Reltio works to continuously improve our performance, functionality and usability. One of the ways we can do this is by finding, building, and developing relationships with other platforms and tools used by the same organizations that use us. Thus, we have developed Reltio Connected Data for Snowflake, a way for both Reltio Master Data Management and Snowflake’s data warehouse to work together.
Please note: Information about ReltiRelo Connected Data for Snowflake provided both here, in related articles, and on the linked webinar is intended for this purpose only- to inform our current and potential customers. No business decisions should be made off of this information alone, please contact us for more information. All of Reltio’s products and their development remain solely owned by Reltio.
In the first article related to the webinar Reltio Connected Data for Snowflake, we discussed a brief overview of how the two products can work together. In our second article, we went over the business solutions that Reltio Connected Data for Snowflake provides, as well as the product’s features, cost, and possible future developments.
Throughout this series, we aimed to answer some important questions. These questions were:
- What business challenges do organizations face? How does Reltio Connected Data for Snowflake work together to solve these challenges?
- How does Reltio Connected Data for Snowflake work? How do users set it up?
- Where can users find resources that enable users to start using Reltio Connected Data for Snowflake?
Much of this content has already been covered in prior articles, however, by the end of this article, we will fully answer all of these questions.
Terms and Reltio Connected Data for Snowflake Demonstration
Remember, to best understand how Reltio Connected Data for Snowflake works, you must understand the following terms.
- Pipeline- a configuration that allows you to create a job.
- Schedule- choose when you would like data transformation to occur.
- Job- the delivery of data into Snowflake.
- Task- at least one task makes up a job.
- Relational view- customer data that follows the relational database model.
- Custom views- created on top of the JSON format.
Before using Reltio Connected Data for Snowflake, you will need to perform four steps. These are:
1. Register Snowflake Account. Before registering, you will need to create one service account for Reltio on the Snowflake side. We have provided the steps you need to follow, as well as information about the kind of access you will need to provide to Reltio here. You will be creating requests from your security side to this single service account for Reltio to access the specific database.2. Create a Pipeline. After registering your snowflake account, everything else is a self-driven API.3. Trigger Job Run. This step is completed after the pipeline is created. 4. Run Queries. Once the job is created, you will track the job to an API call, and monitor its status. If there are any failures, they will be available to you in the job response.
Working in Snowflake
In this demonstration, our presenter, Ganesh, shows a created data warehouse for a particular tenant, as well as a separate database where he is using Snowflake schema. He goes over how Reltio will only communicate certain information to Snowflake, as Reltio has ensured as minimal of access as possible. Keep in mind, if there is any data you do not want to share with Snowflake, like PII data, you will be able to choose which attributes to ignore before the data is sent to Snowflake. You can also export all data, or just updated data, by sending only an incremental batch.
We highly recommend watching his process in real time so you can see exactly how he performs the tasks.
He explains the views within the box Snowflake connector. When working through the Snowflake schema, you will be able to see two things. One of these is the source table. The other is an OV table, or operational value table. OV is intended for business users, thus, we convert the data into operational value and the most simplified form. The OV table will have the transformed simplified operational value structure. This simplified operational value structure will allow the user to create relational views and custom views easily.
The source table is where you can see the data, and is more for integration and analysis. This is a Reltio JSON structure with all cross falls and all information included. This will be your source landing table, with everything residing there. Use the source table to do a cross check analysis or a detailed analysis by the integration teams.
After the source table, the data is automatically converted into a more simplified format, which makes creating a relational view more simple. Furthermore, the JSON is very simple, and is still a JSON structure within Snowflake. If you would like to query it, it will mold like a relational column. Then, you can create a relational view. Or, create a custom view, which is an out-of-the-box relational view.
These are the two things that happen automatically for each entity and relations within the Snowflake connector.
Usability for Business Owners
Reltio could have made this even more complex, but we worked to make sure that anyone could use this product. Therefore, you will notice simplified views throughout. Business users will be able to run typical relational queries, and view a simple attribute or nested attribute. For example, if a business user would like to know who within their data lives in Chicago, they will be able to run this query, and see the results as a relational view.
Architecture Overview
Another valuable resource shared during our webinar is a breakdown of the Reltio Connected Data for Snowflake’s architecture. In the diagram shown, there are two different boxes. One represents the Reltio cloud, while the other is Snowflake. It shows the primary data storage clusters, which expand where your primary data resides.
From there, the data makes its way through one Apache Spark cluster, which runs in an AWS cloud, and enters an analytical data bucket. Another bucket, serving its purpose as a staging environment, is an S3 bucket. Once here, users trigger the Snowflake copy task. The whole process of data transformation begins, and the data is delivered into Snowflake using the Snowflake copy task. Remember, this process does not update you in real time, however, you will have data transformation complete in about thirty minutes. This is something Reltio may develop in the future.
More Information for Reltio Connected Data for Snowflake
At the beginning of this webinar, we focused on three main ideas. The answers to the first two questions, dealing with business solutions and how Reltio Connected Data for Snowflake works, have been discussed throughout the webinar and within two other articles on the subject. Here is the answer for the last question:
- Where can users find resources that enable users to start using Reltio Connected Data for Snowflake?
We have compiled everything you need to know about Reltio Connected Data for Snowflake within Reltio Documentation. To find the resources you need, simply follow Integration and Connectors from Home, and click Reltio Connected Data for Snowflake. Or, search “Integration and Connectors” in the search bar. For easiest access from this article, click here to find access immediately. You will find product terminology breakdowns here, as well as instructions for onboarding Reltio Connected Data for Snowflake, Snowflake schema, a guide to APIs, and videos of the product.
We look forward to our continued practice of making our products even better and more user friendly for our customers. If you have questions, or would like to leave feedback on what you would like to see down the road with Reltio Connected Data for Snowflake, join in the discussions in Reltio Community.
Relevant Content:
Reltio Connected Data for Snowflake: Features and Important Terms
Reltio Connected Data for Snowflake: A Business Solution