Reltio Connect

Hello Reltio Community,

I am currently working on a project where we need to set up data ingestion and consumption between our Snowflake database and Reltio MDM, and I'd like to get your feedback on the approach we're considering.

Data Ingestion (Snowflake to Reltio):

1. Initial Load:

   • Use an ETL tool to extract data from Snowflake and create files.
   • Load these files into Reltio using the Data Loader tool.

2. Incremental Load:

   • Leverage the Snowflake connector in Reltio Integration Hub (RIH) to set up near real-time updates.
   • Snowflake updates happen in batch mode, with data changes often in the range of a few hundred thousand records per batch.

Data Consumption (Reltio to Snowflake):

For sending data back to Snowflake from Reltio, here's my understanding:

1. Initial Export:

   • Similar to the initial data load, we can use an ETL tool to export data from Reltio and send it to Snowflake.

2. Incremental Updates:

   • The Snowflake connector in Reltio Integration Hub (RIH) can be used to send incremental updates directly to Snowflake.

Questions for the Community:

1. Data Ingestion:

   • Is our proposed approach for data ingestion correct?
   • Some data architects are questioning the need to create files for incremental loads when real-time integration via the Snowflake connector is available. Are there better approaches or best practices we should consider?

2. Data Consumption:

   • Can the Snowflake connector in Reltio Integration Hub handle sending both incremental updates and large initial exports effectively?
   • Are there alternative approaches for exporting data to Snowflake that are more efficient or commonly used?

I'd really appreciate any insights, validation, or suggestions you can share to improve our understanding and implementation of this process.

Looking forward to learning from your experiences!

Hi Rushyendar, 

If I follow you correctly, your bullet, ''Leverage the Snowflake connector in Reltio Integration Hub (RIH) to set up near real-time updates" under the section "Data Ingestion (Snowflake to Reltio)", indicates possibly a misunderstanding about the Reltio Snowflake Connector.  The Reltio Snowflake Connector is unidirectional only, delivering data FROM Reltio TO Snowflake. It does not support updates from Snowflake back into Reltio. 

Let me know if I've misinterpreted your  steps. 

Curt

Hello Reltio Community,

I am currently working on a project where we need to set up data ingestion and consumption between our Snowflake database and Reltio MDM, and I'd like to get your feedback on the approach we're considering.

Data Ingestion (Snowflake to Reltio):

1. Initial Load:

   • Use an ETL tool to extract data from Snowflake and create files.
   • Load these files into Reltio using the Data Loader tool.

2. Incremental Load:

   • Leverage the Snowflake connector in Reltio Integration Hub (RIH) to set up near real-time updates.
   • Snowflake updates happen in batch mode, with data changes often in the range of a few hundred thousand records per batch.

Data Consumption (Reltio to Snowflake):

For sending data back to Snowflake from Reltio, here's my understanding:

1. Initial Export:

   • Similar to the initial data load, we can use an ETL tool to export data from Reltio and send it to Snowflake.

2. Incremental Updates:

   • The Snowflake connector in Reltio Integration Hub (RIH) can be used to send incremental updates directly to Snowflake.

Questions for the Community:

1. Data Ingestion:

   • Is our proposed approach for data ingestion correct?
   • Some data architects are questioning the need to create files for incremental loads when real-time integration via the Snowflake connector is available. Are there better approaches or best practices we should consider?

2. Data Consumption:

   • Can the Snowflake connector in Reltio Integration Hub handle sending both incremental updates and large initial exports effectively?
   • Are there alternative approaches for exporting data to Snowflake that are more efficient or commonly used?

I'd really appreciate any insights, validation, or suggestions you can share to improve our understanding and implementation of this process.

Looking forward to learning from your experiences!

Hi

I have the same requirement to enable us to consume data from Snowflake into Reltio. It was my understanding that the connector only works from Reltio to Snowflake (as you indicated), but I need to be able to create a pipeline where Snowflake is the source and have the ability to process data deltas after the initial data sync with csv files through the bulk loader. Is there any guidance on how to accomplish this through the RIH utility? I would hate to look at other solutions, like Mulesoft, if this could be accomplished directly.

Regards

Hi Rushyendar, 

If I follow you correctly, your bullet, ''Leverage the Snowflake connector in Reltio Integration Hub (RIH) to set up near real-time updates" under the section "Data Ingestion (Snowflake to Reltio)", indicates possibly a misunderstanding about the Reltio Snowflake Connector.  The Reltio Snowflake Connector is unidirectional only, delivering data FROM Reltio TO Snowflake. It does not support updates from Snowflake back into Reltio. 

Let me know if I've misinterpreted your  steps. 

Curt

Hello Reltio Community,

I am currently working on a project where we need to set up data ingestion and consumption between our Snowflake database and Reltio MDM, and I'd like to get your feedback on the approach we're considering.

Data Ingestion (Snowflake to Reltio):

1. Initial Load:

   • Use an ETL tool to extract data from Snowflake and create files.
   • Load these files into Reltio using the Data Loader tool.

2. Incremental Load:

   • Leverage the Snowflake connector in Reltio Integration Hub (RIH) to set up near real-time updates.
   • Snowflake updates happen in batch mode, with data changes often in the range of a few hundred thousand records per batch.

Data Consumption (Reltio to Snowflake):

For sending data back to Snowflake from Reltio, here's my understanding:

1. Initial Export:

   • Similar to the initial data load, we can use an ETL tool to export data from Reltio and send it to Snowflake.

2. Incremental Updates:

   • The Snowflake connector in Reltio Integration Hub (RIH) can be used to send incremental updates directly to Snowflake.

Questions for the Community:

1. Data Ingestion:

   • Is our proposed approach for data ingestion correct?
   • Some data architects are questioning the need to create files for incremental loads when real-time integration via the Snowflake connector is available. Are there better approaches or best practices we should consider?

2. Data Consumption:

   • Can the Snowflake connector in Reltio Integration Hub handle sending both incremental updates and large initial exports effectively?
   • Are there alternative approaches for exporting data to Snowflake that are more efficient or commonly used?

I'd really appreciate any insights, validation, or suggestions you can share to improve our understanding and implementation of this process.

Looking forward to learning from your experiences!

Hello Christo and Rushyendar,

Could you please help clarify the following: 

1. For ingestion - What data do you plan to bring in. If it is just interactions, we have zero copy integration with Snowflake. Please check the following documentation
2. For consumption - we have an offering of Reltio Data Pipeline with Snowflake. Please see the following documentation.

Please let us know if you have any further questions on the above. 

Hi

I have the same requirement to enable us to consume data from Snowflake into Reltio. It was my understanding that the connector only works from Reltio to Snowflake (as you indicated), but I need to be able to create a pipeline where Snowflake is the source and have the ability to process data deltas after the initial data sync with csv files through the bulk loader. Is there any guidance on how to accomplish this through the RIH utility? I would hate to look at other solutions, like Mulesoft, if this could be accomplished directly.

Regards

Hi Rushyendar, 

If I follow you correctly, your bullet, ''Leverage the Snowflake connector in Reltio Integration Hub (RIH) to set up near real-time updates" under the section "Data Ingestion (Snowflake to Reltio)", indicates possibly a misunderstanding about the Reltio Snowflake Connector.  The Reltio Snowflake Connector is unidirectional only, delivering data FROM Reltio TO Snowflake. It does not support updates from Snowflake back into Reltio. 

Let me know if I've misinterpreted your  steps. 

Curt

Hello Reltio Community,

I am currently working on a project where we need to set up data ingestion and consumption between our Snowflake database and Reltio MDM, and I'd like to get your feedback on the approach we're considering.

Data Ingestion (Snowflake to Reltio):

1. Initial Load:

   • Use an ETL tool to extract data from Snowflake and create files.
   • Load these files into Reltio using the Data Loader tool.

2. Incremental Load:

   • Leverage the Snowflake connector in Reltio Integration Hub (RIH) to set up near real-time updates.
   • Snowflake updates happen in batch mode, with data changes often in the range of a few hundred thousand records per batch.

Data Consumption (Reltio to Snowflake):

For sending data back to Snowflake from Reltio, here's my understanding:

1. Initial Export:

   • Similar to the initial data load, we can use an ETL tool to export data from Reltio and send it to Snowflake.

2. Incremental Updates:

   • The Snowflake connector in Reltio Integration Hub (RIH) can be used to send incremental updates directly to Snowflake.

Questions for the Community:

1. Data Ingestion:

   • Is our proposed approach for data ingestion correct?
   • Some data architects are questioning the need to create files for incremental loads when real-time integration via the Snowflake connector is available. Are there better approaches or best practices we should consider?

2. Data Consumption:

   • Can the Snowflake connector in Reltio Integration Hub handle sending both incremental updates and large initial exports effectively?
   • Are there alternative approaches for exporting data to Snowflake that are more efficient or commonly used?

I'd really appreciate any insights, validation, or suggestions you can share to improve our understanding and implementation of this process.

Looking forward to learning from your experiences!

Hello Reltio Community,

I am currently working on a project where we need to set up data ingestion and consumption between our Snowflake database and Reltio MDM, and I'd like to get your feedback on the approach we're considering.

Data Ingestion (Snowflake to Reltio):

1. Initial Load:

   • Use an ETL tool to extract data from Snowflake and create files.
   • Load these files into Reltio using the Data Loader tool.

2. Incremental Load:

   • Leverage the Snowflake connector in Reltio Integration Hub (RIH) to set up near real-time updates.
   • Snowflake updates happen in batch mode, with data changes often in the range of a few hundred thousand records per batch.

Data Consumption (Reltio to Snowflake):

For sending data back to Snowflake from Reltio, here's my understanding:

1. Initial Export:

   • Similar to the initial data load, we can use an ETL tool to export data from Reltio and send it to Snowflake.

2. Incremental Updates:

   • The Snowflake connector in Reltio Integration Hub (RIH) can be used to send incremental updates directly to Snowflake.

Questions for the Community:

1. Data Ingestion:

   • Is our proposed approach for data ingestion correct?
   • Some data architects are questioning the need to create files for incremental loads when real-time integration via the Snowflake connector is available. Are there better approaches or best practices we should consider?

2. Data Consumption:

   • Can the Snowflake connector in Reltio Integration Hub handle sending both incremental updates and large initial exports effectively?
   • Are there alternative approaches for exporting data to Snowflake that are more efficient or commonly used?

I'd really appreciate any insights, validation, or suggestions you can share to improve our understanding and implementation of this process.

Looking forward to learning from your experiences!