Reltio Connect

Hi Team,

I want to export data using streaming API to the Azure Event Hub in real-time as soon as the data in Reltio is either created or updated, I know Reltio has out-of-the-box below-mentioned Q services, but I want to notify to Azure Event Hub directly from reltio.

Azure event hub is the streaming platform managed in the Azure cloud.

----------------------------------------

Amazon Simple Queue Service (SQS)
Amazon Simple Notification Service (SNS)
Google Cloud Pub/Sub (Publisher/Subscriber)
Microsoft Azure Service Bus

Hi Abhimanyu,

As you have rightly mentioned, Reltio only supports direct event streaming to AWS SQS/SNS, Azure Service Bus or Google Pub Sub at the moment. As per my understanding, publishing message to Azure Event Hub or Kafka is not possible at the moment. 

So if you want to stream create/update events to event hub or Kafka you have to do the following:

• Configure the External queues to publish events from Reltio to any one of the supported queues/topics. Let's say Azure Service Bus. 
• Create a custom consumer or use an integration tool (MuleSoft, Informatica IICS etc.) which will listen to the Service bus queue and read the message from the queue. You can even leverage Azure functions as well to do this. 
• Once you have read the message from the Azure service bus, you can publish the same message to Kafka or Event Hub using your integration component itself. 

Note:-   You can read messages from Azure Service Bus and publish to Kafka using Reltio Integration Hub (RIH) as well.

Let me know if it helps or if you need any further information.

Hi Team,

I want to export data using streaming API to the Azure Event Hub in real-time as soon as the data in Reltio is either created or updated, I know Reltio has out-of-the-box below-mentioned Q services, but I want to notify to Azure Event Hub directly from reltio.

Azure event hub is the streaming platform managed in the Azure cloud.

----------------------------------------

Amazon Simple Queue Service (SQS)
Amazon Simple Notification Service (SNS)
Google Cloud Pub/Sub (Publisher/Subscriber)
Microsoft Azure Service Bus

Thank you Diparnab,

it looks ok to me . 

I am thinking - Azure service bus ->Custom Listener (get the event)-> Call the reltio API from Custom Listener solution to get a complete snapshot of the entity and then feed to -> snowflake.

Now I have a couple of questions here - 

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?

Hi Abhimanyu,

As you have rightly mentioned, Reltio only supports direct event streaming to AWS SQS/SNS, Azure Service Bus or Google Pub Sub at the moment. As per my understanding, publishing message to Azure Event Hub or Kafka is not possible at the moment. 

So if you want to stream create/update events to event hub or Kafka you have to do the following:

• Configure the External queues to publish events from Reltio to any one of the supported queues/topics. Let's say Azure Service Bus. 
• Create a custom consumer or use an integration tool (MuleSoft, Informatica IICS etc.) which will listen to the Service bus queue and read the message from the queue. You can even leverage Azure functions as well to do this. 
• Once you have read the message from the Azure service bus, you can publish the same message to Kafka or Event Hub using your integration component itself. 

Note:-   You can read messages from Azure Service Bus and publish to Kafka using Reltio Integration Hub (RIH) as well.

Let me know if it helps or if you need any further information.

Hi Team,

I want to export data using streaming API to the Azure Event Hub in real-time as soon as the data in Reltio is either created or updated, I know Reltio has out-of-the-box below-mentioned Q services, but I want to notify to Azure Event Hub directly from reltio.

Azure event hub is the streaming platform managed in the Azure cloud.

----------------------------------------

Amazon Simple Queue Service (SQS)
Amazon Simple Notification Service (SNS)
Google Cloud Pub/Sub (Publisher/Subscriber)
Microsoft Azure Service Bus

Abhimanyu, 
Please find my answers inline

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
  <Diparnab>: You can call Reltio APIs from custom listener solution or you can even use the message payload type SNAPSHOT in your external queue configuration. 
  Using this payload type will mean that the full entity payload will be streamed to the queue for Entity Create/Update Events. The default payload type is snapshot. You will get some more information in the document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/crud-events
  
  
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?
  <Diparnab> : You can use the object filter property (refer to the previous screenshot)  to achieve this - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/message-streaming-capabilities.  Here you can provide the filter condition and specify the entity type for which you want events to be streamed. 

Also, I see that your end goal is to publish the data to Snowflake. So, did you consider leveraging the Reltio Snowflake Connector ( Additional Subscription) for that ? Here is the documentation link for this - https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector

If not, I will definitely recommend reaching out to your Reltio Account Executives to know more about this. 

Thank you Diparnab,

it looks ok to me . 

I am thinking - Azure service bus ->Custom Listener (get the event)-> Call the reltio API from Custom Listener solution to get a complete snapshot of the entity and then feed to -> snowflake.

Now I have a couple of questions here - 

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?

Hi Abhimanyu,

As you have rightly mentioned, Reltio only supports direct event streaming to AWS SQS/SNS, Azure Service Bus or Google Pub Sub at the moment. As per my understanding, publishing message to Azure Event Hub or Kafka is not possible at the moment. 

So if you want to stream create/update events to event hub or Kafka you have to do the following:

• Configure the External queues to publish events from Reltio to any one of the supported queues/topics. Let's say Azure Service Bus. 
• Create a custom consumer or use an integration tool (MuleSoft, Informatica IICS etc.) which will listen to the Service bus queue and read the message from the queue. You can even leverage Azure functions as well to do this. 
• Once you have read the message from the Azure service bus, you can publish the same message to Kafka or Event Hub using your integration component itself. 

Note:-   You can read messages from Azure Service Bus and publish to Kafka using Reltio Integration Hub (RIH) as well.

Let me know if it helps or if you need any further information.

Hi Team,

I want to export data using streaming API to the Azure Event Hub in real-time as soon as the data in Reltio is either created or updated, I know Reltio has out-of-the-box below-mentioned Q services, but I want to notify to Azure Event Hub directly from reltio.

Azure event hub is the streaming platform managed in the Azure cloud.

----------------------------------------

Amazon Simple Queue Service (SQS)
Amazon Simple Notification Service (SNS)
Google Cloud Pub/Sub (Publisher/Subscriber)
Microsoft Azure Service Bus

thank you again.

Can Snowflake Connector feed the data in Snowflake DB in real-time? I do not want the pull-based integration.

Abhimanyu, 
Please find my answers inline

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
  <Diparnab>: You can call Reltio APIs from custom listener solution or you can even use the message payload type SNAPSHOT in your external queue configuration. 
  Using this payload type will mean that the full entity payload will be streamed to the queue for Entity Create/Update Events. The default payload type is snapshot. You will get some more information in the document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/crud-events
  
  
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?
  <Diparnab> : You can use the object filter property (refer to the previous screenshot)  to achieve this - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/message-streaming-capabilities.  Here you can provide the filter condition and specify the entity type for which you want events to be streamed. 

Also, I see that your end goal is to publish the data to Snowflake. So, did you consider leveraging the Reltio Snowflake Connector ( Additional Subscription) for that ? Here is the documentation link for this - https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector

If not, I will definitely recommend reaching out to your Reltio Account Executives to know more about this. 

Thank you Diparnab,

it looks ok to me . 

I am thinking - Azure service bus ->Custom Listener (get the event)-> Call the reltio API from Custom Listener solution to get a complete snapshot of the entity and then feed to -> snowflake.

Now I have a couple of questions here - 

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?

Hi Abhimanyu,

As you have rightly mentioned, Reltio only supports direct event streaming to AWS SQS/SNS, Azure Service Bus or Google Pub Sub at the moment. As per my understanding, publishing message to Azure Event Hub or Kafka is not possible at the moment. 

So if you want to stream create/update events to event hub or Kafka you have to do the following:

• Configure the External queues to publish events from Reltio to any one of the supported queues/topics. Let's say Azure Service Bus. 
• Create a custom consumer or use an integration tool (MuleSoft, Informatica IICS etc.) which will listen to the Service bus queue and read the message from the queue. You can even leverage Azure functions as well to do this. 
• Once you have read the message from the Azure service bus, you can publish the same message to Kafka or Event Hub using your integration component itself. 

Note:-   You can read messages from Azure Service Bus and publish to Kafka using Reltio Integration Hub (RIH) as well.

Let me know if it helps or if you need any further information.

Hi Team,

I want to export data using streaming API to the Azure Event Hub in real-time as soon as the data in Reltio is either created or updated, I know Reltio has out-of-the-box below-mentioned Q services, but I want to notify to Azure Event Hub directly from reltio.

Azure event hub is the streaming platform managed in the Azure cloud.

----------------------------------------

Amazon Simple Queue Service (SQS)
Amazon Simple Notification Service (SNS)
Google Cloud Pub/Sub (Publisher/Subscriber)
Microsoft Azure Service Bus

Hi Abhimanyu, 

Once you set up the Snowflake connector and enable the configuration the data flow from Reltio to Snowflake will be automatic and in near real time. You will not to do anything to pull data from Snowflake. Once the JSON is available in the tables that you will need to configure during set up, you can create custom views/tables to process the Reltio data as needed. 

Refer to these documentations, it will be helpful and provide some insights 
https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector

https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector/reltio-snowflake-connector-architecture

Here is one community webinar link which will help clarify a few things as well - https://community.reltio.com/blogs/chris-detzel1/2023/02/23/reltio-snowflake-connector-show

You can go through these and reach out to your Reltio account team for more information and clarity regarding the Snowflake connector and evaluate whether this will be beneficial for your use cases. 

thank you again.

Can Snowflake Connector feed the data in Snowflake DB in real-time? I do not want the pull-based integration.

Abhimanyu, 
Please find my answers inline

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
  <Diparnab>: You can call Reltio APIs from custom listener solution or you can even use the message payload type SNAPSHOT in your external queue configuration. 
  Using this payload type will mean that the full entity payload will be streamed to the queue for Entity Create/Update Events. The default payload type is snapshot. You will get some more information in the document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/crud-events
  
  
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?
  <Diparnab> : You can use the object filter property (refer to the previous screenshot)  to achieve this - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/message-streaming-capabilities.  Here you can provide the filter condition and specify the entity type for which you want events to be streamed. 

Also, I see that your end goal is to publish the data to Snowflake. So, did you consider leveraging the Reltio Snowflake Connector ( Additional Subscription) for that ? Here is the documentation link for this - https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector

If not, I will definitely recommend reaching out to your Reltio Account Executives to know more about this. 

Thank you Diparnab,

it looks ok to me . 

I am thinking - Azure service bus ->Custom Listener (get the event)-> Call the reltio API from Custom Listener solution to get a complete snapshot of the entity and then feed to -> snowflake.

Now I have a couple of questions here - 

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?

Hi Abhimanyu,

As you have rightly mentioned, Reltio only supports direct event streaming to AWS SQS/SNS, Azure Service Bus or Google Pub Sub at the moment. As per my understanding, publishing message to Azure Event Hub or Kafka is not possible at the moment. 

So if you want to stream create/update events to event hub or Kafka you have to do the following:

• Configure the External queues to publish events from Reltio to any one of the supported queues/topics. Let's say Azure Service Bus. 
• Create a custom consumer or use an integration tool (MuleSoft, Informatica IICS etc.) which will listen to the Service bus queue and read the message from the queue. You can even leverage Azure functions as well to do this. 
• Once you have read the message from the Azure service bus, you can publish the same message to Kafka or Event Hub using your integration component itself. 

Note:-   You can read messages from Azure Service Bus and publish to Kafka using Reltio Integration Hub (RIH) as well.

Let me know if it helps or if you need any further information.

Hi Team,

I want to export data using streaming API to the Azure Event Hub in real-time as soon as the data in Reltio is either created or updated, I know Reltio has out-of-the-box below-mentioned Q services, but I want to notify to Azure Event Hub directly from reltio.

Azure event hub is the streaming platform managed in the Azure cloud.

----------------------------------------

Amazon Simple Queue Service (SQS)
Amazon Simple Notification Service (SNS)
Google Cloud Pub/Sub (Publisher/Subscriber)
Microsoft Azure Service Bus

thank you for the various options and reply, I will validate the fitment.

Hi Abhimanyu, 

Once you set up the Snowflake connector and enable the configuration the data flow from Reltio to Snowflake will be automatic and in near real time. You will not to do anything to pull data from Snowflake. Once the JSON is available in the tables that you will need to configure during set up, you can create custom views/tables to process the Reltio data as needed. 

Refer to these documentations, it will be helpful and provide some insights 
https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector

https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector/reltio-snowflake-connector-architecture

Here is one community webinar link which will help clarify a few things as well - https://community.reltio.com/blogs/chris-detzel1/2023/02/23/reltio-snowflake-connector-show

You can go through these and reach out to your Reltio account team for more information and clarity regarding the Snowflake connector and evaluate whether this will be beneficial for your use cases. 

thank you again.

Can Snowflake Connector feed the data in Snowflake DB in real-time? I do not want the pull-based integration.

Abhimanyu, 
Please find my answers inline

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
  <Diparnab>: You can call Reltio APIs from custom listener solution or you can even use the message payload type SNAPSHOT in your external queue configuration. 
  Using this payload type will mean that the full entity payload will be streamed to the queue for Entity Create/Update Events. The default payload type is snapshot. You will get some more information in the document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/crud-events
  
  
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?
  <Diparnab> : You can use the object filter property (refer to the previous screenshot)  to achieve this - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/message-streaming-capabilities.  Here you can provide the filter condition and specify the entity type for which you want events to be streamed. 

Also, I see that your end goal is to publish the data to Snowflake. So, did you consider leveraging the Reltio Snowflake Connector ( Additional Subscription) for that ? Here is the documentation link for this - https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector

If not, I will definitely recommend reaching out to your Reltio Account Executives to know more about this. 

Thank you Diparnab,

it looks ok to me . 

I am thinking - Azure service bus ->Custom Listener (get the event)-> Call the reltio API from Custom Listener solution to get a complete snapshot of the entity and then feed to -> snowflake.

Now I have a couple of questions here - 

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?

Hi Abhimanyu,

As you have rightly mentioned, Reltio only supports direct event streaming to AWS SQS/SNS, Azure Service Bus or Google Pub Sub at the moment. As per my understanding, publishing message to Azure Event Hub or Kafka is not possible at the moment. 

So if you want to stream create/update events to event hub or Kafka you have to do the following:

• Configure the External queues to publish events from Reltio to any one of the supported queues/topics. Let's say Azure Service Bus. 
• Create a custom consumer or use an integration tool (MuleSoft, Informatica IICS etc.) which will listen to the Service bus queue and read the message from the queue. You can even leverage Azure functions as well to do this. 
• Once you have read the message from the Azure service bus, you can publish the same message to Kafka or Event Hub using your integration component itself. 

Note:-   You can read messages from Azure Service Bus and publish to Kafka using Reltio Integration Hub (RIH) as well.

Let me know if it helps or if you need any further information.

Hi Team,

I want to export data using streaming API to the Azure Event Hub in real-time as soon as the data in Reltio is either created or updated, I know Reltio has out-of-the-box below-mentioned Q services, but I want to notify to Azure Event Hub directly from reltio.

Azure event hub is the streaming platform managed in the Azure cloud.

----------------------------------------

Amazon Simple Queue Service (SQS)
Amazon Simple Notification Service (SNS)
Google Cloud Pub/Sub (Publisher/Subscriber)
Microsoft Azure Service Bus

hi @Diparnab Dey,

I have one more query-

To export data from Reltio, I am thinking of using the Azure service bus to get the event, now how to feed data into Snowflake from the Azure service bus.

Can I use the Azure function or Databrick to carry out this activity?

Secondly, GSK will have an Azure service bus configured in a private network, so how does Reltio push the messages into the Azure service bus, I mean which protocol do you use (HTTPS or AMQP)? will you be able to send a message to the Azure service bus which is exposed via the Azure gateway using HTTPS protocol?

thank you for the various options and reply, I will validate the fitment.

Hi Abhimanyu, 

Once you set up the Snowflake connector and enable the configuration the data flow from Reltio to Snowflake will be automatic and in near real time. You will not to do anything to pull data from Snowflake. Once the JSON is available in the tables that you will need to configure during set up, you can create custom views/tables to process the Reltio data as needed. 

Refer to these documentations, it will be helpful and provide some insights 
https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector

https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector/reltio-snowflake-connector-architecture

Here is one community webinar link which will help clarify a few things as well - https://community.reltio.com/blogs/chris-detzel1/2023/02/23/reltio-snowflake-connector-show

You can go through these and reach out to your Reltio account team for more information and clarity regarding the Snowflake connector and evaluate whether this will be beneficial for your use cases. 

thank you again.

Can Snowflake Connector feed the data in Snowflake DB in real-time? I do not want the pull-based integration.

Abhimanyu, 
Please find my answers inline

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
  <Diparnab>: You can call Reltio APIs from custom listener solution or you can even use the message payload type SNAPSHOT in your external queue configuration. 
  Using this payload type will mean that the full entity payload will be streamed to the queue for Entity Create/Update Events. The default payload type is snapshot. You will get some more information in the document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/crud-events
  
  
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?
  <Diparnab> : You can use the object filter property (refer to the previous screenshot)  to achieve this - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/events-api/message-streaming-capabilities.  Here you can provide the filter condition and specify the entity type for which you want events to be streamed. 

Also, I see that your end goal is to publish the data to Snowflake. So, did you consider leveraging the Reltio Snowflake Connector ( Additional Subscription) for that ? Here is the documentation link for this - https://docs.reltio.com/en/integrate/export-data-from-reltio-in-bulk/export-data-from-reltio-to-business-intelligence-tools/reltio-snowflake-connector

If not, I will definitely recommend reaching out to your Reltio Account Executives to know more about this. 

Thank you Diparnab,

it looks ok to me . 

I am thinking - Azure service bus ->Custom Listener (get the event)-> Call the reltio API from Custom Listener solution to get a complete snapshot of the entity and then feed to -> snowflake.

Now I have a couple of questions here - 

• Calling reltio API from a custom listener solution is recommended or to have a RIH integration altogether in place?
• I want to generate the event and write to azure service bus for some specific entity only not for all the entities mastered in reltio, is that filtration possible?

Hi Abhimanyu,

As you have rightly mentioned, Reltio only supports direct event streaming to AWS SQS/SNS, Azure Service Bus or Google Pub Sub at the moment. As per my understanding, publishing message to Azure Event Hub or Kafka is not possible at the moment. 

So if you want to stream create/update events to event hub or Kafka you have to do the following:

• Configure the External queues to publish events from Reltio to any one of the supported queues/topics. Let's say Azure Service Bus. 
• Create a custom consumer or use an integration tool (MuleSoft, Informatica IICS etc.) which will listen to the Service bus queue and read the message from the queue. You can even leverage Azure functions as well to do this. 
• Once you have read the message from the Azure service bus, you can publish the same message to Kafka or Event Hub using your integration component itself. 

Note:-   You can read messages from Azure Service Bus and publish to Kafka using Reltio Integration Hub (RIH) as well.

Let me know if it helps or if you need any further information.

Hi Team,

I want to export data using streaming API to the Azure Event Hub in real-time as soon as the data in Reltio is either created or updated, I know Reltio has out-of-the-box below-mentioned Q services, but I want to notify to Azure Event Hub directly from reltio.

Azure event hub is the streaming platform managed in the Azure cloud.

----------------------------------------

Amazon Simple Queue Service (SQS)
Amazon Simple Notification Service (SNS)
Google Cloud Pub/Sub (Publisher/Subscriber)
Microsoft Azure Service Bus