Reltio Integration Hub

Hi Team,

We have developed recipe where we are trying to overcome below issue in Reltio connector.

Design details : 

Input from Snowflake (records around 1000 will be taken)  and we need to lookup in Reltio to fetch Entity ID for the same.
When i pass the whole list i am able to get only 200 from the list as first iteration.

Is there a way to get all the 1000 records from the loop, may be use of offsetting.

Image of the design - 



------------------------------
Chetan P
Freshworks
------------------------------

You'll need to use pagination to retrieve all the records

------------------------------
Ayan Basu
------------------------------

Original Message:
Sent: 01-12-2023 01:18
From: Chetan P
Subject: RIH | Reltio API Response limit

Hi Team,

We have developed recipe where we are trying to overcome below issue in Reltio connector.

Design details : 

Input from Snowflake (records around 1000 will be taken)  and we need to lookup in Reltio to fetch Entity ID for the same.
When i pass the whole list i am able to get only 200 from the list as first iteration.

Is there a way to get all the 1000 records from the loop, may be use of offsetting.

Image of the design - 



------------------------------
Chetan P
Freshworks
------------------------------

Thanks for the response @Ayan Basu ..
Quick question on the above pagination. I believe that the above solution works for dynamic records processing as well.

For example - Today i need to process 2k & tomorrow i need to process 5k records. It should work is what i assume.​

------------------------------
Chetan P
Freshworks
------------------------------

Original Message:
Sent: 01-12-2023 12:02
From: Ayan Basu
Subject: RIH | Reltio API Response limit

You'll need to use pagination to retrieve all the records

------------------------------
Ayan Basu

Original Message:
Sent: 01-12-2023 01:18
From: Chetan P
Subject: RIH | Reltio API Response limit

Hi Team,

We have developed recipe where we are trying to overcome below issue in Reltio connector.

Design details : 

Input from Snowflake (records around 1000 will be taken)  and we need to lookup in Reltio to fetch Entity ID for the same.
When i pass the whole list i am able to get only 200 from the list as first iteration.

Is there a way to get all the 1000 records from the loop, may be use of offsetting.

Image of the design - 



------------------------------
Chetan P
Freshworks
------------------------------

@Ayan Basu, can you provide direction on how to use pagination?

------------------------------
Chris McCabe
Clinician Nexus
------------------------------

Original Message:
Sent: 01-12-2023 12:02
From: Ayan Basu
Subject: RIH | Reltio API Response limit

You'll need to use pagination to retrieve all the records

------------------------------
Ayan Basu

Original Message:
Sent: 01-12-2023 01:18
From: Chetan P
Subject: RIH | Reltio API Response limit

Hi Team,

We have developed recipe where we are trying to overcome below issue in Reltio connector.

Design details : 

Input from Snowflake (records around 1000 will be taken)  and we need to lookup in Reltio to fetch Entity ID for the same.
When i pass the whole list i am able to get only 200 from the list as first iteration.

Is there a way to get all the 1000 records from the loop, may be use of offsetting.

Image of the design - 



------------------------------
Chetan P
Freshworks
------------------------------

Original Message:
Sent: 6/4/2024 4:34:00 PM
From: Chris McCabe
Subject: RE: RIH | Reltio API Response limit

@Ayan Basu, can you provide direction on how to use pagination?

------------------------------
Chris McCabe
Clinician Nexus
------------------------------

Original Message:
Sent: 01-12-2023 12:02
From: Ayan Basu
Subject: RIH | Reltio API Response limit

You'll need to use pagination to retrieve all the records

------------------------------
Ayan Basu

Original Message:
Sent: 01-12-2023 01:18
From: Chetan P
Subject: RIH | Reltio API Response limit

Hi Team,

We have developed recipe where we are trying to overcome below issue in Reltio connector.

Design details : 

Input from Snowflake (records around 1000 will be taken)  and we need to lookup in Reltio to fetch Entity ID for the same.
When i pass the whole list i am able to get only 200 from the list as first iteration.

Is there a way to get all the 1000 records from the loop, may be use of offsetting.

Image of the design - 



------------------------------
Chetan P
Freshworks
------------------------------

Thank you for your help.  This works for my current task as the data set is under 5K.   Are other options available when we need to export the full set of golden/OV records (150K+) to our Databricks instance?

------------------------------
Chris McCabe
Clinician Nexus
------------------------------

Original Message:
Sent: 06-05-2024 11:17
From: Gino Fortunato
Subject: RIH | Reltio API Response limit

Hi Chris,

Unfortunately, Ayan is no longer with Reltio.  Generally, pagination involves calling a set of data in the API.  For example GET https://{{environment}}.reltio.com/reltio/{{tenantID}}/tasks/history?max=20&offset=0 will allow you to scroll through the history of the configuration by giving the caller 20 sets of configs. The next time, the offset should be 20 and the caller will get the next 20. Depending on the API you are using, the exact mechanism might be different. Please refer to the documentation for the api you are looking for or submit a ticket.

--

Gino Fortunato Senior Sales Engineer

gino.fortunato@reltio.com

+1.415.867.4195

blog.reltio.com

Get the Study

Total Economic Impact Study Finds Reltio's Modern MDM Delivered 366% ROI

Original Message:
Sent: 6/4/2024 4:34:00 PM
From: Chris McCabe
Subject: RE: RIH | Reltio API Response limit

@Ayan Basu, can you provide direction on how to use pagination?

------------------------------
Chris McCabe
Clinician Nexus

Original Message:
Sent: 01-12-2023 12:02
From: Ayan Basu
Subject: RIH | Reltio API Response limit

You'll need to use pagination to retrieve all the records

------------------------------
Ayan Basu

Original Message:
Sent: 01-12-2023 01:18
From: Chetan P
Subject: RIH | Reltio API Response limit

Hi Team,

We have developed recipe where we are trying to overcome below issue in Reltio connector.

Design details : 

Input from Snowflake (records around 1000 will be taken)  and we need to lookup in Reltio to fetch Entity ID for the same.
When i pass the whole list i am able to get only 200 from the list as first iteration.

Is there a way to get all the 1000 records from the loop, may be use of offsetting.

Image of the design - 



------------------------------
Chetan P
Freshworks
------------------------------

Original Message:
Sent: 6/6/2024 10:52:00 AM
From: Chris McCabe
Subject: RE: RIH | Reltio API Response limit

Thank you for your help.  This works for my current task as the data set is under 5K.   Are other options available when we need to export the full set of golden/OV records (150K+) to our Databricks instance?

------------------------------
Chris McCabe
Clinician Nexus
------------------------------

Original Message:
Sent: 06-05-2024 11:17
From: Gino Fortunato
Subject: RIH | Reltio API Response limit

Hi Chris,

Unfortunately, Ayan is no longer with Reltio.  Generally, pagination involves calling a set of data in the API.  For example GET https://{{environment}}.reltio.com/reltio/{{tenantID}}/tasks/history?max=20&offset=0 will allow you to scroll through the history of the configuration by giving the caller 20 sets of configs. The next time, the offset should be 20 and the caller will get the next 20. Depending on the API you are using, the exact mechanism might be different. Please refer to the documentation for the api you are looking for or submit a ticket.

--

Gino Fortunato Senior Sales Engineer

gino.fortunato@reltio.com

+1.415.867.4195

blog.reltio.com

Get the Study

Total Economic Impact Study Finds Reltio's Modern MDM Delivered 366% ROI

Original Message:
Sent: 6/4/2024 4:34:00 PM
From: Chris McCabe
Subject: RE: RIH | Reltio API Response limit

@Ayan Basu, can you provide direction on how to use pagination?

------------------------------
Chris McCabe
Clinician Nexus

Original Message:
Sent: 01-12-2023 12:02
From: Ayan Basu
Subject: RIH | Reltio API Response limit

You'll need to use pagination to retrieve all the records

------------------------------
Ayan Basu

Original Message:
Sent: 01-12-2023 01:18
From: Chetan P
Subject: RIH | Reltio API Response limit

Hi Team,

We have developed recipe where we are trying to overcome below issue in Reltio connector.

Design details : 

Input from Snowflake (records around 1000 will be taken)  and we need to lookup in Reltio to fetch Entity ID for the same.
When i pass the whole list i am able to get only 200 from the list as first iteration.

Is there a way to get all the 1000 records from the loop, may be use of offsetting.

Image of the design - 



------------------------------
Chetan P
Freshworks
------------------------------