Automate the Reltio export job

5. RE: Automate the Reltio export job

Recommend

Rushyendar Akula

Posted 10-22-2024 23:20

Thank you. I am able to kickoff the export job using an API. Created a python script that gets the token first and then invokes export API using the token....

I can see that the export job is kicked off.....

from the same script, how to check if the job is complete. Also, where this file is available to copy it to one sftp location?

Here is what I am trying in script:

1) Get token

2) Invoke Export API using the token from step1

3) Check the job status, once the job status is complete, download/access the file.

4) Copy the file to exported location.

I am referring to step3 here.

Appreciate your inputs on this. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC
------------------------------

Original Message

Original Message:
Sent: 10-14-2024 10:28
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

The Export jobs are a little different than the data loader jobs, you cannot create the export job definition in advance and execute at a latter point in time. So for scheduling your export, the process you will need to follow is

Schedule a script (or integration flow) to run periodically.
From the script or integration flow execute the Export API based on your filter criteria. For example, all entities that has been created or updated between the last job run to current job run time. Here you will need to dynamically modify the filter criteria from the script or integration.

When you say this export job is scheduled, do you mean export get's triggered every day on a specific time ? If the answer is yes, my guess would be that the export was scheduled using any of the method I specified above.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-09-2024 12:45
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you, Diparnab Dey.

We don't have a license for RIH. If I understand correctly, here is what I need to do:

Define an export job using the API.
Schedule it using a Python/Unix script.

We have an export job scheduled in production. I can view its configuration, but I'm not sure how it was set up and how to replicate it in non-production. However, I am certain we don't have an RIH license.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-07-2024 04:00
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

Currently, we do not have the option to schedule Reltio exports from the UI. However, that doesn't mean it cannot be achieved differently. You can schedule the export API per your preferred window from any integration platform (MuleSoft, Informatica IDMC, RIH, or any custom solution) or a script (Python/Unix etc.) which can be scheduled and can call REST APIs.

If your tenant(s) have RIH enabled, it can be done very easily using RIH, where your recipe should run on a schedule and from the recipe, you will trigger the Export API. You will find a convenient recipe (along with the documentation) which Reltio PS has built earlier, for this activity in the Bitbucket repository - https://bitbucket.org/reltio-ondemand/reltio-rih-recipe/src/master/Export/

Review and let me know if you have any follow up question.

Also, there is no out-of-the-box way to move the files from the storage location (S3/ Blob/ GCS) to SFTP location after they are exported, you will need to build an integration flow to achieve that.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-04-2024 12:52
From: Rushyendar Akula
Subject: Automate the Reltio export job

Hi,

How do we automate and schedule the export process? I dont see an option on UI. Also, once the file is exported, need to push it one sftp location. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC
------------------------------

6. RE: Automate the Reltio export job

Recommend

Reltio Employee

Diparnab Dey

Posted 10-23-2024 04:19

Hi Rushyendar,

Glad to hear that you could trigger the export API from the Python script. You are on the right track here. When you triggered the export job, it should have returned a response like below

{
    "exportType": "ENTITIES",
    "version": "v2",
    "status": "scheduled",
    "details": "Export job has been scheduled. Result will be sent to your email: user@example.com",
    "taskIds": [
        "831272d6-1685-49cc-a98c-c8965d04bc0d"
    ]
}

You will get the information in this documentation as well.

Now, to check for the export job status, you will need to use the Task Id from the Export job response and trigger another API for getting the export job status.

Refer to the documentation in the link above. But I am also sharing the CURL command for the API

curl --location 'https://{{env}}.reltio.com/jobs/{{mdmTenant}}/tasks/{{ExportTaskID}}' \
--header 'Authorization: Bearer {{accessToken}}' \
--header 'Content-Type: application/json'

If the export task status is completed, you will need get a link for downloading the exported data in the response. Please note that based on the volume of the data, the export API will take some time to complete. So you will need to figure out how frequently you are going to trigger the status check after triggering the job.

Also, you have a way to directly store the exported data in your cloud storage location (AWS S3, Azure Blob, Google Cloud Storage Bucket) by providing additional parameters in the export API call. You will get details about those parameters in this document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/export-service-apis/store-export-results.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal
------------------------------

Original Message

Original Message:
Sent: 10-22-2024 23:19
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you. I am able to kickoff the export job using an API. Created a python script that gets the token first and then invokes export API using the token....

I can see that the export job is kicked off.....

from the same script, how to check if the job is complete. Also, where this file is available to copy it to one sftp location?

Here is what I am trying in script:

1) Get token

2) Invoke Export API using the token from step1

3) Check the job status, once the job status is complete, download/access the file.

4) Copy the file to exported location.

I am referring to step3 here.

Appreciate your inputs on this. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-14-2024 10:28
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

The Export jobs are a little different than the data loader jobs, you cannot create the export job definition in advance and execute at a latter point in time. So for scheduling your export, the process you will need to follow is

Schedule a script (or integration flow) to run periodically.
From the script or integration flow execute the Export API based on your filter criteria. For example, all entities that has been created or updated between the last job run to current job run time. Here you will need to dynamically modify the filter criteria from the script or integration.

When you say this export job is scheduled, do you mean export get's triggered every day on a specific time ? If the answer is yes, my guess would be that the export was scheduled using any of the method I specified above.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-09-2024 12:45
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you, Diparnab Dey.

We don't have a license for RIH. If I understand correctly, here is what I need to do:

Define an export job using the API.
Schedule it using a Python/Unix script.

We have an export job scheduled in production. I can view its configuration, but I'm not sure how it was set up and how to replicate it in non-production. However, I am certain we don't have an RIH license.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-07-2024 04:00
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

Currently, we do not have the option to schedule Reltio exports from the UI. However, that doesn't mean it cannot be achieved differently. You can schedule the export API per your preferred window from any integration platform (MuleSoft, Informatica IDMC, RIH, or any custom solution) or a script (Python/Unix etc.) which can be scheduled and can call REST APIs.

If your tenant(s) have RIH enabled, it can be done very easily using RIH, where your recipe should run on a schedule and from the recipe, you will trigger the Export API. You will find a convenient recipe (along with the documentation) which Reltio PS has built earlier, for this activity in the Bitbucket repository - https://bitbucket.org/reltio-ondemand/reltio-rih-recipe/src/master/Export/

Review and let me know if you have any follow up question.

Also, there is no out-of-the-box way to move the files from the storage location (S3/ Blob/ GCS) to SFTP location after they are exported, you will need to build an integration flow to achieve that.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-04-2024 12:52
From: Rushyendar Akula
Subject: Automate the Reltio export job

Hi,

How do we automate and schedule the export process? I dont see an option on UI. Also, once the file is exported, need to push it one sftp location. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC
------------------------------

7. RE: Automate the Reltio export job

Recommend

Utsa Das

Posted 10-23-2024 08:08

Hi Diparnab,

When you are storing the export files in the customer's cloud location , I assuming the type of the file will be zip.

So to access the files inside it , we will need to build a process to unzip the same if the export job is to be automated through RIH .Is that the correct assumption ?

------------------------------
Utsa Das
Novartis
------------------------------

Original Message

Original Message:
Sent: 10-23-2024 04:18
From: Diparnab Dey
Subject: Automate the Reltio export job

Hi Rushyendar,

Glad to hear that you could trigger the export API from the Python script. You are on the right track here. When you triggered the export job, it should have returned a response like below

{    "exportType": "ENTITIES",    "version": "v2",    "status": "scheduled",    "details": "Export job has been scheduled. Result will be sent to your email: user@example.com",    "taskIds": [        "831272d6-1685-49cc-a98c-c8965d04bc0d"    ]}

You will get the information in this documentation as well.

Now, to check for the export job status, you will need to use the Task Id from the Export job response and trigger another API for getting the export job status.

Refer to the documentation in the link above. But I am also sharing the CURL command for the API

curl --location 'https://{{env}}.reltio.com/jobs/{{mdmTenant}}/tasks/{{ExportTaskID}}' \--header 'Authorization: Bearer {{accessToken}}' \--header 'Content-Type: application/json'

If the export task status is completed, you will need get a link for downloading the exported data in the response. Please note that based on the volume of the data, the export API will take some time to complete. So you will need to figure out how frequently you are going to trigger the status check after triggering the job.

Also, you have a way to directly store the exported data in your cloud storage location (AWS S3, Azure Blob, Google Cloud Storage Bucket) by providing additional parameters in the export API call. You will get details about those parameters in this document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/export-service-apis/store-export-results.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-22-2024 23:19
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you. I am able to kickoff the export job using an API. Created a python script that gets the token first and then invokes export API using the token....

I can see that the export job is kicked off.....

from the same script, how to check if the job is complete. Also, where this file is available to copy it to one sftp location?

Here is what I am trying in script:

1) Get token

2) Invoke Export API using the token from step1

3) Check the job status, once the job status is complete, download/access the file.

4) Copy the file to exported location.

I am referring to step3 here.

Appreciate your inputs on this. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-14-2024 10:28
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

The Export jobs are a little different than the data loader jobs, you cannot create the export job definition in advance and execute at a latter point in time. So for scheduling your export, the process you will need to follow is

Schedule a script (or integration flow) to run periodically.
From the script or integration flow execute the Export API based on your filter criteria. For example, all entities that has been created or updated between the last job run to current job run time. Here you will need to dynamically modify the filter criteria from the script or integration.

When you say this export job is scheduled, do you mean export get's triggered every day on a specific time ? If the answer is yes, my guess would be that the export was scheduled using any of the method I specified above.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-09-2024 12:45
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you, Diparnab Dey.

We don't have a license for RIH. If I understand correctly, here is what I need to do:

Define an export job using the API.
Schedule it using a Python/Unix script.

We have an export job scheduled in production. I can view its configuration, but I'm not sure how it was set up and how to replicate it in non-production. However, I am certain we don't have an RIH license.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-07-2024 04:00
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

Currently, we do not have the option to schedule Reltio exports from the UI. However, that doesn't mean it cannot be achieved differently. You can schedule the export API per your preferred window from any integration platform (MuleSoft, Informatica IDMC, RIH, or any custom solution) or a script (Python/Unix etc.) which can be scheduled and can call REST APIs.

If your tenant(s) have RIH enabled, it can be done very easily using RIH, where your recipe should run on a schedule and from the recipe, you will trigger the Export API. You will find a convenient recipe (along with the documentation) which Reltio PS has built earlier, for this activity in the Bitbucket repository - https://bitbucket.org/reltio-ondemand/reltio-rih-recipe/src/master/Export/

Review and let me know if you have any follow up question.

Also, there is no out-of-the-box way to move the files from the storage location (S3/ Blob/ GCS) to SFTP location after they are exported, you will need to build an integration flow to achieve that.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-04-2024 12:52
From: Rushyendar Akula
Subject: Automate the Reltio export job

Hi,

How do we automate and schedule the export process? I dont see an option on UI. Also, once the file is exported, need to push it one sftp location. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC
------------------------------

8. RE: Automate the Reltio export job

Recommend

Reltio Employee

Diparnab Dey

Posted 10-23-2024 08:25

That's right Utsa. Reltio will store the export results in compressed format in your cloud location. You will then have to use a process to uncompress the data for further processing.

Also, the default compression method provided by Reltio is GZIP. However, it can be changed to ZIP. You will just have to raise a Reltio support ticket and request for the fileCompression to be changed from GZIP to ZIP.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal
------------------------------

Original Message

Original Message:
Sent: 10-23-2024 08:07
From: Utsa Das
Subject: Automate the Reltio export job

Hi Diparnab,

When you are storing the export files in the customer's cloud location , I assuming the type of the file will be zip.

So to access the files inside it , we will need to build a process to unzip the same if the export job is to be automated through RIH .Is that the correct assumption ?

------------------------------
Utsa Das
Novartis

Original Message:
Sent: 10-23-2024 04:18
From: Diparnab Dey
Subject: Automate the Reltio export job

Hi Rushyendar,

Glad to hear that you could trigger the export API from the Python script. You are on the right track here. When you triggered the export job, it should have returned a response like below

{    "exportType": "ENTITIES",    "version": "v2",    "status": "scheduled",    "details": "Export job has been scheduled. Result will be sent to your email: user@example.com",    "taskIds": [        "831272d6-1685-49cc-a98c-c8965d04bc0d"    ]}

You will get the information in this documentation as well.

Now, to check for the export job status, you will need to use the Task Id from the Export job response and trigger another API for getting the export job status.

Refer to the documentation in the link above. But I am also sharing the CURL command for the API

curl --location 'https://{{env}}.reltio.com/jobs/{{mdmTenant}}/tasks/{{ExportTaskID}}' \--header 'Authorization: Bearer {{accessToken}}' \--header 'Content-Type: application/json'

If the export task status is completed, you will need get a link for downloading the exported data in the response. Please note that based on the volume of the data, the export API will take some time to complete. So you will need to figure out how frequently you are going to trigger the status check after triggering the job.

Also, you have a way to directly store the exported data in your cloud storage location (AWS S3, Azure Blob, Google Cloud Storage Bucket) by providing additional parameters in the export API call. You will get details about those parameters in this document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/export-service-apis/store-export-results.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-22-2024 23:19
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you. I am able to kickoff the export job using an API. Created a python script that gets the token first and then invokes export API using the token....

I can see that the export job is kicked off.....

from the same script, how to check if the job is complete. Also, where this file is available to copy it to one sftp location?

Here is what I am trying in script:

1) Get token

2) Invoke Export API using the token from step1

3) Check the job status, once the job status is complete, download/access the file.

4) Copy the file to exported location.

I am referring to step3 here.

Appreciate your inputs on this. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-14-2024 10:28
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

The Export jobs are a little different than the data loader jobs, you cannot create the export job definition in advance and execute at a latter point in time. So for scheduling your export, the process you will need to follow is

Schedule a script (or integration flow) to run periodically.
From the script or integration flow execute the Export API based on your filter criteria. For example, all entities that has been created or updated between the last job run to current job run time. Here you will need to dynamically modify the filter criteria from the script or integration.

When you say this export job is scheduled, do you mean export get's triggered every day on a specific time ? If the answer is yes, my guess would be that the export was scheduled using any of the method I specified above.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-09-2024 12:45
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you, Diparnab Dey.

We don't have a license for RIH. If I understand correctly, here is what I need to do:

Define an export job using the API.
Schedule it using a Python/Unix script.

We have an export job scheduled in production. I can view its configuration, but I'm not sure how it was set up and how to replicate it in non-production. However, I am certain we don't have an RIH license.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-07-2024 04:00
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

Currently, we do not have the option to schedule Reltio exports from the UI. However, that doesn't mean it cannot be achieved differently. You can schedule the export API per your preferred window from any integration platform (MuleSoft, Informatica IDMC, RIH, or any custom solution) or a script (Python/Unix etc.) which can be scheduled and can call REST APIs.

If your tenant(s) have RIH enabled, it can be done very easily using RIH, where your recipe should run on a schedule and from the recipe, you will trigger the Export API. You will find a convenient recipe (along with the documentation) which Reltio PS has built earlier, for this activity in the Bitbucket repository - https://bitbucket.org/reltio-ondemand/reltio-rih-recipe/src/master/Export/

Review and let me know if you have any follow up question.

Also, there is no out-of-the-box way to move the files from the storage location (S3/ Blob/ GCS) to SFTP location after they are exported, you will need to build an integration flow to achieve that.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-04-2024 12:52
From: Rushyendar Akula
Subject: Automate the Reltio export job

Hi,

How do we automate and schedule the export process? I dont see an option on UI. Also, once the file is exported, need to push it one sftp location. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC
------------------------------

9. RE: Automate the Reltio export job

Recommend

Utsa Das

Posted 10-23-2024 12:20

Hi Diparnab,

Thanks for the reply.

We are designing the publish process at our end. Now we have two level relationships at our data model like below :-

Entity1 --> Entity2 --> Entity3

Now Entity2 details will be present at Entity1 through reference attribute but Entity3 can not be access directly from Entity1 which we have to go through Entity2 explicitly.

As we have to send one common view to downstream containing Entity1 ,Entity2 and Entity3 considering Entity1 as the parent ,

what will be most efficient way to join all these 3 entities to form a common view with the attributes from all 3 entities for bulk data publish ?

------------------------------
Utsa Das
Novartis
------------------------------

Original Message

Original Message:
Sent: 10-23-2024 08:24
From: Diparnab Dey
Subject: Automate the Reltio export job

That's right Utsa. Reltio will store the export results in compressed format in your cloud location. You will then have to use a process to uncompress the data for further processing.

Also, the default compression method provided by Reltio is GZIP. However, it can be changed to ZIP. You will just have to raise a Reltio support ticket and request for the fileCompression to be changed from GZIP to ZIP.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-23-2024 08:07
From: Utsa Das
Subject: Automate the Reltio export job

Hi Diparnab,

When you are storing the export files in the customer's cloud location , I assuming the type of the file will be zip.

So to access the files inside it , we will need to build a process to unzip the same if the export job is to be automated through RIH .Is that the correct assumption ?

------------------------------
Utsa Das
Novartis

Original Message:
Sent: 10-23-2024 04:18
From: Diparnab Dey
Subject: Automate the Reltio export job

Hi Rushyendar,

Glad to hear that you could trigger the export API from the Python script. You are on the right track here. When you triggered the export job, it should have returned a response like below

{    "exportType": "ENTITIES",    "version": "v2",    "status": "scheduled",    "details": "Export job has been scheduled. Result will be sent to your email: user@example.com",    "taskIds": [        "831272d6-1685-49cc-a98c-c8965d04bc0d"    ]}

You will get the information in this documentation as well.

Now, to check for the export job status, you will need to use the Task Id from the Export job response and trigger another API for getting the export job status.

Refer to the documentation in the link above. But I am also sharing the CURL command for the API

curl --location 'https://{{env}}.reltio.com/jobs/{{mdmTenant}}/tasks/{{ExportTaskID}}' \--header 'Authorization: Bearer {{accessToken}}' \--header 'Content-Type: application/json'

If the export task status is completed, you will need get a link for downloading the exported data in the response. Please note that based on the volume of the data, the export API will take some time to complete. So you will need to figure out how frequently you are going to trigger the status check after triggering the job.

Also, you have a way to directly store the exported data in your cloud storage location (AWS S3, Azure Blob, Google Cloud Storage Bucket) by providing additional parameters in the export API call. You will get details about those parameters in this document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/export-service-apis/store-export-results.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-22-2024 23:19
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you. I am able to kickoff the export job using an API. Created a python script that gets the token first and then invokes export API using the token....

I can see that the export job is kicked off.....

from the same script, how to check if the job is complete. Also, where this file is available to copy it to one sftp location?

Here is what I am trying in script:

1) Get token

2) Invoke Export API using the token from step1

3) Check the job status, once the job status is complete, download/access the file.

4) Copy the file to exported location.

I am referring to step3 here.

Appreciate your inputs on this. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-14-2024 10:28
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

The Export jobs are a little different than the data loader jobs, you cannot create the export job definition in advance and execute at a latter point in time. So for scheduling your export, the process you will need to follow is

Schedule a script (or integration flow) to run periodically.
From the script or integration flow execute the Export API based on your filter criteria. For example, all entities that has been created or updated between the last job run to current job run time. Here you will need to dynamically modify the filter criteria from the script or integration.

When you say this export job is scheduled, do you mean export get's triggered every day on a specific time ? If the answer is yes, my guess would be that the export was scheduled using any of the method I specified above.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-09-2024 12:45
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you, Diparnab Dey.

We don't have a license for RIH. If I understand correctly, here is what I need to do:

Define an export job using the API.
Schedule it using a Python/Unix script.

We have an export job scheduled in production. I can view its configuration, but I'm not sure how it was set up and how to replicate it in non-production. However, I am certain we don't have an RIH license.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-07-2024 04:00
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

Currently, we do not have the option to schedule Reltio exports from the UI. However, that doesn't mean it cannot be achieved differently. You can schedule the export API per your preferred window from any integration platform (MuleSoft, Informatica IDMC, RIH, or any custom solution) or a script (Python/Unix etc.) which can be scheduled and can call REST APIs.

If your tenant(s) have RIH enabled, it can be done very easily using RIH, where your recipe should run on a schedule and from the recipe, you will trigger the Export API. You will find a convenient recipe (along with the documentation) which Reltio PS has built earlier, for this activity in the Bitbucket repository - https://bitbucket.org/reltio-ondemand/reltio-rih-recipe/src/master/Export/

Review and let me know if you have any follow up question.

Also, there is no out-of-the-box way to move the files from the storage location (S3/ Blob/ GCS) to SFTP location after they are exported, you will need to build an integration flow to achieve that.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-04-2024 12:52
From: Rushyendar Akula
Subject: Automate the Reltio export job

Hi,

How do we automate and schedule the export process? I dont see an option on UI. Also, once the file is exported, need to push it one sftp location. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC
------------------------------

10. RE: Automate the Reltio export job

Recommend

Reltio Employee

Diparnab Dey

Posted 10-24-2024 09:08

Hi Utsa,

I will recommend exporting the entity data and relationship data separately. In the entities export, you will get the information for all 3 entities. And in the relations export you will get the relationship with the start and end object information. You can then join those exported data with the relevant entity ids and create the view you want.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal
------------------------------

Original Message

Original Message:
Sent: 10-23-2024 12:20
From: Utsa Das
Subject: Automate the Reltio export job

Hi Diparnab,

Thanks for the reply.

We are designing the publish process at our end. Now we have two level relationships at our data model like below :-

Entity1 --> Entity2 --> Entity3

Now Entity2 details will be present at Entity1 through reference attribute but Entity3 can not be access directly from Entity1 which we have to go through Entity2 explicitly.

As we have to send one common view to downstream containing Entity1 ,Entity2 and Entity3 considering Entity1 as the parent ,

what will be most efficient way to join all these 3 entities to form a common view with the attributes from all 3 entities for bulk data publish ?

------------------------------
Utsa Das
Novartis

Original Message:
Sent: 10-23-2024 08:24
From: Diparnab Dey
Subject: Automate the Reltio export job

That's right Utsa. Reltio will store the export results in compressed format in your cloud location. You will then have to use a process to uncompress the data for further processing.

Also, the default compression method provided by Reltio is GZIP. However, it can be changed to ZIP. You will just have to raise a Reltio support ticket and request for the fileCompression to be changed from GZIP to ZIP.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-23-2024 08:07
From: Utsa Das
Subject: Automate the Reltio export job

Hi Diparnab,

When you are storing the export files in the customer's cloud location , I assuming the type of the file will be zip.

So to access the files inside it , we will need to build a process to unzip the same if the export job is to be automated through RIH .Is that the correct assumption ?

------------------------------
Utsa Das
Novartis

Original Message:
Sent: 10-23-2024 04:18
From: Diparnab Dey
Subject: Automate the Reltio export job

Hi Rushyendar,

Glad to hear that you could trigger the export API from the Python script. You are on the right track here. When you triggered the export job, it should have returned a response like below

{    "exportType": "ENTITIES",    "version": "v2",    "status": "scheduled",    "details": "Export job has been scheduled. Result will be sent to your email: user@example.com",    "taskIds": [        "831272d6-1685-49cc-a98c-c8965d04bc0d"    ]}

You will get the information in this documentation as well.

Now, to check for the export job status, you will need to use the Task Id from the Export job response and trigger another API for getting the export job status.

Refer to the documentation in the link above. But I am also sharing the CURL command for the API

curl --location 'https://{{env}}.reltio.com/jobs/{{mdmTenant}}/tasks/{{ExportTaskID}}' \--header 'Authorization: Bearer {{accessToken}}' \--header 'Content-Type: application/json'

If the export task status is completed, you will need get a link for downloading the exported data in the response. Please note that based on the volume of the data, the export API will take some time to complete. So you will need to figure out how frequently you are going to trigger the status check after triggering the job.

Also, you have a way to directly store the exported data in your cloud storage location (AWS S3, Azure Blob, Google Cloud Storage Bucket) by providing additional parameters in the export API call. You will get details about those parameters in this document - https://docs.reltio.com/en/explore/get-going-with-apis-and-rocs-utilities/reltio-rest-apis/integrate-apis/export-service-apis/store-export-results.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-22-2024 23:19
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you. I am able to kickoff the export job using an API. Created a python script that gets the token first and then invokes export API using the token....

I can see that the export job is kicked off.....

from the same script, how to check if the job is complete. Also, where this file is available to copy it to one sftp location?

Here is what I am trying in script:

1) Get token

2) Invoke Export API using the token from step1

3) Check the job status, once the job status is complete, download/access the file.

4) Copy the file to exported location.

I am referring to step3 here.

Appreciate your inputs on this. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-14-2024 10:28
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

The Export jobs are a little different than the data loader jobs, you cannot create the export job definition in advance and execute at a latter point in time. So for scheduling your export, the process you will need to follow is

Schedule a script (or integration flow) to run periodically.
From the script or integration flow execute the Export API based on your filter criteria. For example, all entities that has been created or updated between the last job run to current job run time. Here you will need to dynamically modify the filter criteria from the script or integration.

When you say this export job is scheduled, do you mean export get's triggered every day on a specific time ? If the answer is yes, my guess would be that the export was scheduled using any of the method I specified above.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-09-2024 12:45
From: Rushyendar Akula
Subject: Automate the Reltio export job

Thank you, Diparnab Dey.

We don't have a license for RIH. If I understand correctly, here is what I need to do:

Define an export job using the API.
Schedule it using a Python/Unix script.

We have an export job scheduled in production. I can view its configuration, but I'm not sure how it was set up and how to replicate it in non-production. However, I am certain we don't have an RIH license.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC

Original Message:
Sent: 10-07-2024 04:00
From: Diparnab Dey
Subject: Automate the Reltio export job

Rushyendar,

Currently, we do not have the option to schedule Reltio exports from the UI. However, that doesn't mean it cannot be achieved differently. You can schedule the export API per your preferred window from any integration platform (MuleSoft, Informatica IDMC, RIH, or any custom solution) or a script (Python/Unix etc.) which can be scheduled and can call REST APIs.

If your tenant(s) have RIH enabled, it can be done very easily using RIH, where your recipe should run on a schedule and from the recipe, you will trigger the Export API. You will find a convenient recipe (along with the documentation) which Reltio PS has built earlier, for this activity in the Bitbucket repository - https://bitbucket.org/reltio-ondemand/reltio-rih-recipe/src/master/Export/

Review and let me know if you have any follow up question.

Also, there is no out-of-the-box way to move the files from the storage location (S3/ Blob/ GCS) to SFTP location after they are exported, you will need to build an integration flow to achieve that.

------------------------------
Diparnab Dey
Technical Consultant
Reltio
Kolkata, West Bengal

Original Message:
Sent: 10-04-2024 12:52
From: Rushyendar Akula
Subject: Automate the Reltio export job

Hi,

How do we automate and schedule the export process? I dont see an option on UI. Also, once the file is exported, need to push it one sftp location. Thank you.

------------------------------
Rushyendar Akula
Datasys Inc
Concord NC
------------------------------

Reltio Connect