Hi Diparnab/Bhaskar,
Thanks for all your suggestions.
We are looking into the option of RIH.
Related to the scheduling option in Reltio data loader, I have checked but seems it will suffice our requirement . Please find the details about the use-case.
Currently we are splitting a large file and storing the split files in S3. For example if we are loading a file having 10 million Individual records, we are first splitting that into 10 files with 1 million records in each file (naming convention is something like a11, a12, a13…..). In this case we want to load these split files one after the other but in scheduler we can only schedule a job definition where the file name is static.
Please let me know your thoughts on this and in-case I am missing anything.
Thanks,
Sourav
------------------------------
SOURAV BANERJEE
Deloitte Consulting India Private Limited
KOLKATA
------------------------------
Original Message:
Sent: 07-21-2022 13:27
From: Bhaskar Nunna
Subject: Automating the Reltio data load process
Hi Sourav,
You can explore Diparnab suggestion to use Reltio Integration Hub but it may require additional contracts/licensing with Workato based on how your agreement was defined with Reltio. Reltio Integrations Hub used Workato framework behind the scenes. We tried that approach at our company but could not proceed further because of the licensing terms I explained above. However, we use Informatica Cloud (IICS) to load data in to Reltio using Reltio APIs. On a high level, this is how we do it.
1. Load Accounts to Staging tables Using IICS (We use Amazon Redshift to stage the data coming from excel files or 3rd party vendors)
2. Validate data integrity and ensure the Account profiles have proper information populated before loading in to Reltio (Ensure Account completeness like Address info etc)
3. Use IICS to load data in to Reltio (Rest APIs and Bulk load APIs)
4. Let the magic happen in Reltio (further Data enrichments, profile completeness, matching/merging etc)
5. Extract data from Reltio and load in the Amazon Redshift tables for further consumption by downstream applications and dashboards using IICS
Hope this helps.
regards,
Bhaskar
------------------------------
Bhaskar Nunna
Original Message:
Sent: 07-20-2022 07:52
From: SOURAV BANERJEE
Subject: Automating the Reltio data load process
Hi,
I am currently exploring options on how to automate the Reltio data load process.
Current approach –
We are using the data loader option from Reltio UI to load csv files (stored in S3 bucket) manually.
Future approach –
We are planning to remove the manual process of loading csv files one by one and automate the process of loading the records which are there in csv files in Reltio from S3 bucket.
Can someone please let me know few options on how to achieve this?
Let me know if you need additional input.
------------------------------
SOURAV BANERJEE
Deloitte Consulting India Private Limited
KOLKATA
------------------------------