Reltio Connect

 View Only
  • 1.  Data Loading Approach

    Reltio Partner
    Posted 08-14-2023 17:43

    While doing data load , in our implementation we have used Console DataLoader. We can also perform the data loading using the API call. Apparently we tested with small sample data using DataLoader and API call. We noticed API call takes lesser time. But API call might need an optimum number of threads and records per thread for a full blown system. Also we have to take care of the complete error handling of data loading  (success / failure) and the traceability of those records. With this understanding we think even though console - data loader is taking little more time to load the data, we  should use that to ensure other aspect is covered.

    Could you please share your recommendation between API call or Data Loader ?  What are the advantages of using API call for data loading ? Also will this impact that maximum Limit API call in data ?


    Basudha Ghosh
    Berkeley Heights

  • 2.  RE: Data Loading Approach

    Founding Member
    Posted 08-16-2023 09:26

    I prefer an optimized pipeline based out of Reltio API, its a one time investment which allows user to perform pre data cleansing operations, business specific data transformation, customized logs, retry mechanism etc. On the other hand data loader can not support all above at its current state and additionally requires set of files specially for nested attribute to be loaded sequentially.
    Few customer do prefer a hybrid approach e.g., IDL via data loader and ongoing delta load via Reltio API as a phased implementation approach. 

    Ashish Rawat
    Sr. Manager
    Fresh Gravity