While doing data load , in our implementation we have used Console DataLoader. We can also perform the data loading using the API call. Apparently we tested with small sample data using DataLoader and API call. We noticed API call takes lesser time. But API call might need an optimum number of threads and records per thread for a full blown system. Also we have to take care of the complete error handling of data loading (success / failure) and the traceability of those records. With this understanding we think even though console - data loader is taking little more time to load the data, we should use that to ensure other aspect is covered.
Could you please share your recommendation between API call or Data Loader ? What are the advantages of using API call for data loading ? Also will this impact that maximum Limit API call in data ?
I prefer an optimized pipeline based out of Reltio API, its a one time investment which allows user to perform pre data cleansing operations, business specific data transformation, customized logs, retry mechanism etc. On the other hand data loader can not support all above at its current state and additionally requires set of files specially for nested attribute to be loaded sequentially.Few customer do prefer a hybrid approach e.g., IDL via data loader and ongoing delta load via Reltio API as a phased implementation approach.
100 Marine Pkwy #275, Redwood City, CA 94065
+1 (855) 360-DATA+1 (855) 360-3282
© 2022 Reltio. All Rights Reserved
Site by eConverse MediaAlso of Interest: Master Data Management (MDM) Reltio Community Reltio Integration Hub Community, Data Management Community Matching, Merge, Survivorship Reltio Workflow Master Data Management (MDM)