Reltio Connect

 View Only
  • 1.  Staggered Bulk Loads

    Reltio Partner
    Posted 01-11-2024 17:45

    Does anyone have experience with taking a staggered approach to loading the Reltio data tenant with D&B information? 

    There's a need to match-and-append D&B information for approximately 14MM records.  Initial load testing resulted in acceptable processing times for approx 2MM records.  We'd like to understand if we could load the full 14MM in 2MM segments over a 7 day period (1 segment processed each night for a week).

    Regards,

    -John 

       



    ------------------------------
    John Caiafa
    D&B
    Ridgewood NJ
    ------------------------------


  • 2.  RE: Staggered Bulk Loads

    Reltio Employee
    Posted 02-05-2024 10:53

    Hi John, 

    Would you please expand on what you mean by 'acceptable processing times'?  Additionally, how are you loading the data? i.e. Data Loader?  ROCS utility?  D&B connector?  

    If I understand correctly, its seems if you are asking if one load would affect the other?  Can you expand on what you are trying to do?

    Thanks,



    ------------------------------
    Gino Fortunato
    Senior Solution Engineer
    Reltio
    ------------------------------



  • 3.  RE: Staggered Bulk Loads

    Reltio Partner
    Posted 02-05-2024 14:07

    Thx Gino - 

    We're trying to get an understanding if we could break up a 14MM record load into Reltio through the D&B connector.  We were able to run approx 2MM records through the Reltio connector in several hours (run during off-peak hours).  Knowing we could manage 2MM records is a good first step.  The next item we like to address is running multiple files (approx 2MM per run) through the D&B connector over several days.  

    You're correct in your understanding.  Would you happen to know if we can run multiple files through the D&B connector without negatively affecting the prior accounts loaded? 

    Appreciate your feedback Gino.

    -John 



    ------------------------------
    John Caiafa
    D&B
    Ridgewood NJ
    ------------------------------



  • 4.  RE: Staggered Bulk Loads

    Reltio Employee
    Posted 02-05-2024 14:43

    Thanks John,

    When data is loaded into Reltio, a crosswalk from that source is created.  Data loaded via the connector. will normally run through the normal match/merge process taking advantage the match rules to find duplicates.  Depending on the data and the match rules, you will find matches that are automatically resolved or sent to data stewards for review.  Matched record will go through survivorship to determine operational values.  I would not normally think of that as negatively affecting the existing data, but don't know your use case.  But from a volume perspective, you are safe.  



    ------------------------------
    Gino Fortunato
    Senior Solution Engineer
    Reltio
    ------------------------------