Reltio Connect

 View Only
  • 1.  Full Data Migration Best Practices

    Founding Member
    Posted 05-06-2021 08:08
    Hi all, 

    Can you please share your best practices/challenges for Full data migration (Full Data Load)? What can Reltio share and how are others are tackling this challenge? 

    ------------------------------
    Ram Dornala
    AMFAM
    ------------------------------


  • 2.  RE: Full Data Migration Best Practices

    Reltio Employee
    Posted 05-06-2021 12:19
    Hi Ram, There is a Reltio training course "Loading Data into Reltio" that is available on Reltio Academy. One of the modules covers some best practices. The training is free.

    ------------------------------
    Terence Kirk
    ------------------------------



  • 3.  RE: Full Data Migration Best Practices

    Founding Member
    Posted 05-11-2021 10:47
    Hi Ram,

    The answer I would give would strictly depend on the volumes involved.

    If the volumes are small (<1MIL), I would definitely recommend Data Loader. This tool has worked great for us. If the volumes are >1 MIL, I have only personally used the java utility for this. This works great from a Linux VM, as you can submit it to run as a background task.

    I know you can also do >1MIL loads via Data Loader using an S3 bucket as the source. I have not used Data Loader with these volumes, so cannot vouch personally. 

    Regards,
    Walt

    ------------------------------
    Walt Feldman
    Digital Data Lead
    Subway
    ------------------------------



  • 4.  RE: Full Data Migration Best Practices

    Founding Member
    Posted 05-13-2021 08:42
    Hi Ram, 

    I agree with Walt,  it depends on the data volume and #sources involved.

    If it is a small update,  I will run it myself via the Console Data Loader. 

    We utilize the Java Data Loader for Initial Data Loads,  our automated jobs that run daily/monthly for Delta data changes.

    We also use the ​Java Data Loader if we have a manual load and it involves multiple sources.  Since the Console data loader requires identifying the source you would need to create a separate load for each source. 

    We have not used the S3 or AWS services but would be interested to learn more if there is a different/ better way!.

    Angie

    Angela Wawrzaszek
    MDM Manager and Application Enablement
    Hillrom Inc.


    ------------------------------
    Angela Wawrzaszek
    Manager, MDM governance and Application Enablement
    ------------------------------