Reltio Integration Hub

 View Only
  • 1.  Processing files in sequence

    Reltio Partner
    Posted 09-18-2025 04:31

    Hi Team,

    Requirement: 
    When 30K records are sent in a file, the client-side process splits it into three files. For each file, an RIH webhook is triggered. These files need to be processed sequentially, one after another and also all child jobs needs to be completed before triggering the next file, in RIH

    Current Design:
    We have a main recipe (Recipe 1) that handles business logic.
    Recipe 1 calls child recipes to:
    Load entity data (Recipe 2)
    Load relationship data (Recipe 3)
    Trigger the workflow for newly registered profiles (Recipe 4)
    All of these are called synchronously. The scheduled runs are working as expected.

    Proposed Change:
    To support sequential processing, we introduced a webhook-based recipe (Recipe 0) as the parent recipe.
    The current main recipe (Recipe 1) was moved under Recipe 0 as a child, with concurrency set to 1.

    Issue:
    When the webhook is triggered with multiple files, instead of processing them sequentially, Recipe 1 is still running in parallel. This is not the expected behavior.

    I understand RIHs concurrency only queues jobs created by scheduling or looping within the recipe, not independent webhook requests.
    Can you please let me know if there is any other ways to do it and acheive this requirement?  



    ------------------------------
    Rahul Rajan
    Fresh Gravity
    ------------------------------


  • 2.  RE: Processing files in sequence

    Reltio Employee
    Posted 09-19-2025 02:55

    Hi Rahul, 

    Are you using RIH itself to load the data using the "Create Records in Reltio" connector action ? Or are using using RIH to just orchestrate the Console Data loader APIs?



    ------------------------------
    Diparnab Dey
    Technical Consultant
    Reltio
    Kolkata, West Bengal
    ------------------------------



  • 3.  RE: Processing files in sequence

    Reltio Partner
    Posted 09-19-2025 09:59

    Hi Rahul,

    Hope you are doing fine. If I understand it correctly, recipe should run through loop sequentially through all the files one by one, right? if so, can you try to add variable at the start and end of each file process and keep it as dependency for each file to process.

    Let's say, set variable "Processing" to No and keep it in if condition to run the recipe. While First file is processed update the variable value to "Yes" so that and at the end of the RIH recipe set it to "No". Hope it will work. Please let me know if my understanding is not correct.



    ------------------------------
    Thanks and Regards
    Chenchu Giri Manduva
    ------------------------------



  • 4.  RE: Processing files in sequence

    Founding Member
    Posted 09-22-2025 07:40

    Hi Rahul,

    Concurrency of parent recipe is enforced on child recipes - you might want to check if that's what is at play here. I would suggest implementing a wait mechanism to control the sequential flow - can be done by using workato queue or by using wait step for recipes by recipe IDs.  



    ------------------------------
    Ashish Rawat
    Associate Director
    Radisson Hotel Groups
    Bangalore
    ------------------------------