Reltio Integration Hub

Python App in RIH can execute for only 90 seconds, Is there any data volume/size restriction too ?

Hi @Chris Detzel tagging you to check if you have someone in mind who can help me with this RIH query

Hi Ashish,

It would have been nice to know if you are trying to solve any specific use case using Python script, or if it was a generic query. As there could be other options available within RIH depending on the use case at hand.

Havin said that, Python scripts within RIH Recipes run on dedicated sandbox machines with restrictions on the memory usage, so there are chances of these scripts erroring out, once this amount of data is hit. If no other RIH component is solving your use case and Python script is the only option, then I would recommend to chunk the data to be processed it into smaller blocks. There is no hard and fast rule to determine the block size, it depends on the complexity of the processing planned within the script and the structure of the input data. The optimum block size to be processed can be determined by doing some trial and error testing.

RIH is a great tool and it has a ton of components/connectors to handle different use cases, please continue to explore and all the best.

Regards,
Arjun

Hi Mallikarjuna,

Thanks for your response. We are using python for a specific use case where a data points need to be filtered based on multiple  business use case. Motive to use python is to reduce the task count for overall execution. We are currently trying hit and trial to learn what's the right batch size to avoid "internal errors" due to memory constraints. Any reference to what volume or execution python app can handle would be helpful as if it is unknown it could lead to lot of re-procession.