Reltio Connect

 View Only
  • 1.  Does the external queues feature allow us to use a fifo AWS SQS queue or a fifo AWS SNS Topic?

    Reltio Employee
    Posted 03-02-2022 14:38

    I understand that RIH supports this feature based on the response in this thread:

    Does Reltio Integration Hub support for AWS SQS and SNS including FIFO?

    However, a customer has a question about our managing external queues unrelated to Integration Hub:

    Managing External Queues

    Does the external queues feature allow us to use a fifo AWS SQS queue or a fifo AWS SNS Topic?

    And if it does, what is the value set for message group id?

    Thanks,
    Julie



  • 2.  RE: Does the external queues feature allow us to use a fifo AWS SQS queue or a fifo AWS SNS Topic?

    Reltio Employee
    Posted 03-03-2022 13:40
    Hi @Julie Strubinger
    For scenarios that include parallel processing, concurrent event generation - the platform does not guarantee the order of processing in the same order that requests are added to the queue, because of various asynchronous processes run within the platform. In such scenarios, the use of FIFO is not relevant, and instead, we recommend the use of object version and message timestamp to order events.

    Is the business use case specific to using a FIFO queue? Can you share additional details? 

    Thanks,
    Shantanu


  • 3.  RE: Does the external queues feature allow us to use a fifo AWS SQS queue or a fifo AWS SNS Topic?

    Reltio Employee
    Posted 03-21-2022 11:29
    Thanks for your reply, @Shantanu Indra.

    The use case is fairly simple: customer would like to make sure we receive and process the messages in the order that the updates are made to avoid racing conditions between parent and child records.

    Object version and message timestamp is only useful to avoid processing two updates in the same object in the wrong order, but it does not give us any assurance of order of updates between multiple dependent objects.

    Just to be clear, it’s fine if the answer is that Reltio cannot publish to FIFO queues or topics. I just need a more direct answer to the question itself if possible.


  • 4.  RE: Does the external queues feature allow us to use a fifo AWS SQS queue or a fifo AWS SNS Topic?

    Reltio Employee
    Posted 04-13-2022 09:28
    @Julie Strubinger, to be clear, Reltio does not use FIFO queues. @Mike Frasca can answer in more detail, but that's the direct answer. ​​


  • 5.  RE: Does the external queues feature allow us to use a fifo AWS SQS queue or a fifo AWS SNS Topic?

    Reltio Employee
    Posted 04-13-2022 09:59
    @Julie Strubinger  As Chris outlined, Reltio does not use FIFO queues, and the recommendation for how to handle this would be via the ObjectVersion or CommitTime values within the message itself.  I am not sure that I understand the usecase of dependent objects as we track updates to all of these objects independently, and there could n number of cascading updates due to a single request.  If we could setup time to discuss in more detail, that would be great.  We use these same fields ObjectVersion and CommitTime for our internal processing and via our connectors to handle streaming of events without any issues with race conditions on that processing.​