@Saurabh AgarwalWe currently have just shy of 500 entries in the DLQ and that number has been relatively consistent since early October.
Visibility into the entries in the DLQ would permit us to know what entities may be impacted so that we can research further to take possible corrective action. A failure to ever process the DLQ entries results in a loss of data. A significant delay results in time sensitive data changes being "missed".
The fact that DLQ entries will be lost after 14 days as we are running on AWS, further requires visibility before the entires are lost forever.
With the 500 entries in the DLQ and no further visibility, I don't know whether it's one entity, 500 entities or something in between.
It would also be helpful to understand the types of errors that are occurring, especially before they are fixed by Engineering so that we might understand how to minimize or mitigate the occurences of same.
Thanks,
Mark
------------------------------
Mark Burlock
Dodge Data & Analytics
Hamilton NJ
------------------------------
Original Message:
Sent: 11-04-2022 11:20
From: Saurabh Agarwal
Subject: Best Practices For Handling Events in Reltio DLQ
Hi Mark,
We do retry 200 times at a gap of 15 mins before the entries are written to DLQ.
Frequently occurring errors are identified and the engg team tries to find permanent fixes for those errors to stop them from happening again.
The IRS is the automated process of identifying data inconsistencies in the tenant and retriggers the failed events to maintain data consistency in the tenant.
Can you provide more details on the need to read the DLQ?
Thanks,
------------------------------
Saurabh Agarwal
Original Message:
Sent: 11-03-2022 21:06
From: Mark Burlock
Subject: Best Practices For Handling Events in Reltio DLQ
What are folks doing to handle entries in the Reltio Deadletter Queue (DLQ):
- How are the specific entries in the queue identified and read
- What approach is used to address the errors that occur especially as the issue would appear to be in the Reltio stack
- Are you using the 200 attempts threshold before entries are written to the DLQ
- What have you been able to automate
- What role is Reltio (Support or otherwise) playing in managing the queues, identifying issues, resolving the root cause and reprocessing the previously queued entry
Thanks,
Mark
------------------------------
Mark Burlock
Dodge Data & Analytics
Hamilton NJ
------------------------------