For the entity. And some additional functionality. I'm going to talk about additional functionality here a bit, but it will be available as well in this release is they'll be streaming payload of data only. So only things that changed in the entity will be streamed as a payload in the same manner as activity log is structured to this same type of JSON.
It'll be documented. It's something we're working on right now. And I think I'll be able to talk more about this later in August 2021.
Sr. Product Manager
Original Message:
Sent: 08-17-2021 16:55
From: Chris Detzel
Subject: Is outbound APIs from Reltio to Pub/Sub constrained by a limit of 300 records?
@Dmitry Blinov
Is the get call for the entity, or is the get call for what would have been in the event?
------------------------------
Chris Detzel
Community Program Manager
Reltio
Original Message:
Sent: 08-17-2021 16:54
From: Dmitry Blinov
Subject: Is outbound APIs from Reltio to Pub/Sub constrained by a limit of 300 records?
In the current release we are working on, which will be available later in October of this year, we add some additional capabilities which will make your workflow with queues recommended workflow with queues like this. You can stream events with payload into the outbound queue.
Our statistics says that for most of our customers, absolute number of events will feed the size of limitation of the SQS, which is kind of 56 kilobytes currently. But we added additional flag. So basically, you just listen to the messages from the queue, and listen all of them, and you stream all of them.
But if for some reason there was a single event that was too big to go through the queue, you will still receive it without actual payload, but with the flag telling you that this event was too big to go through the queue. You will have to do one additional get call through the API to get this specific payload through the API.
------------------------------
Dmitry Blinov
Sr. Product Manager
Reltio
Original Message:
Sent: 08-17-2021 16:53
From: Matthew Gagan
Subject: Is outbound APIs from Reltio to Pub/Sub constrained by a limit of 300 records?
I don't know of any limits on the outbound streaming side of things. The limits that I am aware of are specific to what queue mechanisms or streaming mechanisms you're using. There are limits to payload size, and I'm thinking about difficulties customers have encountered with very large, consolidated records in Reltio, needing to be posted in their entirety onto an SQS queue. And the size of the message is prohibitively, running into the limit of SQS and therefore, you've got a message that doesn't really have the integrity.
for those reasons, upstream in your control center, you do have the ability to configure what happens in those scenarios. And that I believe, can also now be conditional on the payload size. So not a one size-fits-all don't-publish-anything onto the queue, just the entity ID versus okay, publish it if it fits. Else, hold it back. I'm not aware of any limitations.
If you missed it, take a look at the webinar: Event-driven Integration with Reltio, including a look at Reltio with Kafka
------------------------------
Matthew Gagan
Original Message:
Sent: 08-17-2021 16:50
From: Chris Detzel
Subject: Is outbound APIs from Reltio to Pub/Sub constrained by a limit of 300 records?
I'm guessing the Reltio Consumer would need to incorporate pagination for those use cases? #Kafka
#ReltioInsights
------------------------------
Chris Detzel
Community Program Manager
Reltio
------------------------------