Join @Saurabh Agarwal, Senior Product Manager at Reltio as he walks us through building complex survivorship strategies in Data Molder Learn how you can use the enhanced survivorship page in data modeler to build complex survivorship strategies using fallbacks and filters.
Check out the Reltio Community if you have questions: https://community.reltio.com/home
Summary:
- The last Reltio Community show is focused on building complex survivorship strategies in data modelers and will feature a presentation and demo by Saurabh Agarwal, a senior product manager at Reltio, who will showcase the new features and changes in the survivorship strategies configuration on the data modeler, which has been moved from the sources UI and is now more complete and enhanced to allow for configuring groups, fallback conditions, and filter conditions.
- A new UI for survivorship configuration has been introduced and allows for creating, editing, and deleting survivorship groups. The UI shows a list of all existing groups and details such as roles with access and number of attributes defined. The UI also supports adding new attributes to a group and defining survivorship strategies, such as aggregation or fallback strategies, using a drop-down menu. The order of priority for sources can be adjusted and sources can be removed and added back as needed. However, the UI does not allow undoing changes to an attribute.
- Different groups with different survivorship rules can be selected for a user in case of conflict, either by default or by providing an explicit survivorship group parameter in the API.
- "It is possible to create conditional nested survivorship rules for a specific attribute source order, with the default being recency, but the functionality is limited compared to overall survivorship configuration. The default survivorship strategy can be changed in the advanced editor via the JSON file."
- With new updates in the tenant configuration, you can apply metadata security on rule sets and select Proof Sets for a specific service account or user to extract OV data. The selection of rule sets on the UI is read-only and does not become the default rule set, updating only the operational values for the selected profile. However, the ability to test survivorship strategies before saving a configuration is being worked on for the near future.
Transcript from Show:
Chris Detzel (00:07):
So welcome everyone to the last Reltio Community Show of the Year. Promise to have more next year. But this show is building complex survivorship strategies in data modelers. So it's a little bit more technical, which a lot of you guys love that. So that's always exciting. Today's special guest is Saurabh Agarwal. He's a senior product manager here at Reltio. And he actually came to me directly and asked if he could do this. So there's lots of really cool features that he wants to show today. So we'll just have a small presentation that he'll share. And then we'll go deep into more of a demo, which is always better in my opinion. So we're excited about that. I am the director of customer community and engagement here at Reltio.
(01:02):
So the rules of the show are normal, the same. Keep yourself on mute. All questions should be asked in chat, and/or you can take yourself off mute, ask the question. Just be courteous of hey, we need to get through the presentation as well. And then this will be recorded, and it's being recorded and shared out to everyone on this and others that registered. This might not take the entire time, which is fine, but a lot of times it does, just because of the questions that we have. Saurabh, I'm going to hand this over to you and let you introduce yourself a little bit, and then take it over.
Saurabh Agarwal (01:44):
All right, thanks. Thanks, Chris. Hi everyone. I'm Saurabh. And I've been with Reltio for about one and a half a year. I'm the senior product manager. Currently I'm managing the areas of survivorship, console data modeler and so on. And before I start, I just wanted to give you a brief background as to why did we plan to have this show. So in the past there have been multiple webinars where people from Reltio, they've discussed about, what are the different survivorship strategies? And which survivorship strategy to use under what circumstances? What are different fallbacks and filter conditions, and so on?
(02:24):
So all the previous shows, they were mostly about what to do, and why to use a particular strategy. But the focus of the current show is how to do it. So you might have noticed that recently we've made certain changes in the UI. The new UI does not allow you to change the survivorship strategies. And also there have been announcements on the data modeler. On the data modeler now, you can configure your entire survivorship strategy, big groups, the complex strategy using fallback and filter. So all of these new changes that you're seeing on the UI now, I just wanted to walk through those new features so that you get a good feel about what these new features are, and how do we use it. All right, I'll start sharing my screen. So there's a small presentation, followed by a demo. Hi, I hope you guys can see my screen.
Chris Detzel (03:24):
We can. Yes. Thank you.
Saurabh Agarwal (03:31):
Okay. Yeah, so to just give you a brief background, I just discussed this earlier. All the survivorship, so basically whatever was available to you for configuring these survivorship all on the Sources UI. So on the Sources UI, you could see that for any entity and attribute what is the current survivorship strategy. You could also change the strategy, and you would see that the operational value changes according to the strategy that you select. Now in the new UI, they've removed that capability. In the new UI, you will not be allowed to change the strategies on the Sources UI. And in fact the entire configuration of the survivorship strategies has been moved to Data Modeler. And in the Data Modeler, now you cannot just select a strategy for an attribute, but you can create whole new groups. You can choose whether a group is default or not, and what are the different fallback conditions and strategies that we can configure on an entity, if there are any filter conditions.
(04:33):
So the new feature that we've built on Data Modeler, it is much more enhanced and complete as compared to what you already have. And the main reason that we made these changes, because one, we believe that the survivorship strategy, the configuration of the survivorship strategy is more of a Data Modeler kind of work. So it's somebody who has good knowledge of the Data Model, who understands the data well enough, they are the ones who should decide what the strategy should be used for the entire entity. And what happened with the old Sources UI was, if you change something on a particular profile, that change was applied to the entire entity. It was not your profile. So assume that you are on an employee profile, and you look at a particular employee, say John Doe. And you change the strategy for the first name to, say from frequency to aggregation, or something like that. You might assume that this change is for only this John Deere profile. But in reality it impacts the entire entity.
(05:38):
So all the employees will be impacted with this change. So that is where we believe that this kind of a change, which has such a huge impact on the downstream applications that consume the operational values from Reltio, it was very risky to put in a place where a lot of people had access to it. And that is why we wanted to move the configuration for survivorship strategies from the Sources UI to the Data Modeler. Also, another thing on the old Sources UI, a user might have had access to different rulesets. And if they selected a particular ruleset, again the expectation was that if you select the ruleset, you see the operational values corresponding to that ruleset. But in reality the selection of ruleset actually updated the selected ruleset to the default ruleset.
(06:30):
So again, it had a major impact, or major repercussions on the final operational values that would flow to the downstream system. So that is why we had to restrict certain features that were available on the Sources UI earlier, and we moved those features to the Data Modeler. So in the new Data Modeler, these are the features that you see we've started supporting. So we can create and manage survivorship groups, or rulesets as we call it on the UI. You can create primary strategies.
Speaker 3 (07:05):
So one question. So, what I heard from you, so in new Data Modeler or new UI, we'll not able to change the survivorship group?
Saurabh Agarwal (07:17):
Yes.
Speaker 3 (07:18):
Right. But still we can go to the older UI, right? We can switch that?
Saurabh Agarwal (07:25):
Right.
Speaker 3 (07:26):
So for that... Sorry, go ahead.
Saurabh Agarwal (07:31):
Yeah, so this older UI is only supported for this release, from 23.1. The old UI will be discontinued. And all tenants, all customers will necessarily be moved to the new UI, mandatorily moved to the new.
Speaker 3 (07:44):
Okay. Okay. That's what I need to, because if we are in the older UI, and if we need to disable that, then we may need to do some configuration, correct?
Saurabh Agarwal (07:56):
Yes. But this is going to be disabled from the new UI. So beginning 23.1, that is from the next release onwards, old UI will we discontinued and only new UI will be supported. Currently we are in discussion with multiple customers to understand what are the gaps that they see between the old UI and new UI, any blockers that they see. And we are fixing those blockers so that the new UI is acceptable to all the customers before we roll it out.
Chris Detzel (08:26):
And I think that that date is February. So-
Saurabh Agarwal (08:29):
Yes.
Chris Detzel (08:30):
Good. Yeah. There is another question, but Nicole, I'll ask that here in a little bit. But let's get through some of this before I ask. So keep going.
Saurabh Agarwal (08:41):
Okay. All right. So you can create primary strategies. And then in this new feature that we've developed, we are also supporting more complex survivorship strategies. You can create fallbacks via primary strategy. And these fallbacks can be in multiple levels, nested, sub-nested and so on. You can also create filters, and then you can decide for a particular filter, which is the primary strategy you want to use. If you want to have different strategies for different filters, that is also supported.
(09:11):
And in fact, you can also create fallbacks within the filters. So you have a filtered strategy. Within that you can also define different fallback criteria, and fallback strategies. We've also built an Advanced Editor, so that if you want to view the survivorship group and the strategy that you've built, if you find it comfortable to view it in adjacent format, you can do that. Or you can also use it to copy the groups across tenants and environments. So it's like something you've tested in there. When you're comfortable with that configuration, you don't need to rebuild it in test, just copy this survivorship group, and [inaudible 00:09:48] in the Advanced Editor for the tenant and [inaudible 00:09:51]. Okay.
Chris Detzel (09:56):
So before you get into the demo, can I ask this one question?
Saurabh Agarwal (10:00):
Yeah.
Chris Detzel (10:01):
So Miguel says, have we reached that point where changing survivorship generates event that gets pushed to SQS? So last we checked, changing survivorship doesn't tell us automatically what customers have been updated, and required a manual analysis and data extract.
Saurabh Agarwal (10:23):
If you change the survivorship, this would not automatically generate events, no.
Chris Detzel (10:34):
Okay.
Saurabh Agarwal (10:34):
Only further-
Chris Detzel (10:34):
And then.
Saurabh Agarwal (10:34):
Yeah, sorry.
Chris Detzel (10:34):
Go ahead. Go ahead.
Saurabh Agarwal (10:34):
No, I'm saying only further updates to entities is what will generate an event. And there you'll be able to see the operational value corresponding to new survivorship groups. But changing the group itself, or changing the strategy itself does not generate an event.
Dmitry Blinov (10:50):
In general, they're all today that data changes are reflected in SQS stream, and metadata changes or configuration changes are not reflected in SQS stream. So there'll be no event generated for metadata or data configuration change.
Speaker 5 (11:07):
That's right, Dmitry Blinov. I think these metadata changes eventually lead to change in the operational value. So it's still a pain point. So every time you change the survivorship you have to manually figure out what customer groups or entities have been updated and then extract them manually out of [inaudible 00:11:28].
Dmitry Blinov (11:28):
Yeah. We'll evaluate. Yeah, it's a good point actually. Maybe we need to create a separate streaming configuration which will be reflecting the metadata changes. But today it's not supported, but we'll evaluate this idea.
Mark (11:39):
It'd be interesting. I don't know how you would identify it, with the way it is right now. I don't want to delay it, but it'd be interesting to know what techniques, if any, people are using to identify maybe at the end of the call.
Chris Detzel (11:59):
I like that, Mark. And make sure you bring that up here at the end. So could one of you guys explain the fallback strategy just quickly? What does that mean?
Saurabh Agarwal (12:13):
Yeah, I'll also cover it during the demo, but just to give you a brief idea. So assume that you define a survivorship strategy of, say aggregation, or whatever survivorship strategy that you define. That returns a non-unique OV, or that does not return an operational value. In that case, you can define fallback strategies so that you always end up getting an OV. So for example, if aggregation returns two values, you can define a fallback strategy of, say recency, and the recency then may return you one unique [inaudible 00:12:45].
Chris Detzel (12:47):
Thank you. And lastly, Robert mentions, OV changes caused by survivorship not being published has been a big pain point for them. So just as an FYI, as you think about this.
Saurabh Agarwal (13:04):
Sorry, what was that again? OV changes-
Chris Detzel (13:08):
Caused by survivorship not being published has been a big pain for them. It's been a pain point. So- \.
Dmitry Blinov (13:16):
So, Saurabh. [inaudible 00:13:17] this, and it's a good feedback.
Chris Detzel (13:19):
Yeah, just good feedback. That's all it is. Sorry.
Mark (13:24):
Yeah, and by the way, Chris, just one comment on that. It's a problem if it is, and it's a problem if it isn't. So there has to be a way of controlling it. It'll kill you, if it does in some cases. And it hurts you, if it doesn't. So just, not one size fits all, I guess.
Chris Detzel (13:40):
That's a good point.
Saurabh Agarwal (13:42):
Okay. So before I go to the configuration, I just wanted to quickly show you what the new UI would look like. Like I mentioned, the new UI does not allow you to change the survivorship strategies. So the Sources UI, you will still be able to see the rule types which are being used for each of those attributes. But nothing of this is editable. So you cannot change the strategies, or you cannot change these settings for a particular strategy. So now going to the configuration within the Data Modeler, you'll notice that after you select an entity, so for example, I choose this entity, you'll find a new tab called, survivorship. Then you select the survivorship tab on the homepage. First you'll see a list of all the existing survivorship groups. So these are all the different survivorship groups which are currently present for this entity. And I have the option to edit or delete a particular group.
(14:59):
You'll also notice that there are certain details like, this one is the default group. Then different roles which have access to these survivorship groups, and total number of attributes for which the roles have been defined in this group. Click on it and you'll see a summary of what the survivorship group is about, different attributes, and what are the rules which are being used for it, whether any fallbacks or filters are defined or not. And if you want to edit it, just click on edit, and it takes you to this page.
(15:33):
For this general purpose, let's create a new group. [inaudible 00:15:43]. So this is the label and description that you currently see in the API. And if for example, I create this group for the marketing team [inaudible 00:15:56] roles, which is for the marketing team. And here I can go ahead and add attributes. So for example, I choose... Sorry, at times the stage hangs. We are still trying to figure out why this happens. But in case it happens to you, you can click on the Advanced Editor and then go back to Roles Builder, and the page will start doing the properties. \.
(16:47):
All right, so choose all the attributes for which you want to define the survivorship strategy as part of this group. Sorry, I already have. And then all the different strategies that you currently support, within all of these strategies are present as a dropdown. For certain strategies and source system, you also have the second item. Within the source system you can drag and drop to determine the order of the source in order of priority. And also if there are certain sources that you do not want to use, you can also remove those sources. All the removed sources, they're at the bottom. And you can include them back if you need to.
Chris Detzel (17:18):
A couple of questions for you. One is with the new UI, I've also noticed that when you're making changes, and this is not a question, when you're making changes to an attribute, it doesn't let you undo them like the old ones, unless visual's missing something. Is that a true statement?
Saurabh Agarwal (17:42):
Sorry, which attribute are you talking about? Is it this page that-
Chris Detzel (17:47):
Was it this page that we're talking about?
Speaker 7 (17:52):
Oh no, I think this meant if you go to an entity and you just delete an attribute on the right side on the old UI, there'd be an undo button on the top of the page. This is not survivorship related, it's just an observation.
Chris Detzel (18:05):
Okay.
Speaker 7 (18:07):
Yeah.
Chris Detzel (18:08):
No that's good to know, because I can talk to Karina about that.
Speaker 7 (18:10):
Yeah, I wasn't sure if that was intentional, or I'm just missing it. Because I think that's a pretty important feature to have. If you accidentally delete something, it's nice to just click the button, and it just undoes itself.
Chris Detzel (18:23):
Yeah. Did you come to the UI webinar a week or two ago with Karina? Might be able to watch that because she went in depth on some of these new features and buttons, and things like that.
Speaker 7 (18:38):
Yeah, I can take a look at that. Thank you.
Chris Detzel (18:39):
I'll send it to you. Don't worry. So some other questions specifically around this. Are groups a way of giving different BUs the options to own survivorship of groups of attributes?
Saurabh Agarwal (18:57):
Or it is also possible, for example, that for the marketing team, you might want to have different strategy for the same attributes. For example, the marketing data for the marketing team is more accurate in their marketing system. So for them, the source system order might be different. But as compared to what you use for the rest of the teams, or other default that you might want to use. So you might want to have a separate strategy, or a separate survivorship group only for the marketing team. So the control still remains with the data admin. But when somebody with this role pitches the entity, they will see the survivorship values which are corresponding to this survivorship group.
Chris Detzel (19:52):
Okay. Keep the questions going. There's one. Yeah, there's another question for role-based, so non-default survivorship. Do any attributes not included leverage the default policy?
Saurabh Agarwal (20:10):
So the way it works is, if the attribute is not defined here by default, they take the [inaudible 00:20:16] strategy, which is recency.
Chris Detzel (20:19):
Okay.
Mark (20:20):
Yeah.
Saurabh Agarwal (20:21):
Plus-
Mark (20:21):
I'm sorry. The question was more that, if the attributes are there, they're populated, but in this group that's role specific, you're only including three of the hundred attributes. Do the other 97 then use the default survivorship?
Saurabh Agarwal (20:39):
Default survivorship, as in the default survivorship for each attribute, which is recency? Or do you mean the default survivorship group?
Mark (20:45):
Default survivorship as defined in the default survivorship group. I'm sorry. Good question for clarity. Yeah. But we have a default survivorship group. It's attribute specific then, right? Any attribute that's not otherwise overwrited in a role based, we'd use the default?
Saurabh Agarwal (21:02):
I tend-
Mark (21:02):
Default survivorship group.
Saurabh Agarwal (21:10):
Dmitry Blinov, are you sure about this one? Or I can check with the engineering team.
Dmitry Blinov (21:11):
Sorry Saurabh, if you can repeat. I got disrupted basically.
Saurabh Agarwal (21:18):
So if we do not have some attributes defined in a particular survivorship group, the remaining attributes, do they use the survivorship group of the survivorship strategy of the default group? Or do they use recency?
Dmitry Blinov (21:26):
They use recency.
Saurabh Agarwal (21:27):
They use recency? All right. Anyway, I'll confirm, because I cannot recall the complete conversation. But we did have a discussion around it sometime back. But I cannot recall it completely right now. I'll confirm and then let you.
Mark (21:51):
That would be great.
Dmitry Blinov (21:52):
Let's confirm, yeah. But by default, recency is used. I think you can even do a quick test, and try it if we have time after with them.
Saurabh Agarwal (22:01):
Yeah. Yeah. All right.
Chris Detzel (22:03):
All right, we'll keep going. There's a few more questions. Actually a lot more, but we'll ask those here in a minute.
Saurabh Agarwal (22:10):
Okay. All right. Similarly, if you use the Other Attribute Winner Crosswalk, see that it also asks you to select the winner attribute. And all the other survivorship strategies that we support, you can choose any of those here. The ones you've created, the group, you can save it. Or you can also go to the Advanced Editor, and look at the survivorship strategies that we just created. The label description, roles and the strategies for each of those attributes.
Speaker 8 (22:51):
[inaudible 00:22:56].
Saurabh Agarwal (22:57):
All right. So this was the most basic... Sorry, was there a question?
Speaker 8 (23:01):
Yeah, so can you elaborate on that Other Attribute Crosswalk? What is that one? Other strategies, we are familiar. But the new one that you added, it looks new to us. Yeah, Other Attributes Winner Crosswalk. What does that mean when you select this survivorship?
Saurabh Agarwal (23:19):
So what this means is, whichever crosswalk is the winner for full name, that same crosswalk will be used to determine the OV for first name. So Other Attribute Winner Crosswalk.
Speaker 8 (23:26):
Okay, so we are not-
Saurabh Agarwal (23:26):
Full name winner.
Speaker 8 (23:35):
Got it. We are not defining it. It's a dynamic based on whatever other field gets survived, use the same thing for these two fields?
Saurabh Agarwal (23:41):
Yes. Yeah.
Speaker 8 (23:46):
Okay. And one other question. If role, we are defining two different groups. One with, say example, this is marketing role, other with, say maybe that administrator role, something like that, right? If both roles have some fields common, or if a user has both the roles, then which survivorship will apply for him? Either this one, that one or when there is a conflict in between roles?
Saurabh Agarwal (24:12):
If the user also has default, then the default will be applied for him. By default, the default survivorship group is applied. Otherwise we are also enhancing the [inaudible 00:24:21] to accept the parameter, called explicit survivorship group. If you provide that parameter, and you give the group name, the group URI, then this OV value corresponding to that group will be applied. If none of them is true, then in that case I think it could be random. The first group that the user is part of, that would be taken up.
Speaker 8 (24:46):
No, no. My question is, if a user has this role, as well as a role from other group, then what happens to him? Now it's conflict of survivorship, right? The other group might have different roles, and this group has different roles. The user has got-
Saurabh Agarwal (25:01):
Right, so in this scenario, the user goes to the UI, he will see option to select the group.
Speaker 8 (25:11):
Oh.
Saurabh Agarwal (25:11):
Okay? So if I have access to-
Speaker 8 (25:14):
Sure.
Saurabh Agarwal (25:15):
If I have access to three different groups on the UI, I'll be able to see all three different groups. By default, the group which is marked as default, that will be shown. But I can go ahead and select another group, and you'll see that it always will change according to the group selected.
Speaker 8 (25:33):
Thanks. When you say user can select, it means the end user who is weaving the data in the tenant, right? So example, the entity. There you'll have some option to choose which group to select?
Saurabh Agarwal (25:45):
Yes.
Speaker 8 (25:46):
Okay. Maybe if we can show [inaudible 00:25:48] the letter, then it will be helpful. Thanks.
Saurabh Agarwal (25:48):
Right. Yeah. Okay. All right. So Chris, were there more questions? Or you want me to continue?
Speaker 9 (25:56):
I have one question.
Saurabh Agarwal (25:56):
Sure, go ahead.
Chris Detzel (25:56):
Go ahead.
Speaker 9 (26:05):
Yeah. So with this different groups, you said if person has access to three different groups, so three different will be shown. And then the strategy can also be different, with this, the master data which we see. So will the values be different? Because master data is a core data where we can rely on, but with this different growth, because the strategies getting changed. So will that make sense? I'm a little concerned on the business value here.
Saurabh Agarwal (26:43):
So by default, the group which is set as default, that is used. So currently when you call an entity or you export data from Reltio, OV values are set as per the default group. And this is-
Speaker 9 (26:44):
The default group?
Saurabh Agarwal (26:54):
The group which is marked as default. And this is an existing feature. So currently also your tenant might already have multiple survivorship groups defined. But because there's no UI there, you can see it. So maybe it is not easily visible. But when you go to L3 configuration, you might see that for an entity there are multiple groups already defined.
Speaker 9 (27:14):
Yeah, actually my question is more on the business side, because master data is then something which we want to trust. So if with different groups we are still changing those strategies. So it might happen then for one master data, when for one of the group I was seeing some other first name, and then I suddenly see some other first name.
Saurabh Agarwal (27:38):
And that [inaudible 00:27:39] it?
Speaker 9 (27:41):
Yeah.
Saurabh Agarwal (27:42):
So that is when you say that this default group is the one which will be used in most scenarios. So if we want the master data to have just one value, or one OV, or consistent OV across the systems, we'd use this default value. But for specific scenarios, if you want to get data for a specific role, you can provide that extra parameter in the API to get the response for a specific survivorship group also.
Speaker 9 (28:10):
Okay. Yeah, thanks.
Chris Detzel (28:11):
All right. So I want to stop the questions for now, and let's go on with the demo. Because I know anytime you change something big like this, there's going to be a ton of questions. And I want to make sure that we get as many as possible. Let's get through the demo. Push your questions in the chat, and let's continue. This is great. This is awesome.
Saurabh Agarwal (28:30):
All right. All right. So I just took a small example where we defined the primary strategy for three different attributes. Now let's go to the fallback and filter condition. So like I mentioned, fallback is used when the primary strategy does not result in a unique OV. In that case you have the option to define a fallback. And we support two different criterias for fallback. One is called, zero or more than one. And the other one is called more than one.
(28:57):
For example, let's take this zero, or more than one example. So what it means is, for the full name, the Primary strategy: Source system, if the source system returns zero or more than one values, in that case I can add a fallback with rule type, say Frequency. So this is how the system will behave internally. First, it'll execute the source system survivorship strategy. And then it'll check for this condition, zero or more than one. If the source system survivorship strategy returns zero or more than one operational values, then all the crosswalks which qualified for the first pass, they will be passed into this rule type. And those crosswalks will be used to determine the survivorship strategy based on this fallback rule type.
(29:49):
This again can be any of the other strategies that we support, or we can have more fallbacks for the same condition. If also, just assume that frequency also ends up returning zero or more than one value. You can again choose another survivorship strategy. All right? So there's no limit to number of fallbacks that you define, but you just have to be cognizant of the fact that the complex strategies that you make more complicated, it's going to be for you to understand those strategies then. So only define to the level that you think is actually practical, and is the actual scenario that you might see in your data. Secondly, these are for the same criteria. You see that I've defined three different strategies which will be executed one after the other. Now as in the condition there, where for zero or more than one, I execute the frequency. Now I want to have a different condition for the case where it is more than one.
(30:48):
So in this scenario I can add a fallback, a nested fallback. Now with this nested fallback, again I can choose whether it's more than one, or zero or more than one criteria. And they can be a different strategy for this condition. Now in this scenario, the [inaudible 00:31:05] execute is, source system is calculated first. If that returns zero or more than one, then frequency is used. If frequency returns more than one, then it'll go to aggregation. If frequency returns zero or more than one... Sorry, if it returns zero, it'll go to the maximum value [inaudible 00:31:24]. So basically these rule types are executed one after the other sequentially. So in certain conditions you might not even need the nested fallback, but you have this ability here, just in case you have a complex scenario that you want to build using the [inaudible 00:31:40] fallback, you have all the capabilities here in the UI.
(31:42):
So after you've defined all the fallback criteria for it, you can see this. And I'll quickly show it to you. I'll show it to you on the Advanced Editor also. So the survivorship strategy is source system, fallback criteria using zero or more than one condition. And then these are all the different fallback criteria that you just build on the UI.
(32:22):
Okay, next I'll go to filters. Okay, so now assume that I have a situation where, based on the value of a particular attribute, I might want to have different strategies. So for example, if the state is California, I want to use data from a specific source. Whereas, if the state is New York, then I want to use data from a completely different source. Or I might want to have a completely different strategy based on which is the state of the data, whether it's address of the person, or the location or whatever. So here in this condition, for example, it's full name. I'll just take an example. This might not make complete sense in terms of business, but I'll just show you how this is done. So for example, for this full name, assume that I want to have a different source system ordering, if the state is, say California and New York.
(33:36):
So if the birth state is California or New York, I want to use the source system, and the order of the source system is this order. So this is for these two states. Now what if it's a different state? I want to have a different strategy. So I add this full name. The way this is implemented on the UI is, you add full name again. Choose the source system again, you add a filter. And say for Georgia, you want to again have a source system, but the order is different. So this is how the survivorship strategy will work in this scenario. We'll first check which filter is satisfied, whether it is CA or New York. If this is the case, this strategy will be used. If it is GA, this strategy will be used. And the final scenario, when it is neither of these two scenarios which satisfy. In that case I need to have another attribute with a default strategy. So full name without any filters. And this could be the default strategy, that if the state is not CA, New York or Georgia, then use this strategy.
(35:06):
So we can build filters with different conditions, filters, supporting different survivorship strategies. And each of those filter conditions can have their own fallback criteria as well. So this can get really complicated, depending on this scenario that you want to support. If you do not need any fallback, you can just leave it [inaudible 00:35:30]. Again, I'll quickly go to the Advanced Editor and show you what the survivorship strategy now looks like for this. Okay, so this is for our attribute name. Source system, all the different fallback criteria with filter, which is equal to CA or NY.
(36:00):
Okay. Then we have first name and middle name. And this is again the name attribute, with filter equals to GA. It says sources order would be different. And finally the default survivorship strategy full name. Then none of the filter conditions needed. All right, so once I've created the survivorship strategies, I can go ahead and save it. And that survivorship strategy is now available on the UI. And now here you have the option to again edit it and delete it. Okay, this was the demo that I wanted to cover. Chris, before I go back to the slides, and discuss about what are the future updates that we're we working on, is there any questions around it? I think we can take them.
Chris Detzel (37:08):
There is. Yeah, so as [inaudible 00:37:10] mentions, as we're looking now at changing survivorship through Data Modeler, does that mean we cannot change it in the source view like we did in the old UI?
Saurabh Agarwal (37:24):
Yes, in the sources view until the old UI is supported, it would be possible. But once the old UI support is removed, and only new UI is available, it won't be supported. So new UI does not support changing survivorship strategies on the sources UI.
Chris Detzel (37:38):
Okay. How can we do conditional nested survivorship rules for a given attribute? So example, source order for specific sources? Or else most-
Saurabh Agarwal (37:49):
Sorry Chris. It's buzzing. I'm not sure if it's me.
Chris Detzel (37:52):
Yeah. Can you hear me okay now? Are you still there? Can you hear me?
Saurabh Agarwal (37:58):
Am I audible?
Chris Detzel (37:58):
I think-
Saurabh Agarwal (37:58):
Chris, your audible. [inaudible 00:38:08].
Chris Detzel (38:08):
Okay, thank you.
Saurabh Agarwal (38:08):
It looks like it's me.
Chris Detzel (38:12):
Yeah. Dmitry Blinov, I don't know if maybe you can help answer these questions. Are you still there? Yeah.
Dmitry Blinov (38:23):
It's almost... Yeah. I'm here. So I can help.
Chris Detzel (38:23):
Yeah. If you-
Saurabh Agarwal (38:26):
Chris, sorry, I cannot hear anyone. I'll log off and log in again.
Chris Detzel (38:30):
Okay, let's wait. So do you want to wait, Dmitry Blinov? Let's see if you can answer him.
Dmitry Blinov (38:36):
Yeah. Let's continue with the question.
Chris Detzel (38:38):
Yes. How can we do conditional nested survivorship rules for a given attribute? So like source order for specific sources? Else most recent source?
Dmitry Blinov (38:53):
Well on the question of how you refer to the, on the conditional, I'll be able to just explain in a short answer. But some of it is possible. In general, survivorship rules [inaudible 00:39:07] nested for things like ordering. It is a limited functionality compared to overall survivorship configuration. On the question of how, I'm not ready to answer right now.
Chris Detzel (39:21):
Okay.
Dmitry Blinov (39:22):
[inaudible 00:39:22] check the connotation of this. Yeah.
Chris Detzel (39:26):
And then [inaudible 00:39:25] says, default is recency, and Dan says yes, the value is [inaudible 00:39:31]. Not sure what's going on there, but [inaudible 00:39:35] says, I think you'd be able to change the default in the Advanced Editor via the JSON file. Thoughts there, Dmitry Blinov?
Dmitry Blinov (39:43):
Yeah, so the way it works, there was a question around, what will be applied by default for the attributes?
Chris Detzel (39:48):
Ah, that's right.
Dmitry Blinov (39:49):
Which will not participate in any group definition or rule definition. So by default I have, because there was some confusion, I have a clear answer. By default, recency will be used in all cases. And if you want to change that, you'll need to include the specific attribute into the survivorship group. The default group, it is default for the user, or for the rule, for the metadata security role. So default means a rule is not defined. Default role group will be used, but default survivorship strategy is recency, always.
Chris Detzel (40:27):
Okay. And we're just going to go through these questions because I'm not sure if Saurabh is on. Saurabh, can you hear me?
Saurabh Agarwal (40:37):
I can hear now. Sorry. Sorry, I don't know what happened.
Chris Detzel (40:39):
All right. No worries. So, Dmitry Blinov or Saurabh, whoever wants to answer these. And Saurabh, I'll go to you first. But then if you can't, we'll see if Dmitry Blinov has some thoughts. What's the value of using Other Attribute Crosswalk Winners? So what's the main intent of having that feature?
Saurabh Agarwal (40:56):
So this is one good example. So for example, you take full name from a particular crosswalk. The crosswalk which gives you the full name might also be the right source for the crosswalk, or might also be the right source for the first name and last name. So in this case you create interdependency between these two attributes. Like to say that whichever crosswalks gives you the full name, use the same crosswalk to get the first name and last name. So basically it's used to create interdependency between the attributes, where OV for one attribute is dependent on the OV for other.
Dmitry Blinov (41:43):
Another good [inaudible 00:41:43] oftentimes use the same source for remaining attributes. The same as a winner for your address line one. Because address line one usually have the most important part of the address, and this way you make sure that the rest of the address came from the same source, basically.
Chris Detzel (42:06):
Okay, cool. So one comment here, and I love it, and I think we'll do this. And the reason I say it is because you guys are here, but it could be helpful to have some sessions simply for ask-the-experts versus a demo. So I think I like that idea. Maybe once a month have a specific topic. Could be survivorship, could be match merge or whatever. And then have experts on there ready for questions. Maybe we'll have four or five ready, and then you guys can ask. So if you like that, put that in the chat. So a couple questions down here though. When we extract data from Reltio, which group/role will be applied to get the OVs?
Saurabh Agarwal (42:48):
We use the default survivorship group.
Chris Detzel (42:52):
Okay, next question. Maybe a bad example, but what I mean is, this is probably from earlier, but a certain condition of a survivorship rule is not met, such as not a specified source needed that it will fall back to another survivorship rule such as an attribute winner crosswalk minimum value frequency? It's probably from another question. So I don't know, Cameron, if you want to take yourself off mute, and restate it, if you're still there, Cameron. He's not. He's gone. Next question. If there are different OV values for different business units, which OV value is extracted when extracting the data for the entity? Miguel, I think we said that already.
Saurabh Agarwal (43:45):
Yes. So by default, the default survivorship group is used.
Chris Detzel (43:50):
Okay. Yeah. Is there any limit?
Speaker 9 (43:55):
[inaudible 00:43:56].
Chris Detzel (43:55):
Go ahead.
Speaker 9 (43:58):
Yeah. And when you say default, it takes recency. Is it like that? What is the default?
Saurabh Agarwal (44:06):
No, the survivorship group which is defined as default. So you can have multiple survivorship for an entity. The one which is default, that is used. If the default survivorship group does not have a survivorship strategy defined for an attribute, then recency is used for that attribute.
Speaker 9 (44:23):
Oh, okay. Good. Thank you.
Chris Detzel (44:25):
Is there a limit on number of fallbacks?
Saurabh Agarwal (44:27):
There is no practical limit, but just to ensure that we do not necessarily complicate the survivorship strategy, so there's just a guideline that, do not have too many nested fallbacks and fallbacks. Because that might impact the performance of the number, if the calculation is too long.
Chris Detzel (44:51):
That makes sense.
Dmitry Blinov (44:52):
Also, logically I don't think you'll be able to build too long fallback list, because you resolve traditional value in one way or another. [inaudible 00:45:02].
Chris Detzel (45:03):
Yeah. Has any survivorship strategy functionality actually changed? Or is there just a change in how survivorship strategies are configured in the UI? So the UI is now provided for configuration we previously had to do in the L3 directly. So is there any change? Is it just-
Saurabh Agarwal (45:23):
Right, no change in these strategies. It's only that earlier, a thing that you could only do through the API, you can now do it on the UI.
Chris Detzel (45:33):
Thanks. Great question Michael. Are you able to add filters to survivorship strategies from the UI as you are in the L3? I think that's answered already.
Saurabh Agarwal (45:47):
Yeah. During the demo, we did it.
Chris Detzel (45:48):
Yeah. And any capabilities for survivorship not supportive in the UI?
Saurabh Agarwal (45:55):
So right now, currently we are working on few other things. And that is my finance side, which I'll cover. Right now, so for example on the rulesets, sorry, on the sources UI. In the old UI, you can select different rulesets that the user has access to. And earlier on, during this webinar I did mention that, you'll have the same capability with the new UI. But that is not built yet. So apart from this, we have covered everything that is required to configure survivorship strategies.
Chris Detzel (46:32):
Okay. And just two more questions. One is how can API call get a non-default survivorship data?
Saurabh Agarwal (46:39):
So we are adding a new parameter. It was released yesterday, I think. The parameter is called explicit survivorship group. So if you provide that parameter, and you provide the URI of the survivorship group that you only use to fetch the data, that is what the API will return. It's [inaudible 00:46:57] in the API.
Chris Detzel (46:59):
And during extracting, is it possible to get a business specific survivorship group?
Saurabh Agarwal (47:09):
During bulk extract? I need to check that. I'm not sure right now.
Chris Detzel (47:12):
Okay.
Saurabh Agarwal (47:12):
Do you have clue, Dmitry Blinov?
Jacque (47:18):
I can answer this one. It is actually possible. So if you configure rulesets within your tenant, you can apply metadata security towards those rulesets. And if the user or the service account only has some ruleset defined for it, then it'll export data using that ruleset. So you can have specific service accounts, or users that can extract OV data per their ruleset. Yeah.
Chris Detzel (47:57):
Jacque, wow, it's great to hear your voice. Thank you. All right, that's all the questions. So if you want to go ahead and show your last slides on what's coming.
Saurabh Agarwal (48:09):
Yeah.
Chris Detzel (48:14):
I do have another idea for later, but just keep going.
Saurabh Agarwal (48:18):
So I think that is the only thing that we are currently working on. Selection of rulesets on the sources UI. The problem with the old UI was, if you selected a ruleset, that ruleset became the default ruleset for the entity. And again then the problems with some unintentional update that you did not really want to make that default ruleset. But if you only wanted to view the operational values corresponding to the new ruleset. So in this scenario, like the old UI, you always ran into a risk that somebody might unintentionally update the default ruleset, causing a lot of data issues in the downstream system. So now what we're doing is, we will have that ruleset selection feature on the UI. But the ruleset that you select, that does not become the default ruleset.
(49:10):
It'll only update the operational values for the profile that you're currently viewing. So it could be a read only change that will help you to view what the OV values for the selected ruleset will be. But there'll be no changes in the configuration as such. And secondly, with this new Data Modeler enhancement that we've done, although you can configure your survivorship strategies in the Data Modeler, but right now you do not have the ability to see what the profiles would look like when you implement this survivorship strategy. So right now you do not have the ability to test out the strategies before saving it. So this is something that you want to build in near future. You want to give you the ability to also test these survivorship strategies before you save configuration.
Chris Detzel (50:01):
Great. And just a quick thought is, ideas for identifying what entities, so attributes, are changed by a survivorship rule change? I think that's an idea.
Saurabh Agarwal (50:14):
Okay.
Mark (50:16):
Well actually, what I was saying to Chris is that, earlier we had talked about some people were doing that. So there was an enhancement discussed. I'm just wondering how people are doing that today. If they're doing a survivorship rule and it's not resulting an events going out, how would they figure out what's impacted? It sounds like some people are maybe doing that.
Chris Detzel (50:42):
Anybody who wants to speak up? Thoughts? It's a good question.
Speaker 8 (50:49):
We were under the impression that if a data change happens, either via a data update from source, or due to server shift, we expect that event to get triggered so that we can publish that to downstreams. If that event is not triggering then we don't have an option to publish this changed data set to our downstream systems.
Saurabh Agarwal (51:14):
Sorry, I'm not entirely sure. But Dmitry Blinov, do you know if the user runs the re-index job, will it trigger an event? I think it will not, but I'm just-
Speaker 8 (51:26):
Re-index... Yeah, no-
Dmitry Blinov (51:26):
Index will do.
Speaker 5 (51:26):
Re-index will do. Yeah, you have two mods. You can switch it off and re-index without generating the events, or you can re-index with generating the events.
Dmitry Blinov (51:36):
[inaudible 00:51:35] was different. The question was, if I change the operational value survivorship group, and operational value changed as a result of that. In this case in this case I think we discussed it in the beginning of the call. So basically an event will not be generated. And we need to avoid this idea. It's a good idea. We need to work on it. And I understand how this is a pain point today. And as we also discussed in the beginning of the call, we need to make this configurable as well. So in some cases you want to generate these events. In other cases you don't want to generate this [inaudible 00:52:14] event in the event of recalculation of the operational value after the survivorship group change.
Speaker 5 (52:21):
Right. Yeah you can make it as [inaudible 00:52:24]. That will help. Yeah we can-
Mark (52:24):
I think somebody on the call said that it was difficult to do, but they had a way of doing it. I was just wondering in the interim, if somebody has done this, what are they doing with the current capabilities? Because I don't know who thought as far as how to do that. Somebody mentioned it was difficult to do, but it was doable. Somebody else on the call, one of the other listeners, not a presenter. Or maybe-
Chris Detzel (52:52):
I'm just looking back to see if they said that in the chat. Because a lot of people dropped, so I don't know if they're still there.
Dmitry Blinov (53:00):
Maybe search with delta will return you the changed value. So it's not a QS event, I think. So yeah, we can get back to this conversation specifically, because we need to.
Chris Detzel (53:15):
Yeah. Two more questions. And Mark, maybe post that on the community, and as we do these ask me anything, maybe others will speak up to that as well. So do we have an option to build our own survivorship strategy other than the ones listed in the UI?
Dmitry Blinov (53:36):
No, you can't.
Chris Detzel (53:37):
No?
Dmitry Blinov (53:38):
You can only use default strategies.
Chris Detzel (53:39):
A straight no. Okay. One question. Will the UI allow us to understand which survivorship rule is being used when a user is viewing the data? So I'm thinking about the confusion when two people are viewing the UI, one person using the strategy X while second person using strategy Y. So a minor issue, but it still could be confusing. So I think there's an approach to published during a re-index, published event on changes, question mark. Does that make sense?
Saurabh Agarwal (54:13):
For the first question, I think it was whether the UI will let you see which survivorship group or strategy is being used. Yes, that's true. Right now also in the new UI, you can currently see the strategy [inaudible 00:54:25] is being used for each attribute. After we make the ruleset updates, which is planned for release. As part of this current release, you'll also be able to see the current ruleset, or the current survivorship group which you are viewing. So yes, that would be possible.
Chris Detzel (54:47):
Okay. And then the second question is, I think there's an approach to publish during a re-index. Is it publish event on change? Hey Robert. Speak up a little bit. Might be better on that question. You can unmute.
Robert (55:08):
Oh yeah, sorry, I didn't know that I could do that otherwise I would've spoken up. Yeah, so we've used the technique during a survivorship change. I know we've done it, where then you go and you ask for a re-index, and there's an attribute that allows you to essentially say, hey, if you see a change during this re-index, publish an event. We've used that when we've changed survivorship in order to sync to downstream. I'm pretty sure we've used it. I just don't have the exact details on the attributes, or the exact re-index that we used. But I'm fairly certain that we've updated the survivorship, and told Reltio, hey, go look at all the data and publish events. It obviously created a huge cascade, which we had a throttle and stuff. But I'm pretty sure someone was alluding that there is a way people have done this in the past. I'm pretty sure we did that. So anyway, sorry. Messaging is confusing, but-
Dmitry Blinov (56:14):
If you use the re-indexing task specifically, yes, in this case it'll generate all the events. And you're right, it'll be cascaded throughout the [inaudible 00:56:24].
Robert (56:24):
Yep, yep. So Dmitry Blinov, again, someone asked, is there a technique that's been used in the past during survivorship changes? I'm pretty sure that's it. There could be more. I just thought I'd... But anyway.
Mark (56:37):
Yeah, thank you. Okay. Just with that though, would it generate only events for those things that have actually changed? Or everything that's getting touched because of the re-indexing?
Robert (56:46):
No, it should just be on change.
Mark (56:49):
That's what I would hope for. The Reltio guys, is that your understanding?
Dmitry Blinov (56:54):
Yeah, it depends on the way you re-index, I think. Because you can just re-index the actual [inaudible 00:57:02] search index. So you can do the index with full update. That will touch the entire tenant. From the top of my head, I didn't do this for a long time, so I can do a quick test, and get back to you. I just don't have it in front of me [inaudible 00:57:18].
Mark (57:17):
That was great. Yep.
Robert (57:20):
And Mark, we've been able to deal with these throttling, or the cascading effect of change. It's just a challenge. Any kind of streaming platform has similar challenges. It's not a Reltio problem. So, anyway.
Mark (57:35):
Perfect. Thank you.
Chris Detzel (57:38):
That's good. Thank you. That was great. Well that is time for today. Really some great discussion today. And I love the ideas of doing more of these kind of ask the expert anything, specifically around a topic. I think I'm going to do some of that. It's a great idea, Mark. Thank you. But hopefully you liked it. Saurabh, thank you so much. And Dmitry Blinov, thank you for jumping in. And everyone else. Wow, this was really great. But take the survey at the end. Whenever you leave, a survey pops up. Ideas are always welcome. And trust me, I take those and start pushing out some of these shows. So thank you everyone for coming. Come to the community.reltio.com. Ask questions there. You'll get answers pretty quickly. Other than that, have a great holiday and the rest of the year. We'll be back in January. So thank you everyone.
Saurabh Agarwal (58:36):
Thank you.
Speaker 9 (58:36):
Thank you.
Robert (58:37):
Hey, thanks Chris. Appreciate it.
Chris Detzel (58:39):
Yep, of course. All right. I'm going to end it.
Mark (58:48):
All right, have a good one.
Chris Detzel (58:51):
You too.