In this Community Show, Prasad Satam, Senior Product Manager, will go into a deep dive into the Reltio Cleansers. Learn how to use Reltio Cleansers externally and without persisting the data. He will answers questions such as: "How does it work if someone has multiple addresses? Can we call sequences of cleansers on a single attribute?Transcript:
Chris Detzel (00:00:31):
Okay. And today's topic is a deep dive into Reltio's cleansers, because community wanted a deeper dive. We did have an overview of the cleanser a month or so ago, or a little less, and you guys actually wanted this. So we're happy that Prasad has agreed to do this, so thank you, Prasad. Some upcoming community shows, and I am looking at trying to book one or two more for May, but in the meantime we have today's community show on a deep dive into Reltio cleansers, and Empire Life is going to do a show specifically around business benefits of Reltio MDM platform.
Chris Detzel (00:01:12):
If you haven't signed up to that, I will be sending out invites probably in a week or two to that, or you can go sign up yourself directly and get it directly on your calendar. Really excited about that one as well. I did not put the rules up here just mainly because I forgot, but the rules are, please put your questions in chat while you stay on mute, and I'll make sure to ask Prasad the questions from you. You can take yourself off mute, which is fine with us, to ask your question as well. So I'm going to stop sharing, Prasad, and I'm going to give it over to you. We are recording this by the way, and it will be posted to the community in the next, hopefully by Friday, but for sure by early next week. So Prasad, all you.
Prasad Satam (00:02:03):
Thank you, Chris. I see a lot of people joined this call, so it's good, we have a good audience. We can deep dive into the cleansers. And to give some background why we are having this meeting or why we are having this community webinar, is we had a great feedback for the first session about cleansers that was held by Ashley and Kim a couple of weeks ago. And then we got a lot of questions which were more technical, and a deep dive into the cleanser is not easily found on the documentation, or some of the challenges that you have faced during the configuration or during the product use. So as a lead product manager for the cleansers, thought it would be a good idea to have a webinar. We can go through the question and answers in this session. And then I have Thiru, who is the lead engineer for the cleansers, to give us some demo.
Prasad Satam (00:02:56):
The way I have this session schedule is two interesting topics that I have got questions for the last year, I wanted to deep dive into that. And then I have recorded some of the questions from the previous sessions, so I'm going to answer them. And then I'll open up the floor for more questions. And in situations where we need a demo, we have Thiru who can give us some real life example, demo of the situations. Okay. So without further ado, let's get into it. And I think the first topic I want to address, is can we use Reltio cleansers beyond MDM? Can we start an application or a process where we can just trigger Reltio cleansers and cleanse an application which is outside of Reltio domain? And the answer to the question is, yes. We can use the cleansers without persisting the data in the Reltio cloud. We have definitely provided this information on our dock portal. So if you go in the dock portal, there is a session called 'On the Fly Entity Cleanse API', and I will give a demo about how this API works.
Prasad Satam (00:04:09):
But the summary is, this API will trigger the cleansers, which are configured in your tenant without persisting the data. The cleanser configuration is an interesting piece here because every entity type, for example, if you want to cleanse an address, then you will have to trigger the location entity type, if you have an employee or a healthcare professional or a product. So depending on what entity you want to cleanse, it'll trigger the corresponding cleansers, which are configured in your tenant. That's one piece. The second piece is there will be no Reltio cleanser crosswalk associated with the output. So if you provide an input with any source, it doesn't matter, we will do the cleansing, no persistence of the data, and we'll provide an output without a cleanser crosswalk. And the third thing which we have to all understand is, even though you are using Reltio for cleansing, the API calls will get added up towards your entitlement.
Prasad Satam (00:05:14):
So for example, even though we are not persisting the data and you have 1000 records for which you have cleansed, it'll add up to your API entitlements. So this is the summary of this. And the interesting part is you can cleanse one record at a time, or you can do the whole batch. So we have another batch API here. And the difference between the two will be the cleanse versus batch cleanse. This is one of the interesting things which a lot of customers ask. And then you can have address cleansers, phone, email, string, it doesn't matter. All the cleansers can be used outside the Reltio application or Reltio cloud. We have a short demo about how we can get this done, but before we get into the demo, I'm going to take a pause and get questions here. So Chris, if you don't mind, we can open the floor for some questions. This is an interesting topic here, which most customers are interested in.
Chris Detzel (00:06:09):
Great. So does anybody have any specific questions? Because I don't see any yet in the chat. Usually it takes a minute or two, once you go through the demo and things like that, it might...
Prasad Satam (00:06:20):
Okay. All right. So let's get to the demo about how we can use this API. So Thiru, if you want to walk us through about the real life example. So the demo is organized in a way that we will first show how to use address cleansers outside the Reltio, and then phone, email, and string outside the Reltio, so there are two pieces to the demo that we have organized here.
Thanks, Prasad. This is the API. I have set up a tenant in our environment at ACP. And here I have configured a set of cleansers on this, like phone, image, string cleansers. So this is a batch cleanse I'm going demonstrate here. So here, I'm posting an ACP profile with the name, personal details, along with phone and email. This cleanser takes multiple profiles and applies a given cleanser on the given profiles. So here, though the tenant entity has multiples cleansers, at the time we can use only one cleanser name here. So whatever name we mentioned here, only that cleanser will be applied on top of all these profiles.
So here, though I have configured email and phone, but I am passing only the one name here, and it's going to apply phone cleanser on top of this profile. I'm sending here. So here I applied phone cleanser, now I have email and phone, but it's going to cleanse only the phone. So, email will have the version balance, whatever is sent here, and the phone will have all the cleanser values like [inaudible 00:08:18], area code, [inaudible 00:08:20]. You want see the email also, then we can paste the email in here, and then it's going to cleanse only email [inaudible 00:08:31]. And so here in this case, here is the email and phone will return address. So at the time we can apply only [inaudible 00:08:44] under all the profiles.
Prasad Satam (00:08:44):
So Thiru, if I may, from the behalf of the customers here, how can we do all the cleansers in one? Can we say 'type equals' to all?
No. As of now, the platform supports only one cleanse at a time.
Prasad Satam (00:08:54):
Prasad Satam (00:08:58):
Okay. So again, for the audience here, the goal is we want the customers to go beyond Reltio system, Reltio cloud, and use the data quality or improving or enriching your data beyond Reltio application.
Chris Detzel (00:09:15):
Okay. Thank you.
Prasad Satam (00:09:20):
Thiru, do you also have an example where just the address is cleansed?
Prasad Satam (00:09:41):
And for people who have joined late, whatever we are showing, even though we are using the Reltio API, the data is not getting stored. All these API calls will be added up to your entitlements, to your daily API usage or your monthly API usage.
I'm pulling the data for address, so we can [inaudible 00:10:25].
Prasad Satam (00:10:24):
I see a question from Sandra. It's a very good question. Does it matter if it's a production tenant or a test tenant? No, it doesn't matter if it's a production tenant or a test tenant, as long as you have the right configuration in the tenant. So the tenant URL is important because the entitlements will be associated with your account and which involves test [inaudible 00:10:53] fraud. So it doesn't really matter which tenant you're in working as long as it adds up to the API usage. That was a good question, Sandra. Thank you.
Chris Detzel (00:11:03):
Here I have been posting location. For location entity, we have two cleansers from here, one is other, and default. The other is the cleanse confirmation, which takes the individual attributes and cleanses them, so that's why here in passing type as other for the other cleanser, [inaudible 00:11:40]. Here, we can see the cleanser I used.
Prasad Satam (00:11:46):
Yeah. And this is a way where you can also test your tenant's configuration without loading the data. So for example, this is the initial phases of your implementation. You have configured the tenant with the certain cleansers and you want to give it a try. You can use this API for a quick check on your configuration as well. So there are multiple use cases, not just for beyond the Reltio application, but also you can have a quick check of your tenant configuration.
Oh. Just one question.
Prasad Satam (00:12:21):
You have a question? Go ahead.
Yes. Okay, yeah. The values are the responses we are getting, right? So it is in the cleansed values, not the source value. I remember probably when we are using locate, so the both values would be there. The source will be kept and as less the cleansed value would be inward stored as well in the entities. Sorry. Right?
Prasad Satam (00:12:47):
I know that we are going to store within the relative entity, but the response would be there cleansed alone, not the source one, or would two values be there, one is sourced, one is cleansed one?
Prasad Satam (00:13:04):
So in this response that Thiru is showing, again, to repeat the question for everybody else who doesn't know what the question is about. The question is in the response, do we get the source value and the cleansed value both or just the cleansed value? The answer is yes, we get both the source value and the cleansed value, it is separated in two different blocks in the JSON [inaudible 00:13:27]. As Thiru is scrolling up on the top, if you see the incoming value, that's the incoming value. And if you scroll at the bottom of the page, you will see the different cleansed value. It imitates what we do in the system, Pascal, where we store data inside the system, both the source and the cleansed and in the API also we provide both. Does that answer your question?
Sure. Thanks. Thanks. And I have another question. I see that location as meta-distance separated entity. Okay? How does actually it works if we are using as a reference nested attributes within the probably, and let's take further licenses, whether at CCP or ETSU, probably you were familiar with this, how would that can be glanced also?
Prasad Satam (00:14:20):
Great question. The answer is you, all you have to do is change location to HCP. In this case, on the line, number three, in the input, if you change from location to HCP, everything else will work the same because the address cleanser, the type equal to other on the top will be applied to even a cleanse up inside a CP attribute. So it doesn't matter what type it is. It'll work both ways.
Okay. And, and I believe that the A's name would, would be similar to what has been configured within the cleanser. Let's say address line one. It shouldn't be in like ADDR line one, I guess it should be in always address line one state province, city something. I agree.
Prasad Satam (00:15:07):
Absolutely. Yes. So your address attributes will depend on your data models.
Good. Thank you.
So a few other questions, sorry.
I mean just, and then probably an extended one also. So I wanted to understand what you'll be using for this one. Whether we will be knowing what cleanser has been used by relative for cleansing these addresses, or I know that no Canada address has been cleansed. What if I wanted to cleanse for an Asian countries, how effective it would be this cleanser?
Prasad Satam (00:15:49):
Sorry, I didn't understand the question. Can you...
Okay. So, what type of cleansers are using, are locate, are Google or any other cleanser that's first question. The question is how effective this cleansing would be for the countries like India or Malaysias sorry not Malaysia. Probably Pakistan. Okay. Because the US or Canada, I believe the US data would be in what it'll be an open source. Right. Everybody can get it and look locate is kind of, I think almost more than 90 percentage, it works. How would it work for the rest of the other countries. Any countries probably within the South Asian nations or Gulf ones also how accurate would it be, would say recommended or not.
Prasad Satam (00:16:35):
No, it's a good question. So let me answer the first part of your question. Like what cleansers are we using behind the scenes? So today we don't have integration with Google cleansers or anybody else. We only integrate with our partner. The vendor here is located. So no matter what you're doing with the location, the cleanser is always coming from locate. Okay. The second part of your question is how good is the quality of the data? Well, for the four standardized countries, US, Canada, Australia, and England, the location verification is pretty high. Like almost like 99% delivery, but the nonstandard countries like India, Brazil, or anything in the Southeast Asia, we have put it on our website. Like what is the verification level? Whether it is at the lower level verification, device level, street, city. So every country has a accuracy. And then I can, I can share with you on a separate chat here. Like, what is the accuracy for each country? Yeah. I'll once we go into the demo, I'll, paste the link to know the accuracy of each address in each country
Would also be helpful if you had APIs as well. But for both for phone, email, and this one, maybe wouldn't try it in our also.
Prasad Satam (00:17:56):
Yeah. So I see a question from Divya, where would you be able to share the rest APIs? Yes. It's available on the doc portal. I will paste the link on the chat here. So if you go to this the URL, you will see the APIs for both the single instance and the batch cleanse, both.
Chris Detzel (00:18:17):
Great. Let's get to a few more questions here. I'm not sure if you answered this one or not, but it would be nice to see values before and after cleanse on the deck on the postman. Can you highlight the cleanse values in postman? Was that done already?
Prasad Satam (00:18:33):
Yes, Thiru will just walk us through where the cleanse value will be.
Chris Detzel (00:18:38):
Great. And then is there a front end option using Reltio front end to do a batch cleanse?
Prasad Satam (00:18:45):
No, we don't have a front end option right now to do a batch cleanse. The batch cleanse works better with the API today. So the answer to your question is, no, we don't have a front end option to do a batch cleanse.
Chris Detzel (00:18:59):
And then, Gene says, I believe that the survivorship groups affect how the values come to you in the response. Is that, is that correct?
Prasad Satam (00:19:10):
No, the survivorship groups will not affect anything here because there are no multiple crosswalks. We are just sending one address at a time, one profile at a time, or even within the profile. Even if you have multiple addresses, we will execute each address at a time and give you the response. Because what we are trying to do is, it's kind of a black box. You send the data, we enrich the data and provide it. We, we don't perform any reality operations like survivorship, match and merge any of those, just the pure cleansing.
Chris Detzel (00:19:43):
Okay. And then last question for now is there a possibility for on the fly match to be triggered with the existing, similar HCP? So when you send this HCP payload, which you don't plan to persist?
Prasad Satam (00:19:59):
Do you mean on the fly cleanse Divya or do you mean on the fly match?
Hi, Prasad. So basically I'm talking about do you risk... So for example, this use case that you don't want to persist this particular customer, do you kind of fall into the risk of triggering any of the match rules configured when you are really sending all the attribution for this payload?
Prasad Satam (00:20:24):
For example, an existing...
Prasad Satam (00:20:27):
No, I got your questions. As I said, there is no matching involved. There is no survivorship involved. So you do not risk anything. When you trigger this API. This API will only trigger the cleanse section of the data flow. For example, when you load the data in it, first goes through cleanse match and mode survivorship, and it performs multiple operations before a golden record is formed. So we will skip all of those good parts and only focus on cleanse and return the address without going through the rest of the workflow.
So this API basically bypasses all the other configurations done in [inaudible 00:21:05] model, essentially.
Prasad Satam (00:21:05):
Yes. Yes. It focuses only on the plans section of director.
Chris Detzel (00:21:12):
So interesting. And so many people still have questions. One more for you. Is there a way to retrieve a timestamp when the cleanser was invoked? So this would help in case of survivorship. So let's say two crosswalks merged and we are using Reltio cleanser as a source for OV, and if we use LUD, then we get both the cleansed addresses.
Prasad Satam (00:21:37):
I don't think that we provide a time stamp. Thiru, can you go in the top of the response and see if there's a time stamp associated with this? Yes, there is. I think we provide a timestamp when the cleanser was involved in this case. Does that answer your question, Harsha? I don't want to go into the survivorship and everything else because we don't use any of that, but we provided timestamp for this.
And can use that in the regular configuration.
Prasad Satam (00:22:08):
Sorry. I have some trouble hearing, can you repeat?
Can we use that in the Reltio configuration, the time stamp or...?
Prasad Satam (00:22:16):
No, you cannot use this data because this is a single instance of cleansing outside the Reltio, we don't, we cannot use this data back into the... Unless you really want to use it as a created data again, but we recommend to not use it.
Chris Detzel (00:22:34):
Great. That's all the questions for now. A lot of them.
Prasad Satam (00:22:37):
Yeah. That's good. Yeah. Good questions. Keep them coming.
Prasad Satam (00:22:43):
All right, Theo, can I go back to my deck? And then I wanted to address the second kind of questions which I've been getting from the customers similar to what we want cleansers for the non MDM application. The second question, which I was getting is how can we verify our cleanse configuration through a UI interface, right? Suppose this is the initial part of your implementation. You want to see what locate provides in output parameters, and you want to perform the input and the output mapping. Can we do it without loading the data? So the way it works today is if you have seen the Reltio cleanser and then inside the console, we have an application called data modeler. And suppose we are working with a entity type of contact and we have performed some kind of a configurations behind the scene.
Prasad Satam (00:23:41):
So in this case, we have configured two different address cleansers, one in which it takes all the address parameters in one line, which is the default. And the other one is where there is address line one, two, city, state, and country. Now I want to test my configuration. I want to see how the data is provided to locate and what is the output that we are getting. All the fields, not just what I have configured. I want to see everything. And then based on the data that I'm getting, I will decide which attributes I have to consume. So if you, there is a tool that we have released couple of releases ago, probably six months ago, but I have not seen enough adoption and most people have not used it. Maybe people are not aware of it. So I wanted to use this platform to give a quick demo. If you enter...
Prasad Satam (00:24:33):
Let's put Reltio address here and all the input parameters are based on how you have configured the cleansers. So if I enter this data and perform a run test, what you will see on the right hand side is the full list of parameters that locate provides not necessarily what you have configured. So based on this information... So for example, I see that there are multiple ISO codes for country. Maybe I want to use the three characters instead of two characters. Then I can perform the output mapping saying that, Hey, use this characters for my attribution. Similarly, you can see latitude longitude. All this output parameters are coming from the locate cleansers directly. So you can take a look at this, understand which ones you have to use. Like what happens sometimes is there will be a field called building. Sometimes the data will be a part of address line one or a part of the building. So based on this, you can decide which output you have to consume. I see a question.
Prasad Satam (00:25:43):
No, you don't have to key in every field. Like for example, I can decide to skip zip code and then perform the run test. And you can see that it works even without the zip code. This is another way of testing one... Like, for example, if you know that there are a bunch of addresses in a particular country, going back to one of the questions that was asked is, what is the quality of address in a particular country? You can come here, enter that data, and you can see here immediately, what kind of a verification status or how good is the quality of that address, even without performing any data load.
Prasad Satam (00:26:19):
So you can skip as many fields as you want. Like I can remove all these fields and see what is the quality of address which comes out from here. So if I keep only the address line one and the city and do a run test, you can see that it went down from V 55, which kind of, when we say verification level of V five, it's almost like a door to door delivery. Whereas V4 is like a building. So I removed the suite information. So now it provides me the address quality of up to the building level. Where is this located? This is located in Reltio console. There is an application called data modeler. And inside the data model, you have to select which entity type you want to work with. In this case, contact on the top, there is a section for cleanse and here you can see that there is a icon here to test your address.
Chris Detzel (00:27:26):
Thanks, Prasad for getting to the questions faster than I could even get to. So, It's awesome.
Prasad Satam (00:27:33):
Yeah. Yeah. But this is a really powerful tool. I encourage you guys to use it because this will kind of reduce the errors during the implementation cycle, because now you're aware of the whole parameters which are coming out of cleanse. You can decide which one to use in your implementation.
Prasad Satam (00:27:53):
And another use case that I can think of for using this tool is in the middle of your implementation you see that the address quality for a particular pattern is bad. So you can come here and check if that really applies to the whole set of data, maybe you can choose certain addresses for which that error exists and which not. You can use this to create a Zendesk ticket with reality of support, and you can provide a screenshot that this is the address that we input. This is the output that we are getting, and we believe that there is something wrong with the address itself. So you can use it for troubleshooting even after the implementation done. I see another question. Okay. All right. Beyond these two topics. What I want to do is go back to this session, which was held by Ashley and Kim and address some of the questions from there. And then let's see if one of these questions addresses some of the questions that you came up for this session, and if not, we can open the floor for the rest.
Prasad Satam (00:28:55):
One of the questions that Pascal you asked in this session was does the Reltio persist both the incoming address and the cleanse address in the system? The answer is yes. The way the data is managed in the primary store is we associate the crosswalk, or the source information. So your incoming address is stored with whatever crosswalk the data came with. And after the address has gone through the cleanser, we will associate the Reltio cleanser crosswalk with the data. So at any given point of time, if you peek into the system, we will find both the data incoming address and the cleanser address just with different crosswalk associated with each other.
Prasad Satam (00:29:38):
And then the second, the associated question to this was how will this work if the profile has multiple addresses? When we say a profile has multiple addresses, there are two different scenarios. There is one attribute in which multiple sources contributed to the single location profile. So in this case, it will perform all the matching and merging, the operational value based on the survivorship strategy that you have configured in the tenant to identify a golden record or a golden value for that location. Once the operational value has been identified, that will be the only piece which will go through the cleansing and then the rest will stay as it is. And then we will have the cleansed address on the OV selected location or the incoming address for that particular attribute. The scenario two is when the question was, what if the profile has multiple addresses in this case? Suppose the address is a nested attribute and you have multiple addresses home, office, work, et cetera, right? In that case, we will perform the cleansers on each of this address as a separate cleansing process. So these are the two different pieces in which what happens if the profile has multiple addresses.
Prasad Satam (00:30:55):
I see some questions here. Okay. That is just from...
Chris Detzel (00:31:00):
No, not yet. I'm following. Don't worry.
Prasad Satam (00:31:03):
All right. And then the connected to the cleanser's address cleanser specifically, the question was, can we attach an external cleanser, for example, a Google or Melissa data, there are a whole bunch of vendors out there which can provide location cleansing. The answer is no, because if you perform an external API call, it will just slow down the data load process. We don't know the SLAs or the performance of any of these applications out there. And that's why we avoid making an external API call. So rather what we would recommend is if you really want to use an external address cleanser, then we can turn off the locate address cleansers in the Reltio tenant, you can perform the cleansing on the address and then load the data that will be, that will be an ideal scenario, just to make sure that the data load happens with the given SLA.
Chris Detzel (00:31:58):
And Prasad a couple questions did come up, can we exclude some addresses from auto cleansing if they're coming cleansed already by the other data enrichment provider?
Prasad Satam (00:32:10):
Yes, what we can do is based on the country, for example, we won't be able to avoid based on certain attribute values, but what we can do is when we are performing the configuration for a tenant, we can choose which countries we will cleanse and which countries we will not. So for example, if you believe that a certain external address cleanser works well with Southeastern countries, then you can perform that cleansing outside when the data is coming inside Reltio, we can turn off the address cleansers for that country. In that case, it'll avoid performing locate cleansers on those addresses.
Chris Detzel (00:32:51):
Does the OV example also work if not location, but address included in the profile?
Prasad Satam (00:32:59):
I didn't really understand that question. Thiru, can you address that question?
Yeah, I can, this is Mark. I could try to clarify. So a couple things here, the example you have for the address is I guess, as a separate location profile, a reference to it, as opposed to, if I'm understanding correctly, as opposed to just a nested attribute defined within a profile, within the organization profile. So, with that example, when you're talking about the survivorship first and then the cleansing, does that behave the same way in that case? Or does it behave differently? If the nested attribute is literally just a specific say in the organization profile, does that help?
Prasad Satam (00:33:48):
Yeah. That helps. The performance of the behavior will be exactly the same, even for the nested address. Like, for example, if the home has multiple addresses, then depending on what your survivorship strategy is, then it'll perform the cleansing only on the OV for that particular attribute on that particular instance of a home address.
Okay. So I should, if that's the case, then I should only have one cleanse address for a nested item I'm assuming there was only one address type in there. I wouldn't have multiple cleanse values. I would just have one for the address?
Prasad Satam (00:34:24):
One for the address, unless you use aggregate as a survivorship. And then in that case, it'll perform cleanses for all.
Prasad Satam (00:34:35):
It depends on what your survivorship strategy is, right?
Right. That, that makes sense, because you say it's driven by the survivorship first. Okay. All right. Yeah. And the... Okay. And I don't want to take too much of the time, but we've been struggling with survivorship for the addresses when it's embedded in not as a reference attribute, which was how we were told to go originally. And now we're struggling with this a little bit and the sequencing just described is actually helpful to make sure, cause it's not how we thought it was working. So that could be helpful. Thank you.
Prasad Satam (00:35:19):
No worries. That's a good...
Chris Detzel (00:35:21):
That, that's a great question, Mark. So thanks for asking. If you have a nested attribute with aggregation so that the OV has mobile fax work, et cetera. And how does the cleanser as a source play a role in that configuration?
Prasad Satam (00:35:39):
So to repeat the question, it's a nested attribute with the survivorship as aggregate. And so let me take a look at this question, please. Yeah.
Chris Detzel (00:35:49):
Kurt, Kurt, you can speak up if you want.
Yeah. So Prasad, Hey, how you doing? So, we're got this configuration, we're trying to sort of figure out, the cleanser is a source, right. So it has a crosswalk and it looks like a source in many ways and perhaps like any other source. Right. So if you set up aggregation and for, let's say addresses or phones, right. So you have, you said, okay, I want my nested attribute to produce all the phone types in the OV, the mobile, the fax, the work, the home, they're all there. And so you're saying that the cleanser actually takes the OV as input. Right. So all four of those let's say from the OV are going to get fed to the cleanser, and then the cleanser is going to spit out all four cleansed phones?
Prasad Satam (00:36:46):
Yes. And then we will have the Reltio cleanser crosswalk associated with that attribute. Right.
Okay. So then my OV because it's an aggregate of all sources would have the phone cleanser output in it, as well as...
Prasad Satam (00:37:10):
Your source values,
The source values.
Prasad Satam (00:37:12):
So how do you arrange for just the cleansed version of all those things? You know, because basically when you set up the aggregate, what you're saying is I want all my sources represented, but of course I want the cleansed version of all my sources represented. I don't want the raw values of all my sources. So the cleanser being a source, I'm trying to understand how you sort of tell Reltio, you know, you sort of hold the cleanser separate and distinct in some sense by saying, look, these are the sources that I'm interested in and these are the types I'm interested in. But of course I want the cleansed version of all these things.
Prasad Satam (00:37:57):
Maybe I'm not getting it. So Kurt, in that case, would you not have separate versions of phone number? Like for example, if your phone number is a nested attribute and in that case, you can have the type from work to home to office and then you can just separate each of these phone numbers based on the type. I'm just thinking out loud where instead of using aggregate, maybe we can separate it out by type, right?
Yeah. We'll probably have to take it offline because we're also trying to use filter to sort of get those types separated from each other. But yeah, but I want to take,
Prasad Satam (00:38:37):
We can this offline. No, but it's a good...
Actually do me a favor when you take it offline, if you could somehow include a couple other people Mark [inaudible 00:38:46] and Harsha we are kind of dealing with very similar problems right now. So it'd be very helpful for us to hear and maybe contribute. Okay.
And, and what we'll do is, cause we're actually going to work through this offline with professional services as well, Prasad. So I think we come up with a well written answer on this topic. Chris, I'll go back and share it with the community.
Chris Detzel (00:39:10):
Prasad Satam (00:39:11):
Perfect. Thank, thank you, Kurt and Mark, if we have an offline session now I definitely invite you for this.
Okay. Yeah. Thank you. And, and we're actually, we're working with Dan Gage right now and I don't know if maybe he can involve you in our conversations with him. It could be helpful. Okay. Yeah. So I will maybe mention that to him also reach out to him somehow. Okay. Thank you.
Prasad Satam (00:39:35):
Yes. Great, great questions. Thank you, Kurt and Mark. And I definitely help you with this
Chris Detzel (00:39:40):
And Prasad we have a few more, so they keep coming in. What is the best practice when you chain different cleansing settings for addresses?
Prasad Satam (00:39:52):
So can you elaborate on that question? Like when you say different settings.
Chris Detzel (00:40:01):
Gene, if you want to...
Hi. Yeah. So what I mean by that is let's say if I want have two different settings, one will do a cast cleanse. And if it fails, I want, I want it to do a geo code instead. Do you get it what I'm saying?
Prasad Satam (00:40:30):
Yeah. So there are two parts of cleansing. One is just the verify without USPS validation and then the USPS validation, right? Yeah. So you want to choose between one or the other, but what is the criteria for choosing, if I may ask that.
Oh, there's not necessarily a criteria. It's just that the end game here is that I just need to have geo code values at the end for both type of addresses. So if the cast fails, sometimes if I only have one setting, somehow I've noticed that it doesn't give me a geo code inside the addresses. Now, I tried to apply the second setting where if it fails, it will just go through the geocoding check instead. And give me a geo code for the address, which I can use then for my other processes. But somehow I see there's still a lag or disparity when, when it runs. So I was wondering if there's a best practice to apply when you do chain plus chain cleansing.
Prasad Satam (00:41:53):
Thiru, Keep me honest on this answer. But Gene, we have an option where we can provide both the verify, which is what you're referring to as geocoding and cast and the way the system works is before we send the data to the USPS validation, we will enrich the data, try to remove out some of the noise words by using that geocoding before. And then, if we feel that the quality of the address is good enough, then we send it to the USPS so we can provide that kind of a chain cleanser. You don't have to know the exact details of the how it works in the physical configuration of the tenant. But if you really want to try it out, what I will ask you just create a support ticket and ask them, I want to attach the geocoding before the cast. And in that case, you get to see both the values in the output.
Okay. Yeah. Because the way that what you just mentioned is something I was looking into and want a solution to, so I'll do that. I'll open a ticket. Thanks.
Prasad Satam (00:42:57):
Yeah. And I want to see if we have that information on the dock portal, I think we have a document for it and then maybe Gene take a look at it. And if you feel comfortable about it, I think we can just try it on your test tenant.
Gotcha. Thank you.
Prasad Satam (00:43:21):
So in our doc portal, if you search for pre-processing location data, you can see that there is this configuration, which says pre and this happens behind the scenes. But what happens is when the address is brought into the system, it'll perform the geocoding. Before we feed it to the cast.
Speaker 10 (00:43:44):
...is like verify we apply first and we take the verify response and then send cast so that the input will be accurate.
Prasad Satam (00:43:52):
So in this case, if it's changed, if I put the V plus G first, what the results of that, essentially, if it fails in that step, would it carry it over to this next step?
Prasad Satam (00:44:09):
No, it doesn't. If the quality of the address is really bad, then we don't even need to feed to the cast engine.
Chris Detzel (00:44:17):
Right. Still more questions. This is great. This turned out to be a ask me anything for Prasad. No, I love it.
Prasad Satam (00:44:26):
Ask me anything.
Chris Detzel (00:44:30):
What'd you have for dinner? No, I'm just joking. What config is required to use the cleansed address as part of the match.
Prasad Satam (00:44:40):
Can we have some more information to that question? What is required to use the cleansed? Oh, so I think if I may repeat the question, basically the question is based on the cleanser quality, do you want to use it as a part of match or not? Is that the question?
Chris Detzel (00:45:05):
I think so.
Prasad Satam (00:45:08):
Senti, no, I think the question is from you. Can you, can you elaborate on that question a little more here?
Chris Detzel (00:45:15):
Speaker 10 (00:45:16):
Chris Detzel (00:45:17):
There you go.
Speaker 10 (00:45:19):
So actually, what I was looking for that, how to use the cleansed address or the cleansed information as a part of the match rule.
Chris Detzel (00:45:27):
Prasad Satam (00:45:29):
When you have the configuration matching, so when you have the configuration for this matching, you can use verification status as an attribute, and you can use the fact that only if the location is verified, use it for matching. So you can definitely use that as a part of your match rule. I mean, again, I'm thinking out loud, is this what you're looking for? Is that all the cleansed address will be part of matching, but if you want only the good addresses to be a part of matching, you can use the value of an attribute in the part of your match rule.
Speaker 10 (00:46:07):
Prasad Satam (00:46:08):
Does that answer your question Senti or is that what you were looking for?
Speaker 10 (00:46:12):
Yes, actually the plan is instead of using the raw source data, let us just cleanse that one and then put that data as a part of the match so that the match becomes higher.
Prasad Satam (00:46:25):
Yeah. I think we definitely do that. We perform the cleansing before we do the matching. So the cleansed address automatically goes into the match algorithm.
Chris Detzel (00:46:35):
Yeah. So quick question. Does locate support multi linguistic address cleansing?
Think Chris, I have a follow up question on that one.
Chris Detzel (00:46:46):
Sorry about that.
Sorry to intrude, yeah, no problem. Yeah. I think we can set the match rules maybe based on the attributes only if the verification status is in order to, or probably name the value. Okay. Verify. Okay. But what if it is in nested attributes? How can be that touchable if the address is, were stored in nested attributes? I understand part of this entity, but nested one. I couldn't think of it or couldn't visualize it.
Prasad Satam (00:47:17):
Yeah. I may not have the answer to that question, Jay, but it is a good question because today the difference between individual entity, these match rules versus match groups, I think, and I don't see we having a conditional, but again, maybe I'm wrong. I'll have to check with the match expert on our product team and then I'll have to come back with that answer.
Sure. No problem. I think I have similar, kind of a question to in the community as well, and probably I'm waiting for an answer from the admin.
Prasad Satam (00:47:53):
Okay. Chris can help you.
Probably let us know.
Yeah. Sorry. I think it's same way as in location entity, it works like the match rules can on the address list. The other attributes, let's say, if you have ACP has first name, last name and address it, if you define attributes under the name attributes and the address, whatever combination you use in your match rules based on that, it can match.
But that becomes matching against in another entity. But my question was like within the same entity, when you have the two nest attributes, then how come this works. I clearly understand how this can implement it when we have a separate entity, because two entities will having a two separate columns as verified entities, then we can actually implement a match rule. But within that nested one, I'm not sure how can it, so that's the, that's what...
Oh, we can't, I was still thinking that it can be possible that in match rules we can have some filters for operators. Is it in different answer? There it cannot be done?
Prasad Satam (00:49:16):
No, again, we have to check with the match expert Jay. Again, you have a question there on the community. I think Chris will follow up and then we'll have that answer for that question.
Sure. I probably will try to tag Chris also there in the post.
Chris Detzel (00:49:29):
Yeah. Tag me in that, then it comes up and I'll get one of those guys to answer. Does locate support, multi linguistic address cleansing, Prasad?
Prasad Satam (00:49:39):
ADE locate does support multi linguist addressing... Address cleansing, but not in the current product that we have. There is something called as auto complete. What means is that if somebody is entering their data in the UI and in that case, we support, but not in the current format. And we do have plans to integrate that with our UI. The current location engine is not, does not support multi linguistic at this time.
Sorry. I didn't quite get the autocomplete part of it. If you could just...
Prasad Satam (00:50:15):
So what happens is, so instead of an API, if somebody's entering the address through a UI and in that case, so we have a separate feature, which we call it as a capture plus. If we integrate that with your tenant and it has a separate pricing model, Divya, so when the customer, when we enable that, as in, based on the location and where you are, we will have that country's local as a part of the address input. And in that case, we will support multi linguistic. So maybe the way we can take that off, then I'll introduce you to that feature. And if that is the requirement, but the catch here is it's only applicable through UI. And that's why as a whole, a UI versus UI plus API, we don't support multi linguistic location cleansing today.
Chris Detzel (00:51:06):
So Ram has a question about testing features from UI and other cleanse functions like phone, email, as I cleanse function, locate for address, but cleanse function, phone cleanser, FN is not showing up to test from data model or UI to test like address a few.
Prasad Satam (00:51:30):
Yes, no, that's a good question. And thanks for going on the console and testing the feature right away. Yes. In that case, we are only supporting address tool now. And then the phone and the email is definitely in our roadmap to bring it up on the console.
Chris Detzel (00:51:48):
Great. I love that people want to go test it. They're keeping you honest, Prasad. So there it's just where an address was partially verified, but verified earlier. But now we see that it's verified after the recleanse, there's also other scenarios when previously cleanser was sending some value for address line one, and now is sending another value. So in response to that support ticket, we've got to node that there is a release and locate. Is there a way to determine all the address in a tenant for whom we would expect such a update since it's difficult to recleanse all the entities in a tenant and then check?
Prasad Satam (00:52:31):
The, the answer to that. So to rephrase the question, for the rest of the audience here to understand what they're asking is every time we perform a data pack upgrade for location data, or if there's an algorithm change, is there a way to understand the impact that this release is going to do on an existing data? Like, can we find out which locations to recleanse versus recleansing the entire tenant? The answer to that question is it depends. Like for example, if the ticket was filed, that all the addresses in Taiwan for example have changed. So recently Taiwan's postal code, went from three digit to five digit. So we performed a data upgrade for that country. In that case, it is easy for us to tell you that, okay, all the Taiwanese addresses will have to go through the cleaning again.
Prasad Satam (00:53:24):
Similarly, if there a pattern, for example, if the ticket was filed where the building name was getting attached in a part of address line one, or if the suite information was wrong. In that case, we have a pattern to identify which addresses need to be recleansed. But if it is an individual instance, it is very difficult to now connect that instance with the rest of the data and tell you that, okay, X percentage of the locations need to recleansed. So the answer to the impact analysis of a particular release, it depends on the ticket and depends on the situation. Does that answer your question that they, what you are looking for in terms of impact analysis?
Chris Detzel (00:54:12):
That's a good answer. I thought, so. Smith, do you have a question?
Yes. It, it does answer my question, but it would be really hard to actually them back considering that these informations are like going to [inaudible 00:54:35] replications, we would need a analyze impact before it goes there.
Prasad Satam (00:54:42):
So in that case, Smith, can you do the impact analysis on your test tenant? Like instead of production?
Replicating the data and test tenant and then doing the recleanse, yeah. I mean, we can definitely do that.
Prasad Satam (00:54:58):
I mean, generally the strategy is that your test tenant is kind of a middle copy of your production, and then we perform any of these operations on test before we take it to production. So that's why I was thinking you can use your test tenant as a staging environment for this.
Yeah, sure. We do that.
Chris Detzel (00:55:19):
Great. And does a change in survivorship role kick off a cleanser at time of retrieval?
Prasad Satam (00:55:27):
No. No, it doesn't because the survivorship change, like if you're doing it on the UI, it is performed on the get, like every time you perform a get it'll return you the survivor address. But in that instance, we do not trigger a separate cleanser.
Chris Detzel (00:55:46):
So then, what happens then? So if the cleanser goes off of the survivorship value, which I didn't know before the call. I didn't know, at least two days ago. So if the survivorship rule changes, the cleanse value now no longer aligns with the survivorship value. If it's not kicking off the cleanser again. Right? Or am I missing something?
Prasad Satam (00:56:11):
So if I have to like, take your question, Mark, for example, today, your survivorship was source based. And based on that source, we performed the cleansing and then everything is persisted in the system. So tomorrow you go and change the survivorship from say, source to aggregate or some of the strategy...
Or even change it to a different sequence and source even, I mean, as simple as you do.
Prasad Satam (00:56:36):
Yeah. Yeah, do a different sequence, right. And then in that case, your OV will change. And then you will have to... Once you do that, you will have to recleanse that address to get the new value, because we don't perform every time we change. Unless again, Thiru do you think that makes sense? Or we perform the cleansers every time we change the OV as well.
Yeah. Sorry. No, we don't perform recleanse every time. Whenever we see issue then we do a recleanse.
Prasad Satam (00:57:12):
Yeah. So Mark, the answer to your question is we will not perform. So you will have to do a force recleanse in that situation or... The reason why we say this is generally the survivorship strategies don't change as often, once it goes into the production. And if you believe that the survivorship strategy will change, then it'll trigger thee for that particular address.
Okay. I mean, it's understandable. I've just doing this every single time on a performance thing. It's just again, a new data point I'm just trying to reconcile on my understanding, so thank you.
Prasad Satam (00:57:51):
No, it's a good question, Mark. Something that we will have to document, like every time we change the survivorship strategy, then you will have to perform the recleanse for that particular entity type.
Chris Detzel (00:58:03):
Well, that precludes our time. We're, I think we're out of time. So this was, yeah...
Prasad Satam (00:58:11):
We, Yeah. I wanted to go through a few more questions, but we had some really good questions on the floor. And then Chris, maybe we can post this slide deck on the community and people can go through the questions. And the plans are that we will try to address most of these questions in our documentation as well. Maybe we'll add a separate FAQ section for cleansers, and then you can...
Chris Detzel (00:58:32):
Yeah, I like that idea. And you know, what I'll do is I'll post the presentation, just make sure you share that with me Prasad and then I'll post the video or this webinar on the community. If you just go to the community.Reltio.com, click on events and past events, then you'll see this particular community show in addition to all the other ones that we've had in the past.
Prasad Satam (00:58:59):
Chris Detzel (00:59:00):
But wow. You know, this was the most heavy asked questions on a topic. So I think that there was a lot of interest in this area. So certainly appreciate two of you getting on today and answering some of these questions. I mean, this is core to our product. So I think it's of interest to everyone. So hopefully you guys liked it. It was a little bit different than usual, but I liked the Q and A, and all those things. And, and Jay, you might post that one separately on the community and I can get Prasad or somebody else to answer that particular question that you have right now, but thank you, everyone, please post in the chat if you liked it and if it was helpful, so thank you everyone for coming. I'm going to stay on another minute or so just to kind of get the feedback and everything else, but wow.
Prasad Satam (00:59:57):
Yeah. Thank you. Thank you, everyone. Can you stay
For, could you stay on for two more also precise? I just have a burning question if that's okay.
Prasad Satam (01:00:05):
Yeah, yeah. Mark. I can, I can stay for a couple of more minutes.
Thank you. So we are struggling. And I mentioned, we're trying to deal with Dan and so forth. We're using a nested address item within the profile, so it's not a separate location. And I'm just trying to understand, is there a best practice survivorship rule for this? Like, literally I feel like we're trying to invent something that's should be so well documented. And our survivorship is based on recency, not aggregation. Okay. And we were struggling with four back rules cause we were basing on address line one and then sometimes that was sparse. So I'm just wondering, is there something that just has the best practice as far as, you know, what Reltio is recommending to do in this case?
Prasad Satam (01:00:57):
No, that's a good question. Thiru, do you have an answer for this or we will have to go and check with the match team on this.
Yeah, we have.
Prasad Satam (01:01:04):
What mark is asking is there a best practice for nested address for survivorship?
Yeah. With recency.
I was waiting for the same question.
Prasad Satam (01:01:14):
Okay. Thank you for, thank you for waiting on this call you. Yeah, mark. We, we might not have the answer for this question in this, in this session.
Oh, that's fine. I wasn't expecting it. I'm hoping though, if you can somehow get it to us work with Dan Gage. Cause I literally, I feel like I'm trying to invent a wheel and it's probably already invented.
Prasad Satam (01:01:34):
We need to document it. Think...
Chris Detzel (01:01:41):
I agree. And, and I wonder if Dmitri and or Joel Snipes could have an answer to that. Right. So it's another good one to post on the community because I can get them easily to answer that specifically in that forum. But if you need additional guidance and stuff like that, let's see if we can get Dan up to speed on some of that stuff too.
Okay. And Mark, in your nested attribute, this is Kurt. I'm wondering if you're doing something similar to what we're doing or trying to get done, which is yeah. You're using recency, right. But are you also trying to support multiple types of addresses, like work, home...
Right now we are not. And we're having such problem with the address. We kind of we've defined it a different nest for a primary address and then alternate addresses. We're actually not using alternate addresses now, but the intent is that the alternate would have multiple types. The primary address, we have a type right now, primary where we're debating, trying to put something else in there, but because we're struggling so much, we're just the one I'm trying to avoid that right now. But ideally we would like to be able to do that, but right now we can't get the basic thing to work and it's actually stopping us from going live with something.
Prasad Satam (01:02:57):
So yeah. So, so Chris, I need to drop off another call. Mark, I will address that question in the community, but thank you all. Thank you, Kurt for, for those questions.
Thank you. Thank you so much.
Sure. Thanks everyone.
Chris Detzel (01:03:09):
Thanks everyone for coming. Really appreciate it.
Chris Detzel (01:03:12):
Thank you, Evan.
Assemble Park City1389 Center Road, Unit 200Park City, Utah 84098USA
+1 (855) 360-DATA+1 (855) 360-3282
© 2022 Reltio. All Rights Reserved
Site by eConverse Media