Reltio Connect

I attended the 8/19/21 webinar "Rule Based Matching a Deep-Dive" and posted this as agreed during during the session.

The context is initial cut-over and phasing in actual match merges volume to minimize disruption to downstream systems, internal users and actual customers which will result from significant merge of Organizations and Contacts.

Preferred approach would support the following:
1) Initial one-time historically loaded content is not automatically merged (understand this can be accomplished via rules as suggest only)
2) On-going content changes (create or change) are eligible for automatic merge including originally loaded history while at the same time the originally loaded history would not be merged using the automatic rule unless "driven" by newly arriving matched content.
3) In addition to have the capability to drive a controlled volume (based on match rule and other criteria) of specific automatic or suggest rules in an automated fashion.

For purposes of example:

Assume initially loaded "history" data:

a) first name: john   last name: doe       email: john.doe@email.com
b) first name: john   last name: doe       email: john.doe@email.com
c) first name: fred    last name: smith   email: fred.smith@email.com
d) first name: fred   last name: smith   email:fred.smith@email.com
e) first name: jane    last name: doe      email:jane.doe.@email.com]
f)  first name: janet   last name: buck   email:jane.doe@email.com 

Match rule 1:     Automatic after history load;  exact:  first name, last name, email
Match rule 2:     Suggest:   exact email

After history load of a thru f above no automatic merges to be performed

Now on-going process of new content should drive automatic merges:

New content:
g) first name: john   last name: doe       email: john.doe@email.com

      Now we should merge using match rule 1:
              Only records a, b and g since these are "initiated" by new content
              Not records c and d .. even though they meet the rule, don't want to automatically match unless new content drives the match

     The above processing continues with newly arriving create or change content resulting in automatic merges to existing "history", but no automatic unless new matching content arrives.

    At the same time we can initiate in a controlled approach ideally thru API call or some other suggested mechanism:
i) Merge older history for match rule 1 based on maximum count and / or some other filter criteria
ii) Merge older history and / or new content using suggest match rules such as match rule 2

Any specific recommendations or alternative suggestions would be most appreciated.

Thanks,

Mark

------------------------------
Mark Burlock
Dodge Data & Analytics
Hamilton NJ
------------------------------

@Mark Burlock, I reached out to @Suchen Chodankar this morning. Let's see if he can make a recommendation or two.

Chris ​​

------------------------------
Chris Detzel
Community Program Manager
Reltio
------------------------------

Original Message:
Sent: 08-19-2021 23:15
From: Mark Burlock
Subject: Match Rule Merge Control: No Pre-Merge of initial Load, Merge New and Updated Content to full Corpus

I attended the 8/19/21 webinar "Rule Based Matching a Deep-Dive" and posted this as agreed during during the session.

The context is initial cut-over and phasing in actual match merges volume to minimize disruption to downstream systems, internal users and actual customers which will result from significant merge of Organizations and Contacts.

Preferred approach would support the following:
1) Initial one-time historically loaded content is not automatically merged (understand this can be accomplished via rules as suggest only)
2) On-going content changes (create or change) are eligible for automatic merge including originally loaded history while at the same time the originally loaded history would not be merged using the automatic rule unless "driven" by newly arriving matched content.
3) In addition to have the capability to drive a controlled volume (based on match rule and other criteria) of specific automatic or suggest rules in an automated fashion.

For purposes of example:

Assume initially loaded "history" data:

a) first name: john   last name: doe       email: john.doe@email.com
b) first name: john   last name: doe       email: john.doe@email.com
c) first name: fred    last name: smith   email: fred.smith@email.com
d) first name: fred   last name: smith   email:fred.smith@email.com
e) first name: jane    last name: doe      email:jane.doe.@email.com]
f)  first name: janet   last name: buck   email:jane.doe@email.com 

Match rule 1:     Automatic after history load;  exact:  first name, last name, email
Match rule 2:     Suggest:   exact email

After history load of a thru f above no automatic merges to be performed

Now on-going process of new content should drive automatic merges:

New content:
g) first name: john   last name: doe       email: john.doe@email.com

      Now we should merge using match rule 1:
              Only records a, b and g since these are "initiated" by new content
              Not records c and d .. even though they meet the rule, don't want to automatically match unless new content drives the match

     The above processing continues with newly arriving create or change content resulting in automatic merges to existing "history", but no automatic unless new matching content arrives.

    At the same time we can initiate in a controlled approach ideally thru API call or some other suggested mechanism:
i) Merge older history for match rule 1 based on maximum count and / or some other filter criteria
ii) Merge older history and / or new content using suggest match rules such as match rule 2

Any specific recommendations or alternative suggestions would be most appreciated.

Thanks,

Mark

------------------------------
Mark Burlock
Dodge Data & Analytics
Hamilton NJ
------------------------------

Hi Mark,

Thanks for the detailed explanation of the use case. 

By default, the matching engine works exactly the way you described the requirement above. When new match rules are introduced or any changes are made to the existing rules, matching the existing records using the changed matching rules is not triggered automatically. The matching is triggered only when the records are updated or added and the matching is triggered only for the changed or newly added record. 
The matching of the entire record set can only be triggered by running the RebuildMatchTable job. Until then only the changed/new records will go through the matching using the updated matching rules. Please note that the matching will be triggered if one or more conditions listed below are true

• One or more match attributes of the record are changed
• Match rule is changed or added since the record was last updated
• OV value of one or more match attributes of the records are changed

Regarding the second question related to the merge, you can use the merge API to merge any given pair of records.  Here is the link to the documentation of this API
https://docs.reltio.com/mergeapis/mergingtwoentities.html

Hope this answers your question. Please let me know if you have any follow-up questions on this.

------------------------------
Suchen Chodankar
------------------------------

Original Message:
Sent: 08-19-2021 23:15
From: Mark Burlock
Subject: Match Rule Merge Control: No Pre-Merge of initial Load, Merge New and Updated Content to full Corpus

I attended the 8/19/21 webinar "Rule Based Matching a Deep-Dive" and posted this as agreed during during the session.

The context is initial cut-over and phasing in actual match merges volume to minimize disruption to downstream systems, internal users and actual customers which will result from significant merge of Organizations and Contacts.

Preferred approach would support the following:
1) Initial one-time historically loaded content is not automatically merged (understand this can be accomplished via rules as suggest only)
2) On-going content changes (create or change) are eligible for automatic merge including originally loaded history while at the same time the originally loaded history would not be merged using the automatic rule unless "driven" by newly arriving matched content.
3) In addition to have the capability to drive a controlled volume (based on match rule and other criteria) of specific automatic or suggest rules in an automated fashion.

For purposes of example:

Assume initially loaded "history" data:

a) first name: john   last name: doe       email: john.doe@email.com
b) first name: john   last name: doe       email: john.doe@email.com
c) first name: fred    last name: smith   email: fred.smith@email.com
d) first name: fred   last name: smith   email:fred.smith@email.com
e) first name: jane    last name: doe      email:jane.doe.@email.com]
f)  first name: janet   last name: buck   email:jane.doe@email.com 

Match rule 1:     Automatic after history load;  exact:  first name, last name, email
Match rule 2:     Suggest:   exact email

After history load of a thru f above no automatic merges to be performed

Now on-going process of new content should drive automatic merges:

New content:
g) first name: john   last name: doe       email: john.doe@email.com

      Now we should merge using match rule 1:
              Only records a, b and g since these are "initiated" by new content
              Not records c and d .. even though they meet the rule, don't want to automatically match unless new content drives the match

     The above processing continues with newly arriving create or change content resulting in automatic merges to existing "history", but no automatic unless new matching content arrives.

    At the same time we can initiate in a controlled approach ideally thru API call or some other suggested mechanism:
i) Merge older history for match rule 1 based on maximum count and / or some other filter criteria
ii) Merge older history and / or new content using suggest match rules such as match rule 2

Any specific recommendations or alternative suggestions would be most appreciated.

Thanks,

Mark

------------------------------
Mark Burlock
Dodge Data & Analytics
Hamilton NJ
------------------------------

Suchen,

Appreciate the response and definitely moving us in the right direction.

The way you describe behavior for new items is close to what I am looking for with the gap related to including older content in the match inventory only driven by newly arriving content. We can possibly  stitch something together between merge API, entity API for search and the match rules.

Your clarification of what selective updates to existing content would result in that same content being included in match rules was very helpful and is different than the behavior I assumed (dangerous on my side), as I thought a change to ANY attribute would be the data into the "match inventory", but now I understand the additional restrictions.

One other variable that I'm now considering is that we have existing "match" identifier from older legacy systems which is used by downstream systems and I'm now thinking to include that ONLY where our Reltio based match rules are solid so that we look more like the older system but only where the matches also align with our new match rules.

I also see that in addition to the Merge API that you specifically referenced indexed in documentation as "Merging Two Entities", there is also an entry indexed as "Merge Entities". Could this 2nd entry also be of use to us and would it permit match/merge on more than 2 entities at a time.

If by any chance there is some additional guidance on how to best stitch the components together or a customer that is doing same, it would be most helpful for to take that into consideration.

Thank you much,

Mark

------------------------------
Mark Burlock
Dodge Data & Analytics
Hamilton NJ
------------------------------

Original Message:
Sent: 08-22-2021 01:27
From: Suchen Chodankar
Subject: Match Rule Merge Control: No Pre-Merge of initial Load, Merge New and Updated Content to full Corpus

Hi Mark,

Thanks for the detailed explanation of the use case. 

By default, the matching engine works exactly the way you described the requirement above. When new match rules are introduced or any changes are made to the existing rules, matching the existing records using the changed matching rules is not triggered automatically. The matching is triggered only when the records are updated or added and the matching is triggered only for the changed or newly added record. 
The matching of the entire record set can only be triggered by running the RebuildMatchTable job. Until then only the changed/new records will go through the matching using the updated matching rules. Please note that the matching will be triggered if one or more conditions listed below are true

• One or more match attributes of the record are changed
• Match rule is changed or added since the record was last updated
• OV value of one or more match attributes of the records are changed

Regarding the second question related to the merge, you can use the merge API to merge any given pair of records.  Here is the link to the documentation of this API
https://docs.reltio.com/mergeapis/mergingtwoentities.html

Hope this answers your question. Please let me know if you have any follow-up questions on this.

------------------------------
Suchen Chodankar

Original Message:
Sent: 08-19-2021 23:15
From: Mark Burlock
Subject: Match Rule Merge Control: No Pre-Merge of initial Load, Merge New and Updated Content to full Corpus

I attended the 8/19/21 webinar "Rule Based Matching a Deep-Dive" and posted this as agreed during during the session.

The context is initial cut-over and phasing in actual match merges volume to minimize disruption to downstream systems, internal users and actual customers which will result from significant merge of Organizations and Contacts.

Preferred approach would support the following:
1) Initial one-time historically loaded content is not automatically merged (understand this can be accomplished via rules as suggest only)
2) On-going content changes (create or change) are eligible for automatic merge including originally loaded history while at the same time the originally loaded history would not be merged using the automatic rule unless "driven" by newly arriving matched content.
3) In addition to have the capability to drive a controlled volume (based on match rule and other criteria) of specific automatic or suggest rules in an automated fashion.

For purposes of example:

Assume initially loaded "history" data:

a) first name: john   last name: doe       email: john.doe@email.com
b) first name: john   last name: doe       email: john.doe@email.com
c) first name: fred    last name: smith   email: fred.smith@email.com
d) first name: fred   last name: smith   email:fred.smith@email.com
e) first name: jane    last name: doe      email:jane.doe.@email.com]
f)  first name: janet   last name: buck   email:jane.doe@email.com 

Match rule 1:     Automatic after history load;  exact:  first name, last name, email
Match rule 2:     Suggest:   exact email

After history load of a thru f above no automatic merges to be performed

Now on-going process of new content should drive automatic merges:

New content:
g) first name: john   last name: doe       email: john.doe@email.com

      Now we should merge using match rule 1:
              Only records a, b and g since these are "initiated" by new content
              Not records c and d .. even though they meet the rule, don't want to automatically match unless new content drives the match

     The above processing continues with newly arriving create or change content resulting in automatic merges to existing "history", but no automatic unless new matching content arrives.

    At the same time we can initiate in a controlled approach ideally thru API call or some other suggested mechanism:
i) Merge older history for match rule 1 based on maximum count and / or some other filter criteria
ii) Merge older history and / or new content using suggest match rules such as match rule 2

Any specific recommendations or alternative suggestions would be most appreciated.

Thanks,

Mark

------------------------------
Mark Burlock
Dodge Data & Analytics
Hamilton NJ
------------------------------