Reltio Connect

Dear Community, 

Here's a unique situation in our MDM implementation which need expert advise. Say we have 3 source systems.

Source1(SAP), Source2(SFDC), Source3(Legacy). We would like to implement matching/survivorship at 2 different levels

1. Intra source level: SAP can have dupes or similar profiles, based on source specific survivorship/matching, create a golden profile for SAP and similarly, for SFDC and Legacy.
2. Inter Source Level: Once we have essentially mastered individual source profiles, then implement matching/merging across source systems

Question: Is there a way in Reltio to fire matching at different levels, say Rule 1 (SAP), Rule 2(SFDC),  Rule3(Legacy) are triggered first and then Rule 4 (SAP, SFDC, Legacy) to merge profiles across source systems.

Regards,

Waseem

Hi Mohammed,

I'm curious, what is the advantage to this approach?  The end state will end up the same if you just let the matching happen the 'standard' way.  (at least as far as I can tell).  Certainly the crosswalks are the same in the end state.  Further, the data stewarding effort will go up as you rerun the match rules and have to review the matches again.  Configuration effort will be increased as you change match rules and rerun the match job (see below)

That said, the way to do this would be to add an attribute on the data on the ingress that describes the source (i.e. SAP) then make it so these attributes must match as part of the match rule.

Looking forward to your insights.

Dear Community, 

Here's a unique situation in our MDM implementation which need expert advise. Say we have 3 source systems.

Source1(SAP), Source2(SFDC), Source3(Legacy). We would like to implement matching/survivorship at 2 different levels

1. Intra source level: SAP can have dupes or similar profiles, based on source specific survivorship/matching, create a golden profile for SAP and similarly, for SFDC and Legacy.
2. Inter Source Level: Once we have essentially mastered individual source profiles, then implement matching/merging across source systems

Question: Is there a way in Reltio to fire matching at different levels, say Rule 1 (SAP), Rule 2(SFDC),  Rule3(Legacy) are triggered first and then Rule 4 (SAP, SFDC, Legacy) to merge profiles across source systems.

Regards,

Waseem

Hi Gino - Thanks a lot for your response. Thats what I feel, will the end result be same either way?

The reason for this multi-step approach is we have existing rules with Negative Match condition (which eliminates matching/merging profiles from same source, so essentially profiles from SAP doesn't match with other SAP profiles and SFDC doesn't match with SFDC and so on), so I want to create separate new rules (additional attribute as "Source system" Name, so SAP matches only with SAP and SFDC matches with SFDC and so on). This approach will help us in below:

1. Reduce the regression testing effort of changing existing rules with negative conditions
2. Keep track of what resulted in Intra source Merging - For Source system level deep dive analysis to see the extend of data duplication/fuzzy dupes etc.

My concern is, if we cannot control order of priority of these match rule executions (do intra first and then do Inter source (existing rules) later), do we get into a scenario where one rule conflicts/oversteps with other rule and produce un-expected results (might also result in over-merging etc). 

End of the day, it's a case of changing existing rules (regression Testing) vs creating new rules (to avoid regression) kind of scenario and making sure data integrity/efficiency stays intact in either route we choose.

Looking forward for your further guidance in this regard.

Hi Mohammed,

I'm curious, what is the advantage to this approach?  The end state will end up the same if you just let the matching happen the 'standard' way.  (at least as far as I can tell).  Certainly the crosswalks are the same in the end state.  Further, the data stewarding effort will go up as you rerun the match rules and have to review the matches again.  Configuration effort will be increased as you change match rules and rerun the match job (see below)

That said, the way to do this would be to add an attribute on the data on the ingress that describes the source (i.e. SAP) then make it so these attributes must match as part of the match rule.

Looking forward to your insights.

Dear Community, 

Here's a unique situation in our MDM implementation which need expert advise. Say we have 3 source systems.

Source1(SAP), Source2(SFDC), Source3(Legacy). We would like to implement matching/survivorship at 2 different levels

1. Intra source level: SAP can have dupes or similar profiles, based on source specific survivorship/matching, create a golden profile for SAP and similarly, for SFDC and Legacy.
2. Inter Source Level: Once we have essentially mastered individual source profiles, then implement matching/merging across source systems

Question: Is there a way in Reltio to fire matching at different levels, say Rule 1 (SAP), Rule 2(SFDC),  Rule3(Legacy) are triggered first and then Rule 4 (SAP, SFDC, Legacy) to merge profiles across source systems.

Regards,

Waseem

Oh, this is interesting!

First, I'm going to assume you are aware of the special properties of negative match rules documented at https://docs.reltio.com/en/reltio/what-does-reltio-do/what-reltio-does-at-a-glance/data-unification-and-mdm-at-a-glance/data-unification-and-mdm-in-detail/reltio-match-and-merge/match-group-elements---description-and-configuration/negative-rule

But I'm still a little confused at what the hesitation is.  Are your match criteria (other than the negative part for the source) the same between the two sets of match rules?  if so, the negative rule is a subset of the non negative rules.  i.e. the non negative rules will only find entities within the same source.  The concern about over matching is no different if they are a regular set of match rules and the relevant testing.

It would be 'easy' to test this by making the new match rules 'external' only, then running a file with test cases using the external match functionality.  This way you can see what the results would be without loading the data.  I would normally recommend doing that in your test environment.

Hi Gino - Thanks a lot for your response. Thats what I feel, will the end result be same either way?

The reason for this multi-step approach is we have existing rules with Negative Match condition (which eliminates matching/merging profiles from same source, so essentially profiles from SAP doesn't match with other SAP profiles and SFDC doesn't match with SFDC and so on), so I want to create separate new rules (additional attribute as "Source system" Name, so SAP matches only with SAP and SFDC matches with SFDC and so on). This approach will help us in below:

1. Reduce the regression testing effort of changing existing rules with negative conditions
2. Keep track of what resulted in Intra source Merging - For Source system level deep dive analysis to see the extend of data duplication/fuzzy dupes etc.

My concern is, if we cannot control order of priority of these match rule executions (do intra first and then do Inter source (existing rules) later), do we get into a scenario where one rule conflicts/oversteps with other rule and produce un-expected results (might also result in over-merging etc). 

End of the day, it's a case of changing existing rules (regression Testing) vs creating new rules (to avoid regression) kind of scenario and making sure data integrity/efficiency stays intact in either route we choose.

Looking forward for your further guidance in this regard.

Hi Mohammed,

I'm curious, what is the advantage to this approach?  The end state will end up the same if you just let the matching happen the 'standard' way.  (at least as far as I can tell).  Certainly the crosswalks are the same in the end state.  Further, the data stewarding effort will go up as you rerun the match rules and have to review the matches again.  Configuration effort will be increased as you change match rules and rerun the match job (see below)

That said, the way to do this would be to add an attribute on the data on the ingress that describes the source (i.e. SAP) then make it so these attributes must match as part of the match rule.

Looking forward to your insights.

Dear Community, 

Here's a unique situation in our MDM implementation which need expert advise. Say we have 3 source systems.

Source1(SAP), Source2(SFDC), Source3(Legacy). We would like to implement matching/survivorship at 2 different levels

1. Intra source level: SAP can have dupes or similar profiles, based on source specific survivorship/matching, create a golden profile for SAP and similarly, for SFDC and Legacy.
2. Inter Source Level: Once we have essentially mastered individual source profiles, then implement matching/merging across source systems

Question: Is there a way in Reltio to fire matching at different levels, say Rule 1 (SAP), Rule 2(SFDC),  Rule3(Legacy) are triggered first and then Rule 4 (SAP, SFDC, Legacy) to merge profiles across source systems.

Regards,

Waseem

Thanks, Gino for the response. Well one of the hesitations is that sources having dupes (exact + fuzzy) and we have almost 6-7 sources (will increase further), so all these combinations going into a single rule (to perform intra source dupes + inter source dupes) might drastically impact the matching/merging process. So, in order to reduce such overcollisioning (not sure if this makes sense to you), its better to eliminate specific source level duplication via a source system specific rule (mainly fuzzy ones, exact dupes can be removed in integration layer) and then match across source systems via a existing negative rule. But, if we cannot control the order of execution, then this might not work at all and have to go with the option of all source dupes (intra + inter) gets matched within a single rule. Hope this make sense to you

Oh, this is interesting!

First, I'm going to assume you are aware of the special properties of negative match rules documented at https://docs.reltio.com/en/reltio/what-does-reltio-do/what-reltio-does-at-a-glance/data-unification-and-mdm-at-a-glance/data-unification-and-mdm-in-detail/reltio-match-and-merge/match-group-elements---description-and-configuration/negative-rule

But I'm still a little confused at what the hesitation is.  Are your match criteria (other than the negative part for the source) the same between the two sets of match rules?  if so, the negative rule is a subset of the non negative rules.  i.e. the non negative rules will only find entities within the same source.  The concern about over matching is no different if they are a regular set of match rules and the relevant testing.

It would be 'easy' to test this by making the new match rules 'external' only, then running a file with test cases using the external match functionality.  This way you can see what the results would be without loading the data.  I would normally recommend doing that in your test environment.

Hi Gino - Thanks a lot for your response. Thats what I feel, will the end result be same either way?

The reason for this multi-step approach is we have existing rules with Negative Match condition (which eliminates matching/merging profiles from same source, so essentially profiles from SAP doesn't match with other SAP profiles and SFDC doesn't match with SFDC and so on), so I want to create separate new rules (additional attribute as "Source system" Name, so SAP matches only with SAP and SFDC matches with SFDC and so on). This approach will help us in below:

1. Reduce the regression testing effort of changing existing rules with negative conditions
2. Keep track of what resulted in Intra source Merging - For Source system level deep dive analysis to see the extend of data duplication/fuzzy dupes etc.

My concern is, if we cannot control order of priority of these match rule executions (do intra first and then do Inter source (existing rules) later), do we get into a scenario where one rule conflicts/oversteps with other rule and produce un-expected results (might also result in over-merging etc). 

End of the day, it's a case of changing existing rules (regression Testing) vs creating new rules (to avoid regression) kind of scenario and making sure data integrity/efficiency stays intact in either route we choose.

Looking forward for your further guidance in this regard.

Hi Mohammed,

I'm curious, what is the advantage to this approach?  The end state will end up the same if you just let the matching happen the 'standard' way.  (at least as far as I can tell).  Certainly the crosswalks are the same in the end state.  Further, the data stewarding effort will go up as you rerun the match rules and have to review the matches again.  Configuration effort will be increased as you change match rules and rerun the match job (see below)

That said, the way to do this would be to add an attribute on the data on the ingress that describes the source (i.e. SAP) then make it so these attributes must match as part of the match rule.

Looking forward to your insights.

Dear Community, 

Here's a unique situation in our MDM implementation which need expert advise. Say we have 3 source systems.

Source1(SAP), Source2(SFDC), Source3(Legacy). We would like to implement matching/survivorship at 2 different levels

1. Intra source level: SAP can have dupes or similar profiles, based on source specific survivorship/matching, create a golden profile for SAP and similarly, for SFDC and Legacy.
2. Inter Source Level: Once we have essentially mastered individual source profiles, then implement matching/merging across source systems

Question: Is there a way in Reltio to fire matching at different levels, say Rule 1 (SAP), Rule 2(SFDC),  Rule3(Legacy) are triggered first and then Rule 4 (SAP, SFDC, Legacy) to merge profiles across source systems.

Regards,

Waseem