Find the PPT: Using Reltio Integration Hub to automate Reltio workflows PPTJoin Dan Gage , Principal Solutions Consultant and Chris Detzel, Director of Customer Community and Engagement at Reltio for another Community Show on using RIH to automate Reltio Workflows. Reltio workflow provides a controlled way to route profiles to different users for review, enrichment and approval. Initiating workflow in Reltio has traditionally required manual intervention or automation via a Lifecycle Action. With the introduction of Reltio Integration Hub, Reltio configurators now have a simplified way to trigger the automation of workflows based on conditional logic without resorting to LCA. This session will walk through the process to create this automation. Note that this is not session on creating Reltio workflow, nor will it be a tutorial on Reltio Integration Hub, but rather it will focus on leveraging these two modules to automate a data governance process.Transcript:
Chris Detzel (00:05):
Thank you everyone for coming to another Reltio Community show. My name is Chris Detzel and I'm the Director of Customer Community and Engagement, most of you know me. We have our special guest, Dan Gage. Although, he has been on several times in the past. He's a Principal Solutions Consultant and today's topic is, Using Reltio Integration Hub to Automate Reltio Workflows. As most of you know, we've been doing a lot of shows around Reltio Integration Hub and Dan came to and is like, "Hey, I've been doing some really interesting stuff around workflows. Let's do a show." And I said, "Let's do it."
As usual, keep yourself on mute. Please post your questions in the chat and or take yourself off of mute and ask your question. I'll be making sure to moderate those questions and then ask as many as I possibly can to Dan. And then we are recording this, just as an FYI, as usual. And we'll push this out to the community within probably early next week. Probably won't be tomorrow. And then we have some really cool shows coming up today with Dan using Reltio Integration Hub to automate Reltio workflows. Excited about that. On the seventh we have one around building an MDM practice, so harnessing the momentum to evolve. So we have one of our, Chris [inaudible 00:01:35], he's going to be doing that.
And then we have a show around getting started with rest APIs and Reltio. So Joel Snipes is coming back. It's been a few months, but super excited to have him back. And then one that I am really excited about too is, Blazing Trails and Master Data, stories from women succeeding in MDM. Megan, our senior director over our UX and documentation team will be leading the discussion with three other women with Reltio in the MDM space. So hopefully you can come to that.
And then we do have our round table cab discussion around documentation in the UX of the documentation. We have a few of our customers that help kind of make some of those decisions and say, "Yeah, that's good, that's not good." So if you want to become a part of that, let me know and we'll get you in touch with, her name is Mel actually, and she runs that. So without further ado though, Dan, I'm going to stop sharing, and I'm going to let you share.
Dan Gage (02:40):
Great, thanks. All right, can you see my intro slide? I've just got a few slides here just to set the-
Chris Detzel (02:56):
I can. Yes.
Dan Gage (02:57):
... concept. All right, so to start off with, one of the things that we wanted to focus on here is really around automation. So, there's a new term that's been floating around the industry for a few years called augmented MDM. And it's really about the ability to automate processes and to in situations where it's possible have the computer make decisions using machine learning. Those are definitely some areas where Reltio is continuing to invest. But one of the first areas is really going to be around using the Reltio Integration Hub. And the Reltio Integration Hub has been around for over a year now and it's really focusing on a low code, no code way of automating various different processes for Reltio. So this is definitely one where I will be heavily on the no code side today. So just using the available services that Reltio has in the Reltio Integration Hub to allow users to interact with products or customer records that have been in this case newly created.
The second phase in this is going to be the Reltio workflow engine. And Reltio has provided this workflow engine for, gosh, five or six years now. It's been around, it's been part of the core product and it's the ability, as you can see on the right-hand side, just to create really just a flow chart that allows information or records profiles to be routed and notified with different users to allow them to participate in some kind of review and approval process. So, that's the core of Reltio workflow. And then the third area that we'll focus on is really that event-based publication. Because this is really what's going to allow a lot of this automation to happen. So this is somewhat complex, this is really the data management life cycle. I'm going to focus on the red box in the upper left-hand corner.
Really when profiles come into Reltio, and it doesn't matter how they get into Reltio, if you key it in through the Reltio native UI, if you load it through the Reltio Integration Hub, or if you loaded through Reltio data loader. Or even if you're using a third party tool like a Talent or Power Center and you're ingesting that data into Reltio, no matter how it gets into Reltio, it's going to be considered an onboarded record. Reltio is going to cleanse it, we're going to standardize it, we're going to validate the record. I'm going to skip over all this stuff in the middle here. There's a lot of things that we do around matching and merging. I really want to focus on this blue line at the bottom that really comes over to this publication queue. So Reltio provides externally available queues that whenever something happens within Reltio, we're going to put an event on that queue that you can subscribe to, and allow you to take some action based upon that.
So as you can see from this diagram, anytime a record is created or updated, we'll put an event in that queue. Anytime a record is merged, unmerged, split, even when a potential match is detected, we're going to put events in those queues. So the process we're going to see today is going to focus on triggering a workflow when a new product has been created. You can actually subscribe to different events, you can put conditional logic in there and you can do most anything. If you want to send an email, you want to send literally an SMS message to somebody, letting them know that some event has happened in the system. If you want to use our data validation framework and recognize that a new violation has been detected, we're going to put an event on that queue. And this methodology that you're going to see today can be used to notify people or to route it via workflow to remediate those processes.
So, this kind of summarizes that. Starting on the left-hand side, you've really got the Reltio Cloud here, the MDM services. And I show everything here in this larger light blue cloud because it's important to recognize that everything that we're going to do today is all part of the Reltio platform. It's part of your existing license. There's no need for you to go out and buy third-party tools. There's no need for you to upgrade your license or to add additional features. This is all functionality that you can use today and start getting additional value from your organization. So again, the demo process that we're going to go through today is we're going to start in the hub. We're going to create a new product record. When that record gets created, it's going to put an event into that queue. And you'll notice the direction of the lines here. Reltio is going to push that event into the queue, but over here on the lower right-hand side, the Reltio Integration Hub is going to be listening to that queue.
So it's effectively a near real-time monitoring, sometimes refer to in the industry as change data capture. So it's capturing those data change events that are stored in that queue, and Reltio Integration Hub will detect that. And then based upon our process, we're going to invoke a Reltio workflow, which is going to allow somebody to take some action through the Reltio Integration Hub. So, that workflow engine is what's going to be processing that flow based upon the triggered event. So for our scenario today, we are going to go with the simplest of simple workflows. It's literally going to be a one step workflow that simply has one option that clicked that says, "Yep, I reviewed this process."
But we have other sessions that have been hosted by myself and Joel. The workflow that's going to be triggered by this process, however many steps are in that workflow, is completely up to your organization. You can make them really complex, having conditional logic, interacting with other third-party systems or external services. The process we're going to focus on today is literally just using that event to initiate a workflow. What that workflow does is up to your organization.
So, we'll start with a very simple one for our demonstration today. And as I mentioned, whenever there's a change within the Reltio infrastructure, we're going to push those events to a queue. So for most of our customers those queues are already configured and they're available in your system. You can actually have multiple queues. So if you don't have one, I encourage you to reach out to your customer success team. They can help you get it activated. If you want to add another queue, they can certainly work with you there. If you provision a queue in your own environment, then we can hook that in here. And it's important to recognize too that Reltio does work with the three primary queuing platforms that we can deploy to.
So for our demonstration purposes today I will be using the Amazon SQS, but we provide equal support for the Google Pub/Sub service, as well as the Azure Service Bus. So. once that queue has been instantiated, you make a change within Reltio, it's going to put that event in the queue. And what we're going to walk through today is actually going in from scratch and creating a new recipe in the Reltio Integration Hub that's going to listen for that event and then launch that workflow. So let's go into the live software and take a look. Any question before as I switch over there. Chris, do we have anything so far in the chat?
Chris Detzel (10:12):
Nothing yet. Sorry, I was looking for the mute button.
Dan Gage (10:15):
No problem, no problem. All right, so here I am in the Reltio software. As you can see here, I'm on the advanced search screen. It really doesn't matter where you are inside of Reltio. What I want to do is launch the Reltio Integration Hub. So today I'm using the Reltio new UI. So whether you're using the new UI that we've recently released, which is really just the same functionality, we've just moved some of our navigation here to the left-hand side, but everything is fully available. So the menu option here on the left-hand side is going to take me into that integration hub. But there's a couple other things that we want to do before we go there. We're going to go into our console and we're going to check and make sure that that workflow has been deployed. And as part of the assets for this community show, we'll provide a sample copy of these workflows for your team.
So going into the workflow modeler, I'm going to open that existing file. I'm going to go into my workflow directory. And I simply have a workflow called Review BPMN 2.0. So we'll provide this as a downloadable asset. And as I mentioned before, it's an incredibly simple workflow. It literally is going to start, it has one step, and in that step it's going to go to anybody that has role user. So in your choices, you can customize this and you can have it go to a specific role inside your organization. And then on the form, when you look at the decisions, you can see down here at the bottom, we're going to have just a single to say, "Yes, I reviewed it."
So again, through workflow customizations you can make that more complex, but I'm simply going to hit the publish button to make sure that this workflow is now going to be available in my Reltio environment. And you see here at the bottom of the screen, I got success. So that lets me know that this workflow is now available to be invoked.
Next thing we want to do is check that external queue. And that's done under tenant management. So, in tenant management you get some usage statistics where you can see your internal queues. But most importantly, we're going to focus on that external queue here. And I'm focusing on this SQS. Now, you'll see here that this environment actually has two queues. The Google Pub/Sub queue is actually not active. You'll see it's been turned off here. But the SQS queue here is where I'm going to be focusing. And again, you can set it up via whatever provider you're using. Putting in the proper credentials, and you can use the IMs and so forth. And you can look at what events you actually want to publish. So there's a large number of events that Reltio is actually going to trigger through the various different orchestrated processes that we deliver. So I've chosen a wide variety of them here. And then the data that's going to get pushed to that queue, you can get a snapshot, which will include all the fields.
You can get a snapshot, which allows you to select only a subset of those fields that are relevant to the process you're looking. And I'm actually going with a Delta today, which is going to show me the before and after values of any field that's relevant. And you see I have the checkbox that this is enabled, so there's no further action needed at this time. So now we're ready to go into that Reltio Integration Hub. And again, from the Reltio tenant, you can go directly to Integration hub or from the console here, I can jump right in. So Reltio Integration Hub follows the standard best practices around segmentation of different assets in your environment. So I'm going to actually create a project here. And a project is really just synonymous with a folder. So when I create that project, you'll see that it just shows up as a folder here, and I can now go in there and create a recipe. So the recipe is the integration flow that we're going to be defining here.
So other solutions call it a pipeline or an integration. Our term with Reltio Integration Hub is this is a recipe. And you give it a name, and you have various different what are called trigger activities. And again, this is just standard Reltio Integration Hub activity. And what I want to do is I want to listen to a specific event from an application. And this is where the change data capture becomes relevant. Because you could run on a schedule where you could publish it as an API to have it triggered externally. But I'm going to have this process running all the time and it's going to be waiting for something to happen in an application. So when I start building this process here, the first thing it wants me to do is select that application. Well, you can see here I've got some that I've used in the past, but the Reltio Integration Hub does provide a wide variety of applications that you can connect to and listen to for various different events.
But I'm going to be focusing on that Reltio SQS environment. So we have a specific connector here that's been optimized for the Reltio platform. You can listen to single messages or I can say, "You know what, I want to take any message that comes in off of that queue." So I'm going to choose that option. And you'll notice across the top of the screen here, there's a Wizard Apps trigger connection and set up. So in your environment, you're going to want to go in there and define a new connection and put in the credentials and the queue details for your environment. In the interest of time, I'm going to go ahead and just choose one that I had already defined. So I've selected my queue, there's really no additional steps that I need to take. It's now listening, it's going to listen to that queue whenever I run it, and I need to take some further actions.
Chris Detzel (16:11):
Dan Gage (16:13):
Chris Detzel (16:13):
Quick question, sorry. But can these workflows be set up for a specific attribute change? So example, we only need a workflow to alert the user price of a product change and not any other attribute?
Dan Gage (16:26):
Absolutely, absolutely. And we'll get into that. Once we get the process flowing we'll take a look at some of the examples there.
Chris Detzel (16:32):
Okay, thank you.
Dan Gage (16:34):
So, now that we've started this process, we're listening to messages coming in from that queue. We recognize that there could be multiple messages coming in. So I'm going to come in here, I'm going to say, "Well, what do I want to do? Do I want to go into another application?" In this case, I actually want a repeat action. So a repeat action is really a looping mechanism. Because I know that multiple messages can be coming in from this queue, I want to be able to process all of them. So, I'll choose that repeat action and you'll see that it creates this indented loop structure that's going to allow me to process all the incoming messages. And within Reltio Integration Hub we use a concept called a data pill. And the data pill is really the source of data that we're going to iterate on from the previous step, or from a previous step.
So by clicking in that field, it's going to show me, in this case there's only one previous step, so I really can only choose from that area. And it's going to highlight in the white box the area where I'm getting an array. Which is basically a grouping of multiple pieces of information. That's what that data pill is really looking for. It wants to iterate through an array of, in this case, all the messages. So because I've already clicked in this field, I can simply click right on the word messages, or I can drag and drop it into that field. So here I just, you'll see I click and messages gets populated into that input list. and you'll see that it starts populating out some details to make it easier to read. And again, this is where that low code no code concept comes in is that I'm not writing any code, I'm performing actions, and it's helping me visualize the actions that I'm taking.
Again, in the interest of demonstration, I'm going to process these messages inside this loop one at a time. But it's possible to optimize this and allow multiple records to be processed concurrently. But again, for our purposes, we're going to go one at a time. And for those records, what do we want to do? We're going to want to get the payload in each of those messages. So it's important to say, "Well, what is that payload going to look like?" Well, actually I can go out and you can have a library, you can go into the Reltio documentation, it will tell you what that payloads going to look like. Or I can actually run a sample instance and I can get that payload directly from the Reltio Integration Hub.
So in the interest of simplicity, I'm going to go that route to make sure that everybody on the session here has the ability to get the proper payload for their instance. So the first thing I need to do is start a process that's going to create an event. So I'm going to say, "I want to create a new product." And I'll just give it a simple name here and hit save.
Chris Detzel (19:28):
Wow, that's my favorite. Thank you.
Dan Gage (19:32):
So, I've created this as a product. You'll see that I also have a status and a life cycle phase. I've got some default values that got populated in here, but behind the scenes when Reltio created this, in this case I'm using a product, entity type. It can be an HCO, an HCP, a consumer, a digital asset. It can be any type of entity that Reltio defines. But now that that's done, if I simply click the test button in the upper right-hand corner here, it's going to run through this one time. And it's upsetting at me here because I have an action that I haven't actually done anything with. So, I'll just pull it out for now.
And again, what I'm looking for right now is I want to see the structure of the message that the queue is going to provide to me. So because I've already triggered that, I created that product, it's put the event into the queue. And what's happening right now is it's actually spinning up an instance. It's connecting to that queue, it's listening for events. So it takes anywhere from 10 to 20 seconds to find and connect. So it says that it was successful. And if I click on any of these steps, you'll see on the right-hand side it's show me the input and the output to that step. So it's telling me that it got a couple of items. And if I look at the output here, this is really what I'm focusing on is this body of the message. That's what I want to capture because that's where that important information is going to be.
So, I can just copy that information and here I'll just hit the copy JSON and I'm going to go into my favorite JSON Editor here and I'm going to blank it out. And I'm just going to paste in that information that came from that message. And again, it's this body right here. So the outer payload is all the details that came from the SQS, but this payload body, this is the actual Reltio entity and it's telling me what happened. So I want to copy that information that's inside there. So I'll just use a cut action to take it out and then I'm going to remove everything else and put it back in.
So because that payload body starts off as a escaped character set, you'll see it uses the back slash quote here, I'm simply going to do a quick search and replace, and I'm going to say, "Replace back slash quote with a single quote" And then I want my favorite text editor here to reformat this document. Reformat, and therefore I'll get that nice structure that makes it easy to see. And you can see here that the entity was created, it was a product, it was the Reltio Community Webinar. So this is definitely the message that I was expecting. So I'm going to copy this message structure here. And what Realty Integration Hub is going to do, and it's important that I copy that structure, because each customer is ... Your data model can be extended. So your attributes can be different, depending upon which type of event publication, whether you use the Snapshot or the Delta payload, those, that payload can be slightly different. So this process that I'm following here can be fairly helpful.
I'm still here, you see it in the top of the screen, I'm still in the test mode. So if I go back to the recipe edit mode, what I'm going to want to do is parse that JSON. So Reltio Integration Hub has a JSON parser as one of the steps. And this sample document here, that's the whole reason that I went through that whole process of getting that sample document is so that I could paste this in here and Reltio is going to know what kind of information to expect from that event. And then this is the sample structure that it's going to expect, but the actual document itself is going to come from that loop. So because that loop is looping through the body of all those messages, I want to take that body and that's what we're going to parse here. So, I can now take another step.
Now, it's not necessary to store data in intermediate steps, but again, for training and debugging purposes, I found that it's nice to be able to create what's called a variable. So, if you create a variable and I want to do the event type, I'm going to track that and I want to add another variable, and it's going to be the URI of the object that I acted on. So I've defined these two variables and I would have pulled them from that parsed JSON document. So now that I've defined them here on the right, you can see, I can simply click in the data that came out of that previous step. You can see that because I provided that sample document, it's going to let me see what all the available information that is coming out of that queue is going to look like. And I simply want the event type to go here and the URI on the object to go here.
So again, for demonstration purposes, we can run this in a test event again. And just while that runs, we'll come back over here and look and you can see that the entity ends in JE0. That's that Reltio Community webinar URI that we were using. That's the item that was generated by Reltio.
So, if I come in here and I click on that step, you can see that output and there actually were multiples because when it was created, you see it's on that same one. But one of the events that I got was that the analytical attributes had changed. And that's because I have a data validation rule here that I'm checking to see whether or not that in this case we're looking at the G10 was defined with that product. So, there's multiple events that were actually generated. If I go back up into the four loop up here, I can expand that out and I can see the entity created event was there, and then there was another one for the analytical attribute that came down. But we're definitely on the right path. So now what I want to do is I want to launch that workflow if the event type is that the record was newly created.
So by going back into edit mode, I can add a step. And I want to do some conditional logic here. I want to say, if, and by clicking in the data field I want to say, "Is that event type equal to entity created?" Just to double check, entity created. So I've spelled it correctly. So we're only going to be inside this loop here if that happened. So this is where one of the questions came in. If you wanted to look at specific changes to specific attributes, up here I could say, "Add a variable to say what attribute changed." And again, it's not necessary to store this information, intermediate attributes, you can do your validations and your equations directly on the data coming out of this JSON document. But this just makes it a little bit easier to visualize, especially when you're first learning the platform. And as you become more efficient, you could bypass or skip some of these intermediate steps.
But populating that attribute changed, that attribute changes in the Delta. I can see that it was an entity created. Well, in this case it's actually not an entity. There was no entity to be changed. They were all changed because it was a new record. So I'm going to see that they're all in here. But I could grab the ... See, there's the type, product name. So if I was looking for a specific attribute changing, I could drag that type in here. And again, if there's, time permitting, we can go in and we can actually change just the single attribute through an edit action. And instead of being an entity created event, it would be an entity changed event. And then the attribute being changed will also have it before and after value. And you could check to see whether that before and after value was changed within whatever tolerance you feel is necessary in order to trigger workflow. So we're.
Chris Detzel (28:41):
Dan Gage (28:42):
Chris Detzel (28:43):
Quick question. Yeah, do we have any platform for like a POC?
Dan Gage (28:49):
So, the Reltio Integration Hub is actually available to all of our customers. You need to work with your customer success team, and you can use this on your dev environment. So, if you don't have access to Reltio Integration Hub, there is no additional cost to it, it's just a matter of notifying your customer success manager and asking them to activate it. And there's a little bit of paperwork that you need to do. But for anybody that is new to Reltio, you can certainly reach out to your Reltio account team and we can certainly work with you to make that available.
Chris Detzel (29:22):
Dan Gage (29:22):
All right, so going back in our options. When the entity is created, our goal for today was just to launch that workflow. So the action that we take is going to be to call the Reltio Connector. And what action do we want take? So you that the Reltio Connector gives you lots and lots of options here that are specific. But because we're going to be creating a workflow, there's not a prebuilt option for the workflow. So we're actually going to utilize what's called the custom action within the Reltio Connector. And that's going to let me call absolutely any API that I choose. So another area there would be like the Reltio Data Loader and the Reltio export service. We do not yet have specific pre-built connector actions for some of those specialized APIs. They will be coming at a future release. And likewise, I would think that the ability to automate workflow will be coming in a future release. But you'll see here in a moment how easy it really is.
So, I've chosen the custom action. It's going to say, "Okay, what credentials do we want to use?" And again, I've already done that, but in your wizard, if you haven't yet connected, it's going to ask you for the tenant ID, which is going to be here in your URL. You'll know your tenant, your environment, and it'll walk through a little wizard there for you to establish your connections. And if you're unfamiliar with the Reltio Integration Hub, we've got some great training on the Reltio Academy that'll help walk you through that as well. So I'm just going to choose the Reltio account that's out there. And my custom action, I'm going to call it Launch Workflow, and it is going to be a post action. And it wants to know the URL. So in most cases, when you're interacting with the data in the URL, you can accept the default URL, which is going to be that for the tenant.
But in my case, because I'm calling the workflow server, I've actually saved this over here in a temporary folder. I'm going to be invoking the workflow server. So the benefit here is that it's going to use the Reltio Connector, which is going to automatically get my off token. It's going to make sure that it doesn't time out. If it does, it's going to refresh, even if I'm processing many, many events. But in this case I'm going against the workflow API. So it's not the same environment or endpoint as my tenant. So for me, because I'm part of the Reltio sales organization, I'm using the sales workflow. For most of our customers, you'll be using the prod-workflow environment. So again, this is just going to be specific to where the workflow environment engine is that you'll be utilizing. For our customers that are deployed in Europe, there may be a slightly different workflow engine. And again, if you have any issues with this, you can contact your customer success manager or Reltio support, and they'll help direct you to the proper workflow server for your environment.
Chris Detzel (32:28):
So Dan, I want to clarify something. So, I thought I heard you say there's no additional charge, there's just additional paperwork, but Mano says it will be an extra cost if client activates Integration Hub. But I thought I heard you say there's not?
Dan Gage (32:43):
Yeah, no, there is definitely no cost for utilizing the Reltio Integration Hub. There are some quotas and limits on the Reltio Integration Hub specifically, very similar to the fact that you have a limit to the number of Reltio API calls you can make each month. There is a limit to the number of tasks you can make. But it's fairly generous from the standpoint that it's intended that for the vast majority of our customers, you would never have to extend your license to purchase additional tasks. But if you ever did, they're very, very inexpensive. It's really just kind of guardrails very similar to the storage, just because Reltio gets cost from our infrastructure providers, but it's definitely no cost to activate it. If you have not activated it, you will get a $0 order form from your Reltio customer success manager. So there will be no cost to that. You just have to accept the terms of recognizing that this new product has been added. Because if you signed your license prior to Reltio Integration Hub being available, there's some terms in there that you just need to agree to.
Chris Detzel (33:52):
All right. Thank you for clarifying. There's another question. Is there a way to get credentials or values from Amazon Parameters store, instead of hard coding while creating recipes?
Dan Gage (34:05):
I don't know the answer to that. So that's definitely something we'll need to follow up on with Aion or somebody from the Reltio Integration Hub team. But that would be a great opportunity to post a question to the Reltio community.
Chris Detzel (34:17):
You read my mind.
Dan Gage (34:18):
If another member already has the answer or if we need to bring in our product management team, we can certainly help you with that.
Chris Detzel (34:25):
Great. And Chatan [inaudible 00:34:27] what's that?
Yeah, [inaudible 00:34:34].
Chris Detzel (34:34):
Great, thanks Chatan. And then another question, can we call Reltio APIs using this in a similar way?
Dan Gage (34:43):
Oh, a 100%, absolutely. So yeah, if you wanted to update the record, if you wanted to read additional information from a record, if you wanted to read related records. Some of the other actions that you can do, I'm working on a proof of concept with another customer, is to use the, identifying when a match has been created. And to use that to potentially group records together based upon the logic. So, that's very similar to what Reltio was doing with our household functionality. Is the household functionality is generating events that you use to trigger specific actions.
Chris Detzel (35:21):
So one more question. Thanks for that, Dan. Can service tasks inside workflow interact with Reltio Integration Hub? So for example, calling an external API from the service task?
Dan Gage (35:34):
They certainly could, yeah. A service task is just Java code. And any Reltio Integration Hub process, if you write it to be exposed as an API, then it could certainly be called from the workflow itself. So Reltio Integration Hub could call the workflow or a workflow could call back to real integration hub.
Chris Detzel (35:54):
Great. Thanks Dan. That's all the questions for now.
Dan Gage (35:56):
All right, so one other important factor. Because I'm calling the workflow server. As you may be aware, the authentication details are going to be stored as a header parameter for the rest API call. And that's automatically going to be handled by the Reltio Connector. But specific to the workflow server, you need to tell it the environment because I'm connecting to the sales workflow or the prod-workflow or the Europe workflow, whichever workflow server you're connecting to, you need to put in those headers. And you don't see the headers by default. But if I scroll down to the bottom, you'll see this little check box here where the options, there are two additional fields, and if I select that, you'll see that those request headers are available. They're just not displayed by default. So I need to show those optional fields for the request headers.
And now that I see my request headers, I want to go in and add a header. And it's the environment URL. So when I call the workflow server, I need to let it know which environment I want to trigger this workflow for. Because the prod-workflow server, it can work across potentially multiple environments, your test environment, your prod environments and so forth. So for me it's https, just got sales.reltior.com. If you go back and look at your environment, it's really just that prefix for whatever tenant you're going to be accessing. So, that is the only mandatory parameter that you need to add in there. What I also need to do is the input. The data that's going to be passed to start the workflow. And again, Reltio Integration Hub will allow you to learn that payload from a sample JSON document. So I'll choose, use a sample. And again, I had saved over here a sample payload that the workflow server. And again, if you go to developer.real.com, you'll see the API for starting a workflow process instance, and it will give you this payload.
But again, we can provide this as part of the assets that we attach for this community reporting. So I'm simply going to come in here and paste that in as a sample document, and that's going to help Reltio Integration Hub learn the structure. So, it's generated that schema and now have data that I can map to. The process type. You'll recall that I inserted that workflow earlier, and I simply called it review. That was just the name of the workflow when you deploy it. So when you deploy a workflow, whatever name you have on your workflow, that's the name of the process that you'll be calling here. The object URI is going to come from that attribute that I parsed into a variable here earlier. And you see there's that URI. So I can drag and drop and put that up here.
But it's important you see on the left-hand side, it wants it to be in array. So there's a couple different ways to do that. The simplest way is if I just come in here, because I'm only doing one record, I can simply put a left bracket and a right bracket around it and it'll make it an array of this one object. And then the comment is what the user is going to see. So Reltio community is cool. That's just the message that you're going to see inside the system. And I have actually done absolutely everything that's necessary, but actually we'll throw in, like Steve Jobs used to say, "One more thing." We grab because we're listening to this event from the queue, we actually want to take that message out of the queue when we're done processing it. So here, once we've launched that workflow, we're going to add one more step. It's going to be an action that interacts with that queue. And we want to delete that message out of the queue. Because right now that message is just sitting out there.
They will eventually time out, but the receipt handle is going to come all the way back from the beginning in that for loop from those messages that are coming out of the queue. And if you scroll down, you'll see there's that receipt handle that was in there. So for each message that we process, once we're done processing it, we're going to tell the queue, "We're done with that message, you can delete it." Now, again today it's outside of the scope of our exercise to talk about topics and subscriptions and best practices around queue management. So there's definitely a number of other things that people that are very queue savvy, if you have multiple processes that are all feeding from the same queue, you'll want to not necessarily delete messages from the queue, you'll want to set up topics and subscriptions and connect to those instead. But for our purposes, we've done everything we need to do.
So if I hit test here, it's going to take a sec, it's going to spin up the infrastructure that's necessary to execute this process. It's going to reconnect to that queue and that queue should still have ... And actually there were two events out there. One was that analytical attribute was created when the data validation rule was triggered. And the other of course is going to be that entity created event that Reltio generated. So when I come in here, look, so apparently something had timed out, so you see that it came down and it said condition was not met. And if we look in there, we see that yeah, it's only got that analytical attribute was changed event. And that message is actually not being consumed out of the queue because I put that delete action inside of my if. So, I'm actually just going to drag and drop, move that down to the lower section here. So that even if the entity created event isn't, so we can purge that analytical attribute event out of our queue.
Chris Detzel (42:11):
It's a quick question, I think. So you've got folks trying this out on the fly, so that's kind of cool. But Max is getting a 401 from a get and then puts the 361 Reltio.com status. Do you need to be an admin? He's a tenant admin for all environments.
Dan Gage (42:34):
[inaudible 00:42:36] Responded to quite a thing. I did too. If you wanted to find your workflow URL, you can use the status API to get that back.
Dan Gage (42:48):
Yeah, that's correct. Yeah-
Max the question I have for you is, did you authenticate first because you need to do that?
Chris Detzel (42:56):
And he did not. There we go.
Chris Detzel (42:59):
Okay, thanks, Gino.
Dan Gage (43:03):
Yeah, so I think that other event had probably timed out. And again, depending upon your queue, because it's been about 30 minutes here since we started and we created that, we may have a 30 minute automatic expiration on those queue events. So I'm just going to create one new product here. And we're going to go back, and this time, instead of running this in test mode, I'm actually going to back out and I'm actually going to start this recipe. Because this is where you're going to want to be is once you've debugged this, you've tested it, you can actually run it and it's just going to constantly set out there and it's going to keep running in the background looking for events. So right now it's checking for jobs, checking for jobs, and in a few seconds here, again, once it's spun up that node, allocated the resources connected to the queue, it'll see the events of that one new product.
And there you can see it found one job and now it's waiting again and it's going to keep checking. So if I come back over here and I now hit refresh. In the upper right hand corner here, you should see I've got a workflow notification. And I did not get a workflow notification. So something's a little bit off here. Let's stop that and let's go in and look. Ah, we did, we got a duplicate refresh process instance exists with intended. So it's saying that the workflow already existed. Oh, and this is probably for the Reltio Community item.
And there you go. You see in the upper right-hand corner we now have a workflow notification. And the new UI, it puts the workflows over here on the right-hand side. And you'll see that it's automatically assigned to me. I got that message, Reltio community is cool. I've got the priority. If I wanted to reassign this, actually there's nobody else on this tenant. This is my personal sandbox, so none of my colleagues. So if Geno had access to workflow on this, I could reassign this to him. So you'll see that I notice it here. I'm also going to see that event in my Reltio inbox. So there's that review, the Community webinar, there's the data, the information that was created on today's date, actually it's due by the 27th, it was created there today. And I can actually take the action from right here and say, "Yes, I've reviewed it." Or I can click into that item and again, see it in the Reltio UI.
And I will say that I've reviewed it. It will terminate that workflow and that item has now been reviewed. So I think that concludes kind of the session. There's obviously a little bit of debugging and testing. And the end result will be that you'll want to put more conditional logic around, is it just that you want to create a workflow every time an entity has been created? If you want to create a different workflow for different types of entities, I can actually come in here and I can add another variable that says entity type. And that entity type can come from my payload. And there's that type right there. So in this case, this is the workflow that I wanted to have for product. So I could say, if entity type equals product, then I can launch this workflow. And we actually let you copy this. So if you click these three little dots over here and choose copy, and then down here you have, instead of inserting you have the paste option.
And I could come in here and say, "Okay, well if it's an organization, then instead of launching the review workflow, I can trigger the review organization workflow." Now that workflow doesn't exist yet, but you could simply go into the Reltio Workflow Builder, build another more complex workflow. And you now have the ability that when entities are created, if it's a prod record, a product, it would go to one workflow. If it's organization type record, it can go to a different workflow. And again, as the person mentioned, you do have the ability in your queue to send the snapshots, which is going to be with the deltas, the before and after values. So you can look at the before and after value and see if the before value was one choice in the after value was a different. And I actually have a example of that. So if I exit out of this. Looking here, and as part of my practice, and I actually shared this with another customer, you see that I have the same process here listening to the queue, parsing the document, listening to the event changed.
And here I'm looking, if there was an attribute change, get the old and the new value. If the attribute is the product lifecycle phase and the old value was one and new value is two, then I'm going to launch this workflow. But if it's not a change to the lifecycle phase or if it's a change to the lifecycle phase and it's going from phase two to phase three, then this evaluation, this condition is going to evaluate false and it would not launch this workflow. We have to go into some other workflow. And you'll see here that again, in this process we have the entity change logic in this loop. I can collapse that down.
And then I have the entity created performing a different action. So this is where, again, depending upon your business needs through this low code environment, no code environment, you can just go in there and define your process to effectively do whatever you need to do. It's going to be reading more information. And I think I do that here. I don't in this one, but I have other ones where we go back into Reltio and we pull additional information and use that information to help guide towards which route we're going to take. And that also is going to depend upon your queue setup. If the queue is not providing you all of the details necessary, if you only want to take action if the crosswalk that was contributed is from Salesforce. Then you can go in here and you can loop through and find that. And I think I have an example of that as well, maybe not in this environment.
Chris Detzel (51:13):
And I do have a question, Dan, when you get to a stop point.
Dan Gage (51:14):
Sure, go ahead.
Chris Detzel (51:18):
I think it was answered, but let's have you answer it anyway. Is there a way, and Ashish, welcome back. Glad to have you again. Is there a way to control these recipes to avoid making millions of rest APIs from an automated recipe? So I know this is the same as doing it from any integration tool, but wondering if there's a way where a tenant admin can be warned of an unexpected behavior?
Dan Gage (51:42):
So, technically yes. So when you're building out your recipes, you'll notice here on the left-hand side you have your dashboard and I can look at how many tasks have run today. So, just in my testing today, I've actually used, you can see at the bottom of this little pop-up box, 47 tasks have been used. Well, the number of tasks that are being used by each process, that is metadata that is stored inside the Reltio Integration Hub. And we actually have recipes that we can share with you that make API calls into the Reltio Integration Hub to look at that metadata and look at the task utilization. And if you expect that in your environment that you're using, let's say 2,500 tasks per day, you could set up a recipe that monitors your task utilization.
And if that number ever exceeds what your expected threshold is in this scenario, greater than 2,500 tasks, you can send a workflow, you can send an SMS, an email. So you do get a wide variety of things that you can do. I can show you an example here where I could come in here and say, "I want to edit this recipe." And if I want to send an email, there's an email by Rocado. So again, you don't have to have a special server, it's going to come from mailer at Rocado. So I can just put firstname.lastname@example.org.
Chris Detzel (53:27):
I'm loving these community rocks and a bunch of stuff like that. I like it.
Dan Gage (53:31):
So yeah, whatever you put in here is just going to send that email. And then obviously, you could intermingle that with information about the message. So data can be merged in with that email. You also have the ability that if it's a high priority, we also support an SMS connector. So I can come in here and I can put in my phone number.
Chris Detzel (53:54):
Is that really your phone number, Dan?
Dan Gage (54:04):
It will get to me, yes, it's a forwarding number thanks to our good friends at Google. And you can see that's the format, the return value will come out. But sure enough, when this process runs, now I did not put any conditional logic here. So what's going to happen is every single time this process runs, I'm going to get an email and I'm going to get an SMS message.
And I think, no, I've cleared them out. I had to some cached on my phone here. But we are good to go. So, one of the other things that I would encourage you to also explore is air handling of course. Anytime you're going to be taking some action, one of the steps here, you can add a step and you see that it says air handling. And effectively is going to create a try catch block. So you can say, "Hey, I want to send that email. And if for some reason that email doesn't get sent, then I want to send it via SMS."
So again, it's just a matter of saying, "What steps do you want do that you want to be monitoring?" And if any of those steps fail, then it's going to jump down to that error handler and it's going to perform whatever your error action is. So it can be sending an email or an SMS, it can be logging to a various different system. So we do provide a fairly robust error handling, error trapping mechanism within this. And then you also have the full recipe life cycle management. The ability to go out and take those recipes and export them from a dev environment, import them into a production environment and carry over that life cycle. And that's also where it becomes very important that the connectivity information. So connecting to that queue, in your dev environment you're going to be connecting to a dev queue. But when you promote this recipe into your production environment, you'll want to use an alternate connection that connects to your production environment.
So you don't have to go back and rewrite the recipe, you just use an alternate connection object when you migrate that code into production. So again, there's many best practices around securing and encapsulating data. And then as the user mentioned earlier, being able to tie into a credential repository that's likely available. There's a wide variety of different adapters. And you see here in the community library. You can see all the many, many hundreds and thousands of adapters that are already built. Either by our partner Waikato, or in many cases they're built by the community themselves. But there is a full SDK. So if you have an environment that you're going to be integrating with frequently, you can actually build your own connector for.
Chris Detzel (57:05):
And I feel like you got a ton of bonus things in here, so really appreciate it. So one comment is from Ashish, and thank you. This is awesome. Control automation can be effective this way. So good to know. And if there are any other questions, please go to community.reltio.com. We hope that you really enjoyed this. So feel free to push this push comment or two notes section. Again, this will be recorded and posted by next week. And the last thing is, I am looking for guest speakers such as yourself, like Dan did, or we can do even a Q&A show. So if you're open to it, whether you're a customer and or a partner and employee, I would love to speak with you directly. But that's all the time we have today. Thank you everyone. Dan, thank you so much for promoting community into the Reltio Integration Hub workflow stuff today. That was fun. And then the last thing is, Michael did say, "Will you publish this sample recipes?"
Dan Gage (58:17):
Yeah, sure will.
Chris Detzel (58:20):
That's a good idea.
Dan Gage (58:21):
It will be in the sample workflow.
Chris Detzel (58:25):
Dan Gage (58:25):
Will make those available. So Chris, I'll provide those resources to you and you can find the best way to make them available.
Chris Detzel (58:31):
All right, well thank you, everyone. I'm going to stay on for a minute or two, but thanks, thank you all for coming in. It's great, Dan, that you're certified. I just now saw that for some reason, but. So thank you, everyone.
@Daniel Gage take a look at this message below. Is it possible to push this file to a post?
Hi @Christopher Detzel
The BPMN.xml file which is used in workflow modeller, can you please attach that file here.
@sai ram madhamshetty did you download this pdf? https://community.reltio.com/viewdocument/using-reltio-integration-hub-to-aut
Can you please provide that sample BPMN diagram as downloadable asset (in video it is mentioned but it is not present)
100 Marine Pkwy #275, Redwood City, CA 94065
+1 (855) 360-DATA+1 (855) 360-3282
© 2022 Reltio. All Rights Reserved
Site by eConverse MediaAlso of Interest: Master Data Management (MDM) Reltio Community Reltio Integration Hub Community, Data Management Community Matching, Merge, Survivorship Reltio Workflow Master Data Management (MDM)