Reltio Integration Hub

 View Only

Reltio Integration Hub: Advarra Interview, Saved Money, Time and Resources

By Ayan Basu posted 09-20-2022 08:54

  
Reltio Integration Hub: Advarra Interview, Saved Money, Time and Resources


In this live interview with
Sajid Sayed, Senior Director, Data Science Platform at Advarra, Sajid explains how the Reltio Integration Hub enabled integration of the Reltio solution with Snowflake to make accurate, trusted data available to its data scientists.  During this interview, Sajid mentions that Advarra saved $100,000, Reltio Integration Hub was four times as fast, meaning the Reltio Integration Hub helped Advarra achieve quick time to value with rapid completion of its initial use cases.  Advarra didn't need specialists to go do this work.  Instead they were able to leverage their existing teams to build on Reltio Integration Hub.  Read their case study here: Turning data into valuable assets: Making clinical research safer, smarter, and faster.

 

Ayan Basu: Sajid, welcome!  To start, could you please do a quick introduction about yourself and Advarra, your company.  I'll also share a couple slides that can showcase how you guys use Reltio and then more specifically Reltio Integration Hub.

Sajid Sayed: Thank you so much, Ayan, for that introduction. I'm a senior director for Enterprise Data Science Platform at Advarra. Advarra is in the business of making clinical trials safer, smarter, and faster. We have two lines of business. We have a consulting business where we have an independent review board to manage the clinical trials. And then, there is also clinical trial management software that we build, that's a technology side of the house. Currently, I am in the data science space, and I've been with Advarra about two years now and I've been tasked to build this data science platform for Advarra.

 

Ayan Basu: Thank you! Can you talk a bit more about your data platform vision and then we'll jump into Reltio Integration Hub.

Sajid Sayed:  The vision for the data science platform is to turn data into assets. To do that, we were asked to build this data platform with the objective of creating reporting analytics, help create standalone products and data APIs that we could service out to our customers. To do that, we established these four steps to help us get there. First, we wanted to create a data right and governance body.   For that, we wanted data stewards and we wanted to create a data governance council. Now, Advarra grows through acquisitions. What we also wanted to do was unify Advarra data products.  Curate the data, profile it, cleanse it, master it, and then publish it.

 

Ayan Basu:  Great, can you help us understand how Reltio Integration Hub fits into this overall vision?

Sajid Sayed: Advarra has grown through acquisitions.  Some of the sources of its transactional system are multi-tenant, some are single tenant transactional databases. Our objective was to ingest the data, bring all that data into a raw zone. This is where we were cleansing the data, profiling the data, and after that, we transformed the data within Snowflake.  The Reltio Integration Hub was what allowed us to do this.

 

Ayan Basu: How did you first become familiar with Reltio Integration Hub at the beginning of this journey?

Sajid Sayed: The architecture diagram is an overview of how we started. We knew our sources of our data. We had to bring that data to our data cloud for which we had choose Snowflake as a platform and we had an ETL tool that was ELT2 that was bringing the data to this raw zone. Now, we had to do the last hop where we had to move the data to Reltio. Unfortunately, the ETL tools that we picked, and most of the ETL tools, were moving the data from the transaction system to the warehouse but not from the warehouse to the transactional system. So that was the problem that we wanted to solve for and any integration tool that we were looking into would have cost us about $100,000 to do that.  Also we would need specialist skills to work on it.  We reached out to Reltio with our problem statement and Reltio introduced us to RIH over here which was native, which came out of the box with our Reltio implementation, and that's how we were introduced to RIH.


Ayan Basu: It looks like Reltio Integration Hub was the solution that appealed to you early on. What was it about the tool itself that you felt like at that point in time would help you solve this problem? Were there specific aspects of it that really stood out to you?

Sajid Sayed: . I think a lot of it. First, it is native to Reltio. Our source of data was Snowflake, our destination was Reltio.  We were still in our early phase of our implementation, and we were part of a data science team, so we didn't want to purchase this integration solution to just solve for this niche problem that we have. We were able to get the data from all different sources. So RIH came native to Reltio so there was not a whole lot for us to do to enable that. It was a local environment, so again, not a steep learning curve. It came with pre-built connectors for Snowflake and Reltio so that made our job easier. I think those were our conditions, those were the earlier insights that really helped us make the decision.


Ayan Basu:  Let's say Reltio Integration Hub was not available to you. You said it would've cost you about $100,000 for an alternative solution. What would that alternative solution have looked like and how would you have gone down that route?

Sajid Sayed: We were looking at a lot of integration platforms to help us do that. The other problem with some of the integration platforms were that we would have to own and manage those systems. Now, we got Snowflake as a SaaS-based solution whereas we got Reltio which is SaaS-based. Now, just for this problem, having an integration platform and owning, managing and maintaining and the cost for it would just not work for us and we were trying to run away from that. We were not in an integration space. We were not creating a center of excellence around data integration. We are a data science team and we wanted to stay in our zone.

 

Ayan Basu: As you think about Reltio Integration Hub and the points that you mentioned, the fact that it's low code, not a steep learning curve, what did it look like initially for your teams to get used to the tool? How did they learn what to do early in the journey? Can you talk about what that initial experience was like for the development teams?

Sajid Sayed: I think thanks to our CSM team. First, they introduced us to the platform. They gave us an overview of the platform. Once we knew that, then they referred us to a documentation for Reltio. We heavily relied on that documentation to upskill ourselves. It's a local platform so it didn't take us a lot to figure things out. Just from a navigation perspective, we just needed to understand what it was. And then, whenever we were trying to build this, if we would run into trouble, we would either log a support or reach out to a CSM team. It was straightforward for us to do the implementation.

 

Ayan Basu: What did the implementation process look like and more specifically, would you be able to give the audience a comparison of coding up that entire solution, writing hundreds, thousands of lines of code versus using the Reltio Integration Hub and putting a solution like that together?

Sajid Sayed: I think just from an implementation perspective, the first thing  we had to do was make sure Reltio Integration Hub was enabled in our environment, their test production. Once that was there, we had to make sure of our connection endpoints. I think whatever work that we had to do was to ensure the Reltio connection came out of the box, we had to configure for Snowflake connection. Then I think one thing for us to be aware of was best practices so RIH will have its own best practices. We had to understand from an organization widely what were our internal best practices that we wanted to implement. And then from a DevOps perspective, how do we make sure that this integration moves between our dev test and production environment? These were some things that we had to be mindful about. Again, from a coding perspective, there was hardly any but what we still had to learn is what does looping mean, what does logging mean, and how to go about that. When we are handling a large amount of JSON files, how do we break it up? There are things that you need to know, not everything will happen magically but then we didn't have to code around all those.


Ayan Basu : You said it was about four times faster. What are the key things that accelerated that? Obviously, the coding aspect is one. What are the other things that contributed to the speed, do you think?

Sajid Sayed: Everything is UI based.  It's not a separate entity to Reltio.  What we had to do was understand the Reltio data model. We had to understand the destination data model but because it came in connected with Reltio, I think that part was very easy for us. There was no environment management. If we had this different environment, we had to make sure we are managing all these different environments which we didn't have to do so we have zero production support team for integration. So as part of that Reltio Integration Hub piece, we log. We need to understand our system from the entire data pipeline perspective so what we do is as part of that Reltio Integration Hub process, we log everything that's happening into the system and now, we have this central portal where we see this entire data transformation that's happening. And so, it just makes it easy. 

 

Ayan Basu:  How is Reltio Integration Hub performing in terms of latency, data loads, and any challenges and performance that you ran into? Can you talk about the performance testing?

Sajid Sayed: I don't know if I have the benchmarks that we did but I think what we did as part of our first implementation, everything was in batch mode. I think what we could do if we had to do this again and as we improve the process.  Now we can create real-time integrations with Reltio Integration Hub. That is performance that we have to be mindful about. But if you start doing this real-time, and like I said, we are getting the data from the source to this Snowflake area, we are doing a lot of transformation, there are certain data quality checks that we can do in flight.  I think that's where we are leading into. Our future states will really have that.

 

Ayan Basu: Even though you did batch mode and batch loading, and you talked about going real-time which is great, is there anything you encountered that would create any pause or concern in terms of performance or did you feel like the platform itself met your performance criteria?

Sajid Sayed :  Yes, it did meet the criteria but I think just something to be careful about, something that we are still in the learning phase.  Right now, when we are extracting data out of Reltio, you get this major JSON file. What we'll have to understand is how do we loop through that? How do we compartmentalize that and don't take one big file but a certain amount of data at one time, pass through it, and move it? There are certain things that we learn as part of the implementation, it's not performance or a drawback. It's something for us to learn and know how to use the platform.

 

Ayan Basu:  How much time did it take from the decision of using Reltio Integration Hub all the way to going live into production, what was that timeframe?

Sajid Sayed: The Reltio Integration Hub, it was four weeks’ worth of effort, that includes test and then production.  But everything else was not just linear because there were other program and project decisions that we had to make, but just from a dev effort, it took us four weeks to do that. Just a disclaimer, we don't have any complex transformation in that. It's a simple, straightforward pipeline because we had done major of the transformation within the Snowflake area, and this was just the last push that we were doing. As part of that last push, we discovered, hey, there is a lot that we can do in this platform.

 

Ayan Basu:  If you had to code the solution, obviously it would take much longer than four weeks. Is that an accurate statement?

Sajid Sayed: Oh yes. The other problem is then that knowledge is with that developer alone. Now, if we run into any issues with Reltio, it's very easy for somebody to look at it and figure out what's happening. But then we can also go to Reltio Integration Hub support forums and customer support and figure out if there are certain things. But we didn't want to get locked into whether IP is just with that developer, and we cannot use other people in our data engineering team to support the solution.

 

Ayan Basu: You did mention the fact that supporting the solution long-term maintainability, that's much easier with integration hub as opposed to separate infrastructure, separate integration, that is a whole IT development team that's dedicated to supporting that.

Sajid Sayed: Correct. From a data science team, we wanted to go to market really fast. As many, if you start having these different technologies, we need to have super specialists to support each of them and then DevOps process in each of them. If there is something with a larger memory, we need to increase that memory. And then, if there's a different data source, we need to create tunnels or whatever we need to get to that data source. So now, we have avoided all that by going to a SaaS platform which is natively integrated with these sources and comes with pre-built connectors for most of them.

 

Ayan Basu: Can you provide some insight into what the longer-term plan looks like with RIH? Obviously, you've had good experiences so far. As you think about one year down the road, two years down the road, can it be that central connectivity integration platform into your larger, broader ecosystem efforts?

Sajid Sayed: I think that's the area that we need to invest in.  Bring Reltio close to source as much as possible, move most of it, whatever we can, from batch to real-time and to connect all these different transactional applications that, like we said, we have inherited through growth. I think that is the biggest business value that we can get.

 

Ayan Basu: You mentioned one of the biggest appealing factors was the fact that the platform does come with a lot of pre-built out of the box connectors that you've already used, the Snowflake connector that's in there and there's others that you can look at. As you looked at these other platforms versus Reltio Integration Hub, what did that look like, the pre-built connectors out of the box? Was that a criteria for comparison as you looked at different systems?

Sajid Sayed: Definitely. We have a very heterogeneous environment. We have Oracle SQL server, postgres, so we have all of that. For us to make sure that there are connectors for... And then, different versions of Oracle, right? We have some of their pluggable databases in Oracle, so we wanted to make sure that we have connectors for all these environments, so I think that's very critical for us. We didn't want to create a single pipeline for one platform for each transaction system that we had.

Ayan Basu:  Just one stop shop for everything would be obviously the ideal way to go. Looking back on your integration hub journey and experience, what would you have done differently? What were the learnings? What would you recommend to the people that have joined this call that have maybe haven't used Reltio Integration Hub? Any advice or guidance you would have for them?

Sajid Sayed: I think we looked at RIH very late in our journey and maybe it was just released about that time too. We just started with our problem to solve. So we said, "Okay, I just want to connect from this source to this source," and that's about it.  But I think for anybody starting fresh, I would like to tell them that, take a step back, look at what RIH has to offer from a whole platform perspective. There are so many transformations that you could do in flight so see if you can use that.  There are so many jobs that you can schedule in there versus... Some of the challenges that we have like Snowflake is a great platform but when you start putting tasks and all that in there, it's just a little hard from a UI perspective to know which bad job is running where. There are a lot of those that you can shift, that RIH can do that for you.

 

Sajid Sayed: To sum it up, I think for us, especially from a Reltio perspective, we are thinking about integration, we don't have to go out and buy an integration platform. Even if you have an internal integration platform, you don't have to rely on that team to build this integration. You don't need a specialist data integration architect to help you with the integration. Think about these things, these are available to you to build integration. It will just make your life easier and help you get to market faster.

Relevant Content: 

#Reltiointegrationhub
#Featured
#Featured
#Blog
0 comments
4384 views

Permalink