Good afternoon, everyone, and welcome to NCQA's State Spotlight, Data Quality and Interoperability in Action, Lessons from Rhode Island. We're really pleased to have you with us today. A few housekeeping items as we get started and folks are still trickling in. At the end of today's presentation, we will have time for questions. Please use the Q and A function in Zoom. And you can enter those throughout the presentation, but we'll behold those questions until the end. And also, if you'd like to use the raise your hand function, we can unmute you, and you can ask a question live. Lastly, we will be recording and sharing the presentation to attendees after the webinar. Here's our agenda today. As you may have guessed from the title of the webinar, we will spend the bulk of the time today hearing and learning from the state of Rhode Island. Lastly, we'll have some time for question and answer towards the end of presentation. Across states, there is growing attention to how we move beyond simply collecting more data towards building an infrastructure that makes high quality clinical data usable for quality measurement, care delivery, and ultimately, better outcomes. States and health care organizations are investing heavily in interoperability, but exchanging more data is only valuable if that data can be trusted and used. So NCQA is helping to bridge that gap, building toward a future where high quality data flow seamlessly and can be put to work for quality improvement, accountability, and value based care. As a measure developer, we know better quality measurement depends on more than moving data. The data must be standardized, trustworthy, and fit for use in measuring quality. We're helping build that trust layer by bringing together interoperable standards, digital measurement, data quality guidelines, and validation so organizations can have greater confidence in the data they use. Ultimately, we want trusted data to flow once and support many purposes from quality measurement and care improvement to accountability and value based payment with less manual reconciliation and burden. And that's why we're especially excited about today's conversation. Rhode Island offers a very practical example of what this works looks like on the ground. We'll hear not just about the technology, but also about the policy, financing, legal agreements, partnerships, and provider support needed to make data exchange work in practice. Those are often the pieces that determine whether interoperability succeeds. But for now, we'd like to, first of learn a little bit about who's joining us today and where you are in your own interoperability journey. So we have a polling question, if you don't mind filling out and identifying your role, your organization's role so we can get an idea of the stakeholders in the room. Okay. Great. Looks like a majority of folks are from state Medicaid quality programs, but also a smattering of health IT and health plans as well. Next, how would you describe your interoperability journey? Okay. Great. Thank you very much for providing that context. Thank you so much for filling it out. And without further ado, I am pleased to welcome Liv Keaton, state health IT coordinator with Rhode Island's executive office of health and human services. Liv Keaton will take us through Rhode Island's approach, what's working, what's still evolving, and some of the lessons they've learned along the way. So over to you. Great. Hi, everyone. I'm just gonna get right into it if you wanna go to the next slide. My first slide's maybe mostly preaching to the choir, but I think it's worth level setting, going over, what what we're doing, why we're doing it. So as undoubtedly you all know, federal quality reporting requirements are just increasing both in complexity and volume. It it... And more and more, they're, you know, really syncing up with quality reporting at the health plan level. And what this ultimately means is that providers downstream are getting subjected to all of these different requirements. They're really just rolling downhill in various different ways and landing on the providers. And they're often... They're... Even if they are fulfilling them, it's very administratively burdensome. It can be costly, from a technical build side. And, obviously, it's very frustrating because a lot of times their perspective is like, why aren't you guys talking to each other and coordinating and figuring your stuff out and exchanging data? And so we saw that as an opportunity for the state to serve a coordinating role in creating centralized infrastructure, trying to meet the principles of reuse, as much as possible. And, this really began many years ago, actually, under a CMMI grant. Before the... Every... Anyone was really conceiving of digital quality measurement as something they were planning to do in the real world, this really originated from that original, complaint from providers about you're all asking us for the same thing in slightly different ways. Why can't you just standardize? So we've been working this for a long time, and it's really evolved into what it is now. Next slide, please. And so right now, what our goals are is to support providers in having only one outbound clinical interface that ultimately feeds all of the state's managed care organizations as well as the state itself as my... And to commercial plans, Medicare, etcetera, as much as possible, to repurpose that same data feed for care coordination across providers, because, really, the same data we're looking to do quality measurement on should be the same kinds of records they're doing clinical care based off of in an ideal world, and to satisfy quality reporting for multiple state programs. So when we look at the state level within the Medicaid agency, we're looking for fee for service. We're looking for accountable care organizations. Rhode Island has a CCBHC implementation, which is a certified community behavioral health clinic. Obviously, we have health homes. We have state directed payments. Like, it's just getting on and on and on. And a lot of times, the reporting for this kind of grows siloed, as the reporting requirements appear. So how can we have a more coordinated approach? My disclaimer, as we go through the rest of this, is that a lot of this is in progress. So I'm gonna talk about, what we're trying to do. Some of it we have already accomplished. Some of it is very much underway. So we don't have, like, a nice polished final product to show you, so to speak. But what what we do have is kind of, real talk. What are we actually doing? What's actually been successful? And what are we still working on? So next slide. So this is, like, the overall flowchart I'm gonna be going through today. This is essentially the data flow that we're we're moving towards. It's mostly in place so far. Long story short is that providers all submit data to our statewide health information exchange, and then that HIE forwards it to the various health plans and to Medicaid's contracted quality reporting system vendor, who's also here today to show you what that looks like. Separately, we, as Medicaid, have been con... Have contracted with what we've decided to call a provider support, vendor, which really provides technical assistance to the providers in this process. And they have this kind of, like, yellow box behind the whole thing because they are really serving the role of pulling all these different threads of information together and communicating back to the provider what they need to do and what actions they need to take to improve the data being submitted. And so I have this poll question to see what you are most interested in learning about because, frankly, I could talk about this all day. It's very complicated. There's a lot of layers to it, and I wanna make sure I'm spending time on the aspects of this that you're most interested in. So the first thing we'll do, is kind of show you the end product. Like, why are we going through all of this? What's the point of this? And then, it'll come back to me, and I will walk through kind of each step of this flowchart in terms of what policy finance and legal structures were needed to enable the data to actually flow. So I wanted to make sure there was kind of a... The context of, like, what's the point of all this work, before I get into all the details, but then we'll come back to this and and go over that. Okay. Sounds like HIE QRS state, is what's clear winner in this poll, so good for me to know. Thank you. And from here, I will, hand it to my friends at MedicaSoft to show you on the technical side, what is happening, where this is going, and how we're planning to use it. Thanks, Liv. Hi, everybody. My name is Paul Bradley. I'm with, MedicaSoft. We're the team developing the solution. Let me share my screen. Actually, my screen share is disabled. So can someone give me permission? Thank you. Hopefully, everybody can see my screen now. Okay. Yes. So I'll give a quick overview of of our solution Starting from the left, as as we've mentioned, we're receiving receiving both clinical and claims data. The clinical data that we're receiving is in the form of CCDs. These are dab certified CCDs coming from the Rhode Island HIE and and containing, you know, data... Clinical data from various providers in the state of Rhode Island. We are also receiving claims data, and these... This comes to us in the form of flat files from various payers, and this is sent to our platform via Rhode Island HHS. And so we have... As as most most folks are dealing with, we have data from multiple sources, different formats, different level... Levels of completeness and quality from a quite a variety of organizations. And as we know, this presents some challenges when it comes to, computing quality measures. And so this, I think as most folks know, is where FHIR comes into play. And so the NXT FHIR platform is is being used here to to import the data from the various sources in the in the various formats that we receive it and convert it into standard FHIR resources. Okay? And then once the data has been converted to FHIR, the clinical and claims data sits together in the FHIR database. So each patient has a longitudinal record comprised of both that clinical and claims data. And that's and that's the case whether the data comes from any, you know, any source, whether it's clinical or claimed. It's all joined together. And so this gives us a consistent model to work from. Now once the data is sitting in the FHIR database, it is accessible through two APIs. There's a FHIR API, and, an SQL API. So the FHIR API is a standard restful API, that, any consumers can hit and and query the data. We also provide the SQL API so that those who are more comfortable using SQL queries can extract the data using SQL, the same FHIR data using SQL for validation, research, reporting, and so on. So we give two two different options. Now the solution we're using to compute the measures is what we call NXT CQM. It is a quality measure engine, that runs on the FHIR data. And so because it's it's built on top of FHIR, we can we can compute a whole host of quality measures. We're focus... We'll focus here on this demo on HEDIS measures, but this can be used for for all kinds of HEDIS and non HEDIS measures, all built on top of FHIR. Okay? And then once we have the measures, we can construct reports, and those reports can be... Can take multiple forms for Rhode Island. We're using Power BI, and so, we'll we'll... Later in the demo here, we will show some of those Power BI dashboards and walk through the the the measurement flow. So... But one thing about the NCQM that I wanna start with is it's not just a a computation engine. There are also tools that we use, to help, validate and assess, the readiness of the data and the readiness to compute the measures on it. And so we're gonna... Going to start the demo with that. So let me switch over. So so, again, part of the CQM solution are some tools that we we have that we've constructed, again, for the purposes of analyzing the data. And there's there's four tools that we have available today... That we use today on the Rhode Island project. The first tool I'll mention and show is the... What we call just the important metrics tool. And this is just providing a quick high level view of the data. And so this is looking at the data, that we're importing... The characteristics of the data that we're importing into the system. So one view we have is, is essentially the the view of all the FHIR resources created by organization and by location. So we can see that, you know, these are the counts of FHIR resources created by the various locations. This is all coming from Care New England. So for this demo, we only have a couple 100 CCDs imported into this system. So what I'm showing you is not representative of all the data we're going to be working with. So just sort of take the... Take what I'm showing you as a grain of salt. It's not representative. It's just for demo purposes. But in any case, we can we can trace the providence of of all the data. So we can see all the different locations sending us data. And sometimes this is helpful just to do some initial assessments on the data. Okay? In... For a for a... I'll show this diagram, this real quick. So for somebody who... Where we've imported, you know, thousands or millions of messages and documents, that view would look something more like this. We're we're seeing multiple organizations and then the various locations within all of those organizations. But since we've only imported data from Care New England, you're only seeing that one organization here. But we also have some time series graphs where we show the, you know, the... When data was imported, whether whether it was imported successfully or skipped. If anything was skipped, we... You would see some some information around the skips. If we had any validation errors, you would see some information about what what errors occurred, over time. And then here, this is just another chart showing the number of FHIR resource created by resource type over time. So this is really for for for the team to understand, what... What's happening temporarily as we're importing data into the system. The data tracer tool, I'm not going to demonstrate that because this is this is production data in this PII, PHI, which I'm not going to show. But what I'll say about the data tracer tool is, essentially, it gives users the ability to see original messages, the FHIR resources created from those messages, and any of the logs associated with those messages. So it's like a single pane view that lets lets folks do some analysis to understand if if there's a particular field or a particular value that looks suspect, we can very quickly analyze that and trace it back to the original message and see it in the original message side by side with the FHIR data as well. Okay? We also have what we call a CQM readiness tool. What this tool does is it does an assessment, of the data against the measures. Now in this case, this is going to be HEDIS, but this can be used for non HEDIS as well. And we're measuring... You can see it down here. We're measuring completeness, standard coding, and value set hits. Okay? But we actually start the analysis by looking at the clinical dates. Now we... This will be extended to claims dates as well. But, since we only have clinical data here, we're going to focus on clinical dates. And this is important because the user gets to select, you know, what, you know, what the time time frame is that they are... That they wanna do the assessment on. And they also can specify multiple organizations. But as I mentioned, this... All this data is coming from Care New England, so this is the only organization in scope. They can also adjust some of the scoring scoring thresholds and then run it. So here we could see that for the I think it was about 207 CCDs that we imported. It looks... The data looks pretty good, but we could see that the condition, a lot of the conditions are missing, some key dates. The, onset date time and asserted date are missing. And likewise for procedures, the performed period start dates are missing, and you could see about 23, 24% of the data. So that... That's important or that's relevant because if those dates are missing, then they are essentially, you know, counting against us when we look at that particular date range. We don't know what we don't know what measurement year to put some of that data in if we're missing those clinical dates. Okay? So once we run, the computation or the the readiness tool, what it will do is provide completeness, standard coding, and value set score by measure. And so we could see here, for for those 200 or so CDAs... CCDs, some of the scoring that we see. And now from here, we can drill in. So if we're looking at, say, for example, colorectal screening, we can click on this and see, okay. Well, how did we arrive at 70% for completeness, 92% for standard coding, and 3% for value hits? And what this will show is, all the different, FHIR pass or, you know, the FHIR data that's used to compute that score. And so we could see for completeness where data might be missing. Now because we didn't import any, any claims data, you could see that, you know, none of the claims data, is is present, which... So this is to be expected in this particular case. But we can scroll down, and we can see for standard coding. As an example, we can click into that and see what's happening there. Why, why, you know, why did we end up with a score of 91%, 91.2%? And we can see some of the results from there. So we could see where the proper codes... The proper code systems were were found and where improper or unexpect... I shouldn't say unexpected, but nonstandard code systems, or non, code systems that can't be used for the HIE does measures you're found. And so this lets us, you know, kinda drill in and see this in a little bit more detail, and then we go even further in and query, query the actual data and pull back some of the resources to analyze it. And this is where the data tracer comes in. We can trace that all the way back. We can pull that resource and trace it all the way back to the incoming message and do some additional analysis. So that's the, that's one of the tools that we have for the CQM readiness. We also have a a a picky scorecard. And, essentially, what this is... It's... This is doing something similar, although it... What it does is it operates, it operates at the organization level and provides a scorecard, across the organization. And similar to the CQM measure, CQM, readiness scorecard, you have the ability to drill in and see specific issues with with availability, accuracy, conformance, and plausibility. And so we... Again, you can drill in and see particular issues and and coding issues. So we could see in here, there's a... There's actually a mistake with the the units of measure, and so we're able to detect that and, drill into that and then, build reports off of that. Okay. So I know that was kind of, quick, but, we just wanted to give you a sense of how we do some of the assessments in tandem with the the building of the measures. And so with that, I am going to pass it to Jack. Jack, I will show the slide. So you can speak to the slide and then take control of the screen when you're ready, Jack. Alright. Sounds good. Hello, everyone. Thanks, Paul. I'm Jack. I'm a data engineer at Medixoft working on our quality measure solution. So you've just seen how the data comes into our system and the the readiness work we've done around the measures. So I'm going to pick up at the point where that data is ready for computation, and show how it gets from the data tables into the final reports. So we're starting with the health home measure set. The reports currently use test data, including NCQA sample data. The production data load isn't complete yet, so the results I'm showing just demonstrate how the reports work, and they're not real Rhode Island results. But this diagram on screen shows the process at a high level. So we start with FHIR data and Redshift. Our technology stack includes, Python and DBT, which stands for data build tool. For NCQA measures, CQL is our source of truth, and then our, CQM framework translates that logic into SQL, and DBT runs the shared and measure specific models against the data. The shared models prepare information that multiple measures can use, so things like service events, coverage, and patient characteristics. And then each measure applies its own rules to determine, who qualifies and whether any exclusions apply, and who meets the numerator. The framework automates and reuses much of the development and testing process, so that helps us verify and certify, measures more efficiently and adapt to, you know, new measurement years with less manual work. We use the same overall approach for non NCQA measures. Our synthetic data tools create test cases, and we develop, expected results and score keys to check the outputs. The detailed results, which you can see in the middle box, show whether each member or episode met the required criteria. We compare our results with the NCQA's expected results from the test materials, and we also have a Power BI validation page that compares the rates calculated from our pipeline with the rates in the reports. Then once all the results are checked, we prepare the reporting tables for Power BI, and those tables support the rates, counts, and filters, in our final reports. So I'm going to take over the screen share now and show you guys Power BI. Alright. So do you guys see? Yes. Okay. So, this is the landing page for the health home reports, currently available here, but, it doesn't include every measure in the full, health home set yet. But I will, start with IET. So IET measures initiation and engagement of substance use disorder treatment. This is an episode based measure, so the counting unit is a treatment episode rather than simply one group member. And if user wants more information about the measure, we have, an extended description in the tooltip here. But at the top, we have filters for cohort, and geography along with ZIP code, some demographics, and the payer. The cards below summarize initiation engagement for all cohorts and then for each product line, and the two rates show whether treatment starts for a patient and whether people continue into, engagement with that treatment. If we want to see just the opioid use disorder results, we can select that cohort at the top. The rest of the the rest of the page updates to, match that selection, and I can reset that filter so we're back into the full view. The next section shows rates by race and ethnicity. These stratifications are included in the NCQA IET specifications, and the charts, show both initiation and engagement for each category. Then these buttons up top let us focus, the stratified rates on one product line. So I just selected Medicaid briefly if we wanna drill down to see just those patients, and that changes the results. So I'm just gonna reset that. But further down, we can see rates by age group and substance use disorder cohort. These age groups and cohorts are also part of the, IEG specs. And the cohort detail table just puts, the eligible episodes and the initiation and engagement counts, next to their rates, which just makes it easier to compare the cohorts and see what's behind each percentage. And below that is a supplemental section with additional, views by payer sex and geography. The CMS QMR guidance includes sex and geography as stratification categories, and this section lets us explore those results. So the payer table on the left shows the eligible episodes and initiation and engagement counts for each pair grouping. And then the chart below just gives us another way to, compare those rates across payers. And we can also we can also compare the, results by sex and between urban and rural areas as you can see here. And then at the very bottom shows the initiation rate by ZIP code area. So the darker blue areas have higher initiation rates, and the, lighter blue areas have lower rates. So for example, 02879 is darker blue with an initiation rate of 84.6%, whereas 02, 817 is lighter blue with 65.9%. Then the tool tip, shows more detail for each area, including the number of eligible episodes, the rate, and whether the, area is classified as urban or rural. And now when I click engagement, the map updates to show the engagement rates by ZIP code area. I can hover over the same areas to compare their engagement, results as well. K. So that's all I had to show for the IET report page. I'll quickly switch over to, COL, which is the measure for colorectal cancer screening. So this measure page just gives a second example, with a a different measure definition and counting unit. And unlike IET, COL has eligible members rather than episodes and focuses on whether those members received one of the colorectal cancer screenings accounted by the, specification. So like IET, the filters at the top let us explore results across different categories. The cards show the overall screening rate and rates by product line along with the number of members, screened versus those who are eligible. You also noticed that COL uses just one color for its screening rate, while IET use... Uses the two colors to distinguish between, initiation and engagement. And then moving down, we see the screen rates by race and ethnicity, followed by rates by age group. Then the age group detail table just shows the eligible and screen counts alongside each rate. There's there's also a socioeconomic status breakdown for Medicare members. Then finally, the supplemental section just gives us additional ways to explore the results by payer, sex, and geography. And as with IET, the map gives us the ability to see more granular results for the measure across, different ZIP codes. As we built these reports, we've also been, considering accessibility. So we've added alt text, set a logical keyboard, tab order, and checked the color contrast as well. So, yeah, those are two examples of how the same overall pipeline supports the different measures. We start with the standardized data, apply the measure specific rules in, SQL, validate the results, and then, prepare the reporting tables for Power BI. So, yeah, I'll hand it back to you now. That's it for me. Thank you, guys. So this is, as, Amy gets the slides back up. This is, the dream or at least my dream, is to have all of the data in the system. As, Paul and Jack were just saying, we're kind of mid data load at the moment, so you've kind of caught us at a bit of an awkward time. But, we are we are close to having all of the data. So, how do we get there? What what lets any of the data flow in the first place? I mean, obviously, the data starts with the providers. So the first thing that needs to happen is they need motivation. They need reason to send the data. They have a lot of competing demands. So policies... Policy wise, right now, we've kind of started this project within two specific programs that OHHS has stronger authority, like, more direct authority over, I would say, in that in Rhode Island, UHHS sets the certification standards for both our Medicaid ACO and our CCBHC programs. So we started by making it a a certification requirement in each of those programs just to submit data to the HIE. We're just starting very simple. Just just submit your data. Just just get anything flowing at all. Now as we move ahead with actually having the data flow, we're considering some bigger options, to cover gaps like labs or imaging, and we might put that in the provider enrollment requirements for Medicaid enrollment. Importantly, Rhode Island's HIE model is that we have a single statewide HIE. It's been established as a state designated entity in Rhode Island law, And OHHS, me personally, also oversees that contract and that designation. So we're really able to serve as a central coordinator for all of the activity happening at the various vendors. Next slide. Great. So money. Common feedback from providers is that's great, but, my EHR vendor is gonna charge me for this. I would say the average cost I've seen is anywhere from, like, $1,000 to $40,000, to do the interface with some ongoing costs represented. I would say the two methods I found to successfully kind of navigate costs for providers here. The ongoing one is to build it into your alternative payment model cost structure. So our AE and CCBHC rate... Sorry. AE is the Rhode Island Medicaid a... ACO acronym for some reason. So our ACO and CCBHC rate setting, processes incorporate technology costs, and so we encourage them to include their costs for submitting data to the HIE. We also keep our eyes open for any onetime funding opportunities such as the rural health transformation program, and we will have some funds available to support a a variety of health IT activities for providers, but in... Inclusive of any costs to establish HIE connections. So, essentially, we try to pair, the requirement with a funding opportunity. So it's kind of like, yes. You do have to do this, but we'll cover your costs. Next slide. Alright. Legal framework. So we do have to do quite a bit of work on the HIE participation agreement. We have a really clear outline in that agreement of allowable use cases, I think, is the terminology we ended up using. And so everyone's kind of on the same level playing field for how the data is allowed to be used. And this was an important task to take on instead of just being like, oh, we'll just let it sort itself out, you know, through normal standard agreements because we kind of did the work upfront to reconcile HIPAA, 42 CFR part two, as well as Rhode Island state law so that it's really clear exactly how the data and the HIE can be used by whom and for what purpose. So anyone who's signing on to the HIE signs a standard participation agreement that does state that their data will be used for quality reporting, and sent to health plans on their own members. And that's actually supported in Rhode Island law as well to some extent, but we really and we really made it much more precise, within the participation agreement. And the HIE is the, entity that conducts patient matching. So health plans submit their member lists to the HIE. I'm getting ahead of myself a little bit. But, essentially, this is an assurance to providers that plans are receiving data on their own members and not their whole population from the provider side. Next slide. So HIE to health plans. Our HIE operates on a consent model called opt out to disclose, which means that patients can opt out of their records being disclosed to other providers, but our enabling statute calls out that quality reporting is an allowable use even if the patient has opted out of disclosures to other providers. This is actually a fairly recent, statutory change in Rhode Island as of just a few years ago. So this was a big shift for us and, was very exciting. Obviously, opened up a lot of opportunities that we've been pouncing on. And I will state, though, that this did need a technical build to support it because I think we were one of the earlier states... Or at least we were the first state to ask CRISP shared services to support retaining opted out records so that they could be reused for other purposes. They're also allowable for, public health reporting. So, really, the two, main uses are public health reporting and quality reporting, even if someone has opted out. Next slide. So our HIE operates on a voluntary multi payer financing model. And this year, going into next year, I guess, really, this next year is the first year I'm kind of incorporating all of the data aggregator validation costs and NCQA costs, into the overall HIE operations. So if you are a payer that is a member, basically a paying participant of our HIE, and really all of our commercial plans are, then you kind of get access to all of the outbound quality data, that, you know, Paul and Jack were just talking about. So they're, you know, they're working on behalf of the state Medicaid agency, but the other health plans also get to benefit in their own right from the work we're doing as long as they financially contribute to the underlying infrastructure that facilitates the clinical data flow and DAP certification. So OHHS directly contracts with the HIE administrator, and our financial contribution covers all of the Medicaid beneficiaries. And then commercial and Medicare lines of business separately have contracts with the HIE administrator called the RIO, at... On the exact same rate as OHHS does. And we rebase this rate every year based on actual costs. Next slide. Alright. So, the health plan signed the exact same agreement as the providers do with all the same allowable use case list. So we're all on the same page. And there is, however, an addendum that specifies exactly which kinds of entities, which, you know, participant types, pay in, and it clarifies the rate and what work is covered through the rate and what isn't. So if you want anything special or custom, it's not covered. But we have kind of a baseline standard set of services through the HIE that is covered. And, starting next year, that incorporates, outbound DAP CCDs. Next slide. So this is the one everyone was most interested in, so I'll spend a little more time here. This is the relationship between the state, which is the state Medicaid agency in this case, the HIE, and our quality reporting system. And policy wise, this took quite a bit of lead up time. Like, I was having conversations about this in 2024, about what this would look like, and we really did a lot of internal coordination. The first thing we did was kind of a landscape assessment in which I got some project management support, and we ran around really collecting every single place within Medicaid that some kind of quality reporting was happening. And I discovered, a lot of places where quality reporting is happening often in, various, ways. So that also incorporated what are you reporting, what gaps do you see, you know, where are we out of compliance with CMS? Because I don't know about you, but we have a few places we were not fully in compliance, and we've been getting a waiver or, you know, some kind of dispensation for a while, to to work on coming into compliance. And so that really helped inform us in terms of what does this work need to support, what should we prioritize. And it was also really important as a product of that to be clear on what is in scope for, in my case, you know, MedicalSoft, the quality reporting system vendor versus what are our MCOs still obligated to do, and what work will our internal analytics team still be doing. And it really takes a lot of time, thoughtfulness to pull all of these people together, document what's happening, and and make sure we're on the same page. And because, you know, this is slow, procurement is slow, technology is slow, budgets are slow, I have to say this takes a lot of ongoing continuous communication internally because, like I said, I started these conversations in 2024, and now I'm like, hey. It's time for us to get the data, and some people may not recall that clearly what we talked about in 2024. So I strongly encourage, like, on the policy side of the state participation. Like, wherever the state is directly participating, you need to have a lot of ongoing clear communication about what the goals of the project are, why you're sharing what you're sharing, and what program requirements, specifically CMS requirements, you're going to be fulfilling. Next slide. Legal, so OHHS has direct contracts in this case with both Crisp shared services and MedicaSoft, and, in order to share data with them. So as, you know, Paul was saying, we're sharing Medicaid claims with MedicaSoft, and we're sharing the Medicaid enrollment roster to the health information exchange because the HIE needs to identify who is a Medicaid beneficiary in order to send the provider's clinical data over to the quality reporting system. And so we have to send data both places, and so both contracts need to be extremely clear and precise around what data we're providing and for what purpose so that we can establish that this is an allowable use, and we're being appropriate, you know, administrators of our beneficiaries' data. And on top of which, it was our preference internally that each contract have an associated data use agreement. So even beyond the contract terms, we have an associated DUA that really specifies these are the exact data elements we're sending you, and this is how they're to be used and for no other purpose. Next slide. Alright. So finance wise, we do use an advanced planning document, to receive 90% or 75% federal financial participation. We appropriate the state match in the state budget every year. We've been doing that for the HIE for a very long time, so it was for the Medicaid contribution to our multi payer model. So in this case, it was a matter of expanding that scope in that budget. And, actually, as a little side note, the the the important rationale to the state legislature in getting this additional funding approved is that it actually represents an overall cost savings to the state because, otherwise, our managed care organizations need to go through this process separately. So we have three MCOs. All three of them need to separately do interfaces with the providers, do their own chart review or dev certification, do their own quality metrics. You know? Like, they're all gonna be doing this if we don't do it. So we can either pay three different MCOs to do it three times, or we can do it once. And that was a really important rationale, to justify the increase in the state budget to cover this work. And because we're able to get $90.10 for this Medicaid portion of it, it really wasn't a substantial increase, and I was able to get it through. And then over time, it does represent some savings. So we've been able to demonstrate that. However, just as a little side note, the model we have in Rhode Island, essentially, the other plans kind of entrust EOHHS with oversight of the HIE and its operations. So for formality's sake, our actual contracted budget has, you know, a column for the Medicaid contribution and then a column that reflects the other payer's contribution. And the HIE administrator, the RIO, takes in all of the other payer contributions. And when we at Medicaid, OHHS, approve an invoice, they kind of pay themselves out of the other payer's portion. If that's, like, convoluted and complicated, you can feel free to ask questions about it. This is the way I've kind of reconciled CMS cost allocation requirements alongside our health plans preference for the way the system works. This kind of arose out of long ago and continuing into today. No one really wanted the state to have direct management of these finances. There's always a risk that the state legislature will scoop any unspent money if there's a tough budget year. And so health plans were really not comfortable with contributing the money directly to the state. So we came up with this kind of, like, contracted administrator role for the regional health information organization, and that's how that works. Next slide. Okay. So the last kind of component of this, we didn't originally have a provider support vendor. And let me tell you, it is necessary. Because, basically, the long story short is your technical vendors will find all these places there are problems. Like, Paul was looking at, like, real data from Care New England. I already found, like, four different things that Care New England's doing wrong when we were looking at that. Right? But let me tell you, Paul cannot tell Care New England what they're doing wrong because they won't care if it comes from him. Like, I went through years of my tech vendors telling providers where they're finding issues in their data, and it did not translate into any kind of meaningful action in changing the data. So I realized that I really needed a TA provider who was really comfortable and experienced working with the providers, really good at translating technical stuff or, you know, kind of detailed stuff into what it means for you, what do you have to actually do. Because there is a cognitive load associated with looking at all these little data readouts and then deciding what do I actually have to do about this, Whose job is it? Where does it live? Is it the EHR vendor? Is it me? Do I have to train my staff? Do I have to configure something in my EHR? That's a whole process that has to happen. And so we contracted with a local nonprofit that's been a longtime partner for OHHS in primary care transformation and value based care, and they provide technical assistance. They're the ones who actually look at all those data readouts you were seeing earlier and translate that back to the practice in saying, hey. You definitely have a LOIT code mapping problem because it's showing up consistently across all of these readouts, and this is exactly what you need to do to fix it. Again, a lot of this is aspirational, kind of a work in progress. But, ultimately, the the the crux of this is, this is about data confidence. Because what DAV does is it looks for fidelity. It looks for is the data in your... Is is what you're seeing in your EHR as a clinical user matching exactly what we sent to the health plans. Is this the same thing? Are the health plans judging your performance based on the actual clinical record that you use to, you know, dispense care? That's not the same thing as being useful for the quality measure. It could be totally but everyone's recording the wrong diagnosis code. Like, I'm accurately giving you the wrong diagnosis code. Right? So I engage with the provider support vendor to do data confidence assessments. Is your performance accurately reflecting the actual clinical care you're providing? And that resonates with providers a lot more than talking about data quality. Like, they get really defensive if you talk about data quality. They put a lot of work into their data quality. Right? But if you say, hey. I just wanna make sure you're getting credit for the actual work you're doing because the HIE does measures look for specific codes, and you might not be using those specific codes. Right? So next slide. Hey, Liv. Can I interrupt briefly? There's a question in the in the q and a Oh, sure. That might be pertinent to what you were just talking about. What does your internal team structure look like to support all of this? So, it's actually... Let's see. There's actually three of us and, a lot of collaboration from other people. But in terms of staff, let's see. If this is probably, like, 30 to 40% of my FTE, and I kind of provide, like, executive sponsors... Sponsorship and oversight. I have a full time project manager that reports to me who works on both the HIE and the quality reporting aspects. And then I have contract administrator who she probably spends a third of her time, managing contract level invoicing and so on and so forth. And then we work very closely with, the Department of Health as well as some internal Medicaid data folks. So this... That's not, like, the full scale of staff support needed for this. But in terms of, like, dedicated time, I would say it's maybe, like, two FTEs in total, is what we do. And that's largely because we've delegated a lot of this to the, the RIO, really the administrator for the HIE. So on top of which, this provider support contract is very important because they're really the ones... That's the Care Transformation Collaborative is who we work with. They're really the ones communicating back to the practices and the ACOs, and in some cases, the health plans about what are we seeing in the data, where do we have confidence, where do we have concerns. And they coordinate peer learning, individual deep dive technical assistance, as as well as these kind of, like, lighter touch groups. So this has been a really key aspect of making this work because, otherwise, let me tell you, before I had this contract, I was spending, like, a truly absurd amount of my time talking to individual practices on an ongoing basis about their integration status, their connectivity, what quality measures they were, you know, being subject to, so on and so forth. And then there's a whole side of this I haven't even touched on around who picks the quality measures for each program year, you know, how are those specifications chosen, how are they aligned across the various programs and lines of business we have happening in Rhode Island. So dedicated support is like two FTEs, but I would say, we kind of have our little tentacles out touching, like, many, many, many aspects, of the of of the state. And then I think the next slide is just my lessons learned. Yes. Okay. So you might be getting a sense for what I've been saying, but this takes a really long time, and it's very incremental. Everyone's super nervous about this. This is, a ton of data we're talking about. We're changing things. You know? Like, we're judging people, essentially, you know, on performance, on data quality. There's cybersecurity risk aspects. There's the legal agreements. There's a... It it takes a long time. Do not be discouraged by how long it takes, and it is very incremental. Like, you need to be really clear that this needs to be baby steps. So, like, my first requirement to the providers, just submit your data. My next requirement next year is gonna be, okay. We're gonna pick... We're we're picking a couple measures that we think won't take a big hit if we totally switch to electronic quality reporting in your performance. Like, your performance won't be majorly impacted because they're mostly claims based measures, and we're gonna move them to being fully digital. And we're just gonna, year by year, do one measure, do one measure. You know? So everyone has time to kind of work through each process and say, oh my god. Now we're doing depression screening, and I have to go make that all my depression screening results are dropping in my CCDs in the right place, which is what we're doing right now. Second point, you need to focus on relationships, not technology. The technology is solved. Like, I don't know how many times people have heard about FHIR, and then you ask people if they've actually done FHIR, and their answer is like, no. No one's really agreed to do that yet. Right? The the FHIR is fine. Like, the technology is there, but the stakeholder engagement has to come first. You have to really get everyone strategically pointing in the same direction, on the same page, agreeing with the intent of what you're trying to do before you start figuring out the technology. Like, don't get focused on that. And then lastly, this is really messy data. This is clinical practice data that we're trying to use for, like, outcomes monitoring. We're we're kind of, like, taking something that's lived for decades and trying to make it suit another purpose. And so it's it's just a constant task. You need to be super clear with your providers. This is something they're going to be working on forever. They need to review their CCEs, their data every single year. Every year. It's it's, like, part of their general quality improvement processes. They they go through, you know, Fidelity to evidence based practice. They also need to go through Fidelity to data reporting practices. Like, it needs to really become part of their ongoing operations and for you and your vendors. This is not set it and forget it. This is, like, a whole new process you need to take on for essentially as long as you're doing electronic quality reporting. Like, as my last comment says, once you get the data flowing, you just start it. Because now you can actually look at the data, you can pull out all of these pieces of information. Like, I'll just give one last example because I know we're coming up to time. But, we we got all the data flowing. All of our providers were telling us they were doing fantastic on their, quality measure for eye exams for patients with diabetes. They're doing wonderful with their self reported data. You know, we look at all their data. We run the HEDIS measure version. Terrible. Everyone's performance was terrible. We're like, why is there so much mismatch? Why do the providers think they're doing fantastic? But when we run their data through the actual measure, it looks terrible. Well, they were running the CMS version of the measure for one thing, but, also, they had not gone through the trouble of ensuring that when they put the eye exam report results into their system, they ensured it was labeled as conducted by an eye care professional. And guess what? The HIE does measure specifications as it has to actually get done by an eye care professional. They they were getting them done by an eye care professional, but they weren't coding it in their system as flagging it as being done by an eye care professional, so no one was getting credit. And so that's what I'm talking about when I say it's not really... Like, it's it's it's it's not really about the data flowing. It's about assessing the usefulness of the data for specific measures and the confidence you have that the measure performance reflects the care they're delivering. Because in this case, the providers will get all up in arms because they're like, I am getting everyone their eye exams. And it's like, okay. I know you are, but I need you to go in and make this change so that it actually drops the code that says it was done by an eye care professional. And and that takes so much coordination between all the various vendors, you know, and, communicating that back to the provider and so on and so forth. So this really is an ongoing process. It really is about relationships. It is about being patient and and being super clear, with everyone on what the the purpose of this data sharing is, and keeping everyone focused on that. So that's what I have for today. I will let us, in our last few minutes, take any questions. Thanks, Liv, for such a comprehensive overview. That was a lot, and I'm sure folks are still absorbing. We did get some questions about sharing slides. So I know folks will be eager to see, some of this, and they shared that there's a lot of helpful information in here. So if you do have a question, we have a few minutes to take a few questions. Please do enter them into the q and a function in Zoom, or you can raise your hand, and we can unmute you to ask your question live. I see one in the chat about what's been the biggest unexpected benefit from this work. It's a great question. It's, like, weirdly kind of brought us all closer together in a certain way. It it... It's one of the few places that I would say state provider and payer interests all just, like, perfectly align. Like, we are all on the same side here. Right? Like, we all want to lower administrative burden. We all want the data to be good, and we all want everyone's performance to be good. So in a weird way, it it does kind of help, you know, take down some of those bear... Some of those, like, walls between those groups. On the other hand, as the state entity, it's very clarifying sometimes because there's a lot of pointing fingers between MCOs and ACOs and, you know, there's a lot of, like, oh, they're not doing this right. It's because that data didn't come in. Well, guess what? If the state is operating the system directly, I know exactly where the problem is. Right? Like like, when I run reports, I can see exactly, like like, if it's because the provider isn't sending that data or because the MCO isn't ingesting it correctly. And so it really gives you a level of transparency and oversight of the entire process and relationship that is just not possible when you're kind of delegating all of it to the MCOs. And we have something like 90% managed care membership for our Medicaid beneficiaries in Rhode Island. So we do delegate a lot to the MCOs, and, it's it's really allowed us to to have much more clarity into that. But, also, getting all this data in one place is going to also let us do true population wide measurement. Because right now, what we're doing is we're getting results from each MCO. We're trying to cobble together some fee for service results. We have some programs like the CCBHCs where they do their measure results out of their EHR and their population. And it's like... I don't know. It doesn't really feel very comparable to me versus when we have them all in the system, we'll be able to say, okay. Give me the a one c measure results on the whole Medicaid population and then stratify it by race and ethnicity. And we've never been able to do that before because the state and at least in Rhode Island, we don't wanna take in all that clinical data ourselves. Right? So this really is gonna set up both the relationships to get us kind of all working together to get everything going, but also the transparency and oversight, to identify where problems are and who has ownership for resolving them and and then let us really see population wide what some of these results are. Thanks, Liv. I think that's really important, the shared goal and kind of reeducating and reinstating that kind of goal, I think, is also very important in your work. Alright. I think that we don't have any more questions. But as we close, we would love to continue to engage with states and organizations on the call as you advance your interoperability goal. So please make sure to answer this poll, and maybe our next state spotlight is listening in as we speak. So we're eager to connect with those who are interested. And we can leave that poll up. But because of time, I just wanna close also by saying that we do send out quarterly newsletters called public policy notes, which has state federal policy content as well as NCQA updates. If you aren't already, please consider scanning the QR code and subscribing. And then a final thank you, to Liv, her team at Rhode Island, the team at MedicaSoft, and all those who listened today. It was a really helpful look at what it takes to move interoperability from an aspiration into practice. We appreciate all the partners on this call, and those continuing with us on the journey. So thank you, and enjoy the rest of your afternoon. Thanks, everyone.