Stream: IPS
Topic: IPS Maturity Model
Grahame Grieve (Dec 09 2021 at 06:46):
I'm on the JIC openForum. One very interesting idea that emerged - from Stephen Key - is the need for some convergence around an IPS specific Maturity Model that can be applied to the various blocks of the IPS (both established and proposed). I think that's a pretty good idea, and I think it would be a good thing to develop.
Grahame Grieve (Dec 09 2021 at 06:47):
with this in mind:
Grahame Grieve (Dec 09 2021 at 06:54):
- IPS-MM 1 The block is considered ready for testing, and there is a consistent model across various instantiations of the IPS
- IPS-MM 2 PLUS the block has been tested in exchange between at least three independently developed systems leveraging most of the elements (e.g. at least 80%) using semi-realistic data and scenarios. These interoperability results must have been reported to and accepted by the the JIC(?)
- IPS-MM 3 PLUS the block has been verified by the JIC as meeting the IPS Quality Guidelines; has been subject to a round of formal balloting in at least one SDO which has at least 10 distinct implementer comments recorded in the tracker drawn from at least 3 organizations resulting in at least one substantive change
- IPS-MM 4 PLUS the block has been tested across its scope, published in a formal publication by more than one SDO, and implemented in multiple prototype projects. As well, the JIC (?) agrees the block is sufficiently stable to require implementer consultation for subsequent non-backward compatible changes
- PIS-MM 5 the block has been published in two formal publication release cycles by two SDOs and has been implemented in at least 5 independent production systems in more than one country - supporting real clinical exchange
Grahame Grieve (Dec 09 2021 at 06:54):
there's a draft for people to think about/iterate on
Giorgio Cangioli (Dec 09 2021 at 07:04):
Hi Grahame, I support the Steve's idea , my main question is what this maturity model is expected to measure..
Giorgio Cangioli (Dec 09 2021 at 07:06):
is the technical capability to exchange data, is the appropriateness of the agreed content, both ?
Grahame Grieve (Dec 09 2021 at 07:06):
well, it communicates, more than it measures, and establishes common expectations across the community and helps ease the adoption of new blocks
Grahame Grieve (Dec 09 2021 at 07:07):
but I think - for IPS at least - it's about the appropriateness of the agreed content
Giorgio Cangioli (Dec 09 2021 at 07:09):
so can we think that there will be a generic IPS MM and then FMM that will indicate the maturity of the specific IPS FHIR profiles implementing that building block ? (even though for the time being there is no FMM for the components of a FHIR IG...)
Giorgio Cangioli (Dec 09 2021 at 07:11):
in that case a CDA template might be less mature than the correspondent FHIR profile with a well assigned more generic IPS block maturity ?
Grahame Grieve (Dec 09 2021 at 07:11):
... probably? We are certainly introducing FMM to IG artifacts. But I would think that in general, the FMM of a the FHIR IG would very closely shadow the IPS-MM - why would it be different?
Grahame Grieve (Dec 09 2021 at 07:13):
I guess that implies to me that the CDA representation (or the FHIR representation) might be in question, even though the content is not in question. It's hard for me to think that there's much question about the representation in FHIR or CDA? (ok, there is, in the question of what the profiles look like, so perhaps there's some interesting differences?)
Giorgio Cangioli (Dec 09 2021 at 07:22):
For sure I'd expect that the CDA representation will have a lower maturity than the FHIR one because of lower number of implementations and also of testing events (maybe only the IHE connectcathons) .... but there are not (or not yet) CDA template maturity levels :-)
Giorgio Cangioli (Dec 09 2021 at 07:25):
.. in any case these are details.. thank you for starting this thread on IPS MM
John Moehrke (Dec 09 2021 at 13:37):
I emotionally like the idea. But analytically I am worried that we are creating a mechanism that will slow progress, not help. I get that it is important for the more mature sections to be seen as more mature so as to encourage accelerated use. But once a section is declared immature, it may never recover from that stigma. What are the milestones needed to climb the ladder, what is the proof that the mature sections already did that climb. I fear that this will turn into governance overload.
Rob Hausam (Dec 09 2021 at 13:55):
I didn't join the Open Forum until after Steve's presentation, so I haven't yet seen it. But from this discussion I have a question. When we refer here to IPS "block", I assume that tracks with what we have been describing as the IPS "building block" approach - and the "block" is either a FHIR profile or a CDA template. And if so, in FHIR wouldn't this be the same as, or implemented by, assigning an FMM level to the FHIR profiles in the IPS IG? It's not obvious to me why there would be an additional or different IPS-MM?
Giorgio Cangioli (Dec 09 2021 at 14:51):
because IPS is not (in principle) bound to a specific representation (e.g. FHIR)
Giorgio Cangioli (Dec 09 2021 at 14:53):
FMM can rate the maturity of a specific FHIR IPS profile
Giorgio Cangioli (Dec 09 2021 at 14:55):
...having said that I'd expect that the maturity of an "IPS block" will be at the end measured referring to the FHIR IPS implementation ...
Rob Hausam (Dec 09 2021 at 16:03):
Giorgio's last comment I think fits exactly with what I was thinking.
Christof Gessner (Dec 09 2021 at 16:12):
Giorgio Cangioli said:
measured referring to the FHIR IPS implementation ...
I can read this as a proposal to attribute maturity to those who implemented (and are operating) such a service. Matches the idea of labeling "Hospitals on FHIR".
Rob Hausam (Dec 09 2021 at 16:51):
I should also add that the Focus Task Group (which I am part of) is proposing and working on a general SMM (standards maturity model) - which may be applicable.
Grahame Grieve (Dec 09 2021 at 19:05):
.having said that I'd expect that the maturity of an "IPS block" will be at the end measured referring to the FHIR IPS implementation
I generally agree with that, but there's more to the process than the FHIR IG, so I was abstracting the language
Grahame Grieve (Dec 09 2021 at 19:06):
I am worried that we are creating a mechanism that will slow progress
Well, Stephen Kay's concern was that there's no path to include developmental content in the PIS at this time, because people will push back against mixing experimental parts into the spec - which is what the maturity model is for
Rob Hausam (Dec 09 2021 at 19:20):
Reasonable points.
Peter Jordan (Dec 09 2021 at 19:59):
While I think that this is a good idea for the future, I think that there are more urgent considerations to discuss about the IPS right now. In particular, TTBOMK there is no known instance that validates, without errors or warnings, against the current IPS Bundle or Composition Profiles so we are hardly in the position to discuss the maturity of individual blocks. Then there is the R4 v R5 discussion - the IPS contains resources that range from FMM 0 to Normative in the R4 specification and at least one, MedicationStatement, that is slated to be removed in R5. If we are going to create a global standard for a patient summary then I believe it needs to be backwards-compatible with any future changes and I would be interested in hearing thoughts on how that might be achieved without tethering it to a specific version of FHIR.
Rob Hausam (Dec 09 2021 at 21:13):
Agree, Peter.
Grahame Grieve (Dec 10 2021 at 20:59):
there is no known instance that validates, without errors or warnings, against the current IPS Bundle or Composition Profiles
@Peter Jordan can we get to this place?
Peter Jordan (Dec 11 2021 at 00:13):
We can if the Profilers employ test-driven development and ensure that their work can be supported by working examples. The current set of warnings, are too opaque for implementers to diagnose and it's even possible that someone will have to step through the Validator Code in debug mode to pinpoint exactly where and what these (likely slicing) issues are...
"This element does not match any known slice defined in the profile http://hl7.org/fhir/uv/ips/StructureDefinition/Composition-uv-ips"
Rob Hausam (Dec 11 2021 at 01:31):
It's taking a while (too long) - but that analysis of the errors is exactly what I'm working on.
Grahame Grieve (Dec 11 2021 at 01:43):
This element does not match any known slice defined in the profile http://hl7.org/fhir/uv/ips/StructureDefinition/Composition-uv-ips
have you tried verbose mode so that the validator shows it's reasoning?
Jens Villadsen (Dec 11 2021 at 10:28):
If I was to prioritize the work effort I would put much focus on the R5 conversion
Peter Jordan (Dec 12 2021 at 00:38):
Grahame Grieve said:
This element does not match any known slice defined in the profile http://hl7.org/fhir/uv/ips/StructureDefinition/Composition-uv-ips
have you tried verbose mode so that the validator shows it's reasoning?
That doesn't tell me anything further, noting the following caveat for Verbose Mode in the Validator Documentation
"Note that the validator knows why a particular resource fails to meet the definition of a slice. What it can't know is whether you expected it to or not, so verbose mode has a low signal to noise ratio, providing obvious reasons for resources don't match slices."
Grahame Grieve (Dec 13 2021 at 05:29):
well, can we work on it? do you have an example for me to work on?
Peter Jordan (Dec 13 2021 at 09:05):
Maybe try one of the examples in the IG... http://build.fhir.org/ig/HL7/fhir-ips/downloads.html#examples
Rob Hausam (Dec 13 2021 at 10:54):
We'll need to have a discussion again on the IPS slicing for the 'code' (CodeableConcept) elements. We've generally concluded that our approach is useful, and the IG Publisher/Validator explicitly allows it (at least it has been) - even though it doesn't follow all of the "rules".
Peter Jordan (Dec 13 2021 at 22:04):
The warning messages would suggest that the problem is with the slicing on the Bundle entries, rather than code elements within those entries.
Last updated: Apr 12 2022 at 19:14 UTC
