Stream: IG creation
Topic: Requiring meta.profile be populated
Saul Kravitz (Sep 17 2020 at 21:46):
In the Plan-Net IG, we want to require the meta.profile field be populated and the _profile search parameter be supported for two reasons:
- Easier for clients, since the server attests that the data instances it is delivering conform to the Plan-Net Profiles, and its associated extensions/constraints.
- There are two profiles of USCore Organization -- Plannet-Network and Plannet-Organization. If one wants to retrieve only the Networks, which is a meaningful and reasonable query, the only way to differentiate is by constraining the search to _profile=Plannet-Network unless you traverse the relationships with other instances.
Two questions:
- Is this an appropriate and reasonable profile constraint? (If not, why have a meta.profile field?)
- If so, should the constraint on meta.profile include:
- only the Plan-Net profile, or
- the Plan-Net profile AND the underlying US-Core profile?
Thx,
Saul. @Robert Dieterle
Lloyd McKenzie (Sep 17 2020 at 22:07):
The problems with requiring declaration are:
- if a system is already producing instances that are otherwise conformant, you're requiring them to change code in order to conform
- in a general space, you might not have control over the client systems and have no way to make them declare the profile, but still want their data (so having your system rely on its presence is going to eventually cause grief)
- the declaration isn't a guarantee of accuracy. Once someone puts a fixed value profile declaration in their code, odds are at some point in the future they'll make a change that breaks compatibility but will leave the profile declaration in place - so having the profile doesn't remove the need (or the cost) of validating the instance anyhow
Lloyd McKenzie (Sep 17 2020 at 22:08):
You should never use profile as a way of searching for meaningful data. If you want to know whether something is a network or organization, that must be inferrable from the data in the instance if you strip the profile declarations away. If you can't, then your IG is non-conformant.
Lloyd McKenzie (Sep 17 2020 at 22:08):
I.e. you should be using an extension or some other data element to distinguish, not meta.profile
Saul Kravitz (Sep 17 2020 at 22:32):
So, if I can't require that someone populate meta.profile to declare conformance, you view it as an informational declaration of a data producer that is always optional?
Lloyd McKenzie (Sep 17 2020 at 22:42):
I'm not saying you can't require it, I'm saying it's not good practice - and it's certainly not something anyone should ever look at for semantic information. It must always be possible to wipe all profile declarations and then redetermine them solely by validating instances against all of the possible applicable profiles.
John Moehrke (Sep 17 2020 at 23:04):
The automated conformance testing that some reference implementations do when the .meta.profile is populated is not the only way to get conformance testing. You can have a server that forcefully requires conformance.
Mark Kramer (Sep 18 2020 at 14:16):
The trouble is the chicken-and-egg situation with respect to conformance testing. If you want to verify that a server conforms to US Core, one part of that involves testing that all labs conform to US Core Observation Lab. To do this, you query for all Observations, sort the Observations into two piles, labs and non-labs, and then run the validator on the labs. Simple! Except that....
How do you determine which Observations are supposed to be labs? Sorting them requires using the information in the Observation, which is exactly what you are testing. The server may have improperly populated Observation.category or used the wrong LOINC code (or didn't use LOINC at all), so the testing software cannot identify which resources are supposed to be labs.
If everything that the testing software identified as a lab passes validation, there still might be resources that should have conformed to US Core Lab, but were never identified as labs in the first place. So conformance question is undecidable.
This is a normal feature of black box testing. You can never really decide if the software you're testing is 100% correct via black box testing. What Saul suggests, and many have suggested before, doesn't change this. It only changes the scope of what you are testing, namely, if the server declares via meta.profile that "I intend this resource to conform to US Core Lab" then a test can easily decide if what the server thinks are labs do or do not conform to US Core.
Basically, conformance testing has limitations, and unless there is a out-of-band method of sorting out what is intended to be what, we're stuck.
Saul Kravitz (Sep 23 2020 at 21:56):
@Mark Kramer you appear to have the last word....
Grahame Grieve (Sep 23 2020 at 21:59):
no having the last word on this is on my to do list
Grahame Grieve (Sep 23 2020 at 21:59):
I won't have a chance until after the WGM
Mark Kramer (Sep 24 2020 at 15:35):
The FHIR documentation requires meta.profile to be populated if the server declares it a supported profile:
"To help a consumer find the correct set of reports for a use-case, a producer of resources also SHALL, for any profile declared in CapabilityStatement.rest.resource.supportedProfile:
- Mark resources with profile assertions documenting the profile(s) they conform to (this enables indexing by the profile)
- (if a server) support searching by the _profile parameter for the declared profiles
Beyond these requirements, a producer of resources SHOULD ensure that any resource instance that would reasonably be expected to conform to the declared profiles SHOULD be published in this form."
See https://www.hl7.org/fhir/profiling.html#CapabilityStatement.rest.resource.supportedProfile
Mark Kramer (Sep 24 2020 at 15:36):
Last word regained :laughing:
Grahame Grieve (Sep 24 2020 at 15:37):
... it's not over yet
Igor Sirkovich (Oct 07 2020 at 22:28):
@Lloyd McKenzie, going back to this discussion, I see that it's not a good practice to have meta.profile as a required element for client data submission and it's not appropriate to rely on searches by a "_profile" parameter (even though it's a standard parameter). However, I'm wondering whether it wouldn't make sense to require a read-only server (e.g. a CARIN BB or Plan Net server) to always declare meta.profile.
Lloyd McKenzie (Oct 08 2020 at 00:05):
You can certainly mandate declaring a profile
Lloyd McKenzie (Oct 08 2020 at 00:08):
However, that creates a cost for systems that might comply with the profile without having been custom designed for your IG. Is that something that's even a vague possibility for compliant systems?
Igor Sirkovich (Oct 08 2020 at 13:25):
Thank you Lloyd. I believe all the payers are implementing CARIN BB compliant servers and will implement the Plan Net ones next year to meet the new CMS/ONC rule, so possibly having meta.profile as required would make sense for all these implementations. @Saul Kravitz, @Pat Taylor, maybe some additional discussion of this requirement would be helpful.
Pat Taylor (Oct 08 2020 at 15:07):
@Lloyd McKenzie The CARIN BB profiles were designed to align with claims submission data standards adopted by HHS. The reimbursement methodologies, benefits applied and value sets are different between the profiles. As the source of the data is the payer and the end user of the data is a consumer, from a payer perspective, conveying data with a high degree of data quality is an important consideration. Our preference is to require the meta.profile to be populated. @Igor Sirkovich @Amol Vyas @Saul Kravitz
Last updated: Apr 12 2022 at 19:14 UTC
