Stream: hapi
Topic: dynamic creation of resources
Marc Günther (Feb 02 2021 at 15:18):
How to dynamically create Resources based on their StructureDefinitions?
Hi,
I'm quite new to FHIR and HAPI, so forgive me if I don't know something obvious.
We want to implement a conversion from our internal representation of medical data to specific FHIR resources. For this the idea is to annotate each of our internal fields with the kind of FHIR resource it corresponds to.
For example, the data field which contains a patients body weight would be annotated with the following StructureDefinition (this is from the German Covid dataset):
Once everything is annotated this way, we can write some generic conversion function, which iterates over our data, and generates the FHIR output, based on their StructureDefinitions (there will also conversion of values to be considered, but let's leave that for later).
So for example, we would like to have some Java method, which takes:
- the value of a form field, i.e. 84kg, and
- the BodyWeight StructureDefinition,
and then outputs a FHIR data record as in the example here:
As far as I understand them, the StructureDefinitions contain all the necessary information to construct data records out of them in a generic way.
Unfortunately, I have not really found any way to do this. I can read the StructureDefinitions just fine, but there seems to be no support to create actual data records from them.
All I can do at the moment, is to use the fluent API to create a data record just as the BodyWeight example above, but then everything is hard coded in my Java code. I would then need to create Java methods for every possible data type that's part of the Gecco dataset. Which seems like a lot of effort, given that the structural information is already available.
Is there anything I missed, is somebody else already doing that, or is this idea maybe ridiculous and wouldn't work anyway? What are all those StructureDefinition used for normally, and how do you create data records for them?
Thanks for any insight,
Marc
Noemi Deppenwiese (Feb 02 2021 at 15:38):
Without exactly knowing what HAPI can or can't do with StructureDefinitions, usually more than one "data item" is necessary to create a resource. In your example, the GECCO Body Weight Profile also requires status, subject and effective for which no fixed values are provided. Other GECCO Observation profiles have different constraints, so they would require different parameters. (Of course, you could write a "generic" method for all observations that supports all possible parameters, but than why not add category & code as parameters too.)
Lin Zhang (Feb 02 2021 at 15:41):
Maybe you need an integration engine such as Mirth Connect to do the conversions using your mappings.
Frank Oemig (Feb 02 2021 at 15:51):
With an integration engine you have to describe the same stuff. Instead you need a semantic model with an appropriate description that integrates between your data and a representation in FHIR. as Noemi says it depends on the item what is necessary and how it fits resp. interrelates.
Marc Günther (Feb 02 2021 at 17:31):
More than one data item:
Ah, yes, that is an added complexity. In the BodyWeigt example, "subject" and "effective" would come from the metadata that is attached to our specific data field, and "status" I'm not sure what it is yet, but will probably figure it out.
But there are also Resources that really have true multiple data items, like BloodPressure. For this we would annotate our data field that contains the systolic value so that it will map to the Resource BloodPressure and in there to the id "Observation.component:systolicBloodPressure.value[x]".
Does this make sense, or is this not the way to go about it?
but than why not add category & code as parameters too.
But aren't these fixed values, at least in the BodyWeight example? They definitely are not part of our existing data.
I think this is the main point that I do not understand. When I want to create an Observation that contains a BodyWeight, how do I know that it needs these entries (these are all taken from the example):
category.coding[0].system: http://terminology.hl7.org/CodeSystem/observation-category
category.coding[0].code: vital-signs
code.coding[0].system: http://loinc.org
code.coding[0].code: 29463-7
code.coding[0].display: Body weight
code.coding[1].system: http://snomed.info/sct
code.coding[1].code: 27113001
code.coding[1].display: Body weight (observable entity)
code.text: Body weight
I would like to avoid to hardcode this info in my Java source code for all 64 Gecco Resources.
These are already part of the gecco BodyWeight StructureDefinition, and it's baseDefinition, so shouldn't there be some easy way to get that info, and construct new Resources from it?
How does everyone here create FHIR data records for various Profiles? Is it all hard coded for the specific use case?
Noemi Deppenwiese (Feb 03 2021 at 08:30):
Well, you could add the code & category codes as field metadata too? Alternatively, you could process the StructureDefinition and check all ElementDefinitions in the StructureDefinition.snapshot for cardinality & fixed values and set them if required.
I don't believe I have ever processed StructureDefinitions for Resource creation (only for metadata-focused uses), how much generalized vs specialized code there is for me depends heavily on the profile set and the source data structure. For how good generalization works with GECCO profiles, I suggest you ask for experiences on german (d-a-ch) or german/mi-initiative, questions on GECCO often get asked there.
Marc Günther (Feb 03 2021 at 10:28):
Hi Noemi,
thanks for your answer.
Well, you could add the code & category codes as field metadata too?
Yes, but how do I know what they are for the 68 StructureDefinitions that are eg. in Gecco? Without reading them from the StructureDefinitions as you suggest?
As you have never used them in that way, how do you yourself create Resources then? If I were to manually implement 68 methods to create all the Gecco Resources with all the details hardcoded in them, how would I know what to write in those methods?
Sorry, I think there is something fundamental that I do not yet understand about FHIR. It is advertised as an exchange data format between different systems, so the problem of converting a proprietary data format into FHIR has to come up all the time and already has to be solved in several ways, but my searches have turned up absolutely nothing about the topic.
I am very grateful for any help!
Noemi Deppenwiese (Feb 03 2021 at 10:49):
You need to look at the profiles and identify what information from your source system corresponds to which FHIR attribute in which resource. The profiles tell you what the resources should look like at the minimum (you can always add additional information for attributes not required in the profile, e.g. the performer for bodyweight.) Also, i would suggest to look at the actual implementation guide instead of just the profiles (for gecco body weight https://simplifier.net/guide/GermanCoronaConsensusDataSet-ImplementationGuide/Bodyweight-duplicate-2), since there is usually some explanatory text. If you are lucky, you can identify a general pattern and generalize some of the mapping, if you are unlucky you need to handle everything separately. For example, the weight observation is very different from the sex assigned at birth observation (e.g. effective is important for weight, but sex assigned at birth should not change so effective is not as important. On the other hand, all Laboratory values are very similar (except the different LOINC codes of course), so there one method would probably be enough for all profiles (if you have the correct LOINC for each test or a mapping in your source database of course).
What we actually use the StructureDefinitions for is validation of our complete resources to catch any errors.
In MIRACUM, we have one of our older ETL jobs publicly available on Github https://github.com/miracum/etl-p21-to-fhir , it might give you an example of what a transformation might look like. (My colleagues who are more involved in our ETL development do not read this stream, so if you have questions about it again i would suggest the german/mi-initiative stream.)
Marc Günther (Feb 05 2021 at 09:00):
You need to look at the profiles and identify what information from your source system corresponds to which FHIR attribute in which resource.
Ah, Sorry, I think I didn't explain that properly. I am not concerned with the mapping, in fact I do not even know what those source systems look like, as they are completely generic. What I am concerned with is providing a way for others to define that mapping for their particular system.
So, short explanation, we provide an EDC system, where people have defined forms for entering data, and these forms consist of text fields, number fields, date fields, etc. What they contain is not known to me or the system.
We want to provide a way to export these fields to FHIR. So, as these fields are completely generic, the idea is to provide a configuration UI, which allows to annotate each field with its corresponding FHIR Resource and path inside that Resource. This defines a mapping from those particular forms to FHIR/Gecco. We can than use this mapping to export the data into FHIR Resources.
So, what I need is:
- a way to list all the Resources and their possible value pathes, and present them to the user, so they can define the mapping,
- and a way to then convert our generic data fileds into FHIR Resources, given that mapping.
My post here was concerned with the second point, but of course the first one is also interesting.
When I understand all of this correctly, I need someone with Gecco specific domain knowledge, in order to:
- provide that list of possible mappings,
- and provide some Java code to create the corresponding HAPI Fhir objects.
I was hoping that I could extract that info out of the StructureDefinitions, which are already there, and contain all the necessary information, in some generic way, but that doesn't seem to be possible.
Have I missed something?
Marc Günther (Feb 05 2021 at 09:35):
And of course in the final and sad conclusion this means, if at some point we want to support not only Gecco, but some other set of profiles, we have to again find someone with domain specific knowledge for THAT specific profile, in order to get the necessary information and Java code?
I was hoping there would be some generic way to implement this, so users can upload a definition like one of these zip files available for download here:
and the system parses them, and provides the necessary mapping options and conversion methods.
I still think that all the information which is needed to do this is in those zip files, no? But apparently, no one has ever used them for this purpose?
Noemi Deppenwiese (Feb 05 2021 at 09:51):
Well for the first step (mapping) I believe you will always need someone with Domain-Specific knowledge. However, loading profiles into a Mapping UI to provide a user with a visual mapping method should be possible. For step 2 (executing the mapping), i believe given the mapping an the profiles it should be feasible to build the HAPI objects without specialized code. You would need to translate each element's id to a HAPI method call, and find a way to deal with slicing. (Plus some other issues that will probably come up during implementation...)
Of course, if you are not bound to using HAPI, using a more generic JSON Library may be easier. Or take a look at the FHIR mapping language. If you manage to generate mapping language from your mapping UI, you can use an existing engine to execute it.
Or have a look at Pyrog: https://github.com/arkhn/pyrog)It provides a mapping UI and can execute that mapping, maybe you can build a module to make it work with your EDC system?
Marc Günther (Feb 08 2021 at 15:35):
Well, the actual mapping (this data field corresponds to that element in that FHIR Resource), would be done by whoever setup those data fields in the first place, so they should have the necessary domain knowledge.
Providing the Mapping UI, and executing the mapping (generating the Resources) needs to be done by us beforehand.
So if I understand your answer correctly, something like this has never been done with HAPI before, and it might not even be the best tool for it?
I found a similar topic here (although they are talking about code-generation), but the problem is described as "super hard":
Regarding the FHIR Mapping Language, can it read in existing StructureDefinitions? Or is there a way to convert the StructureDefinitions into that language? If not, then I would need to manually write code in that language for every Profile, and if I need to do that, I could also do it in Java Hapi, which is a language I already understand. :-)
Grahame Grieve (Feb 08 2021 at 22:35):
someone somewhere has to write 'code' that explains how to migrate from one form to another.
Grahame Grieve (Feb 08 2021 at 22:37):
the mapping language can leverage the structure definitions, but the structure definitions themselves don't generally have the capability to capture the mappings. (the mapping facilities in the structure definition support 'conceptual' mappings, which state the elements map to each other, but doesn't have the facilities to deal with value domain exceptions, or all the typical kinds of data wrangling that goes on
Sean Muir (Feb 08 2021 at 23:02):
Marc Günther said:
Well, the actual mapping (this data field corresponds to that element in that FHIR Resource), would be done by whoever setup those data fields in the first place, so they should have the necessary domain knowledge.
Providing the Mapping UI, and executing the mapping (generating the Resources) needs to be done by us beforehand.
So if I understand your answer correctly, something like this has never been done with HAPI before, and it might not even be the best tool for it?
I found a similar topic here (although they are talking about code-generation), but the problem is described as "super hard":
Regarding the FHIR Mapping Language, can it read in existing StructureDefinitions? Or is there a way to convert the StructureDefinitions into that language? If not, then I would need to manually write code in that language for every Profile, and if I need to do that, I could also do it in Java Hapi, which is a language I already understand. :-)
For what its worth - the MDHT project implemented similar feature for CDA to populate template ids and other static content - happy to walk you through the code if interested
Noemi Deppenwiese (Feb 09 2021 at 10:40):
Well, I can say I have never seen it done with HAPI before :) About the FHIR Mapping language, my knowledge on the topic is only superficial. If you plan to explore that direction, there is a mapping-framework stream where someone might be able to answer your question.
Marc Günther (Feb 09 2021 at 13:18):
Grahame Grieve said:
someone somewhere has to write 'code' that explains how to migrate from one form to another.
Yes, but this code could be generic, no? I mean, instead of 68 incantations like the following:
And with those, I don't even know how to code all these 68 cases... :(
Noemi Deppenwiese (Feb 09 2021 at 13:41):
Well, you could write something that takes 1..* value/path pairs explaining where each value should be stored in a FHIR structure and a structure definition. Then, you parse the StructureDefinition for fixed values and set them, then you write each value from your value/path pairs to the given path. For this to work, you would need some "generic" way of building resources which is your main problem as i understand it? So, you could write some method that takes a ElementDefinition.path and translates it to a HAPI method call. An maybe easier solution would be to use e.g. a JSON Path library. Translating the path value to JSON path should be possible, given that you can find out if something is an element or a list by looking at the ElementDefinition in the StructureDefinition ? (maybe you would need an additional layer because of slicing but in general it should work?) So, you could build a JSON object which when you serialize it is a FHIR resource. At least, that is the approach i would explore.
Marc Günther (Feb 09 2021 at 14:01):
That sounds very interesting, i will try.
Thanks :)
Last updated: Apr 12 2022 at 19:14 UTC
