FHIR Chat · FHIR DSTU2 to STU3 converter · implementers

Stream: implementers

Topic: FHIR DSTU2 to STU3 converter

Jeff Chung (Feb 06 2017 at 20:31):

Does anyone have code to convert DSTU2 observations to STU3? I will be using this for a Himss demo.

Grahame Grieve (Feb 06 2017 at 20:36):

there's code in the Java reference implementation - see org.hl7.fhir.convertors.VersionConvertor_10_20.

Jeff Chung (Feb 06 2017 at 21:26):

That is awesome. Thanks!

Vadim Peretokin (Feb 07 2017 at 08:15):

I'm looking at converting DSTU2 to STU3 as well - is that the only tool available (not that it's a problem), or are there any others as well?

Grahame Grieve (Feb 07 2017 at 08:38):

the only other option I know of, which is actually more reliable, is the R2/R3 conversion scripts - but using these requires more infrastructure which is not in place right now

Grahame Grieve (Feb 07 2017 at 08:38):

a to do for me

Vadim Peretokin (Feb 07 2017 at 08:38):

Do those scripts use these maps? https://github.com/hl7-fhir/fhir-svn/blob/master/implementations/r2maps/R2toR3/Account.map

Grahame Grieve (Feb 07 2017 at 08:42):

yes that's the scripts I was referring to

Vadim Peretokin (Feb 07 2017 at 08:42):

Ah man, I got pretty excited about getting to use those. How much more is needed to make them work?

Grahame Grieve (Feb 07 2017 at 10:28):

well, they work (or they worked 4 weeks ago, and I will need to reviist them), and the code to make them work is in the unit tests for the java reference implementation - but that's the only place and it's not wrapped it elsewehre in a nice reusable form

Vadim Peretokin (Feb 07 2017 at 10:30):

Sounds similar to VersionConvertor_10_20.convertResource which I've just finished grasping (haven't tried running yet though)

Grahame Grieve (Feb 07 2017 at 10:31):

that doesn't require any infrastructure. You just parse the resource, convert it, and then save it.

Grahame Grieve (Feb 07 2017 at 10:31):

the maps require an IWorkerContext, which means you need definitions.x.zip

Vadim Peretokin (Feb 07 2017 at 10:32):

Aah, okay. I'll try the convertResource first then

Jeff Chung (Feb 08 2017 at 17:04):

Is there a way to convert the FHIR objects to JSON? I noticed the FHIR-SVN library uses its own DSTU2 and DSTU3 structures.

Jeff Chung (Feb 08 2017 at 18:05):

Oh I think I found the answer. I didn't see the JsonParser class.

Vadim Peretokin (Feb 08 2017 at 19:52):

Are these projects exported as gradle/maven projects at all? I got an all-Eclipse build working, but a gradle project would be nicer

Jeff Chung (Feb 08 2017 at 22:27):

I would like it as a maven project. I added jar commands to the ant build files to create jars. Then added them to my maven project.

Jeff Chung (Feb 08 2017 at 22:28):

Is there a conversion for MedicationAdministration? I was able to find conversions for others except for that one.

James Agnew (Feb 09 2017 at 00:56):

My hope is to get these into HAPI sometime soon, at which point they'll be available as Maven artifacts and kept up to date that way..

Jeff Chung (Feb 09 2017 at 00:59):

So far I have tried Device, Location, Observation, and Patient which have converted successfully. I'm hoping for MedicationAdministration.

Grahame Grieve (Feb 09 2017 at 03:54):

yes it's my intent that the maven etc distro is as part of HAPI

Vadim Peretokin (Feb 09 2017 at 07:26):

@Jeff Chung Yeah it seems to be missing, but Grahame did make a map in https://github.com/hl7-fhir/fhir-svn/blob/master/implementations/r2maps/R2toR3/MedicationAdministration.map. So if you want to pull out the infrastructure for running those from the unit tests...

Jeff Chung (Feb 09 2017 at 18:12):

Thanks Vadim

Vadim Peretokin (Feb 10 2017 at 12:20):

For StructureDefinition, VersionConvertor_10_20 converter mostly works well, but it misses putting the @id on elements in a few spots - is it expecting some values to be present in the DSTU2 StrucDef, or should I patch it for the cases it's missing?

Lloyd McKenzie (Feb 10 2017 at 20:59):

If it's not working, then patching or a change request may be necessary

Grahame Grieve (Feb 10 2017 at 22:46):

the algorithm for @id changed a lot between STU2 and STU3. I didn't try and get them right in the converter - I expect to use ProfileUtilities.setElementIds after the conversion

Vadim Peretokin (Feb 11 2017 at 12:54):

ProfileUtilities.setIds? I'll give it a go, looks like the right thing.

Vadim Peretokin (Feb 13 2017 at 10:06):

I'm getting ids like MedicationStatement:m1prescribing.effectivePeriod.start:starttidspunkt. The colons look nice and I can make sense of them - just checking that's the expected output?

Grahame Grieve (Feb 13 2017 at 10:26):

yes

Vadim Peretokin (Feb 13 2017 at 10:26):

Great thanks.

Vadim Peretokin (Feb 13 2017 at 14:08):

It's sticking sliceName on all elements - is that a big deal outside of slices? The sliceName is also what was the ElementDefinition.name before, whose value wasn't constrained before - and sliceName now needs to follow a regex. Is there a method to strip invalid characters or should I do that as post-processing?

Vadim Peretokin (Feb 13 2017 at 15:02):

Nevermind, it's an issue with the http://build.fhir.org/definitions-r2.xml.zip, it has names on everything.

Grahame Grieve (Feb 13 2017 at 19:22):

then it's wrong.

Mritunjay Dubey (Mar 13 2017 at 12:46):

Hey guys, how can I try these scripts ? which maven artifact will contain these ?

Grahame Grieve (Mar 13 2017 at 20:33):

@James Agnew - can we get the R2 / R3 converter code into HAPI?

James Agnew (Mar 13 2017 at 20:53):

very soon! :)

It's in a git branch right now, will be moved into master and deployed as a snapshot within probably the next 24 hours. The only holdup has been getting the validator updated. The 1.9.0 round of model changes broke a lot of HAPI's unit tests so it's taken a while to get those running. I actually just finally finished that work this morning.

Mario Hyland (Mar 14 2017 at 18:53):

Seems to me that this approach to consider a DSTU2 to STU3 converter is not a simple translation or mapping exercise. Sure you can force DSTU2 resources to be STU 3 (and vice-versa) - but this will not work in all (FHIR Resource or Profile type) cases and with all situations without incuring some type of "data-loss" or potential "data-coruption". A simple mapping exercise we used to test out conversion includes the following exercise Example-A (v1.0) converted to Example-B (v2.0), followed by Example-B (v2.0) converted to Example-C (v1.0) (same format as Example-A) - if Example-A and Example-C are the same your conversion works (with no data-loss). I would want to ensure folks understand these challenges - unless I am missing something (which has been known to happen - on occassion).

Grahame Grieve (Mar 14 2017 at 19:13):

you could read the introduction at http://build.fhir.org/r2maps.html

Michael Lawley (Mar 15 2017 at 06:06):

Are there any plans for R2:ValueSet <-> R3:CodeSystem?

Grahame Grieve (Mar 15 2017 at 19:07):

that works. ValueSets with an inline code system are converted to a value set and a code system

Michael Lawley (Mar 15 2017 at 23:27):

ah, good. it's just not clear from the table of conversions

Mritunjay Dubey (Mar 17 2017 at 06:07):

Hey @James Agnew , As you mentioned on 14th march, the converters will be out as a snapshot. Are they already published ? If yes, where can I find them? Is this a separate artifact or part of an existing one

Mritunjay Dubey (Mar 17 2017 at 13:21):

Hey, for now I am building myself and using the jar in my project, just to test out. I got a problem, since MedicationOrder is renamed to MedicationRequest how I am supposed to convert a bundle having MedicationOrder.

Mritunjay Dubey (Mar 17 2017 at 13:22):

Right now I am getting an error Unknown resource MedicationOrder ?

Mritunjay Dubey (Mar 19 2017 at 09:17):

Any ideas about above? I am getting same error for MedicationOrder and DiagnosticOrder. Though I didn't see on fhir-build-diff thatDiagnosticOrder is replaced by which resource.

Grahame Grieve (Mar 19 2017 at 10:42):

It sounds like there's something wrong with your loading conditions.

Grahame Grieve (Mar 19 2017 at 10:42):

how are you loading?

Mritunjay Dubey (Mar 19 2017 at 12:12):

@Grahame Grieve , This is the code which I have written

r2ToR3Loader.loadBundle(fileInputStream, false) String s1 = FhirContext.forDstu3().newXmlParser().encodeResourceToString(bundle);

It works for fileInputStream containing Immunization/Condition/Procedure/Observation/DiagnosticReport/FamilyHistory/ProcedureRequest, but it fails for DiagnosticOrder/MedicationOrder. I suspect it might be because these are deleted in STU3.

Grahame Grieve (Mar 19 2017 at 19:38):

how did you prepare you r2tor3Loader?

Mritunjay Dubey (Mar 20 2017 at 02:57):

I had checkout svn source and built the jar. I found only one way to prepare r2ToR3Loader which is R2ToR3Loader r2ToR3Loader = new R2ToR3Loader(); and after that above lines are called.

Mritunjay Dubey (Mar 20 2017 at 04:22):

@Grahame Grieve I think one possible reason could be I tried to use these as my dependencies

compile ("ca.uhn.hapi.fhir:hapi-fhir-base:2.1") compile ("ca.uhn.hapi.fhir:hapi-fhir-structures-dstu3:2.1") compile ("ca.uhn.hapi.fhir:hapi-fhir-structures-hl7org-dstu3:2.1") compile files('lib/org.hl7.fhir.igpublisher.jar')

org.hl7.fhir.igpublisher.jar is the latest which I built rest I am getting from maven. Which might be the cause, I will try to get snapshots of above thing as well

Mritunjay Dubey (Mar 20 2017 at 04:38):

Even this didn't help.

Mritunjay Dubey (Mar 20 2017 at 04:58):

I looked at this page as well, http://build.fhir.org/r2maps.html. Even this doesn't talk about DiagnosticOrder and MedicationOrder.

Grahame Grieve (Mar 20 2017 at 05:09):

I will have time to look at this in 48 hours or so, once Release 3 is out

Mritunjay Dubey (Mar 28 2017 at 06:00):

Hey guys any updates on this.

Mritunjay Dubey (Mar 29 2017 at 06:18):

One new question, since codesystem and Valueset will be two different systems now. How I am supposed to reffer a CodeSystem from valueset ?

Oliver Egger (Mar 29 2017 at 06:30):

You can include a CodeSystem in a ValueSet with compose/include (see https://www.hl7.org/fhir/valueset.html) e.g.

 <compose>
    <include>
      <system
              value="http://..../>
    </include>

Mritunjay Dubey (Mar 29 2017 at 06:51):

Hey @Oliver Egger, Thanks for the response. I just converted a valueset having an inline codeSystem with R2->R3 converter. I just got below after conversion this is the extra part we got

<compose>
        <include>
            <system value="http://acme.com/config/fhir/codesystems/cholesterol"/>
        </include>
    </compose>

And in R2 was as below

    <codeSystem>
        <system value="http://acme.com/config/fhir/codesystems/cholesterol"/>
        <version value="4.2.3"/>
        <caseSensitive value="true"/>
        <concept>
            <code value="chol"/>
            <display value="SChol"/>
            <definition value="Serum Cholesterol"/>
            <designation>
                <use>
                    <system value="http://acme.com/config/fhir/codesystems/internal"/>
                    <code value="internal-label"/>
                </use>
                <value value="Obdurate Labs uses this with both kinds of units..."/>
            </designation>
        </concept>
    </codeSystem>

I am not sure why others are not populated.

Mritunjay Dubey (Mar 29 2017 at 06:54):

If I remember corectly @Grahame Grieve mentioned ValueSets with an inline code system are converted to a value set and a code system.

Oliver Egger (Mar 29 2017 at 08:48):

Why is written "codesystems" in R2, should I not be CodeSystem? In the R2R3 map it looks like that a CodeSystem is generated: http://build.fhir.org/valueset-version-maps.html http://build.fhir.org/mapping-language.html (ValueSet-codeSystem" : for src.codeSystem as vs0 make create("CodeSystem") as vt0 then codeSystem(src, tgt, vs0, vt0))

Mritunjay Dubey (Mar 29 2017 at 09:48):

Hey thanks! One new question now ? I just tried converting a bundle with DiagnosticOrder. The performer element is missing from the output bundles.

      <performer>
            <reference value="http://localhost:9997/providers/19.json"/>
      </performer>

This is what input file contains. The output file doesn’t have anything called performer.

Grahame Grieve (Mar 29 2017 at 20:05):

where is performer defined?

Mritunjay Dubey (Mar 30 2017 at 03:34):

This is what my original xml looks like

                <DiagnosticReport xmlns="http://hl7.org/fhir">
                <id value="86571847-d982-4af9-887c-ee8d0fccfde5"/>
                <code>
                    <coding>
                        <system value="http://localhost:9997/openmrs/ws/rest/v1/tr/concepts/07a9e3a1-5206-11e5-ae6d-0050568225ca"/>
                        <code value="07a9e3a1-5206-11e5-ae6d-0050568225ca"/>
                        <display value="Hemoglobin [Mass/volume] in Blood"/>
                    </coding>
                </code>
                <status value="final"/>
                <issued value="2015-09-04T15:17:08.000+05:30"/>
                <subject>
                    <reference value="http://localhost:9997/api/default/patients/98001046534"/>
                    <display value="98001046534"/>
                </subject>
                <performer>
                    <reference value="http://localhost:9997/providers/19.json"/>
                </performer>
                <encounter>
                    <reference value="urn:uuid:cfd88241-3af9-4f5b-892a-980a70506f20"/>
                </encounter>
                <identifier>
                    <value value="urn:uuid:86571847-d982-4af9-887c-ee8d0fccfde5"/>
                </identifier>
                <request>
                    <reference value="http://172.18.46.156:8081/patients/98001046534/encounters/5c2714ce-3b5a-43e3-afd8-50fbb4c32fa4"/>
                </request>
                <effectiveDateTime value="2015-09-04T15:15:17.000+05:30"/>
                <result>
                    <reference value="urn:uuid:b0a674b5-486a-4975-af43-60fe74b6ed8e"/>
                </result>
            </DiagnosticReport>

If you see there is a performer field. Below is the converted xml which doesn't have performer field.

            <DiagnosticReport xmlns="http://hl7.org/fhir">
                <id value="86571847-d982-4af9-887c-ee8d0fccfde5"/>
                <identifier>
                    <value value="urn:uuid:86571847-d982-4af9-887c-ee8d0fccfde5"/>
                </identifier>
                <status value="final"/>
                <code>
                    <coding>
                        <system value="http://localhost:9997/openmrs/ws/rest/v1/tr/concepts/07a9e3a1-5206-11e5-ae6d-0050568225ca"/>
                        <code value="07a9e3a1-5206-11e5-ae6d-0050568225ca"/>
                        <display value="Hemoglobin [Mass/volume] in Blood"/>
                        <userSelected value="false"/>
                    </coding>
                </code>
                <subject>
                    <reference value="http://localhost:9997/api/default/patients/98001046534"/>
                    <display value="98001046534"/>
                </subject>
                <context>
                    <reference value="urn:uuid:cfd88241-3af9-4f5b-892a-980a70506f20"/>
                </context>
                <effectiveDateTime value="2015-09-04T15:15:17+05:30"/>
                <issued value="2015-09-04T15:17:08.000+05:30"/>
                <result>
                    <reference value="urn:uuid:b0a674b5-486a-4975-af43-60fe74b6ed8e"/>
                </result>
            </DiagnosticReport>

Mritunjay Dubey (Mar 30 2017 at 03:39):

I am converting this as r2ToR3Loader.loadBundle(fileInputStream, false). I have build the igpublisher.jar from my local checked out svn copy which has the latest build from 29th march,
`r11865 | grahameg | 2017-03-29 07:40:03 +0530 (Wed, 29 Mar 2017) | 1 line

update vsCache`

Mritunjay Dubey (Mar 30 2017 at 03:56):

I just checked the code, these lines are commented out, not sure why

//    tgt.setPerformer(convertReference(src.getPerformer()));
//    for (org.hl7.fhir.dstu2.model.Reference t : src.getRequest())
//      tgt.addRequest(convertReference(t));

Mritunjay Dubey (Mar 30 2017 at 10:21):

Any idea about above, Even I found out we are not converting MedicationOrder and DiagnosticOrder. The VersionConvertor_10_30.convertResource() are doesn't check for these two resources.

Mritunjay Dubey (Mar 31 2017 at 04:15):

Hey guys any idea about above. If we are not converting above two Resources can we not throw an error. Right now if there is a bundle with Encounter, Condition and MedicationOrder it fails to convert the whole bundle itself because the medicationorder is unknown.

Grahame Grieve (Apr 03 2017 at 02:22):

so i

Grahame Grieve (Apr 03 2017 at 02:22):

so I'll have time to look at this stuff this week.

Grahame Grieve (Apr 03 2017 at 02:23):

note that there are 2 different conversion routines: the java one, and the mapping language one.

Grahame Grieve (Apr 03 2017 at 02:24):

going forward, the java one is only really supported for conformance resources, unless someone else wants to take that on. The mapping langauge one is what I intend to invest in from now on. Which is the one your invocation code relates to. I had a quick look at performer, and the mapping language looks all right to me

Mritunjay Dubey (Apr 03 2017 at 03:56):

We are using the Java library which is published in igpublisher.jar. As you said *The mapping langauge one is what I intend to invest in from now on*, Does this mean if there are any missing parts we will have to do that manually in java code by following scripts ?

Grahame Grieve (Apr 03 2017 at 03:59):

there's the Java library that does the actual conversion directly, and there's the java library that implements the mapping language, and uses the mapping language files published in the spec.

Grahame Grieve (Apr 03 2017 at 03:59):

the first is only maintained for the conformance resources (though I didn't bother removing the other reosurces in case someone wnated to own that).

Grahame Grieve (Apr 03 2017 at 04:00):

the second is what I think you are using

Mritunjay Dubey (Apr 03 2017 at 04:02):

yeah!

Grahame Grieve (Apr 03 2017 at 04:05):

so I think we want to sort out this so that there's an easy to run it, which we don't have now. how are you running? as a java class, or as a jar?

Mritunjay Dubey (Apr 03 2017 at 04:06):

We have added igpublisher.jar as a dependncy to project and then invoking R2ToR3Loader with DSTU2 bundles.

Grahame Grieve (Apr 03 2017 at 04:22):

ok, so I will add a new class for this

Mritunjay Dubey (Apr 03 2017 at 04:40):

I think I am missing something. What is happening now R2ToR3Loader is working. Internally it calls VersionConvertor_10_30 which is not taking care of few things
1) DiagnosticOrder
2) MedicationOrder there is below commented line I found in java code

//    if (src instanceof org.hl7.fhir.dstu2.model.MedicationOrder)
//      return convertMedicationOrder((org.hl7.fhir.dstu2.model.MedicationOrder) src);

3) Performer for DiagnosticReport.
4) FamilyMemberHistory.condition.onset thing is changed from onsetQuantity to onsetAge. I think the mapping-scripts/java-code is not taking care of this.

Mritunjay Dubey (Apr 03 2017 at 04:41):

@Grahame Grieve As you said "so I will add a new class for this" what this new class will be doing ? I am a bit confused.

Grahame Grieve (Apr 03 2017 at 09:41):

yes. so you're using the java conversion code directly, but not using it the way I expected it to be used. The conversion code isn't supported anymore.

Grahame Grieve (Apr 03 2017 at 09:41):

I'll add a new class for you that makes it easy for you to use the right way

Mritunjay Dubey (Apr 03 2017 at 09:44):

Okay! After adding the new class will the above issues be fixed ?

Grahame Grieve (Apr 03 2017 at 10:01):

well, we'll have to see

Mritunjay Dubey (Apr 03 2017 at 10:08):

By when can we expect this class to be there? If it is checked in SVN I can pull and build it locally as well.

Grahame Grieve (Apr 03 2017 at 11:24):

see org.hl7.fhir.convertors.R2R3ConversionManager

Mritunjay Dubey (Apr 03 2017 at 11:30):

Hey We just checked out latest code base. There is this new file, but We are not sure how to use this one. Right now what we were doing calling r2ToR3Loader.loadBundle() and it was doing things in expected way. The new class has not implemented anything.

Mritunjay Dubey (Apr 03 2017 at 11:31):

I am not sure how to go forward about it.

Grahame Grieve (Apr 03 2017 at 11:31):

no at present it's just the interface. See the notes at the head for how you would use it. I'll work on the implementtation

Mritunjay Dubey (Apr 05 2017 at 05:10):

We tried these scripts. It is converting MedicationOrder as well now, didn't check deep though yet. We found below things are failing
1) Condition notes are not converted
2) DiagnosticRepoet Performer are not converted
3) FamilyMemberHistory condition onset is not converted. shouln't it rename <onsetQuantity> tags to <onsetAge> tag. While comparing I got both datatypes are Age but the tag name is changed.
4) ProcedureRequest with status request is throwing an error. I feel it has to be taken care in converter only, because outside I won't have control over it. I can't change these in DSTU2 bundle since the new status won't be available. As soon as it reaches to converter it fails saying the old status is unknown in STU3.

Grahame Grieve (Apr 05 2017 at 05:21):

@Rob Hausam @Eric Haas DiagnosticReport.performer is not converted becaue neither of you answered my question about this while I was working on the conversions. I don't have a clue how to understand the new DiagnosticReport.performer as compared to the old

Grahame Grieve (Apr 05 2017 at 05:21):

Condition.notes - I overlooked this, and I don't know why

Grahame Grieve (Apr 05 2017 at 05:21):

fixed, will update the posted scripts soon

Mritunjay Dubey (Apr 05 2017 at 05:57):

Hey @Grahame Grieve any idea about the last two ? We just found a new issue, Condition with clinical status as active is not getting converted.

Mritunjay Dubey (Apr 05 2017 at 06:11):

What I was thinking while analysisng DiagnosticReport.performer change. Initially it had performer with reference to (Organization/Practitioner). Now it has a performer.actor with same reference. We can populate the stu3 diagnosticreport.performer.actor same as dstu2 diagnosticreport.performer.

Mritunjay Dubey (Apr 05 2017 at 06:13):

The stu3 diagnosticreport.performer.role information cannot be populated because this information doesn't exist in older structures. And it cannot be put default also because it has an example binding. That's what I was thinking.

Mritunjay Dubey (Apr 05 2017 at 09:58):

Hey guys, The converters are failing for converting the DiagnosticOrder having more than one Item. Why are we restricting it to be only one? How should this be fixed ?

Mritunjay Dubey (Apr 05 2017 at 12:40):

Hello guys, I will just put what We have got today.

1) Condition notes are not converted.

2) Condition with clinical status "active" is converted to clinical status null.

3) FamilyMemberHistory condition onset is not converted. Ideally it should rename <onsetQuantity> tags to <onsetAge> tag. While comparing I got both has datatype as Age but the tag name is changed.

4) ProcedureRequest with status "requested" is throwing an error. It should map all existing status to one of the new status. I feel it has to be taken care in converter only, because outside I won't have control over it. I can't change these in DSTU2 bundle since the new status won't be available. As soon as it reaches to converter it fails saying the old status is unknown in STU3.

5) MedicationRequest.dosageInstruction.timing.repeat.boundsQuantity is not getting converted. Ideally it should rename <boundsQuantity> tags to <boundsDuration> tag. While comparing I got both has datatype as Duration but the tag name is changed.

6) DiagnosticRepoet Performer is not converted. (What I was thinking while analyzing DiagnosticReport.performer change. Initially, it had the performer with reference to (Organization/Practitioner). Now it has a performer.actor with the same reference. We can populate the stu3 diagnosticreport.performer.actor same as dstu2 diagnosticreport.performer.)

7) The DiagnosticOrder with more than one Item is failing. Why are we restricting it to be only one? How should this be fixed?

8) The MedicationOrder dateEnded is not present in MedicationRequest. What is the idea behind removing it ? I couldn’t find a similar field.

Eric Haas (Apr 05 2017 at 15:47):

R3 performer = R2 performer when R3 performer.role = AUTauthor (originator). R3 performer could be some other role as well and <> R3 performer. Probably need to create a tracker to change the binding to the v3 codes and binding strength to extensible @Rob Hausam what do you think?

Eric Haas (Apr 05 2017 at 15:53):

ProcedureRequest is for a single item. See how composite requests are handled here

Rob Hausam (Apr 05 2017 at 17:59):

@Eric Haas @Grahame Grieve Yes, that's along the lines of what I was expecting to be able to do with DR.performer.role. The current binding to the performer-role value set doesn't work for that (or at least doesn't provide an example for it), but changing it as you suggest I think would make sense. I think we could have some discussion as to whether we should make it extensible and whether it should be V3 codes or a FHIR mapping to V3 (or something else).

Mritunjay Dubey (Apr 06 2017 at 06:00):

@Rob Hausam @Eric Haas @Grahame Grieve, thanks for the help. Right now is there a way to tell ignore the errors in the conversion. Like in issues mentioned above the 4th and 7th are throwing an error. Which causes the whole bundle to fail.

Mritunjay Dubey (Apr 07 2017 at 02:39):

Hey guys, any updates on above.

Mritunjay Dubey (Apr 07 2017 at 02:43):

I meant is it possible anyway to ignore errors while converting, and convert them in my code.

Mritunjay Dubey (Apr 07 2017 at 02:56):

The above will be userful while converting something like Diagnostic Order with multiple Items, since the converter expects just one Item per DiagnosticOrder.

Grahame Grieve (Apr 07 2017 at 03:23):

this is on my list to get to - but I have a lot of things to work on

Mritunjay Dubey (Apr 07 2017 at 03:43):

ohhh okey! I wanted to know will something like this will be possible.

Mritunjay Dubey (Apr 09 2017 at 11:31):

@Grahame Grieve is there any workaround of above as of now ?

Grahame Grieve (Apr 09 2017 at 21:46):

not right now. Hoping to get to it this week, but I'm not keeping up right now

Grahame Grieve (Apr 19 2017 at 12:27):

ok this is on my list for tomorrow (pinging this to get it back to the top of the topic list)

Mritunjay Dubey (May 04 2017 at 03:49):

Hello guys, is converter it not being published as a maven artifact? Right now I am using it as a jar file dependency for my project but I want to avoid it becasue it is causing conflicts with lot of other libraries.

Grahame Grieve (May 04 2017 at 03:53):

it should be released via HAPI for maven use. I thought that would have happened by now @James Agnew

Mritunjay Dubey (May 04 2017 at 03:59):

Yeah I gave a try but I was thinking of using R2R3ConversionManager class which comes with just igpublisher.jar file. The same doesn't come with hapi-fhir-converter. I was just checking if this is also published as part of some artifact.

Grahame Grieve (May 04 2017 at 04:00):

that is what is supposed to come with HAPI now

Mritunjay Dubey (May 04 2017 at 04:01):

Do you mean even R2R3ConversionManager is supposed to come with HAPI in latest release ?

Grahame Grieve (May 04 2017 at 04:01):

yes

Mritunjay Dubey (May 04 2017 at 04:04):

Well, Then @James Agnew will be able to answer. I don't see it in their latest artifact on maven hapi-converter-2.4, neither it's in hapi-fhir github on master, so I can expect it to come as a snapshot.

James Agnew (May 04 2017 at 10:04):

At this point we have the java converters bundled into HAPI, but not yet the R2R3ConversionManager one. I tried really hard to get this into HAPI 2.4, but wasn't able to. The main reason for this is that R2R3ConversionManager has a dependency on the RI parsers, which don't ship with HAPI. I need to spend some time untangling the co-depoendencies between R2R3ConversionManager and other parts of the RI and I haven't been able to find the bandwidth to do that yet.

Help obviously welcomed if anyone wants to join into this effort.

Grahame Grieve (May 04 2017 at 12:29):

hmm. I'll have a look at that

Grahame Grieve (May 04 2017 at 12:29):

but i absolutely need the elementmodel parsers. Is that a problem?

Mritunjay Dubey (May 04 2017 at 12:34):

Hey @James Agnew thanks for the reply. The problem I am facing right now is becasue hapi artifacts have same classname as well as package name but the code is not same.

For Example :- https://github.com/hl7-fhir/fhir-svn/blob/master/implementations/java/org.hl7.fhir.utilities/src/org/hl7/fhir/utilities/xhtml/XhtmlParser.java and https://github.com/jamesagnew/hapi-fhir/blob/master/hapi-fhir-base/src/main/java/org/hl7/fhir/utilities/xhtml/XhtmlParser.java, both have same package and class name, but in hl7-fhir file has a method public XhtmlNode parseHtmlNode(XmlPullParser xpp) which is not present in the file from hapi-fhir. same problem with XmlParserBase#loadXml(InputStream stream).

I am not able to get a way out because I need to import igpublisher.jar which has code from hl7-fhir and I need to do some modification in the resources for that I need hapi-stu3 and hapi-stu2 structures artifact as well. Which ends up in incompatible code.

James Agnew (May 04 2017 at 12:38):

@Grahame Grieve no, the elementmodel parsers are no issue, we bundle those already.

@Mritunjay Dubey yeah, there is no way to use the RI stuff and HAPI on the same classpath. HAPI contains modifications to the RI, including stripping out the XmlPullParser as well as a few other dependencies on libraries that are either not needed or that have questionable licenses. I've been slowly rationalizing all of this, but it's a gradual process.

Grahame Grieve (May 04 2017 at 12:38):

let's talk through this in Madrid

James Agnew (May 04 2017 at 12:38):

..that is to say, you can't use an RI JAR like igpublisher in the same JVM as HAPI. You can absolutely use RI classes that are included with HAPI of course :)

James Agnew (May 04 2017 at 12:39):

Sounds good.

Lloyd McKenzie (May 04 2017 at 12:40):

Is there any chance of us putting the igpublisher into Maven? (Question from a client)

Grahame Grieve (May 04 2017 at 12:44):

there's a chance. If someone does it or pays me to do it.

Craig McClendon (Sep 26 2018 at 18:53):

Can someone summarize the current options for converting DSTU2 or DSTU2.1 resources to STU3? The converters bundled with HAPI appear to only address the conformance resources (at least the 2.1 converters do). I assume the R2R3ConversionManager piece has not made it into HAPI? Thanks!

Craig McClendon (Oct 01 2019 at 17:02):

DELETE POST - answered above.

Mridul Sharma (Jun 09 2020 at 14:01):

Hello all, we recently upgraded spark FHIR DSTU2 version and getting error "Mongo document contains invalid BSON to parse" with existing data while migrating to new version. Please share your experience if any faced such issue or any recommendation how it can be addressed.

Will it work fine if we migrate directly to STU3?
We also moved infrastructure from Azure to AWS, Windows to Linux not sure if that is affecting it some way.

Please share your thoughts.

Lloyd McKenzie (Jun 09 2020 at 14:16):

@Ward Weistra

Ward Weistra (Jun 09 2020 at 14:22):

Perhaps @Kenneth Myhra knows

Kenneth Myhra (Jun 09 2020 at 14:41):

@Mridul Sharma when was the last time you upgraded Spark? If it has been a while it could be that you need to run the RemoveDuplicateId.js script to remove a Id-field that caused trouble in newer versions of MongoDB. https://github.com/FirelyTeam/spark/tree/master/scripts

Also, is there an inner exception that contains some more information you could share?

Mridul Sharma (Jun 09 2020 at 14:49):

@Kenneth Myhra thanks for your response. We will follow your suggestion.

Actually Spark FHIR DSTU2 version we were using is from 2017 and it was never upgraded, below is the inner exception.

{
"resourceType": "OperationOutcome",
"issue": [
{
"severity": "error",
"diagnostics": "InvalidOperationException: Mongo document contains invalid BSON to parse."
},
{
"severity": "error",
"diagnostics": "FormatException: Type checking the data: Encountered unknown element 'Id' at location 'Resource.Id[0]' while parsing"
}
]
}

Kenneth Myhra (Jun 09 2020 at 14:50):

Yep, you will need to run the mentioned script. But as always back-up before you do so :)

Mridul Sharma (Jun 09 2020 at 14:52):

Sure, let me try it.

Kenneth Myhra (Jun 09 2020 at 14:53):

And to answer your other question. Since the resources are stored as JSON and in DSTU2 format in MongoDB you would need to run a migration process to convert them to STU3

Mridul Sharma (Jun 09 2020 at 15:01):

Sure @Kenneth Myhra will run migration process, any recommendation / best practices or any ready to use tool which can be make this process smooth for us?

Kenneth Myhra (Jun 09 2020 at 15:14):

@Mridul Sharma There are some tools in the HL7 FHIR specification that can do this for you, that is actually how resources in the specification are transferred from one version to the next. In a production system it will need some kind manual supervision to make sure you do not loose data when converting from one version to another.

The tool mentioned uses the mapping language. You can see maps under the diff section on each resources in the FHIR specifcaion, like this for DocumentReference http://hl7.org/fhir/documentreference-version-maps.html

Depending on how many resource types you are currently using it could be trivial to just code the conversion yourself in your favorite language.

Mridul Sharma (Jun 09 2020 at 15:19):

Thanks @Kenneth Myhra , will explore these options.
I really appreciate your help.

Last updated: Apr 12 2022 at 19:14 UTC