Stream: implementers
Topic: new tool for example loading?
Jens Villadsen (Dec 26 2021 at 21:00):
Hi all - I'm building a tool as I have the need my self to be able to send the example resources from an IG to a designated FHIR server. Before I spend too much time on it, I would like to know that I'm not building something that already exists out there. FYI I'll be piggybacking on the HAPI FHIR stack as well as the utilities from the org.hl7.fhir.utilities so much of the heavy lifting is already done. (@Jose Costa Teixeira )
Jens Villadsen (Dec 26 2021 at 21:01):
@Grahame Grieve / @Martijn Harthoorn and others - are you familiar with such a tool already?
Grahame Grieve (Dec 26 2021 at 21:32):
well, it sounds like bulk import to me. I believe that the bulk data spec covers that now?
Grahame Grieve (Dec 26 2021 at 21:33):
I have some code somewhere to do it in hapi core, but it's not organised or packaged up for a command line thing. And the code itself isn't a very big deal
Jens Villadsen (Dec 26 2021 at 22:13):
I agree - it isn't a big deal codewise
Jens Villadsen (Dec 26 2021 at 23:34):
@Grahame Grieve - having had a quick look at https://github.com/smart-on-fhir/bulk-import/blob/master/import.md I'd say that my need is of a different nature. My need is to 'prepare' a given server for a given set of scenarioes exemplified by the example instances that follows an IG. Nothing more. It puts no other requirements on the server but the need to support transaction as there are so few example instances in the IG's. I guess that the IGpublisher just as well could do it - simply bundling all example instances in a transaction bundle as part of the build process - but whether that is a good idea I cannot say.
Jens Villadsen (Dec 26 2021 at 23:36):
@John Moehrke / @Oliver Egger what tools do you guys use for bootstrapping IHE test cases when doing IHE Connectathons? Do you still heavily rely on the textual descriptions in Gazelle, or could you just as well have the need for at least formulat a given state with a certain set of test resources?
Grahame Grieve (Dec 27 2021 at 02:40):
I guess that the IGpublisher just as well could do it - simply bundling all example instances in a transaction bundle as part of the build process
a typical IG contains a set of incompatible resources that represent different scenarios or transations covered by the IG
Oliver Egger (Dec 27 2021 at 07:10):
Jens Villadsen said:
John Moehrke / Oliver Egger what tools do you guys use for bootstrapping IHE test cases when doing IHE Connectathons? Do you still heavily rely on the textual descriptions in Gazelle, or could you just as well have the need for at least formulat a given state with a certain set of test resources?
During the connectathon vendors can add FHIR resources to a hapif-fhir instance. Test descriptions are done in gazelle, i don't think they need much bootstrapping. In case of MHD there is special tooling support developped with asbestos.
Oliver Egger (Dec 27 2021 at 07:21):
@Jens Villadsen the hapi-fhir CLI command tooling has code to upload FHIR resources using the transaction approach as you mentioned, see ExampleDataUploader.java
Oliver Egger (Dec 27 2021 at 07:27):
i usually just load an implementation guide and do then form the client side the actions I want to do: https://github.com/ahdis/matchbox/blob/main/src/test/java/ch/ahdis/validation/IgValidateR4TestStandalone.java#L55
Oliver Egger (Dec 27 2021 at 07:28):
if somebody has already typescript support for loading implementation guides I would be very interested, without wanting to hijack jens thread ..
Jens Villadsen (Dec 27 2021 at 08:23):
@Grahame Grieve whats 'incompatible resources'?
Jens Villadsen (Dec 27 2021 at 08:29):
@Oliver Egger it looks like the hapi-cli only considers the FHIR releases and not 'random' IGs
Oliver Egger (Dec 27 2021 at 08:36):
Jens Villadsen said:
Oliver Egger it looks like the hapi-cli only considers the FHIR releases and not 'random' IGs
some work left for you to do :grinning:
Jens Villadsen (Dec 27 2021 at 08:42):
Ok - Im doing the tool regardless if anyone needs it
Jens Villadsen (Dec 27 2021 at 09:34):
@Grahame Grieve is this the full list of dependencies returned here (https://github.com/hapifhir/org.hl7.fhir.core/blob/master/org.hl7.fhir.utilities/src/main/java/org/hl7/fhir/utilities/npm/NpmPackage.java#L787) - or do I need to recursively look into those?
René Spronk (Dec 27 2021 at 10:14):
Sounds like something the Firely Terminal toolkit would also support, but as a trainer/analyst I tend not to use command line tools.
René Spronk (Dec 27 2021 at 10:15):
(but then again: I use a Py script to seed a FHIR server with resource examples prior to a training course)
Grahame Grieve (Dec 27 2021 at 10:16):
@Grahame Grieve whats 'incompatible resources'?
content of the example resources represents sets of transactions that imply different things about the state of the data - the overall picture is not always consistent with being treated as a single transaction
Grahame Grieve (Dec 27 2021 at 10:17):
you have to look into the dependencies recursively - that's just the dependencies for this package
René Spronk (Dec 27 2021 at 10:20):
The examples in the IG would have to jointly form a consistent (in terms of references etc, but also clinically) web of resources. That's rarely the case in an IG. But still, it makes sense to have these 'fragments' as instances on a server, for people to play with.
John Moehrke (Dec 27 2021 at 13:49):
Mostly what IHE has been doing is validation with the SD profiles. There is movement to create IGs that are the peer of a definition IGs. These second IGs would be more focused on defining the test scenario. In this way they can be versioned independent of the definition IG, and thus more time can be spent on making, and keeping, them consistent with the testing needs. That is to emphasize that testing needs tend to change over time as it is determined what implementers have tended to not have any problem implementing, vs the specific scenarios where implementers didn't notice a constraint. These IGs can also contain non-conformant resources, that are leveraged by the test plan to be sure the system-under-test behaves properly (proper Postel's Law).
John Moehrke (Dec 27 2021 at 13:51):
As @Oliver Egger points out, we would very much like an easy way to standup a HAPI server instance with a set of IGs loaded. @Jose Costa Teixeira was working with @Jens Villadsen .. right? I presumed this was ready for experimentation.
John Moehrke (Dec 27 2021 at 14:07):
We do have a TestScript based testing for the IHE-MHD IG.
And we are working on integrating Inferno.
@Lynn Felhofer can you provide other details about how IHE-Connectathon prepares (pre-loads servers)?
Oliver Egger (Dec 27 2021 at 14:07):
@John Moehrke the hapi-fhir-jpasever-starter has the capability to load ig's. what we do with matchbox is to load the ig's and the expose a $validate operation, which then can be used with the IHE EVS Client tooling, we did this already in switzerland on our own gazelle instance and are looking how to incorporate that also for future connectathons
John Moehrke (Dec 27 2021 at 14:08):
so is that what matchbox is? I have seen that lately, and couldn't quite figure out what it was.
Jose Costa Teixeira (Dec 27 2021 at 16:05):
@John Moehrke just one thing about the testing need changing more often than the content and being separate - I am very much in favour of not separating those. Otherwise there goes TDD, right?
John Moehrke (Dec 27 2021 at 16:17):
I don't think that separation is a negative to TDD. It is more a recognition that even in pure TDD, the tests continue to evolve as the code is developed. This is exactly why I think it is useful to separate the specification IG from the test IG. This is not to say that I am against them being the same IG, I would love for that to happen. I would also love a perfect world where the tests (happy-path) are written before any constraints. But I don't think it is practical today. I expect that evolution of the test IG that I am promoting is never contrary to the spec IG. The evolution of the test IG would be around the edges, failure-modes, etc. If there is a need to change the spec IG, then one does that and everyone get a revised spec IG.
Jose Costa Teixeira (Dec 27 2021 at 17:34):
My experience tells me specifications and testing criteria belong together. Otherwise where would we put things like Acceptance Criteria?
Jose Costa Teixeira (Dec 27 2021 at 17:35):
If testing evolves at a different pace, sounds like waterfall
Jose Costa Teixeira (Dec 27 2021 at 17:36):
Versioning would also be more difficult if we have different IGs
Jose Costa Teixeira (Dec 27 2021 at 17:42):
fwiw: Happy path testing does not need to be written before specs. But should be published at the same time and versioned together with the specs.
Grahame Grieve (Dec 27 2021 at 18:58):
I could add something to the FHIR toolkit like this
Grahame Grieve (Dec 27 2021 at 18:58):
Jens Villadsen (Dec 27 2021 at 19:35):
Requirements (profiles) should be bundled in the same set (IGs) as its verifications (test instances) at all times, IMHO.
Jens Villadsen (Dec 27 2021 at 19:37):
Test IGs sounds like the wrong way to me
Jose Costa Teixeira (Dec 27 2021 at 21:28):
@René Spronk the examples in an IG would have to jointly form a consistent set of resources - or a set of consistent sets?
Jose Costa Teixeira (Dec 27 2021 at 21:30):
I explain: if an IG puts together different types of content (e.g. a medication IG that mixes supply chain, pharmacovigilance, prescription and analytics) those may be different sets of resources with consistency within the set but not necessarily consistent altogether.
Jose Costa Teixeira (Dec 27 2021 at 21:31):
Aldo resources may refer to intermediate steps (e.g. Task, CarePlan and their evolution) and thus may not be consistent as a snapshot altogether
Jose Costa Teixeira (Dec 27 2021 at 21:33):
@Grahame Grieve I presume this feature in th toolkit would take those resources, make a transaction Bundle and post to the server?
Grahame Grieve (Dec 27 2021 at 21:34):
Well, wouldn’t need to be a transaction
Grahame Grieve (Dec 27 2021 at 21:35):
But otherwise yes
Jose Costa Teixeira (Dec 27 2021 at 21:35):
Question: would it make sense to optionally allow the editor to define their (transaction) Bundles? In which case we can select which ones to upload?
Jose Costa Teixeira (Dec 27 2021 at 21:36):
E.g. Bundle1 to set the server to KnownState1, Bundle2....
Jose Costa Teixeira (Dec 27 2021 at 21:37):
This idea (and it's just a new idea) would mean that the upload logic would be:
Jose Costa Teixeira (Dec 27 2021 at 21:38):
For GUI: if there is no Bundle, show the resource types and allow user to select. If there are Bundles, show those
Jose Costa Teixeira (Dec 27 2021 at 21:38):
For CLI: default to upload all, or if there are Bundles, allow user to select which bundles by ID in the command line?
Jose Costa Teixeira (Dec 27 2021 at 21:39):
@Jens Villadsen for your thoughts on CLI
Jose Costa Teixeira (Dec 27 2021 at 21:39):
(I don't know how much this diverges from your initial idea and I don't know how much Java work this is)
Jose Costa Teixeira (Dec 27 2021 at 21:40):
@Grahame Grieve I can perhaps do some coding on the toolkit once I have it compiling and after you set the direction
John Moehrke (Dec 27 2021 at 23:12):
So the idea is that a bundle could be defined that is a starting-point? Where the IG contains many steps in a workflow, and thus the whole IG would not put the server in the starting-point?
Grahame Grieve (Dec 27 2021 at 23:15):
Those bundles would be tied to example scenarios, wouldn’t they?
Grahame Grieve (Dec 27 2021 at 23:16):
As for toolkit compile.. should be sorted?
Grahame Grieve (Dec 27 2021 at 23:16):
But I’m out at the cricket today…
Jens Villadsen (Dec 27 2021 at 23:31):
CLI-wise I now have the default implementation ready - including recurse lookups
Jens Villadsen (Dec 27 2021 at 23:32):
Whitelist of bundle id's can be added
Jose Costa Teixeira (Dec 27 2021 at 23:57):
I'll look at the toolkit after I manage some digital detox (today wasn't it). I think Bundles could be tied to TestScripts - perhaps also ExampleScenarios but I wasn't thinking of those
René Spronk (Dec 28 2021 at 07:56):
The problem I quite often see is that example resources are always in the 'present tense' (which makes sense: show an active medication or problem), and rarely any historic data (inactive conditions, irrelevant observations from 10 years ago, etc.). But having something to play with, even if not very realistic nor complete, is still better than having nothing at all.
Jose Costa Teixeira (Dec 28 2021 at 11:40):
When we handle testing and dynamic ExampleScenarios and CarePlans we'll surely have to have data at different times, right?
John Silva (Dec 28 2021 at 18:27):
The Firely VonkLoader tool might be considered: https://docs.fire.ly/vonkloader/
(currently it doesn't support authentication methods; hopefully that will be available in the future.)
Jens Villadsen (Dec 28 2021 at 18:29):
VonkLoader is a free, multithreaded tool for Windows / macOS / Linux to upload resources in bulk to any FHIR STU3 server. To install, go to simplifier.net/downloads and log in.
Jens Villadsen (Dec 28 2021 at 18:30):
STU3
Jens Villadsen (Dec 28 2021 at 18:31):
Anyways ... I'm done now with the first version now @Jose Costa Teixeira
Jens Villadsen (Dec 28 2021 at 18:31):
You can run it using eg. docker run jkiddo/ember --igIdAndVersion=ch.fhir.ig.ch-core#2.0.0
Jens Villadsen (Dec 28 2021 at 18:35):
There are 3 parameters: serverBase, igIdAndVersion (mandatory) and loadRecursively. If serverbase is omitted, the total bundle transaction is printed in the terminal - if provided it will send the bundle transaction to the server address. The igIdAndVersionis the IG id and version. loadRecursively is by default false - if set to true, all example instances from parent dependencies recursively are also included in the total transaction bundle
Jens Villadsen (Dec 28 2021 at 18:35):
Let me know what you think. It is FHIR versino agnostic
Eric Haas (Dec 28 2021 at 21:01):
I have done this for USCore. But as in the case of US Core, there are several examples in this (as many other IG's_ where the references are stubs using the display element or "example/org" urls and that may be insufficient for the testing purposes, so I am not sure how useful it would be to be as an IG artifact. You will Likely will need to do some post processing to clean em up.
Jens Villadsen (Dec 28 2021 at 23:13):
@Eric Haas All the samples Ive seen in US core looks fine to me. Have you tried the tool against us core?
Eric Haas (Dec 28 2021 at 23:33):
I have not. However, I loaded the examples successfully using my script. My point is for testing you may need to bulk up the examples with actual working references where they are stubbed in.
Eric Haas (Dec 29 2021 at 00:59):
OK I tried it part way and it creates a transaction using POSTs .. But a couple of issues I had ....
- included some search bundle which causes servers to choke (at least HAPI server).
- includes the text elements (I like to pop them off to make it less bulky)
- includes the id elements even though a POST transaction (doesn't choke on HAPI but may on other servers)
- POSTs instead of PUTs so can't predefine the ids for easier referencing downstream. ( US Core has >100 examples so would need to map the input ids to the the output ids from the operation outcome for searching and testing )
- perhaps could output a table of old and new ids too ?
- allow a PUT option with a unique prefix (like '2021-01-USCore') to allow PUTing so the reference mapping is known and done at the input.
Jens Villadsen (Dec 29 2021 at 07:08):
@Eric Haas - ok, 1 is now done. By default search bundles are now no longer included - they can be by setting includeSearchBundles to true.
Jens Villadsen (Dec 29 2021 at 07:09):
2 - I don't like changing the actual resources in the IG.
Jens Villadsen (Dec 29 2021 at 07:12):
3 - sound like you found yourself some servers that do not follow the spec: http://hl7.org/fhir/http.html#create - on POSTs the server shall ignore the id. If its a cardinal issue, this can ofc be removed
Jens Villadsen (Dec 29 2021 at 07:19):
4 - (US Core has exactly 84 examples, including search bundles - at least according to my tool ;) - named ember). You cannot assume that the server allows client assigned Id's when using PUT's. HAPI allows this if configured so, but doesn't by default, if I recall correctly. Since you cannot assume client assiged Id's is allowed, you instead should rely on the use of identifiers, or - I could add the feature that client assigned Id's is expected and then do it as PUT and as well include a configurable prefix.
Jens Villadsen (Dec 29 2021 at 07:21):
PS - I just overwrote the latesttag on ember
Jens Villadsen (Dec 29 2021 at 17:35):
source can be found here https://github.com/jkiddo/ember/tree/main
Eric Haas (Dec 29 2021 at 18:10):
OK thanks for the follow-up. Still a mapping table that shows new fhir_ids, old fhir_id would be a nice to have output. That's what I did anyway.
Jens Villadsen (Dec 30 2021 at 00:04):
Well - the mapping table is pretty hard to compute - if not impossible. If you have eg. two patient resources that you POST in a single transaction, you cannot deduce from the transaction response which patient ended up with which Id.
Eric Haas (Dec 30 2021 at 01:01):
The order is the same...
Eric Haas (Dec 30 2021 at 01:03):
...the response the server SHALL return a Bundle with type set to batch-response or transaction-response that contains one entry for each entry in the request, in the same order, with the outcome of processing the entry.
(http://hl7.org/fhir/http.html#transaction-response)
Eric Haas (Dec 30 2021 at 01:07):
I meant to say I created this table on the front end of the transaction since I used PUTs. But could create from the backend of the transaction using the OO. ( again is just a thought and may not be worth the effort)
Srinivas Mylarapu (Jan 03 2022 at 18:17):
Hi All,
I installed ibm-fhir-server on ubuntun in AWS from dockerhub (https://hub.docker.com/r/ibmcom/ibm-fhir-server). To do some configurations, I need to get access to server.xml file. But, I don't find /opt/ol/wlp folders etc... Does anyone run into this issue and found resolution? Thanks in advance
Lloyd McKenzie (Jan 03 2022 at 18:24):
@Paul Church
Paul Church (Jan 03 2022 at 18:46):
sorry, not my cloud
Lee Surprenant (Jan 04 2022 at 02:58):
hi srinivas, feel free to reach out on #ibm
David Simons (Feb 15 2022 at 17:27):
John Moehrke said:
... These IGs can also contain non-conformant resources, that are leveraged by the test plan to be sure the system-under-test behaves properly (proper Postel's Law).
How do you mark these non-conformant resources, so you know they will trigger an error/warning/information?
Filename? A tag? A list?
a meta.!profile= or meta.profile!=would be nice, to indicate explicit non-conformance.
Jose Costa Teixeira (Feb 15 2022 at 17:48):
I think declaration of non-conformance would be in the TestScript itself.
David Simons (Feb 16 2022 at 09:00):
Yeah, I am also leaning toward separating Example instances (happy path with meta.profile conformance claims), from strict Test instances (with an expected validation test-result in the test script, as you say) to help get the profiles to be valid _and_ strict (ie. raise info/warnings/errors if and only if desired).
Last updated: Apr 12 2022 at 19:14 UTC
