Stream: implementers
Topic: testscript executor
Jens Villadsen (Sep 17 2020 at 08:52):
what open source clients are available to execute FHIR testscripts?
Jose Costa Teixeira (Sep 17 2020 at 13:24):
I'm interested in that too
Lloyd McKenzie (Sep 17 2020 at 14:48):
I think Crucible is open-source? Not sure about Inferno (and not sure if Inferno is driven by TestScript)
Robert Scanlon (Sep 17 2020 at 15:18):
Crucible's plan_executor (which is essentially its test runner) can execute STU3 TestScripts, but we hadn't finished the R4 implementation. As you can probably tell by the lack of activity, MITRE isn't currently funding development of Crucible.
Robert Scanlon (Sep 17 2020 at 15:28):
Inferno is not currently driven by TestScript.
Jens Villadsen (Sep 18 2020 at 07:33):
The idea making a 'x'Unit test runner out of FHIR Testscripts also crossed my mind. But it more sounds like very few people actually uses testscripts
Grahame Grieve (Sep 18 2020 at 09:58):
it's circular. there's no many implementations cause no one uses it and no one uses it because there's no implementations
Grahame Grieve (Sep 18 2020 at 09:58):
it's circular. there's no many implementations cause no one uses it and no one uses it because there's no implementations
Jose Costa Teixeira (Sep 18 2020 at 10:44):
We have a colleague looking at generic "test planning and execution artifacts" and hopefully that will be somewhat mappable to TestScript.
Jose Costa Teixeira (Sep 18 2020 at 10:47):
this will allows us to either make our test execution (what is execution? also test reporting?) scripts or just feed the FHIR testscripts to the tool of choice
Jens Villadsen (Sep 18 2020 at 11:05):
Ideally (for me) I would prefer a tool that would convert Testscripts to (java) unittests. From that point, I would be able to throw all kinds of tooling after the actual execution and what not
John Moehrke (Sep 18 2020 at 11:51):
I understand Inferno is open-source, but does not yet support TestScript. @Reece Adamson @Reece Adamson
John Moehrke (Sep 18 2020 at 11:53):
IHE is looking at using Cucumber as an overall orchestration of test plans and scripts, using a FHIR based test tool for FHIR specifics.
Jose Costa Teixeira (Sep 18 2020 at 11:58):
gherkin/cucumber was part of our investigation (in BE). IIRC we found that we needed something broader
Jens Villadsen (Sep 18 2020 at 11:58):
@John Moehrke cucumber/gherkin is usually on a quite higher abstraction level so I don't see a natural bridge to testscripts
Jose Costa Teixeira (Sep 18 2020 at 11:59):
Gherkin as a whole is just a language to express things that need to be tested. We are interested in running the execution from a script
Sandy Vance (Sep 18 2020 at 13:05):
Touchstone.com is Open Access and hosts a ton of TestScripts, including the Da Vinci scripts to meet the new CMS Interoperability Rules. You can run up to five test executions (which can include multiple scripts) in the open free lane. :-)
Todd Cooper (Sep 18 2020 at 15:07):
FWIW - The Gemini #SDPi+FHIR group has been looking at using Gherkin (and possibly Cucumber Studio) to capture detailed use case / scenario requirements that can eventually be mapped to test tooling configuration & execution; possibly leveraging ReqIF as well.
Alexander Kiel (Sep 18 2020 at 16:57):
I also started an implementation of TestScript execution in a command line tool written in Go. But I never finished it. I actually like the idea of test scripts but as Grahame said, it's a chicken-and-egg situation.
Ryan Harrison (Feb 06 2021 at 06:27):
@Todd Cooper @Jose Costa Teixeira Has there been progress on "FHIR Gherkin", e.g. a Cucumber implementation using SUSHI/fsh?
@John Moehrke Discusses in his blog: https://healthcaresecprivacy.blogspot.com/2021/02/from-implementation-guide-to-ihe.html
Re-posting my blog comment for context
Thanks John, for yet another informative post! I agree that Gherkin is great high-level BDD DSL for (1). Are you aware of a FHIR-specific Gherkin test runner for (3)? If not, there may be a gap, in that someone would need to write executable tests based off the Gherkin in (1) [^1]. Partial duplication and an invitation for (human) error, drift and incomplete test coverage.
Since Cucumber is already implemented many languages, I can imagine a sushi/fsh executor for Cucumber. One writes sushi/fsh Gherkin which is executable by Cucumber.
I found one mention of this approach in the FHIR chat, but as of yet no implementation.
https://chat.fhir.org/#narrow/stream/179166-implementers/topic/testscript.20executor[^1] Alternatively, are you of the opinion that there is no gap. That the human-readable Gherkin is _intended_ as non-executable contract between the Actors and the IG authors. That the executable tests need to be written separately in (3) is a feature not a bug: https://news.ycombinator.com/item?id=10194242
Last updated: Apr 12 2022 at 19:14 UTC
