Stream: TestScript Resource
Topic: Testing Integration with the FHIR Spec
Richard Ettema (May 19 2021 at 12:08):
Per our Connectathon 27 Session 2 discussion, this topic will focus on how we can integrate testing requirements into the FHIR specification. The focus will be on the RESTful API of the FHIR operations.
Initially we will look to define test definitions, TestScripts, that provide appropriate coverage of the FHIR operations starting with the read operation. How we collect and present these test definitions is TBD but one possible approach is within a "Testing" implementation guide.
Jose Costa Teixeira (May 19 2021 at 12:18):
apologies for not being participative as I wanted. I just wanted to say that we may be adding a "Testing" tab to each ImplementationGuide artifact page.
Jose Costa Teixeira (May 19 2021 at 12:19):
So if you have a StructureDef, or Operation, or CapStatement, you would have another tab which explains "how this is tested"
John Moehrke (May 19 2021 at 12:20):
that does not seem like the right perspective. an artifact is already specifically defined. The TestScript is needed when one puts that artifact into some context, interaction, actor, etc..
John Moehrke (May 19 2021 at 12:23):
or are you thinking that this artifact based test tab would be just an up-pointer to the various places in which that artifact is engaged in a testable situation?
Jose Costa Teixeira (May 19 2021 at 12:23):
right, so the interaction will have its TestScript, and the actor as well
Richard Ettema (May 19 2021 at 12:24):
No worries. We actually started out our Session 3 discussion along those lines. Yesterday, I put together a quick "Example IG" that shows a new high level "Testing" section using the existing igpublisher capabilities for FHIR 4.0.1. I tried to build it using FHIR 4.6.0 so I could use the new TestScript.scope element but had issues with the downloaded 4.6.0 packages.
I'll get it publicly available "soon" so we can start using it as a straw man.
Jose Costa Teixeira (May 19 2021 at 12:24):
I used StructureDefs because those are more relatable. We are not high enough in the Zachman framework to define interactions
Jose Costa Teixeira (May 19 2021 at 12:25):
I think "Testing" should not be only at the level of the IG
Jose Costa Teixeira (May 19 2021 at 12:25):
(hence the tab)
Richard Ettema (May 19 2021 at 12:25):
Agreed. Had to start somewhere.
Jose Costa Teixeira (May 19 2021 at 12:26):
right, and I was focusing on the mechanism to add the tab, so I didn't worry much about the content
Richard Ettema (May 19 2021 at 12:26):
Once I get the igpublisher working with FHIR 4.6.0, I'll start to play with the "Testing" tab at the artifact level.
Jose Costa Teixeira (May 19 2021 at 12:26):
(once we manage to have such a tab)
Jose Costa Teixeira (May 19 2021 at 12:26):
we need to define an extension
Richard Ettema (May 19 2021 at 12:27):
Right. :wink:
Jose Costa Teixeira (May 19 2021 at 12:27):
because for R4, "scope" was not there
John Moehrke (May 19 2021 at 12:43):
an example that I have is the IHE-MHD implementation guide that has a narrative page 'test plan' that references the test plan, test procedures, and test tools. There is no computable linkage, just narrative.
This said, I presume that it is fully understood that any structureDefinition is already understood as a testing artifact itself.
John Moehrke (May 19 2021 at 12:44):
so it seems to me that the obvious thing a testing tab on a structureDefinition can say is... "here I am".
John Moehrke (May 19 2021 at 12:45):
the obvious thing a testing tab on a valueSet would say... "here is where I am used"... which is already said on that artifact page
John Moehrke (May 19 2021 at 12:46):
this is why I think the testing in an IG would be more at the actor-interactions level, not at the artifacts level.... This actor-interactions level is not always clear, that is indeed a problem.
John Moehrke (May 19 2021 at 12:47):
here is the MHD page http://build.fhir.org/ig/IHE/ITI.MHD/branches/master/testplan.html
John Moehrke (May 19 2021 at 12:48):
which points at the NIST test tool, which has TestScript in it.. but the MHD ig does not specifically contain them (one reason is that the tests are currently written to the previous version of MHD, and not this brand new one)
Jose Costa Teixeira (May 19 2021 at 13:10):
the thing with the narrative page is that it is not a SMART requirement
Jose Costa Teixeira (May 19 2021 at 13:11):
the testing should be on any "testable" artifact.
Jose Costa Teixeira (May 19 2021 at 13:11):
this can mean several resource types
Jose Costa Teixeira (May 19 2021 at 13:12):
a transaction should be a testable artifact
Jose Costa Teixeira (May 19 2021 at 13:12):
I'd split that testplan page
John Moehrke (May 19 2021 at 13:12):
totally agree
Jose Costa Teixeira (May 19 2021 at 13:12):
there's quite some content in that page, right? tools, artifacts..
John Moehrke (May 19 2021 at 14:23):
Given that TestScript is part of FHIR-I responsibility... this makes it quite noisy for people that want to contribute to TestScript topics, as they need to join FHIR-I and deal with all the other responsibilities of FHIR-I... could testing be created as a formal sub-workgroup off of FHIR-I? I seem to recall some sub-workgroup concepts for mailing lists and calendar.
Jose Costa Teixeira (May 19 2021 at 14:31):
if you are suggesting to split Testing approach vs TestScript resource, that would make sense
Jose Costa Teixeira (May 20 2021 at 13:57):
pinging @Bert Peters
Last updated: Apr 12 2022 at 19:14 UTC
