Stream: implementers
Topic: Testscript for clients
Jose Costa Teixeira (May 08 2020 at 09:37):
Can we use TestScript to say to a system
- send your CapStatement
- create a valid patient with last name Smets (we're not going to tell you how that resource looks like because that's cheating)
- here's an allergy - btw, send us a screenshot of how it looks like on your end
?
John Moehrke (May 08 2020 at 14:29):
you can in cucumber
Jose Costa Teixeira (May 08 2020 at 15:01):
Is there an example TestScript with some scripting in it?
Jose Costa Teixeira (May 11 2020 at 07:17):
Any idea about this?
I want to see if I can use testscript to support testing of clients. I know i can use scripting for the clients, what I'm looking for is how this plays with the TestScript resource
Lloyd McKenzie (May 11 2020 at 19:33):
@Richard Ettema ?
ryan moehrke (May 11 2020 at 19:41):
For the actual requests you expect from the client you can use the same operation/assert structure as for server testing and to test the request for things like validation of the Patient POST you can just use assert.direction to specify request asserts.
As for not giving the tester a proper patient resource, in Touchstone you could just define a invalid patient as the source fixture and describe in the operation.description what you actually want the testers to do.
As for sending the screenshot, I don't believe that there is any way to do that besides again just adding description text for the testers to follow.
Jose Costa Teixeira (May 11 2020 at 21:11):
ryan moehrke said:
For the actual requests you expect from the client you can use the same operation/assert structure as for server testing and to test the request for things like validation of the Patient POST you can just use assert.direction to specify request asserts.
How would this look like in a TestScript for a client?
Jose Costa Teixeira (May 11 2020 at 21:12):
ryan moehrke said:
As for not giving the tester a proper patient resource, in Touchstone you could just define a invalid patient as the source fixture and describe in the operation.description what you actually want the testers to do.
I prefer not to give an invalid resource (what is an "invalid patient"?) but just tell the client "last name should be Smet. Do your thing". I don't want to instruct the client that this should be in name.family.
Jose Costa Teixeira (May 11 2020 at 21:13):
ryan moehrke said:
As for sending the screenshot, I don't believe that there is any way to do that besides again just adding description text for the testers to follow.
Ok, but description text seems short - what I would want is a way to say "ah, here is a script"
ryan moehrke (May 11 2020 at 21:23):
The reason why I said to give an invalid Patient is because you wanted to not tell the tester how the client is set up and if I remember correctly Touchstone (the testing platform I have experience in) requires a sourceId for create operations, and if there is a sourceId fixture testers can see it.
So if you want to not give the tester a valid Patient resource to work off of then you must have an invalid patient (which could literally just be the string "Patient with last name Smets")
ryan moehrke (May 11 2020 at 21:24):
Ok, but description text seems short - what I would want is a way to say "ah, here is a script"
What are you looking to display? I don't believe there is any character limit on String and I'm not sure what "ah, here is a script" is trying to convey..
Jose Costa Teixeira (May 11 2020 at 21:24):
I'd hope there is something in between string and a valid resource.
Jose Costa Teixeira (May 11 2020 at 21:25):
I would like to tell the client "here is where you will find the script for you to run in this step"
ryan moehrke (May 11 2020 at 21:27):
that doesn't seem in scope for anything in the testscript, because if they're reading any part of the test script then they shouldn't have to be pointed to said test script.. That sounds more like something you'd put on a Confluence site
Jose Costa Teixeira (Aug 17 2020 at 12:33):
Back to this,
for 3. I will see if I can need to add an extension to say "in this step, take a screenshot".
for 2. I will think of something. I don't want to maintain confluence pages with my test data
for 1. I still have no idea
Robert Scanlon (Aug 17 2020 at 14:47):
For 1, I would think some kind of extension is required unless it is a requirement of the client in your context to send its capability statement in a specific RESTful FHIR-conformant way. There is no /metadata equivalent for clients that I'm aware of, so you'd have to define it. And then you could use TestScript to test it. That wouldn't make sense to do unless the server is expected to do something with that information though (I can think of some contexts where it would matter, but for most it wouldn't).
Robert Scanlon (Aug 17 2020 at 14:50):
For 2, could you use test.action.assert.expression for the specific value, and maybe validateProfileId to ensure the rest of the resource is valid?
ryan moehrke (Aug 17 2020 at 16:01):
Now knowing that you don't actually want a test runner to be able to interpret and automate the testscript,
I don't see any reason why for 2 you couldn't just use a descriptive txt file loaded into a fixture as your sourceId for the Patient create operation; then the testers could see and read it without any 'cheating' of how it looks like
3 does feel like a situation where you would need an extension, but depending on where/when in the execution you want the user to screenshot you may be able to use an extended code on operation.type to mean that the operation with that 'screenshot' code is not a restful operation but rather a test step for the user (and you could have them POST the screenshot as a DocumentReference.content etc. if you wanted to store it elsewhere)
1 could be something similar to the operation.type 'screenshot' code where you define a post of the client capability statement to whatever validation you are using, bypassing the target fhir system completely.
I don't know any system that could interpret those codes properly today, but if these are test steps you need to do I see no reason why they shouldn't be operation codes.
But yes, as Rob noted you could still use asserts to validate 1 and 2 regardless of how you ask the tester to send it, and if you choose to have them POST the screenshot as well you could do some minimal assertion on that as well.
Jose Costa Teixeira (Aug 17 2020 at 16:18):
for 2 I can use test.action.assert.description or an extension of it.
.expression is a fhirpath which is yucky for the end users
Jose Costa Teixeira (Aug 17 2020 at 16:22):
for posting the screenshot I expect a Binary is sufficient
Last updated: Apr 12 2022 at 19:14 UTC
