Stream: Da Vinci
Topic: Validation tools
Egor Shundeev (Mar 10 2021 at 15:10):
Hi guys, are there any tools to validate server for Da Vinci profile conformance?
Lloyd McKenzie (Mar 10 2021 at 15:37):
There are a variety of tools that can help, but no one tool that will fully test a system. The FHIR validator can test that emitted instances conform to the profiles. There are test scripts in Touchstone that help test server behavior for some of the IGs. Keep in mind that there are multiple 'server' roles and numerous IGs. Most systems won't conform to all roles.
Carie Hammond (Mar 10 2021 at 16:12):
@Egor Shundeev The Touchstone tool does have some tests for many of the DaVinci IGs (not all, but many). They are publicly available at touchstone.com. Depending on the use cases the tests are built for, several have test steps that evaluate against the DaVinci specific profiles. If there is a particular IG you are looking for, let us know and we can point you to specific tests.
Ilya Beda (Mar 12 2021 at 07:39):
Hi @Carie Hammond
We have just started using Touchstone.
We are using FHIRSandbox/DaVinci/FHIR4-0-0-Test and now we are trying to embed Touchstone checks in the CI/CD pipeline.
It is sad to say but we are very unsatisfied with tooling and the ecosystem.
We may use the system in the wrong way or test sites themselves may cause the issue.
I will really appreciate it if you could help us with using Touchstone to meet our needs.
So, test suites from FHIRSandbox/DaVinci/FHIR4-0-0-Test only validate existing resources. Furthermore, I have to manually assign IDs for patients, practitioners, or any other resources to test. Is it possible for Touchstone to create resources on an FHIR server, validate create process and then read them back to validate read API?
Also, it seems that Touchstone is designed to be used as a web app. Is it possible to run the test programmatically via API and get a response in machine readable format?
Carie Hammond (Mar 12 2021 at 15:17):
Ilya,
First, you will want to be using the DaVinci/FHIR4-0-1-Test tests, as the 4-0-0 tests are quite old. We keep them there for reference only.
We have some different kinds of tests, but if you are testing some of the IGs that are mostly centered around the Patient Access API (such as Plan-Net), we do not have creates for test data, as the servers that usually would test those IGs are not usually supporting creates or updates, only reads, etc. We usually point systems in need of data for testing to the Mitre Synthea tool: https://synthetichealth.github.io/synthea/
There ARE tests such as the FHIR4-0-1-Basic and FHIR4-0-1-Advanced tests that have creates and updates of the resources under test. https://touchstone.aegis.net/touchstone/testdefinitions?selectedTestGrp=/FHIR4-0-1-Basic&activeOnly=false&contentEntry=TEST_SCRIPTS or https://touchstone.aegis.net/touchstone/testdefinitions?selectedTestGrp=/FHIR4-0-1-Advanced&activeOnly=false&contentEntry=TEST_SCRIPTS. Users can also create their own testscripts and upload them to Touchstone with an appropriate subscription level, and those tests can include 'fixtures' of the data you want to load as part of the test that can then be read/searched back, updated, etc.
Touchstone is only available as a web application at this time. That said, it DOES have a set of API's for launching and retrieving tests that is very popular with our customers who want to integrate Touchstone into their CI process. See https://touchstone.aegis.net/touchstone/userguide/html/continuous-integration/index.html.
Please feel free to reach out to Touchstone_Support@aegis.net if you want to set up a short live demo and discussion about how to use some of these features in Touchstone.
Ilya Beda (Mar 12 2021 at 16:40):
Thank you @Carie Hammond
Last updated: Apr 12 2022 at 19:14 UTC
