Stream: implementers
Topic: FHIR Sample Data
jsolana (Aug 02 2017 at 09:14):
Hello,
I am performing database load tests and would need thousands or millions of FHIR instances to load.
Do you know where I can download FHIR data?
Thanks
Jose Costa Teixeira (Aug 02 2017 at 15:03):
Hi. You want to mock up data, or consistent data, or realistic data? I guess if it is "just" for load tests, you could use a data generation tool for generating dummy-but-conformant XML/JSON?
Eric Haas (Aug 02 2017 at 15:53):
maybe this is what you are looking for?
https://projectcrucible.org/testdata
Tilo Christ (Aug 02 2017 at 18:22):
I use MiHin PatientGen. Very large, realistic but synthetic data set. Thus no PHI concerns. Available through a Fhir endpoint. https://mihin.org/services/patientgen/
Igor Sirkovich (Aug 02 2017 at 19:33):
You can try https://github.com/synthetichealth/synthea
Jarkko Koistinaho (Sep 22 2017 at 12:53):
I asked related question in HAPI FHIR Google Group: https://groups.google.com/forum/#!topic/hapi-fhir/FfnBk-xwxh0
How it sounds? Would you use FHIR data generator with my requirements?
Jason Walonoski (Sep 22 2017 at 21:02):
The Synthea project generates reasonably detailed FHIR JSON data (STU3 or DSTU2) with Patient, Encounter, Condition, Observation, DiagnosticReport, Immunization, MedicationRequest, CarePlan, AllergyIntolerance, Procedure... any maybe a few others, all cross-referenced appropriately. https://github.com/synthetichealth/synthea
The SyntheticMass project hosts 1+ million of these STU3 records at https://syntheticmass.mitre.org/fhir
Disclaimer: I am a developer on both of these open-source projects.
Abbie Watson (Oct 10 2017 at 21:55):
Thank you for all the work on the Synthea project!
I bookmarked it last year, with the intention of revisiting it after we completed implementing the Argonaut resources and CCD on FHIR. We just wrapped that up, and I was kicking around thinking 'how are we going to populate this thing with FHIR data?' when I remembered the Synthea project.
Looks like we'll be doing a deep dive into Synthea and Synthetic Mass, and doing a bulk upload to stress test the Symptomatic infrastructure. I'll keep you posted!
Jason Walonoski (Oct 25 2017 at 14:27):
Any updates on that @Abigail Watson ? Interested to hear what did/didn't work...
Abbie Watson (Oct 25 2017 at 14:42):
Any updates on that @Abigail Watson ? Interested to hear what did/didn't work...
Oh, I don't have that quick of turnaround time. ;) Hopefully will get to it this month; but end-of-year is more likely. Now that we've got the CRUD UI in place, load testing is queued up as a next major initiative; but there's consulting work and refactoring that's delaying it.
Abbie Watson (Aug 22 2019 at 20:19):
@Jason Walonoski - Hi! I recently received NIH grant funding for a project that has finally allowed me to take a deep dive into Synthetic Mass. So far, it's being a godsend, and I'm generating notes. I noticed that the GitHub repository is archived, though... I take it the project is no longer even in maintenance mode?
Alexander Kiel (Aug 23 2019 at 14:05):
Is Synthetic Mass just superseded by https://github.com/synthetichealth/synthea? I use the latter with success.
Paul Church (Aug 23 2019 at 14:15):
SyntheticMass is a specific dataset generated using Synthea. Perhaps they archived the syntheticmass repo because that project is completed? Ongoing work on the generation framework is in the synthea repo, which looks quite active.
Abbie Watson (Aug 23 2019 at 16:41):
Yeah, after researching a bit, we're probably going to use Synthea. Only missing piece is loading it up into a FHIR server. Does anyone know of a gist or script file for loading the resulting Synthea dataset into a HAPI server or Vonk server or something?
We have a great FHIR Client, but our Server is pretty basic and doesn't even support things like the operations framework or chained queries. I need to get the output of the Synthea utility into a FHIR Server that supports DSTU2 and STU3 that has full chaining and operations framework support.
Alexander Zautke (Aug 23 2019 at 16:46):
We at Firely developed vonkloader for exactly this issue. Have a look here: http://docs.simplifier.net/vonkloader/
Alexander Zautke (Aug 23 2019 at 16:48):
Despite the name, it's not tied to Vonk, works with most FHIR servers
Abbie Watson (Aug 23 2019 at 16:49):
Ah, it reads the Synthea output files, and then does PUT/POST queries into the FHIR server? Yeah, looks like that's what it does. Okay, going to pull out our Windows workstation and see what we can do.
Alexander Zautke (Aug 23 2019 at 16:51):
Yes, and for huge Synthea data sets you can do it in parallel
Paul Church (Aug 23 2019 at 16:52):
SyntheticMass is hosted on Google Cloud Healthcare, which has a bulk import API.
I've been meaning to test vonkloader against GCP, it should work out of the box if it can pass an auth token.
Josh Mandel (Aug 23 2019 at 20:48):
We're also working on a standardized $import API to help with this.
Alexander Kiel (Aug 26 2019 at 09:05):
I have written an alternative to VonkLoader in Golang: https://github.com/life-research/blazectl
It can also upload transaction bundles in parallel. It runs under Windows, macOS and Linux without the need of any SDK's installed.
Jason Walonoski (Aug 26 2019 at 12:43):
The synthetic mass data is still online for download or FHIR access. See https://synthea.mitre.org/fhir-api and https://synthea.mitre.org/downloads. The synthetic mass repo was just the original website used to serve the data, which has been replaced. The data was generated by Synthea, which is still active.
John Moehrke (Aug 26 2019 at 14:02):
These tools should be added to the available servers and test apps https://wiki.hl7.org/index.php?title=Publicly_Available_FHIR_Servers_for_testing
Abbie Watson (Aug 28 2019 at 04:45):
I'm happy to say that we've had amazing success with Synthetic Mass so far, and will probably be moving forward with using Synthea to set up a Synthetic Michigan or Illinois dataset. Thank you everybody for the help and guidance!
One quick question... does anybody know if PDFs and CCDA documents were loaded up into Synthetic Mass website? The Synthea project lists having sample C-CDA docs, and there's a DocumentReference listed in the CapabilityStatement, but I've not been able to get any documents returned. I'd love to be able to fetch even a single PDF or C-CDA from Synthetic Mass before we do the deep dive into setting up our own setup.
Otherwise, the ability to fetch thousands and thousands of synthetic documents has been a game changer, and really made possible getting some more advanced workflows set up. Thank you again!
Jason Walonoski (Aug 28 2019 at 12:13):
@Abigail Watson We no longer have the corresponding C-CDA documents available to download. That was lost when we migrated architectures.
If you run Synthea, you can modify the synthea.properties file to export both FHIR and C-CDA. You should be able to link the two files by patient ID, so if you want to use that for your project, you can do so. See https://github.com/synthetichealth/synthea/wiki/Common-Configuration
Right now, Synthea doesn't output any PDFs, but we're working on doing that for clinical note support. TBD on when that will happen.
Abbie Watson (Aug 28 2019 at 13:55):
Ah, thank you so much. That saves us a lot of time and effort, and we'll adjust our project plans accordingly.
Natasha Singh (Oct 31 2019 at 17:04):
Hi @Abigail Watson were you able to use the vonkloader to load the synthea data? Also did you try loading into a vonk server?
Thanks!
Abbie Watson (Oct 31 2019 at 17:06):
Mmmm... someone else on my team did, I believe. I believe the answer is yes and yes, but I'd need to check and confirm.
Last updated: Apr 12 2022 at 19:14 UTC
