Stream: implementers
Topic: FHIR Database
Muhammad Abubakar Ikram (Feb 22 2017 at 14:17):
I want to ask some questions about FHIR database and storing data in FHIR server's database
1. Is FHIR has a specific database schema? or we can use custom database schema with FHIR?
2. Let's say I have stored a practitioner in FHIR server and a patient in FHIR server now I want to create a relationship between practitioner and patient for example patient with id=1 is belong to a practitioner with id=2? how can I achieve this being under the umbrella of FHIR?
nicola (RIO/SS) (Feb 22 2017 at 14:18):
Take a look at fhirbase, it uses advanced features of PostgreSQL - http://fhirbase.github.io/
nicola (RIO/SS) (Feb 22 2017 at 15:32):
You could link patients with practitioners using http://build.fhir.org/careteam.html resource, using standard *references* approach
Lloyd McKenzie (Feb 22 2017 at 20:15):
@Muhammad Abubakar FHIR is an exchange standard, so it's silent about how data is persisted. There is no official database schema - and there are a variety of relational and noSQL approaches to storing data, including managing references between both local and remote resources.
Muhammad Abubakar Ikram (Feb 23 2017 at 12:36):
@nicola (RIO) Thanks. please tell me, for using fhirbase
1. I just have to stand up a fhir server
2. download and install fhirbase
3. connect fhir server with fhirbase
4. start creating resource, Done
I just have to do this? and I don't have to create the database?
Muhammad Abubakar Ikram (Feb 23 2017 at 12:36):
@Lloyd McKenzie Thank you
nicola (RIO/SS) (Feb 23 2017 at 18:15):
@Muhammad Abubakar fhirbase is not fhir-server - it's just partial implementation of FHIR spec inside database and you could build your own server on top of it. If you are looking for ready to use FHIR servers, take a look at Spark (.net), HAPI (java), Grahame server (delphy) or our commercial cloud server (aidbox.io)
Muhammad Abubakar Ikram (Feb 24 2017 at 06:43):
@nicola (RIO) yea yea understand this. Thank you
wassim (Feb 26 2017 at 08:26):
hello , i have some questions about the database using fhir : 1/ how could i use FHIR database and modify it so it can fit my UML shema of database 2/ i can't find the UML shema of all FHIR in the documentation exemple ( practionner with patient are not linked) 3/ im working with intellij with maven project and in the documentation of hapi fhir they said you have to use the JAR , where could i download this JAR's , because on github i found just source codes , THAAAANKS
Lloyd McKenzie (Feb 26 2017 at 15:21):
Hi @wassim, there is no official FHIR database. FHIR's purpose isn't to define databases, just interfaces. If you want to generate a database schema from the resource definitions, you can, but figuring out the design parameters, indexing, etc. and how best to map the complex structures to database tables is up to you. The tradeoffs around how you break things down will be driven by how you expect your database to be used.
Lloyd McKenzie (Feb 26 2017 at 15:22):
2. There's no UML representation of FHIR that integrates all resources. We've tried and it's just unreadable. Too many things connect to too many other things.
Lloyd McKenzie (Feb 26 2017 at 15:23):
The easiest way to get the HAPI jars is to use Maven. See http://hl7api.sourceforge.net/getting_started.html
Igor Sirkovich (Feb 26 2017 at 17:54):
@wassim , have a look at http://hapifhir.io/download.html and https://github.com/jamesagnew/hapi-fhir/releases
wassim (Feb 27 2017 at 08:42):
@Lloyd McKenzie thanks for your answer , i have my own oracle database , data definitions are not the same as the umd representation of ressources ,1/ i don't understand how could i communicate with another EHR , 2/ now i have started a maven project on intellij , i will add the dependencies , and then connect to a database oracle , just this ? what about the JPA server , tomcat and hibernate that i had to install and configure ?? sorry for all the questions i'm little bit lost
wassim (Feb 27 2017 at 08:43):
@Igor Sirkovich thank you for your answer
wassim (Feb 27 2017 at 09:15):
@Lloyd McKenzie if FHIR doesn't have a database where i'm i gonna use the resources if i had already defined my entities and associations on UML
Grahame Grieve (Feb 27 2017 at 09:23):
you have your own persistent store. and you have the FHIR API - an interface. You have to write some kind of server adaptor that maps from the FHIR API to your persistence model. That's what nearly everybody does
wassim (Feb 27 2017 at 09:51):
@Grahame Grieve okey thanks how do i do that ?
Lloyd McKenzie (Feb 27 2017 at 09:57):
@wassim - You're going to have to do some reading on hibernate. This isn't really the forum to get that sort of help. If you have questions about some particular mappings you're having trouble with, that might be an appropriate question. But the actual writing of the code to link a persistence layer to the reference implementation objects you'll have to manage on your own. (To get a basic sense of how it's done, you can look at some of the open source java servers, but be aware that because you have a legacy data base, the mappings won't necessarily be as straight-forward and some transformation will likely be needed.)
Kevin Mayfield (Feb 27 2017 at 09:59):
Presuming your environment is java. Have a look at this HAPI page http://hapifhir.io/doc_rest_server.html I've followed that page and then built my own providers that access our data using Hibernate.
wassim (Feb 27 2017 at 10:02):
@Lloyd McKenzie thanks , please my second question ...
wassim (Feb 27 2017 at 10:02):
@Kevin Mayfield thank you
Lloyd McKenzie (Feb 27 2017 at 10:03):
The purpose of FHIR is to share data between systems. For example, an iPhone app talking to an EHR system or two hospital systems talking to each other or a hospital system sending data to a regulatory system or clinical trial system.
Lloyd McKenzie (Feb 27 2017 at 10:05):
FHIR is not focused on how data is stored. Every system stores things differently. Our purpose is to define a standardized way of sharing the data regardless of how it's stored. Every system maps their particular data store (whether relational or noSQL or whatever, using whatever data structures they happen to have) to and from the FHIR standard so they can send and receive data exchanged with other systems.
Lloyd McKenzie (Feb 27 2017 at 10:06):
So your first question should be: "Who will I be sharing data with?" If the answer is no-one, you may have no need of FHIR, particularly if you've already got a database defined.
wassim (Feb 27 2017 at 10:06):
so can i build my own EHR java application with a relational database ,, and then use hapi FHIR to communicate with the hospital for example ?
Lloyd McKenzie (Feb 27 2017 at 10:08):
Yes. But you'll need to write the code that converts between your native java object representation and HAPI's FHIR-based object representation
wassim (Feb 27 2017 at 10:10):
okey , what i wanted to do is to build my database FHIR-BASED object representation and make it fit to what i need , so then i will not be obliged to write the code that converts , what do you think
Lloyd McKenzie (Feb 27 2017 at 10:11):
If you're building your database from scratch, that can work. You might take a look at one of the existing open source FHIR servers as a starting point
Lloyd McKenzie (Feb 27 2017 at 10:11):
http://wiki.hl7.org/index.php?title=Open_Source_FHIR_implementations
Jose Costa Teixeira (Feb 27 2017 at 10:11):
@wassim building a storage from the standards is not a bad idea (you can build upon the outcome of several discussions and save yourself many design flaws), but I am afraid if you are doing it to save implementation effort, then you must take care.
Kevin Mayfield (Feb 27 2017 at 10:12):
You'll still need to write a fair bit of code. I'd started with HAPI's JPA server but found I needed to store the data in more traditional SQL tables, so although my underlying database structure mirrors FHIR I still need to code many of the queries (which is simplified by using hibernate)
Jose Costa Teixeira (Feb 27 2017 at 10:13):
either the scope is really under your control, or eventually that may backfire. I do think that aligning a design with FHIR by default is a good point
wassim (Feb 27 2017 at 10:14):
so you're saying work step by step , developping an EHR with java and sql database , and when the project is finished and ready , preceeed with hibernate , jpa server of hapi fHIR ??
Kevin Mayfield (Feb 27 2017 at 10:15):
I'd agree, starting with FHIR model works really well and fits majority data storage requirements well - mostly for administrative resources but in a number of areas it doesn't (e.g. terminology services)
wassim (Feb 27 2017 at 10:16):
i'm working on a military EHR based on their paper health record , so the informations are differernt a little bit
wassim (Feb 27 2017 at 10:16):
paper record i mean
Lloyd McKenzie (Feb 27 2017 at 16:32):
If you're implementing FHIR, just recognize that it's an interface. You will have to map and some of the mappings won't be straight-forward. Whether you base your persistance model on the paper forms and do the mapping in code between persistance and the interface or base your persistance on FHIR and do the mapping between your database and your user interface, you're going to be doing the mapping analysis and writing translation code either way. You'll need to figure out what approach is most advantageous for your circumstance.
wassim (Feb 27 2017 at 18:01):
@Lloyd McKenzie i found on github a project wich name is (fhir-base) i've found in it sql files and a postagre sql database , i could use this database and modify it to fit my project ??? it's gonna be fhir model , so it's won't take me long to make it work with hapi-fhir for the communication with the hospital later ??
Kevin Olbrich (Feb 27 2017 at 18:45):
@wassim One thing to consider is how you will migrate to a newer version of FHIR when it's released and also how you will communicate with servers running a different version of FHIR.
wassim (Feb 27 2017 at 21:12):
@Kevin Olbrich yes this is important too , for me this is the first time im trying to use fhir , so i've decided to build my database a fhir object model based database , so that i won't do a big coding to mapping data after that
wassim (Feb 28 2017 at 09:20):
@Lloyd McKenzie i've decided to devlop my application with javaFX and relational database oracle , and when it's finished i will do the data mapping and use hapi fhir , you think i should learn hibernate in same time ??
Lloyd McKenzie (Feb 28 2017 at 16:07):
Hibernate is a technology that helps map between object representations and relational representations. It's certainly not mandatory to use, but you may find it helpful.
Abbie Watson (Mar 01 2017 at 02:01):
If a JSON data store is used (such as MongoDB), FHIR resources can be used as database schemas. It's fabulously productive. We use Meteor, a full-stack javascript platform, and happily use FHIR on the server, client, and database. It allows different apps to seamlessly connect to each other, using the JSON datastore to seamlessly cache what's happening on the wire. The major downside to this architecture is that it requires a denormalized NoSQL data architecture.
Grahame Grieve (Mar 01 2017 at 02:20):
scanned it quickly - it looks nice, but it seems that you didn't get to the big gotcha around resource identity...?
Lloyd McKenzie (Mar 01 2017 at 03:37):
@Grahame Grieve - response to wrong thread?
Grahame Grieve (Mar 01 2017 at 03:42):
oh yes - it was in response to the right person though ;-)
nicola (RIO/SS) (Mar 01 2017 at 05:18):
New SQL standard (released couple of month ago) includes JSON type and partially supported by Oracle, PostgreSQL, MySQL, MSSQL - i think, in a year all of them will support essential part of specification (at least standard is mostly based on Oracle impl. and PostgreSQL team work agressivlly to support it to) - https://www.iso.org/standard/63556.html
Abbie Watson (Mar 01 2017 at 06:10):
Yup. JSON type will be invaluable in interoperability efforts; for both writing incoming messages to disk; and prepping outgoing messages. It will definitely keep MySQL, Oracle, and others in the game for quite a while; but they'll still have clustering and scale out issues after the first silo, due to their CAP theorem architecture. In particular, their support of map/reduce over distributed JSON collections will be precisely as challenging as Mongo's lack of a JOIN operator. It really all boils down to whether we're talking about a FHIR database or a FHIR data warehouse.
Binu DGIT (Mar 01 2017 at 06:29):
We are thinking about National Ehealth repository in Oman based on FHIR standard.
As per the current requirement, the architecture should be provisioned to receive
average of 500,000 messages daily basis. We are using Oracle db to store the messages
and it is stored as clob data. We made a setup of another Oracle DB to extract
relation structure out of the json messages in order to serve the web and mobile portal clients.
I am skeptical about this data architecture to manage the future requirements.
Bigdata arachitecture scale better in this case?
nicola (RIO/SS) (Mar 01 2017 at 06:41):
@Abigail Watson Health IT mostly is not big data and not high load :) One instance of PostgreSQL could keep medical records of small country without any problems. But with NoSQL you are without transactions and sql and this is paintfull. And people who do not understand this trade-offs - after write articles like this http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/
nicola (RIO/SS) (Mar 01 2017 at 06:44):
At least for Heath IT, i think, "C" is very important.
Abbie Watson (Mar 01 2017 at 06:52):
@Binu DGIT At 500k messages per day, that's like 186m messages per year. So a 32 bit index would last about 23 years, if the volume of messages stays consistent. Now figure your average FHIR record size.... is it 1kb or 10kb? If the former, you're talking 200GB a year; the later would be 2TB a year. The 10kb assumption results in a number that's just at the lower end of when you need to start clustering disks. That's per year, though. So, over 20 years, you'll generate 40TB of data. Honestly, you could run it either way.
Abbie Watson (Mar 01 2017 at 06:55):
@nicola (RIO) - You're not working with genomics data or biometrics, are you? :) You might be surprised how many healthcare use cases where convergence-to-consistency is an acceptable trade-off for availability and scale.
nicola (RIO/SS) (Mar 01 2017 at 07:00):
@Abigail Watson i agree about genomic and imaging - but this is also not use case for document databases. @Binu May i ask you what kind of 500k messages do you have?
nicola (RIO/SS) (Mar 01 2017 at 07:06):
@Binu DGIT in high load messages processing now is very popular persistent queues like kafka.
Binu DGIT (Mar 01 2017 at 07:18):
@nicola (RIO) We have almost 300 health institutions varying from health centres, poly clinics to bigger hospitals sending FHIR messages to central repository based on their patient visits everyday. Initially we work with hospitals under govt. later to extend to the private and other non-govt institutions. There are 14 FHIR resource types selected for communication. So based on patient visits in the hospitals, average 5Million FHIR messages are expected (correction from the earlier statment 500K). As I explained earlier, right now we are using Oracle in database level, Apache camel and Active MQ for message processing. Apache kafka is a good suggestion to be explored again..
nicola (RIO/SS) (Mar 01 2017 at 07:20):
Typical architecture for such systems is get raw messages into Kafka, then feed from Kafka into RDBMS, analytical databases like vertica, timeseries DBS like druid and/or do some stream processing. Kafka could handle millions messages per second.
nicola (RIO/SS) (Mar 01 2017 at 07:26):
Because Kafka is persistent - you could reprocess or connect new subscribers without big pain - so you will be able add new databases downstream. Most of analytical guys usually do some nightly reprocessing - to fix daily inconsistency, because lost or reordered messages
Binu DGIT (Mar 01 2017 at 07:33):
Thats interesting. Since we are considering asynchronous messages Kafka would be a better option but I am not much familiar about time series db like druid and all. Infact we have Please find the attached snap tells you the current architecture in place.MessageArhiecture.JPG
with the current architecture itself, I have a problem in managing the retry Q.
nicola (RIO/SS) (Mar 01 2017 at 07:55):
Looks quite reasonable ;) we've designed architecture for very similar use case for Saint-Peterburg government for ~300 EHR integration. I hope, I will share it with community. The core point in our design was some inversion of control: EHR generate log of all internal changes encoded as FHIR transactions and with increasing transactions counter. Central system polls and merges all EHRs logs and provide it's own TX log for backward integration.
nicola (RIO/SS) (Mar 01 2017 at 08:00):
The good property of this design - that everything is async and persistent - so we have some time travel (reprocessing, node fails ) ability to fight with a lot of problems in such a distributed system.
nicola (RIO/SS) (Mar 01 2017 at 08:02):
Also this architecture is more sutable to create "really" distributed system, i.e. without critical dependency from central integration service at all - for example one EHR could feed tx log of another to sync.
Binu DGIT (Mar 01 2017 at 08:19):
@nicola (RIO) It would really appreciated if you can extend the details of the above mentioned project. Reprocessing to manage failed messages is a critical task..
Radu Craioveanu (Mar 07 2017 at 20:54):
@Lloyd McKenzie you are mentioning some noSQL DBs. Can you please point me to an implementation using MongoDB for a Java platform. We are currently using HAPI FHIR, but the HAPI JPA version is using RDBMS underneath. I know Furore is using Mongo for a .NET implementation, but I am hoping we can find something for JPA. (question also for @James Agnew )
James Agnew (Mar 07 2017 at 21:29):
I'm not aware of any currently..
Out of curiosity, why do you feel that Mongo would be better/preferable to using a relational database? All of my tests have shown that performance is similar, and you get the added advantage of atomic transactions across multiple documents, referential integrity, etc. when you use an RDBMS like Postgres or MySQL
Radu Craioveanu (Mar 07 2017 at 21:43):
@James Agnew , @Satya Hi James, you have been also chatting with some other folks on my team re Mongo and Hibernate on the HAPI FHIR google group. The reason I am interested in Mongo is that we have built a multi data center highly available Mongo infrastructure that is relatively inexpensive as compared to our Oracle licensing. It is not a simple task for us to introduce a new DB, such as postgres. It took me years to make Mongo an enterprise acceptable solution. I could go with Oracle with the HAPI FHIR JPA implemetation, but I am trying to avoid that path. If you can help out Satya with the Hibernate issues and questions, it may be sufficient for us to use most of your stack and just plop it on top of Mongo. Do you think that is doable?
James Agnew (Mar 07 2017 at 21:54):
That does sound theoretically possible, but it's certainly not how the library works today so it would require a fair bit of engineering effort.
Radu Craioveanu (Mar 07 2017 at 22:13):
Ok. We already have implemented about 40 resources using the HAPI FHIR in OSGI containers, on top of Mongo for those resources that do not currently exist in our EHR (like CarePlans, Tasks..) and require Read/Write and on top of MS SQL, ORACLE and/or HL7 for those resources that data already exists in our EHR (we use Siemens SOARIAN, now part of CERNER) (observations, medications, encounters...). We are looking for ways to accelerate the development cycle by jumping onto something like the JPA so that our developers can focus more on building out integration components between the existing EHR and the JPA servers for those cased where the EHR needs to be updated with data that is written to the JPA server as well as updating the JPA server with data as it it being updated in the EHR in real time (orders, lab results, ...). Maybe I am missing something, so I wonder if we can get on a call at some point to better explain what we are trying to do and how we can get there the most efficient way.
Abbie Watson (Mar 07 2017 at 22:26):
+1 for Mongo. Former Cerner/Oracle admin here. Mongo has better approach to CAP theorem for use-cases that require clustering scale out and can accept convergence-to-consistency. Particularly relevant with biometrics, wearable devices, interfaces, monitoring, etc.
James Agnew (Mar 07 2017 at 22:33):
Sure thing, happy to jump on a call if that's useful. Shoot me an email at jamesagnew@gmail.com
Jack Anelson (Mar 08 2017 at 18:28):
@Peter Hendriks Could you please provide the required steps to build hapi-server using mysql.? I have gone through your site.Its great(http://hapi.simplifier.net)
Mention the link to download HAPI packages and steps to deploy it in the server .Thanks in Advance
James Agnew (Mar 08 2017 at 22:55):
There are some instructions on how to do this here: https://www.openhealthhub.org/t/howto-build-a-health-database-and-fhir-api-server-in-15-mins-using-open-source/155
(Shoutout to @Kevin Mayfield for these!)
Peter Hendriks (Mar 09 2017 at 10:58):
Hi Jack,
Peter Hendriks (Mar 09 2017 at 11:27):
@Jack Anelson
Assuming that you just want to have the HAPI-FHIR JPA server example that stores it's data in MySQL instead of (standard) Derby.
Personally I used Netbeans IDE, . Note that you need a basic servlet container so I suggest that you setup that first (Tomcat/Jetty/Glassfish etc.). Of course you need MySQL running with proper user rights.
I just cloned the enire HAPI-FHIR repo from github and opened the HAPI FHIR JPA Server project. Then I edited the dataSource() in FhirServerConfig.java class so that it would use MySQL over Derby and added the MySQL dependencies to the POM file.
Then I ran the project and noticed that resources where arriving as BLOB in MySQL when posted to the JPAServer example.
public DataSource dataSource()
{
BasicDataSource retVal = new BasicDataSource();
//retVal.setDriver(new org.apache.derby.jdbc.EmbeddedDriver());
//retVal.setUrl("jdbc:derby:directory:target/jpaserver_derby_files;create=true");
try
{
retVal.setDriver(new com.mysql.jdbc.Driver());
retVal.setUrl("jdbc:mysql://localhost:3306/db_hapi");
} catch (Exception e)
{
e.printStackTrace(System.out);
}
retVal.setUsername("myusername");
retVal.setPassword("mypassword");
return retVal;
}
Jack Anelson (Mar 09 2017 at 18:28):
Thank you @Peter Hendriks I have done...:raised_hands::big_smile:
Last updated: Apr 12 2022 at 19:14 UTC
