Stream: implementers
Topic: FHIR and EventSourcing
Håkan MacLean (Nov 18 2020 at 15:52):
I'm attending the DevDays conference and one topic that I would love to know more about is combining FHIR with Event Sourcing. We are looking at using event sourcing to communicate between our micro services and it would be nice to integrate FHIR into this. Does anyone know of any previous sessions or resources about this? I'm thinking someone should have done this before :) Ping content lead @Ward Weistra :)
Lloyd McKenzie (Nov 18 2020 at 16:03):
Can you provide a bit of information about what "Event Sourcing" is?
Håkan MacLean (Nov 18 2020 at 16:09):
Lloyd McKenzie said:
Can you provide a bit of information about what "Event Sourcing" is?
It's a design pattern. Instead of classical CRUD, where we mainly model and persist the current state, we model everything through a series of events. Services interested in knowing about the say, the current state of a Patient X in the PatientService, would not query the PatientService for this information, but instead listen to all the events PatientService publishes about the changes about Patient X. If you Google you will find tons of resources.
Grahame Grieve (Nov 18 2020 at 16:43):
plenty of implementers are doing this. We've occasionally asked the community whether we should standardise something about this, but most users have a framework they are or want to use, and don't want us to get in the way by making rules about how it's down
Håkan MacLean (Nov 18 2020 at 16:54):
I was just wondering if anyone knew of any previous presentations or people who have implemented this combination. I tried searching here https://www.devdays.com/events/ but could not find anything useful. For example I was thinking if anyone had any experiences using the FHIR Event Resource as the actual event object in the Event Sourcing world.
Grahame Grieve (Nov 18 2020 at 16:56):
Event is a design pattern rather than an actual resource type.
Grahame Grieve (Nov 18 2020 at 16:57):
all the uses I've seen are using some underlying event framework that provides infrastructure, flow control, etc.
Rik Smithies (Nov 18 2020 at 21:36):
Can someone say what the advantages of this pattern are, and for what healthcare applications?
The current state of an EHR, for instance, is already an accumulation of all the events that lead to it, most of which are written once and never updated. It doesn't tend to be necessary to view the data at some previous point in time.
Daniel Thomson (Nov 18 2020 at 23:45):
Wouldn't FHIR subscriptions cover this? https://build.fhir.org/subscriptions.html
Kevin Mayfield (Nov 19 2020 at 06:46):
The ones I've seen tended to have a requirement for resource and CRUD status. I suppose a transactional stream?
In v2 for patient event streams we used ADT and so would be tempted to look at da vinci alerts as a potential solution.
Kevin Mayfield (Nov 19 2020 at 06:48):
In our hapi cdr we output a CRUD plus resource stream for replaying updates to another fhir server.
Rik Smithies (Nov 19 2020 at 08:42):
I’m no expert, hence the question, but I think that a stream of interop messages is not the same as event sourcing. V2 messages are an exchange mechanism, but each one causes state to be saved into a database, and then the message is effectively thrown away, and the database is the master. You do keep the messages just in case, and replay in the event of having to roll back etc, but the messages are not the primary information. I assume that event sourcing keeps the individual system changes e.g. field changed value from X to Y (which may be caused by the messages, but are not the same thing as the messages) and uses those as the primary store.
Ward Weistra (Nov 19 2020 at 08:58):
@Håkan MacLean If this is PubSub (https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern) I would agree with @Daniel Thomson that FHIR Subscriptions should probably cover that?
@Gino Canessa is speaking about that on Friday: https://whova.com/portal/webapp/hfdn_202011/Agenda/1252021
Myles Bell (Nov 19 2020 at 14:47):
I am working with a similar event sourced pattern currently. Micro services publishing events, translated to fhir and then stored. I think the first learning I had was that the mircoservices themselves need to at least have data models that assign ids to all entities that are resources in fhir, otherwise you have trouble with updating/deleting fhir resources. Or all services commit to fhir of course.. I would like to possibly do a talk about this in the future but I dont know of any materials to point you to right now
Håkan MacLean (Mar 09 2021 at 20:17):
I know the deadline has already passed, but did you submit anything for the upcoming DevDays @Myles Bell? Would be super interested in hearing about it. https://www.devdays.com/june-2021/Call-for-Presentations/?utm_campaign=devdays-us-2021-call-for-presentation-reminder&utm_medium=email&utm_source=insightly
Myles Bell (Mar 13 2021 at 15:26):
Yes I did submit a proposal, so hopefully will get chance to talk about it at the conference @Håkan MacLean
Håkan MacLean (Mar 13 2021 at 15:33):
looking forward to it!
Vibin_chander (Mar 15 2021 at 04:54):
The Event-Driven and Event Sourcing are different but it depends on how we use. So, In Event-driven mechanism basically the events don't travel, they just occur. Whenever a change happens in a state that's a event. In Event sourcing a state is stored as sequence of events.
Doctor leaves the clinic. His presence (state) changes from available to not available is an event. But tracking the encounter of a patient over a period of stay is not event, but event sourcing can store this as sequence of encounters. So, it completely depends on the usage. A series of encounters over a period of time can be managed through Episode of care resource as well, if you want it event-driven.
Håkan MacLean (Apr 26 2021 at 10:02):
Myles Bell said:
I am working with a similar event sourced pattern currently. Micro services publishing events, translated to fhir and then stored. I think the first learning I had was that the mircoservices themselves need to at least have data models that assign ids to all entities that are resources in fhir, otherwise you have trouble with updating/deleting fhir resources. Or all services commit to fhir of course.. I would like to possibly do a talk about this in the future but I dont know of any materials to point you to right now
Hi! Picking up this old thread again. I understand your problem and I was recently thinking about the same thing. However, I was thinking about solving it at the other end.
I think it would be nice if the publishing services did not need to know the FHIR ID of the objects that has been generated from its data (and it should be the FHIR server creating the IDs of its resources anyway, if I understand the standard correctly). Instead I was thinking of having the FHIR server being able to map "Originating ID" to FHIR Resource ID.
I was thinking of different ways of solving this. One would be to put the info in the Resource.identifier field, but this doesn't seem like a perfect match since the FHIR object might be only be a subset of the Originating object or combination of multiple other objects from other microservices. Another idea that seems promising is using a Bundle and also provide a FHIR Provenance Resource, where you encode this link to the Originating object. Did you consider any of these options?
Lloyd McKenzie (Apr 26 2021 at 17:27):
If you're creating an object on a target system, there may not be a persisted resource in the source environment. If there is, then conveying a Provenance to establish that linkage is reasonable. That's different than presuming systems will automatically map the identifiers though.
Håkan MacLean (Apr 28 2021 at 14:03):
In our case, and I suspect in @Myles Bell case as well, we have a bunch of microservices that have their own persistence, and we want them to share medical data via FHIR events. So in this case there will always be a persisted resource in the source environment.
Slight side track regarding Provenance: I was thinking that if you do a Create (POST) towards a FHIR server with a Bundle containing Resource A and Provenance pointing to this resource, how should the Provenance object point to the Resource? The FHIR ID is not set yet, so I suppose the Source Service would just make up a (temporary) ID to be used as an internal reference in the context of the Bundle?
Josh Mandel (Apr 28 2021 at 14:15):
Yes; see http://hl7.org/fhir/bundle.html#references and urn:uuid examples for resolving references via Bundle.entry.fullUrl
Myles Bell (Apr 29 2021 at 13:28):
One would be to put the info in the Resource.identifier field
This is the right approach to me, a business identifier should identify the underlying concept consistently across all contexts. All updates are done using the identifier. Generating a separate FHIR ID makes sense but doesnt stop there needing to be these unique identifiers.
subset of the Originating object or combination of multiple other objects from other microservices
Big picture I would avoid this if possible, complicates the architecture and ownership of data. Know that doesnt help in reality when the business needs it... If its a subset then usually it will be different resource types so the ids are still unique, if not you can add extra info to the system of the identifier. Combinations should contain a unique id you can use.
using a Bundle and also provide a FHIR Provenance Resource
Provenance is very useful, was going to include this in dev days talk. Though we use this more to track the originating service/domain/entity rather than for identifiers. We started using a bundle always including provenance, now we actually store the provenance alongside every resource rather than needing reference. Diverging from fhir spec of course but is easier (both logically and performance wise) on read and write
Håkan MacLean (May 07 2021 at 13:16):
Thanks for taking the time to reply @Myles Bell !
This is the right approach to me, a business identifier should identify the underlying concept consistently across all contexts. All updates are done using the identifier.
Using Resource.Identifer to keep track of "source system identifiers" sure makes for an easier implementation and I like the fact that you can then easily query the FHIR server based on the source ID. However, it feels like we are slightly misusing the field. When there is decent 1:1 mapping between the source object and FHIR object thismakes sense, but say the FHIR object is put together of the subset of two different domain objects, i.e. there will be (at least) two entries in Resource.Identifier, then I'm wondering if the IDs still correspond to the "identifier" of this new object and it starts feeling like we are starting to misuse the standard slightly.
You said that you ended up usingsome "embedded Provenance" approach. Was there a reason for doing this compared to just using Resource.Identifier?
I have also started thinking about the topic of Access Control in an event driven world where data originally owned (and Access Controlled) by our micro services will start to also exist in a FHIR server, which needs to adhere to the same Access Control Lists (ACLs) and keep up to date with any changes to them. Have you guys done any work on this topic?
Myles Bell (May 07 2021 at 14:38):
No problem @Håkan MacLean
subset of two different domain objects
Yes this is tricky. Our approach long term is define the business domains to some extent closer to FHIR resources so you dont need to join domain objects. Right now when this happens though we would probably treat one domain as the "owner" and use the id in that domain, storing the reference to the other domain in the provenance. Not ideal but like you say has been much easier from the querying perspective which when trying to get more people using FHIR is quite important
The provenance captures quite a few useful datapoints about the source. Means we can have performant queries on those datapoints which are useful for things like establishing how trustworthy a resource is. e.g in the simplest case if the resource is validated by a dr or came straight from a patient
Not a great deal to be honest. Have considered being able to redact fields as needed but right now authorization is only per resource. We have a common authorization service so I guess that makes it easy to manage rules.
Last updated: Apr 12 2022 at 19:14 UTC
