Stream: implementers
Topic: Transaction Log
nicola (RIO/SS) (Nov 04 2016 at 06:11):
We were discussing subscriptions and streams and came to well known in databases design idea - transaction log: Any creating/deleting or changing resources operations (including transaction) could be expressed as transactions, server could provide stream/ log of all transactions. This could be used for FHIR server *replication* - external (downstream) server just connect to upstream server transaction log , listen and apply all (or filtered) transactions. This is quite generic, but looks powerful. Also filtered by some criteria transaction log could be used to create simplest and natural subscriptions, which are easy to implement ( In general, it's not trivial (if possible) to make arbitrary search reactive - i mean current subscriptions).
nicola (RIO/SS) (Nov 04 2016 at 06:17):
*history-all* operation is similar, but has different granularity - transaction boundaries are missed.
Grahame Grieve (Nov 04 2016 at 06:21):
yes transaction boundaries are missed. that's explicitly documented. Is it a problem? why?
nicola (RIO/SS) (Nov 04 2016 at 06:24):
May be not :) but that's integrity concern - usually replication has transaction granularity guaranties.
John Moehrke (Nov 04 2016 at 12:10):
Transaction log is an implementation and deployment detail. FHIR is an interoperability standard, it doesn't cover many implementation and deployment details. FHIR does have a Provenance resource and when used with Resource versioning, will provide a version granularity.
nicola (RIO/SS) (Nov 04 2016 at 12:11):
@John Moehrke Replication of FHIR server - isn't it interop?
John Moehrke (Nov 04 2016 at 12:20):
so your use-case is a replication of a FHIR server... okay... so what is the failure-mode you are looking to protect against? I ask because at the Interoperability layer, we have integrity through the http(s). So both sides can confirm communication. A good recipient will not return success indication until it has committed the transaction (to storage, to transaction queue, etc). A 'bad' recipient won't... so this 'risk' is an implementation detail. What is your identified 'risk'?
nicola (RIO/SS) (Nov 04 2016 at 12:22):
You could use replication not only for failover, but to localize data on another server
John Moehrke (Nov 04 2016 at 12:23):
Note, FHIR also has AuditEvent that can be recorded on both sides (sender records the sending of an identified set of records, recipient records the reception of a set of records, recipient records commitment of a set of records, etc)
John Calvin Young (Nov 04 2016 at 18:12):
This is a powerful thought, and potentially more useful than a typical FHIR subscription. One of my integration partners recently told me that an approach like this would be far easier for them to implement than the last-updated approach with polling.
Grahame Grieve (Nov 04 2016 at 18:53):
what's the difference?
John Moehrke (Nov 04 2016 at 19:31):
The difference is "state". In a typical REST subscription it is the client that remembers what it has seen. In a case like SOAP (e.g. XDS Document Subscription) the server must remember the state, that is the server is told that the client is interested in notification on a specific set of filters (query), the server looks at the set of subscriptions everytime something changes, and notifies the client when something happens. The second problem is directionality, but that can be resolved with tricks of connection state management.
Grahame Grieve (Nov 04 2016 at 19:32):
I don't follow. there's push vs pull, and then there's transaction boundaries that might be missing. but I don't know what the difference between 'transaction log' and last-updated is
John Moehrke (Nov 04 2016 at 19:35):
Sorry, your question is then not for me... I too am confused as to the transaction log risk.
Last updated: Apr 12 2022 at 19:14 UTC
