Stream: implementers
Topic: Treat AuditEvents as Logs Events
Alexander Kiel (Sep 13 2019 at 14:03):
Should a FHIR Component such as a server, client or intermediate just log AuditEvent resources to stdout like all other log events or are there arguments to handle AuditEvents special? My question arised after reading https://12factor.net/logs freshly.
Lloyd McKenzie (Sep 13 2019 at 15:51):
@John Moehrke
John Moehrke (Sep 13 2019 at 16:03):
Well, that depends. I have been on one project that did decide ( 1 ) to use the normal log mechanism, as it was broader than just the events driven by FHIR aware apps. Thus the downstream analysis wanted to have all the logs from the database, web-server, app-server, etc... and also the nicely formatted ones using the JSON encoded AuditEvent to give it structure.
I was on a project ( 2 ) that wanted to redirect all stdout into AuditEvent logs. Thus doing the reverse of option ( 1 ). They would intercept all stdout/stderr and make an AuditEvent out of them -- this was less successful as it is possible to do this, but you end up with a rather stupid AuditEvent that has only static values and a blob of a string. They stopped this.
I might suggest a more realistic ( 3 ) that when apps are dealing with FHIR data, they should audit using AuditEvent to a repository that is FHIR aware. This might be a special purpose, especially as there will be volume issues and purge processes. But in this model the analysis at the Security office or during an event audit, the inspection will need to look across both the AuditEvent repository AND the normal system logs.
Most of the time when IHE, DICOM, FHIR writes and implementation guide, there is only an expectation that you well form the audit events that the IG defines. Persistence might be in a text log, database, or fhir repository.
so... it depends on what you want...
nicola (RIO/SS) (Sep 13 2019 at 16:19):
:) we are just discussing with partners - where to write AuditEvents into db or into logs stream?
John Moehrke (Sep 13 2019 at 16:27):
The AuditEvent is a nice format, but reality is that the IT infrastructure from general IT won't use it... So there will be a combination. The hope in AuditEvent is that where we can we should try harder to get full and coded log entries... The retention issues are very different for logs, than they are for clinical data.. which is why I recommended an independent fhir repository for AuditEvents from that being used for everything clinical.
Alexander Kiel (Sep 14 2019 at 14:13):
@John Moehrke @nicola (RIO/SS)
I think we have a consensus that AuditEvents are just better, more structured log events. So putting them into the general log stream of an application would be reasonable.
Than however John, you are right, there is a question whether one has access to the general log stream to analyze the special AuditEvents. In other words, it depends on the execution environment wether AuditEvents from log stream can be used to there full advantage. As a side note: we run an execution environment capable of this at our site. But other sites still use single Windows VM's to run my FHIR server.
The retention issues are very different for logs, than they are for clinical data.. which is why I recommended an independent fhir repository for AuditEvents from that being used for everything clinical.
I also thought about this. At the end one possibility would be to have a special FHIR server implementation tuned for storing AuditEvents only. It should be optimized on the write side to be able to store AuditEvents with low latency. Because storing AuditEvents doesn't need transactions (in my eyes), it could also scale horizontally. I have pretty advanced ideas of the architecture of such a system. So if someone likes to discuss this, I would be happy.
So if we don't think, we have the execution environment to handle AuditEvents from logs stream in the way we like it, would it be an idea to just POST them directly to such a special FHIR server from our application servers using HTTP connections? In my eyes that would complicate the implementation but would be possible.
Jens Villadsen (Oct 21 2019 at 12:08):
FWIW - we use auditlogs as a reporting format for national logging services - we use proprietary json format for regular 'normal' ops logging
Last updated: Apr 12 2022 at 19:14 UTC
