Stream: implementers
Topic: Auditing a transaction
Anand Mohan Tumuluri (Jun 24 2016 at 20:47):
I have a question regarding auditing of a FHIR transaction. Are we supposed to create one AuditEvent for the entire transaction bundle with type/subtype as rest/transaction? Or should we create one AuditEvent for each resource operation within the transaction bundle?
Grahame Grieve (Jun 25 2016 at 09:11):
I do both. But there's no formal policy
John Moehrke (Jun 25 2016 at 20:56):
Hi Anand,
John Moehrke (Jun 25 2016 at 21:00):
You can create one audit event that indicates as entities each of the items in transaction. You could create individual audit events for each item in the transaction. You could even just do one audit event for the transaction, and N simple create/update audit events for the items. Or you could jut do one audit event for the transaction, create a Provenance for anything created/updated, and point the transactional audit evnent at the provenance. This last part is what confuses the most, as it really indicates the overlap of provenance and auditevent. They are overlapped, but they have different downstream uses. So it is important to look at the downstream use-case, that will tell you which mode is likely most useful.... That said, not knowing the downstream, it is far better to create too many audit events as they are by nature less expected to be permenant and allowed to be analized and purged.
John Moehrke (Jun 27 2016 at 13:48):
The security wg would like to help the community understand proper use of auditEvent and Provenance through some connectathon focus. Today we have the EHR lifecycle that is fine for that specific purpose, but doesn't compel people to come test it. So we are thinking that what we need is some existing connectathon track that does something overall useful but also involved enough to have multiple actors and act upon multiple resources in an overall useful scenario. Security would then have an overlay on this indicating the points where Provenance might be critical, or useful; and where auditEvent might be criticalor useful. -- recognizing that policy in a specific deployed implementation will actually define what is really required. This way people can first focus on the primary purpose of the connectathon track, working on the interactions and clinical resource use; and then realize that for a bit more effort they can add support of the security layer...
John Moehrke (Jun 27 2016 at 13:50):
Unfortunately it seems all connectathon tracks are single resource focused and consist of only a client and a server. So it is hard to show value of audit or provenance; and adding these in without a larger context would look bizarre as the context less scenario would demand all involved create both audit events and provenance events; as there is no context to lend reasonable expectations.
Chris Grenz (Jun 27 2016 at 14:39):
+1 for security connectathon track(s)! I'm concerned that the conformance resources as getting "firm" without enough information protection scrutiny...
John Moehrke (Jun 27 2016 at 15:15):
Interesting... the Security WG could take a concentrated look at the security impact of the conformance resources... We are approaching an empty CP backlog...
John Moehrke (Jun 27 2016 at 15:16):
That said... my point is that we have had a security track that no one signs up for.. the EHR lifecycle track... So, my proposal is not to have a security track, but rather to have security overlays on other tracks.
Chris Grenz (Jun 27 2016 at 15:53):
My concerns are that the conformance resources are useful in accurately conveying what's happening in a real-world system as information is redacted and/or updated during FHIR service calls.
Chris Grenz (Jun 27 2016 at 15:55):
Many interactions with clients are going to be on resources subsets...will the standard adequately communicate with the client regarding what information is being exchanged? Or will the server simply not send some information forcing the client to guess the meaning of missing data.
Chris Grenz (Jun 27 2016 at 15:56):
Or, prior to exchange, will it be possible for the client to determine a configuration based on what data the server will provide? Or will the client just have to start asking for data and then determine what it can do based on what is returned?
Chris Grenz (Jun 27 2016 at 15:57):
And now I'm off-topic for this thread....my apologies....
John Moehrke (Jun 27 2016 at 16:06):
Chris, we need to be careful. What you are asking for would expose the data that the system is protecting: "Here is the data I have on the patient, except for that STD that the patient has asked us to not disclose".
John Moehrke (Jun 27 2016 at 16:07):
We have mechanisms in FHIR to indicate that you have received ALL the data, or a SUBSET. This primarily so that you know you have a subset and thus an UPDATE would be difficult/impossible... (an operational problem we are kicking down the road, but at least we can detect it).
Chris Grenz (Jun 27 2016 at 16:07):
Absolutely....but making a general statement that "STD Observations will not be included in any request" might be important
Chris Grenz (Jun 27 2016 at 16:08):
Referring to the SUBSETTED tag?
John Moehrke (Jun 27 2016 at 16:08):
SUBSETTED is used when you have asked for a _summary... as an indication you didn't get it all.
John Moehrke (Jun 27 2016 at 16:08):
REDACT (or other) would be used if it was removed for a Security or Privacy reason
John Moehrke (Jun 27 2016 at 16:09):
but telling you WHY, is wrong.
Chris Grenz (Jun 27 2016 at 16:09):
"kicking down the road" is exactly my concern...
Chris Grenz (Jun 27 2016 at 16:10):
My lack of authorization to see birth date shouldn't (necessarily) prevent me from being able to update communication...
John Moehrke (Jun 27 2016 at 16:10):
we have enough problems to solve today... right? This problem is where a user has less than full access, yet somehow has the rights to UPDATE? I don't understand why that use-case is important.
John Moehrke (Jun 27 2016 at 16:11):
your example is not likely... right?
Chris Grenz (Jun 27 2016 at 16:11):
Because we want this standard to be useful in all kinds of environments! Sure it is
John Moehrke (Jun 27 2016 at 16:11):
you have rights to do an UPDATE on a resource, but you don't have rights to READ the whole thing?
John Moehrke (Jun 27 2016 at 16:11):
how is that even possible?
John Moehrke (Jun 27 2016 at 16:12):
AND the security layer is independent of the data-model... we are working on the data-model first.
Chris Grenz (Jun 27 2016 at 16:12):
Think app market apps....there may be many scenarios where apps will operate only parts of resources
John Moehrke (Jun 27 2016 at 16:12):
not that I want to paint myself into a corner.. which is why early this morning I suggested that this is useful for security to look at.
John Moehrke (Jun 27 2016 at 16:14):
If apps work on only parts of resources, then we should take that as evidence that the resource is too big. I have argued many times that we need to be careful about making reources that are too big. That a criteria for how big a resource is should be how likely there would be policies that need to subdivide a resource. This should be used as clear evidence that the resource should be split.
Chris Grenz (Jun 27 2016 at 16:14):
If we were only working on the data model, I'd be calm. But we're also very much working on the service model...
Grahame Grieve (Jun 27 2016 at 20:57):
I have worked on a system where I had the right to enter a data value, but not the view the data I entered
Grahame Grieve (Jun 27 2016 at 20:57):
HIV result.
Grahame Grieve (Jun 27 2016 at 20:58):
I generally agree that the bigger a resource gets. the more likely it is to want to secure part of it, and that when they are too big, this will be a problem. But I don't agree that the correct size is 'when it's so small that no one will want to secure bits of it', so we will have to deal with that problem.
Grahame Grieve (Jun 27 2016 at 20:59):
the problem with security at connectathons is time... security would have to be an outcome, not an overhead. (specifically with regard to the connectathon itself)
James Agnew (Jun 27 2016 at 21:17):
I wonder if it would be feasible to build a reusable HAPI server interceptor that automatically creates an AuditEvent for every incoming request. That would be really neat actually.
Something to try at next connectathon perhaps. :)
Grahame Grieve (Jun 27 2016 at 21:31):
I already do that internally
Brian Postlethwaite (Jun 28 2016 at 01:30):
Same here.
James Agnew (Jun 28 2016 at 01:33):
Yeah, your respective servers were the inspiration for this. :)
I'm just wondering if it can be done generically enough to be useful (ie. could it be used on anyone's server if it's built with the HAPI server framework)
Grahame Grieve (Jun 28 2016 at 02:20):
that sounds like a nice idea
John Moehrke (Jun 28 2016 at 02:25):
so, this is nice... in that it does record AuditEvents for some security relevant events. However there are many more auditevents that need to be recorded. I would never discourage automation, I just want to keep the pressure on for complete auditevents. So, what events? Access Control decision failues, authentication failures, communication failures, storage failures, boot, system start, service managment, configuration management, and --- execution of services, workflows, algorithms, etc...
John Moehrke (Jun 28 2016 at 02:28):
@Grahame Grieve I would never say to go too small... They need to be 'just right'... which is a tradeoff between many things. Sorry if I implied that we should make everything micro-resources. Not my intention. All I wanted was recognition that sometimes we should look to breakup a resouce because it is getting too big, too hard to manage. Yes I did propose that a trigger for this evaluation is when security would need to segment a resource, but this is just a trigger for evaluation, it is not a mandate that it must be broken up. I think we need some of these guiding principles.
Grahame Grieve (Jun 28 2016 at 02:30):
agree. In fact, that particular rule is written somewhere, I think
Mohammad Jafari (Jun 30 2016 at 17:53):
@James Agnew having thought of creating audit events inside security labeling interceptor, I was wondering whether there is a mechanism to create/update resources from within an interceptor without actually establishing a TCP connection to the local server. Something like a special HAPI FHIR client that connects to the data source directly but maintains the HAPI client interface.
Grahame Grieve (Jun 30 2016 at 21:28):
sounds dangerous from within an interceptor... locking problems arise easily
Mohammad Jafari (Jun 30 2016 at 22:27):
@Grahame Grieve I think if one uses the JPA transaction manager those issues are going to be taken care of.
Brian Postlethwaite (Jul 01 2016 at 03:28):
And if the transaction fails and is rolled back, need to be careful that the audit doesn't rollback too.
James Agnew (Jul 04 2016 at 22:21):
If you mean within the JPA module you could do this by asking spring for a copy of the IResourceDao<AuditEvent> and wiring that into your interceptor. Then in your interceptor you could use that to write new AuditEvent resources.
Look at the IJpaServerInterceptor interface, which adds some new methods to the interceptor. These methods are called within the scoping transaction so you could get all-or-nothing behaviour when writing an audit event alongside the thing you are actually writing (presumably you'd want this behaviour).
Anand Mohan Tumuluri (Jul 06 2016 at 04:39):
We had done a lot around avoiding this behavior i.e. failure in creating/updating a resource should NOT fail writing the audit event. At the same time failing a resource create/update if the creation of audit event fails is a desirable behavior.
John Moehrke (Jul 06 2016 at 13:45):
I don't think I agree. We even made statements to this in the core ATNA standards. The loss of an audit event is less important than the loss of medical data. Don't roll back something because you couldn't record the audit event. The audit needs to be as clean and fast as possible.
John Moehrke (Jul 06 2016 at 13:46):
This article is about this concept, although the reason the article was written was different. But the concept is the same. https://healthcaresecprivacy.blogspot.com/2011/12/atna-syslog-is-good-enough.html
James Agnew (Jul 06 2016 at 13:55):
@Anand Mohan Tumuluri FWIW the interceptor in this case does also let you record a different audit event scoped outside the transaction in the event that the transaction rolls back, via the handleException method. It seems weird to me that someone would want the same audit event recorded for a write transaction, even if that write didn't actually succeed.
The behaviour John is describing would be a bit more work though. Since the write and the audit-write are grouped, either could fail the other. I guess a queueing mechanism of some sort would be needed to separate these.
John Moehrke (Jul 06 2016 at 14:15):
If the primary write failed, then the auditEvent.outcome should indicate the failure vs success. So it isn't really radically diffeent audit log entry.
Anand Mohan Tumuluri (Jul 06 2016 at 16:50):
@James Agnew I think I didn't say that right. If a resource CRUD failed, we write a failure audit event. The audit event write (success/failure) is not in the same transaction as the resource. In our case, the resource write is a sub-transaction yielding a operation outcome. The outcome is audited no matter what in the parent transaction.
John Moehrke (Jul 06 2016 at 17:00):
Well done. I expected so, but wanted to be clear.
Anand Mohan Tumuluri (Jul 06 2016 at 17:02):
@John Moehrke Isn't it subjective to allow/disallow resource actions without a audit trail? Is there enough consensus that it is OK to allow the resource action even if the audit couldn't be recorded.
Anand Mohan Tumuluri (Jul 06 2016 at 17:04):
BTW very informative article, thanks for sharing
John Moehrke (Jul 06 2016 at 17:15):
@Anand Mohan Tumuluri correct to note that this is clearly a policy statement; however I would argue that the in healthcare treatment the dominant policy is the one I ascribe. If we were in the Financial market, they would set the policy completely the opposite way; that is if it can't be written into the ledger then it can't happen. The difference is that in Finance the harm of delaying a transaction until the full system is operational is minimal, where in healthcare treatment the delay can cause pain, suffering, and death. -- as a general-purpose toolkit, you don't know if you are being used for life-critical treatment, or if you are being used for delayable use-cases.
Anand Mohan Tumuluri (Jul 06 2016 at 17:23):
@John Moehrke Thank you, I am more convinced than before.
Mohammad Jafari (Aug 13 2016 at 00:45):
Hi @James Agnew . Working on a consent enforcer interceptor for the Baltimore connecrathon, and following your advice above, I am now facing this exception trying to get a Dao for Patient or Consent. Wondering how I can trigger the creation of these beans:
No qualifying bean of type [ca.uhn.fhir.jpa.dao.IFhirResourceDao] found for dependency [ca.uhn.fhir.jpa.dao.IFhirResourceDao<org.hl7.fhir.dstu3.model.Patient>]: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}; nested exception is
org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type [ca.uhn.fhir.jpa.dao.IFhirResourceDao] found for dependency [ca.uhn.fhir.jpa.dao.IFhirResourceDao<org.hl7.fhir.dstu3.model.Patient>]: expected at least 1 bean which qualifies as autowire candidate for this dependency. Dependency annotations: {@org.springframework.beans.factory.annotation.Autowired(required=true)}
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:569)
James Agnew (Aug 26 2016 at 11:27):
That's very strange. Those should be created automatically.. Does it help if you ask for it by name? myPatientDaoDstu3
It gets declared in BaseJavaConfigDstu3.java FWIW
Last updated: Apr 12 2022 at 19:14 UTC
