Stream: Security and Privacy
Topic: AuditEvent pattern for POST search
John Moehrke (Jan 04 2021 at 16:25):
The security WG has a CR https://jira.hl7.org/browse/FHIR-26938 that is asking us to define a pattern for auditEvent when recording a POST based search. I would rather define something that is in use, than define something no one has implemented.
John Moehrke (Jan 04 2021 at 16:50):
Would seem the post body is simple enough to record in the query element. Does someone have an example?
Brian Postlethwaite (Jan 10 2021 at 23:54):
Need both the URL and the post body (as can have parameters in both places, and the base URL would indicate which operation, or resource type)
I've also added notes into the Jira ticket too.
John Moehrke (Jan 11 2021 at 13:42):
yes, seems even with a GET query, one does need to preserve the http headers. So, is there already a definition of how one combines the URL and http headers (and body for POST) into a blob?
Brian Postlethwaite (Jan 12 2021 at 11:21):
I'd be interested to know.
Brian Postlethwaite (Jan 12 2021 at 11:21):
For my server, I'd only log a few specific headers.
John Moehrke (Jan 12 2021 at 13:21):
@Brian Postlethwaite what do you log? as is typical for FHIR and for Implementation Guides, they tend to start with what is currently done.
Jens Villadsen (Jan 12 2021 at 17:44):
Our auditevents looks the same regardless of the FHIR query being done as GET or POST
Jens Villadsen (Jan 12 2021 at 17:45):
They are semantically equal
John Moehrke (Jan 12 2021 at 17:46):
so... what do they look like?
Jens Villadsen (Jan 12 2021 at 17:48):
Ill find an example for you tomorrow
Brian Postlethwaite (Jan 12 2021 at 23:05):
Here's a rough one
{
"resourceType": "AuditEvent",
"id": "f1df2bacce8e4c4198e12a90f3a7197c",
"meta": {
"versionId": "1",
"lastUpdated": "2020-12-18T14:14:01.6713288+11:00"
},
"extension": [
{
"url": "http://fhir.example-org.com.au/StructureDefinition/TimeTaken",
"valueDecimal": 7.45
},
{
"url": "http://fhir.example-org.com.au/StructureDefinition/X_CorrelationId"
}
],
"type": {
"system": "http://terminology.hl7.org/CodeSystem/audit-event-type",
"code": "rest"
},
"action": "R",
"recorded": "2020-12-18T03:14:01.6539829+00:00",
"outcome": "0",
"_outcome": {
"extension": [
{
"url": "http://fhir.example-org.com.au/StructureDefinittion/httpstatus",
"valueInteger": 200
}
]
},
"agent": [
{
"who": {
"reference": "Organization/2bfe6cfc-38e5-4b84-a4d9-efaebabe110b",
"display": "Generic Medicines"
},
"altId": "mvf_client_id",
"name": "Generic Medicines",
"requestor": true,
"network": {
"address": "::1",
"type": "2"
}
}
],
"source": {
"site": "Development",
"observer": {
"display": "HCX-LP-1303"
},
"type": [
{
"system": "http://terminology.hl7.org/CodeSystem/security-source-type",
"code": "3",
"display": "Web Server"
}
]
},
"entity": [
{
"role": {
"system": "http://terminology.hl7.org/CodeSystem/object-role",
"code": "24",
"display": "Query"
},
"name": "http://localhost:2483/Patient?_id=3d03619fefa74dae93ce0d93c65b2803&given=Jonathan&family=Curt",
"description": "Search Patient - Total: 1 Entries: 1"
},
{
"what": {
"reference": "Patient/3d03619fefa74dae93ce0d93c65b2803",
"display": "Curt, Jonathan Mr"
},
"role": {
"system": "http://terminology.hl7.org/CodeSystem/object-role",
"code": "1",
"display": "Patient"
}
}
]
}
(yes I know it's not valid as has some missing values, but is in development build)
And happy to get feedback if I have some of these values wrong.
Note that we extract the patient ID from the query parameter, and put that into the entities so that the searching for that patient's content comes up in the audit searching by that patient ID - this is also done on other things like MedReq searching by Patient ID.
John Moehrke (Jan 13 2021 at 00:42):
so that looks like an audit of a GET, not a POST... and properly what you have in entity[0].name should really be in entity[0].query.
I am noticing that even this is missing information like the mime-type negotiation...
John Moehrke (Jan 13 2021 at 01:35):
do you have a FHIR server available that is configured to auto record AuditEvents for REST (and possibly transactions)?
Jens Villadsen (Jan 13 2021 at 11:27):
GET
{
"resourceType": "AuditEvent",
"type": {
"system": "http://hl7.org/fhir/audit-event-type",
"code": "rest",
"display": "RESTful Operation"
},
"subtype": [
{
"system": "http://hl7.org/fhir/restful-interaction",
"code": "search-type"
}
],
"action": "R",
"recorded": "2021-01-13T12:21:12.303+01:00",
"outcome": "0",
"outcomeDesc": "Patient",
"purposeOfEvent": [
{
"coding": [
{
"system": "http://ehealth.sundhed.dk/fhir/PurposeOfUse",
"code": "INTERNAL_AUDIT_ONLY"
}
]
}
],
"agent": [
{
"extension": [
{
"url": "http://ehealth.sundhed.dk/fhir/StructureDefinition/ehealth-responsibleOrganization",
"valueReference": {
"reference": "https://organization.fut.trifork.com/fhir/Organization/33405"
}
}
],
"userId": {
"system": "http://ehealth.sundhed.dk",
"value": "https://organization.fut.trifork.com/fhir/Practitioner/107950"
},
"requestor": true
}
],
"source": {
"identifier": {
"system": "http://ehealth.sundhed.dk",
"value": "http://localhost:8080/fhir/"
},
"type": [
{
"system": "http://hl7.org/fhir/security-source-type",
"code": "4"
}
]
},
"entity": [
{
"identifier": {
"value": "f7ded3e5-6db7-42de-ba30-65cd8d0b4015"
},
"type": {
"system": "http://hl7.org/fhir/security-source-type",
"code": "4",
"display": "Application Server"
},
"role": {
"system": "http://hl7.org/fhir/object-role",
"code": "24",
"display": "Query"
},
"description": "search entity",
"query": "eyJfaWQiOlsiMTExIl19"
},
{
"reference": {
"reference": "http://localhost:8080/fhir/Patient/111"
},
"role": {
"system": "http://hl7.org/fhir/object-role",
"code": "1"
}
}
]
}
Jens Villadsen (Jan 13 2021 at 11:27):
POST
{
"resourceType": "AuditEvent",
"type": {
"system": "http://hl7.org/fhir/audit-event-type",
"code": "rest",
"display": "RESTful Operation"
},
"subtype": [
{
"system": "http://hl7.org/fhir/restful-interaction",
"code": "search-type"
}
],
"action": "R",
"recorded": "2021-01-13T12:22:02.980+01:00",
"outcome": "0",
"outcomeDesc": "Patient",
"purposeOfEvent": [
{
"coding": [
{
"system": "http://ehealth.sundhed.dk/fhir/PurposeOfUse",
"code": "INTERNAL_AUDIT_ONLY"
}
]
}
],
"agent": [
{
"extension": [
{
"url": "http://ehealth.sundhed.dk/fhir/StructureDefinition/ehealth-responsibleOrganization",
"valueReference": {
"reference": "https://organization.fut.trifork.com/fhir/Organization/33405"
}
}
],
"userId": {
"system": "http://ehealth.sundhed.dk",
"value": "https://organization.fut.trifork.com/fhir/Practitioner/107950"
},
"requestor": true
}
],
"source": {
"identifier": {
"system": "http://ehealth.sundhed.dk",
"value": "http://localhost:8080/fhir/"
},
"type": [
{
"system": "http://hl7.org/fhir/security-source-type",
"code": "4"
}
]
},
"entity": [
{
"identifier": {
"value": "b4b77e5f-9658-4fa4-8a1c-530a5f2ce0e7"
},
"type": {
"system": "http://hl7.org/fhir/security-source-type",
"code": "4",
"display": "Application Server"
},
"role": {
"system": "http://hl7.org/fhir/object-role",
"code": "24",
"display": "Query"
},
"description": "search entity",
"query": "eyJfaWQiOlsiMTExIl19"
},
{
"reference": {
"reference": "http://localhost:8080/fhir/Patient/111"
},
"role": {
"system": "http://hl7.org/fhir/object-role",
"code": "1"
}
}
]
}
John Moehrke (Jan 13 2021 at 12:44):
@Jens Villadsen so it appears you convert both get and post into some other form. What is that other form? I simply don't recognize it
{"_id":["111"]}
John Moehrke (Jan 13 2021 at 12:44):
this method has the drawback that the audit log will not contain the query as it was received. It will have been cleansed. Thus there is not data there to be used to determine if an attack probe is being done.
Jens Villadsen (Jan 13 2021 at 12:53):
the query was "_id=111"
Jens Villadsen (Jan 13 2021 at 12:54):
you can say it has been 'cleansed' - by HAPI that is ... to the canonical internal representation as conceived by HAPI
Jens Villadsen (Jan 13 2021 at 12:55):
They are syntactically identical
Jens Villadsen (Jan 13 2021 at 12:56):
We have a full request/response log next to the auditlog
Jens Villadsen (Jan 13 2021 at 13:05):
@John Moehrke
John Moehrke (Jan 13 2021 at 13:08):
so the intermediate form I am looking for is a HAPI invention. unfortunate... and as I said, for security or privacy purposes this clensed form is not as good.... I do wonder if this brings up an opportunity to use the intermediate form for positive purposes (find me the audits of queries for this). while dual recording the exact inbound query also for negative purposes.
Jens Villadsen (Jan 13 2021 at 13:11):
note - all auditevents are linked to the actual request in the request/response log as the entity.identifier in the auditevent is a b3 zipkin ... meaning that we can always find the original request will all headers and URL's and stuff
Jens Villadsen (Jan 13 2021 at 13:11):
The AuditEvent will always be a condensed form of the the original request as I see it
John Moehrke (Jan 13 2021 at 13:12):
but that is not "standard"... I am trying to come up with a "standard" that everyone can follow ... hence my questions
John Moehrke (Jan 13 2021 at 13:13):
I am not looking to implement, I am looking to fix or provide guidance in AuditEvent
Jens Villadsen (Jan 13 2021 at 13:13):
ok ... then the AuditEvent should reflect the request in its entirety
Jens Villadsen (Jan 13 2021 at 13:14):
meaning: ALL headers, ALL of the Body
Jens Villadsen (Jan 13 2021 at 13:20):
IE - I don't see a partiular field for HTTP header fields in AuditEvent
Jens Villadsen (Jan 13 2021 at 13:37):
@John Moehrke
John Moehrke (Jan 13 2021 at 13:51):
yes, that is the intentioni
John Moehrke (Jan 13 2021 at 13:52):
that is why I was asking for how others have solved this.
John Moehrke (Jan 13 2021 at 13:55):
This is why the AuditEvent.entity.query is a base64, so that it captures exactly what came in. Thus for any kind of query technology there needs to be a way defined for how to be complete while also not sanitizing the data. With DICOM this was easy, with SQL this was easy, for XDS this was easy... for http GET we thought it was easy until it was pointed out the http headers are important, and for http POST we could grab the http header/body but miss the URL.
Jens Villadsen (Jan 13 2021 at 13:58):
HTTP headers are also important in XDS
John Moehrke (Jan 13 2021 at 13:59):
in what way are you thinking they are important?
Jens Villadsen (Jan 13 2021 at 13:59):
they may contain security tokens
Jens Villadsen (Jan 13 2021 at 13:59):
or other important context
John Moehrke (Jan 13 2021 at 13:59):
security is not a different layer, handled by a different audit event
Jens Villadsen (Jan 13 2021 at 14:00):
sure ... but the header may set the context of the xds invocation
Jens Villadsen (Jan 13 2021 at 14:00):
right?
John Moehrke (Jan 13 2021 at 14:00):
in FHIR http, I don't mind keeping the security headers or removing them... not the most important problem right now
Jens Villadsen (Jan 13 2021 at 14:02):
it would not recommend removing headers in an audit trail
Jens Villadsen (Jan 13 2021 at 14:02):
especially in a RESTful setup as FHIR
John Moehrke (Jan 13 2021 at 14:02):
right. I agree
John Moehrke (Jan 13 2021 at 14:02):
I am not trying to remove them
John Moehrke (Jan 13 2021 at 14:02):
I am trying to get the POST query fully recorded
Jens Villadsen (Jan 13 2021 at 14:03):
you don't even have the GET fully recorded
John Moehrke (Jan 13 2021 at 14:03):
if along the way I discover the GET query needs some more guidance, so be it
John Moehrke (Jan 13 2021 at 14:03):
then help me... I know the points you are making.
Jens Villadsen (Jan 13 2021 at 14:04):
and your intention is to build a model that fully encapsulates a HTTP request
Jens Villadsen (Jan 13 2021 at 14:04):
?
John Moehrke (Jan 13 2021 at 14:04):
I asked for examples.. you gave them to me... I pointed out how they are not sufficient... now onto the next step
Jens Villadsen (Jan 13 2021 at 14:05):
and the next step would be ... ? to fully encapsulate HTTP requests (at least for RESTful FHIR)?
Jens Villadsen (Jan 13 2021 at 14:06):
is that your goal? to design such a model?
Jens Villadsen (Jan 13 2021 at 14:07):
the specific HTTP method is irrelevant ... you would have to capture all of them anyway
Jens Villadsen (Jan 13 2021 at 14:07):
any by the way ... HTTP GET can actually contain a body ... just an FYI
Jens Villadsen (Jan 13 2021 at 14:08):
and it is used from time to time ... seldom however
Jens Villadsen (Jan 13 2021 at 14:09):
just to stir things up
John Moehrke (Jan 13 2021 at 14:11):
yes that sounds right
Jens Villadsen (Jan 13 2021 at 14:17):
well ... to simplify matters ... for calls made over HTTP you would need an array of fields for for HTTP headers, one for the method and a field for the body
Jens Villadsen (Jan 13 2021 at 14:17):
regardless of the HTTP method
John Moehrke (Jan 13 2021 at 14:20):
sounds good... does this exist, or will we be defining this on the AuditEvent page?
Jens Villadsen (Jan 13 2021 at 14:21):
The word 'header' does not exist on https://www.hl7.org/fhir/auditevent.html so my guess would be 'no'.
John Moehrke (Jan 13 2021 at 14:21):
I don't mind defining it on the AuditEvent page, but would like it better if it was already specified somewhere.. like in an IETF RFC...
Jens Villadsen (Jan 13 2021 at 14:22):
arh...
John Moehrke (Jan 13 2021 at 14:22):
what do web server audit logs natively look like?
Jens Villadsen (Jan 13 2021 at 14:22):
there's no such thing
Jens Villadsen (Jan 13 2021 at 14:22):
in general terms
John Moehrke (Jan 13 2021 at 14:23):
surely doesn't hurt to ask
Jens Villadsen (Jan 13 2021 at 14:24):
usually like https://httpd.apache.org/docs/2.4/logs.html#accesslog
Jens Villadsen (Jan 13 2021 at 14:24):
well ... thats access logs ...
Jens Villadsen (Jan 13 2021 at 14:24):
that not the same thing as auditlogs
Jens Villadsen (Jan 13 2021 at 14:26):
https://tools.ietf.org/html/rfc2616 specifies how HTTP requests look like
Jens Villadsen (Jan 13 2021 at 14:26):
and what is mandatory
John Moehrke (Jan 13 2021 at 14:27):
so we have the other stuff... we just need the "The request line from..."
John Moehrke (Jan 13 2021 at 14:29):
we just need %r
Jens Villadsen (Jan 13 2021 at 14:30):
The https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
Jens Villadsen (Jan 13 2021 at 14:31):
so the method and the request uri
John Moehrke (Jan 13 2021 at 14:31):
that would do it... right?
Jens Villadsen (Jan 13 2021 at 14:31):
seems correct ... where would you place the headers?
Jens Villadsen (Jan 13 2021 at 14:32):
or are you 'just' collecting stuff that needs to be in the model?
John Moehrke (Jan 13 2021 at 14:32):
unfortunately... the code that would be building the AuditEvent likely only has sanitized data (like your observing)... so it might be hard to get this...
Jens Villadsen (Jan 13 2021 at 14:33):
nop ... and FHIR server building auditevents should have access to the actual request ...
John Moehrke (Jan 13 2021 at 14:33):
ah. okay.
Jens Villadsen (Jan 13 2021 at 14:33):
well ... you are right .... it can probably be hard to get it under some circumstances
John Moehrke (Jan 13 2021 at 14:33):
so... this would solve the core problem. satisfy the need for both GET and POST query, grabing stuff that we don't grab today...
Jens Villadsen (Jan 13 2021 at 14:34):
and all the other Methods
John Moehrke (Jan 13 2021 at 14:34):
I just wonder if we should have a different way to record the sanitized parameters.
Jens Villadsen (Jan 13 2021 at 14:34):
PATCH, DELETE
John Moehrke (Jan 13 2021 at 14:34):
this is just for Query type transactions... but the pattern might be useful elsewhere.. agreed
Jens Villadsen (Jan 13 2021 at 14:34):
well ... given that POST and GET searches are semantically equivivalent ... it shouldn't be necessary
Jens Villadsen (Jan 13 2021 at 14:35):
You need to support all HTTP methods for the AuditEvent
Jens Villadsen (Jan 13 2021 at 14:35):
at least the one's that are supported for FHIR
Jens Villadsen (Jan 13 2021 at 14:35):
or specified for FHIR
Jens Villadsen (Jan 13 2021 at 14:36):
will this come up on the upcoming WGM?
John Moehrke (Jan 13 2021 at 14:36):
I think you are addressing another topic.. that of should there be a pattern in AuditEvent to record all http requests regardless of what they are.
John Moehrke (Jan 13 2021 at 14:36):
to that, I don't disagree, but to that we expected that was already handled by the infrastructure that is not FHIR specific
Jens Villadsen (Jan 13 2021 at 14:37):
well ... we just agreed that the current model does not contain headers
John Moehrke (Jan 13 2021 at 14:38):
and for Query we do expect that if you can identify the patient(s) that you do... so we are not replacing the web server log, we are doing a log at the FHIR infrastructure level
John Moehrke (Jan 13 2021 at 14:38):
Jens Villadsen said:
well ... we just agreed that the current model does not contain headers
yes, and you just helped solve that one
Jens Villadsen (Jan 13 2021 at 14:38):
sure
Jens Villadsen (Jan 13 2021 at 14:40):
I just dont understand how the auditevent model can be seen as complete for the other HTTP methods when HTTP headers currently are not part of the model
Jens Villadsen (Jan 13 2021 at 14:40):
but i agree - that is not part of the topic
Jens Villadsen (Jan 13 2021 at 14:41):
but if you are working on fixing the model i general ... now you know more of its shortcommings ;)
John Moehrke (Jan 13 2021 at 14:43):
Jens Villadsen said:
so the recommendation is that the AuditEvent.entity.query is base64 (later we can argue the base64 point) encoded of this
Request = Request-Line ; Section 5.1
*(( general-header ; Section 4.5
| request-header ; Section 5.3
| entity-header ) CRLF) ; Section 7.1
CRLF
[ message-body ] ; Section 4.3
does this work for @Grahame Grieve @Brian Postlethwaite @James Agnew and other FHIR server platforms ?
Jens Villadsen (Jan 13 2021 at 14:47):
- an obvious note to that is that the query can be in either the
message-bodyor theRequest-Line
John Moehrke (Jan 13 2021 at 14:50):
I read this to say that the "Request" includes both the Request-Line and the message-body... so we are covered.. right?
John Moehrke (Jan 13 2021 at 14:50):
would be good to see two examples (GET vs POST) of a simple FHIR query
Jens Villadsen (Jan 13 2021 at 14:52):
The Request is the full monty
John Moehrke (Jan 13 2021 at 14:54):
likely more than we want... as you pointed out it will include security headers... but better to get it than invent some format that will be harder to address.
Josh Mandel (Jan 13 2021 at 15:09):
I have not read all the messages in this thread but wanted to point out http://www.softwareishard.com/blog/har-12-spec/#request which I don't think was ever formally standardized despite being drafted in w3c, but it is widely used in software tools for saving durable records of http requests and responses (for example, in chrome developer tools you can save any requests made from a web page in this format by right clicking the request in the network tab and choosing save as)
Jens Villadsen (Jan 13 2021 at 15:34):
seems like an ok option
John Moehrke (Jan 13 2021 at 16:49):
that seems like an alternative we should consider. It is a bit more readable to a JSON friendly person. But it seems to contain the same stuff, right? So the difference is encoding, and one is a standard vs one is a a quasi-standard. I don't see that as a bad thing, but just want to be clear.
Josh Mandel (Jan 13 2021 at 16:56):
I think "quasi-standard" sounds about right for HAR; https://en.wikipedia.org/wiki/HAR_(file_format) has a bit more background.
Last updated: Apr 12 2022 at 19:14 UTC
