FHIR Chat · Information about source for each information element · implementers

Stream: implementers

Topic: Information about source for each information element

Ellen Gregusson (Aug 08 2019 at 14:41):

We are creating the patient resource based on information from both external registers and from internal systems. In order to make sure that internal systems do not update or delete information received from external systems we need to indicate on each element in the document what the source for the information is. Does anyone have input on how to solve this requirement?

Michele Mottini (Aug 08 2019 at 16:00):

You can indicate where each element comes from using Provenance resources and the (proposed) targetElement extension (see https://gforge.hl7.org/gf/project/fhir/tracker/?action=TrackerItemEdit&tracker_item_id=21284)

David Hay (Aug 08 2019 at 17:56):

@Ellen Gregusson - as a side question, are you maintaining a central registry of patients (ie a directory)? If so, will you be supporting the updating of data by clients? If so, interested in your thinking of which API you'll be using...

Ellen Gregusson (Aug 09 2019 at 07:34):

@David Hay We are maintaining a registry of patients which have a relationship with the Health directorate in Norway. The clients will add and update part of the information on the patient. We will store some information that cannot be displayed in all clients so we will have to use patch-api's in order to add or update information from clients. We are using Microsoft FHIR server, so we are really looking forward to them implementing patch api's for person, patient, practitioner, related person and organization.

David Hay (Aug 09 2019 at 08:04):

By patch - do you mean the patch operation?

David Hay (Aug 09 2019 at 08:06):

see: https://fhirblog.com/2019/07/09/thoughts-about-updating-registries/ . - The team I work with isn't keen on PUT updates, and operations (as described in the post) have also received some criticism. We didn't really look at patch in detail, so interested in your approach to this...

Ellen Gregusson (Aug 09 2019 at 09:18):

What I am thinking of is PATCH as opposed to PUT, GET, POST in REST. We are still at en early stage so we have not thought this really through yet, but in order to make sure that data received from external sources is not changed by clients we would like to have information about source on each element in order to not allow patches on them. As you mentioned in your document, we will probably also need an identifier for addresses and contact points/telecom in order to be able to update the correct item.

Ellen Gregusson (Aug 09 2019 at 09:20):

@Michele Mottini The information in the Provenance resource, is this available in the api to get for example patient, or do we need to do something special in the urls in order to get the provenance information related to each information element?

Grahame Grieve (Aug 09 2019 at 09:43):

have you looked at the X-Provenance header?

Ellen Gregusson (Aug 09 2019 at 12:10):

The use of the header is not documented for other HTTP methods than PUT og POST. How do we use it for GET?

Grahame Grieve (Aug 09 2019 at 12:11):

also PATCH

Grahame Grieve (Aug 09 2019 at 12:11):

you wouldn't use it with a GET since nothing is changing

Ellen Gregusson (Aug 09 2019 at 12:14):

What we need is to show to the client which element has been received from which source when they get patient/practionerinformation. The client should not be able to change information provided by an external source, but should be able to change information created by internal systems.

Michele Mottini (Aug 09 2019 at 12:34):

Every patient search would have to include also Provenance: .../Patient?_revinclude=Provenance:target&. . . .
Note that this is just a way to make that information available, clients are still free to do everything they want in term of PUT/POST/PATCH - you server would have to implement checks to prevent forbidden modifications

Ellen Gregusson (Aug 09 2019 at 13:13):

Is my understanding correct that we will not be able to get provenance information by using the GET method in REST, but that we have to do a search in order to find it? Will it be easy for the client to map the information from provenance with the information elements in Patient.

John Moehrke (Aug 09 2019 at 13:22):

you can use _revinclude=Provenance:target on any GET... provided the server supports it, which you can require that it does

John Moehrke (Aug 09 2019 at 13:24):

note that Provenance would indicate who made the change in a specific version of a resource. There is some client side processing one would need to do to determine which version changed a specific element (who updated the phone number?)

Ellen Gregusson (Aug 09 2019 at 14:47):

Does this mean that we have to fetch a provenance for each version of the Patient object, and the client will have to look through all of them in order to know who updated which element? If yes, I am not sure all our clients will be able to do this.
In the Norwegian person registry they have a field for source under each element in the profile. See example from test below where Kilde = source. Is there any possibility that the field can be added to the profile in FHIR, or do we have to solve this with extensions?

    "identifikasjonsnummer": [
        {
            "ajourholdstidspunkt": "2019-06-05T09:33:23+02:00",
            "erGjeldende": true,
            "kilde": "dsf-folkeregister-synkronisering",
            "gyldighetstidspunkt": "2019-06-05T00:00:00+02:00",
            "status": "iBruk",
            "foedselsEllerDNummer": "090524xxxxx",
            "identifikatortype": "foedselsnummer"
        }
    ],
    "status": [
        {
            "ajourholdstidspunkt": "2019-06-05T09:33:23+02:00",
            "erGjeldende": true,
            "kilde": "",
            "aarsak": "Innvandring",
            "gyldighetstidspunkt": "2019-06-05T09:33:23+02:00",
            "status": "bosatt"
        }
    ],

Michele Mottini (Aug 09 2019 at 15:16):

You'll have to go with extensions. Using Provenance would be the standard way to do it - so I think using an extension where a standard way to do the same thing (although more complex) exist would be discouraged (we have been there ourselves)

Michele Mottini (Aug 09 2019 at 15:18):

See https://chat.fhir.org/#narrow/stream/179166-implementers/topic/Source.20of.20information

John Moehrke (Aug 09 2019 at 15:20):

we do have an improvement request (CR) to add more detail to the Provenance as to what element within the target resource was updated. The problem being that this is not in the model today, and is not clear how it would work.

John Moehrke (Aug 09 2019 at 15:22):

It seems that CR (referenced in the chat Michele points at) did get approved. it has not yet been integrated into the build.

Grahame Grieve (Aug 09 2019 at 19:54):

you can use _revinclude=Provenance:target on any GET.

I think that this is not true; only on search. But's only a small change in syntax for a client:

GET [base]\Patient\X
Get [base]\Patient?_id=X&_revinclude...

Grahame Grieve (Aug 09 2019 at 19:55):

I agree that doing revinclude leaves the client sorting out which provenance is the most recent for which element, and that logic to do that belongs on the server.

Grahame Grieve (Aug 09 2019 at 19:56):

The client should not be able to change information provided by an external source, but should be able to change information created by internal systems.

So if they were the source of the information, they can change it, but if it come from elsewhere, they can't?

Brian Postlethwaite (Aug 11 2019 at 23:06):

There is also the VerificationResult resource that is similar to provenance however is appropriate when you're checking the content against multiple sources.
Yku might have received the data from X, however you've cross referenced it with Y and Z.

Ellen Gregusson (Aug 13 2019 at 12:51):

@Grahame Grieve If the information comes from an official registry our internal systems will not be allowed to change the information. For example gender, identifiers and official addresses. We will be able to change non official phone numbers, e-mail adresses and temporary addresses.

Grahame Grieve (Aug 14 2019 at 08:51):

so I think that this is definitely extension territory

John Moehrke (Aug 14 2019 at 14:33):

This level of fine-grain access control and data manipulation is not the kind of problem REST intends to solve. So you will need to deal with this in ways that REST is not going to help you with. REST model sees the resource object boundary as the smallest atom to manage.

Grahame Grieve (Aug 15 2019 at 02:35):

that doesn't mean it's impossible; it just explains why it's an extension

John Moehrke (Aug 15 2019 at 15:31):

yes. an alternative is that we re-examine the layout of each Resource to determine if they need to be split up due to logical authorization boundaries. Such as the given that the addresses could be independent resources that are pointed to by the Practitioner. I recommended this be part of the governance, especially when a resource gets very large.

Grahame Grieve (Aug 15 2019 at 19:04):

we do examine this. But we cannot divide in a way to make everyone happy - particularly in a case like this, where the granularity is at the element level

Jose Costa Teixeira (Aug 15 2019 at 21:06):

I think we (my project) will need to implement this in a way that we do not need to define on the standard what are those boundaries (which elements, or which sources)

Jose Costa Teixeira (Aug 15 2019 at 21:10):

I am not sure those boundaries are at design level (use case specific) or at runtime level (looking at k-anonymity). But I believe we cannot get a consensus on which elements are more critical/sensitive than others.

Jose Costa Teixeira (Aug 15 2019 at 21:29):

Ellen's description seems to be relying on a fixed set of rules, so a flag extension + provenance could do it.

Thomas Tveit Rosenlund (Aug 23 2019 at 13:20):

you can use _revinclude=Provenance:target on any GET.

I think that this is not true; only on search. But's only a small change in syntax for a client:

GET [base]\Patient\X
Get [base]\Patient?_id=X&_revinclude...

Would this be any different than searching for any Provenance resource referencing a specific Patient resource in target?
GET [base]/Provenance?target:identifier=X

John Moehrke (Aug 23 2019 at 14:31):

Provenance is about the resource it is pointing at in .target. Thus Provenance itself does not have a .subject element. Thus you could search on Provenance with a target of the Patient, but that would only give you Provenance records about Create or Update events on that Patient resource itself.

John Moehrke (Aug 23 2019 at 14:35):

The specific question that started this thread is asking for a method of having more distinguishing Provenance. That has been addressed in GF#21284

Thomas Tveit Rosenlund (Sep 04 2019 at 14:00):

Provenance is about the resource it is pointing at in .target. Thus Provenance itself does not have a .subject element. Thus you could search on Provenance with a target of the Patient, but that would only give you Provenance records about Create or Update events on that Patient resource itself.

Why only the Create or Update. The specific information about what is changed in the resource (recorded in the extensions of the target element discussed above). Should the information about what is changed in the Resource replace the information about the target resource itself?

Grahame Grieve (Sep 04 2019 at 20:39):

Should the information about what is changed in the Resource replace the information about the target resource itself?

I'm not sure what you mean

Thomas Tveit Rosenlund (Sep 06 2019 at 08:24):

The "subject" of the Provenance resource is called target. There is a proposal for two extensions to the target element to be able to describe what element is changed in the target resource. What I proposed was to search for all changes to a specific patient record using this transaction, Instead of the revinclude:
GET [base]/Provenance?target:identifier=X

@John Moehrke stated that this would only return the Provenance record about Create and Update events of the Patient record. I did not understand why it should not also return the delete, nullify transactions as well.

I guess his point was that the transaction result would not include the patient resource itself.

Grahame Grieve (Sep 06 2019 at 12:02):

search does not return deleted records. that's the most obvious reason why. And most systems do not retain the deleted records since they are deleted

John Moehrke (Sep 06 2019 at 12:22):

@Thomas Tveit Rosenlund delete is another action that a Provenance record can be made for. I simply was not exhaustive when I left it out. I also left it out as recording Provenance on delete will not be common, and thus domains that need it will specifically drive for it.

Thomas Tveit Rosenlund (Sep 06 2019 at 12:33):

search does not return deleted records. that's the most obvious reason why. And most systems do not retain the deleted records since they are deleted

Sorry I don't get it. The search I tried to specify does not search for the deleted Patient resource but instead searches for all Provenance resources documenting the changes of the Patient resource, including the delete record. That Provenance resource should still exist even though the Patient resource is deleted? My guess would be if you put in the reference to the Patient that is not version specific, you should indeed get all changes to that Patient resource, even delete.

After reviewing the Provenance resource again the documentation of the target reference states "The Reference(s) that were generated or updated by the activity"
If the target element of a delete transaction is left empty a search on target would not return any Provenance resource documenting the deletion. Is that even allowed as the target reference has cardinality of 1..*?

Thomas Tveit Rosenlund (Sep 06 2019 at 12:34):

Thanks for the reply @John Moehrke In that case some clarification of that point could be added to the documentation. It is not totally clear to me how that should work based on the existing documentation.

John Moehrke (Sep 06 2019 at 13:01):

Please submit a CR with improvement recommendation.

John Moehrke (Sep 06 2019 at 13:03):

The effect of deleting a resource is not defined in the core given the various policies on what that might mean in various context (clinical is different than research is different than registry ...). The effect upon the Provenance records is even less defined. It is possible to have a policy where delete of a resoruce does indeed delete all Provenance (EU right to be forgotten implemented in a non-clinical system).

Thomas Tveit Rosenlund (Sep 10 2019 at 11:31):

The effect of deleting a resource is not defined in the core given the various policies on what that might mean in various context (clinical is different than research is different than registry ...). The effect upon the Provenance records is even less defined. It is possible to have a policy where delete of a resoruce does indeed delete all Provenance (EU right to be forgotten implemented in a non-clinical system).

I agree, GDPR could demand the delete of all versions of Person and Provenance resources for the information about a given Person in non-clinical systems.

John Moehrke (Sep 10 2019 at 12:20):

if you are referencing the regulated fact that you must keep the data due to medical records retention regulation.. correct, but then no delete can be done on anything that might have been used...
More important overall is that once that regulated reason under Medical Records Retention expire, you MUST delete all data unless you get authorization from the patient to keep it for a defined purpose. So the delete must happen eventually.
See the GDPR analysis HL7 has done relative to FHIR https://confluence.hl7.org/display/SEC/FHIR+-+GDPR

Joost Reuzel (Sep 11 2019 at 12:45):

Would for this use-case the source field in the meta section of a resource not suffice? By setting the source field to the origin system in each of the resources, would it not be possible quickly identify what the 'external' vs 'internal' resources are?

Lloyd McKenzie (Sep 11 2019 at 12:55):

Depends what your rules are around distinguishing internal from external. Does something that's updated become 'internal'? What about something that came from outside but was "approved" to become part of the local record?

Joost Reuzel (Sep 11 2019 at 13:10):

Reading further, I see that Ellen needs source information per element of the resource. As the source field exist on the resource (meta) level, this won't be granular enough. Provenance indeed seems the way to go here...

Jose Costa Teixeira (Sep 11 2019 at 13:35):

@David Pyke @John Moehrke do you want to entertain a discussion for that? I am thinking we may need a standard extensions that can be used for DS4P and things like this. The concepts should be also clarified in any guidance - IMO it's less about "internal vs external" but "changeable acc policy X, not readable acc policy Y, accessible via consent X1"...

Jose Costa Teixeira (Sep 11 2019 at 13:36):

(i'm pinging you 2 because of the chat we had earlier an because i do not know the secprivacy agenda for next week...)

Thomas Tveit Rosenlund (Sep 12 2019 at 12:51):

if you are referencing the regulated fact that you must keep the data due to medical records retention regulation.. correct, but then no delete can be done on anything that might have been used...
More important overall is that once that regulated reason under Medical Records Retention expire, you MUST delete all data unless you get authorization from the patient to keep it for a defined purpose. So the delete must happen eventually.
See the GDPR analysis HL7 has done relative to FHIR https://confluence.hl7.org/display/SEC/FHIR+-+GDPR

Is there any meaningful difference differencing the Delete and Nullify codes described in https://www.hl7.org/fhir/valueset-provenance-activity-type.html
The way DELETE is described in this discussion, they seem to be similar.

John Moehrke (Sep 12 2019 at 20:52):

so that is a comprensive valueset, and therefore not uncommon for there to be overlapping terms. I read the definition and Delete is actual delete, where as Nullify seems to be setting the .status element in a way that results in the data being not normally visible.

Thomas Tveit Rosenlund (Sep 13 2019 at 07:09):

@John Moehrke agreed, it is actually described as a state change. Thanks for the clarification.

Last updated: Apr 12 2022 at 19:14 UTC