Stream: implementers
Topic: currentAsOf extension?
Josh Mandel (Mar 16 2021 at 05:17):
FHIR's Meta.lastUpdated element tells you how long something has been stable (time of last update). But in data aggregation scenarios there's a distinct desire to know when data are stale, e.g. with something like a currentAsOf extension, so even data that hasn't changed could be marked with the time they were last refreshed by the aggregator from primary sources. https://github.com/smart-on-fhir/smart-scheduling-links/issues/16#issuecomment-799954590 has an example.
But... If an aggregator actually populated this extension it'd be causing an update, and presumably Meta.lastUpdated would tick up because of this. On this evaluation, it seems fine for aggregators to just skip the currentAsOf extension and repopulate lastUpdated as $now each time they refresh content.
Anybody have a different take?
René Spronk (Mar 16 2021 at 07:40):
AFAIK updates to meta don't cause _lastUpdated to be updated. But whether that's mandated by the spec.. that's the q.
John Moehrke (Mar 16 2021 at 12:35):
yes, unfortunately there is some hints in the spec that say that updates to meta don't update _lastUpdated. I think this is a bad general rule. It is not a bad consideration for policy. That is to say to declare that a server can have a policy that would allow some specific kinds of updates to meta to not be reflected in _lastUpdated.
The big problem I see with any policy that does not update the _lastUpdated, is that it puts provenance in question overall. If there is any rule that allows _lastUpdated to not be changed, then one must question what other rules are around that don't do something.
John Moehrke (Mar 16 2021 at 12:39):
Josh, Your use-case seems reasonable within a closed system. That is that there is a database level solution. But to expose this at the FHIR API brings up many problems. Why does it need to be exposed at the FHIR level?
Provenance could do what you are asking, but would be very heavy as even a minimally populated Provenance is bigger than a timestamp extension. However, one Provenance record can tag thousands of resources as "all of these are current as of today"
Josh Mandel (Mar 16 2021 at 14:04):
Thanks for the call out that changes to mets needn't affect lastUpdated. (Is there a reference for this?)
John Moehrke (Mar 16 2021 at 14:14):
I think it was in the $meta-add operation, but don't see it there now
John Moehrke (Mar 16 2021 at 14:14):
but using the $meta-add was at a time defined as not updating the _lastUpdated
John Moehrke (Mar 16 2021 at 14:17):
I think I successfully got the bad fixed policy removed.
Jean Duteau (Mar 16 2021 at 15:32):
In my FHIR SPL work, we need to know that exact question - how current is this data - and I came to the same conclusion as you did - any element that I added to track that would just duplicate the Meta.lastUpdated, so we are just using that and are telling users to just send a no-change update to make things current.
Lloyd McKenzie (Mar 16 2021 at 15:47):
There are technical operations that don't reflect a 'review' of the data that could change _lastUpdated. _lastUpdated is also server-specific. If I receive a record from another system that represents "knowledge as of 5 years ago", the _lastUpdated in my system could be 'today' if I received the record today. That doesn't mean that that information is 'current' as of today.
Josh Mandel (Mar 16 2021 at 15:52):
In the context of this thread, "currentAsOf" represents: "I [the aggregator] last queries the information source for this data at ____". (Agreed the word "current" could be misleading; don't read too much into that @Lloyd McKenzie.)
Rik Smithies (Mar 16 2021 at 15:53):
@Jean Duteau I seem to remember doing a no-change update to a server (HAPI?) and it just ignored it, noticing that there was no change. It didn't update the version number. I am not sure about lastUpdated but I would assume not.
Lloyd McKenzie (Mar 16 2021 at 15:58):
If you go through and update tags or profile declarations or anything else purely locally (without grabbing the data from source of truth) your lastUpdated will still change
Jean Duteau (Mar 16 2021 at 16:21):
Rik Smithies said:
Jean Duteau I seem to remember doing a no-change update to a server (HAPI?) and it just ignored it, noticing that there was no change. It didn't update the version number. I am not sure about lastUpdated but I would assume not.
Well, that would suck! We've been assuming that sending a PUT with a complete instance would create a new version even if the data was exactly the same. Something to note for our testing!
Lloyd McKenzie (Mar 16 2021 at 16:59):
I presume you meant PUT? It is up to a server whether it actually creates a new instance. (It's also up to the server whether it changes what you asked to be changed...)
Jean Duteau (Mar 16 2021 at 17:00):
Isn't that what I said? :) I changed it as I constantly get POST and PUT backwards. That's news to me that it's up to the server on whether it actually creates a new instance. Hmm...
Lloyd McKenzie (Mar 16 2021 at 17:02):
Even more fun, on a POST, you might get back the id of a pre-existing resource rather than a new one created :)
Paul Church (Mar 16 2021 at 17:03):
I think you have to look at the response to a POST or PUT to make sure the server did what you expected.
Jean Duteau (Mar 16 2021 at 17:04):
okay, so this tells me that to ensure that Meta.lastUpdated actually gets updated when I do an update, that I have to add a new field via an extension that is the equivalent of Meta.lastUpdated and that I will update when I do the update so that the update will get done?
Lloyd McKenzie (Mar 16 2021 at 17:21):
If the server happens to support that extension. Rather than doing that, I'd probably set some expectations in the IG about updates forcing a new version or maybe define a 'touch' operation rather than adding a useless data element.
Jean Duteau (Mar 16 2021 at 17:23):
upon re-reading, I'm presuming this means that if I'm a client, I can't necessarily expect the server to do what I ask. But if I'm designing the server and using IBM's server or HAPI's server, I'm hoping that I can force an update and/or an update of the lastUpdate.
Lloyd McKenzie (Mar 16 2021 at 17:26):
Yes, that should be possible. Though you'd have to check with the author of the server in question.
Lloyd McKenzie (Mar 16 2021 at 17:26):
(And also make sure they don't 'optimize' things in an update and you don't recognize the change...)
Rik Smithies (Mar 16 2021 at 18:07):
is this behaviour not standardised? we wouldn't want a solution that would fail if a new server vendor was switched in one day.
Jean Duteau (Mar 16 2021 at 18:56):
okay, having read the relevant parts of the specification, then the SHOULDs about data in matching data out can become SHALLs for an implementation's specific use-case. If the concerns about data integration and transactional integrity are not actually concerns, then one can develop one's server appropriately. As to whether the OTS servers support that is a question that everyone deciding on using one of those would need to ask.
Rob Brackett (Mar 18 2021 at 00:30):
Hi, I’m the person who posted the GitHub issue @Josh Mandel referred to in the OP. I’m pretty new to HL7 & FHIR, so apologies if I’m missing the obvious. The currentAsOf extension here is hypothetical right? Am I understanding correctly that we’re talking about something roughly like:
{
// some domain resource
"extension": [{
"url": "http://hl7.org/fhir/StructureDefinition/currentAsOf",
"valueInstant": "2021-03-17T12:08:36Z"
}]
}
?
Josh Mandel (Mar 18 2021 at 00:49):
Welcome @Rob Brackett! Yes hypothetical extension for the moment :) And indeed, the example you're writing out here is precisely what I had in mind.
Josh Mandel (Mar 18 2021 at 00:50):
We could suggest this as a new core extension, too. I'll submit a Jira issue for it.
Josh Mandel (Mar 18 2021 at 00:54):
Added FHIR-31567
Rob Brackett (Apr 19 2021 at 19:56):
Checking in here… the current status (not finalized, but ready for a vote) is that we’d have this?
{
"resourceType": "Slot", // Or whatever
"meta": {
"extension": [
{
"url": "http://hl7.org/fhir/StructureDefinition/lastSourceSync",
"valueDateTime": "2021-03-15T23:00:01Z"
}
]
},
"id": "123",
// etc...
}
Last updated: Apr 12 2022 at 19:14 UTC
