Stream: implementers
Topic: Has there been a use case for valueDate?
John Silva (Apr 12 2019 at 19:04):
Has there been a use case for having valueDate (instead of valueDateTime) as one of the types for Observation (and maybe other resources)? For the same reason that birthDate is date only, there may be a need to report an Observation with only the date (and not worry about timezone issues).
Grahame Grieve (Apr 12 2019 at 20:49):
dateTime can do date only
John Silva (Apr 12 2019 at 21:05):
OK, but the need is to know that the observation value[x] property is a date (only) so the receiver knows to handle it as a date, that is why I was looking for a specific choice of valueDate
Eric Haas (Apr 12 2019 at 23:58):
use a profile, add an invariant and regex the datetime to date.
John Silva (Apr 13 2019 at 12:04):
Yes, that might work but the observation coming in can have either a valueDateTime or a valueDate; this needs to be determined at runtime. It would be much easier for the receiver's logic if it knew it was a date (only) because of the property name. I suppose the standard way would be to define an extension for the Observation.value[x] to have a valueDate property and bind it to the 'date' type not the 'datetime' type; I can do that. However, my question is asking if others have this use case; that is, needing to specific something in an observation as a date (only) field? I can imagine there are many measurements that do not have the granularity of a dateTime, especially patient-taken observations like "what was your weight yesterday?" or similar and to represent these as date/times is not really an accurate representation of when the observation was measured.
Lloyd McKenzie (Apr 13 2019 at 14:13):
At runtime you can just look at the length of what you received. I wouldn't recommend an extension nor count on receiving one. In general, the complexity should live on the server side.
John Silva (Apr 13 2019 at 14:27):
Not sure why complexity should be on the FHIR server side ... however, if I want to ensure that the sender (not the server) is sending dates only, then I either need a profile or an extension (or built-in) support for valueDate. If I know that a measurement (obs) should not be reported with a granularity of more than a day, I don't want to sometimes deal with dates only and other times date/times (and tz offsets). For example, if I want to make sure that last date of having a cigarette is only specified as a date, then I need a mechanism to ensure that and not have to make a decision on a length calculation based on what was sent to me (the server). [However, my use case is about ensuring the FHIR client knows that it is always dealing with a date-only field.]
Eric Haas (Apr 13 2019 at 16:26):
They could inspect the meta profile.
Jean Duteau (Apr 13 2019 at 16:58):
Not sure why complexity should be on the FHIR server side ... however, if I want to ensure that the sender (not the server) is sending dates only, then I either need a profile or an extension (or built-in) support for valueDate. If I know that a measurement (obs) should not be reported with a granularity of more than a day, I don't want to sometimes deal with dates only and other times date/times (and tz offsets). For example, if I want to make sure that last date of having a cigarette is only specified as a date, then I need a mechanism to ensure that and not have to make a decision on a length calculation based on what was sent to me (the server). [However, my use case is about ensuring the FHIR client knows that it is always dealing with a date-only field.]
The situation you are describing is where the client and server are going to live and die by the profile, i.e. you are want assurance that the client is only ever going to send you dates. If that is the case, then you don't need anything more than the dateTime and a constraint in your profile that you can only send dates and anything else is an error.
The bigger FHIR picture is that you could be receiving observations from anywhere so you're going to have to validate the incoming observation value - and whether the profile asserts that it is a date or you do the above server logic to make sure it doesn't have more than 8 characters, validation is being done by the server. That is Lloyd's point. You can't be assured that every client is going to abide by your profile and only send you dates, so you have to validate. If you're validating, then you might as well leave the observation as-is and just check lengths.
Lloyd McKenzie (Apr 13 2019 at 20:06):
If someone sends you a dateTime down to the millisecond for an Observation "date of last cigarette" (unlikely :>), it would be best to not treat it as an error. Just store the date and throw away the rest. That's the lowest cost solution for everyone involved and is still a safe thing to do.
Grahame Grieve (Apr 14 2019 at 07:40):
the use case : constrain to just date. Can be done, as Eric said above, by making a constraint in the profile for the length of the date time. Or, as you say, by also allowing the type 'date' and constraining the type to date. That's slightly cleaner for the profiler, but much messier for implementers as now any observation might have a value that is either a date or a date time. Yuck. So we don't add the type date and accept a slightly clunkier way to do the constraint
John Silva (Apr 14 2019 at 10:52):
The explanations do not fit my use case because I didn't explain it completely. My FHIR server needs to be able to accept both valueDateTime and valueDate (if there was such a thing) for FHIR Obs. It is NOT the server that I want to make the decision about if something is a full date/time, it's the sending client. (It's sort of like the precision discussion; the client knows the precision of the data, not the server.) So @Lloyd McKenzie* answer of 'truncate the date' doesn't make sense for two reasons, first, it would cause problems with timezone boundaries (just like the birthDate problem), and second because the server needs to be 'dumb' about this; it just stores whatever is sent (though if I had a valueDate dataType the server could do validation checking for that data type). I also have a second client that also wants to be sure that when it has a date only, it only formats the result as a date (for display purposes), as indicated by the fact the first client sent in the date as valueDate. It doesn't seem that the alternative approaches suggested address this use case.
Grahame Grieve (Apr 14 2019 at 10:54):
I'm not sure how it's not addressed. The creator of the resource provides just a date in the datetime if that's what it wants to do. The system displaying the resource finds just a date, so that's all it displays.
John Silva (Apr 14 2019 at 11:47):
Still not addressed -- how does the 2nd client know that the sender intended to send a date only if it received a date and time? I see this similar to the precision issue --- it's really only the sender of the data that knows the source system's data precision, NOT the FHIR server, NOT the (2nd) FHIR client. In healthcare, the sending devices are calibrated to a certain precision (and accuracy) and the receiving client should not be making arbitrary decisions about 'truncating' or 'adding digits' to that precision. That's what would be occurring with the datetime being truncated by the FHIR server or subsequent FHIR clients.
Lloyd McKenzie (Apr 14 2019 at 15:08):
Why would any system add digits?
Jean Duteau (Apr 14 2019 at 18:51):
Still not addressed -- how does the 2nd client know that the sender intended to send a date only if it received a date and time?
Why would the sender have sent a date and time if it only meant to send a date? If it wants to send just a date, why isn't it just sending a date?
Lloyd McKenzie (Apr 14 2019 at 19:35):
It's totally possible to send just a bare date (or even just a year) in a dateTime.
John Silva (Apr 15 2019 at 10:10):
It's not that the sending system would arbitrarily add digits (of precision), i.e. add a time, it's that the '2nd client' has no idea if the first system intended to send date and time or not. (That's to Lloyd's suggestion of truncating the time when the 2nd receiver receives the date/time -- it has no idea that the original sender's intention was to send date only.) It's also true that a sending system could send less that a full year/month/day; that is syntactically allowed by FHIR's date (and datetime) data type. That begs the question, in any good system, what would a receiver do with a year only if it was expecting a full date? Just because the FHIR server allows it by following the 'letter of the law' of the data type, doesn't mean that it makes clinical sense to a user of that data. So, all this theoretical discussion -- my practical use case is to be able to use the valueDate datatype (if it existed) to communicate the sender's intention of sending ONLY date so that the 2nd FHIR client knows that it should expect date only and doesn't have to worry about trying to do 'dangerous' truncation or adaptation of a full date/time with TZ info.
Grahame Grieve (Apr 15 2019 at 10:49):
if the sender provides only a date, then the obvious and only possible conclusion to draw is that it meant to send a date. None of us are following you when you say:
the '2nd client' has no idea if the first system intended to send date and time or not.
John Silva (Apr 15 2019 at 12:26):
The '1st client' sends many FHIR Observations, many (most) of them are intended to communicate date/time however, a small subset of these FHIR Observations are meant to communicate date only. Neither the FHIR Server nor the 2nd client (the receiving client if you will) would like to be able to differentiate those Observations that were intended as dates with the valueDate field (date data type). Without an a priori agreement (e.g. based on a subset of FHIR identifier codes) the 2nd client wouldn't know which Observation's dates are date vs datetime data types. Yes, if they received a data-only there's not much else they can do but if they received a date/time in an observation that should be date only, then what would the receiving client do?
Grahame Grieve (Apr 15 2019 at 13:04):
if you get just a date, what else could it be but intended to be a date? if you get a date/time when you were expecting a date, how does it make any difference whether you use a different type or not?
John Silva (Apr 15 2019 at 13:58):
If the sender sends me a date/time and my code expects only a date, then I can't just truncate the time portion because it might not give the correct date (i.e. not account for timezone). @Lloyd McKenzie has explained before about why the Person.birthDate was using date not datetime datatype for this exact reason (timezone concerns). Why can't that same logic be used for the Observation.value[x] property?
Lloyd McKenzie (Apr 15 2019 at 14:55):
I don't understand why it's a problem to truncate the date. If you capture that last cigarette was at 2:52pm 3 weeks ago and the receiver just captures the date from 3 weeks ago and throws away the timezone information, who cares? Why is that not safe? If the sender wants to send only a date, then they send only a date within the dateTime and omit the time information. If they send more precision than the receiver can handle, they throw away the extra precision. (E.g. some systems might be able to store hours and minutes but not seconds or sub-seconds - they'd throw away the extra digits too.)
John Silva (Apr 15 2019 at 17:50):
My cigarette example doesn't illustrate the point. It is clinically relevant in some cases what exact date an observation occurred, take for instance some value being totaled for a day or a feeding tube being inserted into a patient. It is not 'just ok' to truncate the seconds and timezone info because that might not produce the correct date and then the total would be for the wrong day. It is also not correct to just truncate the time and timezone info for a birth date, which is why FHIR (as I think you explained) has the Patient.birthDate as a date (only) property. I don't know enough about how FHIR specifies data precision but the same kind of thing happens for other measurements (observations) too; precision is important and there doesn't seem to be a way to communicate that without an a priori, out-of-band, agreement between exchanging clients. (and JSON, as we know, doesn't maintain precision without an external 'helper library' so that's implementation specific, not part of the FHIR standard, per-se.)
[I guess I'm not able to make this point so I'll stop. I was hoping someone else had a similar use case but maybe this chat only reaches a small audience so that others that might have a similar use case are just 'not here'.]
Grahame Grieve (Apr 15 2019 at 21:50):
I don't know about use cases. there's quite a lot of people following the stream, but they're probably not following this topic any more - it's a busy stream. But we don't follow what the problem is. If the sender sends a date time, and it's not easily truncatable to date, how does changing the data type make any difference. I don't think any of us have followed your argument that allowing 'date' as a type makes a difference to the underlying problem you describe, which is inherent in the data itself
John Silva (Apr 16 2019 at 10:24):
The sender should NOT send a date time if the Observation.value[x] is (or could be) specified as valueDate not valueDateTime. It would then be an error for a client which 'fills in' the valueDate field with a full date/time; the FHIR server validation could reject the bundle if this was sent. Again, the client (because of a clinical use case) knows that it is sending date only so it would use the valueDate property to 'signal' to the FHIR server and downstream FHIR clients that was its intention. Later, a FHIR client, seeing the Observation.valueDate property would know that it would (and can only if FHIR server did validation) receive a date only. I'm not sure how to say this any more clearly?
Lloyd McKenzie (Apr 16 2019 at 12:49):
In FHIR, the general rule is to not restrict a client from sending data that they have, but to instead have the receiver ignore what they don't need/support. The former forces the client to have a different interface for each receiver. If one receiver wants a time and the other wants only a date and you force different types, then the client needs two interfaces. It's much better for the client who only wants a date to strip the data they don't want. If a server sends only a date in a dateTime, that very clearly signals its intention. Remember, dateTime allows you to send only a date - there's no zero-padding.
John Silva (Apr 16 2019 at 14:12):
@Lloyd McKenzie -- what you describe misses my use case --- what you say might be completely true from a FHIR spec perspective but doesn't match the clinical use case I'm trying to describe.
Lloyd McKenzie (Apr 16 2019 at 14:15):
I'm not understanding the clinical case. If you have an Observation where you only want a date and the source system captured a time, then you'll get the time and can throw it away. If you accept a time and the source system only captured a date, then you'll see that they only captured a date and will have to live with that.
Lloyd McKenzie (Apr 16 2019 at 14:16):
(or could raise an error and say you need more information)
John Silva (Apr 16 2019 at 14:23):
If the source system has a date only field (e.g. on a UI), that's ALL it can send but FHIR Observation forces the valueDateTime property to be used because that's the only way to do this. Yes, the source system can fill in only the yyyy-mm-dd but now a subsequent receiver does not know that the source system intended this whereas if there was an Observation.valueDate this intention would be a given. I don't know if profiling can do this or communicate this intention or not. (The other point is that many/most Observations would send full date/time, only specific ones would send date only, maybe identified by a certain set of identifiers.)
Lloyd McKenzie (Apr 16 2019 at 14:47):
How would a subsequent receiver not know that's what the source system intended? The value transmitted would be something like "2019-04-16" - and that would be what is transmitted to subsequent receivers too.
Lloyd McKenzie (Apr 16 2019 at 14:48):
"2019-04-16" is a valid dateTime value just as it's a valid date value.
Yunwei Wang (Apr 16 2019 at 18:23):
I think what he said was that server sitting between sending client and receiving client may change the date value to dateTime. @John Silva Please correct me if my understanding about your use case is wrong.
I don't think that is an FHIR issue. If the server can interpret the inbond date value correct, then the server should be able to save and resend the vaule in the correct format. If not, that server has a business logic problem.
Lloyd McKenzie (Apr 16 2019 at 18:47):
The date value is a dateTime
Lloyd McKenzie (Apr 16 2019 at 18:47):
If you're talking about how it's persisted, we don't care. Systems are expected to retain the received precision. So if you add zeros in your database, you have to strip them when you ship the data out again.
Lloyd McKenzie (Apr 16 2019 at 18:48):
(The same as when they receive a decimal)
John Silva (Apr 16 2019 at 18:48):
@Yunwei Wang -- good point. If the FHIR server always stored valueDateTime as a, e.g. full datetime object in a database, then when another client makes a request of it, how does the FHIR server know to send it only date? The 'precision' is missing; dateTime implies 'precision' of full date/time (yes, FHIR allows subsets of full date time in the data type but there's no way to communicate that precision.) The same problem of precision happens with numeric data (though I seem to remember someone here talking about FHIR developing an approach to deal with this problem.)
John Silva (Apr 16 2019 at 18:49):
@Lloyd McKenzie -- yes, and FHIR doesn't seem to have a way to deal with the precision of a decimal value (it's even worse with JSON persistence since JSON doesn't deal with this well)
Yunwei Wang (Apr 16 2019 at 18:56):
@Lloyd McKenzie Maybe I didn't make it clear. For example, client send '2019-04-16' to server and server MAY resent the value as '2019-04-16T00:00:00-05:00' based on its implementation. Though both a dateTime type but the meaning is different.
I agree with you that this is NOT a FHIR issue. Server need to implement correctly to deal with that.
@John Silva I have seen different solution on server persistance layer. The simplest is save the value as string so the server always has the correct precision as what received.
Lloyd McKenzie (Apr 16 2019 at 19:05):
As I said, if I send a server 2019-04-16, when I subsequently query, the server MUST return 2019-04-16. It can't send 2014-04T00:00:00-06:00. Similarly, if I send 2019-04, then when I query, that's what it must give back. The simplest may be to store a "precision" element along with the dateTime value, though storing a string is also a possibility. FHIR doesn't really care how the persistence layer works, only that the data received is retained (including precision). It's no different than sending "5.00" in and expecting "5.00" to come back, not "5.0" or "5".
John Silva (Apr 16 2019 at 19:29):
@Lloyd McKenzie OK, need to read up on the FHIR specs as to what a compliant server MUST do. However, in the case of numerics and with JSON exchange, precision is a problem since JSON would represent 5.00 as 5 and the original precision is lost. Does FHIR have a way to communicate 'original precision' ?
Lloyd McKenzie (Apr 16 2019 at 19:44):
The documentation on JSON warns implementers about this and tells them they may need a custom parser because of this (see the 'dragon' section here: http://build.fhir.org/json.html#primitive)
John Silva (Apr 16 2019 at 20:51):
Yes, I remember reading about this and I already knew about the JSON problem with not being able to preserve precision on numbers. However, that now becomes a server implementation detail. It still seems like there should be some specification way of stating precision, that is not buried in implementation details. A system I worked on (not based on FHIR) specifically stored precision and offset and base, along with units, because all of these are important aspects of clinical measurements.
Hmm, see how OpenEMR represents a measurement (DV_QUANTITY) with real number, precision, units, and accuracy.
https://specifications.openehr.org/releases/RM/latest/data_types.html#_dv_quantity_class
It's also interesting to see how they model Date, Date/Time and Partial Date/Times (with uncertainties)
https://specifications.openehr.org/releases/RM/latest/data_types.html#_date_time_package
And they have a date-only data type (class): https://specifications.openehr.org/releases/RM/latest/data_types.html#_dv_date_class
Lloyd McKenzie (Apr 16 2019 at 21:34):
The precision is clearly represented on the wire. How systems manage it internally is up to them. Our support for partial values is driven by what the XML schema types can manage (and it's past the point where we can really make changes there).
Grahame Grieve (Apr 16 2019 at 22:21):
@John Silva the fundamental issue you have is that it's not enough to simply provide a date, you have to *additionally say that you intended it be just a date". Where as we hold that just saying a date is enough. From my perspective, you're trying to denormalise things in order to achieve... something that we're not clear on. The writer has a way to communicate clearly, the reader has a way to understand clearly, and anyone writing a profile has a way to say what is expected clearly. You haven't shown anything that can't be done with what we have, only a complaint that isn't being done in the way you think it should, to double say things.
The issue with decimal in JSON is different, in that we use the JSON number format, and json processing libraries often trash the precision. We argued about this a long time ago, whether to also include precision as an extra field or not - and chose to do it the way do it now with a warning. But the date issue is different - it's a string in json, so it's not the json libraries that trash the date or not, it's the processing application that decides. And the specification is clear: conformant applications have to handle the data type correctly across multiple precisions
Brian Postlethwaite (Apr 17 2019 at 12:37):
If your servers persistence had valueDate or valueDatetime, and you're selecting to put them in seperate DB cols, you can use the same storage, and when not time portion, store in that col, then you're in the same place and conformant.
This is what my server actually does with searching data. So using the valuseDate doesn't really add anything new.
Last updated: Apr 12 2022 at 19:14 UTC
