Stream: fhirpath
Topic: date math
Lloyd McKenzie (Nov 09 2020 at 04:14):
The validator is blowing up saying that toDate is 'not done yet' with this expression:
((now().toDate() - $this) <= 24 months) or value.length()=4
(context is Patient.birthDate)
Is there a way I can perform this function without using toDate()? (If I just remove it, the publisher yells that you can't use the '-' operator with two different types - dateTime and date. )
Richard Townley-O'Neill (Nov 09 2020 at 05:25):
Could you make them strings?
Richard Townley-O'Neill (Nov 09 2020 at 05:25):
:nauseated:
ryan moehrke (Nov 09 2020 at 15:09):
looks like there's a today() instead of now() does that work for you? http://hl7.org/fhirpath/#current-date-and-time-functions
Lloyd McKenzie (Nov 09 2020 at 19:40):
Awesome! Never thought to look for a date-specific function. That fixed my issue. Thanks @ryan moehrke
Brian Postlethwaite (Nov 11 2021 at 10:36):
With the date math functions, I'm adding support for this into the dotnet libs and have an issue with @2016 + 365 days adding to return 365. (defined in this section in the spec http://hl7.org/fhirpath/N1/#addition-2)
where is the "standard year length" defined, and is there a standard month length too?
What would happen if we said @2016-01 + 365 days should that also return the same value?
How about @2016-02 + 28 days?
The fhirpathjs engine returns 2016 which is what my initial implementation has done too, and @Ewout Kramer pointed out that it was wrong...
I know this doesn't have the partial date semantics either, but...
https://www.timeanddate.com/date/dateadded.html?d1=01&m1=01&y1=2016&type=add&ay=&am=&aw=&ad=365&rec=
Brian Postlethwaite (Nov 11 2021 at 10:54):
I personally think this should do what I (and what I assume the fhirpathjs implementation have done) for partials, assume was 1st of Jan that year (or whatever partial it was) perform the addition, then trim the days/months off (whatever the original precision was)
This magic average year surprised me - and there isn't a magic month length defined also...
ryan moehrke (Nov 11 2021 at 15:52):
there does seem to be a conversion table defined
http://hl7.org/fhirpath/N1/#quantity-conversion-functions
so according to this a standard month is 30 days
ryan moehrke (Nov 11 2021 at 15:54):
but it does seem rather strange that the spec requires you respecting variable length periods for when precision matches (?) but once there is a precision mismatch just use a standard conversion table, I'm not sure I understand the reasoning behind that decision
Bryn Rhodes (Nov 11 2021 at 18:14):
The motivation was to provide a consistent algorithm for applying to date/time arithmetic when the precision of the date/time value was lower than the precision of the time quantity being added. It does result in some surprises like those examples, but that's why we added those example specifically, to illustrate that behavior. The only time we can guarantee calendar arithmetic semantics is when the time quantity can be anchored to the date/time value, and that requires at least the precision of the time quantity.
Brian Postlethwaite (Nov 11 2021 at 21:28):
Thanks Bryn, at least I see the full conversion routines now.
@Paul Lynch I guess we need to update the fhirpathjs implementation too here.
Are those specific examples in the fhirpath test suite? (and the Java implementation also doing the same thing)
Paul Lynch (Nov 12 2021 at 16:48):
@Brian Postlethwaite -- Yes, thanks for pointing that out. I will add a ticket for it.
Brian Postlethwaite (Nov 15 2021 at 21:57):
In implementing the date math on a partial month value, there is no logic defined on how to add weeks...
As I see it there are 2 choices here:
case "weeks":
if (dateValue.Precision == DateTimePrecision.Year)
dto = dateValue._parsedValue.AddYears((int)(addValue.Value / 52));
else if (dateValue.Precision == DateTimePrecision.Month)
dto = dateValue._parsedValue.AddMonths((int)(addValue.Value * 7 / 30)); // convert to days in a week over a month
else
dto = dateValue._parsedValue.AddDays(((int)addValue.Value) * 7);
break;
or
case "weeks":
if (dateValue.Precision == DateTimePrecision.Year)
dto = dateValue._parsedValue.AddYears((int)(addValue.Value / 52));
else if (dateValue.Precision == DateTimePrecision.Month)
dto = dateValue._parsedValue.AddMonths((int)(addValue.Value/52*12)); // convert to weeks in the year over a month
else
dto = dateValue._parsedValue.AddDays(((int)addValue.Value) * 7);
break;
the latter of these makes more sense to me.
Bryn Rhodes (Nov 15 2021 at 22:44):
The spec has language that talks about dealing with weeks as 7 days, so I think it would make sense to say, multiply the weeks value by 7 and then do the same thing you would do with days.
Brian Postlethwaite (Nov 16 2021 at 02:54):
(checking with Excel) the difference between the 2 options gives different answer at 30 and 43 under 52, but when you go over 100 weeks the 2 options give the same result only 50%(ish) of the time.
So I think we should get this defined...
I'll log a tracker, and I do think given the way the others are defined, we should go with the more accurate intention of average weeks in a year.
And beyond 377 weeks, they are never the same
Brian Postlethwaite (Nov 16 2021 at 05:06):
https://jira.hl7.org/browse/FHIR-34315
Bryn Rhodes (Nov 16 2021 at 15:18):
Is there a reason not to handle it the same as days? i.e. always treat it like weeks * 7?
Brian Postlethwaite (Nov 16 2021 at 23:06):
Accuracy? Ave days in a month is less precise than avg weeks in a month, it's all feeling very random to me.
(you don't have days precision, and you haven't provided days precision)
Richard Townley-O'Neill (Nov 17 2021 at 00:56):
Do they work better with the value for weeks in a year not being 52, but being 365/7 (~52.143) or 365.25/7 (~52.179)?
Bryn Rhodes (Nov 17 2021 at 14:39):
We already don't have accuracy with un-anchored calendar calculations. Seems like consistency with days behavior is preferable, especially given that we say in the spec "Calculations involving weeks are equivalent to multiplying the number of weeks by 7 and performing the calculation for the resulting number of days."
Brian Postlethwaite (Nov 17 2021 at 18:33):
Ok. That's what I'll put in the dotnet implementation then l, and reference back to the tracker.
Seems that's what you and Ewout both concluded too.
Brian Postlethwaite (Nov 24 2021 at 10:00):
In updating the dotnet implementation to handle the date math, we fixed the handling quite a lot and in doing so we've come across a few unit tests from the spec that are failing now that we believe are actually wrong according to the spec.
==== Running tests from file 'TestData\fhirpath\tests-fhir-r4.xml' ====
FAIL: testTypes - testStringQuantityDayLiteralToQuantity: '1 day'.toQuantity() = 1 'd'
(AssertFailedException) Assert.Fail failed. Expected a non-empty result
According to the fhirpath spec, 1 year is approximately 1 a, not equivalent, and same with day(calendar) and d (UCUM)
http://hl7.org/fhirpath/N1/#time-valued-quantities
1 year'.toQuantity() = 1 'a'
And this one too - under what basis does the toString on the quantity do this conversion?
(as the fhir type Quantity can hold both, and thus needs to be able to hold it and why would a to string on a fhir type produce something different? - note that we couldn't spot anything in the spec explicitly saying that it SHOULD do this)
FAIL: testTypes - testQuantityLiteralWeekToString: 1 week.toString()
(AssertFailedException) Assert.AreEqual failed. Expected:<1 'wk'>. Actual:<1 'week'>
Brian Postlethwaite (Nov 24 2021 at 10:02):
Note that there is also this section that talks about using quantity too, which further confuses the issue. http://hl7.org/fhir/fhirpath.html#quantity
If they are forced to be calendar based - then there are issues with the equivalence to equality operations defined in the core spec...
Ewout Kramer (Nov 24 2021 at 10:51):
Hi @Brian Postlethwaite - I remembered we discussed this some time ago, and I found the tracker: https://jira.hl7.org/browse/FHIR-28144
Ewout Kramer (Nov 24 2021 at 10:51):
And the previous discussion: https://chat.fhir.org/#narrow/stream/179266-fhirpath/topic/Date.2FTime.20comparison.20vs.20equality
Ewout Kramer (Nov 24 2021 at 10:54):
This also means updates to this table: https://hl7.org/fhirpath/#time-valued-quantities.
Brian Postlethwaite (Nov 24 2021 at 10:55):
So does the outcome then mean that there is no difference between equivalence and equality?
(as they are considered the same, and always forced to calendar units?)
Ewout Kramer (Nov 24 2021 at 11:16):
The difference remains for units above weeks (so month and year)
Ewout Kramer (Nov 24 2021 at 11:18):
It does not explain why (1 week).ToString() should return the string "1 'wk'" according to the unit tests. They may be considered equal under the "=" operator, but does this mean ToString() does a normalization to UCUM?
Brian Postlethwaite (Nov 24 2021 at 11:27):
I don't understand how 1 year to quantity compares equal to 1a and not equivalent.
Ewout Kramer (Nov 24 2021 at 12:09):
It's not equal (=), but it is equivalent (~), which was already the case.
I think the new table looks like this:
Ewout Kramer (Nov 24 2021 at 12:09):
Ewout Kramer (Nov 24 2021 at 12:10):
As an extension, I guess then that date addition is now also allowable for mo/month, wk/week, etc ?
Paul Lynch (Dec 02 2021 at 16:43):
Ewout Kramer said:
As an extension, I guess then that date addition is now also allowable for mo/month, wk/week, etc ?
You are referring to a tracker item (FHIR-28144) that is in a Triaged state, and not yet approved. If it is approved (as seems likely, but when is less clear), will that affect N1, or a future release of FHIRPath? We are trying to correct the fhirpath.js behavior, and I am not sure whether we should follow the current specification or the tracker item.
Bryn Rhodes (Dec 03 2021 at 22:56):
That tracker is a technical correction, so it is auto-approved, I'd follow the tracker on that.
Last updated: Apr 12 2022 at 19:14 UTC
