Stream: dotnet
Topic: FhirDateTime.ToDateTimeOffset() unexpected bahaviour
Brian Postlethwaite (Oct 18 2018 at 23:38):
I've done a partial write-up of an odd behavior I've found in the FHIR client.
new FhirDateTime(“2015-01-02”).ToDateTimeOffset(new TimeSpan(-5, 0, 0)).Day != 2
http://brianpos.com/2018/10/19/dates-and-timezones-in-the-fhir-net-api/
Planning to do more research and a recommendation on how we could proceed.
@James Agnew, not sure if there is something similar in the Java Client libs too.
FhirDateTime testDate = new FhirDateTime("2015-01-02");
Trace.WriteLine($"Undeclared Timezone: {testDate.ToDateTimeOffset().ToString()}");
Trace.WriteLine($"Melbourne: {testDate.ToDateTimeOffset(new TimeSpan(11, 0, 0)).ToString()}");
Trace.WriteLine($"Boston: {testDate.ToDateTimeOffset(new TimeSpan(-5, 0, 0)).ToString()}");
This test gives this result
Undeclared Timezone: 2/01/2015 12:00:00 AM +00:00 Melbourne: 2/01/2015 11:00:00 AM +11:00 Boston: 1/01/2015 7:00:00 PM -05:00
Angus Millar (Oct 19 2018 at 07:37):
I can see how to get the values you are asking for, something like this still needs work:
public static DateTimeOffset? ToNewDateTimeOffset(this Hl7.Fhir.Model.FhirDateTime FhirDate, TimeSpan TimeSpan)
{
if (FhirDate.Value.Length < 11 )
{
DateTime? TempDateTime = FhirDate.ToDateTime();
if (TempDateTime.HasValue)
{
TempDateTime = DateTime.SpecifyKind(TempDateTime.Value, DateTimeKind.Unspecified);
return new DateTimeOffset(TempDateTime.Value, TimeSpan);
}
return null;
}
else
{
return FhirDate.ToDateTimeOffset(TimeSpan);
}
}
Angus Millar (Oct 19 2018 at 07:38):
But I don't understand how you would use it in your server search indexing, how do you know what Timezone to use on index commit or on the search query?
Brian Postlethwaite (Oct 22 2018 at 03:49):
Yes, Hence my desire to redo the implementation in our server.
Angus Millar (Oct 25 2018 at 08:46):
Brian if it helps I tackled this somewhat differently. Still not great but as close as I could possibly get with this problem to make it reasonable. Love to hear what you come up with.
So below are my notes on what I did. I will also point out that every date and datetime index I store is stored with two DateTimes, a Low and a High. So every single one is a range, not a single dateTime. Then all the searchers on these indexes are range on range searches. They are also all converted and stored in the database as Zulu time UTC +0000. The server's default timezone is used when no timezone information is given, such as Dates and any DateTime without time zone info.
Given all this here are my notes about the Date with no time zone problem and what I do:
To deal with the problem of no time zones on Dates, e.g 2018-10-05 we treat the search as a 36 hour day rather than a 24 hours day when the precision is one on Year, Month or Day. For more fine-grained precisions such as Hour, Min, Sec we expect to have the time zones information supplied if time zone differences are a concern of the searcher.
So to do this I subtract 6 hours from the beginning of the date range 2018-10-05T00:00 and add 6 hours to the end of the day 2018-10-05T23:59. This gives us a 36 hour day range. The idea is that it is better to return more than less for the search.
This is a compromise as we really do not know what is meant by a date with no time zone. We can assume the servers default time zone as a starting point but this is only a guess to what the true time zone was for either the supplied search date or the stored FHIR resource dates when dealing with only date and no time.
So the range we actually use for this example is not:
2018-10-05T00:00 to 2018-10-05T23:59
but rather:
2018-10-04T18:00 to 2018-10-06T05:59
Which in a 12hr clock is 04/10/2018 6:00PM to 06/10/2018 6:00AM when the search date was: 05/10/2018
Also bear in mind that all date times are converted to UTC Zulu +00:00 time when stored and searched in the database.
Brian Postlethwaite (Oct 25 2018 at 21:50):
My server also does the high low thing that you've described. But it happy with the other parts I've done.
I'm considering doing 2 indexes, one for partials (no times), and one for precise (with tines/zones)
Brian Postlethwaite (Oct 25 2018 at 21:51):
Then these compromises aren't needed, but does cost storage space
Brian Postlethwaite (Oct 25 2018 at 21:52):
A 3 integer index.
Angus Millar (Oct 26 2018 at 07:38):
I still don't see it.
If I receive a resource with a Date only 2018-10-05 I will store that in three indexes as follows:
Date: 2018-10-05
LowDateTime: 2018-10-05T00:00 Z
HighDateTime: 2018-10-05T23:59 Z
I've already had to make an assumption on time zone for the Low and High DateTimes .
Now a search comes in with a timezone: 2018-10-05T21:00 -04:00 which converted to Zulu will be: 2018-10-06T01:00 Z
That will not match on the indexes we have , but the assumption I think we wanted is that it would.
What about the other way around?
If I receive a resource with a DateTime and time zone 2018-10-05T21:00 -04:00 I will store that in three indexes as follows:
Date: 2018-10-05
LowDateTime: 2018-10-06T01:00:00 Z
HighDateTime: 2018-10-06T01:00:59 Z
Now a search comes in with a only a date: 2018-10-05
Its a match, but then so is :2018-10-06
I don't know if I have you idea wrong but I'm still stumped on how to solve it.
Michel Rutten (Oct 26 2018 at 08:41):
That will not match on the indexes we have , but the assumption I think we wanted is that it would.
Why? A date without a timezone is ambiguous. My Friday in The Netherlands partially overlaps with your Friday. Without a specific timezone, a date represents a broad and imprecise range.
I like your interval arithmetic approach, as it provides clearly defined behavior for imprecise ranges. You could also transform the incoming search date to an interval (possibly wider in case of missing timezone), then search db intervals for non-zero intersections.
Brian Postlethwaite (Oct 26 2018 at 22:21):
Sorry, the 3 indexes I meant were year, month, day depending on the precision required. Plus the high low for precise data.
Christiaan Knaap (Oct 29 2018 at 08:17):
FYI: Vonk also translates every date(time) to a range based on the precision and translated to an offset of 0. Both the date(time)s in resources and in search arguments, and then match on overlap. Pretty much the same as you do. @Angus Millar idea of a 36 hour day is a nice workaround.
@Brian Postlethwaite solution for y/m/d/high/low is most precise. Reality is that no Vonk user has reported an issue with our current solution yet.
Brian Postlethwaite (Oct 29 2018 at 08:35):
Thanks for the feedback Christiaan, great to know.
Last updated: Apr 12 2022 at 19:14 UTC
