FHIR Chat · Date Type Choices · methodology

Stream: methodology

Topic: Date Type Choices

Grahame Grieve (Dec 12 2019 at 10:58):

ok, we said that we were going to discuss creating a new type to deal with the choice elements that can be something like dateTime | Period | duration | string. This is the topic thread for that

Grahame Grieve (Dec 12 2019 at 10:59):

the basic notion we had is that there is semantics involved in this choice, and it's not that infrequent.

Lloyd McKenzie (Dec 12 2019 at 12:17):

We're excluding Age and Range from the discussion?

Grahame Grieve (Dec 12 2019 at 12:21):

no.

Lloyd McKenzie (Dec 12 2019 at 12:36):

Ok. This is a situation where (unlike CodeableReference), we definitely don't want there to be more than one option here. Instead, we are grouping to make things easier for transformation and rendering, correct? There are two very different situations here:

• communicating an instant with uncertainty
• communicating a duration with uncertainty.

dateTime | Period | Age | Range | string is for communicating things that happen at (roughly) a single point in time. Commonly used for stuff like 'onset'. dateTime allows for precision or uncertainty (though certainty down to the level of a particular 'time' is uncommon here. Period allows for greater (and more precise) uncertainty. Age + Range allow the same, but expressed with respect to the patient's date of birth. String allows for any of the above to be expressed as human-readable, but also allows non-computable things like e.g. "Summer of '69", "When she was a toddler"

date | Period | duration | Range | string is for the communicating things with duration - i.e. with a distinct start and end. With date, you either specify low precision (year, year+month, date) and know that start and stop finished within that boundary but don't specify exactly when and gives you a maximum on the duration (1 day, 1 month or 1 year, respectively). Period allows you to specify specific stop and end and allow calculation of specific duration. Duration gives you specifics about how long it was, but no information about start and end. Range gives you an imprecise duration (e.g. 60-90 minutes) with no information about start and stop. String allows any of the former to be expressed as pure text, but more typically is used for concepts that don't fit neatly into any of the boxes - as for the prior paragraph. dateTime doesn't really make sense here because the time aspect requires full precision down to the second and means the event was pretty much an instant - in which case you ought to be looking at the first combo.

Lloyd McKenzie (Dec 12 2019 at 12:39):

One of the tricky bits here is that the interpretation of date, Period and Range are different depending on which of the combos you're using

Lloyd McKenzie (Dec 12 2019 at 12:42):

In the first, if I have a period of Feb-1990 - July-1991, the answer to the question "was this happening in March 1990" is "maybe" for the first combo is "maybe" - because it's saying "the event happened sometime in this range". The same range with the second combo would have an answer of "definitely" because it's communicating an event with duration where the only uncertainty is the precision of the start and stop. You definitely know the event was happening throughout the entire middle.

Lloyd McKenzie (Dec 12 2019 at 12:43):

We'd previously chosen not to get fussed about that, but if we're introducing distinct types, this is probably an appropriate time to choose.

Yunwei Wang (Dec 12 2019 at 16:14):

Constraints of CodeableReference is "At least one of reference, identifier and coding SHALL be present (unless an extension is provided)." In which use case you need more than one specified?

Lloyd McKenzie (Dec 12 2019 at 16:24):

When satisfying consumers where one wants a code and one wants a reference

Yunwei Wang (Dec 12 2019 at 16:35):

When providing both, does one need to be subsumed by the other? Like concept is Snomed-71388002 Procedure and reference is Procedure resource with coding Snomed-48537004 Bypass Graft? Or they could be pointing to different concepts, such as for the concept is a Snomed finding code and reference is a Procedure resource?

Lloyd McKenzie (Dec 12 2019 at 17:03):

My expectation is that they should be pointing to concepts that are roughly equivalent. That's certainly something the the type should make clear

Vassil Peytchev (Dec 12 2019 at 18:25):

Should the constraint read more like

At least one of reference.reference, reference.identifier or concept.coding SHALL be present (unless an extension is provided).

?

Lloyd McKenzie (Dec 12 2019 at 18:27):

I would expect concept.text or reference.display to be allowed with nothing else

Yunwei Wang (Dec 12 2019 at 18:41):

Oh. that brings another question. What is the "identifier" mentioned in the constraints?

Lloyd McKenzie (Dec 12 2019 at 19:01):

Was added to Reference in R4. Let's you reference by a business identifier. (No expectation for resolution during query - or at any other time)

Thomas Beale (Dec 13 2019 at 11:49):

date | Period | duration | Range | string is for the communicating things with duration - i.e. with a distinct start and end. With date, you either specify low precision (year, year+month, date) and know that start and stop finished within that boundary but don't specify exactly when and gives you a maximum on the duration (1 day, 1 month or 1 year, respectively).

That would not be the normal understanding of a low precision date. Indeed in ISO8601, a period is specified differntly from a point in time (even with low precision) - reqquires the '/' notation.

Thomas Beale (Dec 13 2019 at 11:55):

More generally, I think the main job FHIR needs to do is to just get the data that is in a source system to a requesting system / application - with a modicum of normalising different representations of the same semantic thing. I don't think anyone expects FHIR to answer questions such as 'was X happening in March 1990'. Inferring that is a question for the client and back-end systems, not for the communication protocol between them.

Thomas Beale (Dec 13 2019 at 11:56):

I agree with your separation of 'point in time' and 'duration' so I suggest that a clean-ish model of those two things will solve quite a few fields that are currently choice[x].

Thomas Beale (Dec 13 2019 at 11:57):

One question to consider is: how useful is it to expose so many representations to the client side?

Lloyd McKenzie (Dec 13 2019 at 19:21):

Low precision date that's expressing an event with a duration has uncertainty in two aspects - exactly when did things start and stop, as well as how long did the event last.

It's true that FHIR's purpose is the sharing. However, the purpose of combining these options into a single type is to simplify the processing of the data (in FHIRPath, CQL and other logic), in user interfaces, etc. Thus the impact on those uses is relevant - it's the only reason for combining the choices into a parent type.

The multiple representations reflect what existing systems collect. If systems are currently doing it, that's defacto evidence that it's useful.

Thomas Beale (Dec 16 2019 at 15:30):

@Lloyd McKenzie
I may not have been clear - what I meant was, let's say we agree that there are numerous data types trying to represent duration as you said: date | Period | duration | Range | string (we can argue about 'date' later); I agree of course that all of those things are likely to be found in source systems (and remembering that the 'string' form is likely to be 5-10 variant syntax formats). My question is: why not present the data as a Duration type to the client programmer? The motivation is make things easier for the O(10,000s) client devs, for the price of a bit of pain forthe O(100s) source system devs/vendors. I don't really see the benefit of exposing these 5 data types (and ?10 syntax possibilities?).

Lloyd McKenzie (Dec 16 2019 at 15:42):

The string variant isn't expected to be formatted at all - it's expected to be free text. The reason to not express it as a Duration is because the programmer needs to send what they have. How is a developer supposed to take "Summer of '69" from their database and map that to a duration.
We have to be able to convey the data systems actually have. And if the use-case is to allow capturing free text, the specification needs to support that. We don't get to change the legacy data and, by and large, don't get to change what systems capture on a go-forward basis. We just get to standardize how they share what they have.

Thomas Beale (Dec 16 2019 at 17:52):

Well, the most likely case in an EMR DB for a string field representation a duration is that it will be a string form of a date, maybe ISO 8601, maybe something else, but I agree, it could sometimes be something like 'summer of NNNN'. In any case there is a basic question about computability. In a field in a resource with this kind of type specification (i.e. date | Period | duration | Range | string) there is an attempt to say something like:

• the intention is a duration and...
• the data could actually be a duration
• the data could be a range of dates, aka a Period
• the data could be an approximate date whose accuracy provides an effective duration (I think this is a wrong specification, but it's what's there right now)
• the data could be a range of durations, recorded as quantities, aka Range
• the data might be an unprocessible string

So, apart from the last, the idea is that the datum is computable.

Thomas Beale (Dec 16 2019 at 18:02):

Now, this is a standard situation of course, the String v computable form. So there's a general situation in terms of specification which could be understood as a type of the form:

class DataType<T: VariantDataType>{
    unstructured: String [0..1];
    computable: T[0..1];
}

Rename those two any way you want. So one part of the work is to determine meaningful VariantDataTypes, ones that deal with the structured multiple possibilities in the above list, and the other similar lists of types.

Thomas Beale (Dec 16 2019 at 18:05):

I think if a generic approach like the above were used to model these 'variable data' situations, there would no longer be any need to have ad hoc type lists, nor to have String as one of the types. Taking that approach would make things easier for the modelling folks (they just choose some VariantDataType like VariantDuration for a 'duration' field, and if String is possible, then they choose DateType<VariantDuration>); it would also make life for the client programmer a lot better.

Grahame Grieve (Dec 16 2019 at 19:30):

sounds like moving the deckchairs around to me... what exactly has changed?

Thomas Beale (Dec 16 2019 at 19:41):

haha your favourite phrase. I'm suggesting a disciplined approach to this ongoing challenge of dirty data. Right now, every committee has to have a presumably lengthy discussion on what data might be in real systems that would be used to populate the field they are working on, which is logically intended as a duration, or a date or whatever it might be. It is clear from look at the choice[x] page that slightly different answers come up to these ad hoc discussions all the time, quite a few of which are not quite right. I'm suggesting doing this properly. Treat the dirty data problem as one to deal with models that are designed to deal with it explicitly.

Outcome a) defining committees no longer need to waste time trying to re-solve the same problem for the 10-th time, instead they just re-use the appropriate type. Outcome b) client devs get a type that can generate the logically intended computable type as a Duration, or whatever , or else, they can easily tell that there is only a String available, and that no computing will happen.

Most of my suggestions are about re-use. Right now, there is a lot of repeated problem-solving with numerous varying results (and loss of time), and on the dev side, there is not much clarity or reuse - they have to hand-write switch statements or some equivalent to deal with N different possible types. We can greatly reduce a lot of this pain.

Grahame Grieve (Dec 16 2019 at 19:50):

Outcome (a) sounds like a retrograde step, honestly. Instead of the committee analysing what the actual problem is, they just pick something near... that's a good thing? I don't follow how that's actually better. Easier, sure...

Outcome (b) I don't follow that either. Either there isn't any computable data, or there is. There's still a choice that they have in their code somewhere.... ('easily tell', you said). WHy is this method easier than the choice method?

Grahame Grieve (Dec 16 2019 at 19:51):

I'm definitely in favour of re-use in general, but it seems that you propose that reuse happens by providing a description of the problem that is 'near enough' so that's more generally applicable. Is that a fair characterisation?

Grahame Grieve (Dec 16 2019 at 19:51):

If so, when is that better?

Lloyd McKenzie (Dec 16 2019 at 20:00):

Side point: This isn't "dirty" data. It's a question of whether there's a need for free-text informality or not. Our expectation is that if what you have is a date, you'll send it as a properly formatted date. (It's not that hard to write code that can figure out what you've got and transform it appropriately.) What we don't want to do is force arbitrary precision when the captured statement is not precise. If the system captures "a few summers ago", then that's what should be shared.

Grahame Grieve (Dec 16 2019 at 20:02):

it's not clear to me when only having text should be treated as an exception (an extension), and when it should be part of the choice of data types.

Thomas Beale (Dec 16 2019 at 20:52):

Outcome (a) sounds like a retrograde step, honestly. Instead of the committee analysing what the actual problem is, they just pick something near... that's a good thing? I don't follow how that's actually better. Easier, sure...

Outcome (b) I don't follow that either. Either there isn't any computable data, or there is. There's still a choice that they have in their code somewhere.... ('easily tell', you said). WHy is this method easier than the choice method?

The committees are not doing good modelling. What is happening is ad hoc conversation, with numerous repeated attempts to solve the same / similar problems in isolation, rather than any proper analysis that can be re-used. For each committee, the choice of data types is what they happen to have discussed, based on who happened to be in the room, until the next time that discussion is updated. Having proper types to represent the properly analysed list of possible concrete types that might represent the intended logical type (duration or whatever) isn't a 'close enough' solution (what they have now is a 'close enough' solution); it's a precise response to the intended type required for the field, and it would enable this particular kind of ad hoc discussion to be dispensed with.

With a linear list of types that might or might not have String in it, and might or might not be the same as other such lists supposedly representing Duration or Date or whatever, the client application developer is stuck with ad hoc processing of all of that. They can't even re-use the code in other places, because no-one can say for sure whether the ad hoc list of types represents the same semantics, or should be the same, only in another location, the relevant committee didn't think of how (say) a partial ISO 8601 date could represent a duration. Every single data element is its own ad hoc use case. Well, I'm sorry, but that just has no hope of scaling.

Thomas Beale (Dec 16 2019 at 20:56):

The large amount of inconsistency across the resources shows pretty clearly what is going on. In my first analysis of inconsistency a few months ago, I stopped counting after about 50 items. And I had not at that stage even noticed the differences in choice [x] shown by your reverse table.

Thomas Beale (Dec 16 2019 at 20:58):

Side point: This isn't "dirty" data. It's a question of whether there's a need for free-text informality or not. Our expectation is that if what you have is a date, you'll send it as a properly formatted date. (It's not that hard to write code that can figure out what you've got and transform it appropriately.) What we don't want to do is force arbitrary precision when the captured statement is not precise. If the system captures "a few summers ago", then that's what should be shared.

I'm not advocating any fake precision, just typing. If you only have a String, then it should be easy to find that out. If you have anything else, it should be easy to work with, and it should include any appropriate accuracy or range or whatever. This is all just routine type design.

Jean Duteau (Dec 16 2019 at 21:06):

My argument may not be as strong as Thomas' but I believe that if we gave a core set of data type "choices", that committees would start with those and only change them if they had specific requirements. As it is, I've seen a lot of type choice addition as I map from FHIR version to FHIR version. That seems to me that the committee started with three choices and then added another and then another. If we could provide advice to committees to say "when you have dealing with effective dates, look at this data type choice pattern because this seems to repeat itself in nature", I think many committees would agree and we wouldn't have these discrepancies that Thomas and Grahame have listed previously

Lloyd McKenzie (Dec 16 2019 at 21:54):

I'm in favor of having "common patterns" that we invite work groups to consider and to diverge from only if they have justification. But I strongly support the appropriateness of them diverging when the set of requirements they have differs from the standard set. Certainly I'd expect that what gets captured for something like Observation.effective[x] and Condition.onset[x] would be different because the level of precision that systems care about is last different. Systems don't capture a blood pressure value from "a few of summers ago", but capturing an onset date with that value is much more reasonable/common. Having common patterns would also allow us to provide guidance around user interface and rendering without necessarily incurring a cost in our instances.

Grahame Grieve (Dec 16 2019 at 21:56):

Having proper types to represent the properly analysed list

it seems that you have some magic associated with 'properly analysed list' which doesn't start with requirements. I don't know how else to understand what you wrote

Grahame Grieve (Dec 16 2019 at 21:58):

large amount of inconsistency across the resources

What you haven't addressed - ever, anywhere, so far as I've seen - is whether the inconsistency is real and due to requirements, or not. You assume that it's not real, but most cases we've looked at, it's real

Grahame Grieve (Dec 16 2019 at 21:58):

I've seen a lot of type choice addition as I map from FHIR version to FHIR version

Is that heading towards convergence? I think it leads towards divergence, but I haven't done any formal analysis of that

Grahame Grieve (Dec 16 2019 at 21:59):

I'm in favor of having "common patterns"

We could certainly extend the tooling to allow this. Should it be more explicit than committee business? But I don't really see how we establish the list from where we are....

Jean Duteau (Dec 16 2019 at 22:00):

I'm not sure if it leads to convergence in every case. I think that sometimes it leads to convergence and sometimes to divergence. I just think that, given a pattern, a committee would have a good starting point as opposed to now having a blank slate where they try to determine what datatypes they would need.

Lloyd McKenzie (Dec 16 2019 at 22:03):

Methodologically, my preference is "start from a blank state" - so you're sure you're really driven by what's common. People will often take something that exists even if it's not right both as a path of least resistance and also because "it already exists, it must be right". If they create their own first, that give them a bit more of a solid foundation to consciously evaluate whether alignment with the pattern is appropriate and, if not, why not.

Lloyd McKenzie (Dec 16 2019 at 22:04):

As an example, someone proposing a new 'event' resource should never start by looking at the Event pattern. Looking at Event should happen only after their initial design has settled.

Jean Duteau (Dec 16 2019 at 22:07):

I don't agree. The problem is that you don't want either the blank slate or the full pattern, you really want implementers to tell you what types of data go in that slot. But when I'm creating a resource, I don't have the luxury of waiting until I've canvassed my implementers. I have to put some datatype there. In that case, I think starting with the pattern makes the most sense. Then I can drive out the differences with my implementers.

Grahame Grieve (Dec 16 2019 at 22:12):

where's our actual design documentation?

Grahame Grieve (Dec 16 2019 at 22:13):

I guess the current correct link is https://confluence.hl7.org/display/FHIR/Guide+to+Designing+Resources

Grahame Grieve (Dec 16 2019 at 22:13):

we could certainly improve this documentation

Lloyd McKenzie (Dec 16 2019 at 22:33):

It's much harder to get implementers to remove something once it's there. It's also harder to get them to identify a preferred/better name once they've seen a 'standard' one. Agree we could make the documentation better. The question has been how much to focus on that given that 'most' resources already exist.

Grahame Grieve (Dec 16 2019 at 22:34):

right typically we review. but some we didn't review enough.

Grahame Grieve (Dec 16 2019 at 22:34):

we'll still get 50+ more yet

Thomas Beale (Dec 16 2019 at 22:46):

Methodologically, my preference is "start from a blank state" - so you're sure you're really driven by what's common. People will often take something that exists even if it's not right both as a path of least resistance and also because "it already exists, it must be right". If they create their own first, that give them a bit more of a solid foundation to consciously evaluate whether alignment with the pattern is appropriate and, if not, why not.

Well you need to start from a blank slate when looking at requirements, but if you have no library of previous analyses, or models, everyone is just going to lose time trying to reproduce an analysis that has undoubtedly been done before, and refined, formalised and published. Not to mention that you want to internally standardise identical semantics when you can. Scalability of any large effort is only possible through the discipline of good analysis and re-use.

Grahame Grieve (Dec 16 2019 at 22:53):

well, sure. so here we are seeking semantic patterns. because it's when there's inter-related meaning implicit in a choice that it's actually a re-use problem

Lloyd McKenzie (Dec 17 2019 at 00:06):

When defining a new resource, we absolutely look at analyses that have been done before - generally what other standards exist in this space. But we ask designers to look at them with caution as many don't adhere to the design principle of "focus on what most systems do". It's true that there's a possible cost in time of not starting out with a pattern, but there can be a cost in bias of starting with an existing design. The current recommended sweet spot is to bring in the patterns after you've done your initial design and beaten it up a bit with implementation, but before you've really pushed the specification out there for widespread use. (We don't necessarily always achieve that sweet-spot, but that's the objective.)

Grahame Grieve (Dec 17 2019 at 00:07):

but we do expect the committees to use the defined datatypes... so there seems like an arbitrary difference there

Lloyd McKenzie (Dec 17 2019 at 00:17):

We expect (and see) a high degree of re-use in data types that makes consistency of high importance. Resource elements having different sets of choices or different element names when it better reflects reality/provides improved clarity to implementers doesn't (generally) have that same impact. We come from a place where we tried to drive everything from a consistent model - and that failed miserably. That doesn't mean that models don't have a place, but we need to be cautious about letting the elegance of the model supplant alignment with what systems do. The further we drift from the latter to make complex things easier, the harder we make the easy things, the more we create barriers to adoption and the less implementers can count on the standard to reflect the data they're likely to to actually get.

Thomas Beale (Dec 17 2019 at 13:01):

We come from a place where we tried to drive everything from a consistent model - and that failed miserably. That doesn't mean that models don't have a place, but we need to be cautious about letting the elegance of the model supplant alignment with what systems do.

Well the RIM wasn't comprehensive enough, and had its own formal problems, so I wouldn't regard its failure as saying much about good modelling in general versus that particular model (which had one very useful pattern, but needed another 3 or 4 - I wrote a paper on that). Let's not go there.

However, I agree that the game is not to try to replace real world messiness with fake elegance. The only useful 'elegance' of a 'model' is that which truly models what you really need to represent. If the thing we want to model is a logical duration field whose concrete data in source systems could be a string, or else 4 kinds of structural type, then let's model that exact situation properly. Similarly for the earlier discussion about presence/absence Boolean + a datum (a Date or whatever). These are recurring patterns of messiness in the real world; at the moment they are re-discovered in separate meeting rooms and disconnected, repeated and approximate re-analysis is taking place, and the results are ad hoc, and often incomplete and/or inconsistent, when viewed across the whole model base. There is little re-use at any level - of analysis, design or implementation.

Doing good modelling means really looking properly at the problem in question: understand what is really going on - an ontological POV if you like. It's very easy to be superficial (as we all are sometimes, usually when having no time to be otherwise) but building a standard is not the place to be superficial.

Lloyd McKenzie (Dec 17 2019 at 15:03):

What we need to represent are the data elements that systems generally have and are able to exchange. We want to be very cautious about the model supporting capabilities that systems don't have (and obviously we want to ensure the model does support capabilities most systems do have). So the problem isn't "how do we convey a duration", but rather "how do systems in this particular space convey a duration when they're talking about this particular property of this particular resource". Consistency of models should be a reflection of consistency in implementation. If implementation isn't consistent in two different elements, then we'd expect inconsistency in the models, even if the elements are conceptually dealing with the same notion.

If we can agree on that as foundational methodology, it'll be easier to get agreement on what point standard/pattern-based modelling should be introduced to the design process.

Richard Townley-O'Neill (Dec 17 2019 at 22:51):

Fascinating debate. :+1:

Thomas Beale (Jan 09 2020 at 15:01):

This page contains a full list of all type choices in DSTU4, reversed and grouped according to design intent (as far as I can determine it). I'm working on formal models that could replace most of these combinations. https://openehr.atlassian.net/wiki/spaces/stds/pages/441581569/HL7+FHIR+choice+x+analysis

Thomas Beale (Jan 09 2020 at 15:02):

Note that this page will shift to the HL7 wiki soon, just waiting on someone with the right kind of access and knowledge to do that, and it will then be editable by anyone.

Grahame Grieve (Jan 09 2020 at 21:32):

you should have access

Grahame Grieve (Jan 09 2020 at 21:34):

it looks like your analysis is based on an older copy of FHIR before I made the CodableReference changes? (though it only changes one of the tables...)

Grahame Grieve (Jan 10 2020 at 09:22):

@Thomas Beale

these probably should be 2 data points in all cases, because the general case is a Boolean + a data structure if the Boolean is True

Why would that be better? it would allow false + a structure, which is incoherent. Otherwise, would true be inferred by the presence of data? If yes, why is it different? if no, why not? Either way, how is not just moving the deck chairs around? Either way, the code I'm going to write is going to look pretty much the same, except that I'd need more defensive programming if there were 2 attributes instead of one

Grahame Grieve (Jan 10 2020 at 09:25):

in many cases, it is assumed that the value will be supplied in one or more structured forms, OR a string (marked in grey);
this should be dealt with in a generic way across all the data types;

Really? why would this better? To have this possibility everywhere, instead of the 2% of elements where it's a thing? I really don't follow your logic there. But perhaps there's something that I haven't thought of that makes a difference to this analysis

Grahame Grieve (Jan 10 2020 at 09:27):

Age vs Duration... there's not much difference in the value domain, but there is a semantic difference in intent. Not a big point, but I think it's too late to collapse them

Grahame Grieve (Jan 10 2020 at 09:30):

yes we could have handled instant vs dateTime using a metadata constraint. I did consider that in early drafts, and we discussed it at length. It was believed that it was simpler if there's a type that means a specific instance of time

Grahame Grieve (Jan 10 2020 at 09:31):

instant should not be used on Observation.effectiveTime. I lost that argument - kind of the past coming back to bite me. We defined instant because it was simpler for peope to get their head around, and then they insisted that it had to be available as a constraint mechanism for Observation.effecticeTime. I think that's wrong, but wasn't able to get the committee to bend on that

Grahame Grieve (Jan 10 2020 at 09:34):

I do not know whether there's rhyme or reason for when TIming is on the time list or not, but given it's fiendish complexity, I'm very happy for us to be as inconsistent as possible in order to minimise it's presence. I, for instance, do not think that it's presence on Observation.effectiveTime is at all useful; I cannot imagine a useful observation that was made at a set of discontinuous times. Yuck to that idea. But again, the committee didn't agree with me

Grahame Grieve (Jan 10 2020 at 09:35):

Why is the boolean presence flag is use for Patient.deceased[x] but not for Immunization.occurrence[x] and potentially other occurrence[x] elements?

Because they are all known to have happened or not by the other contents of the resource. The deceased state is not otherwise known

Grahame Grieve (Jan 10 2020 at 09:35):

so it shouldn't be in the same list

Grahame Grieve (Jan 10 2020 at 09:37):

I certainly would like to see the Historical Event time consistent across the resources

Grahame Grieve (Jan 10 2020 at 09:37):

Why does the 3rd choice list include Reference() and not the other two?

that's an error. It will be removed in future versions

Grahame Grieve (Jan 10 2020 at 09:39):

The money things... grrr.

Thomas Beale (Jan 10 2020 at 10:01):

Boolean+data item first: the cases I can find in FHIR are:

• deceased data | Boolean (= is deceased or not)
• nr multiple births | Boolean (= multiple births or not)
• MedicationRequest.reported: Reference | Boolean
  It's true that these are all atomic data items rather than structures. Nevertheless, what you want in terms of interface is to always be able to ask the question in the Boolean sense, and optionally to obtain the detailed data item. That means 2 data items, with a Boolean-valued accessor function. Then all software can process that data item with the same code.

Grahame Grieve (Jan 10 2020 at 10:05):

Then all software can process that data item with the same code.

no, I'm not seeing how inlining it makes that difference. What I see is that you can't get your head around inlining

Thomas Beale (Jan 10 2020 at 10:08):

For stringified data: to get to the bottom of how to deal with this properly, we need to be able to answer a simple question: how do we know that any given data item can never be a String? How can you know what is in all systems? Just because the people in some committee meeting didn't happen to include someone whose brand of EMR has string in field xyz doesn't make it go away. As far as I can see, any legacy system data item could be a string in the FHIR world. Personally I would aim to be parsing a lot of those strings, likely to be dates, and even complex things like "110/80", and converting them at source, but that's a different question.

Generically, it seems to me that a String / non-computable form of the data is possible in any data item.

Thomas Beale (Jan 10 2020 at 10:08):

What do you mean by 'inlining' - do you mean the type union?

Grahame Grieve (Jan 10 2020 at 10:10):

sure. We can take a a choice of boolean / dateTIme and turn into a class with boolean and dateTime fields. So what?

Thomas Beale (Jan 10 2020 at 10:13):

It means you can always assume the presence of a Boolean accessor, 100% of the time. So every time someone needs to write a piece of code of the form if (not patientDeceased()) then yyz; it will work, regardless of whether the original data item was a Boolean or a dateTime or whatever else. Right now, that's not the case.

Grahame Grieve (Jan 10 2020 at 10:14):

well, if you're going to write accessors, then what difference does it make? you put your if/else logic here or there....

Thomas Beale (Jan 10 2020 at 10:15):

Then for code that wants an actual date, you also have the same: if (patientDeceased.hasData()) then // stmt using patientDeceased.item;

Thomas Beale (Jan 10 2020 at 10:17):

a) the accessor code and logic is done once (in each language), published and re-used
b) the choice on that data item goes away, simplifying the modelling effort
Both of these will reduce errors. Right now, 5,000 developers will all hand-write their own code. At least 20% of them will get it wrong.

Grahame Grieve (Jan 10 2020 at 10:19):

it seems to me that you're anticipating something I can't imagine. You didn't make the choice go away... you just moved it around. Everyone still needs all the same logic, somewhere, and everyone has to write it somewhere, unless they get it from a reference implementation

Thomas Beale (Jan 10 2020 at 10:21):

It does go away. Choice is always a typing cheat - it's non-typing. Boolean+other is just the simplest form of it. I'm saying: do proper typing and on the way, standardise the logic that deals with the specified instance possibilities.

Thomas Beale (Jan 10 2020 at 10:22):

When you look at the timing type choices, it is even clearer. Everyone trying to figure out their own logic will be a giant mess.

Thomas Beale (Jan 10 2020 at 10:23):

And that's assuming the specs are right, which they are guaranteed not to be, due to the arbitrary way the choices are determined (by who happens to be in the room, and if they happen to have seen or heard of some product that might have some specific representation of some data item xyz that might be the equivalent of the named data item abc in the Resource 123)

Grahame Grieve (Jan 10 2020 at 10:25):

maybe someone else who is watching this can follow and understand how you think that this makes a difference, because I sure don't

Thomas Beale (Jan 10 2020 at 10:26):

Well I can only explain it so many ways. Choice[x] is not typing, it is avoiding typing. Without typing you are in an ad hoc place both at the modelling phase (getting the choices right) and in the implem phase (coding for them). These are just standard consequences of this approach. It's why no-one uses it.

Grahame Grieve (Jan 10 2020 at 10:35):

so you have a data type that can be a boolean or a dateTime. Or you can have a element that is a class that can contain a boolean or a dateTime... I'm not seeing the big difference...?

Grahame Grieve (Jan 10 2020 at 12:31):

I suppose that central to your argument is that fact that casting is required, or not

Thomas Beale (Jan 10 2020 at 12:39):

Casting is usually wrong. Occasionally, you have a genuine check on sub-types of the static type, but generally speaking, the whole point of polymorphism in modern OO languages is to avoid that.

Grahame Grieve (Jan 10 2020 at 12:43):

but then they invented the visitor pattern....

Thomas Beale (Jan 10 2020 at 13:19):

The visitor pattern is for traversable structures, mainly trees.

Thomas Beale (Jan 10 2020 at 13:20):

and is also for separating functionality that doesn't belong inside the central classes in question, i.e. add-on functionality, e.g. new serialiser, tree displayer etc

Thomas Beale (Jan 10 2020 at 13:51):

Also, visitor doesn't use casting at all. It relies on polymorhpic calls, just like everything else.

Lloyd McKenzie (Jan 10 2020 at 16:02):

Let's talk about it differently. Rather than having deceased with two types, lets say you have two different properties - a deceasedBoolean and a deceasedDate - but there's an invariant that says you're only allowed to share one or the other to eliminate the possibility of conflicting data. (In internal code, you'd always be able to infer the boolean, but in terms of what gets shared over the wire, you only want there to be one or the other.

Lloyd McKenzie (Jan 10 2020 at 16:02):

Would you see that design as problematic?

Lloyd McKenzie (Jan 10 2020 at 16:03):

Because that's essentially what our 'choice' structure does. If you'd prefer to think about it as two elements with an invariant rather than one element with a choice type, you're welcome to.

Thomas Beale (Jan 10 2020 at 17:19):

well you are still relying on the programmer to figure out that 'deceasedDate' and 'deceasedBoolean' are really variant forms of a notional 'deceased' data item. Putting the name of a type in a variable name is what people used to do in SmallTalk, and still do in JS (there's a reason TypeScript exists...) - it's a sign that real typing is absent.

Thomas Beale (Jan 10 2020 at 17:20):

And your invariant would have to be at the Resource level, along with other invariants of the same form. It would be a lot cleaner to just have a proper type for a single data element 'deceased'.

Thomas Beale (Jan 10 2020 at 17:21):

It could even be a generic type like 'SmartValue<Date>' where SmartValue<T> adds the boolean member, boolean accessor and any invariants you want.

Thomas Beale (Jan 10 2020 at 17:22):

All of the problems I am pointing out here are multiplied quite significantly when you look at data elements with choice[x] of various dateTime, Period, Interval etc etc, and most other choice[x] uses.

Grahame Grieve (Jan 10 2020 at 18:03):

so this is something that you could choose to generate now - instead of generating the element as a polymorphic one, generate it as a class, with elements.

Grahame Grieve (Jan 10 2020 at 18:04):

I maintain the java generated classes, and no one has ever raised this issue. Nor have I seen it raised for DotNet

Grahame Grieve (Jan 10 2020 at 18:09):

but it does make me think that the problem is of your own making - it's how you choose to understand the definitions

Bryn Rhodes (Jan 10 2020 at 18:34):

For what it's worth, choice types are a fairly common feature of modern languages. For example, TypeScript supports Union Types.

Grahame Grieve (Jan 10 2020 at 18:43):

yes I think that's pretty much what is making Tom barf

Bryn Rhodes (Jan 10 2020 at 18:56):

I find them quite elegant :)

Bryn Rhodes (Jan 10 2020 at 18:57):

Strengthens reasoning about types in the type system.

Grahame Grieve (Jan 10 2020 at 18:58):

well, I should let Tom speak, but I think he would say that it won't need strengthening if you did it properly in the first place

Bryn Rhodes (Jan 10 2020 at 18:58):

And I love the analysis of choice types that Tom did, but the fact that he could do it is evidence of that "strengthening" :)

Bryn Rhodes (Jan 10 2020 at 18:59):

And we use them in the CQL translator to provide some pretty sophisticated type inferencing.

Grahame Grieve (Jan 10 2020 at 19:03):

I love the analysis of choice types

did it lead to any changes for you?

Bryn Rhodes (Jan 10 2020 at 19:07):

Yes, triggerDefinition.timing, we're considering making that consistent, though that's up for alignment with Subscription as well, so we're not totally sure what that looks like yet.

Bryn Rhodes (Jan 10 2020 at 19:07):

Other than that, it was confirmation that we're being consistent about the choices we're using across resources.

Grahame Grieve (Jan 10 2020 at 19:07):

I already created a task to remove Schdeule from there as well

Bryn Rhodes (Jan 10 2020 at 19:09):

It also raised the question for us about the choice of Range in some of the elements (like Procedure.performed. What does it mean for Performed to be a Range?

Lloyd McKenzie (Jan 10 2020 at 19:09):

It's conveying duration with uncertainty

Bryn Rhodes (Jan 10 2020 at 19:10):

Yep, just found by looking at the comments in the latest build. An Age range. Thank you.

Grahame Grieve (Jan 10 2020 at 19:10):

from the comments:

Range is generally used when the patient reports an age range when the procedure was performed, such as sometime between 20-25 years old

Grahame Grieve (Jan 10 2020 at 19:10):

but I kind of feel that performed is performing a wide range of semantics there

Bryn Rhodes (Jan 10 2020 at 19:11):

It is, and it definitely causes complexity when writing logic against it.

Bryn Rhodes (Jan 10 2020 at 19:11):

We've taken to building helper functions that "normalize" those different representations to an interval.

Thomas Beale (Jan 11 2020 at 10:48):

@Bryn Rhodes I'm sure you know that although unions or anything similar are in a few languages, they are not recommended for use, since they make things brittle. If they are used carefully and judiciously, the problems can be contained. However, making them the routine way to do modelling is a bad idea - it makes people think they can put anything in there without thinking too much about the real semantics. All those variations on time/timing are evidence of that. Each one requires custom programming, not to mention possible new permutations due to profiling. (Note BTW in that TS example that the formal static type is 'Any'. That's just bad design for that routine).

Thomas Beale (Jan 11 2020 at 10:50):

Anyway, I'm glad that the analysis has some potential value.
Here's a random question: why does Observation.value[x] not include Duration or Date?

Thomas Beale (Jan 11 2020 at 10:53):

It's conveying duration with uncertainty

How can the Range meaning uncertainty be distinguished from a Range meaning the real times during which the procedure was performed (e.g. a 3h period)? Wouldn't it be better to include the notion of accuracy in the quantitative data types (including date/times)?

Thomas Beale (Jan 11 2020 at 10:55):

BTW I am a fan of CQL, but I have to say that most programmers out there are not even close to the level of skill of its authors. You have to consider the downsream fan-out of consequences of any modelling choices, not whether it might work nicely for one competent dev group.

Grahame Grieve (Jan 11 2020 at 11:35):

How can the Range meaning uncertainty be distinguished from a Range meaning the real times during which the procedure was performed (e.g. a 3h period)?

right. that's a very pertinent question which I have too. It's not right to hide something like that in an off-hand comment. I'll talk to the committee

Grahame Grieve (Jan 11 2020 at 11:38):

@Thomas Beale you didn't answer my question about whether you would think differently if we generated choices as containing classes instead of inline poly-morphic elements?

Thomas Beale (Jan 11 2020 at 12:43):

You mean some sort of synthesised mixin types? I probably need more detail on your idea.

Thomas Beale (Jan 11 2020 at 12:51):

Général comment: although Grahame occasionally says things like would xyz 'make me happy', it should be understood that all my analysis is from the POV of the downstream dev community, also the downstream data use community (and to be transparent, one major org is the VA, for which I am a consultant); I personally will probably never write FHIR consuming code (I tend to work at EHR, workflow, CDS level). So I am coming at all this from a statistical POV of possible errors / inefficiencies in large communities, also errors /inconsistencies made upstream in committee modelling efforts. Just so we are all clear :)

Grahame Grieve (Jan 11 2020 at 21:03):

observation.change.svg

Grahame Grieve (Jan 11 2020 at 21:03):

something like that

Thomas Beale (Jan 12 2020 at 10:11):

Well now you have a class containing a combination of data types specific to Observation. It would be be better if you could just put the type as DataValue or DataType or whatever, and know by inheritance that any of the data types could be bound at runtime. Your proposal is literally moving deckchairs (one deck out), and in a way, obfuscating further the design intention of Observation.value, which is surely 'any data type'. Although SampledData, which is a structure of potentially any other type, shouldn't be shoe-horned in to the position of atomic data item. But then we get on to the question of modelling structured observational data properly in time, which FHIR Observation doesn't do anyway, e.g. 50 BP samples over 24h.

Grahame Grieve (Jan 12 2020 at 10:51):

the design intention of Observation.value, which is surely 'any data type'.

Sigh. no. It's not. we keep having this discussion, where you eel free to make up requirements as you see fit rather than deal with the real ones we encounter

Grahame Grieve (Jan 12 2020 at 10:55):

I agree that I am moving the deck chairs around. but it's way that gives full typing, instead of using inheritance, which you have a strong aversion to if it's constrained.

Thomas Beale (Jan 12 2020 at 13:07):

the design intention of Observation.value, which is surely 'any data type'.

Sigh. no. It's not. we keep having this discussion, where you eel free to make up requirements as you see fit rather than deal with the real ones we encounter

I'm not exactly making up requirements, after 25y of working on health data and EHRs. It's not a supposition. We have 15y of empirical knowledge about the data that turn up in the representation of 'observations' in real systems. When I look at the FHIR data types, there are only a couple that might not make sense as the type of an Observation, e.g. Money (and ignoring complex structures like Address that are not really clinical data types).

What I'm really pointing out (again) is that the committee-based method that concentrates solely on the 'real [requirements] we encounter' just misses basic things which we already know more generally, e.g. Duration and Date, which are missing from Observation.value[x].

Thomas Beale (Jan 12 2020 at 13:11):

As I've pointed out before, if we get a different group of people to analyse the requirements of just about any choice data Element in FHIR, the result will be different from what is currently published. That's because the current method of arriving at requirements ignores any theoretical considerations, past lessons etc, and focuses solely on what current committee groups come up with in ad hoc (non-methodological) discussions. That approach is guaranteed to lead to incomplete results. The fact, coupled with choice being a very brittle construct is similarly guaranteed to lead to problems over time.

Lloyd McKenzie (Jan 12 2020 at 15:23):

As a very simple example, we don't want to allow both string and CodeableConcept - because they both let you convey a bare string type and we don't want there to be more than one way of doing that. We also don't want to let you send just an integer - because in the context of an Observation, that always makes sense to be expressed as a Quantity. (And having both would also give two ways of sending the same sort of thing.) We really really don't want all possible data types.

Thomas Beale (Jan 12 2020 at 15:31):

@Bryn Rhodes

For what it's worth, choice types are a fairly common feature of modern languages. For example, TypeScript supports Union Types.

I missed on first reading the typing 'string | number'. This is I suppose better than Any in one sense, but it's really just a shorthand for overloading, allowing what would be 2 routine declarations to be made in one, as long as the number and meaning of arguments is the same in all variants. Overloading is indeed a kind of choice-typing, but its proper use is almost always limited to format variation of semantically identical data items, e.g. the data item has to be a number, but could arrive as an Integer or as a String containing an integer.

Personally I've never seen overloading as a good idea, and in languages that don't have it, I have always used differently named methods. Sometimes its convenient. About a decade ago, Eiffel introduced a truly nice way of dealing with the most common use of overloads (which is constructors handling different physical formats of arguments), which is the 'convert' section of a class definition - https://www.eiffel.org/doc/eiffel/ET-_Other_Mechanisms#Convertibility

Thomas Beale (Jan 13 2020 at 10:37):

As a very simple example, we don't want to allow both string and CodeableConcept - because they both let you convey a bare string type and we don't want there to be more than one way of doing that. We also don't want to let you send just an integer - because in the context of an Observation, that always makes sense to be expressed as a Quantity. (And having both would also give two ways of sending the same sort of thing.) We really really don't want all possible data types.

Well maybe you do, maybe you don't, it depends on what you are trying to do. One problem is that in FHIR is there is no supertype of the types you would normally consider 'clinical data types', so there's no way to specify those types while excluding raw primitive types like String, Integer etc. Back to the argument for a DataType or DataValue type (as per this post - https://wolandscat.net/2019/09/12/fhir-fixes-the-observation-value-problem/) .

In openEHR, we do maintain such a separation, and you are right, we don't want raw Strings, Integers etc. However, I happen to know that typical Obs data is recorded in the Epic system in String form, e.g. heart rates and BPs. Are you saying that such data should be turned into proper Quantities etc (I would agree)? But I also hear that you want to maintain the original data visible to a FHIR requestor app, which implies you want to ship it as Strings in that case. What is the guideline for FHIR for this kind of thing?

Separately from that, the question of which data types might be applicable to some specific Observation data depends on what the observation is. It can't be known in the Observation resource; it has to be specified in specific Obs profiles.

So if a clear parent type of what you consider to be all possible data types that may appear in Observation can be established, Observation.value can just be that type. It doesn't matter if it includes a couple that will (probably) never be used, e.g. Money. Because all real observation data will be profiled to just the types you want, just as we do with openEHR and CIMI archetypes.

Lloyd McKenzie (Jan 13 2020 at 16:17):

Actuallly, it looks like I lied. Observation does allow both CodeableConcept and string (not clear on why). It also supports Quantity and integer. It needs time and dateTime. It doesn't allow decimal or timestamp. The key point is that there are specific types we need and there are others we don't want. And it's not a neat division of "simple type" vs. "complex type". (And in FHIR, even a boolean is a complex type given that it can have an id or extensions.)

Thomas Beale (Jan 13 2020 at 16:25):

Timestamp seems to me to be a data type (like some others, e.g. Age, Distance) that are named not based on what they are, but on what they are intended to be used for. But Timestamp is really just a date/time that is not partial (in the ISO 8601 sense) and Age is just a Duration that is presumably meant to be used for 'age' attributes within resources. Given that, it's not clear to me why Timestamp is not allowed. And if Integer is allowed, why not Decimal? Surely you can only say which types are really allowed when you know what Observation data you are talking about (heart rate, ECG lead data etc)?

Thomas Beale (Jan 13 2020 at 16:26):

Quite a lot of these restrictions are constraints that in reality would apply in specific situations, in my view they don't belong in the based model.

Lloyd McKenzie (Jan 13 2020 at 16:55):

Timestamp is not allowed because catching time where you MUST be precise down to the millisecond doesn't really make sense as the value of an Observation. The constraints in the base model reflect what we expect most systems to support. (Not everything that might be theoretically possible in some tiny fraction of systems to support a narrow edge case - that's what extensions are for.)

Thomas Beale (Jan 13 2020 at 19:00):

Back to the more general point: it's pretty clear that the type choice lists are brittle in the sense that re-analysis often shows up some anomaly, and also there are most likely some missing types, simply because not all use cases are known. This gets back to the primary objection to modelling this way - how can it ever be right. For example, let's say someone comes to a WG meeting and convinces everyone that in fact Duration and/or Date should be Observation.value [x] types. Let's say this happens after the spec is made normative. What happens then? Because this will happen...

Lloyd McKenzie (Jan 13 2020 at 19:24):

Adding additional types to an existing choice structure is a permitted change even for normative specifications so long as the minimum cardinality on the element is 0. (Which is true for most, if not all choice types)

Thomas Beale (Jan 13 2020 at 19:36):

And what about removing? Marked as obsolete I guess...

Lloyd McKenzie (Jan 13 2020 at 19:49):

Deprecated first, but eventually obsolete, yes. We haven't talked about doing that at a 'type' level, but it would align with other similar change processes

Thomas Beale (Jan 13 2020 at 20:20):

These changes are going to have knock-on effects in software, code generators, ...

Grahame Grieve (Jan 13 2020 at 21:17):

the changes we've already made out of this discussion have generated 4 weeks of work for me

Grahame Grieve (Jan 14 2020 at 03:33):

I no longer have any clear vision of where we are on this. It's increasingly sounding like going around in circles.

• we're fixing up a few miscellaineous elements where things aren't being done well
• we're supposed to have a reasonabel talk about a times/durations data type
• Observation.value type is now DataType and the only remaining issue is how constrainedthe type set should be. But you wouldn't think so reading this
• we have a much more consistently defined set of abtract types (no secret knowledge now)
• we could talk a little bit more about the few special cases for string representations of human remembered data like in FailyMemberHistory

Diego Bosca (Jan 14 2020 at 12:30):

Deprecated first, but eventually obsolete, yes. We haven't talked about doing that at a 'type' level, but it would align with other similar change processes

Toms proposal would help in cases such as the Observation.valueAttachment attribute, which seems to come in and out each new version

Lloyd McKenzie (Jan 14 2020 at 15:44):

"Observation.value type is now DataType" - I don't recall agreeing to that and don't think we can make any change there - certainly nothing that will impact what the instances look like. The instances will continue to behave as a choice type. And I'd be opposed to not listing the choices directly in the published resource model. Obfuscation doesn't help anyone.

Thomas Beale (Jan 14 2020 at 19:07):

I no longer have any clear vision of where we are on this. It's increasingly sounding like going around in circles.

Fair enough. Would it be useful if I do some work on a better model of time? We have to solve this better in openEHR Clinical Workflow anyway, but I'm happy to take into account the requirements implied by the current FHIR choices for time.

For the rest of your bullet list, should we consider DSTU4 out of date now, and just stick to the current build?

Grahame Grieve (Jan 14 2020 at 19:07):

"Observation.value type is now DataType"

That was just changing the abstract type.

Grahame Grieve (Jan 14 2020 at 19:08):

"Observation.value type is now DataType"

That was just changing the abstract type.

Grahame Grieve (Jan 14 2020 at 19:08):

"Observation.value type is now DataType"

That was just changing the abstract type.

Grahame Grieve (Jan 14 2020 at 19:09):

current build, yes

Grahame Grieve (Jan 14 2020 at 19:09):

better model of time is certainly something productive to work on

Grahame Grieve (Jan 14 2020 at 23:37):

ok we reviewed @Thomas Beale's document around times this week. We didn't see a lot of commonality and simple patterns, but we did come up with a number of action items:

• create a task to standardise FamilyMemberHistory.condition.onset[x], FamilyMemberHistory.deceased[x] with Condition.onset[x] (though keep boolean as an additional choice for deceased)
• Task to remove string from FamilyMemberHistory.age[x] or explain how it's going to be used
• Define AgeRange, and create task for using it in these elements
• Methodology change : define pattern for re-usable types to make re-use easier, and prototype it's use [somewhere]. Test out how this might impact on implementers
• New Methodology rule: if string is allowed as a type choice, documentation must explicitly call out why it's present in the choice list
• Create a Task for better documentation on Goal.target.due[x]
• Propose an IG for a data type constraints library
• Create a task for much better documentation for ActivityDefinition.timing[x], and documentation for the relationship between ActivityDefinition.timing[x] and CarePlan.activity.detail.scheduled[x] including a conversion table (and providing guidance for string)
• Make a task for better documentation TriggerDefinition.timing[x]
• Make a task asking why date & codeableConcept are mutually exclusive on Goal.start

Grahame Grieve (Jan 14 2020 at 23:38):

we ran out of time to finish up with the timing lists, so that's not a finished list

Grahame Grieve (Feb 06 2020 at 03:55):

• ask why supply delivery has a Timing Datatyype rather than making separate supply delivery resources

Grahame Grieve (Feb 06 2020 at 03:59):

• make a rule in the tooling that if you use Timing in an element, you need to describe why in the documentation

Thomas Beale (Mar 23 2020 at 22:16):

I have done some UML modelling to show how to replace the set of types in the first table with a designed (complex) type that covers the semantic needs. It probably doesn't cover them 100% yet, I need to do a bit more work. But this shows in principle how to get rid of numerous combinations with a single, intentional type that people would always whenever they need to represent a history of occurrences of anything in the past. https://openehr.atlassian.net/wiki/spaces/stds/pages/441581569/HL7+FHIR+choice+x+analysis#UML (and then read backward).

Thomas Beale (Mar 23 2020 at 22:18):

NB: I don't have the UML for the FHIR data types, so I've just used some local ones I have like Coded_term; assume it really means code; same for the Iso8601_xx types; assume the normal FHIR atomic types. I'll fix that later.

Thomas Beale (Mar 23 2020 at 22:19):

I think this principle can be applied to most groups of 'choice' on that page, which would change the modelling mentality from a guessing game to an intentional analysis, which I think is likely to lead to quicker decisions and resolutions . I'll add models for the other choice groups as I get to them.

Last updated: Apr 12 2022 at 19:14 UTC