Stream: implementers
Topic: Polymorphic elements
nicola (RIO/SS) (Dec 08 2016 at 15:25):
And again about polymorphic elements wire format:
- impossible to have collection of different types (without realistic reason, just because format)
- problematic validation (it's not so simple to express exclusiveness & required elements)
// what we have
{ valueNumber: 42, valueString: "some"}
// how it could be
{value: {type: "number", value: 42}}
// this isomorphic to {resourceType: "...."} solution, or we could use something like @type for both :)
// if it's collection
{value: [{type: "number", value: 5}, {type: "string", value: "some"}]}
Abbie Watson (Dec 08 2016 at 17:38):
Not 100% sure what the question is here, but it really just depends on your database. MongoDB can certainly handle collections of different resource types. You could even have an index on the resourceType field, for fast collapse/lookup of resources. It wouldn't be the most efficient lookup ever, but would be perfectly fine for inbound queues and caches.
Schema validation is another area where it just depends on your implementation. We're using a schema constructed called SimpleSchema, which supports nested schemas, schema extensions, and partial schemas. So we constructed a BaseSchema and MetaSchema, which all of our other resource schemas inherit from. So, if we want to validate on a set of required elements, plus a base schema, we can do so.
That being said, we were careful not to bake the schema validation into the database itself. MongoDB, NoSQL, and all that.
Grahame Grieve (Dec 08 2016 at 19:06):
We used to be able to do that, back when we had the format for primitive types as :
"code": {
"value" : "xxxx",
"extension" : {
}
}
Grahame Grieve (Dec 08 2016 at 19:07):
instead of
"code" : "xxxx",
"code_" : {
}
Grahame Grieve (Dec 08 2016 at 19:07):
but we continually had implementers come to us and say "you guys just don't even understand JSON do you. It's just not xml"
Grahame Grieve (Dec 08 2016 at 19:08):
so eventually we bowed to the pressure and went to the split representation. We knew, when we did this, that it would mean no polymorphic collections
Grahame Grieve (Dec 08 2016 at 19:08):
from my point of view, this was a step backwards. But it's not going to change now
nicola (RIO/SS) (Dec 08 2016 at 19:33):
For json purists we could use something like this: {type: "string", string: "...value..."}, {type: "date", date: "...value..."}
Grahame Grieve (Dec 08 2016 at 19:35):
@Josh Mandel might want to comment but changing the name of "value" to the name of the type wasn't going to do it.
nicola (RIO/SS) (Dec 08 2016 at 19:38):
Changing value to datetype could simplify fhir/fluent path expressions and will play well with db indexes - so every type has it's own path
nicola (RIO/SS) (Dec 08 2016 at 19:39):
and it looks uniform with {resourceType: "...."} polymorphism
nicola (RIO/SS) (Dec 08 2016 at 19:40):
let say {dataType: ".....", ...} :)
Grahame Grieve (Dec 08 2016 at 19:41):
it wouldn't change fhirpath at all. but it would complicate navigation of the json where you just wanted to value, irrespective of the type. e.g. string, integer, boolean, uri, I don't care
Josh Mandel (Dec 08 2016 at 19:41):
Looking back at all of the pain this has caused, I certainly can't say that I'm happy with where we've landed. Extensible primitives with our current json representation is hard for everyone.
Josh Mandel (Dec 08 2016 at 19:42):
But yeah, calling the property value vs dateTime (or whatever) doesn't seem to matter much to me.
nicola (RIO/SS) (Dec 08 2016 at 19:43):
It matters for example for elasticsearch :)
Josh Mandel (Dec 08 2016 at 19:43):
Anyway, @nicola (RIO)'s comments were about polymorphic elements, not about standalone primitive values.
nicola (RIO/SS) (Dec 08 2016 at 19:44):
and fhirpath "element.date" vs "element(type=date).value
Grahame Grieve (Dec 08 2016 at 19:44):
they're related
nicola (RIO/SS) (Dec 08 2016 at 19:44):
I think, extending primitives is bad idea by themself :)
Josh Mandel (Dec 08 2016 at 19:44):
They could be. But even pre-DSTU1, we disallowed polymorphic lists, no?
nicola (RIO/SS) (Dec 08 2016 at 19:44):
I's like OOP of brain :)
Josh Mandel (Dec 08 2016 at 19:45):
@nicola (RIO) yes, that's my perspective too.
Josh Mandel (Dec 08 2016 at 19:45):
(I mean, about extensions on primitives. I don't know what "OOP of brain" means :))
Grahame Grieve (Dec 08 2016 at 19:46):
yes, we never allowed it. But I always knew that we could allow it if we really needed to, but we just had to define our own equivalent of xsi:type (with all the pain that it cases, along with defining something of our own). but once we made the json change, that was no longer an option
nicola (RIO/SS) (Dec 08 2016 at 19:46):
You could always create custom extension for additional info, instead "extending primitives" :)
Josh Mandel (Dec 08 2016 at 19:47):
Yeah, I'm thinking about doing that in the use cases I have.
Grahame Grieve (Dec 08 2016 at 19:48):
about 5% of our extensions are on primitive types. And in most of those cases, saying to put the extension on the parent not the child would just create confusion.
nicola (RIO/SS) (Dec 08 2016 at 19:48):
I think the name "primitive" means it's not extendable
nicola (RIO/SS) (Dec 08 2016 at 19:49):
i.e. not complex
nicola (RIO/SS) (Dec 08 2016 at 19:51):
We have polymorphic lists in Bundle.entry, do we?
Grahame Grieve (Dec 08 2016 at 19:51):
yes,f or resource type
nicola (RIO/SS) (Dec 08 2016 at 19:53):
I do not see essential difference with polymorphic lists for elements
Grahame Grieve (Dec 08 2016 at 19:55):
no, perhaps not. But perhaps you could comment about where in particular the resource content models are wrong because of this rule
nicola (RIO/SS) (Dec 08 2016 at 19:57):
We just have a lot of bugs and confusion with [x], that's why i've started this discussion :)
Grahame Grieve (Dec 08 2016 at 19:58):
we'd still have a lot of bugs. That's definitely a case of moving the deckchairs around
nicola (RIO/SS) (Dec 08 2016 at 20:01):
Usually when concepts are fuzzy - you've got a lot frustration. Simple & concise one always better :)
Paul Knapp (Dec 08 2016 at 20:19):
Is the burden of the 'confusion' for 5% of the extensions worth the burden for accomodating extensions on primatives? I'd think not. I think you'd consider an extension in the parent to either add an element or override an existing element.
Grahame Grieve (Dec 08 2016 at 20:38):
you haven't really considered all of the ways this plays out. if you 'decorate' a primitive with an extension, by putting it somewhere else, you're taken a step back - you're still going to need to consider the extension, but no it's not even in the right place. You stll haven't dealt with the actual problem, which is that there *are* extensions on primitive types
Paul Knapp (Dec 08 2016 at 20:47):
Well, it's not in the 'right place' in json and we seem to be coping with that. So your concern is that you have a business need to replace or augment the concept with a different or richer thing - is that really the same concept?
Paul Knapp (Dec 08 2016 at 20:50):
If the use case is that normally the concept foo can be represented by a primative but sometimes you need a complex to be used then adding fooComplex as an extension on the parent would not seem confusing.
Grahame Grieve (Dec 08 2016 at 20:51):
unless it repeats. And in json it works as well as a millipede with a wooden leg, really.
Paul Knapp (Dec 08 2016 at 20:52):
I expect that this would lead to less reuse of purely generic extensions and more element-specific extensions but that would further help reduce confusion.
Grahame Grieve (Dec 08 2016 at 20:52):
have you looked at the extensions we have on primitives?
Paul Knapp (Dec 08 2016 at 20:53):
Unless it repeats, you mean it has a cardinality greater than 1 or that there are several places where the same generic extention may apply?
Paul Knapp (Dec 08 2016 at 20:54):
No I was just starting to skim through the extensions. But certainly the 21090 extensions, not sure which may apply to primatives, could be cast as extensions which augment or overide a spcific element.
Grahame Grieve (Dec 08 2016 at 20:55):
most of them, it would not be very cool to permutate the extension identity with the element identity.
Paul Knapp (Dec 08 2016 at 20:59):
Then you could add an optional element to extension to indicate the element being extended: use referencing rather than positioning to provide context. On one level it makes things harder but I think it may slightly increase point complexity while overall reducing complexity.
Paul Knapp (Dec 08 2016 at 20:59):
We could explore Tuesday Q1 in ITS in San Antonio. I think we need to stand up some examples to really assess the pros and cons.
Grahame Grieve (Dec 08 2016 at 21:01):
we could just put all the extensions in the root of the element, and refer to the context by :id.
Grahame Grieve (Dec 08 2016 at 21:01):
that was my preferred approach for a long time. a bitch to serialise properly
Paul Knapp (Dec 08 2016 at 21:02):
root of the resource or at nodes?
Grahame Grieve (Dec 08 2016 at 21:02):
yes resource, that's what I meant
Paul Knapp (Dec 08 2016 at 21:02):
ok, so literally a bag and a map
Paul Knapp (Dec 08 2016 at 21:04):
I'm not championing any particular solution, but I think that not extending primatives is important, and if we are going to make that change that we should make it soon - not in haste but soon.
Grahame Grieve (Dec 08 2016 at 21:04):
well, we aren't making it for STU3. that's for sure.
Paul Knapp (Dec 08 2016 at 21:08):
I don't have any extensions I'm using which are applied to primatives so I'd need to refresh on what the impacts are of losing the context. We could point to an id attribute in xml on the primative, but in json what would we do - privide the path? or inject a marker element and point to it?
Grahame Grieve (Dec 08 2016 at 21:10):
well, this is the cool thing - in json, we'd need an id. So we could change the representation of the primitive to
"code" : {
"id" : "x2",
"value" : "xxxx"
}
Paul Knapp (Dec 08 2016 at 21:10):
it would mean you would need to pre-process to serialize and deserialize but I this we are there already so it is just changing what you are doing.
Grahame Grieve (Dec 08 2016 at 21:10):
that way, we could move extensions out to the root of the resource so we didn't need to have a complex primitive, and we could just have "code": "xxxx"
Grahame Grieve (Dec 08 2016 at 21:10):
;-)
Grahame Grieve (Dec 08 2016 at 21:11):
honestly, I prefer the json-ld format. it's more verbose, yes, but it's brutally simple.
Paul Knapp (Dec 08 2016 at 21:11):
Alternately we could get rid of primatives and make all data types complex which may contain complex or primatives - just a thought.
Grahame Grieve (Dec 08 2016 at 21:11):
you don't have to pre-process right now
Grahame Grieve (Dec 08 2016 at 21:12):
in effect, all data types are complex that contain primitive values right now. You're just mocving deck chairs around
Paul Knapp (Dec 08 2016 at 21:14):
No, integer is: <foo value="int"/> or foo: int - how is that already complex?
Paul Knapp (Dec 08 2016 at 21:15):
Yes we are moving the chairs, but that is what you do with chairs to make them efficient for the purpose
Grahame Grieve (Dec 08 2016 at 21:17):
integer is already complex because it has an id and extensions. The value is not. This is crystal clear in the structure definitions / typing system, and in the XML, and RDF.
Grahame Grieve (Dec 08 2016 at 21:17):
it's only the json format that complicates things by trying to simplify things
Paul Knapp (Dec 08 2016 at 21:18):
It's 00:16 so I'm going to crash, but I think a lot of people would appreciate a solution so that we didn't extend primatives directly.
Grahame Grieve (Dec 08 2016 at 21:18):
a lot of people would appreciate not having extensions. Until they see the alternative
Paul Knapp (Dec 08 2016 at 21:19):
Agreed - xml isn't dealing with the same 'primative' as json - and that has ramifications.
Paul Knapp (Dec 08 2016 at 21:21):
Actually I think a lot see the need for extensions and for them not to get out-of-hand like Z segments. The 2 pushbacks I hear are them being applied to primatives and their perceived overuse.
Josh Mandel (Dec 08 2016 at 21:45):
I'm not sure the json ld is nearly as simple as you describe. At least: do you expect it to be parsed according to json ld semantics? Should apps know how to parse general json-ld graphs, or is the precise layout in the FHIR examples what's required? Would apps recognize a flattened representation of the same graph (all nodes moved to the top) as equivalent? What about renaming the prefixes to arbitrary new values (as for namespaces in xml)?
Grahame Grieve (Dec 08 2016 at 21:49):
1. yes, it should be able to be, but I'm suspicious that polymorphism isn't right. @Eric Prud'hommeaux says I shouldn't worry about that
2. I don't know the answer
3. I don't whether they would. Should they? I don't know. I think it depends whether they use json-ld pattern for reading it, or just read it directly
4. what prefixes? I don't think we have any in json-ld
Josh Mandel (Dec 08 2016 at 23:16):
http://build.fhir.org/observation-example-TPMT-haplotype-two.html has two "@context" properties (!) which is an error
Grahame Grieve (Dec 08 2016 at 23:18):
no that's very useful is it
Grahame Grieve (Dec 08 2016 at 23:18):
typo. the second is supposed to be @id
Josh Mandel (Dec 08 2016 at 23:18):
We don't have prefixes but a bunch of properties defined in our contexts like http://build.fhir.org/Observation.jsonld and in theory one of could give these fields any name one liked, and client should understand that they mean the same
Grahame Grieve (Dec 08 2016 at 23:19):
by remapping in the definition? But how, when we publish the definitions, and the @context must refer to our definition?
Josh Mandel (Dec 08 2016 at 23:19):
Also, the reference to a contact is wrong because this context only to find properties about observation and does not define the rest of the properties that are used
Josh Mandel (Dec 08 2016 at 23:20):
In general in a Json LD file, you could change the contact and rename properties. I client would recognize that the file has the same meaning by expanding the properties to full Uris.
Josh Mandel (Dec 08 2016 at 23:20):
Something tells me we are not proposing using Json LD in this way.
Grahame Grieve (Dec 08 2016 at 23:20):
.... no I don't understand that one
Josh Mandel (Dec 08 2016 at 23:20):
But I'm not sure we have captured a clear explanation of how we are using it.
Grahame Grieve (Dec 08 2016 at 23:21):
we haven't captured that at all. You won't find any write up of any of this, just the examples and the jsonld definitions
Josh Mandel (Dec 08 2016 at 23:21):
In theory every property name can be expanded to its full URI by looking things up in the context. You could replace the instance data with a new copy where each property that was a full URI. That file that have the same meaning as the current instance data.
Grahame Grieve (Dec 08 2016 at 23:21):
yes indeed, but that requires different @context definitions. which we don't provide
Josh Mandel (Dec 08 2016 at 23:23):
What I am saying is that if we are using Json LD, and we expect clients to understand json-ld semantics, then we should expect clients to understand a payload like the one I just described.
Josh Mandel (Dec 08 2016 at 23:23):
If not, we need to explain what we *do* expect
Grahame Grieve (Dec 08 2016 at 23:24):
umm, I'm not sure how. Perhaps you could explain how that would work with an actual example
Josh Mandel (Dec 08 2016 at 23:24):
But in any case, this is the kind of thing I mean when we talk about the Simplicity of the format. One variant of the payload might look simple while another might look complex. And they both have the same meaning.
Grahame Grieve (Dec 08 2016 at 23:24):
and I mean, an actual resource. Cause I understand how that works in principle, but I believe that we eliminate much of this by fixing the value of @context
Josh Mandel (Dec 08 2016 at 23:25):
If you want to see an example, you can put one of the current resources into the sandbox at jsonld.org and click "expand". I will share link when I'm at a computer.
Grahame Grieve (Dec 08 2016 at 23:25):
I don't believe that will show me an example of what you're talking about
Grahame Grieve (Dec 08 2016 at 23:26):
for a start it doesn't work... I broke something...
Grahame Grieve (Dec 08 2016 at 23:28):
but that will show me the transform from one particular input to a canonical representation. And there could be other inputs that lead to the same canonical representation - in theory. But by locking down @context, we prohibit that, no?
Josh Mandel (Dec 09 2016 at 00:08):
A context is just a way to list a bunch of shortcuts that you are allowed to use. A given document can always expand everything fully and ignore those shortcuts even though they are available.
nicola (RIO/SS) (Dec 09 2016 at 07:23):
As I understand json-ld it's just json representation of RDF, where attributes are URIs and this approach is similar to what we've discussed for extension with embedded schema and extensions as key/value instead collection. I like it, because there is no difference between FHIR core elements and extensions.
Grahame Grieve (Dec 09 2016 at 09:26):
umm, no. not quite, I didn't see a way to pull off making them all the same, but happy to talk about ideas there
nicola (RIO/SS) (Dec 09 2016 at 12:56):
I mean something like this:
{
$schema: { // or @context
race: {url: "http://hl7.org/fhir/StructureDefinition/us-core-race", type: 'CodeableConcept'},
name: {url: "fhir", type: "HumanName", collection: true}
},
race: {coding: ....}, // extension
name: [...names....] // core element
}
````
Josh Mandel (Dec 13 2016 at 11:00):
You could make extensions just the same as regular elements, but you lose your trivial mapping back/forth with FHIR's other serialization formats. Sticking them all behind an "extension" property keeps this mapping very straightforward (but gives less idiomatic rdf).
Last updated: Apr 12 2022 at 19:14 UTC
