FHIR Chat · JSON Extensions · implementers

Stream: implementers

Topic: JSON Extensions

Grahame Grieve (Mar 05 2017 at 19:44):

Someone else has taken a stab at the syntax for extensions in JSON: http://community.fhir.org/t/suggestion-for-extensibility-in-json/369/2. This is a new variant we haven't discussed before.

Grahame Grieve (Mar 05 2017 at 19:44):

procedurally, It's really hard for me to imagine that this is something we'd change at all, let alone now, but I know that not everyone is happy with where we are...

nicola (RIO/SS) (Mar 05 2017 at 19:57):

@Grahame Grieve i've written about this 5 month ago and we implemented it in aidbox as additional json+aidbox format :) https://gist.github.com/niquola/0f66419274af5e3add9a33766e21b8c7

Grahame Grieve (Mar 05 2017 at 19:57):

this particular syntax? I don't recall this particular variant being discussed?

nicola (RIO/SS) (Mar 05 2017 at 19:59):

https://chat.fhir.org/#narrow/stream/implementers/topic/Extensions.20-.20Representation

Grahame Grieve (Mar 05 2017 at 19:59):

again, not this particular syntax

nicola (RIO/SS) (Mar 05 2017 at 20:00):

The core idea - it should be key-value, not collection

nicola (RIO/SS) (Mar 05 2017 at 20:04):

Please, read gist above - there are also couple of ideas about polymorphics & primitive extensions

nicola (RIO/SS) (Mar 05 2017 at 20:14):

Or may be i have to ask Mostafa Sholkamy to present it :)

Michael Lawley (Mar 05 2017 at 23:49):

As you ask in that thread, there's the open issue of where the outermost extension key comes from.
Also, the XSLT example is brittle, so it's not as simple as it makes out; the matching doesn't restrict to http://hl7.org/fhir/StructureDefinition/patient-clinicalTrial and could be broken by other extensions that have overlapping sub-parts

nicola (RIO/SS) (Mar 06 2017 at 06:38):

@Michael Lawley this issue could be solved by embedding schema like this:

....
extension: {
  $schema: {
    livingArrangement: {url: "http://hl7.org/fhir/ValueSet/v3-LivingArrangement", type: 'CodeableConcept'}
    race: {url: "http://hl7.org/fhir/StructureDefinition/us-core-race", type: 'CodeableConcept'},
    adoption: {url:"http://hl7.org/fhir/StructureDefinition/patient-adoptionInfo", type:  'string'},
    whenAdopted: {url:"http://narus.com/when-adopted", type: 'date'},
    occupation: {url:"http://hl7.org/fhir/ValueSet/anzsco-occupations", type: 'string'}
    marriedDate: {url:"http://narus.com/fhir/date-married", type: 'date'},
    employer: {url:"http://narus.com/fhir/supportive-employer", type: 'boolean'}
  },

  livingArrangement: {coding: ...},
  race: {coding: ....},
  adoption: "no",
  whenAdopted: "1980"
  occupation: "Telecommunications Engineer",
  marriedDate: "2001"
  employer:  true
}
...

nicola (RIO/SS) (Mar 06 2017 at 06:43):

@Grahame Grieve we can introduce new JSON format as not breaking change - let's just support both current and new one with different mime types

Lloyd McKenzie (Mar 06 2017 at 06:44):

So the element names could vary from instance to instance and would essentially be made up by the serializer?

nicola (RIO/SS) (Mar 06 2017 at 06:49):

This is well known namespacing problem and solution. If we have name clash - we could use aliasing like "alias1/key" and "alias2/key" like in most of modern programming languages

nicola (RIO/SS) (Mar 06 2017 at 06:51):

Probability of name clash is not so heigh, and, i think, usability profit worth it

Michael Lawley (Mar 06 2017 at 06:51):

@nicola (RIO) Sorry, I was responding to @Grahame Grieve . However, the problem with the $schema syntax is that a client can't just say, for example, resource.extension.whenAdopted since 'whenAdopted' could be mapped to any extension URL (or none at all). Yes, this is the well known namespacing problem, but it's clients that pay the price

nicola (RIO/SS) (Mar 06 2017 at 06:52):

all extensions go into separate key extensions so you always know it's extension

nicola (RIO/SS) (Mar 06 2017 at 06:54):

If you are not sure about it's semantic or have a conflict or merge, you lookup $schema resolution object - but in many of applications you will know your extensions

Michael Lawley (Mar 06 2017 at 06:57):

If I'm writing a generic client that's looking for a well-known (to me) extension (let's say "http://narus.com/when-adopted"). I can't assume that every server is going to serve resources with that URL mapped to "whenAdopted". How should I consume/navigate this JSON in a safe way?

nicola (RIO/SS) (Mar 06 2017 at 06:58):

You could do schema resolution (i.e. very simple transformation):

extension: {
   $schema: {key: "narus.com/whenAdopted"}
    whenAdopted: ....
}

=>
extension: {
    "narus.com/whenAdopted": .....
}

Michael Lawley (Mar 06 2017 at 07:00):

Which we could have had in the first place?

nicola (RIO/SS) (Mar 06 2017 at 07:01):

This pure transformation (i.e. do not require any external knowledge) will provide you with safe aggregation format (very simmilar to json-ld)

nicola (RIO/SS) (Mar 06 2017 at 07:01):

@Michael Lawley excuse me, do not understand question :(

Michael Lawley (Mar 06 2017 at 07:05):

What I meant was, why not start with

extension: {
    "narus.com/whenAdopted": .....
}

nicola (RIO/SS) (Mar 06 2017 at 07:06):

Yes it's valid, but developers prefer simple short names - so why do not give them aliasing tools

Michael Lawley (Mar 06 2017 at 07:07):

The $schema stuff just looks like a compression technique? If you want short names, do the reverse transformation where you're in control of the target names.
It's just my gut saying that they should not be part of the wire protocol.

nicola (RIO/SS) (Mar 06 2017 at 07:07):

We introduced this format half year ago and our clients prefer it for internal API calls, while for external integrations we automatically convert to current FHIR JSON format

nicola (RIO/SS) (Mar 06 2017 at 07:09):

$schema also could be used to put some critical metadata like types or isModifier

nicola (RIO/SS) (Mar 06 2017 at 07:10):

But this is details - if we agree on key-value representation of extensions - that's cool

Grahame Grieve (Mar 06 2017 at 08:31):

so you can do that transformation now. We don't need to change anything for that. This is pretty much where we ended up last time...

nicola (RIO/SS) (Mar 06 2017 at 09:09):

Ok, let's wait more feedback on JSON from real users :)

nicola (RIO/SS) (Mar 06 2017 at 09:12):

Essential problem with current JSON, that it does not play well with design of JSON schema - you could not express some constraints (polymorphics, primitive extensions, extensions) in JSON-schema bacause current JSON format not enough jsonish :( So standard JSON schema (without custom extensions) able only partialy validate FHIR resources.

nicola (RIO/SS) (Mar 06 2017 at 09:15):

I could prepare talk for May 2017 HL7 International Conference & Working Group Meeting about JSON related issues and possible solutions

Abbie Watson (Mar 06 2017 at 09:15):

You do realize that json-schema isn't the only JSON schema package around, right?

nicola (RIO/SS) (Mar 06 2017 at 09:17):

@Abigail Watson what do you mean - concrete JSON schema engine? We've implemented json-schema in clojure and used several npm packages.

nicola (RIO/SS) (Mar 06 2017 at 09:17):

JSON-schema is specification not package :)

nicola (RIO/SS) (Mar 06 2017 at 09:20):

Or you are talking about antoher schemata for json. JSON schema is very popular and core of Swagger specification - that's why, i think, it's right direction for FHIR to have good support for it.

nicola (RIO/SS) (Mar 06 2017 at 09:23):

There is some "internal logic" or "JSON thinking" and other specs like GraphQL, avro etc expect it - but FHIR breaks this expectations

nicola (RIO/SS) (Mar 06 2017 at 09:24):

Just try to express in JSON schema required value[x] parameter, primitive extensions or validation for custom extension - and you will understand, what i'm talking about.

Abbie Watson (Mar 06 2017 at 09:30):

I'm talking about other schemata. The Meteor ecosystem has an isomorphic schema layer that handles most of those things on the server and client, and lets us use FHIR schemas as our database schema. We've got extentions, multi parameter matching, versioning, and the other things you're talking about mostly solved. I've been implementing them. So I basically know exactly what you're talking about.

Abbie Watson (Mar 06 2017 at 09:33):

It's super late here and I'm crashing for the evening; but um... I think the ecosystem is a bit bigger than you seem to think it is. (Although I'll grant you one thing... JSON Schema having IETF Draft is a compelling argument. I'm going to bring it up to the Simple Schema folks and do a bit more research. )

Michael Lawley (Mar 06 2017 at 10:06):

Interesting - I presume you mean https://www.meteor.com/ -- there are many "Meteor"s out there and this is one I had not come across

nicola (RIO/SS) (Mar 06 2017 at 12:24):

Thx @Abigail Watson I will take a look at meteor's schema. If you have your FHIR extensions in open source - please send me a link. Does meteor's schema has specification? After we could discuss it in more details ;) We also played a lot with graphql and avro - have you seen it?

Igor Sirkovich (Mar 06 2017 at 16:31):

Yes, Meteor refers to meteor.com. I believe all Abigail's Clinical Meteor codebase is open source and can be found at https://github.com/clinical-meteor

Abbie Watson (Mar 06 2017 at 17:24):

The following package implements the FHIR Patient Resource and passes the Touchstone API tests for version 1.6.0.
https://github.com/clinical-meteor/hl7-resource-patient

It basically uses 'node-simple-schema, and does the same thing that Swagger does, using some Meteor specific packages calledJsonRoutesandCollection2`. I'm not saying that everybody should drop what they're doing, and use Meteor. However, we've put a year or two of work into this ourselves, and have got an isomorphic schema layer that's working for API endpoints, server collections, client collections, our database schemas, *and* subscriptions. Understandably, we're reluctant to toss all that work away, and sort of like things how they are.

In particular, lines 73 through 75 show how we're building composite resources that can handle extensions.
https://github.com/clinical-meteor/hl7-resource-patient/blob/master/lib/Patients.js#L73-L75

And the latest node-simple-schema package has support for SimpleSchema.oneOfwhich can support the Resource.value[x] pattern.
https://github.com/aldeed/node-simple-schema#multiple-definitions-for-one-key

Abbie Watson (Mar 06 2017 at 17:33):

As for graphql, there's the Apollo project from the Meteor Development Group.
http://www.apollodata.com/

We've been waiting until it had a Subscription implementation before moving over. Classic Meteor included a pub/sub technology called DDP/Livedata that was loosely based on the old PubSubHubBub protocol. So Meteor has had Subscriptions for like 4 years now; in an implementation very familiar to what's used in Radiology and Laboratories. We've been waiting for Apollo to catch up with it's Subscription implementation, so we can do a clean refactor and swap the one infrastructure out for the other. It's probably going to break isomorphicity on the database schemas layer though; and break the rest of our tooling pipeline.

So... we're heading in the GraphQL direction, but we're happy to take our time. We may not adopt for another year or two yet. The DDP/Livedata implementation is 4 years mature, and meets the needs of Radiology, Laboratory, etc; whereas GraphQL is only 1 or 2 years mature at this point.

Lloyd McKenzie (Mar 06 2017 at 19:27):

Keep in mind that *no* schema language (XML or JSON) is going to be able to enforce all the constraints of a profile - slicing, terminology bindings, FHIRPath invariants, etc.

nicola (RIO/SS) (Mar 07 2017 at 08:52):

@Lloyd McKenzie just as example if extensions will be key-value - it's possible to express slicing by JSON schema very naturally, JSON schema could be easyly extended to validate terminology bindings. FHIR tools also unable to enforce all buisness validations on resources :( - so we are fighting to express as much as possible in a declarative way.

Grahame Grieve (Mar 07 2017 at 08:53):

can you be specific about all these please

nicola (RIO/SS) (Mar 07 2017 at 08:57):

@Grahame Grieve do you mean, how to describe slicing by JSON-schema or validate terminology bindings?

Grahame Grieve (Mar 07 2017 at 08:58):

both of those, and also which business validations that the FHIR validation tooling can't enforce that json schema could

nicola (RIO/SS) (Mar 07 2017 at 08:59):

I mean, not FHIR not JSON schema could enforce a lot of real-world app invariants

nicola (RIO/SS) (Mar 07 2017 at 09:00):

Because this invariants involve a lot of resources and relationships

nicola (RIO/SS) (Mar 07 2017 at 09:02):

Schema or FHIR could validate structure of resource and some invariants of it's internal parts

nicola (RIO/SS) (Mar 07 2017 at 09:08):

About slicing - it's activelly used to describe extensions, because they are collections.
If extensions would be objects like:

extension: {
    "us/race": ....
    ...
}

It's easy to specify schema for it:

extension: { 
    type: object,
    properties: {
          "us/race":   { type:  object, ................}
    }
   required: ["us/race"]
}

Grahame Grieve (Mar 07 2017 at 09:09):

but you can still slice without that. It's just a little less direct

nicola (RIO/SS) (Mar 07 2017 at 09:13):

Yes but this approach is more natural, at least for me.

nicola (RIO/SS) (Mar 07 2017 at 09:15):

As a developer i want to access race as Patient.extension.race not Patient.extension.where(url="race")[0], i would like to say there is only one race per patient etc

nicola (RIO/SS) (Mar 07 2017 at 09:16):

Because in FHIR i do Patient.name not Patient.elements.where(prop="name")

nicola (RIO/SS) (Mar 07 2017 at 09:18):

That's the diff between what FHIR force me and if i would do it by myself or how it's done by my framework etc (i.e. without standard) - this impedance will be in mind of many developers

nicola (RIO/SS) (Mar 07 2017 at 09:28):

I consulted several projects, which thinking about or starting using FHIR - and most of developers compare standard approach to their personal approach or industry approach to model entities, build API etc. The more impedance means less chances to choose FHIR as foundation for system design and decide - it's only transport format (like others).

Lloyd McKenzie (Mar 07 2017 at 18:29):

How would this work with JSON schema - where the presumption is all FHIR instances in the world need to validate against a single schema?

Michael Lawley (Mar 08 2017 at 00:05):

@nicola (RIO) I think this would best done client-side as a transformation on the JSON coming in and again in reverse before the JSON goes out.
Then, as a client, I could know that the extension was mapped to race and not, for example, us_race.

nicola (RIO/SS) (Mar 08 2017 at 08:45):

@Michael Lawley yes for clinet side we do this, but we still have some problems with schema and database, which are on a backend (https://gist.github.com/niquola/0f66419274af5e3add9a33766e21b8c7).

Mostafa Sholkamy (Mar 08 2017 at 16:27):

@nicola (RIO) this solution is looking good

Mostafa Sholkamy (Mar 08 2017 at 16:51):

@Abigail Watson I'm building a GraphQL server for FHIR here using express.js server for apollo, if you can take a look

Abbie Watson (Mar 08 2017 at 18:27):

Hi Mostafa. Fantastic bit of work there. If you don't mind, I'm going to include a link in the Clinical Meteor SDK, so people can keep track of this project. Very promising. I'm also checking up on the recommended build pipeline. We use a Meteor pipeline, but Webpack/Babel isn't necessarily a deal breaker. This architecture actually aligns well with the SNOMED-CT server from IHTSDO.

Michael Lawley (Mar 09 2017 at 05:51):

@nicola (RIO) Do you have a publicly available javascript function that implements the transform?

nicola (RIO/SS) (Mar 09 2017 at 07:55):

@Michael Lawley it's in clojure, but we can port to js, if you are interested

Michael Lawley (Mar 10 2017 at 03:42):

If you want to port it that would be awesome, but I'm pretty sure I could do it myself if I could take a look at your clojure implementation

Last updated: Apr 12 2022 at 19:14 UTC