FHIR Chat · jsonschema · implementers

Stream: implementers

Topic: jsonschema

Pietro Ghezzi (Dec 28 2017 at 18:36):

Hi,
I'm new to FHIR. Just trying to use the json schemas, available on fhir for each resources, to validate input data on the backend. I have a problem with self referencing (for example the element schema, referencing to itself https://www.hl7.org/fhir/Element.schema.json), because most of the libraries like ajv crash going in infinite loop. There is a reason for self referencing and this is something that will stay in the future and if there is a solution to this. Thanks

Grahame Grieve (Dec 28 2017 at 21:28):

json schema is a challenge for us.

Grahame Grieve (Dec 28 2017 at 21:29):

I'm expecting to meet with several json schema tool maintainers in march to see if we can figure out how to do a json schema that works across multiple tools - all of them instruct us to do different things with the schema

Pietro Ghezzi (Dec 28 2017 at 21:33):

So this means that the schema will change soon?

John Moehrke (Dec 28 2017 at 22:02):

Grahame, has the new update to the IETF RFC for JSON helped? RFC8259 (STD90) https://www.rfc-editor.org/info/std90

Grahame Grieve (Dec 28 2017 at 22:19):

I don't know. I'll have to look

Pietro Ghezzi (Dec 29 2017 at 17:40):

I saw another problem with the schema. For example in Patient.schema.json is it correct that the dependencies : Patient_Contact, Patient_Link,... aren't inside dependencies, but on the same level as Patient? Shouldn't it be more like this https://gist.github.com/gpietro/20775ec6d6a53d50426a573a51e8ffeb ?

Abbie Watson (Dec 31 2017 at 05:22):

I saw another problem with the schema. For example in Patient.schema.json...

As far as I can tell, those Foo.schema.json files auto-generate the documentation, and aren't actual JSON-Schema files for the resource in question.

Grahame Grieve (Dec 31 2017 at 10:02):

nup I generate them with the intent that they are actual json schema files

Pietro Ghezzi (Dec 31 2017 at 10:43):

Yes they are valid json schemas. I'm trying to make them work with ajv library for validating data input.

Abbie Watson (Dec 31 2017 at 12:46):

Interesting. A couple of rough edges, but I think the Meteor community could start supporting the raw JSON Schema files then.

We've been refactoring/extracting the FHIR Simple Schemas into their own package:
https://github.com/clinical-meteor/fhir-simple-schemas

And our Patient schema is roughly analogous. We invert the required array into an optional field. And use a class based shorthand for describing cardinality and type.
https://github.com/clinical-meteor/fhir-simple-schemas/blob/master/schemas/Patient.js

But it looks like there's a JSON Schema to SimpleSchema converter:
https://github.com/bshamblen/meteor-json-simple-schema

So, with a little bit of work, we can help get the Meteor/Node/NPM communities using JSON Schema files, as published.

Abbie Watson (Dec 31 2017 at 13:22):

I guess my question would be: why all the underscore fields? Coming from the other direction, with a validation infrastructure that has been using simple-schemas for the past 2 years, it seems that the json-schemas are cluttered with underscore variables and extensions. Is there a way to rethink that?

Pietro Ghezzi (Dec 31 2017 at 14:06):

@Abigail Watson I still have some problems with that. https://github.com/epoberezkin/ajv/issues/659. I'm looking to find the correct format.
I got rid of underscore fields. They aren't necessary for input validation.

Abbie Watson (Dec 31 2017 at 14:38):

Huh. Looks like you're not resolving the urls for the datatypes. We had to implement those as separate schemas, and pull them in via ES6 imports. But it's not clear at all that a FHIR server ought to serve them up at any endpoint. Perhaps resolve to https://www.hl7.org/fhir instead of http://localhost:3030/fhir?

The problem with those $reffields is figuring out where they resolve to. You've approached the problem assuming that they would be endpoints, but the only thing they really specify is the name of a file. Could be accessed via file reader, or any number of other approaches.

Pietro Ghezzi (Dec 31 2017 at 16:57):

I have now a "working" solution with ajv without the use of Extension and ResourceList schemas.
The problem is that those schemas have too many $ref.

Grahame Grieve (Jan 01 2018 at 19:01):

the json schemas contain the underscored fields because the json source does too

Pietro Ghezzi (Jan 01 2018 at 19:41):

@Grahame Grieve What do you mean with json source?

Grahame Grieve (Jan 01 2018 at 20:19):

the json resources

Pietro Ghezzi (Jan 05 2018 at 10:07):

I'm expecting to meet with several json schema tool maintainers in march to see if we can figure out how to do a json schema that works across multiple tools - all of them instruct us to do different things with the schema

@Grahame Grieve can you tell some of the those instructions? I'm trying to elaborate the schemas. One try was to put all the references inside the resources definitions. It works but it is unacceptable since there are too many and files get too big. I'm curious to know some suggestions on how to solve this problem and try an implementation. Thanks

Pietro Ghezzi (Jan 05 2018 at 10:34):

Interesting. A couple of rough edges, but I think the Meteor community could start supporting the raw JSON Schema files then.

We've been refactoring/extracting the FHIR Simple Schemas into their own package:
https://github.com/clinical-meteor/fhir-simple-schemas

And our Patient schema is roughly analogous. We invert the required array into an optional field. And use a class based shorthand for describing cardinality and type.
https://github.com/clinical-meteor/fhir-simple-schemas/blob/master/schemas/Patient.js

But it looks like there's a JSON Schema to SimpleSchema converter:
https://github.com/bshamblen/meteor-json-simple-schema

So, with a little bit of work, we can help get the Meteor/Node/NPM communities using JSON Schema files, as published.

@Abigail Watson I don't see a good reason in converting json schema to another format like json-simple-schema. It's not even a standard, it would create only a dependency. I did try to solve the issue by importing refs into resources definitions but the file gets too big. Another solution could be: importing only the circular refs but I don't know if it's doable, or, simplify the schemas as you did in https://github.com/clinical-meteor/fhir-simple-schemas with an official full supported version. Any thoughts?

Abbie Watson (Jan 05 2018 at 22:06):

:shrug: It may not be suitable for your use case.

We like it because meteor-simple-schemasupports isomorphic client/server/database design, came out of the Meteor.js community, which was started and currated by MIT alumni, and is now one of the most popular web frameworks on the internet, and has been widely proven in production. But you're correct, it uses an adhoc schema library.

So, if we want to harmonize with the FHIR standards, we're probably going to want to migrate the fhir-simple-schemas from SimpleSchemas to JsonSchemas; which means moving towards meteor-json-simple-schema.

But all the source code is there. We've got a strategy for converting FHIR Json Schemas into Meteor SimpleSchemas, which can be attached to Mongo cursors on the server and minimongo cursors on the client. And it all supports hooks and triggers.

Guess what I'm trying to say is 'this is what we had to go through to get a working schema validation infrastructure working' and 'this is our plans for harmonizing with FHIR initiatives'.

Pietro Ghezzi (Jan 05 2018 at 22:55):

@Abigail Watson I didn't mean to criticize the choice of the framework. Meteor is perfectly fine. What I was trying to say is that if we stay with a standard format it would be better for everyone (you could choose, js, python, go,...) and there are already the fhir schemas for each resource (155). I think it needs just a little effort to make them work with validation libraries. It would be more difficult the other way around, I didn't find any library supporting self referencing. So far I see AJV as the best choice. But I would like to hear also the option of @Grahame Grieve about this, I have limited knowledge of FHIR.
Also Mongo version 3.6 supports now json schema validation:
https://docs.mongodb.com/manual/core/schema-validation/#json-schema

There are also client side library for rendering forms with json schemas. One is from mozilla:
https://github.com/mozilla-services/react-jsonschema-form

This would save a lot of work.

I'm available on helping with this if anyone like the idea.

Abbie Watson (Jan 05 2018 at 23:35):

No criticism taken. I'd very much like to move towards using the standard json-schemas that FHIR provides directly; and the Meteor community is in a great position to make that migration and promote it. The Mongo 3.6 support for json-schema in the database is pretty fantastic; and we could continue using meteor-json-simple-schema when we need validation on the server and client cursors.

I will say that we've had problems with using schema based form rendering in the past. We had a templating layer which people tried to add an additional templating layer on top of; and it became too much abstraction. But people continue asking about it; and the React approach with its pure methods and functional programming might make the approach work. The real win, of course, would be in autogenerating forms for the long tail of lesser-used FHIR schemas.

Abbie Watson (Jan 05 2018 at 23:35):

Anyhow, count me in. I'd be happy to test out the json-schemas for Mongo 3.6 and the react-jsonschema-form. We're doing a refactor/redesign right now, and I'm happy to take recommendations and ideas. (Still on the fence about ajv.js though. It's probably a solution for non-Meteor node apps.)

Abbie Watson (Jan 05 2018 at 23:48):

Now here's the important question: are the FHIR json-schemas available on NPM anywhere?

Joel Schneider (Jan 06 2018 at 19:28):

Is Swagger (aka Open API) a viable alternative or complement to other JSON schema technologies?
https://github.com/OAI/OpenAPI-Specification

Grahame Grieve (Jan 06 2018 at 20:30):

complement - it uses JSON schema, though with some technical restrictions that are very... painful... for us

Abbie Watson (Jan 06 2018 at 23:27):

So, I took the liberty of publishing the JSON Schemas onto NPM:
https://www.npmjs.com/package/fhir-schemas

And I managed to get Ajv.js working. Implementation is in the following file:
https://github.com/clinical-meteor/meteor-on-fhir/blob/schemas/webapp/server/main.js

A few notes:

1. Element, Extension, and ResourceList were the resources that caused the pain points.
2. The way they're structured requires importing ALL of the schemas into Ajv, so it can parse across the entire dependency graph. It's the allOf and oneOfdeclarations that wind up spanning across the entire API surface.
3. The $schema field in the published JSON Schemas isn't being picked up by Ajv. I had to delete it from each resource.
4. Likewise, the Foo.id field needed to be assigned to Foo.$id.
5. Lastly, and most importantly, the Foo.schema.jsonreferences needed to be replaced by http://hl7.org/fhir/json-schema/Foo

In summary, some of the $ fields seem to be misconstructed, and the $ref fields should point to the schema id, not the name of the .json file.

Abbie Watson (Jan 06 2018 at 23:29):

Later this week I'll do the meteor-json-simple-schema integration; and see if the $ref values work in that library also.

Pietro Ghezzi (Jan 07 2018 at 11:00):

The problem with swagger are the remote refs that is not resolved.

Abbie Watson (Jan 09 2018 at 00:14):

@Grahame Grieve Can you give any opinion on the viability of replacing the .json file names in the $ref fields with the $id? i.e. changing from $ref: "Foo.schema.json" to $ref: "http://hl7.org/fhir/json-schema/Foo"?

Grahame Grieve (Jan 09 2018 at 01:03):

One json schema tool author said that would be a wrong thing to do. Other than that, I don't know. I get conflicting advice from the json schema tools guys. That's why I'm hoping to meet them face to face in March

Abbie Watson (Jan 09 2018 at 01:40):

Gotchya. It sort of boils down to different implementations, then. Hate to suggest that we need a spreadsheet, but it sounds like it may boil down to doing some empirical research and tracking which schemas are supported in which tools. Thx!

Pietro Ghezzi (Jan 12 2018 at 12:59):

As I said it would be great to know what those advice from library maintainers are to try out some solutions and make a proposal.

nicola (RIO/SS) (Jan 28 2018 at 22:12):

@Grahame Grieve https://github.com/fhir-js/fhir-schema - source code & here is a demo http://fhir-js.github.io/fhir-schema/

For contained resources and resource list we could use reference to the root $ref: "#/" - so if engine support discriminator - it will report better errors

Abbie Watson (Feb 03 2018 at 16:40):

During the Working Group FHIRChat, someone mentioned that a new set of R4 schemas were published last week, after the connectathon? Is there a link?

John Silva (Feb 03 2018 at 18:59):

Graham posted in the JSON Schema(R3) thread that it's available for download on build.fhir.org - took me a while to find it -- here's the direct URL: http://build.fhir.org/fhir.schema.json.zip

Abbie Watson (Feb 04 2018 at 14:10):

Thank you! Exactly what I was looking for. It blipped by as desktop notification, and I couldn't find it again. Will get to work publishing it on NPM...

Abbie Watson (Feb 04 2018 at 16:55):

Okay, got it loaded up into the fhir-schemas package. Anybody have an Ajv.js syntax example for the new all-in-one file format? I'm not seeing id or $id properties on the resources.

Grahame Grieve (Feb 04 2018 at 20:31):

$id - ?

John Silva (Feb 04 2018 at 21:00):

Do you mean the meta.id property?

Abbie Watson (Feb 06 2018 at 01:10):

Well, I mean an $id field on the schema:
https://github.com/clinical-meteor/fhir-schemas/blob/master/fhir.schema.json/Patient.schema.json#L3

We're having a bit more success with the Json Schemas from the download page than the all-in-one schema. The schemas that are separated out in different files at least have an id field. We used that to add a $ and arrived at an $id field which our validator could pick up and stitch together into a graph. Having all the schemas in the same file solves the problem of building an API graph, I think; but now I'm not entirely sure how to query a single schema, if not by id.

I'm going to experiment with syntax a bit more. Having them all in a single file could be a good idea; but there's still a bit of room for improvement.

Grahame Grieve (Feb 06 2018 at 01:14):

so we decided to get rid of the individual schemas - they were generating all the problems with references, and they weren't real useful because they all depended on each other anyway

Grahame Grieve (Feb 06 2018 at 01:14):

I think you're doing something with the graph that's not clear to me - can you explain more?

Grahame Grieve (Feb 08 2018 at 20:07):

@Abigail Watson ping on this - I'd like to close JSON schema issues out

Abbie Watson (Feb 09 2018 at 16:26):

I think we're all set on the NPM side. We had to add the following $id field at the top of the file, like so:

"$id" : "http://hl7.org/fhir/json-schema",

Which lets us do the following:

import { FhirApi } from 'fhir-schemas';
import Ajv from 'ajv';

var ajv = new Ajv;
var validate = ajv.addSchema(FhirApi).getSchema('http://hl7.org/fhir/json-schema/Patient');

var newPatient = {
    "resourceType": "Patient",
    "name": [{
        "family": 'Doe',
        "given": ['Jane']
    }],
    "identifier": [{
        "value": '123'
    }]
};

var isValid = validate(newPatient);

Apparently, as long as that initial $id is there, Ajv will autogenerate the $id fields throughout the rest of the schema. We can even override it with a second parameter (but it's not as clean syntax).

var validate = ajv.addSchema(FhirApi2, "http://hl7.org/fhir/json-schema").getSchema('http://hl7.org/fhir/json-schema#/definitions/Patient');

Anyhow, it's all working great, with minimal maintenance on our part. Great work, everyone.

Abbie Watson (Feb 09 2018 at 16:30):

Also, it's now published in the fhir-schemasNPM package.
https://www.npmjs.com/package/fhir-schemas

I'm still refining the README with examples, and deprecating the older schemas that Meteor was using. Regardless, I have a list of list of Meteor apps that will be migrating towards it, and and a number of business clients using Node who are asking about it as well.

Grahame Grieve (Feb 09 2018 at 20:08):

ok added the root $id to the generator

Abbie Watson (Feb 09 2018 at 20:11):

Fantastic. Should we think about including semver or release version in there?
http://hl7.org/fhir/json-schema/v3.0.1/

Lloyd McKenzie (Feb 09 2018 at 20:14):

If we were to do that, we should trim it to "3.0" rather than "3.0.1"

Abbie Watson (Feb 09 2018 at 20:15):

Works for me. I'd only want to release on major versions, anyhow.

Grahame Grieve (Feb 09 2018 at 20:18):

I'll do that

Grahame Grieve (Feb 15 2018 at 23:14):

@Ewout Kramer - all this work was kicked off because you wanted to add more metadata to the json schema. Now would be a great time to start adding to it, now that it's valid

Ewout Kramer (Feb 16 2018 at 12:47):

@Ewout Kramer - all this work was kicked off because you wanted to add more metadata to the json schema. Now would be a great time to start adding to it, now that it's valid

Yes! I have a few:

• xml-order
• binding information (strength, valueset)
• is choice type?
• target profile
• aggregation
• mustsupport, ismodifier

just to get started...

pattern values - I doubt we could do that.

Michael Lawley (Feb 28 2018 at 06:03):

yikes - just loaded the R3 into Swagger UI and effectively killed Chrome as the mem usage swelled to over 20GB

nicola (RIO/SS) (Feb 28 2018 at 12:12):

That's swagger ui problem - create them issue on github :) - they should be lazy

Michele Mottini (May 08 2018 at 14:31):

...are the json schema files for DSTU2? Or a way to generate them?

Dread Hawk (May 08 2018 at 14:35):

http://hl7.org/fhir/DSTU2/allergyintolerance.html#resource
Check 'JSON'

Michele Mottini (May 08 2018 at 14:36):

That's a example of JSON coding, not the JSON schema...

Michele Mottini (May 08 2018 at 14:36):

There are JSON schemas for STU3 at https://www.hl7.org/fhir/fhir.schema.json.zip, but not for DSUT2

Dread Hawk (May 08 2018 at 14:37):

Derp. Sorry.

Michele Mottini (May 08 2018 at 14:46):

..no problem...thanks for trying!

Grahame Grieve (May 09 2018 at 04:24):

I thought I generated json schema for R2 somewhere -- but can't retrospectively add them to the spec itself.

Michele Mottini (May 09 2018 at 19:33):

If there is a way to get those files - or to generate them somehow - that would be great

Michele Mottini (May 09 2018 at 19:33):

(We have a customer asking for Swagger description of our FHIR DSTU2 implementation, and the json schemas are the closes thing to that)

Grahame Grieve (May 11 2018 at 19:03):

umm. I'll have a look - but I'm pretty backlogged up...

Dread Hawk (Jul 02 2018 at 19:58):

Bump - I need json schemas for testing purposes.

John Silva (Jul 03 2018 at 11:16):

They are 'hidden' ;-) in a non-obvious place under Documentation -> Downloads

http://build.fhir.org/downloads.html (R4) or https://www.hl7.org/fhir/downloads.html

The JSchema file is one BIG file with many internal references. I've had some problems using it (though not the most recent version) with Newtonsoft's JSchema implementation.

Dread Hawk (Jul 03 2018 at 16:54):

Sorry, I meant that I need STU2 json schemas, preferably with Argonaut profiles.

Grahame Grieve (Jul 03 2018 at 20:10):

I do not think that I have generated these, or that it's very easy for me to do so

Simone Heckmann (Aug 02 2018 at 06:22):

I received the following question today:

we experience inconsistencies in the FHIR json files from R4 Ballot #1 (FHIR 3.3.0) ( http://hl7.org/fhir/2018May ).
The experience with the current version (FHIR Release 3 STU) was much better and does not contain any of these inconsistencies.
Is that a “normal” behavior and we can expect the metadata quality to improve when it comes to the final R4 release?

Find below the list of problems with the JSON encoded files of the FHIR definitions and examples for version 3.3.0:

In http://hl7.org/fhir/2018May/definitions.json.zip

valuesets.json
Multiple entries missing 1..1 field CodeSystem.content: http://hl7.org/fhir/2018May/codesystem.html
We patched the standard to allow 0..1

fhir-versions CodeSystem with entry in concept list missing the 1..1 field code:
At lines

32913 {
32914 "display": "1.0.2",
32915 "definition": "DSTU 2 (Official version) with 1 technical errata"
32916 },

We changed to

32913 {
32914 "code": "1.0.2",
32915 "display": "1.0.2",
32916 "definition": "DSTU 2 (Official version) with 1 technical errata"
32917 },

feeding-device CodeSystem missing 1..1 field status
At lines

36431 "fullUrl": "http://hl7.org/fhir/CodeSystem/feeding-device",
36432 "resource": {
36433 "resourceType": "CodeSystem",
36434 "id": "feeding-device",
36435 "meta": {

We changed to

36431 "fullUrl": "http://hl7.org/fhir/CodeSystem/feeding-device",
36432 "resource": {
36433 "resourceType": "CodeSystem",
36434 "status": "unknown",
36435 "id": "feeding-device",
36436 "meta": {

dataelements.json
Multiple entries missing 1..1 field
We ended up ignoring them in the tests as it was not obvious how to fix

Here is the list of issues we found because we have tests validating the example files:

· Files with entry(ies) missing some mandatory base field:
o codesystem-extensions-CodeSystem-{author,effective,end,keyword,workflow}.json
o valueset-extensions-CodeSystem-{author,effective,end,keyword,workflow}.json

· Files with entry(ies) missing some mandatory type field:
o definition.json
o event.json
o request.json
o fivews.json

· Files with entry(ies) missing some mandatory linkId field:
o All files whose name end with -questionnaire.json, except for operation-structuredefinition-questionnaire.json

· Files with entry(ies) missing some mandatory code field:
o codesystem-fhir-versions.json
o valuesets.json
o dataelements.json

· Files with entry(ies) missing some mandatory status field:
o codesystem-feeding-device.json
o codesystem-fhir-versions.json

...can someone comment on these issues?

Gabriel Bezerra (Aug 02 2018 at 13:55):

Hi Simone, thanks for posting it. I'm the one that found these issues in your comment. Another case is the file plandefinition-example-cardiology-os.json in examples. On line 551 the field action contains null values without a companion _action field containing corresponding {id: “”, extension: {…}} values.

    551       "action": [
    552         null,
    553         {
    …
    601         },
    602         null,
    603         null,
    604         null,
    605         {
    …
    700         }
    701       ]

Simone Heckmann (Aug 08 2018 at 06:23):

@Grahame Grieve ...?

Grahame Grieve (Aug 08 2018 at 06:35):

json schema is on my list for the net week or so

Pietro Ghezzi (Jan 27 2019 at 23:25):

I just released a work I have done two years ago with the jsonschema. It's a javascript generator that helps to create a REST API with FHIR resources. It only support STU3 for now. If somebody has the time to check it out I would interested in some feedback.

Pietro Ghezzi (Jan 27 2019 at 23:25):

https://www.npmjs.com/package/generator-fhir

Last updated: Apr 12 2022 at 19:14 UTC