Stream: implementers
Topic: JSON Schema
Grahame Grieve (May 21 2016 at 01:14):
I have committed a draft JSON schema for all resources and data types. I haven't tested it - the java options for enforcing JSON schema don't seem very attractive - one requires Java 1.8, and one it not maintained. I'm open to suggestions for testing the JSON schemas as part of the build
Lloyd McKenzie (May 22 2016 at 15:35):
Is requiring Java 1.8 a problem?
Vadim Peretokin (May 22 2016 at 22:32):
1.7 hasn't had any security updates for about a year now
Grahame Grieve (May 23 2016 at 01:00):
Don't know. I'm pretty conservative about this. Past experience is that if I make the build dependent on a new version of java, it's going to create heaps of work for me
nicola (RIO/SS) (Nov 06 2016 at 10:46):
Hi @Grahame Grieve some notes about JSON schema coming with FHIR build:
- Separate schemata is generated for nested elements (like Patient_Link etc), but JSON schema support nested declarations, which is more idiomatic.
- Using "oneOf" for validating Bundle and/or contained resources is slightly wrong semantically, we use JSON schema extension from Swagger - "descriminator" (https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#fixed-fields-13) .
- We also were able to extend JSON schema to support validation of ValueSets and Referential integrity.
Such extensions is quite simple and we could extend popular JSON schema engines for them.
In aidbox we would like to provide Swagger API to FHIR server and would like to actively collaborate on JSON schema and Swagger support in FHIR (let's discuss it in Amsterdam :)
Grahame Grieve (Nov 06 2016 at 10:50):
ok sure let's talk about that in DevDays
Jose Montoya (Dec 21 2016 at 02:00):
Hey all. We're currently trying to use schema.json files to provide the validation for our content. I'm running into errors related to looping schemas.
fatal: validation loop: schema "file:/home/.../definitions/DomainResource.schema.json#/definitions/DomainResource/allOf/0" visited twice for pointer "/contained/0" of validated instance
The error is thrown by anything with a "contained" non-empty array
Grahame Grieve (Dec 21 2016 at 02:19):
e.g. everything. In which case, you need the single json schema fhir.schema.json
Jose Montoya (Dec 21 2016 at 03:10):
I'm not sure I understand how this would help. It seems to me that this would still lead to some recursive validation.
Grahame Grieve (Dec 21 2016 at 03:16):
typically, this is a problem because there's a library trying to resolve all the library references before starting to do type based reasoning.
Grahame Grieve (Dec 21 2016 at 03:16):
there's not much likelihood of running into a schema library that doesn't allow types to reference each other
Jose Montoya (Dec 21 2016 at 03:18):
But for the that do try to resolve references first then this would not work...?
Grahame Grieve (Dec 21 2016 at 03:21):
that's why there's a single schema that contains everything - for those tools.
Ville Lindholm (Apr 15 2017 at 07:14):
Hi! I downloaded the STU3 JSON Schema files so that I can validate a QuestionnaireResponse in a Node backend service. (I use the ajv module for the validation.) It seems that the schema files use an inconsistent referencing method. Each resource id is of the form "http://hl7.org/fhir/json-schema/SomeResource" but when referencing resources with "$ref" the schema files use "SomeResource.schema.json#/definitions/SomeResource". I had a chat with the ajv package maintainer and this is apparently not valid JSON Schema. Either the id should end with "SomeResource.schema.json" or the $ref should use "SomeResource#/definitions/...". As things stand now, ajv can't follow the references unless a custom mapping is provided to "rename" the id.
Ville Lindholm (Apr 15 2017 at 07:15):
SomeResource.schema.json is also the file name, but JSON Schema references don't really know about file names apparently.
Ville Lindholm (Apr 15 2017 at 07:18):
I thought about providing patch for this but couldn't find the schema files in SVN :D
Ville Lindholm (Apr 15 2017 at 07:19):
Oh, and as a mini bug report, the stu3 home page link to this chat actually links to "char.fhir.org" ;)
Grahame Grieve (Apr 15 2017 at 20:27):
thx. can you createa a gForge task for this - see http://gforge.hl7.org/gf/project/fhir/tracker/?action=TrackerItemBrowse&tracker_id=677
Grahame Grieve (Apr 15 2017 at 20:27):
and which schema are you using? the single schema file, or the per-resource schema?
Grahame Grieve (Apr 16 2017 at 21:39):
@Ville Lindholm - can you put me in contact with the author of the tool? thanks
Grahame Grieve (Apr 16 2017 at 21:43):
and also try this set of json schema:
Grahame Grieve (Apr 16 2017 at 21:51):
Ville Lindholm (Apr 17 2017 at 09:00):
@Grahame Grieve Sure, the ajv project has a chat similar to this one on gitter: https://gitter.im/ajv-validator/ajv and the person I spoke to was epoberezkin. The new schema package you provided looks like it should work!
Ville Lindholm (Apr 17 2017 at 09:01):
I was using the per-resource schemas to optimize things a bit
Ville Lindholm (Apr 17 2017 at 10:05):
So to summarize, dropping .schema.json from refs is the quickest fix. If you want to host the files so that the URIs become actual downloadable files the filenames should also drop the .schema.json extension OR the web server should map Element -> Element.schema.json
Ville Lindholm (Apr 17 2017 at 10:06):
i.e. the zip file above works
Ville Lindholm (Apr 17 2017 at 10:06):
but tools that automatically resolve the ID with HTTP won't get the file at the moment
Grahame Grieve (Apr 17 2017 at 21:15):
Perhaps I should generate the schema twice with both sorts of references?
Grahame Grieve (Jun 28 2017 at 12:43):
@Ville Lindholm - I have updated all the redirections on the web site for JSON schema. Interested to know if I've fixed it any. But generally, the JSON schema problem looks intractable to me - $ref is critical to any JSON scheam for FHIR, and the tools are all over the place on it
Jose Montoya (Dec 11 2017 at 23:37):
@Grahame Grieve -- Im currently working on a tool that'd make use of these json schemas taken from the downloads page. Are these kept up to date automatically? Is there a better source?
Just as a note, since the link does not offer any sort of cache related headers I'll probably have to download the zip every time and maybe use the hash to check whether I already have the latest ones. Any advise is appreciated.
Grahame Grieve (Dec 12 2017 at 05:45):
those are the best, and they are generated on the fly. What kind of cache related headers are you talking about? (that's a standard IIS server there)
Jose Montoya (Dec 12 2017 at 16:16):
I ask because the files within the zip are dated to April of this year. And some ETag functionality would be great, if we'd send an If-None-Match and get a 304 if we already have the latest zip then we wouldn't download the file again.
Grahame Grieve (Jan 27 2018 at 23:45):
@Ewout Kramer
nicola (RIO/SS) (Jan 28 2018 at 03:18):
Here is discriminator from latest OpenAPI - https://swagger.io/docs/specification/data-models/inheritance-and-polymorphism/
nicola (RIO/SS) (Jan 28 2018 at 03:20):
https://swagger.io/specification/#discriminatorObject
Grahame Grieve (Aug 09 2018 at 03:42):
One of the reasons we are having problems with json schema is because the Java json schema validator doesn't seem to be very conformant. So I'm going to try changing the json schema validation library that we're using in the build to check the json schema. The question is, what library? It needs to be able to run from java code, and needs to run without extensive set up requirements on windows, mac, linux.
Grahame Grieve (Aug 09 2018 at 03:43):
the other question is: I validate it by running every valid instance through the schema. But I don't have a library of invalid instances. But the candidate normative resources are stable enough to make it worth investing in that now - if any one wants to volunteer
Grahame Grieve (Aug 16 2018 at 01:00):
... back to this....
Grahame Grieve (Aug 16 2018 at 01:00):
here's a json schema, a little test case derived from the fhir json schema:
{ "$schema": "http://json-schema.org/draft-06/schema#", "id": "http://hl7.org/fhir/test-json-schema/3.4", "description": "for unit tests", "discriminator": { "propertyName": "resourceType", "mapping": { "A": "#/definitions/A", "B": "#/definitions/B" } }, "oneOf": [ { "$ref": "#/definitions/A" }, { "$ref": "#/definitions/B" } ], "definitions": { "boolean": { "pattern": "^true|false$", "type": "boolean" }, "id": { "pattern": "^[A-Za-z0-9\\-\\.]{1,64}$", "type": "string" }, "integer": { "pattern": "^-?([0]|([1-9][0-9]*))$", "type": "number" }, "string": { "pattern": "^[ \\r\\n\\t\\S]+$", "type": "string" }, "Element": { "properties": { "id": { "$ref": "#/definitions/string" } }, "additionalProperties": false }, "Coding": { "properties": { "id": { "$ref": "#/definitions/string" }, "system": { "$ref": "#/definitions/string" }, "version": { "$ref": "#/definitions/string" }, "code": { "$ref": "#/definitions/string" }, "display": { "$ref": "#/definitions/string" }, "userSelected": { "$ref": "#/definitions/boolean" } }, "additionalProperties": false }, "A": { "properties": { "resourceType": { "const": "A" }, "id": { "$ref": "#/definitions/id" } }, "required": [ "resourceType" ] }, "B": { "properties": { "resourceType": { "const": "B" }, "code": { "$ref": "#/definitions/id" }, "string": { "$ref": "#/definitions/string" }, "integer": { "$ref": "#/definitions/integer" }, "boolean": { "$ref": "#/definitions/boolean" } }, "additionalProperties": false, "required": [ "resourceType", "code" ] } } }
Grahame Grieve (Aug 16 2018 at 01:01):
here's a little sample instance that's valid - I believe - against that schema:
{ "resourceType" : "A" }
Grahame Grieve (Aug 16 2018 at 01:02):
of the json schema engines I've tried, 3 of 6 think that's valid. ! of the 3 that failed at least reported that they don't support the 'const' keyword (guaranteed fail) but didn't comment on the schema version. the other 2 simply reported some variant of :
2 subschemas matched instead of one
.. indicating that they don't support one of the key words...
Grahame Grieve (Aug 16 2018 at 01:03):
.... I really don't like JSON schema much....
Grahame Grieve (Aug 16 2018 at 01:03):
the java engine is one that fails :-(
Grahame Grieve (Aug 16 2018 at 01:08):
https://github.com/everit-org/json-schema/issues/205
Brian Lind (Aug 29 2018 at 20:54):
I like the new flattened out JSON schema in a single file, but I noticed the _id property is no longer present. Is this intentional? Isn't _id needed to hold extensions to the id element?
Brian Postlethwaite (Aug 29 2018 at 22:57):
That property would be for the XML id attribute, and is a great question on extending the Id element in Json. Not sure if that's come up before.
<Patient xmlns="http://hl7.org/fhir">
<id id="2" value="5">
<extension url="ext-url">
<valueCode value="3" />
</extension>
</id>
is represented in JSON as
{
"resourceType": "Patient",
"id": "5",
"_id": {
"id": "2",
"extension": [
{
"url": "ext-url",
"valueCode": "3"
}
]
},
nicola (RIO/SS) (Aug 30 2018 at 09:07):
my eyes are bleeding :(
nicola (RIO/SS) (Aug 30 2018 at 09:09):
Here was a long discussion - https://chat.fhir.org/#narrow/stream/4-implementers/topic/JSON.20Format.20for.20Primitives.20.26.20extensions
Brian Lind (Aug 30 2018 at 19:17):
I looked through the long conversation, but still don't understand why "_id" existed in the 3.2.0 schema, but is not in the 3.3.0 schema.
Grahame Grieve (Aug 30 2018 at 22:42):
when would you use _id?
Grahame Grieve (Aug 30 2018 at 22:42):
I think Nicola is right about the reference
Grahame Grieve (Aug 30 2018 at 22:42):
we decided you can't extend the id, and so I didn't generate that
Patrik Sundberg (Sep 01 2018 at 01:28):
@Grahame Grieve where in the spec is the decision that id can not have extensions written up? i don't see it here: http://build.fhir.org/id.profile.json.html, here http://build.fhir.org/element.profile.json.html or here http://build.fhir.org/resource.profile.json.html ?
Grahame Grieve (Sep 01 2018 at 20:32):
it would appear that we don't say it anywhere
Jim Taylor (Nov 10 2018 at 16:00):
Hello everyone,
Jim Taylor (Nov 10 2018 at 16:05):
Let's try this again. We are working to implement a FHIR product offering based on the HAPI FHIR specification. We are now focused on providing functionality for JSON resources. Can you please let me know if the FHIR community has established a best practice or policy regarding JSON validation? Is the plan to use JSON schemas? Thank you, Jim
Lloyd McKenzie (Nov 10 2018 at 16:40):
Best practice is to use the Java validator - it can test FHIRPath constraints, do terminology validation and other things that JSON schema (and XML schema + Schematron) can't do. It also works for JSON, XML, RDF and any other syntaxes that might be supported going forward.
nicola (RIO/SS) (Nov 11 2018 at 08:24):
In aidbox we use json schema as a primary validation tool. There is official JSON schema (https://www.hl7.org/fhir/downloads.html). To do proper validation we extend JSON schema with additional keys - for example to validate valueset's bindings or references). If you interested - we can share our ideas and artefacts.
Abbie Watson (Nov 11 2018 at 08:32):
And in the Meteor ecosystem, we've packaged up the JSON schemas that Nicola is referring to above, and tried to provide various validators, including SimpleSchema, simpl-schema, and Ajv. We recently completed both DSTU2 and STU3 of the Argonaut resources, and should have an update for that package by the end of the year.
https://www.npmjs.com/package/fhir-schemas
Mohammad Jafari (Nov 15 2018 at 00:52):
@Abigail Watson curious: how do you keep this project synched with the evolving specs? Is it a manual process?
Abbie Watson (Nov 15 2018 at 00:54):
For the time being yes. A few years from now, I thoroughly expect it to all get automated. But until the patterns are sorted out for what is actually needed, it's all manual.
Grahame Grieve (Dec 30 2018 at 20:52):
Question about JSON schemas for each resource
Grahame Grieve (Dec 30 2018 at 20:53):
(see https://chat.fhir.org/#narrow/stream/179240-Announcements/subject/R4.20release.20is.20happening)
Grahame Grieve (Dec 30 2018 at 20:54):
The syntax for referring between schemas is not well supported by JSON schema tools. There's question about the relationship between the logical and literal URLs, and I couldn't get it working at all.
Grahame Grieve (Dec 30 2018 at 20:54):
Further, there's no tooling benefit to separating them, since any tools must resolve all of them.
Grahame Grieve (Dec 30 2018 at 20:55):
Given that, and the extreme difficulty that JSON schema represents, We only support a single schema
Lloyd McKenzie (Dec 30 2018 at 21:23):
Meaning that we only provide a schema for the "current" release of FHIR and abandon schemas for prior versions?
Grahame Grieve (Dec 30 2018 at 22:04):
I'm not sure what you mean by abandoning.
Lloyd McKenzie (Dec 30 2018 at 22:35):
The original query was about the disappearance of the JSON schemas at the STU3 endpoint. You said that was intentional - by which I presume the intention is that right now we will only support a JSON schema for R4. And presumably when R5 is published, we'll cease supporting the JSON schema for R4. I'm confused why we need to do that for JSON when we don't do the same for XML.
Grahame Grieve (Dec 30 2018 at 22:38):
you misread it. The query was why R4 is different to R3. Nothing was removed from the published R3
Lloyd McKenzie (Dec 30 2018 at 23:24):
k
John Silva (Dec 31 2018 at 13:00):
@Grahame Grieve - in the past I've had problems with Newtonsoft's JSchema implementation not being able to resolve even the internal #ref's in the FHIR JSON Schema. I'll have to try this again with the R4 JSON to see if it works any better. (I end up manually cutting a specific resource's schema from the 'full schema', e.g. Patient, and then have to also remove the references to the FHIR data types, not ideal but that's what I was able to get working. I guess this suggests that I should revisit that [and report back here what I find].
Grahame Grieve (Jan 01 2019 at 20:41):
sure let me know
anay (Jan 08 2019 at 16:30):
"egvs": [
{
"systemTime": "2018-02-06T09:12:35",
"displayTime": "2018-02-06T01:12:35",
"value": 122,
"realtimeValue": 121,
"smoothedValue": 122,
"status": null,
"trend": "flat",
"trendRate": -0.5
}]
I want to add the above json into an observation , I was able to add "egvs" how do I create extensions to an extension...
Lloyd McKenzie (Jan 08 2019 at 16:40):
The preferred solution to send that information would be to break out all of the elements into separate Observation components (with the possible exception of systemTime which should presumably correspond to Observation.effectiveTime)
anay (Jan 08 2019 at 17:04):
The preferred solution to send that information would be to break out all of the elements into separate Observation components (with the possible exception of systemTime which should presumably correspond to Observation.effectiveTime)
So how Do I create custom properties like trend and trend rate realtime and smoothedvalue ?
Lloyd McKenzie (Jan 08 2019 at 17:53):
They're not custom properties - they're Observation components. You'd have one component for the realtime value, one component for the smoothed value, etc. Each would have a component.code and a component.value
anay (Jan 08 2019 at 20:40):
They're not custom properties - they're Observation components. You'd have one component for the realtime value, one component for the smoothed value, etc. Each would have a component.code and a component.value
I wanted to apply some fixed string values on the trend rate based on the value, do i apply some kind of constraints in this case?
"doubleUp"
+3 ≤ trendRate ≤ +8
"singleUp"
+2 ≤ trendRate < +3
Lloyd McKenzie (Jan 08 2019 at 21:41):
Are you talking about constraining what's allowed to appear in a given Observation? If so, yes you could enforce those sort of constraints with a profile
Grahame Grieve (Jan 12 2019 at 21:02):
@Corey Spears here's a present for you
Grahame Grieve (Jan 12 2019 at 21:02):
Rajesh Garlapati (Jan 12 2019 at 21:04):
Grahame, It seems you sent to wrong person.
Grahame Grieve (Jan 12 2019 at 21:07):
no was public on purpose, though Corey asked for it
Pietro Ghezzi (Feb 05 2019 at 11:55):
@Grahame Grieve where is the definition for the id.schema.json, present in resources like Patient and Observation. Should it be a primitive type or there is also a schema as the identifier? I couldn't find it. Thanks
Grahame Grieve (Feb 05 2019 at 11:56):
I don't know
Grahame Grieve (Feb 07 2019 at 00:09):
it looks like I missed all the primitive types
Grahame Grieve (Feb 07 2019 at 00:10):
I'm overloaded... but the solution is for someone to pull them out of the single json schema
Shinji KOBAYASHI (Nov 18 2019 at 02:22):
I found trivial bugs in JSON Schema of MedicationRequest resource(line 36691-36693 in fhir.schema.json, that fails to mandate "status" and "intent". In the official document, their cardinality is 1..1 and XML schema set minOccurs="1".
Shinji KOBAYASHI (Nov 18 2019 at 02:56):
Two more, the JSON Schema of MessageHeader also fails to mandate "endopoint" in Source and Destination elements. Where should I send pull requests?
https://github.com/skoba/HL7-FHIR-JP-WG/commit/16036ea3906e03619ca77801acd022cae17955d6#diff-c907b9dd199507218860e113d7ee51b4
Shinji KOBAYASHI (Nov 18 2019 at 02:57):
Grahame Grieve (Nov 18 2019 at 08:28):
since this is generated, I'll have to debug this, and since it's generated, whatever pattern it is will be consistent in lots of places
Shinji KOBAYASHI (Nov 18 2019 at 10:07):
I guess so. Cardinality conversion seems loose.
Grahame Grieve (Nov 29 2019 at 12:09):
Actually, @Shinji KOBAYASHI this is deliberate; the way the json format works means that primitive elements can be present in either format - eg. status or _status. So we can't make status required
Chidamber Kumar (Mar 24 2020 at 19:57):
Hi
An observation : The schema downloaded from http://build.fhir.org/fhir.schema.json.zip has 'id' defined in 'definition.coding' while FHIR website doesn't mention any id for 'coding' datatype.
Any thoughts ?
@Grahame Grieve
Michele Mottini (Mar 24 2020 at 19:58):
It's inherited from Element
Chidamber Kumar (Mar 24 2020 at 20:10):
Oh yeah! missed it. Thanks @Michele Mottini :)
Chidamber Kumar (Mar 26 2020 at 11:53):
Hi
I wanted to check if JSON schema also suggests about cardinality of elements ? I could see "type":"array" for some elements but what about elements with cardinality 1..1 ? How those are suggested in the JSON schema ?
Lloyd McKenzie (Mar 26 2020 at 14:20):
If something isn't an array, then it can't repeat...
Chidamber Kumar (Mar 27 2020 at 08:41):
Lloyd McKenzie said:
If something isn't an array, then it can't repeat...
Hi @Lloyd McKenzie If something isn't array, then it could be 0..1 or 1..1 ? How is that distinguished in JSON schema ?
Lloyd McKenzie (Mar 27 2020 at 14:13):
They're declared towards the bottom as required. E.g.
"required": [
"code",
"resourceType"
]
Lloyd McKenzie (Mar 27 2020 at 14:15):
Note that simple types (e.g. Observation.status) can't be enforced this way because there's a choice of having "status" or "_status", so neither are 'required', even though you must have one of the two.
Chidamber Kumar (Mar 28 2020 at 05:49):
Lloyd McKenzie said:
Note that simple types (e.g. Observation.status) can't be enforced this way because there's a choice of having "status" or "_status", so neither are 'required', even though you must have one of the two.
Got it ! @Lloyd McKenzie Thanks
Chidamber Kumar (Apr 06 2020 at 09:08):
Hello guys!
I have been going through the JSON schema downloaded from http://build.fhir.org/fhir.schema.json.zip and noticed that 'Narrative' has only one required key, which is 'div'. While, the html page http://build.fhir.org/narrative.html#Narrative says both 'status' and 'div' are required.
Thoughts ?
Grahame Grieve (Apr 06 2020 at 09:12):
This is very much an FAQ. status is required but may appear in the JSON as either status or _status - there's another section of the JSON schema that requires oneOf these to be present
Chidamber Kumar (Apr 07 2020 at 10:55):
Hi @Grahame Grieve
I could find two sections JSONschema.OneOf and JSONschema.definitions.ResourceList.oneOf. None of these specifies that status may appear as either status or _status. Could you please point me to the right section?
Chidamber Kumar (Apr 11 2020 at 09:01):
Hi
I have been going through JSON schema and noticed that Patient.deceasedDateTime has 'type:string'.
Shouldn't there be a child here as "$ref": "#/definitions/dateTime" ? after all , it's primitive data type is dateTime .
Chidamber Kumar (Apr 11 2020 at 09:22):
(deleted)
Lloyd McKenzie (Apr 11 2020 at 14:23):
@Grahame Grieve
T M (Jun 02 2021 at 20:48):
Hi everyone. We're using JSON Schema for FHIR. It works well on validation, but the error messages are very hard to figure out. It looks like the JSON Schema was written with draft 6 and Swagger in mind. The issue is the giant oneOf, which attempts to validate against all resource types. Swagger supports discriminator, but JSON Schema does not. JSON Schema draft 7 supports if/then/else, where maybe we could map subschemas by resource type and get much better error messages, and possibly faster performance. Has anyone tried this?
Lloyd McKenzie (Jun 02 2021 at 21:25):
@Grahame Grieve
Grahame Grieve (Jun 02 2021 at 23:51):
I haven't truid if/then/else and I'm interested in how it would be different if someone wants to try it. but don't think we'll ever get a really good outcome. and Swagger is more important than json schema 7, I think
T M (Jun 04 2021 at 21:46):
Is there a good way to validate with swagger in Python? We just want to see if we have valid FHIR or not in a dictionary. We're trying to work entirely with Python. We use the full Java validator as a manual step here and there, but the jsonschema is extremely useful.
Eric Haas (Jun 05 2021 at 00:15):
check out the fhir.resources library with fastAPI it uses pydantic and is natural fit for fastAPI and swagger.
T M (Jun 08 2021 at 23:30):
Those error messages are a lot cleaner. It doesn't seem to validate types, though. Like I can change a 3 to a "3" and it doesn't complain. Will keep looking at it. Thank you, Eric.
Last updated: Apr 12 2022 at 19:14 UTC
