FHIR Chat · R Unconference · analytics on FHIR

Stream: analytics on FHIR

Topic: R Unconference

Grahame Grieve (Oct 26 2017 at 04:45):

I am at the Australian R Unconference today, working on the R client for FHIR

Grahame Grieve (Oct 26 2017 at 04:45):

It was written by @Sander Laverman - thanks.

Grahame Grieve (Oct 26 2017 at 04:45):

I've beefed it up and I know it pretty well now

Grahame Grieve (Oct 26 2017 at 04:46):

I've just reviewed it with one of the leads for R for Australia, and the clear message is that it's basically useless without methods for restructuring the data to make it square

Grahame Grieve (Oct 26 2017 at 04:47):

restructuring the data.... needs some new ideas....

Grahame Grieve (Oct 26 2017 at 04:48):

I'm wondering about beefing up graphQL to allow for flattening, or exploring some OData conversion specification...

Grahame Grieve (Oct 26 2017 at 11:10):

I followed this:

Grahame Grieve (Oct 26 2017 at 11:10):

https://github.com/graphcool/framework/issues/268

Grahame Grieve (Oct 26 2017 at 22:17):

Data Reorganization Directives

GraphQL is a very effectively language for navigating a graph and selecting subset of information from it. However for some uses, the physical structure of the result set is important. This is most relevant when extracting data for statistical analysis in languages such as R. In order to facilitate these kind of uses, FHIR servers should consider supporting the following directives that allow implementers to flatten the return graph for easier analysis

@flatten

This directive indicates the the field to which it is attached is not actually produced in the output graph. Instead, it's children will be processed and added to the output graph as specified in it's place.

Notes:
If @flatten is used on an element with repeating cardinality, then by default, all the children will become lists
When using @flatten, all the collated children must have the same FHIR type. The server SHALL return an error if they don't

@singleton

This directive indicates that an field collates to a single node, not a list. It is only used in association with fields on which a parent has @flatten, and overrides the impact of flattening in making it a list. The server SHALL return an error if there is more than on value when flattening

@first

This is a shortcut for a FHIR path filter [$index = 0] and indicates to only take the first match of the elements. Note that this only applies to the immediate context of the field in the source graph, not to the output graph

@slice(fhirpath)

This indicates that in the output graph, each element in the source will have "." and the result of the FHIRPath as a string appended to the specified name. This slices a list up into multiple single values. For example

name @slice($index) @flatten {given @first @singleton family}

For a resource that has 2 names will result in the output

{
"Given.1" : "first name, first given",
"Family.1" : "first name family name",
"Given.2" : "second name, first given",
"Family.2" : "second name family name"
}

Other uses might be e.g. Telecom @slice(use) to generate telecom.home for instance.

Notes:
In general, the intent of @slice is to break a list into multiple singletons. However servers do not treat the outputs are singletons unless this is explicitly specified using @singleton

Grahame Grieve (Oct 27 2017 at 08:35):

http://www.healthintersections.com.au/?p=2740 - for anyone interested in graphQL or data analysis

Anthony Master (Jul 20 2021 at 15:51):

I am having a hard time wrapping my head around how the flatten directive is not breaking the GraphQL official specification. It takes it from a "get what you ask for" to "ask and see what you get" kind of shift. Doesn't returning fields not requested break the GraphQL specification? And then what about returning fields that are not part of the schema? That would definitely break spec. And this adds even more logistical problems considering even the basic example:

For an example, take this graphQL, and apply it to the patient example:

{
  identifier { system value }
  active
  name { text given family }
}
This will give the output:

{
  "identifier": [{
      "system": "urn:oid:1.2.36.146.595.217.0.1",
      "value": "12345"
  }],
  "active": true,
  "name": [{
    "given": ["Peter","James"],
    "family": "Chalmers"
  },{
    "given": ["Jim"]
  },{
    "given": ["Peter","James"],
    "family": "Windsor"
  }]
}
Adding the @flatten directive changes the output:

{
  identifier @flatten { system value }
  active
  name @flatten { text given family }
}
This has the output:

{
  "system":["urn:oid:1.2.36.146.595.217.0.1"],
  "value":["12345"],
  "active":true,
  "given":["Peter","James","Jim","Peter","James"],
  "family":["Chalmers","Windsor"]
}

I understand that by using @first as exemplified we can remove the second given name for each patient allowing the response to be better understood as 3 patients instead of 5, but what about the family name? We see two family names and no null field for the missing value. I would expect the example response to be: ["Chalmers",null,"Windsor"] since the second patient did not have a family name. Without having a null value though, the data becomes ambiguous very quickly and I don't see much good out of the flatten directive if it is even supported anywhere?

https://github.com/Asymmetrik/graphql-fhir seems like the biggest open source project for supporting GraphQL on a Node.js server and it does not support the flatten directive still.

I see this spec is still in a draft and has been for quite some time. Is there any consensus to using GraphQL for FHIR or is this a lost cause? I think it is needed as data is becoming more graph like and would be great if we could see FHIR implementations on services like Dgraph or Neo4j. I'm still a newb so maybe they exist, I just haven't found them yet.

Paul Hellwig (Aug 02 2021 at 16:14):

Hi @Anthony Master couple of thoughts that might help:
Your example patient is actually 1 patient. "Windsor" is his maiden name (see original fhir resource in github).
This shows a problem of the suggested "flatten" directive which seems to lose too much meta information. I wouldn't understand why this was required at all, maybe this was an R thing. Now, as there is the FHIR bulk data api, there shouldn't be an issue with loading data into neo4j for analytics with a little bit of schema config.

Grahame Grieve (Aug 17 2021 at 03:15):

ok it's a fair comment that @flatten and @singleton are problematic with regard to schema.

Grahame Grieve (Aug 17 2021 at 03:16):

And maybe that will mean that they can't be supported, though my implementation does. it's just such a useful thing for a client to save them walking through deep json structures. It's definitely really useful in R, which is orientated towards processing square json structures, but it's useful for any client that is going to process the json returned.

Grahame Grieve (Aug 17 2021 at 03:18):

but your example output looks wrong to me, and would certainly not be preocessible. It's not the same as in the spec.

Grahame Grieve (Aug 17 2021 at 03:18):

@Anthony Master

Anthony Master (Aug 17 2021 at 14:08):

Grahame Grieve said:

ok it's a fair comment that @flatten and @singleton are problematic with regard to schema.

Thank you for the reply. I have accepted a position as an Interface Specialist and start tomorrow. I was mainly just trying to get the lay of the land a little bit and see where things stood. Implementing GraphQL directives seems to be a big grey area with what they are allowed and not allowed to do in regards to the GraphQL specification. I recently had a similar discussion about implementing a directive from another service provider and there was mixed feedback as to what the spec actually means concerning directives. I have a lot of GraphQL, MySQL, JavaScript experience, no experience with R and obviously a rookie with HL7 (both 2.x and FHIR). Looking forward to the challenges ahead.

Last updated: Apr 12 2022 at 19:14 UTC