Stream: implementers
Topic: Sharing AI Models
Grahame Grieve (Oct 18 2019 at 19:42):
I just watched a presentation here at UT in Houston, a bunch of different teams that shared the work trying to predict Sepsis from a clinical data set provided by Cerner. Good and interesting work. One of the teams asked me about whether there's interest in sharing the model outcomes interoperably. Apparently that would mean sharing weighting values for the input variables, along with some information about pre-processing and which AI algorithmic frameworks are being used.
thoughts?
John Silva (Oct 18 2019 at 21:19):
Some of that might be IP ???
Grahame Grieve (Oct 19 2019 at 11:37):
there's always IP. That would be up to the authors
natus (Oct 19 2019 at 19:22):
sure. however, the model weight would need some deidentication process such differential privacy
Adam Flinton (Oct 21 2019 at 07:54):
Would be interested in the outcomes. We have about 120 million care records & I have been pondering (once we complete the move to aws) the AI/ML implications & indeed how they could be shared out e.g. to the NHS in general etc. or indeed what form/format such AI driven conclusions would look like (e.g. to help drive Decision support systems (DSS)). e.g. how does the mass of data resolve down to "these symptoms > check for these problems".
Bryn Rhodes (Oct 21 2019 at 18:43):
I'm definitely interested in this as well. There's been some discussion about sharing models, but the difficulty I've heard raised is that the models tend to be sensitive to local variation in ways that are difficult to characterize. I'd be interested in taking a look at how we might share the models with FHIR. Any examples we could look at?
Jason Walonoski (Oct 23 2019 at 15:08):
There are lots of standards out there for ML models... for example, popular Python libraries like TensorFlow and Keras have the ability to output trained models as HDF5 (https://en.wikipedia.org/wiki/Hierarchical_Data_Format). Unless we're talking about resources that essentially refer to files like these, I don't think we should be creating resources to actually encode the models.
Abbie Watson (Oct 23 2019 at 15:24):
It would be great to have a resource for TensorFlow trained models. Although my suspicion is that these models are going to be all over the place. TensorFlow is great. But I feel that any such resource should also be able to support something more trivial like a SimCity or Sims or SimHospital model just as well. So my inclination would be to treat the resource as mostly an envelope, similar to the Media or Binary resources. Don't assume too much about the structure of the model right now; let's just provide a resource with some metadata for tracking it, and include the model along in a rider/attachment.
Bryn Rhodes (Oct 23 2019 at 15:38):
The Library resource could be used for this, you could just encode the model in the content element, and would provide authoring/publishing metadata.
Josh Mandel (Oct 23 2019 at 15:44):
There's also https://onnx.ai, as an open format for models with pretty broad support (see "Model Zoo" as an example of a non-healthcare-specific model collection. I'm in agreement with Jay that FHIR shouldn't be defining generic infrastructure in this space, but should look at integrating with well-supported existing efforts.
Grahame Grieve (Oct 23 2019 at 17:34):
:+1: Sounds like we should note this as a valid use of the Library resource, and maybe provide an example
Bryn Rhodes (Oct 23 2019 at 17:43):
Submitted GF#25046
John Moehrke (Oct 23 2019 at 18:24):
or use DocumentReference for models prior to them being worthy of Library.
Last updated: Apr 12 2022 at 19:14 UTC
