Stream: data extraction services
Topic: Expressing confidence
Simone Heckmann (Nov 15 2018 at 12:54):
For further discussion: here's the extension that has been introduced to qualify Patient matches:
http://hl7.org/fhir/extension-match-grade.html
...is that something we can reuse for our purposes?
Simone Heckmann (Nov 15 2018 at 12:55):
The ValueSet consists of certain|probable|possible|certainly-not
Philipp Daumke (Nov 15 2018 at 13:55):
To add on that, in MachineLearning based applications, you usually have a "confidence" with a value range of [0..1].
Simone Heckmann (Nov 15 2018 at 13:59):
Yes, I have been thinking that it doesn't make much sense to create a Condition with a confidence of "certainly-not" from a piece of text that indicates that a Patient doesn't have a particular diagnosis. You'd much rather create a "refuted" Condition with a confidentiality of "0.8"
Morten Ernebjerg (Nov 16 2018 at 10:46):
A thought on using numerical scores for confidence: Just in classical statistics, we've got a number of different ways of indicating confidence: p-values (where smaller is more confident), odds-ratios (which can be smaller or larger than 1), confidence intervals etc. So I would guess that (1) it would be hard to force people to use one fixed numerical scale to indicate confidence, and conversely (2) it would be very hard for people to evaluate what a given standardized score actually means in a given case (e.g. how was this value produced or mapped from another kind of score?). So if some sort of numerical indicator of confidence was added, my feeling is that it should always be combined with an indication (encoded or otherwise) of what that number represents (e.g. a p-value from a specific kind of test) - sort of like value and unit go together in valueQuantity. If it is an output of a complicated AI-pipeline, that indication would presumably be a reference to the software that outputted it. Independently of that, a ValueSetfor indicating an overall conclusion could of course still be useful.
Simone Heckmann (Nov 20 2018 at 15:30):
For the purpose of interoperability, I think it’s important to find a simple measure that all parties can agree on and map to. Also, it should be a human readable/understandable representation, so a suitable ValueSet with 3-5 codes would be ideal, IMO.
John Moehrke (Nov 21 2018 at 17:10):
somewhat related. IHE has produced a set of profiles that work together. Starting with MHD that provides access to an HIE that is document centric, yet recognizing that most consumer apps don't want to consume documents. So the mXDE profile orchistrates MHD with QEDm. Where QEDm is a thin profile of the core FHIR resources in query mode (e.g. Query on Observation, Medication, etc...). The orchistration includes the specification of Provenance to link the various FHIR Resources to the documents from which that data came from. There is no confidence identified, except that one can determine for any one Observation, that it has been found in N documents. This addresses only how often that thing has been observed, not the confidence specifically. There is however a well defined profile on Provenance for communicating this, and communicating the 'agent' that did the extraction. see https://wiki.ihe.net/index.php/Mobile_Cross-Enterprise_Document_Data_Element_Extraction
Last updated: Apr 12 2022 at 19:14 UTC
