Stream: Orders and Observation WG
Topic: Device-issued Observations
Gino Canessa (Sep 23 2020 at 23:05):
Apologies for the cross-post, but this didn't seem to get any traction in implementers, and figured this would be the next place to ask. Currently, Device is not allowed in Observation.performer. Given that, how would you record "AI Algorithm X" performed an Observation? (e.g., AI analysis of an Imaging Study)?
Lloyd McKenzie (Sep 24 2020 at 01:35):
I'd be in favor of a change request to extend Observation.performer. There are definitely Devices that perform (and have sole responsibility for) certain Observations. I don't think the fear of confusion between Observation.performer and Observation.device is a valid reason to omit it.
Gino Canessa (Sep 24 2020 at 15:36):
... or we could use FHIR#22147 . Do I need to open another tracker?
Jose Costa Teixeira (Sep 24 2020 at 19:41):
J#22147 is still open, so I think we should use the same also to avoid repeating the discussion.
Eric Haas (Sep 29 2020 at 16:39):
I think it is a terrible idea that will lead to unending confusion where to stick the device.
Jose Costa Teixeira (Sep 29 2020 at 17:53):
I don't understand what is the problem. If my watch tells me my heart rate, or that I don't seleep enough, what is the confusion / alternative?
Lloyd McKenzie (Sep 29 2020 at 18:07):
It's two different devices. "What made the decision" vs. "what was used". You can't stick both in the same element because they mean different things. Implementers get confused about lots of things. Our job is to create the needed elements and make it as obvious as we can how to do the right thing. We don't "not" include needed capabilities because we're concerned about confusion.
Hans Buitendijk (Sep 29 2020 at 18:12):
If the intent of device as a performer is to indicate what/who made the decision, I can understand that distinction. Problem will remain where to shift from was used into performed. Do you see that if a machine deducted a result (whether simple lab result, or some AI algorithm) that that observation is performed by a device and then a human indicates whether it can be used? whether they acted on it? Is the lab tech the performer, or "just" the operator?
Lloyd McKenzie (Sep 29 2020 at 18:22):
There are multiple workflows. In general if a human decides whether the value exposed, then human has responsibility. However, if the device has sole responsibility for both determining the value and deciding the Observation is going to be added to the patient record, then that's definitely the performer.
Gino Canessa (Sep 29 2020 at 18:22):
I think it lowers confusion. If a practitioner uses a device, the practitioner is the performer and the device is just a device used. The change is to allow the case when there is no practitioner directly involved in the workflow - e.g., organization ABC uses an AI system to perform initial (or secondary, etc.) observations. The practitioner has no input into what is run, it's just part of the workflow.
You could make the argument that some sort of organization or person should be listed as the 'one who decided this step in the workflow', but I don't think it is reasonable (or likely) that it would be filled out (read: most people have no idea who made the decision).
Andrea Pitkus, PhD, MLS(ASCP)CM, CSM (Dec 22 2021 at 03:42):
and I don't see any mention in this thread of autoverification rules where most LISs indicate which (especially when multiple) instrument "reported" an autoverified result. Knowing which instrument is also a regulatory requirement (for traceback/root cause analysis). How is this requirement met?
Eric Haas (Dec 22 2021 at 18:09):
Gino Canessa said:
I think it lowers confusion. If a practitioner uses a device, the practitioner is the performer and the device is just a device used. The change is to allow the case when there is no practitioner directly involved in the workflow - e.g., organization ABC uses an AI system to perform initial (or secondary, etc.) observations. The practitioner has no input into what is run, it's just part of the workflow.
You could make the argument that some sort of organization or person should be listed as the 'one who decided this step in the workflow', but I don't think it is reasonable (or likely) that it would be filled out (read: most people have no idea who made the decision).
lawyers care (a lot) and therefore the providers care
John Moehrke (Dec 22 2021 at 18:36):
lawyers only care when it was a bad/wrong decision
Alan Pinder (Dec 22 2021 at 18:59):
Gino Canessa said:
Apologies for the cross-post, but this didn't seem to get any traction in implementers, and figured this would be the next place to ask. Currently,
Deviceis not allowed inObservation.performer. Given that, how would you record "AI Algorithm X" performed an Observation? (e.g., AI analysis of an Imaging Study)?
Would the Device route allow us to capture eg AI Algorithm Version?
John Moehrke (Dec 22 2021 at 19:04):
I have proposed Provenance be used to express the AI algorithm, and possibly the inputs (Provenance.entity) it was given.
Lloyd McKenzie (Dec 22 2021 at 22:47):
Devices absolutely make decisions and perform things without human supervision. They do that under the auspices of an Organization, but you definitely want to be able to list the Device as the performer.
Eric Haas (Dec 23 2021 at 02:16):
Lloyd, I think you are is being too (absolutely and definitely) dogmatic about this. IMO The device element is intended for all of this.
Richard Townley-O'Neill (Dec 23 2021 at 02:21):
IMO If the device element did all of this, it would need to be a backbone element with a role and a reference, so that the different roles played by the device can be expressed.
Lloyd McKenzie (Dec 23 2021 at 02:42):
Performer and 'device' are different things. If a Device is the thing that is responsible for making the Observation without supervision, then it's the performer - just as a device can be the author of a condition or performer of a procedure or author of a QuestionnaireResponse. Observation shouldn't handle the situation any differently. Doing so creates discrepancy and confusion.
Andrea Pitkus, PhD, MLS(ASCP)CM, CSM (Dec 23 2021 at 05:13):
@Gino Canessa actually for lab data, there's regulatory requirements to know exactly which device (unique id/serial number if multiples of the same model are used in a lab), a particular result originated from. It's helpful for troubleshooting, root cause analysis and FDA I think requires it for any investigations that may lead to recalls, etc. For COVID reporting, HHS wants the UDIs for the analyzer (device) and test kits (device) used to generate the lab results.
Not quite the same as an algorithm/AI produced result, but since many "algorithms" are essentially calculations, calculations need to be validated according to lab regulatory requirements too (usually before they go into use and when inspected.)
Last updated: Apr 12 2022 at 19:14 UTC
