Stream: implementers
Topic: Thumbnail Images
Rowan Foster (Oct 31 2016 at 22:59):
Hi, I was wondering if it was reasonable to return thumbnail versions of images when requested with _summary=true ?
For example if I have images, amongst other things, stored in the Observation resource and I want to list all recent Observations in my UI. I don't want to download all the images as on a mobile client this may be slow & expensive, but I do want to give some indication as to what is contained in the Observation ie a thumbnail. So is it reasonable to return a thumbnail version of the image if _summary is specified, then if the user clicks on the image I can request that Observation by id and get the full image data?
James Butler (Nov 01 2016 at 00:38):
It might be surprising for the same Observation to have different value data at different times - how about a thumbnail extension?
Grahame Grieve (Nov 01 2016 at 06:04):
yes, an extension would be preferred; I suggest that you define an explicit parameter asking for this behaviour instead of defaulting to it
Jenni Syed (Nov 01 2016 at 14:43):
What if the server, in non-summary, returned all sizes, and in summary, returned only the smallest? (thumbnail)
Jenni Syed (Nov 01 2016 at 14:44):
Though this would depend on where you put the attachment
Elliot Silver (Nov 01 2016 at 17:03):
@Rowan Foster If you're trying to store images in observations, I'd suggest you lobby for Media as an observation value type.
I don't like returning any imaging in a _summary view because _summary should only be the key information and I don't think all clients would agree that a thumbnail is key info. (And what is the correct thumbnail size and quality for all clients?) Asking for a thumbnail to be rendered on the fly is potentially expanding the role of the server, and, since it could be rendering (instead of just storing) diagnostically significant information, may cause the server to fall under a different FDA classification. I don't think this should be an expected behavior of any server.
How are you determining which attachments to create thumbnails for? Only image/jpg mimetype? image/png? what about application/pdf, application/word, or text/xml?
I'd suggest that you define an operation on Observation (or on Media) that lets you specify the rendering parameters requested. (Unfortunately, I don't think you can specify an operation on a datatype, which could then be available for all attachments.)
Alternatively, and depending on what these images are, look at using ImagingStudy with a DICOMweb server to handle the image storage and rendering.
Rowan Foster (Nov 01 2016 at 19:13):
Ok, thanks for you assistance
Alex Goel (Jun 20 2017 at 19:39):
Hi, I was wondering how one might get these thumbnail images out of PACS/RIS? I'm trying to figure out what the transaction would look like to embed the image in a DiagnosticReport, Questionnaire, or Observation. Would the radiologist need to flag key images? (http://wiki.ihe.net/index.php/Key_Image_Note & ftp://medical.nema.org/medical/dicom/final/sup59_ft.pdf). In the example on the ImagingStudy page: "Dr. Skann interprets the study on a PACS workstation, and he selects two key image frames to be included in the diagnostic report; this selection is stored back to the PACS as a DICOM Key Object Selection with the title "For Report Attachment", and the PACS makes it available (transcodes it) as a FHIR ImagingManifest resource. Dr. Skann dictates the report using a structured data entry report writing program, including a recommendation for a cardiac catheterization procedure, and signs it. The report writing program formats the report as a CDA document, retrieves the ImagingManifest resource, and inserts the referenced key images into the report." (https://www.hl7.org/fhir/imagingstudy.html) In this example is the PACS assigning some kind of UID to the key images which appears in the ImagingManifest? If that's the case, would the key image be linked to the study in order to retrieve the entire study later in the example? I'm not sure if DICOMWeb supports querying, and if that's the case the assumption would be that the EHR would need all the IDs necessary to get both the key images and the full study. Is that correct?
Elliot Silver (Jun 21 2017 at 22:11):
Hi Alex, a lot to unpack here.
In this example is the PACS assigning some kind of UID to the key images which appears in the ImagingManifest?
All DICOM images have an identifying UID, whether they are flagged or not. The ImagingManifest would list the UID of the flagged imaged.
I'm not sure if DICOMWeb supports querying,
Yes it does, through the QIDO-RS service.
and if that's the case the assumption would be that the EHR would need all the IDs necessary to get both the key images and the full study. Is that correct?
All the information needed to get the key images should be in an ImagingManifest, without querying. The ImagingManifest could link to ImagingStudies which list all the images in the study; or if the ImagingManifest lists the QIDO endpoints, you could query for the remaining images in the study.
Elliot Silver (Jun 21 2017 at 22:19):
If you want to link images to a DiagnosticReport, I see two approaches.
1. DiagnosticReport.imagingStudy can reference an ImagingManifest. You can either create your own ImagingManifest of the images you want to include, or use one that represents a DICOM KOS (key object selection file) or KIN (key image note). An advantage of this approach is that the consumer is free to determine how to render the images (size, crop, format, etc.); a disadvantage of this approach is that the consumer has to determine how to render the images.
2. DiagnosticReport.image references Media. You can put spedific renderings of images in the Media.content.attachment (preferably referenced by URL, not binary inline). The advantage of this approach is that the image presentation can be specified at report composition time, and the consumer just has to invoke the URL.
Anand Mohan Tumuluri (Jun 21 2017 at 23:57):
Both WADO-URI and WADO-RS support the ability to retrieve rendered images as JPEG/PNG, so I am not sure if it is any disadvantage. Ofcourse, the consumer needs to understand single/multi frames and retrieve individual frames etc.
Grahame Grieve (Jun 22 2017 at 01:54):
the disadvantage is around security: if you package the image with the report, the only security to manage is to distribute to the recipient. If the recipient has to dial-back to some base to get the entire report, that's a whole other layer of security to manage.
Grahame Grieve (Jun 22 2017 at 01:55):
right now, in Australia, this security is 'managed' by using GUIDs for the image links, and serving up the image to anyone who has the right GUID
Elliot Silver (Jun 22 2017 at 05:44):
@Anand Mohan Tumuluri The difference I was refering to is that ImagingManifest and ImagingStudy only contain the base URL (well, actually, they reference Endpoints which contain the base URL). Anyone who wants to use an image, builds the image URL from [WADO-RS base]/study/{studyUID}/series/{seriesUID}/instance/{instanceUID} (or equivalent for WADO-URI. Then they tack on the query parameters for rendering if they want. On the other hand, Media contains a complete URL including any rendering parameters the report author wants to add.
This isn't a WADO-URI vs. WADO-RS issue. The one gives full control over the rendering to the consumer; the other gives control to the reporting system.*
(I* suppose a system could strip rendering parmeters off a URL and then reapply the rendering parameters it wanted, but that's another level of effort by the consumer.)
John Moehrke (Jun 22 2017 at 12:44):
Grahame, that is not security. that is obscurity... I don't disagree that security is hard, but.... Are they atleast one-time-use GUIDs? System of audit watching for abuse?
Grahame Grieve (Jun 22 2017 at 13:10):
I hope so. I don't know thought
Alex Goel (Jun 22 2017 at 13:55):
Thanks everyone this was very helpful! The PACS has to assign a UID to the key images for this to work. I think that using ImagingManifest may be the better than media so that the consumer can determine how to render the images. I didn't think of QIDO-RS, and that makes this a little simpler
Last updated: Apr 12 2022 at 19:14 UTC
