Stream: implementers
Topic: Media & Imaging Study Resource
Grahame Grieve (Nov 26 2016 at 18:19):
While doing another task, I came across a rather incoherent paragraph on the media resource:
The Media resource may contain medical images in a DICOM format. While such images may also be accessible through an ImagingStudy resource, the Media resource enables "ready for presentation" access to a specific image. Such images would preferentially be made available in a FHIR ecosystem by the Media.content.url providing a reference to a WADO-RS service to access the image. That WADO-RS service may include rendering the image with annotations and display parameters from an associated DICOM presentation state. Although the Media resource is allowed to contain images collected by a DICOM based system, DICOM images would preferentially be made available in a FHIR ecosystem by provision of a resource with references to a WADO-RS server.
I have rewritten it to:
The Media resource is able to contain medical images in a DICOM format. These images may also be made accessible through an ImagingStudy resource, which provides a direct reference to the image to a WADO-RS server.
For such images, the WADO-RS framework is a preferred method for representing the images - the WADO-RS service may include rendering the image with annotations and display parameters from an associated DICOM presentation state, for instance.
On the other hand, the media resource allows for a robust transfer of an image across boundaries where the WADO-RS service is not available. For this reason, medical images can also be represented in a Media resource, but the Media.content.url should provide a reference to a source WADO-RS service for the image.
but this has subtle differences; @Brad Genereaux @John Moehrke @Elliot Silver please consider my revision
John Moehrke (Nov 27 2016 at 00:25):
I think we should not be suggesting that Media can hold a DICOM image. I recall that Media specifically said it was for images not managed in DICOM form. That would seem to be a more consistent differentiation.
Grahame Grieve (Nov 27 2016 at 02:52):
well, but it can, unless we specificially say it can't. Which is also the same as saying that an image cannot contain a dicom image. Which seems to me to be not what we want
Elliot Silver (Nov 27 2016 at 02:55):
We could do something simple like "For DICOM images, see the ImagingStudy and ImagingManifest resources." That directs people away from Media, but doesn't prohibit it.
Elliot Silver (Nov 27 2016 at 03:01):
The two use cases where I see Media interacting with DICOM images are (a) a URL for a rendered WADO-RS/WADO-URI image, and (b) the need to store a file that the patient brought in to a PCP's office, or similar where they don't have a PACS. In the first case, it is essentially no longer a DICOM image, just a URL to a JPEG (or whatever rendered format), so it seems reasonable to use Media. In the second case, I'm not sure what the correct approach is.
Grahame Grieve (Nov 27 2016 at 05:04):
the 2 most likely reasons for me are:
- I need to give the patient a copy of a DICOM image from a FHIR interface, and I have no knowledge about the availability of a WADO-RS Server to the patient
- I need to stick a DICOM image in a document.
Grahame Grieve (Nov 27 2016 at 05:04):
which does raise the idea of having a connectathon around shared smart-on-fhir for wado-rs?
Elliot Silver (Nov 27 2016 at 14:26):
Yes, @Brad Genereaux and I have already had discussion about an appropriate forum for an imaging-focused FHIR connectathon.
John Moehrke (Nov 28 2016 at 13:56):
@Grahame Grieve I think your use-case is an image from a DICOM Object... right? That is the mime type of image/jpg. I am differentiating a simple JPG that is a representational image, from a DICOM object. Or do you mean for Media to include a DICOM object with mime-type of application/dicom? I think Elliot is trying to cut along this axis as well.
Grahame Grieve (Nov 28 2016 at 18:42):
I'm thinking of the second - keeping the format as application/dicom, though I'm not sure that is matters functionally? Say we said, 'can't put application/dicom in meda' --- we imposed a format transform that might be a little lossy, but did we make any meaningful difference?
Brad Genereaux (Dec 01 2016 at 20:53):
regarding "smart-on-FHIR for WADO-RS" @Grahame Grieve - this is a concept I've been thinking about during RSNA this week - like a "smart-on-DICOMweb" for embedding/interfacing applications into PACS applications. The general concept is now starting to gain traction through the re-activation of DICOM WG-23 as of this week.
Grahame Grieve (Dec 01 2016 at 20:53):
great. Any way we can help that gain traction?
Brad Genereaux (Dec 01 2016 at 20:55):
Also - after discussions at FHIR DevDays - I've taken up the cause of adding a thumbnail API to DICOMweb. This could reduce the need of using Media - or at the very least, evolve how it is used. The proposal for the addition of this API will be taken to the next DICOM standards committee meeting.
Grahame Grieve (Dec 01 2016 at 20:59):
media would be a FHIR specific redistribution for the thumbnail (in DICOM context)
Brad Genereaux (Dec 01 2016 at 21:00):
@Grahame Grieve Definitely. The first step of the committee is to generate use cases / scenarios. I've got a few already lined up - and other committee members are working on this too. The first t-con of this re-activated WG is kicking off I think Jan 6. Once I'm back home from Chicago and have some time to decompress, I'm going to start reviewing some of the SMART-on-FHIR stuff and line up how PACS and DICOM fit. Maybe we should set some time up in January to discuss.
Grahame Grieve (Dec 01 2016 at 21:07):
sure
Brad Genereaux (Dec 01 2016 at 21:08):
Yup, understood. Media contains a content, content is an Attachment that can contain a URL, and the URL could be the thumbnail API URL reference. There's lots of reasons why this isn't perfectly locked up (thumbnail API doesn't exist yet; can't assume that the DICOMweb URL is on the same network segment as the FHIR server; maybe what is desired is "not the thumbnail the PACS determines is appropriate, but rather some other desired representation"). Including a binary (instead of a URL) is certainly not a bad thing, but does create different questions on ensuring that thumbnail remains correct (i.e., if the series the image is in is marked "rejected for quality reasons"), that someone wipes out the binary attachment. That might be a "bridge too far" in keeping things in sync for some implementations.
Elliot Silver (Dec 01 2016 at 21:22):
@Brad Genereaux , @Grahame Grieve , there are three issues here. (Well, actually there are more.) (1) Following DevDays, Brad has proposed a new DICOMweb thumbnail API to allow smart selection of thumbnail images for studies. (2) Use of Media binary content for DICOM imaging, regardless of whether that's a thumbnail, or any other object. (3) Discussion of a SMART-on-FHIR-like framework for imaging use (e.g., PACS call out to a CAD system, dictation system, rendering system, etc.)
The last item was Brad's proposal to discuss in January, I think. I'd like to be involved too.
The first item will likely make its way into ImagingStudy, but may be a standard extension. A report could also insert the thumbnail as the href image to lauch a viewer for the study/image. I expect thumbnails will be a useful addition to DICOMweb, but won't raise structural issues in FHIR.
For the middle item, I don't think we've reached consensus on use of application/dicom content in Media resources, either by URL reference or by binary inclusion.
Grahame Grieve (Dec 02 2016 at 03:30):
well, we have some agreement on the middle item. How about this:
In general, it's better to refer to the source image directly. And if you're doing that, there's not a driving need for the Media resource. And in general, if you have a DICOM image, you have a PACS system to refer to.
Possible times when it's appropriate to put a DICOM image in a Media:
- you need to distribute the image with other FHIR content (e.g. in a document)
- you don't have a PACS system, just a single source providing DICOM content
- you need to do other kinds of workflow management inside FHIR and you don't have enough information for an ImagingStudy
but realistically, the only one of these that is mainstream is the first case - documents. Further, in general, if you have a DICOM image, and you're constructing a Media resource, our recommendation would be to convert it to a PNG or JPEG (or MPEG) because that's much easier to present across all platforms, and if you care about quality or metadata, you shouldn't be using Media
John Moehrke (Dec 02 2016 at 15:26):
so when you have a use-case for a simple ready-to-display representative image, from a DICOM object; then use Media for the representative image? How is it to refer to the ImagingStudy from which it comes? Or how is the ImagingStudy to refer to various representative ready-to-display images? Seems useful, logical, and achievable. The problem I was faced with when I proposed something similar is that for many DICOM objects there is no such thing as a representative ready-to-display image; or so I was told. Second, I was told that if there was then you would use DICOM web methods to get it, not FHIR methods. --- Ready... fight!
Grahame Grieve (Dec 02 2016 at 19:16):
I'm suggesting that if you have a use case for ready to display, you can use Media, but an image format of application/dicom is probably less good than something like image/png or image/jpg
John Moehrke (Dec 02 2016 at 19:22):
I agree... but others seem not ...
Grahame Grieve (Dec 02 2016 at 20:10):
who does not agree?
Elliot Silver (Dec 05 2016 at 20:21):
I think DICOM may be addressing the "representative ready-to-display image" issue with the new thumbnail proposal. One of the things that was discussed is that for non-image objects, an icon-style thumbnail (eg. a page icon for a DICOM report) might be a reasonable rendition.
John raises good points about the linking of a ready-to-display image with the original applicaiton/dicom content and vice-versa. I'm not sure how that should be addressed (standard extensions?)
I think we're converging on a statement like "DICOM objects are preferrably, and most flexibly, shared using an image archive (e.g., a PACS) and the ImagingStudy resource. In cases where this is not feasible, to share a relatively small number of DICOM objects, they may be communicated in a Media resource. As an alternative to the application/dicom media type, consider a ready-to-display format, such as image/png, since many systems are unable to render DICOM files."
We have an II-WG meeting this week, I'll add this discussion to the agenda.
Grahame Grieve (Dec 05 2016 at 21:06):
that language works for me. Note that a media resource can contain both the pixel data in whatever format, and a reference to the source. But the source reference is a reference to exactly the same data. There might be an argument for a 'source' reference that is a reference to the end point from which the data was originally sourced - that's a different thing
Brad Genereaux (Dec 06 2016 at 14:08):
I'm interested in how this will relate to the work being done in the XCA world (relating to a presentation at SIIM Wisconsin). There, it was described how to share the existence of studies across enterprises, without directly exposing DICOM / PACS back-ends to foreign destinations - but the solution was somewhat unknown. Using the media as payload where access to underlying image systems (re: network primarily, authentication/authorization secondarily) where it is more challenging - that might be a use case.
John Moehrke (Dec 06 2016 at 16:48):
I am getting multiple requests for a 'ready to display' representative image. Specifically as Grahame has outlined, in a format that is disconnected from needs for special software like a DICOM viewer or PACS web portal. I recognize this is potentially dangerous ground related to PACS ownership, and medical effectiveness. So, there is need. I think we might need to recognize the need, and inform our audience that the solution is in process of being created. -- Seem s this should be explained in Media and ImagingStudy; but also seems needs to be elevated to a 'module' page (FHIR spec front-page breakdown).
Grahame Grieve (Dec 06 2016 at 19:26):
y. Ours is not to tell people how to do their medicine; just note the potential issues with leaving the DICOM cacoon, and let them choose. Some will choose badly, but some will choose well
Brad Genereaux (Dec 06 2016 at 23:08):
I see it quite simply as - the Media resource can include the URL to the soon-to-be-proposed DICOMweb thumbnail API call, and include the binary representative image indicated by that thumbnail call. That thumbnail URL call is disconnected from needs for special software etc.
It's a bit of a tricky place to navigate "PACS ownership", but I think people would figure that out quickly if they didn't already know, wouldn't they? I would maybe suggest in text "attachment" isn't a wise place to stuff all of the binaries of a 10000-slice CT, maybe only good for "ready to display representative images".
Grahame Grieve (Dec 06 2016 at 23:21):
y
John Moehrke (Dec 07 2016 at 13:42):
I have been using the term ready-to-display, for something of reasonable quality for patient or clinician. That is something screen-sized. Your use of thumbnail seems clearly to not be the same thing. A thumbnail seems to me to be something, although ready to display, is not of any useful size other than to support overlay on a button. Are we talking the same thing?
Brad Genereaux (Dec 07 2016 at 16:10):
I've seen "thumbnails" that are even 200x200 or 300x300 - I guess that's bigger than a typical thumbnail description :) But yes, I think we are talking about the same thing. It's an image used to guide selection; it's not an image I would use directly for primary diagnosis.
John Moehrke (Dec 07 2016 at 18:35):
I don't think that is the same thing. I am not expecting primary diagnosis. I expect useful enough for a GP to help understand the radiologists report, and give instructions to Patient. Not sure a thumbnail is useful for that.
Elliot Silver (Dec 07 2016 at 18:53):
I think the introduction of "thumbnail" in this discussion is muddying things. The idea, as I understand it, is to simplify the selection and rendering of a "representative" image for the study (or series or instance). Although it may be used in the context of Media, I don't think that's the key issue.
The key issue, again as I understand it, is whether Media should be used for DICOM instances or ready-to-display renderings of DICOM instances, and if so, whether those should be included or by url (or both)? And, if Media can contain renderings, are links to the DICOM instance (or FHIR ImagingStudy) needed?
On tomorrow's Imaging Integration call (2pm Thurs), we will be discussing this issue. Anyone with some thoughts on the issue is welcome to join.
Grahame Grieve (Dec 07 2016 at 19:05):
I can't be there. My comments:
- "should" is enough here: we recommend that you should use a simpler image format in Media but we should not make it a SHALL NOT use DICOM unless we're confident that we've enumerated all the possible use cases and know that there's no reason
- I agree that 'thumbnail' is confusing. For must people, a thumbnail = icon
Hui Liu (Dec 07 2016 at 22:14):
I got a problem that I have some measurement data ( like Echo measurements), that can potentially have multiple values associated with it. In other words, you have a single measurement, but multiple values associated.
Is there a standard way to represent multiple values for a single measurement as a Observation? If so, what is the best way?
I notice that under observation, you can have multiple components, should I put my LOINC code for my measurement just at observation level and put each value at component level? Or I have to use extensions?
Thanks!
Grahame Grieve (Dec 08 2016 at 00:43):
can you be more specific about the multiple data? and have you looked SampledData?
Lin Zhang (Dec 08 2016 at 11:07):
Could you provide some example(s)?
Lin Zhang (Dec 08 2016 at 11:07):
@Hui Liu
Hui Liu (Dec 08 2016 at 13:54):
@Lin Zhang ,for example, for an echo study, we could have multiple values entered for ejection fraction. Currently in our system, we only have a single code for ejection fraction. So ideally, we hope that we can send these values together, so that user can query with a single key for ejection fraction and they can decide later which value to keep or keep them all
Stefan Lang (Dec 08 2016 at 14:49):
How about Observation.component?
Or a DiagnosticReport with multiple results?
Depending on what fits best in your use case.
Hui Liu (Dec 08 2016 at 15:27):
@Stefan Lang , so is it a standard way to use component in such case? the way I can think of is to put all values as components, and set value as absent at observation level, put LOINC code for my measurement at observation level, and put some pseudo-general code for components. But it sounds hacky, I am not sure if there is a pseudo code i can use..
Stefan Lang (Dec 08 2016 at 15:39):
Basically, I would consider a set of components as something that leads to a final result which you at some point in time would put into Observation.value*
When you say, one is kept as the final value, and later in the workflow put into Observation.value* then this might be your choice.
The components in your case would all have the same LOINC code as the main Observation. No pseudo code needed.
Stefan Lang (Dec 08 2016 at 15:42):
On the other hand, if all the single ejection fraction measurements are equal and a practitioner at some point in time just decides, which are "good" and which are "bad", then I'd go with DiagnosticReport
Hui Liu (Dec 08 2016 at 16:01):
@Stefan Lang , OK! I would prefer a single code but on https://www.hl7.org/fhir/observation.html, it says "Component code SHALL not be same as observation code" ...
I am not familiar with DiagnosticReport , I think we use it to pack all the data including measurements. Not sure if we can use it to just send measurement data though..
Stefan Lang (Dec 08 2016 at 16:15):
Ah, you're right. So, if the available LOINC codes for ejection fraction don't fit your needs for the initial measurements (i.e. components), you would indeed need to look for/define a ValueSet for that.
Stefan Lang (Dec 08 2016 at 16:19):
But probably DiagnosticReport would fit better.
You could define one DiagnosticReport to report the results of the actual ejection fraction measurements and another to put the "good" ejection fraction data in, along with whatever other Observations you might have.
Grahame Grieve (Dec 08 2016 at 19:11):
I think that these are components of a single observation. They're all made at once, right? what is the applicable LOINC code?
Hui Liu (Dec 08 2016 at 19:36):
@Grahame Grieve , yeah, they are components of the same measurement. Actually I am not sure if I can find a applicable generic LOINC code for such use. ..
Grahame Grieve (Dec 08 2016 at 19:37):
can you find a LOINC code for any related usage?
Hui Liu (Dec 08 2016 at 19:41):
@Grahame Grieve , or maybe just some generic code, according to specifications seems like it doesn't have to be LOINC code (and it doesn't have to be meaningful for my use case)
Grahame Grieve (Dec 08 2016 at 19:42):
then you get to make up your own code. But you should ask LOINC for a code
Hui Liu (Dec 08 2016 at 19:47):
@Grahame Grieve , agree. Thanks for the suggestion!
Lin Zhang (Dec 08 2016 at 21:44):
@Hui Liu Are these items the variables used to calculate the Ejection Fraction? Or do they belong to the formula of Ejection Fraction?
Lin Zhang (Dec 08 2016 at 21:55):
Could you please list these example items in order to determine if there are LOINC codes for them?
Stefan Lang (Dec 09 2016 at 09:19):
For completeness: Observation.related is basically a third option for such cases. But maybe not the best choice in this special case.
Hui Liu (Dec 09 2016 at 17:49):
@Lin Zhang , actually I am not sure. My impression is that these should be the final ejection fraction values themselves. And there are measured in different ways. And somehow users just send over all of them to our system without removing duplicates
Elliot Silver (Dec 09 2016 at 17:52):
@Grahame Grieve Where did you find the paragraph that started this thread? I don't see it in Media.
Grahame Grieve (Dec 09 2016 at 19:19):
it was in media, but I replaced it with the revised paragraph after commentary earlier in the thread
Elliot Silver (Dec 09 2016 at 19:36):
What I'm seeing on http://build.fhir.org/media.html doesn't match anything we batted around here.
Grahame Grieve (Dec 09 2016 at 19:37):
it matches our very early discussion - my original proposal with some comments
Elliot Silver (Dec 09 2016 at 19:39):
Oops, yes. right. Thanks.
Grahame Grieve (Dec 09 2016 at 19:39):
we can continue to propose changes....
Grahame Grieve (Dec 09 2016 at 19:40):
I felt it was better to reword the incomprehensible paragraph to something people could decide whether they actually agree to or not.
Grahame Grieve (Dec 09 2016 at 19:40):
for this review
Elliot Silver (Dec 09 2016 at 19:42):
I'm going to send an email out shortly to the II/DICOM WG-20 mailing lists to make sure we get concensus on a proposed change.
Grahame Grieve (Dec 09 2016 at 19:47):
k thx
Elliot Silver (Dec 09 2016 at 20:27):
If I want to record that a Media resource was derived from a particular DICOM instance, do I use Provenance, or should we create an extension to cover that?
Grahame Grieve (Dec 09 2016 at 20:31):
there's a 3rd option - an identiifer with use = official
Grahame Grieve (Dec 09 2016 at 20:32):
but you probably want an endpoint as well as an id?
Elliot Silver (Dec 09 2016 at 20:34):
at least optionally, yes.
Elliot Silver (Dec 09 2016 at 20:36):
I can see us wanting to say this rendering is derived from the DICOM instance identified by study, series, and instance UIDs, and also saying that instance can be found at a particular endpoint, or is described in an ImagingStudy, or is stuffed in another Media instance.
Grahame Grieve (Dec 09 2016 at 20:39):
you want to provide all those ids? isn't the instance enough?
Elliot Silver (Dec 09 2016 at 20:43):
well, although a DICOM object is theoretically identifyable by instance UID alone, traditionally, we've provided the study and series as context. Historically at least, some PACS couldn't respond to a request for an instance by instance UID alone, and needed the study to scope the query.
Elliot Silver (Dec 09 2016 at 20:47):
Or are you questioning the need for endpoint, ImagingStudy and Media on top of the ids?
Grahame Grieve (Dec 09 2016 at 20:47):
i'm just trying to understand what is required.
Last updated: Apr 12 2022 at 19:14 UTC
