Stream: implementers
Topic: Server supporting _contained searches
Rik Smithies (Mar 09 2020 at 12:09):
Do any servers support this, and how would the capability statement declare it?
test.fhir.org/r4 appears to ignore _contained, unless I am using it wrongly (there seems no example usage in the spec)
http://test.fhir.org/r4/Provenance?_contained=true&patient.name=rik
finds the same resource, whether or not I use _contained=true (and the self link strips it).
Rik Smithies (Mar 12 2020 at 12:13):
So, maybe no servers implement searching into contained resources
Grahame Grieve (Mar 12 2020 at 12:30):
mine does
Rik Smithies (Mar 12 2020 at 12:43):
ah thanks, good to know. When I tested it as above links it did not seem to look at the _contained parameter, and the self link didn't echo it back, implying it is not supported. Is the URL above correctly formatted?
Rik Smithies (Nov 05 2020 at 11:43):
http://test.fhir.org/r4/ didn't work when I tried it and isn't available for testing currently. Do any other servers support "_contained"? @James Agnew @Michael Hansen @Paul Church
Paul Church (Nov 05 2020 at 15:59):
No for Google, and no for Microsoft based on their docs. Not sure if HAPI supports it but it's not listed in their search docs so I'm guessing no.
Rik Smithies (Nov 05 2020 at 16:28):
Thanks Paul. It's strange that no one supports what seems a fairly important feature and one for which there is no workaround possible.
Paul Church (Nov 05 2020 at 16:34):
Searching inside contained resources is technically tricky. I have never heard of anyone needing this feature.
Rik Smithies (Nov 05 2020 at 16:36):
Well let me be the first ;-)
If you need contained resources - and people do seem to - it doesn't seem unreasonable to be able to search them.
Rik Smithies (Nov 05 2020 at 16:38):
It seems that this basically breaks contained resources.
Paul Church (Nov 05 2020 at 17:13):
I do see some use of contained resources, but not searching on them. Because the contained resource is generally a stub or incomplete, it lacks a lot of the things one would normally search for. What's an example of a search that you need to do?
Rik Smithies (Nov 05 2020 at 17:23):
Contained is not only for stubs as such, it is also used when there is no desire to make a whole endpoint for a resource type,
- there is no independent life cycle - but you still want to use that resource.
Hence you may have a full Medication resource contained in a MedicationRequest. Or a Substance in a Medication. And you would want to be able to search on the codes at least. But basically the search params of that resource type. There are examples of both of those in the build, so it's not completely esoteric.
Paul Church (Nov 05 2020 at 17:26):
But the general purpose fhir servers have an endpoint for every resource type. So our best practices are to split them out into normal resources.
Josh Mandel (Nov 05 2020 at 17:27):
The bullet item in this your message @Rik Smithies -- is this written somewhere in the FHIR spec, or supported by what you see there? The examples in http://build.fhir.org/search.html#containedType are (I'm assuming!) intended to represent a case where the medication has no identifying information other than a code.
Paul Church (Nov 05 2020 at 17:28):
Idle speculation, I wonder if we can replace a lot of use cases for stub contained resources with identifier references.
Josh Mandel (Nov 05 2020 at 17:28):
"No desire to make a whole endpoint" is hard for me to interpret in terms of technical requirements/constraints/rationale, and introduces questions
Rik Smithies (Nov 05 2020 at 17:31):
If you don't need to let people read or write them independently, then no endpoint is needed - less to bother with.
Paul Church (Nov 05 2020 at 17:33):
The Google implementation has one API endpoint called GetResource, it handles every DSTU2, STU3, and R4 resource type. The idea of separate API endpoints is not how we think about the world.
Rik Smithies (Nov 05 2020 at 17:34):
@Paul But where would the identifier reference point to? We want the resource to be embedded and not need its own endpoint, and still to exist in FHIR. An identifier reference is mainly for things that cannot be referenced by a FHIR url.
Rik Smithies (Nov 05 2020 at 17:35):
What I mean is if you only ever use Substance contained in Medication, then you need /myserver/Medication, but you don't need /myserver/Substance "endpoint".
Paul Church (Nov 05 2020 at 17:36):
It wouldn't point anywhere, in the case where the identifier is the only thing the server knows about the entity. I agree that if you know more about the resource, then this idea does not apply.
Rik Smithies (Nov 05 2020 at 17:37):
yes if you only have an identifier, then you don't need contained at all. But that is a different issue.
Rik Smithies (Nov 05 2020 at 17:54):
@Josh Mandel it wasn't meant to emphasised as a bullet but zulip intervened :-) It's not a quote.
But it's just a truism about contained resources. They don't have a lifecycle of their own.
And there are examples of this e.g. http://hl7.org/fhir/medicationadministration0302.xml.html
If you only use that resource type as contained (which I would imagine is common, if you use it as contained at all), then you don't need the machinery of a restful endpoint for that resource type.
You don't need people to write one resource then reference it from the other, or use a transaction. They just only ever use it as if it is an inline datatype.
I'm not suggesting some sort of parallel API endpoint. Only that an implementation may want to not support that resource as an endpoint, but still use it contained. I want to use Task within a resource, to indicate a change request for it. I won't ever use that Task un-contained, so it doesn't need a /Task endpoint.
Lloyd McKenzie (Nov 05 2020 at 19:15):
Normally you'd use chaining for that. You wouldn't need _contained
Vassil Peytchev (Nov 05 2020 at 19:26):
I want to use Task within a resource, to indicate a change request for it.
Oh my... Is this really a good idea?
In general, I think the guidance that contained resources should not be used to "package" things together is very important. In this case that one side doesn't want to expose an endpoint for Task, maybe they can receive a notification of a Task existing on the requester side and act on it?
In the case of prescriptions, I would rather MedicationRequest have a reference to Substance (where the compound medication is described) in addition to code or a Medication resource. Having contained resources almost mandated for certain uses seems odd.
Jean Duteau (Nov 05 2020 at 19:28):
Vassil Peytchev said:
In the case of prescriptions, I would rather MedicationRequest have a reference to Substance (where the compound medication is described) in addition to code or a Medication resource. Having contained resources almost mandated for certain uses seems odd.
No, Medication is where the compound medication is described. And the MedRequest/Dispense/Admin/Statement does have a reference to a Medication resource (CodeableReference). I'm just pointing out that implementers are containing the Medication resource rather than creating them separately.
Lloyd McKenzie (Nov 05 2020 at 19:33):
Containing Medication is fine. Containing 'Task' is more problematic. It'd be pretty unusual for Task to not have an independent identity. "I don't want to maintain an endpoint" isn't a justification for using 'contained'.
Vassil Peytchev (Nov 05 2020 at 19:50):
No, Medication is where the compound medication is described. And the MedRequest/Dispense/Admin/Statement does have a reference to a Medication resource (CodeableReference). I'm just pointing out that implementers are containing the Medication resource rather than creating them separately.
I was trying to point out that the requirement to have a Medication resource to describe compound medications is used as the reasoning behind containing Medication within MedRequest (I assume because there is no single record of the compund medication to be exposed as Medication resource?). If we had a substance backbone element within MedRequest that contained references to or codes of Ingredients to describe the compound medication, there would be no need to contain a Medication resource.
With the addition of the new MedicinalProduct-related resources, think we need to revise the way medications are handled, and hopefully simplify some of the things that seem to get way too complicated.
Rik Smithies (Nov 05 2020 at 20:11):
@Vassil Peytchev yes sometimes a backbone would be good, but sometimes you want the same structure as a resource. If something wants to act like a backbone sometimes and wants to act like a resource at others, that is what contained can do.
Rik Smithies (Nov 05 2020 at 20:24):
But if you can't search on it the use is limited
Rik Smithies (Nov 05 2020 at 21:28):
Excepting any other uses (or misuses), if Medication is going to be contained I can imagine that people will want to be able to search on it.
Vassil Peytchev (Nov 05 2020 at 21:35):
I think Lloyd's point was that contained or not, you can use chaining - did I misunderstand?
Gino Canessa (Nov 05 2020 at 21:38):
Just as an aside - this sounds like it would be a lot of work to implement in a performant way.
If the contained resource is an element attached to its parent, the fields will not have been indexed for searching (and even things like full text index don't help for searches on date ranges, etc.).
On the other hand, you cannot assume that contained fragments can be inserted into any normal tables directly. At the very least, you'll need to modify any references/ids/etc. to restrict them to that particular bundle and make sure all the references link back up.
Paul Church (Nov 05 2020 at 21:40):
I do not expect that a chained search can navigate into a contained resource. Is that something that other implementations support?
Vassil Peytchev (Nov 05 2020 at 21:47):
I probably misunderstood...
Lloyd McKenzie (Nov 05 2020 at 22:41):
A chained search can absolutely navigate into a contained resource
Lloyd McKenzie (Nov 05 2020 at 22:42):
That's how they're typically accessed when searching. And that's certainly something supported by HAPI and Grahame's server
Lloyd McKenzie (Nov 05 2020 at 22:42):
The notion of using _contained was very late to the game and isn't widely supported
Rik Smithies (Nov 05 2020 at 22:49):
does _contained only affect the output? I thought it affected the search itself?
Lloyd McKenzie (Nov 05 2020 at 23:06):
_contained changes what's being searched.
Rik Smithies (Nov 05 2020 at 23:19):
right, yes. But I now see that you can search into a contained Y in an X, using e.g. X?Y.code
That doesn't need _contained.
Rik Smithies (Nov 05 2020 at 23:20):
As long as you know what is contained in what, that is enough.
Lloyd McKenzie (Nov 06 2020 at 00:04):
When you're doing chaining, you don't care if it's contained or not. MedicationRequest?medication.code=foo works whether the medication is contained or not.
Paul Church (Nov 06 2020 at 00:08):
How is that implemented? Do the contained resources have their own database rows that can be joined?
Lloyd McKenzie (Nov 06 2020 at 00:16):
@Grahame Grieve @James Agnew
Josh Mandel (Nov 06 2020 at 02:48):
I think that's the usual approach (it's what the SMART server did, back in the day)
James Agnew (Nov 06 2020 at 03:05):
HAPI certainly does not support this, for all of the same reasons listed above: tricky to implement in a performant way, and at least personally most use cases I've seen where someone has wanted to do it have ultimately been because the person was trying to misuse contained resources (and therefore a workaround existed to just use a non-contained resource, or an identifier reference, or an alternative to the reference (e.g. reasonCode instead of reasonReference).
OTOH I've seen a handful of people work around this limitation by creating a custom search parameter with a path that explicitly dives into the contained resource. Oddly enough one such instance was for compound medications like Vassil described above.
Paul Church (Nov 06 2020 at 03:47):
Does HAPI support chaining into a contained resource though?
Paul Church (Nov 06 2020 at 04:44):
Also, does _content search inside contained resources? Does _text search the narrative of contained resources, if they have one?
Paul Church (Nov 06 2020 at 04:58):
Now it's midnight and you've got me writing a design doc for how this could be implemented.
Rik Smithies (Nov 06 2020 at 13:09):
Presumably this would also work with reverse chaining? e.g. to search into a contained Provenance (which refers to the containing resource, not the other way around).
Lloyd McKenzie (Nov 06 2020 at 22:19):
@James Agnew HAPI doesn't support _contained (which I'm fine with) or HAPI doesn't support chaining into contained resources (which is a pretty serious limitation because contained practitioners inside PractitionerRole, contained Medication inside various resources and lots of other scenarios like that will be pretty common. There are lots of valid reasons for contained resources and you certainly don't want to drop to identifier references which are definitely non-searchable.
James Agnew (Nov 09 2020 at 00:06):
The chaining use case is currently only achievable using custom search parameters with a FHIRPath that dives right into the contained resource for the indexed content.
Paul Church (Nov 09 2020 at 16:02):
It seems like there's a real disconnect between Lloyd/Rik saying this is important and all of the implementers on this thread saying they're not doing it. Is there implementation experience on the EHR side?
Michele Mottini (Nov 09 2020 at 16:23):
Most EHR do not have chained search at all
Jens Villadsen (Nov 09 2020 at 17:15):
Most EHRs are arcane monoliths from the 80'ies or 90'ies which was based and build on different grounds of such search capabilties
Rik Smithies (Nov 09 2020 at 18:58):
FHIR isn't only for facades onto big iron EHRs. When used that way, FHIRs capabilities are always likely to be a superset. Why bother with chaining at all?
But in general FHIR should provide a consistent feature set. Contained resources without search are not much use. And they were invented for good reasons.
Paul Church (Nov 09 2020 at 19:19):
Yes, I agree with that but the useful ideas with good reasons need to be tested through implementation experience! The search spec has made it all the way to normative and yet we're having this conversation about whether anyone actually supports _contained or chaining into contained resources.
James Agnew (Nov 09 2020 at 19:30):
Completely agree with Paul on this.
I still don't understand the use cases for needing this (with 'this' meaning searches reaching into contained resources).
The only place I've personally seen it used for was the compound medications one, and if I'm honest, that just felt like it was working around a weakness in the resource models (i.e. clearly it was important to the implementer to be able to track ratios of products in an IV solution as an atomic unit-- so why don't we provide a way to do this that doesn't require contained resources?)
Are the other use cases similar?
Lloyd McKenzie (Nov 09 2020 at 19:33):
I've seen 'contained' resource in lots of cases. Contained 'Provenance' where the Provenance isn't maintained independently. Contained Questionnaire within QuestionnaireResponse when doing adaptive questionnaires. Contained Practitioners when the source data captures the provider as part of the record rather than as a stand-alone registry entry.
Lloyd McKenzie (Nov 09 2020 at 19:34):
Contained is going to be pretty common - and being able to chain into contained resources is also going to matter
Vassil Peytchev (Nov 09 2020 at 19:56):
I've seen 'contained' resource in lots of cases.
Seen in specifications, or seen in actual use?
Lloyd McKenzie (Nov 09 2020 at 20:02):
Actual use
Jean Duteau (Nov 09 2020 at 20:17):
I'm involved in a number of implementations that are using contained resources. One healthcare FHIR server uses contained resources for many of the reasons that Lloyd suggested. Medications are contained because they need to provide the information in that resource but they aren't maintaining them independently. Practitioners are contained because they have practitioner information but they aren't serving those up as independent resources.
Lloyd McKenzie (Nov 09 2020 at 20:23):
Lots of legacy repositories will have to use contained because of how their data is organized. Even those with a strictly FHIR-based repository will still have to use contained where the resource doesn't have independent identity (e.g. the adaptive forms approach defined in Argonaut and SDC)
Rik Smithies (Nov 09 2020 at 20:39):
We use contained Provenance - and cannot find a way to search. It needs reverse chaining.
(I was wrong about Task, we use that but it is not contained...).
Lloyd McKenzie (Nov 09 2020 at 20:46):
If you have a contained provenance, it would be unusual to search on it. To do that you would need _contained
Rik Smithies (Nov 09 2020 at 20:58):
Well I guess that depends if you want to search on Provenance itself or to find containing resources that have some embedded Provenance aspect. The use case of Provenance here is to hold the actual updated date of the (legacy) data, and not the timestamp of the resource. Then search on it.
Paul Church (Nov 09 2020 at 21:16):
@Lloyd McKenzie but have you seen _contained search or chaining into contained in lots of cases? It sounds to me like out of the production implementations represented here, nobody supports them, nobody has stated plans to support them, and nobody is quite sure if they're technically feasible at all.
Lloyd McKenzie (Nov 09 2020 at 21:27):
I have never seen _contained. That's pretty esoteric. DaVinci and SDC both require searching by the URL of the contained Questionnaire to retrieve completed questionnaires for instruments like promise. Searching by dispenses by medication or lot number would also generally require hitting contained resources. (There's zero chance of them being stand-alone resources, particularly in community pharmacy and even in inpatient, it'd be unusual.)
Paul Church (Nov 11 2020 at 17:38):
If anyone wants to try implementing chaining into contained resources I would like to highlight the case where Patient/1 refers to a contained PractitionerRole #pr, and that PractitionerRole refers to a contained Organization #o (which is contained in the Patient, since contained resources aren't nested) with a name of X.
There are two interesting cases that should both return Patient/1:
- PractitionerRole?organization.name=X&_contained=true
- Patient?general-practitioner.organization.name=X
Grahame Grieve (Nov 12 2020 at 10:10):
catching up with this... I absolutely support chaining into contained resources. Contained resources are a row in my database just like any other resources, with a key to the container resource (so I can decide whether I want to see them or not). I think I don't implement _contained fully but chained search into contained resources is important
Last updated: Apr 12 2022 at 19:14 UTC
