Stream: implementers
Topic: Nested Bundle
Dheeraj Kumar Pal (Nov 03 2020 at 13:50):
We are working on Creating the FHIR Central services for New York State.. Where different RHIOs will be Queried to pull the data for a patient. Single query fan-out to multiple based on the patient existence in multiple RHIOs i.e. single query will have multiple response (bundle) from different sources. We would like to aggregate these responses in one single bundle as it is (nested bundle). It seems FHIR do not support nested bundle. What is recommendation for this use case.
John Moehrke (Nov 03 2020 at 13:55):
Nested bundles is supported. IHE uses that in a few of their implementation guides... Can you point at the place in the FHIR specification where you see a restriction?
Dheeraj Kumar Pal (Nov 03 2020 at 14:51):
Thank you John !! That's based on my communication with other implementers .. Would you please share IHE implementation link
Vassil Peytchev (Nov 03 2020 at 15:08):
If by query you mean FHIR RESTful search then I think a search result Bundle is not expected to contain other Bundles. If you define a specific operation, however, you can define the result to contain multiple Bundles.
Dheeraj Kumar Pal (Nov 03 2020 at 15:16):
FHIR central service is HUB , which can take a query from one if the participant and queries back all other participant. Each participants returns a bundle against the query. Now FHIR central service needs to bundle all these search responses and create a super bundle that means bundle contains the multiple bundles as entries
Bundle
entry
--0
-- bundle
-- 1
-- bundle
Is this possible?
Vassil Peytchev (Nov 03 2020 at 15:17):
It depends on what you mean by "query"
Michele Mottini (Nov 03 2020 at 15:18):
No, a search cannot return bundles, you have to combine the bundle received from the participants
David Pyke (Nov 03 2020 at 15:19):
Yes, that is possible but does not happen automatically. You'd create the bundle and the receiver of the information would need to customize a handler to debundle the responses.
Dheeraj Kumar Pal (Nov 03 2020 at 15:23):
That was my understanding ... but the challenge is that .. We are HUB and are not permitted to aggregate the different source data in one bundle .. We have to pass on all the bundles separately in another container. What we have done is created a super bundle where each entry of the super bundle is response from each source. Just would like to confirm if this aligns with FHIR specification?
David Pyke (Nov 03 2020 at 15:23):
Yes, a Bundle of Bundles is legal syntax
Dheeraj Kumar Pal (Nov 03 2020 at 15:24):
Perfect !! Thank you
Michele Mottini (Nov 03 2020 at 15:24):
No, it is not
David Pyke (Nov 03 2020 at 15:25):
There is no restriction on the type of resource allowed in a bundle.
Michele Mottini (Nov 03 2020 at 15:25):
If you receive a request /Condition?patient=xxxx to get all Conditions for a patient the response MUST be a Bundle of Condtion resources, not a bundle of bundlers
David Pyke (Nov 03 2020 at 15:27):
Yes, that is true. However, in the situation he's describing, he's passing through the request to many endpoints and they each return a response. THe receiver will need to know that they're receiving a bundle and debundle it prior to processing the query response.
David Pyke (Nov 03 2020 at 15:28):
It's a custom process, absolutly, but you can put bundles in bundles, as long as the receiver knows to expect and can process that
Michele Mottini (Nov 03 2020 at 15:28):
The receiver is issuing a FHIR search, the response should match what FHIR specifies- that is a bundle of the resources being searched, not a bundle of bundles
David Pyke (Nov 03 2020 at 15:29):
Absolutely true, as I said, it will require that the receiver of the query response knows this will happen and will need to debundle each individual response prior to processing.
Michele Mottini (Nov 03 2020 at 15:30):
That would not be a FHIR server any more then
David Pyke (Nov 03 2020 at 15:31):
Technically, the FHIR server would have a front end process, that handles the debundling. THe FHIR server would be behind it
Dheeraj Kumar Pal (Nov 03 2020 at 15:33):
That's my understanding.. behind such use cases..
David Pyke (Nov 03 2020 at 15:34):
Yes, sometimes you need to bend rules to handle use cases. Really, there should be a FHIR IG that defines a custom operation that outline all of this so that it all can be done in a standard way
Dheeraj Kumar Pal (Nov 03 2020 at 15:36):
I agree .. we are working on IG for New York state and would like to request FHIR team to advise the right way to discuss this use case... and conclude in standard way
David Pyke (Nov 03 2020 at 15:42):
Likely you'll need to create a custom operation that, instead of receiving the actual query, you receive a transaction bundle that you push to the RHIOs. The originator of the query, knows that the response will be a bundle of bundles and will process it appropriately.
David Pyke (Nov 03 2020 at 15:43):
As I said, it's a very custom process/operation but it is legal to bundle Bundles.
Vassil Peytchev (Nov 03 2020 at 15:45):
You should also look at alternatives. For example, the HUB can provide a service to return the endpoints for a particular search. That way you can let the initiator of the query do the searches and combine the results as they need to.
Dheeraj Kumar Pal (Nov 03 2020 at 15:50):
FHIR Central service is state initiative and we would like to keep all the request and response flowing through central service. Also , would like to keep everyone's response separate in our final response to requestor. In case IHE there will more such scenarios .. where you are querying different FHIR servers but do not want to consolidate the response but would like to pass on as it. Is there possibility to have one level of nesting for bundle.
John Moehrke (Nov 03 2020 at 16:07):
as Dave indicates, you can do this with an $operation. You can define an operation that takes Parameters resource for the query, and returns a bundle of bundles. But, as Vassil points out, this would not be natural for a typical FHIR "RESTful" client, that is going to just do a RESTful GET with parameters.
John Moehrke (Nov 03 2020 at 16:09):
so, you have a choice.. a FHIR Operation, which can be defined. Or the hub does more of a federation of configuration endpoints, and let clients do what comes natural. The centeral authority could handle OAuth authorizations, and thus smooth the authorization pathway. This authorization pathway does give it ability to manage and record audit logs (to the granularity of the authorization request)
Vassil Peytchev (Nov 03 2020 at 16:17):
As long as you realize that you are asking clients to do something that is not part of the core specification, you can go that route. You will probably need to consider why clients would find a need to do something custom.
Dheeraj Kumar Pal (Nov 03 2020 at 16:18):
We are working as Trusted entities and not going through the OAUTH . We might go that route in future when we open up this as an API. Our Current implementation is within trusted entities (RHIOs) and we would like to keep it
- All queries going thorough HUB and HUB making call to other RHIOS
- HUB also, Aggregate the data and return one response against the query received from requestor.
I am writing the scenario and will post here in some time.. It's bit complicated use case and scenario and Policy adds more complication to it it. That's why we are seeking if it is use case that can be considered for change in bundle definition and allow at-least one level nesting for bundle?
David Pyke (Nov 03 2020 at 16:19):
It is legal to put bundles in bundles. YOU'll just need to have a custom operation to deal with it
John Moehrke (Nov 03 2020 at 16:20):
and... what you are doing is NOT "http REST", so not the natural core use of FHIR... not wrong, just not on the beaten path.
Dheeraj Kumar Pal (Nov 03 2020 at 16:22):
It is httsp rest service (FHIR central service)..
Dheeraj Kumar Pal (Nov 03 2020 at 16:23):
We need our participant to Query Central service to get the patient data across state i.e. from all other RHIOs. we are also using the State MPI to know where all patient exists.
Dheeraj Kumar Pal (Nov 03 2020 at 16:25):
I have the architecture diagram that will help you all to see the actual scenario .. but not sure . if I can send here in chat
David Pyke (Nov 03 2020 at 16:26):
I'm very familiar with the NYeC SHINNY use case. We can discuss it off-line if you like
Dheeraj Kumar Pal (Nov 03 2020 at 16:29):
Sure David !! I know we worked together in past under SHINNY
David Pyke (Nov 03 2020 at 16:31):
Give me a call and we'll work through this
Josh Mandel (Nov 03 2020 at 17:58):
we would like to keep all the request and response flowing through central service.
You can accomplish this with the central service acting as a proxy, allowing clients to interact in a "normal FHIR REST API fashion " with the different endpoints. The more you can make your implementation look/feel like a regular FHIR server (or a collection of regular FHIR servers) the better, in my opinion.
Dheeraj Kumar Pal (Nov 03 2020 at 18:19):
Requesting Source does not know where all patient exists in community. FHIR central service (HUB) is integrated with State Wide MPI which is again managed centrally. Also, we would have requirement to create a central service to avoid multiple infrastructure build for the same thing.
Gino Canessa (Nov 03 2020 at 18:51):
Was walking through this and I have a question: is the hub going to rewrite all links/urls?
- within resources themselves
- within Bundles (e.g., paging on bundles from the sites)
If you're already parsing and changing data, why not consolidate all of the resources into the same bundle too? Then everything is standard. If you're not planning on rewriting links, I think there will be problems.
John Moehrke (Nov 03 2020 at 19:10):
no problem for a hub to be a patient identity manager... we are all speaking about the queries for observations, those should go direct to the FHIR Servers that are custodians of that data. The Patient resource that is managed centerally would just have a bunch of Patient.link (see also) that itemize each of these FHIR Servers.
John Moehrke (Nov 03 2020 at 19:12):
If the hub insists on being the ONLY endpoint for REST queries, then it will need to do what Gino just mentioned and need to be fiddeling with ALL of the data returned to re-write all references to be hosted at the central hub.... If that is done, then there needs to be Provenance of that fiddeling... and many will question this level of messing with ALL data
Dheeraj Kumar Pal (Nov 03 2020 at 20:16):
@Gino Canessa -- We are not re-writing anything.. We are putting each response as it is in a bundle (we call it as super bundle).. For e.g.
If we make call to 5 different sources and receive 5 bundle... HUB response to requestor is going to be super bundle with 5 bundle (response) from sources. I am not sure what problem you are referring to? Why we have to re-write the links? Our requirement is to keep the responses as it is.
Vassil Peytchev (Nov 03 2020 at 20:21):
At any place in a resource that you a have a Reference, there can potentially be a URL to get that referenced resource. The URL will likely point to the server of the responding system. The requester now has a link they can either not able to access, or they can access without going through the hub.
David Pyke (Nov 03 2020 at 20:23):
THe FHIR requester would have to know to route follow-up queries through the HUB or the RHIOs would need to use a URL for the references that points to the HUB that the HUB would know to forward to the data source RHIO
John Moehrke (Nov 03 2020 at 20:26):
You would need to re-write the Reference.... else it looks like the Resource is on your hub....
For example a Observation.subject = "Patient/1234" would be looked for in the bundle, and when not found in the bundle would be looked for at the URL root of the bundle... thus all relative references would appear to be at your HUB.
John Moehrke (Nov 03 2020 at 20:26):
and there is no way to assure uniqueness in that case
John Moehrke (Nov 03 2020 at 20:27):
even if you replaced all relative URI to full original source URL.. then you end up with the issue Dave is outlining
John Moehrke (Nov 03 2020 at 20:28):
which would give a consuming client an impression that they can retrieve (GET) that full original source URL... and if they can do that, then you have a trust framework that would allow the client to do original http query direct to that endpoint.
Dheeraj Kumar Pal (Nov 03 2020 at 20:32):
@John Moehrke In our implementation , Client needs to extract the all bundles from super bundle and parse the those bundles .. which are maintained as it is. As an requirement , we are not proposing to include super path in XPATH or FHIRPATH.. we are saying requestor in receipt of response needs to extract the bundles from super bundle and parse each one of them
John Moehrke (Nov 03 2020 at 20:34):
okay, but in the sub-bundle are resources (e.g. Observations) with reference values (e.g. Observation.patient) that is dereferencable to a fully specified URL.
John Moehrke (Nov 03 2020 at 20:34):
see The fantastic screen cast from @Josh Mandel https://youtu.be/ZK0AKB5PqGM
Josh Mandel (Nov 03 2020 at 20:54):
we are saying requestor in receipt of response needs to extract the bundles from super bundle and parse each one of them
The question is why do you want to create these special processing rules and obligations? This approach is likely to cause confusion and development mistakes for clients; what is the problem you're trying to solve here?
Lloyd McKenzie (Nov 03 2020 at 21:22):
The key thing to know is that no existing systems will be able to use your interface. It won't work for SMART on FHIR apps, CDS Hooks services, etc. It'll be a pure custom interface. Because the data will be pure FHIR, it won't be as hard for someone to write an adapter to use your interface, but they'll still have to write an adapter. As well, if you're not touching any of the 'self' links or fullUrls in the bundled search Bundles, then when the recipient of the results tries to hit those URLs - which will point back to the original services, not the hubs, you may have an issue.
The reality is that a hub that can't aggregate the data and finesse differences in identity across the servers behind the scenes isn't necessarily going to provide a lot of value. It may be worth expending some energy to try to get the "thou shalt not touch the search bundles" requirement revisited, because it's severely limiting the value of the interface you're trying to provide.
Dheeraj Kumar Pal (Nov 04 2020 at 16:08):
Thank you all for your valuable input ... I am taking this input to share with team .... and will come back with next steps or questions
Last updated: Apr 12 2022 at 19:14 UTC
