FHIR Chat · GF#17408 - FHIR end-point discovery · implementers

Stream: implementers

Topic: GF#17408 - FHIR end-point discovery

---

Grahame Grieve (Jul 01 2018 at 22:12):

GF#17408 proposes that a capability statement be able to reference other end -points with different version (though presumably the same... something). That doesn't really make sense to me - you have to find one of the end points first... and we haven't done anything about end point discovery (other that via provider directory). I'm interested in implementer comment on this...

Andrew Marcus (Jul 02 2018 at 13:19):

Suppose, for example, that we've implemented a server where each supported version of FHIR is at its own endpoint. For instance: fhir.asymmetrik.com/dstu2, fhir.asymmetrik.com/3.0.1, fhir.asymmetrik.com/r4

We know that r4 will give us the ability to implement each of these versions behind a single URL by using Accepts headers. But there may be reasons why it makes sense to continue to split it out like this, particularly because the older versions of the APIs don't support the Accepts header and may already be implemented.

Now suppose fhir.asymmetrik.com were listed in a directory. Under the current situation it would need to be listed 3 separate times, once for each of the versioned endpoints.

What the GF I submitted proposes is that we can list the endpoint once, and then a client can hit a well-known discovery URL at the root hostname to discover which additional endpoints are supported. For example, something like this:

fhir.asymmetrik.com/discovery.json

[
{
url: fhir.asymmetrik.com/dstu2
supportedVersion: [1.8.0, 2.0.0, 'dstu2']
},
{
url: fhir.asymmetrik.com/3.0.1
supportedVersion: [3.0.1, 'stu3']
},
{
url: fhir.asymmetrik.com/r4
supportedVersion: [4.0.0, 'r4']
}
]

Leaving aside for a moment whether the version numbers I listed actually make sense, a format like this would allow directories to stay cleaner (they just need a root domain name), while allowing clients to figure out which versions are supported and where to go to find them.

Andrew Marcus (Jul 02 2018 at 13:24):

Obviously the supportedVersion syntax should be cleaned up from what I posted above. On possibility would be to support version ranges the same way npm does: https://docs.npmjs.com/misc/semver. For example, something like [1.8.0 , >=2.0.0 <2.5.0]

I think it would also be useful to include the string versions (dstu2, r4, etc) as well though, as a less-precise but more human-readable alternative that could be faster for clients to get up and running

Kevin Olbrich (Jul 02 2018 at 13:55):

I think you can effectively achieve this goal by using a load balancer to route traffic to the correct server after inspecting headers. I strongly recommend that you not use versioned urls if you can avoid it. I would also recommend that you only use the semantic version. The urls don't need to be human readable, and frankly, using designations like dstu2 are just confusing and imprecise.

Kevin Olbrich (Jul 02 2018 at 13:57):

One of the goals of the accept header approach is to use standard HTTP content negotiation processes to figure out which version to use instead of building a custom way of figuring this out. Your proposal is to build that alternative approach.

Kevin Olbrich (Jul 02 2018 at 13:58):

@Andrew Marcus given your system, is it technically difficult or infeasible to move to the accept header approach? If so, that would be an important use case and I would like to understand your limitations better.

Grahame Grieve (Jul 02 2018 at 15:33):

@Brian Postlethwaite right now, there's now way to have an end-point for different versions or purposes?

Grahame Grieve (Jul 02 2018 at 15:33):

looks like a problem in end-point to me

Andrew Marcus (Jul 02 2018 at 15:34):

Last I checked, each CapabilityStatement has a single fhirVersion element: http://hl7.org/fhir/2018May/capabilitystatement-definitions.html#CapabilityStatement.fhirVersion This makes sense given that the resources and search parameters available in each version of FHIR are different.

Do you propose showing a client a different CapabilityStatement solely based on the Accepts headers?

Suppose, for example, that you as a FHIR server implementor are slowly building out support for a new version. You've implemented Patient in 3.2.0 but haven't yet implemented Observation. However, your 3.0.1 implementation supports Observation. If a client requests a CapabilityStatement with Accepts: 3.0.1, 3.2.0 they'll get back your latest one which includes Patient but not Observation. It will then be difficult for the client to determine that if they instead make the same request with Accepts: 3.0.1, they'll get back a different CapabilityStatement that does include Observation.

Our recommendation is that both ConformanceStatements be discoverable, which lets the client decide for themselves which version supports all the features they need. The client could still choose to request a 3.2.0 Patient and a 3.0.1 Observation, or the client could decide it would rather handle everything in version 3.0.1 for simplicity.

Also, consider that your FHIR server implementation may consist of different codebases setup at different times in order to support each new version as it is published. A server implementor may wish to upgrade their existing server to handle the new version, which will include a new CapabilityStatement and may include additional endpoints or additional parameters on existing endpoints. But it may be easier to build a new independent server implementation and keep both running in parallel. It is potentially a huge maintenance burden on the load balancer configuration (which is likely a different technology stack than your server) to be able to dispatch every possible endpoint supported by any version you implement.

Andrew Marcus (Jul 02 2018 at 15:50):

An alternative to adding a new discovery endpoint and statement format would be to merge all the versioning information into every aspect of the CapabilityStatement. So, for example, when defining a resource, also define which fhirVersions that resource supports. This would make the CapabilityStatement represent the union of all possible URLs and parameters available for any supported version of FHIR.

However, some obvious challenges with this approach are:
1. Things like supported search parameters could be different between versions, so you'd have to list them out separately for each version, or have some way to codify the differences
2. DSTU2 used a conformance statement instead, so merging DSTU2 endpoints into a CapabilityStatement would be challenging and time consuming for each server implementor
3. this would be a breaking change for any client already consuming a CapabilityStatement
4. the CapabilityStatement would no longer be tied to any particular codebase. So, for instance, if you as a FHIR server implementor would prefer to build a new server stack to support the latest version, you'd need a different build process to create a union CapabilityStatement that also works will all your previous versions.

For these reasons, a new discovery endpoint would be much simpler and more robust.

Grahame Grieve (Jul 02 2018 at 15:55):

so have you looked at the EndPoint resource?

Andrew Marcus (Jul 02 2018 at 16:01):

@Grahame Grieve Thanks. It looks like EndPoint resources could be used to represent the locations of CapabilityStatements for the different versions. And the payloadType could potentially use the Accepts header mime-type encoding to represent the version supported.

But is there a place where you would compose multiple endpoint resources? Would you perhaps list them all in the Organization resource? What if a FHIR server instance supports multiple organizations?

Grahame Grieve (Jul 02 2018 at 16:04):

well, most servers will have many organizations - trading partners etc in the health eco-system . There's no way to indicate which organization owns/runs the server. But that's thinking at the wrong point; the question is, how did you find out where to look for the capability statement at all? And the answer is, in some registry somewhere, however that is. And that's where the version/purpose/endpoint map belongs. and for is, this is the provider directory...

Andrew Marcus (Jul 02 2018 at 16:14):

So, back to the original question then. Suppose I want my server listed in server provider directories. Since there isn't a single standard for how those provider directories operate, each might have a different process for listing your server and its capabilities. Perhaps you send someone an email, perhaps they have a REST endpoint or a developer console...

If I decide to add support for a new version to my server, I shouldn't have to contact each directory individually to get them to update my listing. This quickly turns into a maintenance nightmare. I should just be able to update a statement somewhere in my own infrastructure that each of the providers references. As long as they are pointing to my hostname, the provider can extract whatever information they want to display about my FHIR server.

Andrew Marcus (Jul 02 2018 at 16:59):

Zooming out a bit, the problem comes to this:

The two main best-practice ways to version APIs are either through URLs or Accept headers. https://restfulapi.net/versioning/ There are advantages and disadvantages to both methods, and typically which you choose would depend on your use cases, users, and server stack.

Adding Accept headers to FHIR lets a server developer loosely version their FHIR endpoints, which is great. However, the CapabilityStatements that describe the endpoints are tightly versioned, and Accept headers alone can not effectively loosen them, for reasons I mentioned above.

In order for the new loose-versioning features in R4 to be useful, there must also be a consistent way to loosely version the CapabilityStatement, or to allow a client to find all the relevant tightly-versioned CapabilityStatements.

Andrew Marcus (Jul 10 2018 at 18:22):

More discussion here: https://chat.fhir.org/#narrow/stream/4-implementers/subject/GF.2310205.20-.20Cardinality.20of.20Conformance.2Erest/near/163970

Andrew Marcus (Jul 10 2018 at 20:14):

@Grahame Grieve @Kevin Olbrich

The Accept header specification https://build.fhir.org/versioning.html#mt-version currently says the following:

The most common strategy for handling change between versions of FHIR is to use different end-points for different versions. e.g. http://acme.com/fhir/r2 and http://acme.com/fhir/r3. However, this can be problematic because it means that the same record has a different identity depending on the version of FHIR in use - though this can also be useful; it depends on the context.

A server can support multiple versions on the same end-point, if the client specifies which version to support using the fhirVersion mime type parameter. GET [base]/metadata Accept: application/fhir+json; fhirVersion=3.0 This is a request to return the CapabilityStatement for the server’s support of FHIR Release 3. The client will know that the server doesn’t support R3 if it gets an error in return, or if the capability statement that comes back has a different fhirVersion (which would mean that the server does not support the fhirVersion parameter). The client must include the fhirVersion parameter on all following interactions.

Let me try to spell out some use cases a little more clearly.

1. As a FHIR server implementer, I would like to upgrade my server to support a newer version of FHIR

Suppose I've already invested money in my current FHIR implementation. My two options are to either add support for the new FHIR version to my existing stack, or to build a new parallel server stack for my new version. In the first case I can easily add an Accept header and add code to my server to dispatch to the old or new version based on that header. However, if I choose a new parallel stack (perhaps to clean up technical debt, upgrade the architecture, use a different vendor, etc) it is way cleaner to implement a new set of endpoint URLs. If I need to upgrade the existing endpoints to dispatch based on an accept header, I either need to build Accept-based dispatching into the old and/or new codebase, or spin up a set of load balancer rules to handle dispatching for every possible endpoint. This essentially becomes a 3rd stack that needs to be maintained.

Similarly, the cleanest way to maintain metadata about two independent stacks is for each to have its own CapabilityStatement maintained along with their respective codebase. If I need to publish a single /metadata endpoint with a single CapabilityStatement that describes all supported versions (as is being discussed at https://chat.fhir.org/#narrow/stream/4-implementers/subject/GF.2310205.20-.20Cardinality.20of.20Conformance.2Erest), then I need some additional set of technology that is able to merge together the capability statements from each stack, which might use different technologies, into a single document.

If I want to keep my endpoint URLs separate and avoid introducing a 3rd technology, I could publish my endpoints independently to some online directory of known FHIR endpoints. But each may have their own mechanisms for submission and updates.

2. As a FHIR app implementer, I would like to discover FHIR endpoints that I can talk to and have the capabilities I need

Given a URL to a FHIR server, I'd like to be able to discover which version it supports that my client can also support. If I query the /metadata endpoint with a specific version in the Accept header, I will get back a boolean response; either it supports that version or it doesn't. If it doesn't accept my preferred version, I get back no information about which alternate versions it supports instead. I have to try the request again with another version and see if I get a different response. It is also not clear what happens if I query with no Accept header at all: do I get back the latest supported version, or an arbitrary version? Perhaps the server only supports one version? There is no way for a client to tell which is the case under the current Accept header specification.

In essence, each FHIR client has to attempt a shotgun approach, interrogating the /metadata endpoint with every possible version that the client could potentially accept, without any information about which may be successful. At the moment when there are only a handful of FHIR versions, this is not a big deal. But it will grow into an increasingly large number of queries as the number of FHIR versions increases.

Now suppose that the FHIR server implementer is slowly building out resources in a new 3.3.0 FHIR version. They publish a new capability statement that defines Patient in 3.3.0 but no other resources. They also have an existing endpoints in 3.0.1 that implements both Patient and Observation. My client app requires both resources. If I simply interrogate the /metadata endpoint and discover the 3.3.0 version, I may conclude that the server does not support the features I need, even though I could have asked it for 3.0.1 instead. Without a way to discover which versions are supported and what each of their capabilities are, my client app is flying blind.

Potential Solutions:

1. Move the CapabilityStatement.fhirVersion and CapabilityStatement.url parameters down a level into CapabilityStatement.rest, allowing each CapabilityStatement to specify all the endpoints for each version it supports.

Downside: this refactors the CapabilityStatement definition in a potentially breaking way. Also, there may be situations in which this is hard to implement technically, especially if different versions are implemented using different stacks.

2. Add an optional element to CapabilityStatement which allows it to link to other supported versions on the same server. Then, if a client requests a version with or without an Accept header, the resulting CapabilityStatement points that client to other versions.

Downside: the list of other versions would need to be maintained in every version. It would be potentially problematic to have to iterate recursively through lots of statements in order to make sure you found them all.

3. Add a new code to CapabilityStatement.kind allowing a particular statement to act as an index to other statements. It would need to include a mechanism to list the url and fhirVersion of every other capability statement on the same hostname or installation.

Downside: there does not appear to be a field in CapabilityStatement already to link to these endpoints, so new fields would need to be added that only support this use case. However, they could potentially support both solution 2 and 3 depending on the implementer.

4. Define a new type of version discovery statement, as proposed in GF#17408

Downside: introduces a new type of FHIR resource

5. Define explicit standards for submitting new FHIR endpoints to directory providers, and for querying for endpoints that implement specific FHIR versions

Downside: introduces new types of FHIR resources, requires buy-in from existing directory providers

Andrew Marcus (Jul 10 2018 at 20:38):

Actually, combining solutions 1, 2 and 3 above could be pretty interesting, and allow implementers to choose the best approach for publishing their versions. It also solves the version discovery problem while avoiding introducing a new FHIR resource type, and remaining backward compatible.

Proposal:

• Add CapabilityStatement.rest.fhirVersion and CapabilityStatement.rest.url elements with cardinality 0..1
• Add "directory" to the list of code values for CapabilityStatement.kind
• Define invariants:
  • If CapabilityStatement.kind == 'directory', then CapabilityStatement.rest.fhirVersion and CapabilityStatement.rest.url must be set, but no other elements in CapabilityStatement.rest need to be provided. The URL must point to another CapabilityStatement. Or alternately, if all of the URLs are the same and differ only by Accept header, CapabilityStatement.url could be used instead.
  • CapabilityStatement.fhirVersion and CapabilityStatement.url will be assumed to apply to each REST definition for which CapabilityStatement.rest.fhirVersion or CapabilityStatement.rest.url are not provided.
  • Redefine cnf-8: There can only be one REST declaration per combination of mode and fhirVersion (expression:rest.select(mode).select(fhirVersion).isDistinct())

These three pieces open up a whole range of possibilities for the server implementer. For instance, /metadata without an Accept header could return:

1. a CapabilityStatement with the "directory" kind linking to the CapabilityStatements of each of the supported versions. These could all be at the same URL distinguished by Accept header, or could be at different URLs.
2. a complete aggregate CapabilityStatement which lists the capabilities of every supported version.
3. a CapabilityStatement in the existing format referencing a single version but using the instantiates field to point to another CapabilityStatement which lists other alternate versions.
4. a CapabilityStatement which lists all the capabilities for one version and also includes rest definitions that link to the CapabilityStatements for the other supported versions.

If a client receives a CapabilityStatement which does not include any of these elements (i.e. backward-compatibility mode), the client can safely assume the server at least supports the version listed in CapabilityStatement.fhirVersion.

Lloyd McKenzie (Sep 24 2018 at 20:25):

We don't need anything beyond what we already have. Anyone who wants can host a server that supports the CapabilityStatement endpoint where you can search to find all servers who participate in a particular community, what their capabilities are and what their server addresses are. There's no need for a kind of "directory". If you're hosting a registry of instance capabilities, then all of the CapabilityStatements would be of type "instance".

Lloyd McKenzie (Sep 24 2018 at 20:26):

You could even have multiple "instance" CapabilityStatements for the same endpoint - each with a different version, reflecting what you get when you'd hit the metadata endpoint for that server with an Accept header for that version.

Thomas Tveit Rosenlund (Jan 25 2019 at 14:35):

I would like to discuss a different approach to support more granulated use of the CapabilityStatement. We are currently looking into the case of offering registry data using HL7 FHIR format using FHIR Restful API. There will be three different kind of data offered from the platform regarding Practitioners, Organizations and Persons/Patients. As a part of the Organization scope one of the things we want to describe is the actual FHIR Capabilities and other form of electronic communication that the Organization supports for interacting with other Organizations. A part of the registry will be describing the Endpoints exposed from the Organization as well as their Capabilities.

I was a little suprised to find that there is no way to describe the capability of a FHIR Endpoint using CapabilityStatement. I can think of use cases where a reference from an endpoint to its CapabilityStatment could come in handy. There are two main hurdles that makes it difficult to implement this using FHIR R4.
1. The description of CapabilityStatement is wery specific about the fact that the CapabilityStatement expresses one server's Capabilities. Making the use-case of using CapabilityStatement Resources to describe actual Endpoints illegal according to the normative spec?
2. There is no reference from Endpoint to CapabilityStatement, although this can easily be solved by an extension.

Lloyd McKenzie (Jan 25 2019 at 15:21):

To retrieve the CapabilityServer of an endpoint, all you'd need to do is query 'meta' on that endpoint. Is the issue that you want to retrieve the CapabilityStatement using _include with a query on Endpoint?

Grahame Grieve (Jan 25 2019 at 19:06):

I can see why you'd want to refer to a CapabilityStatement from an end-point in a registry without wanting to hit the end-point. But a factor to consider is: some servers issue different CapabilityStatements based on the role of the user who is authenticated...

Grahame Grieve (Jan 25 2019 at 19:07):

Since

Specifically, capability statements are used in one of three ways

I don't think you're right that

The description of CapabilityStatement is wery specific about the fact that the CapabilityStatement expresses one server's Capabilities

Brian Postlethwaite (Jan 25 2019 at 21:01):

Usually the _include you're describing @Lloyd McKenzie wouldn't work as the capability statement is likely to be remote (on the target server) and not on the directory server with the endpoint resource.
(if the cap stmt was on the directory server too, you would potentially have sync issues, and others like Grahame describes above)

Lloyd McKenzie (Jan 25 2019 at 21:10):

I don't think we've ever said that _include references have to be local - though I agree that many servers might choose to enforce that.

Thomas Tveit Rosenlund (Jan 28 2019 at 14:00):

The CapabilityStatement can be used in one of three ways, correct. But all the three ways discuss different levels of description of a software solution.

Grahame is correct in the statement that we want to refer th a CapabilityStatement without hitting the actual endpoints. It here are dynamics in the capabilities exposed to different users, I still think this must be handled by the authorization the client has when accessing the actual endpoint.

The way I think of it: an organization can have a lot of software solutions available. Some of these software solutions can provide different technical endpoints that can expose a subset of the software solutions capabilities. One way to describe the software solution would be a set of nested CapabilityStatements where some are exposed throug one endpoint and others are exposed through a different endpoint. It would be neat to be able to describe what every endpoint does using the CapabilityStatements drawn for the CapabilityStatements of the entire software solution.

Grahame Grieve (Jan 28 2019 at 19:23):

I still don't see why this is true:

Making the use-case of using CapabilityStatement Resources to describe actual Endpoints illegal according to the normative spec?

Grahame Grieve (Jan 28 2019 at 19:23):

it's certainly not intended to be true, and I didn't find any language suggesting it is

Lloyd McKenzie (Jan 28 2019 at 21:28):

I think the question is whether there should be a core element on EndPoint that points to CapabilityStatement. My leaning right now might be to define a standard extension.

Thomas Tveit Rosenlund (Jan 30 2019 at 08:13):

I agree @Grahame Grieve that the documentation of CapabilityStatement does not state explicitly that it can not be used to describe actual Endpoints. But the fact that this kind of use-case is not mentioned in the spec, and the fact that you can not reference Endpoints from CapabilityStatements does suggest that this kind of use-case is out of scope for the CapabilityStatement.

@Lloyd McKenzie I would very much like to see an addition to the Endpoint resource for a reference to a CapabilityStatement or a set of CapabilityStatements to describe it. A standard extension could work to it the use-case is deemed to be in the 20% realm.

Lloyd McKenzie (Jan 30 2019 at 15:42):

@Thomas Tveit Rosenlund feel free to submit a change request. The work group can evaluate whether it should be core or extension.

---

Last updated: Apr 12 2022 at 19:14 UTC