FHIR Chat · any work backporting r5 subscriptions to r4? · subscriptions

Stream: subscriptions

Topic: any work backporting r5 subscriptions to r4?

Isaac Vetter (Mar 25 2020 at 20:43):

I'd thought that I'd seen something about Basic somewhere, but now can't find it.

Paul Church (Mar 25 2020 at 20:46):

https://github.com/argonautproject/subscriptions/blob/master/backport-to-r4.md

Isaac Vetter (Mar 25 2020 at 21:01):

How basic!

Paul Church (Mar 25 2020 at 21:13):

when I find a link for someone it technically counts as googling it, just less efficiently than normal and with fewer ads

Josh Mandel (Mar 25 2020 at 21:14):

(You can always paste ads here. Pretty sure our community guidelines don't address this... yet.)

Isaac Vetter (Mar 26 2020 at 02:27):

With the cms deadline for ADT notifications, and the r5 publication date seemingly forever away -- I wonder if there aren't additional options for stabilizing our subscriptions work sooner, so that it can be used in prod.

Isaac Vetter (Mar 26 2020 at 02:30):

I think that subscriptiontopic (and the changes to subscription) are stable, why must we wait for the rest of r5 to use it? I don't consider a profile on basic to be sufficiently accessible to implementers.

Gino Canessa (Mar 26 2020 at 02:57):

They aren't all that stable right now given that I have a backlog of items to update (modeling changes around the bundles+extensions is major, plus a laundry list of minor ones).

Been slightly derailed, or the next pass would be up already.
Edit: current plan is to have it up before April 1.

Isaac Vetter (Mar 26 2020 at 03:00):

Thanks, Gino. I'm pretty sure that the adt notifications requirement is six months from now-ish. What are creative ways to enable fhir subscriptions to meet this requirement?

Gino Canessa (Mar 26 2020 at 03:10):

No problem Isaac. I'm trying to get this done as aggressively as possible and I'm hopeful that after the next pass we'll be done with breaking changes (the April 1 deadline is so that we can test at the Connectathon in May).

I'd say that we should know where we stand after the Connectathon. Assuming nothing else major pops up, the models will pretty much be what we need for ballot in the following cycle. With push from (announcer voice) vendors like you! (/announcer voice), we can try to accelerate more, but I'm drawing a blank on how (will think on it).

Lloyd McKenzie (Mar 26 2020 at 04:56):

Theoretically we could do an STU update. Technically those can happen without ballot, though we'd have to squint pretty hard to decide that was an appropriate mechanism for such a radical revamp. Tricky bit is that if we wanted to ballot, the earliest we could do at this point would likely be Sept. - though we might be able to push for an out-of-cycle and do the balloting sooner.

David Pyke (Mar 26 2020 at 12:24):

Until that time, would it be possible to add Extensions to R4 Subscription to handle the basic use cases and address the issues?

Gino Canessa (Mar 26 2020 at 14:57):

We've looked at this before, and I'm not sure it's possible to merge the two with all of the breaking changes. Once I have the next pass of updates done I'll revisit to see if it's possible (and encourage everyone else to do the same).

Eric Haas (Mar 27 2020 at 19:39):

Until that time, would it be possible to add Extensions to R4 Subscription to handle the basic use cases and address the issues?

We started there and decided to use Basic instead... Why is using Basic a problem?

David Pyke (Mar 27 2020 at 19:44):

In my opinion, because of the depth of the extensions necessary to turn basic into a useful Resource. If we add capabilities to Subscription, we can minimize the need for customization.

Eric Haas (Mar 30 2020 at 17:09):

In my opinion, because of the depth of the extensions necessary to turn basic into a useful Resource. If we add capabilities to Subscription, we can minimize the need for customization.

Have you looked at theBackported R5 Subscrition proposal? Is not deeply nested extensions rather a single json string...

e.g.

{
  "resourceType": "Basic",
  "code": {
    "coding": [
      {
        "system": "http://hl7.org/fhir/resource-types",
        "code": "R5Subscription"
      }
    ]
  },
  "subject": {
    "reference": "Patient/58"
  },
  "extension": [
    {
      "url": "http://hl7.org/fhir/StructureDefinition/json-embedded-resource",
      "valueString": "{\"resourceType\":\"Subscription\",\"channel\":{\"endpoint\":\"https://client.subscriptions.argo.run/Endpoints/b962286c-d1c6-4b13-a409-8df2a7208fb8\",\"header\":[],\"heartbeatPeriod\":60,\"payload\":{\"content\":\"id-only\",\"contentType\":\"application/fhir+json\"},\"type\":{\"coding\":[{\"code\":\"rest-hook\",\"display\":\"Rest Hook\",\"system\":\"http://terminology.hl7.org/CodeSystem/subscription-channel-type\",\"userSelected\":false}],\"text\":\"REST Hook\"}},\"end\":\"2019-09-20T15:21:28.427Z\",\"eventCount\":0,\"filterBy\":[{\"matchType\":\"=\",\"name\":\"patient\",\"value\":\"Patient/58\"}],\"reason\":\"Client Testing\",\"status\":\"requested\",\"topic\":{\"reference\":\"https://server-r4.subscriptions.argo.run/Basic/1\"}}"
    }
  ]
}

David Pyke (Mar 30 2020 at 18:06):

What I see there, if I'm reading it correctly, is that you've embedded a resource in a valueString. I'm not sure that's better than nested extensions nor better than profiling the R4 Subscription resource. It's likely that I'm coming into this late and the reason for this design isn't clear to me.

Gino Canessa (Mar 30 2020 at 18:24):

No worries! From my perspective:

• There are multiple resources needed (e.g., Subscription, SubscriptionTopic). Using Basic lowers the implementation burden as far as endpoints, etc.
• Various fields in Subscription have moved and/or been renamed - this requires mapping in addition to extensions.
• Clarity for versioning (e.g., no interference with existing R4 implementations)
• Basic works.
• R5 is still under development.
  • Any R4 modeling would need to be updated consistently.
  • I haven't heard of anyone volunteering to model the extensions and mappings and keep them in sync.

Beyond that, both the 'how' on defining notification triggers and the 'what' of payloads are quite different. I'm unclear what the advantage is for trying to conflate the two versions, given that an implementation would likely just fork out one vs. the other anyway.

Eric Haas (Mar 30 2020 at 18:37):

I personally think this is better than adding a dozen extensions or really complex extensions to Subscription

David Pyke (Mar 30 2020 at 18:47):

You're probably right. If the goal is to duplicate R5 in R4, then this is likely better/easier. I was hoping that we could put a uncomplicated fix in to the R4 resource to make something usable for now and let the R5 one get more mature.

Gino Canessa (Mar 30 2020 at 18:48):

Haven't figure out a path for that - if it were just adding a few fields we wouldn't have needed a redesign.

David Pyke (Mar 30 2020 at 18:51):

That makes good sense

Vassil Peytchev (Apr 17 2020 at 21:08):

Getting back to this, I know Gino is really swamped with many more urgent tasks. In the mean time, is there anything that I can help with the state of R5 subscription, notification and topic changes in the current build? Is there a need to prepare for WG approvals? I am hoping that if there is a decent R5 set of proposed changes, there can be a second look at the Argonaut Subcriptions IG that can go beyond using the Basic resource.

Gino Canessa (Apr 17 2020 at 21:32):

Vassil, you have great timing! I am cleaning up the build right now to push the next pass of changes up. I hope to have it before end of business today.

From there, I am hopeful that it's mostly documentation (writing/reviewing/etc.).

Vassil Peytchev (Apr 17 2020 at 22:28):

Great! Is this the R5 CI build?

Gino Canessa (Apr 17 2020 at 22:34):

Yes, still in the branch I've been working on for now. Unfortunately the resource changes/renames are difficult to sort through.

Gino Canessa (Apr 18 2020 at 00:57):

Blah. Issues with the new bundle type and sorting out all the invariants. Giving up for tonight.

Vassil Peytchev (Apr 18 2020 at 02:48):

If you want to share your branch, I can try to help in figuring these out next week...

Eric Haas (Apr 18 2020 at 21:45):

I don't think there is an easy way to backport to R4 as discussed above the R4 and R5 are completely different and trying to shoehorn in extensions into R4 is more complex and more work that the proposed Basic approach.

Grahame Grieve (Apr 18 2020 at 21:54):

I would much rather that the R5 subscription adaptation in R4 still be an actual Subscription. What is the problem with just using extensions?

Keith Boone (Apr 19 2020 at 02:34):

This is the difference between the two:
image.png
And SubscriptionTopic doesn't even exist in R4, and is a critical component.

Grahame Grieve (Apr 19 2020 at 08:52):

so put some random fixed value in criteria, and use extensions for the other things.... then you're still using Subscription

Grahame Grieve (Apr 19 2020 at 08:52):

SubscriptionTopic isn't so obvious so I didn't raise that one...

Grahame Grieve (Apr 19 2020 at 08:53):

but hiding R5 resources inside basic.... I don't know why to even bother

Keith Boone (Apr 19 2020 at 15:47):

I was thinking for SubscriptionTopic we could actually have an extension that used valueReference(Binary), where the Binary would actually be the R5 SubscriptionTopic resource content.

Isaac Vetter (Apr 20 2020 at 19:49):

Or even, an extension in R4 Subscription pointing to a canonical, non-resolvable SubscriptionTopic, and then the R4 IG's define the meaning of the topic using language that equates to an R5's SubscriptionTopic's elements.

Gino Canessa (Apr 20 2020 at 21:27):

To be clear, I'm not opposed to something "more modeled", but I would like to get the R5 version sorted first. If we're still modifying those (which we are), then we're just multiplying our work. This was one of the reasons Basic was proposed when we were pressed for something last year.

That said, if we want to go with a more defined backport for R4:

• Subscription needs extensions/mappings
• Need a way to define and link to a SubscriptionTopic
• Payloads need sorting: current R5 is a new type of Bundle (subscription-notification) containing a new resource (SubscriptionStatus)
• anything I've missed? (spent a few minutes thinking, but I haven't been thorough)

Nothing feels insurmountable, but there's work there. I'm happy to help sketch out what is needed, but I don't have cycles to spare until the R5 stuff is ready.

David Pyke (Apr 20 2020 at 21:37):

Perhaps a meeting could be scheduled for this just to review? I think I have cycles that could be used to work on the backport

Vassil Peytchev (Apr 20 2020 at 22:54):

I am also interested in a meeting.

Eric Haas (Apr 20 2020 at 23:52):

I am not....I am am curious why there is a rush to discard the current Basic approach which is easy peasy and straightforward but lets go ahead and make it insanely complicated. ... I thought about adding all these extensions to R4 Subs a year ago when @Josh ( or @GIno or somebody smarter than me) suggested Basic his reasoning then is equally sound today.

Keith Boone (Apr 21 2020 at 08:23):

Others think the Basic approach is complicated and their approach simpler. There's no hard and fast rule as to what is easy or hard, it depends on what the assessor of easy/hard is comfortable with. Rework of any approach sucks, but later rework sucks a lot more than earlier rework. So it might be worth rehashing. Your mileage may vary.

Grahame Grieve (Apr 21 2020 at 09:43):

I don't like the Basic use for subscription because subscription gets special treatment in the server. I don't really want to look at Basic like that

Gino Canessa (Apr 21 2020 at 16:08):

Josh initially proposed Basic, but we both fleshed it out with feedback from the Argonaut (and other group) calls.

If implementers think another solution is better now... that's why we do this over time and in the open. I will stand with Past-Me that Basic was the right call at the time. I'll even go as far as to say that I'd prefer continuing with Basic until the R5 version is ready for ballot because:

• I don't want to thrash on this
• I don't want backporting to interfere with new development
  • double the work (core and extensions need updating)
  • change resistance from the extra work
  • trying to shoehorn things in instead of defining them properly
  • etc.

Personally, I think we need to focus on the getting it right for R5 before spending effort backporting (e.g., documenting and revising the current models on this branch so it can be merged into master and we can have a good connectathon). Just looking at the models from when Basic was proposed versus where the models are today, they are vastly different.

With all of that said, if people want to spend time on it - it's not my time to spend :-).

Lloyd McKenzie (Apr 21 2020 at 18:14):

Being able to perform 'special behavior' for Basic resources will, I think, be somewhat normal...

Grahame Grieve (Apr 21 2020 at 19:48):

@Gino Canessa I typiclly cut a new version to support the connectathon 5-6 weeks before the connectathon so that people have a stable version to target. I have not done that this time, but I've been asked to do it now. The subscription track is the primary user of R5 in the coming connectathon - what is the likely course towards a stable version?

Gino Canessa (Apr 21 2020 at 19:59):

@Grahame Grieve , I'm fixing an error Josh noticed (made the channel type a code instead of Coding) right now, but hope to have that pushed in the next few minutes (taking longer because I'm fixing all the codes that are used in multiple resources to be value sets as well).

Documentation is lagging, but as far as I know this has all* the model changes that have been sent to me. As of right now, we asked for a slot on Monday's FHIR-I call to look it over before merging into the master branch, but to my knowledge that isn't a requirement - so it can be done any time people are happy with it.

*There is one that's been brought up to me but not discussed: adding the ability to define a SubscriptionTopic trigger based on an defined event e.g.: system: HL7V2, code: GMC-C3456789. I think it makes sense, but would prefer discussion before adding it.

Grahame Grieve (Apr 21 2020 at 20:00):

so early next week after the FHIR-I call then

Gino Canessa (Apr 21 2020 at 20:01):

That would be my preference.

Jenni Syed (May 15 2020 at 16:49):

Hey all, this is a topic at our connectathon that came up and I want to make sure I understand the desired endstate and use cases for the backport. We were considering having a breakout on this. If I look at one of the use cases that @Isaac Vetter brought up, it was to meet an upcoming CMS use case around ADT. I am curious if there are others.

Jenni Syed (May 15 2020 at 16:51):

Some thoughts: this isn't "backporting" R5 to R4, since R5 won't be released in time/timing of R5 is the main driver of the backport. Given that, we're actually going to need to choose a snapshot to backport. As an example, even the changes in build today have trackers we're logging during the connectathon.

Jenni Syed (May 15 2020 at 16:51):

So even if we think it's "stable" I would assume we'll get feedback to change between the final ballot version for R5 and the final R5. I think that's fine, but want to make sure this is a shared understanding

Jenni Syed (May 15 2020 at 16:52):

Second, as a personal opinion: I don't think the target would be to back patch a fully functional R5-like subscription. You may need to give up some functionality to make sure it's "simple"

Jenni Syed (May 15 2020 at 16:53):

In that hope, understanding what people want to use this for will help identify what all we need to worry about

Jenni Syed (May 15 2020 at 16:53):

IE: does dynamic topic query qualify as a "must have" ?

Jenni Syed (May 15 2020 at 16:54):

Or would agreement out of band work for that? IE: here are the canonical "topics" I support (meets the ADT use case), and an extension (there would be more than one) on subscription to define which they are subscribing to?

Isaac Vetter (May 15 2020 at 16:54):

Jenni - I think that there really is a narrow, do-able path for a backport, involving: rest-hooks only, id-only only, canonical topics only and Notifications. cc/ @David Pyke
Would it make sense to talk about this today?

Jenni Syed (May 15 2020 at 16:55):

It would be great if we could find a time. @Gino Canessa @Lloyd McKenzie @Grahame Grieve @Keith Boone not sure if you all are available or interested?

David Pyke (May 15 2020 at 16:55):

I'm good with where things stand. As we've end-run the need for the project I'm working on and will move to R5 subscriptions when released, I'm happy.

Gino Canessa (May 15 2020 at 16:55):

I'm cleaning up a list of issues I've run into while tinkering with the backport stuff. I can post it here and hopefully we can chat.

Gino Canessa (May 15 2020 at 16:56):

I'm fairly flexible after about the next hour.

Lloyd McKenzie (May 15 2020 at 17:08):

Availability is uncertain. My primary objective is to be able to point to something that's implementable in R4 that EHRs are actually likely to roll out (given that the consensus seems to be that the subscription approach defined in R4 is problematic). Some of Da Vinci just needs to be able to monitor for the response to a request (so Task or maybe something else). The Notification project has a bit wider requirements

Grahame Grieve (May 15 2020 at 17:54):

I'm interested, but it's not clear whether i missed this?

Gino Canessa (May 15 2020 at 18:01):

Hasn't had an agreed-to time yet

Gino Canessa (May 15 2020 at 18:01):

We talked some in the earlier meet-time, and would welcome more at the next at 5:00 EDT.

Grahame Grieve (May 15 2020 at 18:11):

well, I'm awake and I don't yet have other break outs

Gino Canessa (May 15 2020 at 18:14):

Sounds good. Jenni is in another meeting for the next hour or so, but the room is open for discussion any time beyond the check-in as well.

Eric Haas (May 15 2020 at 20:01):

We have talked about this ADT using Backported Subscriptions specifically for DaVinci Notifications next R4 version. So I would like to be involved

Jenni Syed (May 15 2020 at 20:03):

We can talk about this at our normal 5 EDT Subscription check in if you all can join @Grahame Grieve @Eric Haas @Isaac Vetter @Lloyd McKenzie

Jenni Syed (May 15 2020 at 20:04):

zoom link so you don't have to go to the track page: https://zoom.us/j/2565638242

Jenni Syed (May 15 2020 at 21:04):

@Isaac Vetter were you going to be able to join or conflicts?

Jenni Syed (May 15 2020 at 22:19):

Here's my attempt at a summary of the May 2020 Connectathon break out call where we discussed this:

• Use cases are primarily to get subscription functionality as fast as possible - waiting until R5 is final won't work
• It is ok to target to specific use case needs
  • Limit to RestHook and email (public health and ADT notifications, DaVinci needs fit here)
  • The SubscriptionTopic agreement can be out of band - not dynamic as R5 allows today.
  • limit to id-only and empty
• Goal: A small IG that defines how to do this
  • limited to use cases above
  • Target date is Sept. DaVinci (or other IGs) can reference the Ballot version.
  • Done as part of the Subscription PSS, but will need to follow IG proposal process
  • Will need to be wrapped up by June to meet Sept timeline
  • @Eric Haas will help with the IG editing/process
  • @Gino Canessa @Isaac Vetter and @Jenni Syed will work on getting examples drafted for extensions on Subscription and the notification (history bundle) with extensions vs. what Basic would look like.
• Challenges
  • History bundle may be wonky still. @Grahame Grieve will look at what we propose and see what other options we may have.
  • We think the Topic canonical URL can be provided on Subscription as an extension to indicate topic, could reuse criteria for filters?

Jenni Syed (May 15 2020 at 22:19):

Let me know if I missed something you all want in there or misstated

Isaac Vetter (May 15 2020 at 22:24):

just a clarification, we'd also limit to "id-only", right?

Gino Canessa (May 15 2020 at 22:27):

I'd lean towards empty and id-only since that's what we landed on for all the other Argonaut pieces (long discussions around them)

Jenni Syed (May 15 2020 at 22:28):

I added it above, but @Eric Haas may need to confirm

Eric Haas (May 16 2020 at 02:22):

(deleted)

Eric Haas (May 16 2020 at 02:43):

I agree that this IG should focus on empty and id-only but disagree that the we constrain out resource. I was under the impression the IG would talk cover the content model and transactions but all the business externalities that cover safety and security and risk are version agnostic and don't need to be rewritten. I think that there will be use case where they want to push resources.

Gino Canessa (May 16 2020 at 15:38):

For me, the concern is what it means to be compliant with the IG. I don't intend to mean full-resource is be prohibited, but rather that empty and id-only are required for compliance, while servers may choose whether or not to implement full-resource.

This is based on the feedback that many servers do not intend to support full-resource until more of the security implications are understood.

Eric Haas (May 19 2020 at 03:45):

R4 Subscriptions next steps...

1. agreed to publish R4 IG ASAP (FHIR-I/Argo) - limited scope for R4
2. Gino to write/I'll help edit and publish it.

What about the ADT use case? :

• add to Argo guide or DaVinci?

Isaac Vetter (May 19 2020 at 03:49):

Eric, I'd suggest that adding this to the da vinci guide would have the broader impact.

Gino Canessa (May 27 2020 at 19:01):

Backport notification question: while talking with @Josh Mandel earlier today, he brought up the idea of making the notifications closer to the R5 versions by making Bundle.entry[0] be a Basic with all the extensions we need instead of putting them on Bundle.meta.

I feel like it's more in line with R5 (easier for consumers to migrate), and will allow extensions on the root level instead of in meta. But, it's a change from what's been discussed, so figured I would put it out there... Thoughts?

Grahame Grieve (May 28 2020 at 00:22):

sounds good to me

Grahame Grieve (May 28 2020 at 00:22):

though you could also consider Parameters

Eric Haas (May 28 2020 at 18:11):

Parameters in R4 in Normative and states:

This resource is a non-persisted resource used to pass information into and back from an operation. It has no other use, and there is no RESTful endpoint associated with it.

So this would be breaking our own rules. ( not that we have not done that before)

Vassil Peytchev (May 28 2020 at 18:23):

There is FHIR#26368 and the subsequent FHIR#26390 that try to address the use of Parameters beyond operations. This could be a use case for it, but arguably not in an IG based on R4.

Gino Canessa (May 28 2020 at 18:34):

Hmm... I think I'd like to propose that we try to avoid such prohibitive text in the future.

Playing with all the variations, I've liked Parameters the best so far. I guess that makes it a question for the group-at-large: can it be used here? I can see the argument either way: the text prohibits it, but can an IG override?
@Grahame Grieve @Lloyd McKenzie
Edit: or can this be ignored?

Grahame Grieve (May 28 2020 at 19:20):

We already agreed that the text Eric quoted is too narrow, though I think that this is a case that meets that intent anyway

Grahame Grieve (May 28 2020 at 19:20):

so that text should not prevent using Parameters here

Gino Canessa (May 28 2020 at 19:21):

If people are comfortable with it, that works for me.

Gino Canessa (May 28 2020 at 20:38):

Super early pass of the backport IG is online at GH. Only parts of interest right now are the profiles, since documentation is essentially nonexistent (only partially-bulleted out).

Have a voice! Thoughts and feedback welcome/encouraged.

Jenni Syed (May 29 2020 at 16:03):

For those of you reviewing, I took a stab at an example Subscription in R5 (snapshot 2) and what it would look like in R4: https://github.com/argonautproject/subscription-backport-ig/issues/1

Jenni Syed (May 29 2020 at 16:04):

We would need to describe how to create the criteria... I assume it's using the resulting resource in Topic with the filter as query params, so that's what I modeled there

Gino Canessa (May 29 2020 at 17:07):

I started adding the examples. If you see anything wrong, please let me know.

Gino Canessa (Jun 02 2020 at 17:26):

Question: writing up the finer details on what's required for conformance with the guide vs. what is described and optional. Right now, I am leaning towards the same restrictions we came up with during the Argonaut calls: empty and id-only, rest-hook are required, and everything else is optional.

Thoughts?

Jenni Syed (Jun 02 2020 at 19:22):

ok, so out of the body/content you get, empty and id-only are required for the server to support, and then for channel type rest-hook?

Jenni Syed (Jun 02 2020 at 19:22):

I don't know if I realized id-only was actually required :) lol.

Jenni Syed (Jun 02 2020 at 19:26):

I assume that's the argo-specific one since the spec doesn't require it.

Gino Canessa (Jun 02 2020 at 19:53):

Yeah, just trying to open discussion on it really. We could say that any one supported combination is valid as well.

Grahame Grieve (Jun 04 2020 at 03:23):

Looking at @Jenni Syed's example, I wonder why http://hl7.org/fhir/us/subscriptions-backport/StructureDefinition/backport-topic-canonical is a modifier extension? What does it modify?

Jenni Syed (Jun 04 2020 at 13:58):

I was originally wondering that as well, but realized it changes the interpretation of the criteria field

Jenni Syed (Jun 04 2020 at 13:58):

IE: you can't just understand criteria to be able to send the notifications that the client requested

Gino Canessa (Jun 04 2020 at 17:41):

My thought process is that linking to a SubscriptionTopic modifies the intention/processing of many other fields. It changes the meaning of the subscription in a substantial way and if you cannot process the extension (e.g., you don't support topic-based subscriptions), you should not attempt to interpret the subscription request.

Grahame Grieve (Jun 04 2020 at 20:01):

you can't just understand criteria to be able to send the notifications that the client requested

This is not the same as 'it's a modifier extension'. Does it make the interpretation of the criteria wrong? My interpretation (of the undocumented example) was the criteria is incomplete, not interpreted differently

Gino Canessa (Jun 04 2020 at 20:26):

At a high level, I thought that changing the processing from R4-style to R5-style would change the meaning enough to merit a modifier (since the resource has different meaning/must be interpreted differently). If that isn't the case, I'm fine with changing it back to a regular extension.

My thinking is that if a server receives one of these subscription resources and don't use topics, the behavior will not match what the client expects (e.g., what is sent to the endpoint, when it is triggered, etc.). I'm not sure what that entails across each endpoint type and if it would cause problems or not. Since the subscription 'looks' otherwise valid, the server may begin processing it immediately.

Grahame Grieve (Jun 04 2020 at 20:38):

can we list out the ways in which it would be wrong explicitly? That will help decide the question

Gino Canessa (Jun 04 2020 at 21:04):

Sure, here's a quick list (for the sake of current discussion). This type of info should be included in the IG as well, but I am working on testing the implementation before I spend too much time documenting it (e.g., most of the info is still just in artifacts).

• Server-side
  • Without a Topic, the criteria is evaluated as dataset to look for changes.
  • This may cause performance issues for the server
    • Trying to trigger on every change to every observation (expecting the topic to filter for a particular set of interest)
  • This may create a subscription for a completely different intention
    • Particularly around method-based topic definitions (e.g., topics specifying 'delete' or 'create'), this will produce incorrect behavior
• REST
  • R4 subscriptions send an empty post to the endpoint. Backported REST notifications POST a bundle.
  • Servers expecting a bundle will not process an empty post
    • The server will think it is correctly sending notifications and the client will not be receiving them
• Email
  • The expectations around email are established differently in R5/backport (e.g., body is human-readable, attachments are machine-readable).
  • Paired with a server (potentially) sending too many messages, a server may get blacklisted, etc.
• Messaging
  • This wasn't well defined in R4 either, so servers may have different implementations vs expectations

Grahame Grieve (Jun 04 2020 at 21:11):

so... the email thing doesn't sound different to me. At least, no different for my server. The Messaging thing doesn't seem clear. The server side things:

• performance issues - that doesn't change anything, so far as I can see. The issues already exist...
• different intention. Because things might be missed?

REST - so you're saying that the presence of that extension triggers different server behavior when sending notiifcations?

Eric Haas (Jun 04 2020 at 21:23):

@Gino Canessa I been looking at the build and examples and playing with a simple custom liquid template for better visualization ...
image.png

Eric Haas (Jun 04 2020 at 21:25):

I'll make a pr and create on for the bundle profiles as well

Gino Canessa (Jun 04 2020 at 21:46):

Grahame Grieve said:

• performance issues - that doesn't change anything, so far as I can see. The issues already exist...
• different intention. Because things might be missed?

A topic will often specify a different intention from the criteria. Blah, docs say you can have tables here, but they aren't working for me:

Topic Trigger                    | Criteria | Result
---------------------------------+----------+-------------------------------------------------------
Delete on Patient                | None     | Notification for every patient Created or Updated
Create on Patient                | None     | Notification for every patient Created or Updated
Create on vital sign observation | None     | Notification for every observation created or updated

REST - so you're saying that the presence of that extension triggers different server behavior when sending notiifcations?

Yes, a major part of the changes were to allow for both subscription information and different levels of content(empty, id-only, or full-resource) in notifications. The IG defines how to do that here.

Gino Canessa (Jun 04 2020 at 21:47):

Eric Haas said:

I'll make a pr and create on for the bundle profiles as well

I'm still using SUSHI for generating everything, so please be aware of that for the PR

Grahame Grieve (Jun 04 2020 at 22:16):

The presence of a topic extension changing the behavior of the channel doesn't sit well with me; we should be more explicit, I think.

Grahame Grieve (Jun 04 2020 at 22:17):

waht does Criteria = none mean?

Gino Canessa (Jun 04 2020 at 22:21):

Just highlighting that with a Topic, no criteria is valid

Grahame Grieve (Jun 04 2020 at 22:21):

but it's a required element...

Gino Canessa (Jun 04 2020 at 22:21):

Could just as easily be an additional filter (e.g., for a specific patient)

Gino Canessa (Jun 04 2020 at 22:22):

Hmm.. missed that. Would likely want to specify something like just the base resource in that scenario (e.g., allow Encounter for the criteria)

Grahame Grieve (Jun 04 2020 at 22:23):

so, there must be a criteria, and it must minimally specify the resource(s) that will appear in the channel

Grahame Grieve (Jun 04 2020 at 22:23):

is that a reasonable position?

Gino Canessa (Jun 04 2020 at 22:23):

That sounds right to me.

Grahame Grieve (Jun 04 2020 at 22:24):

so the topic specifies additional filtering applied over and above the criteria

Gino Canessa (Jun 04 2020 at 22:25):

For R5, the other way around - but yes. The distinction is important because implementers can optimize the topic-level filtering based on what they support, with the subscription filtering working on the reduced set.

Grahame Grieve (Jun 04 2020 at 22:26):

I don't follow that. I'm not interested in what's happening behind the scenes, only in the API. The topic applies additional filtering to the criteria - how is that not true?

Gino Canessa (Jun 04 2020 at 22:27):

Yes, as far as the API is concerned it's the overlap of the two sets of filters.

Grahame Grieve (Jun 04 2020 at 22:28):

with additional complexity around delete/modify, since in the absence of a topic, a criteria can't pick them up.

Gino Canessa (Jun 04 2020 at 22:30):

Yes. Also, the criteria is restricted per topic - e.g., if a topic only allows Encounters filtered by patient and only by equality test, the server should reject a subscription using unsupported criteria.

Grahame Grieve (Jun 04 2020 at 22:31):

I don't actually understand that. I assume that somewhere the relationship between what the topic says and what the criteria can say is documented clearly

Gino Canessa (Jun 04 2020 at 22:31):

It obviously wouldn't have to, but that was one of the goals for allowing servers to reduce complexity and yet have good communication around what is supported

Grahame Grieve (Jun 04 2020 at 22:32):

so let's say that a client posts a subscription to a server. The server is running unmodified R4 subscription semantics. The client adds a topic extension (not a modifier extension, in this thought bubble), but the server doesn't know about that extension, and so it ignores the topic, and proceeds as if it doesn't exist.

In this case, the client can immediately tell that the subscription is being processed wrongly when it receives the first notification, right?

Gino Canessa (Jun 04 2020 at 22:32):

Yes, in Subscription.canFilterBy

Gino Canessa (Jun 04 2020 at 22:33):

The client likely wouldn't get the notification, even though the server thinks it went through fine. My implementation wouldn't forward a POST with no body - it doesn't have context to do so.

Gino Canessa (Jun 04 2020 at 22:33):

So, the server thinks all is well and continues sending and the client just never gets a notification.

Grahame Grieve (Jun 04 2020 at 22:34):

I don't understand why the client doesn't get a notification. The server is sending something through the channel that the client nominated, no?

Gino Canessa (Jun 04 2020 at 22:35):

In R4, a rest-hook subscription does a POST with no body. In R5 (and the backport), the body is always a Bundle. Without a bundle, my server has nothing to forward.

Gino Canessa (Jun 04 2020 at 22:36):

(sorry to run - family is waiting on dinner, will check back after)

Grahame Grieve (Jun 04 2020 at 22:36):

I don't follow. I'm not asking about your server, I'm asking about a client that expects the new subscription behavior connecting to a server that supports the old behavior

Gino Canessa (Jun 04 2020 at 23:23):

Yep - there is a (client) http server that is hosting the REST endpoint.

Grahame Grieve (Jun 05 2020 at 00:04):

right. and that's under the control of the client through some private arrangement. So as soon as it gets the first notification, it knows that the server doesn't support the new subscription framework.

Gino Canessa (Jun 05 2020 at 00:24):

Fair enough. I have been keeping the http server as dumb as possible to emulate proxies (e.g., converting the POST to a push notification), but that could be added as guidance on that side

Grahame Grieve (Jun 05 2020 at 00:24):

it could. The obvious problem is that the client doesn't know the server doesn't support topic support until the first notification. Which in some (corner) cases, never comes. Is this a problem?

Gino Canessa (Jun 05 2020 at 00:37):

Hmm...the flow is different. The client will never receive the handshake, so it could use some sort of timeout there to assume it's broken. That would still likely end up with manual resolution (e.g., the server will say it's working, the client will know it isn't).

We could just put the onus on the client to check the Capabilities Statement (haven't added that yet), and not send in the subscription if it doesn't find the IG.

Grahame Grieve (Jun 05 2020 at 00:39):

I think that latter is where I was heading towards. And I think that the extension is not a modifier. There is clearly risk here, but it's not around the statement that the topic is making, but in the interaction expectations of the client and server

Gino Canessa (Jun 05 2020 at 00:40):

It still feels odd to me that it doesn't use a modifier, given that it is changing the meaning and the behavior, but :shrug:
In that case, it probably makes sense to move the topic to the criteria (where I had it originally) as an extension, yes?

Grahame Grieve (Jun 05 2020 at 00:40):

I

Grahame Grieve (Jun 05 2020 at 00:40):

I'm not sure

Gino Canessa (Jun 05 2020 at 00:43):

I can do a poll :grinning:

Gino Canessa (Jun 05 2020 at 00:56):

/poll Should the Backport IG have a ModifierExtension for linking to a topic?
Yes - It changes the intention of the subscription (criteria)
No - Extension on Subscription
No - Extension on Criteria

Eric Haas (Jun 05 2020 at 01:01):

I would keep as an extension on Subscription

Grahame Grieve (Jun 05 2020 at 01:02):

I think that's what (1) means. But @Gino Canessa your poll is either ambiguous or not binary

Grahame Grieve (Jun 05 2020 at 01:02):

(or both)

Gino Canessa (Jun 05 2020 at 01:04):

There, first no leaves it on subscription - I'd edit but it doesn't let me

Eric Haas (Jun 05 2020 at 19:30):

I was going to volunteer drafting the CapStatements., but what is the current status of this whole effort? The whole R4a thing is a huge potential monkey wrench from left field? I assume it is wait and see the outcome before devoting any more time to it?

Gino Canessa (Jun 05 2020 at 20:50):

I'm moving forward with making sure the models are there and working for now. If R4A happens and includes Subscriptions, I'll pivot then.
For the capabilities, I thought a base one for indicating core support (e.g., $topic-list, $status, etc.) and combinations for server support would make sense. E.g., separate statements for:

• rest-hook + empty
• rest-hook + id-only
  ...
  That way, a client can determine if the server generally supports the backport and what combination of channels/payloads it can work with.

With all that said, I understand if you'd prefer not to invest anymore time right now. Please just let me know so that we don't duplicate effort (capability checking is about two items down on my list right now).

Grahame Grieve (Jun 05 2020 at 20:58):

what combination of channels/payloads it can work with

That could be a profile on subscription, right?

Gino Canessa (Jun 05 2020 at 21:00):

Probably? Haven't gotten to it yet, so "profile on subscription" sounds like a good starting place to me.

Grahame Grieve (Jun 05 2020 at 21:01):

what options are there in supporting back port? What topics are allowed? Can you support back-port without any topics?

Gino Canessa (Jun 05 2020 at 21:10):

I have been thinking I would define all the options from R5 (rest-hook, email, websocket, messaging :: empty, id-only, full-resource) and letting servers establish from there. This would also show a model for if people wanted to extend into custom channels, etc.

For topics, there's an operation which returns a Bundle with SubscriptionTopic canonical URLs. Looking at it right now, I think I have them in the entry.fullUrl field, though they should be in entry.link instead.

I've been struggling with what 'minimum' compliance would look like, and so far have what I described above - compliance means you have the required operations and understand the extensions. This allows servers to only implement the parts they are planning on actually using (e.g., if I just need public health emails, I don't want to implement rest-hooks at all), while allowing clients to figure it out without the guessing game of what's supported (e.g., try every combination until you figure out what's supported).

Gino Canessa (Jun 05 2020 at 21:11):

I don't think any particular topic/s should be specified in the guide - just like having R5 support doesn't include a minimum set of topics

Gino Canessa (Jun 05 2020 at 21:15):

So far in R5, we've left topics out of capabilities altogether (other than resource support). Clients can query that resource in order to find desired topics

Eric Haas (Jul 21 2020 at 01:41):

FYI on the procedural end there is an HL7 PSS here to be reviewed by:

• FHIR-I (voted and approved)
• FMG

• Steering Committee and TSC

• Will move the repro to HL7 shortly - let me know if you want access to contribute ( along with you github name ( @Jenni Syed ?)

• Need a list of implementers / projects who intend to implement (only need 2)
• Next steps Notice of Intent to ballot , draft, connectathon, publish for ballot

Eric Haas (Jul 22 2020 at 20:29):

Project Punchlist

Last updated: Apr 12 2022 at 19:14 UTC