FHIR Chat · CDS Hooks feature request · cds hooks

Stream: cds hooks

Topic: CDS Hooks feature request

---

Matt Varghese (Sep 03 2019 at 23:47):

Hey guys - I have a few things we have identified as we need here at Epic, in working through some implementations.

#1 - A Card Identifier. This needs to do three things for us.

A. The Card ID tells us at some "abstract"/vague level what the card is about. So one card could be about "diabetes" and another card could be about the "chest-x-ray" the provider orders.

B. The Card ID can serve for deduplication of alerts. So for example, a health system could decide that I'm subscribing to both services X and Y. However, if service X and service Y both return alerts about "diabetes" I think they are duplicative and result in alert fatigue for the provider. Besides I know service X is a better source than service Y for "diabetes" related information. So if both returns "diabetes" cards, then I want to suppress the card from service Y and only show the card from service X.

C. The health system has already made a CDS Hooks request and is showing the cards to the provider. The provider thinks, because they did something to the patient after the CDS request, one of the cards shown is no longer up to date. They want to just selectively refresh that card. (or they simply, for no reason want to refresh that one card to get up to date info). This needs a separate field in the request as well - I'm writing up a separate comment on selective refresh below.

It could be argued even that there are two concepts being intermixed here - a Card ID and a Card Category. Open to feedback; let me know how to proceed with this.

Matt Varghese (Sep 03 2019 at 23:51):

#2 - A request field for specifying what cards to refresh.

As stated above, sometimes, after the initial request for CDS, the EHR may want to selectively refresh some but not all the cards being shown.
This refresh can happen in two ways.
A - refresh (a) specific card(s), by specifying (a) Card ID(s) in the request, as stated in the previous issue.
B - refresh all cards of (a) specific indicator type(s) (indicator field in the card is defined as either info, warning, or critical.)
C - possibly there are other refresh types I'm not considering?

How can we incorporate this in the CDS Hooks request? Ideally, I don't want to do a full refresh and drop specific cards (and even dropping specific cards needs a Card ID field.) since then I'm dropping CDS information sent back by the service into the void.

Matt Varghese (Sep 03 2019 at 23:54):

#3 - Flexibility on selectionBehavior of Suggestions in the card.

Right now, the Suggestion + Action structure doesn't really work for our use. We are planning to add a constraint that each suggestion only contain one action for our initial implementation of Suggestions because of this. So a CDS Service can return N different suggestions. And we want the provider to be able to select a subset of them. This means, the value set for selectionBehavior (currently only 'at-most-one') also does not work for us. Can we add a new value to the valueset: Any, which implies the provider can accept none/some/all of the suggestions returned?

Matt Varghese (Sep 03 2019 at 23:57):

#4 - finally, given the above Any option for the selectionBehavior, we would like to specify a default selection status for each suggestion.

Basically, when there are multiple suggestions, some suggestions are "preferred"/"recommended" suggestions, and others are "alternate"/"less-preferred" suggestions. So we want to mark the preferred suggestions, so that when the card is returned, the preferred suggestions are selected by default, and the provider can accept the preferred suggestions by a single click. They only need to tweak selections etc, if they want to disregard the preferred suggestion and go with an alternate suggestion returned by the CDS Service.

Matt Varghese (Sep 03 2019 at 23:58):

Writing these up so that I can have your feedback on what is the best way to accomplish the objectives. Please provide feedback here, but I will also be present at the connectathon if anyone would like to talk more.

Lloyd McKenzie (Sep 04 2019 at 00:30):

My opinions (for what they're worth)
#1 - I'm ok with a card id, but I can't imagine a situation where there could be meaningful de-duplication except from a single server. Servers might well provide all sorts of different information about diabetes - protocol recommendations, rules for diagnosing, tests that should be performed, etc. You could use the id to suppress a particular card for that patient (possibly for a period of time) or independent of the patient forever if the information is particularly useless/annoying. But the notion that we could come up with an ontology that would be specific enough to allow de-duplication across servers seems far-fetched to me given the potentially infinite set of recommendations CDS systems might come up with. And if we did get one that was sufficiently all-encompassing, the likelihood of everyone coding recommendations consistently would also be poor. I'd very much like to see this in action (particularly across non-coordinating CDS systems) using extensions before we consider semantic identifiers.
#2 - My understanding was that the service gets hit every time a field changes, so all the cards should effectively get refreshed pretty much all the time and nothing should be out of date. If an action results in changes to the resource in question, that should certainly trigger a refiring of the hook services, no? (I wouldn't expect most clinicians to be paying enough attention to the cards they're getting back to notice if they've changed and expecting a couple of additional clicks to force refresh of each card seems like a lot of imposed cost on the users for something that definitely shouldn't be the user's problem.)
#3 - There are good reasons to have multiple actions in a single suggestion. If you want to replace a record with a new record, that's a delete + an add (2 actions). In CRD, one of the suggestions is to fill out a Questionnaire (create questionnaire + create task). If you have multiple actions for a single suggestion it's because they must be executed together to make sense. Fine with adding clarification to that effect, but not so fine with prohibiting multiple actions per suggestion.
#4 - In practice, there can be logic to the suggestions where some are mutually exclusive, some could be done together, etc. The notion of 'preference' could be challenging in a lot of those situations. Also, 'preference' implies that the service knows what's better than the practitioner - which a) might not be true; and b) might be true, but not appreciated by the practitioner. My take on the 'default' for CDS Hooks was always "ignore the cards and continue with what you were doing" - particularly when the cards may well change as you continue filling out the screen.

Matt Varghese (Sep 04 2019 at 02:23):

@Lloyd McKenzie - as I mentioned, these are all issues that came up from real world implementation efforts going on right now. Epic does not, ideally, want to go ahead and make extensions to support this, since we think these are valuable to others as well and so merit being standardized.

#1 - We actually have some meaningful deduplication examples which were designed with input from CDS Service and are being developed right now for real world use with multiple sources of decision support. Granted, this would require communication between services, a mix of EHR configuration for mappings, and CDS Hooks card ID support - so I'm also not expecting a complete ontology, and in your terminology this would not be a non-coordinating CDS system, but coordinating CDS systems are rather common in the real world.

#2 - EHRs usually have workflows where only a subset of decision support may be refreshed. These are usually actively triggered by a clinician's interaction, so that there is no risk of the clinician not noticing. Essentially this is a clinician driven refresh of a single specified card / single indicator type group that the clinician clicked on. This is existing EHR behavior that clinicians are used to, that we need to make sure works with CDS Hooks as well.

#3 - We cannot currently support multiple actions inside a suggestion. So Epic's initial implementation will constrain that there be only one action within a suggestion, but there can be multiple suggestions in a card. Eventually we plan to take this limitation away so both ways become supported.
#3.1 To be clear, I am not proposing prohibiting multiple actions. We also want to support that eventually, and we really think it is a very valuable thing. I am only asking that both options / both ways of doing things be allowed. selectionBehavior can retain existing value of 'at-most-one', plus asking to add a new value of 'any' (or some other equivalent word) representing multiple selections being allowed.
#3.2 Also note that, what I am asking for is how EHR decision support works in many examples out in real world today (not driven by CDS Hooks, but built natively in EHR) - so what CDS Hooks has formulated doesn't fit existing decision support implementations / what clinicians are used to. What I'm asking for is one way to bridge the gap - open to other ways to solve the problem as well.
#3.3 For example, if I place an MRI, and the CDS comes back and says use a CT preferably or an X-Ray, but not an MRI. Then there are two ways to represent this in suggestions:
#3.3.1 with a selectionBehavior of 'at-most-one', there will be a suggestion of CT+remove MRI, and another suggestion of X-Ray+remove MRI.
#3.3.2 with the new proposed selectionBehavior of 'any', there will be three suggestions: CT, X-Ray, and remove MRI. The CT and the remove MRI will be selected by default, but the X-ray being the less preferred suggestion, will not be selected by default. So if the clinician just accepts the Card without tweaking suggestions, they place a CT and remove the MRI.

#4 The concept of a preference is already baked into existing decision support mechanisms (EHR-native) that are out there in real world use. So we are not introducing anything new to a practitioner, but rather introducing something that gels comfortably into what they are already used to.
#4.1 Even in 3.3.1 above where I have the example with the current CDS Hooks way of doing things, it is still extremely useful for the CDS service to be able to mark that the CT+remove action is preferable to the X-Ray+remove action in the 'at-most-one' case (as I understand there are real world use-cases which can actually make such distinctions based on radiation exposure, implants that the patient has etc.). EHRs will very likely default to making the first suggestion be the preferred suggestion if an explicit marking of the preferred suggestion is not supported. But I would say, explicit marking makes for a better standard than allowing arbitrary behavior like the first suggestion becoming the preferred suggestion.

Grahame Grieve (Sep 04 2019 at 02:44):

t could be argued even that there are two concepts being intermixed here - a Card ID and a Card Category

I definitely think that. id needs to be some kind of generated UUID thing. category needs to some managed code from some useful ontology that doesn't currently exist (and feels like a nightmare)

Matt Varghese (Sep 04 2019 at 02:52):

@Grahame Grieve Category is attempting to capture the set of types of alerts that a CDS Service can generate. Therefore, I would say it is preferable to make it a string and leave it to the CDS service, or make it a code where there are code systems defined by each CDS Service. I can't imagine a universal ontology - each CDS service is going to differ in how they categorize alerts.

Matt Varghese (Sep 04 2019 at 02:52):

Also, I don't think CDS Hooks should bring it under its scope to start defining clinical content such as code-systems/ontologies etc.

Grahame Grieve (Sep 04 2019 at 02:56):

I certainly wasn't proposing that we do it. but if you're going to try and do meaningful de-duplication based on the category code (however represented) then someone has to impose some consistent meaning on it; you can't just leave it to the CDS service - all their incentive is to game the system by choosing a code no one else uses

Matt Varghese (Sep 04 2019 at 03:00):

What about making it a code? Then different vendors can define different code systems, and they can work together with the EHR for the translations. This is already how things are done in many other situations.

Also, I would say the statement "all their incentive is to game the system by choosing a code no one else uses" comes across as rather harsh. At least all services that I have worked with, and afaik all that are used in the real world, are pretty good at cooperating and collaborating to make really cool things happen.

Grahame Grieve (Sep 04 2019 at 03:04):

yes ok it was a bit harsh. But I had just read this: https://twitter.com/jasonfried/status/1168986962704982016, so that colored my opinion a little. But nevertheless, it needs some curation centrally. So yes, it needs to be a code of some kind. And with multiple supporting code systems.... yes....

Matt Varghese (Sep 04 2019 at 03:07):

Ok - I agree code will meet the need and avoid many of the unfortunate situations that could potentially arise out of a plain string.

Matt Varghese (Sep 04 2019 at 03:08):

Maybe the difference is that these CDS Services needing to be configured in the EHR, if they are malicious actors, the Health System will just end their contract - so there is good incentive on the part of the CDS Services to play good actors.

Grahame Grieve (Sep 04 2019 at 03:09):

so I think that's a thing that's true for your use of CDS hooks, but not true in other uses outside pay-to-play EHRs. EHRs are obviously an important use case, but not the only one

Matt Varghese (Sep 04 2019 at 03:12):

Agreed - but I'm only asking category in the standard. How that is used to deduplicate isn't baked into the standard. So the category can be optional, and in a non EHR case it can be ignored. Is that fair?

Matt Varghese (Sep 04 2019 at 03:15):

Also, for CardID being a UUID, can we say then that if nothing has changed so that the same card is being returned by the service, the UUID will/can remain the same?

Grahame Grieve (Sep 04 2019 at 03:17):

well, i'd just make it an id and say that if the CDS service returns an id, it must be something it recognises when it's returned in the request, and then say what it functionally means in the request

Matt Varghese (Sep 04 2019 at 03:18):

Ok - I think I can make that work.

Grahame Grieve (Sep 04 2019 at 03:18):

and I think yes about the category, though I'm not sure whether it should be system+code (like a FHIR coding) or a URI (which is kind of system#code)

Matt Varghese (Sep 04 2019 at 03:19):

Open to either - my preference would be to keep it like it is done in FHIR.

Grahame Grieve (Sep 04 2019 at 03:21):

so would mine, but Coding isn't directly available

Matt Varghese (Sep 04 2019 at 03:22):

Well, everything in CDS Hooks is custom JSON anyway? Except for the JWT and the contained resources.

Grahame Grieve (Sep 04 2019 at 03:22):

yes so just have to define an object with system/code properties - looks the same, and refer to FHIR Coding for the entire edifice it sits in front of

Matt Varghese (Sep 04 2019 at 03:23):

I can agree with that.

Grahame Grieve (Sep 04 2019 at 03:26):

then of course I ask the logical corollary... is only one enough? .. :unamused:

Matt Varghese (Sep 04 2019 at 03:27):

That depends on how complex we want to make this.

Matt Varghese (Sep 04 2019 at 03:31):

I am open to multiple. I don't necessarily need multiple. But maybe someone else does?

Grahame Grieve (Sep 04 2019 at 03:31):

typically, we define a singleton and wait for someone to protest

Matt Varghese (Sep 04 2019 at 03:32):

Okay - I'm fine with that strategy.

Lloyd McKenzie (Sep 04 2019 at 04:12):

I'm not totally clear on why Epic can't support multiple actions within a suggestion - is it issues with the transactional nature of both must succeed? You should provide feedback to the CRD specification because right now, two very key suggestion types depend on multiple actions. (And I don't expect the need for them will be limited to CRD.)

The concern about "preference" is that if you're calling 5 different decision support services and each of them are providing cards, what does the notion of "preference" mean? Sure, if all the cards are coming back from one place, you could indicate preference if that's sensible in the context of what's being returned (i.e. the actions are intended to be mutually exclusive). My my expectation is that most of the time there will be numerous services involved. And at that point, the notion of "preferred" is going to become pretty meaningless.

Matt Varghese (Sep 04 2019 at 12:53):

@Lloyd McKenzie Every EHR ideally wants to integrate CDS Hooks into existing decision support mechanisms. So major differences from how existing decision support mechanisms work will require a much larger development effort. Like I said, we also want to eventually support multiple actions within a suggestion and see it as desirable; however, that is not how things work today. Also we really feel most multiple action situations can be equivalently accomplished by single actions as I showed with the example, maybe that won't be as nifty as with a grouped action, but it meets the need. This is really not in my power to change, I'm trying as best as I can.

Preference exists only within a card. So I don't think I follow your concern with preference.

Josh Mandel (Sep 04 2019 at 19:41):

Catching up here --

1) regarding a card ID, having a semantically meaningless unique id is fine/feasible; trying to use this ID for context-sensitive de-duplication is like requiring the ID semantics to "read your mind" (i.e., it's expecting the id to reflect exactly whatever hard-to-pin-down de-duplication semantics you want to enforce). Seems infeasible to me.

2) With respect to multiple suggestions per card (selectionBehavior), I'd be in favor of adding all and any (in addition to at-most-one), and making this a required field; right now, it seems like "all" semantics only happens if you omit the field.

Matt Varghese (Sep 04 2019 at 19:53):

@Josh Mandel
1. In the conversation with @Grahame Grieve above, we are saying there will be Category which will be a Code-System+Coding and can work for deduplication (but only with co-operating services and additional configuration in EHR to map different codesets), and there will be a separate UUID which must be recognizable to the CDS Service if returned in a subsequent request.
2. selectionBehavior is already required field. From the specification (Card Attributes, text against the suggestions property)

If suggestions are present, selectionBehavior MUST also be provided.

So sending suggestions without a selectionBehavior is currently precluded.

Josh Mandel (Sep 04 2019 at 19:56):

Wait, I see

this content at https://cds-hooks.org/specification/1.0/#http-response

Josh Mandel (Sep 04 2019 at 19:56):

Ah. "If suggestions are present, selectionBehavior MUST also be provided" is what I was missing!

Josh Mandel (Sep 04 2019 at 19:56):

Thanks.

Josh Mandel (Sep 04 2019 at 19:57):

So to restate: I'd be in favor of adding all and any (in addition to at-most-one).

Josh Mandel (Sep 04 2019 at 19:58):

(Incidentally, I'd prefer the word CONDITONAL to OPTIONAL for selectionBehavior. Will add a PR.)

Josh Mandel (Sep 04 2019 at 20:00):

https://github.com/cds-hooks/docs/pull/492

Matt Varghese (Sep 04 2019 at 21:40):

Agree with both the values for selectionBehavior and the CONDITIONAL phrasing.

Dennis Patterson (Sep 13 2019 at 14:08):

@Matt Varghese these are topics that would be worth discussing further this weekend. Outside of integrating and testing CDS Hooks workflows at the Connectathon, let's try to talk through some of your use-cases, whether in a break out session or otherwise. Sound good?

Matt Varghese (Sep 13 2019 at 14:42):

@Dennis Patterson Yes, that sounds good.

Jacob Crump (Sep 14 2019 at 15:40):

I'm not sure if this has been brought up, but in the fhirAuthorization object, the key names are in snake case format (i.e. access_token). Why are the key names not camel cased like all of the other fields in the specification (i.e. accessToken)? Is there sound reason for this difference or is this something that could be changed for 1.1? It just seems inconsistent in my limited lens of view.

Isaac Vetter (Sep 14 2019 at 15:50):

Hey Jacob, I think it's because those are names defined by OAuth.

Jacob Crump (Sep 14 2019 at 15:53):

Gotcha that makes sense. I just found the rfc documentation for anyone else interested: https://tools.ietf.org/html/rfc6749. Thanks Isaac.

Matt Varghese (Oct 09 2019 at 15:32):

Hey folks, I've created a few pull requests, based on this thread:

• ruleCode (https://github.com/cds-hooks/docs/pull/505) based on the discussion about the Card Category above, as well as some discussions at the Connectathon.
• uuid (https://github.com/cds-hooks/docs/pull/496) to add a Card ID to the card.
• (https://github.com/cds-hooks/docs/pull/498) adds selectionBehavior value any.
• preference indication (https://github.com/cds-hooks/docs/pull/497) on suggestion.

Frank Pandolfe (Jan 20 2021 at 21:30):

@Matt Varghese Has there been any further discussion, decisions, consensus regarding the card.uuid? Is the UUID typically associated with the card or "instance of the card" i.e. as you had mentioned, if nothing has changed so that the same card is being returned by the service, does the UUID remain the same, or is it different?

Matt Varghese (Jan 20 2021 at 22:50):

Hi @Frank Pandolfe , my above requests have made their way into CDS Hooks, and have been in the spec for some time now.
https://cds-hooks.org/specification/current/#card-attributes

card.uuid represents a unique instance of the card.

We also have card.source.topic which is a coding object, and captures the type of card, or rule or logic that led to the generation of the card.

Frank Pandolfe (Jan 20 2021 at 23:28):

Thanks @Matt Varghese

Do you consider every time the card is issued (returned to the calling application) a unique instance of the card?

I have been thinking about the cards as more static objects. Generating a new UUID for the same card every time it is issued seems to present difficulties when trying to use the feedback for analysis. For example, if a service determines to return Card_A , for an A1c between 7 and 9 and Card_B for an A1c greater than 9. Both cards may have some unique suggestions but may also have a shared one (suggestion_1). Additionally, both of these cards will have the same topic (diabetes).

Since we are generating a new UUID every time the card is issued, do you have any recommendations regarding re-associating the UUIDs to the same card? How else would one try to determine how often suggestion_1 was taken for Card_A vs for Card_B?

Matt Varghese (Jan 21 2021 at 00:58):

@Frank Pandolfe , I personally don't have an opinion or preference here :( .

In my CDS Hooks client implementation, I assume that the CDS Service will make the determination if and when card UUIDs can stay the same, suggestion UUIDs can stay the same, what is the boundary between things that fall in two different topics etc. At least in my implementation, I am able to handle the same UUID returned across multiple invocations, but I cannot speak for the broader community, and will defer to others to chime in.

What I can say is, if I get the same value for topic, I treat that as the same conceptual card / template card. So for example, if a physician says "snooze this topic for the next four hours", I can skip presenting cards with that topic but different (or same UUIDs) for the next four hours even if the CDS Hooks response returned those.

Frank Pandolfe (Jan 21 2021 at 15:17):

Hi @Lloyd McKenzie . Based on some of your thoughts from the original thread, wondered if you have comments or thoughts you'd be willing to share. Thanks.

Lloyd McKenzie (Jan 21 2021 at 19:00):

The key thing for Da Vinci CRD is that there's a clear ability to audit on both sides - so an EHR can capture "we received card X from payer Y" and be able to verbally share an identifier for that card so they payer can find it in their own log. As such, for us, it's fine if the identifier changes every time. It's also ok if the identifier remains if identical information is sent multiple times in close proximity (e.g. during order create)

Frank Pandolfe (Jan 21 2021 at 21:16):

Lloyd McKenzie said:

The key thing for Da Vinci CRD is that there's a clear ability to audit on both sides - so an EHR can capture "we received card X from payer Y" and be able to verbally share an identifier for that card so they payer can find it in their own log. As such, for us, it's fine if the identifier changes every time. It's also ok if the identifier remains if identical information is sent multiple times in close proximity (e.g. during order create)

It seems like the combination of the UUID and a timestamp could help resolve the exact identify. Additionally, in your case the CDS Hooks service provider would have to log every time a card was sent, not just feedback received. i.e. server sent Card_A at 12:01 and the card is ignored on the client... no feedback is generated. The client could log the receipt of the card, but unless the server is also logging every card it sends, you're not going to find it in the feedback log. This could also possibly help with figuring out an "ignore rate" and potentially "dwell time"

Lloyd McKenzie (Jan 21 2021 at 21:30):

The notion of feedback didn't exist at the time we adopted CDS Hooks. We don't have an expectations to require use for that at the moment.

Matt Varghese (Jan 21 2021 at 22:04):

I don't know why it would be hard either way? Even if you have a card UUID that is unique to each instance / each response, wouldn't the service still be able to do analysis like that on the data?

Matt Varghese (Jan 21 2021 at 22:05):

In fact, I would wonder if "dwell time" isn't harder if the UUID is reused?

Frank Pandolfe (Jan 21 2021 at 22:56):

sorry... my stream of consciousness was probably hard to follow. I agree now that a UUID for every card instance is a good idea. I think the trick is the service provider logging the cards template name, the UUID and timestamp (along with any optional details that may differ from the card template) when the card is generated and sent to a client. If and when feedback is received from the client, that UUID row can be updated with the additional details from the feedback response. This will allow for ignore rate and dwell time. If the client chooses to keep its own details based on the UUID it can. Potentially it can be merged/compared with the log kept on the service provider, depending on the use case... i suppose possibly for auditing reasons.

Matt Varghese (Jan 21 2021 at 23:16):

@Frank Pandolfe , I agree with that.

---

Last updated: Apr 12 2022 at 19:14 UTC