Stream: smart
Topic: Web messaging, scratchpad
Josh Mandel (Feb 12 2021 at 17:52):
@Matt Varghese thanks again for the discussion in the call just now. One thing I realized I didn't understand about your proposal is how search would be used by an app if it is searching against the full state of the fhir server. When you have a scratchpad in place, that is necessarily and explicitly specific to a user session and only includes things like draft orders that the user has in their shopping cart right now. But if you are issuing a fhir API query, you might pull up a whole pile of draft orders that are part of other people's sessions, or abandoned ideas, or medical student suggestions, or other data that would not be relevant to a shopping cart review experience. How would the app query for just the "ephemeral" items relevant and visible in the current UI session?
Matt Varghese (Feb 12 2021 at 17:56):
Isn't that already covered in the Scopes for SMART on FHIR?
Chuck Feltner (Feb 12 2021 at 18:23):
I agree @Josh Mandel this is one of the main benefits of the scratchpad concept in the SMART Web Messaging spec. It shows the current items in progress for that specific user session.
Josh Mandel (Feb 12 2021 at 18:31):
Definitely not covered in existing SMART on FHIR scopes (which are tied to data a in a specific record, rather than session-scoped data)
Lloyd McKenzie (Feb 16 2021 at 23:44):
One of the side discussions Matt and I are having are about what I see as a fundamental property of the scratchpad (or "whatsit" as we're currently calling it as the name seems to be a source of conflict). I believe that information accessed from the scratchpad SHALL NOT be written by the receiving app to the REST API. I.e. if you get data from the scratchpad and you want to change it, you have to write it back to the scratchpad. I think this is also an unwritten assumption with CDS-Hooks. If you receive a resource in the CDS-Hook invocation, you're not allowed to write that to the RESTful server as part of your hook processing. You can pass back cards and recommendations, but the choice of what to write (and whether to write anything at all) to be exposed to the RESTful API remains in the launching system, never the launched app. @Matt Varghese believes the "SHALL NOT" is too strong a statement. Would welcome other's thoughts.
Matt Varghese (Feb 16 2021 at 23:46):
All draft MedicationRequest / ServiceRequest resources are candidates for showing up in CDS Hooks draftOrder context.
So such a take essentially excludes any MedicationRequest / ServiceRequest resource with status of draft from the REST-ful endpoint.
Lloyd McKenzie (Feb 16 2021 at 23:50):
No. Status of draft has nothing to do with it. If you are editing an order (regardless of what hook or what status) and you get that order from CDS Hooks, you SHALL NOT write the order to the RESTful endpoint within the hook service.
Matt Varghese (Feb 17 2021 at 00:02):
Are you saying what's in the scratchpad will never show on the REST-ful endpoint, or just that the App / CDS Hooks service shouldn't use the REST-ful endpoint to modify that?
Josh Mandel (Feb 17 2021 at 00:10):
I mean if a CDS Hooks service receives a resource as input to a request and then turns around and creates something similar at the client EHR's FHIR REST API endpoint... it is acting outside of the expected scope of its job as a CDS service, and this is a weird pattern that I would not personally encourage. But we don't go out of our way to prohibit all kinds of weird behavior.
Josh Mandel (Feb 17 2021 at 00:10):
So I'm not sure I would agree with imposing any specific prohibitory conformance language around it. But if this is something that developers are thinking about doing, we should certainly have language in the specification that helps them understand why this is not the expected pattern and steer deftly away from it.
Josh Mandel (Feb 17 2021 at 00:12):
As far as I'm aware this has never come up in real life; it's more of a hypothetical example that came up because of discussions about a different use case. I wouldn't worry too much about this happening in CDS Hooks Services unless you see some indication that it is a real world issue @Lloyd McKenzie . I have learned over time that I'm not very good at anticipating all of the strange and wrong things that people want to do with specifications; so I rarely try to prohibit them ahead of time :-)
Matt Varghese (Feb 17 2021 at 00:13):
I think Lloyd was saying the scrathpad and REST-ful endpoint are disjoint sets. But I'll let Lloyd clarify.
Josh Mandel (Feb 17 2021 at 00:16):
I'm not seeing Lloyd state that they are disjoint; just that a receiving system should never make assumptions of overlap, and should instead interact with the EHR using an explicit mechanism like a "SMART Web Messaging what'sit API" or a CDS Hooks suggestion/action, rather than assuming that it is okay to make changes by directly interacting with a fhir server.
Josh Mandel (Feb 17 2021 at 00:21):
I agree with this perspective. (it's just that I would stop short of prohibitory language because there may be cases where a system could knowingly and acceptably violate this rule, if arrangements are made out of band.)
Lloyd McKenzie (Feb 17 2021 at 00:21):
I'm saying that the purpose of the scratchpad is to make available objects that are (or at least may be) edited versions of the underlying resources - or which might not even exist yet. And the user of the system that sources those objects should have final say over whether they get written to the REST endpoint. In the case of CDS Hooks, that's done by passing back cards, the user interacting with the cards and then determining what (if anything) eventually gets stored. In the case of SMART Web Messaging, it's by writing information back to the scratchpad which the user can then determine what gets written back.
Josh Mandel (Feb 17 2021 at 00:22):
I agree 100% with that Lloyd
Lloyd McKenzie (Feb 17 2021 at 00:24):
Part of the discussion we've been having is "what happens if a temporary resource is written to the REST endpoint and subsequently made available over the scratchpad - is it ok to write to the RESTful resource". My answer is: if you get the data from the RESTful endpoint, you're free to write to the RESTful endpoint. If you get from the scratchpad interface and you want to make changes, you SHALL do that through the scratchpad.
Josh Mandel (Feb 17 2021 at 00:31):
I basically agree with this; I'd just worry that the strict "shall" might close off some potentially reasonable avenues that you could pursue with appropriate out of band arrangements in place.
Lloyd McKenzie (Feb 17 2021 at 00:32):
SHALL NOT (without out-of-band agreements)? :smile:
Josh Mandel (Feb 17 2021 at 00:35):
Yeah. Which is to say "SHOULD NOT" ;-)
Lloyd McKenzie (Feb 17 2021 at 00:36):
Not quite. SHOULD says You're free to do what you like, but best practice is not to. SHALL NOT (some qualifier) sets a specific circumstance for when the SHALL NOT can be violated.
Matt Varghese (Feb 17 2021 at 01:24):
I honestly don't understand how SHALL NOT is possible.
A pended order must still undergo decision support. So it can show up in CDS Hooks / is part of scratchpad.
Yet, it is also going to show up on the REST-ful endpoint as it is filed to the database.
Matt Varghese (Feb 17 2021 at 01:27):
To me, a pended order is an example of something that is in the intersection of scratchpad and REST-ful endpoint, thereby showing they are not disjoint.
Josh Mandel (Feb 17 2021 at 03:22):
Not quite. SHOULD says You're free to do what you like, but best practice is not to. SHALL NOT (some qualifier) sets a specific circumstance for when the SHALL NOT can be violated.
Agreed that's the technical distinction. But I'd rather have us specify "SHOULD" if we specify anything here. (To reiterate, I don't think we're talking about anything that has come up as a real world issue, just a theoretical concern.)
Josh Mandel (Feb 17 2021 at 03:23):
It doesn't matter if they are disjoint or not; what matters is that app and service developers have a clear expectation of how to interact with the EHR once a specific conversation has been started.
Josh Mandel (Feb 17 2021 at 03:24):
(if the EHR says "here is an order in progress" the app needs to know where and how to bring up proposed changes.)
Lloyd McKenzie (Feb 17 2021 at 05:01):
If an app receives something from the scratchpad API, it has no authorization to propagate what information it has to the RESTful API. If the information received from the scratchpad API happens to be identical to the version available from REST, the interaction is still expecting the information to be passed back to the scratchpad (which is how the launching system is prepared to receive it). If it instead gets sent back to the RESTful API, the launching system has no reason to expect it to be there and, at best, will end up out-of-sync with what's in REST.
Matt Varghese (Feb 17 2021 at 14:21):
So @Lloyd McKenzie , building upon your patient demographics example, what is the behavior expectation when App 1 is manipulating patient demographics via scratchpad and SIMULTANEOUSLY App 2 is manipulating patient demographics on the REST-ful endpoint?
Lloyd McKenzie (Feb 17 2021 at 14:33):
The invoker of App 1 will run into a collision if/when it chooses to apply the changes from App1because the ETag will have changed since it last queried due to the updates from App 2.
Lloyd McKenzie (Feb 17 2021 at 14:33):
The same as if App 1 had never been involved and the invoker of App 1 had been making changes on its own when App 2 cut in with an update.
Matt Varghese (Feb 17 2021 at 14:35):
So
(1) you're saying the same data could be manipulated on the REST-ful endpoint and scratchpad simultaneously, and the EHR has to manage this Access.
(2) what is stopping App 1 and App 2 being the same - one app capriciously accessing the scratchpad or REST-ful endpoint, and the EHR applying the same guardrails?
Lloyd McKenzie (Feb 17 2021 at 14:37):
- The EHR has to manage access regardless. There's always the possibility for resource contention - even if the only interface is REST.
- I've proposed a SHALL saying that it's non-conformant to behave that way. If we don't do that, then nothing prevents it, but you will end up with a frustrated end-user because their experience will suck.
Matt Varghese (Feb 17 2021 at 14:39):
Based on (1) I'm still not sure I understand there is a scratchpad different from the REST-ful endpoint.
Matt Varghese (Feb 17 2021 at 14:41):
So I'm still not sure what the scratchpad is, and that could use more definition.
On the other hand, if we say, SMART Web Messaging has an EHR endpoint that provides managed access to work in progress resources, I can see that being reasonable.
Lloyd McKenzie (Feb 17 2021 at 14:47):
The scratchpad is the stuff that isn't committed to the REST endpoint yet. System 0 does a read, makes changes in memory, launches app 1 passes the 'dirty' resource, app 1 makes further changes, passes back to System 0, user of system 0 may adjust further and eventually commits to the RESTful endpoint. Only in that last step does a new version get exposed on the RESTful endpoint. If some external system has changed the data on the REST endpoint in the meantime, then System 0 has to deal with the fact there's been a collision. But very clearly there's a difference between the scratchpad and the REST endpoint because there have been multiple updates to the resource by different systems without creating a new version or exposing anything on the RESTful endpoint.
Matt Varghese (Feb 17 2021 at 14:49):
That is only one way to implement things.
In the example of two Apps 1 and 2, one hitting scratchpad, and the other the REST-ful endpoint, I can see a different managed access mechanism syncing changes, so that both Apps can manipulate successfully, and the above picture of scratchpad wouldn't work.
Matt Varghese (Feb 17 2021 at 14:50):
In that sense, the above picture is not defining interface, but defining EHR implementation..
Lloyd McKenzie (Feb 17 2021 at 14:55):
And that synching mechanism doesn't involve anything being written to the REST endpoint by System 0 or anything launched from System 0 before the user decides they want it to be?
Matt Varghese (Feb 17 2021 at 14:57):
Doesn't have to..
Matt Varghese (Feb 17 2021 at 14:58):
In that implementation, there is no "in-memory" store - everything always persists.. with versioning..
Josh Mandel (Feb 17 2021 at 15:09):
Again "scratchpad" is a mental model for developers interacting with resources that may not be permanently stored anywhere. This doesn't prevent you from persisting scratchpad data somewhere in your own EHR implementation, but it avoids system level guarantees that would prevent other EHR implementations from taking different choices. It is an abstraction that provides the functionality for CDS/app interactions with potentially ephemeral data, without impacting how you implement it under the hood.
Matt Varghese (Feb 17 2021 at 15:16):
It is an unnecessary mental model that does not need definition, nor has clear definition.
On the other hand, if we say, SMART Web Messaging has an EHR endpoint that provides managed access to work in progress resources, I can see that being reasonable.
Josh Mandel (Feb 17 2021 at 15:19):
When you write the word "endpoint" here, do you mean what SMART Web Messagingc calls a message group / message type? Or do you mean a fhir endpoint? Or something else?
Matt Varghese (Feb 17 2021 at 15:20):
"endpoint" is a generic word as I used here. This would still be an endpoint that listens to postmessages.
But instead of "scarchpad." it will be "ui." or "host." or "ehr."
Some word that captures the "entity that provides managed access"
Matt Varghese (Feb 17 2021 at 15:20):
That removes the issue currently asking for definition of "scratchpad"
Matt Varghese (Feb 17 2021 at 15:21):
Also, the same entity today gives "launchActivity", "done" etc. messages - so that I would say they both should be the same..
Josh Mandel (Feb 17 2021 at 15:21):
I think you're just talking about renaming the word "scratchpad" to use a different word, and maintaining the semantics. As I've said before, I am open to discussing new names within the scope of this ballot.
Matt Varghese (Feb 17 2021 at 15:23):
Yes, I have said before that changing from using a word that stands for an "arbitrary and not clearly defined space" to a word that stands for the "entity that provides managed access" will resolve my concern.
Our reasons for agreeing on that solution seem very different, but that doesn't really matter to me.
Josh Mandel (Feb 17 2021 at 15:24):
again, if you want to update your tracker from last week make this request, or if you want to close last week's tracker and create a new one that's focused on this renaming request, that's something we could take up in the course of our ballot reconciliation.
Matt Varghese (Feb 17 2021 at 15:24):
The key point to note is, what I am proposing avoids JIRA issue https://jira.hl7.org/browse/FHIR-29349 altogether.
Matt Varghese (Feb 17 2021 at 15:25):
Well, my issue already documents this - https://jira.hl7.org/browse/FHIR-31052
Josh Mandel (Feb 17 2021 at 15:25):
I am confident that whatever we call it, we will need a definition :-)
Matt Varghese (Feb 17 2021 at 15:26):
Defining "arbitrary and not clearly defined space" - is defining something that I'm not sure has a clear definition.
Defining "entity that provides managed access" - is defining the EHR that is hosting the App, which can actually be left implicit.
Josh Mandel (Feb 17 2021 at 15:27):
it might help guide discussion if you can get a pull request together implementing the cleanest way you see to make this change, and we can iterate on this before trying to bring it up for a vote.
Josh Mandel (Feb 17 2021 at 15:27):
I'm not saying this is requirement, just trying to make a productive suggestion, since we won't be able to devote another hour to this topic in our call.
Matt Varghese (Feb 17 2021 at 15:28):
Sure, I can do that.
Lloyd McKenzie (Feb 17 2021 at 15:28):
It's not ok for content to be persisted with versioning unless the user says they want it to be. Users sometimes want to be able to draft things without the outside world ever seeing what they're contemplating. Also, the changes that they're contemplating sometimes can't be exposed until they declare 'done'. E.g. processing a patient discharge.
Lloyd McKenzie (Feb 17 2021 at 15:29):
The requirement of "don't write to the REST endpoint" is a firm one.
Josh Mandel (Feb 17 2021 at 15:30):
I'm pretty sure Matt is not planning to document a set of changes that involves writing to a REST endpoint or exposing a versioned history API -- I am expecting he is just going to rename the scratchpad message group and perhaps tweak or refine the associated definitions.
Matt Varghese (Feb 17 2021 at 15:30):
Users sometimes want to be able to draft things without the outside world ever seeing what they're contemplating.
That is access control.
Matt Varghese (Feb 17 2021 at 15:31):
Also, @Lloyd McKenzie , again you're using the word "draft" :P
Josh Mandel (Feb 17 2021 at 15:31):
@Carl Anderson it would be good to make sure the scratchpad definition we voted on last week is incorporated before this PR work happens, so Matt can either tweak or delete as he sees fit in his proposal
Matt Varghese (Feb 17 2021 at 15:32):
@Josh Mandel , I think it shouldn't be incorporated..
My PR is going to say "scratchpad.<>" is deprecated since it is unclear what scratchpad is, and instead have "ui.create", "ui.delete", "ui.update".
Josh Mandel (Feb 17 2021 at 15:32):
That's fine; I just want to document this in the PR. It might look like we delete that paragraph. But since we voted on putting it in I want to make the lineage of these changes explicit in your proposal
Josh Mandel (Feb 17 2021 at 15:33):
(I would also suggest you may be able to avoid talking about deprecation since we have not yet published the API and in this first version it will be called whatever it is called.)
Matt Varghese (Feb 17 2021 at 15:34):
Okay
Josh Mandel (Feb 17 2021 at 15:34):
(I would also suggest you not directly propose new language to include in the specification that tries to explain problems with deleted language that readers will not be looking at -- best to publish something clear and proactive.)
Matt Varghese (Feb 17 2021 at 15:35):
I was going to make a PR that included all three issues..
Josh Mandel (Feb 17 2021 at 15:36):
If you are reworking things that we voted on last week, it would be good to make this explicit. I think there's a lot of useful informative context in the definition we put together even if we don't keep those words.
Matt Varghese (Feb 17 2021 at 15:38):
Let me discuss process with you and Carl offline?
Josh Mandel (Feb 17 2021 at 15:38):
Also, one concept that emerged in discussion on zulip over the last week for me is the usefulnessof explaining equivalence with the CDS hooks programming model; we already state equivalents in the specification but it didn't come up in our discussion, and I thought it was clarifying to note that the same kinds of access and manipulation that happen in CDS hooks "suggestions" are the sorts of things that apps can accomplish with SMART Web Messaging.
Last updated: Apr 12 2022 at 19:14 UTC
