Stream: implementers
Topic: Distinguishing test patients
Brian Kaney (Jun 22 2016 at 15:35):
We have an EHR with mostly real patient records, and a few test ones. How could this be exposed? I was thinking about an extension on Patient, as using Patient.active isn't the correct semantics.
James Agnew (Jun 22 2016 at 15:57):
Using tags might be a good option for this
Pascal Pfiffner (Jun 22 2016 at 16:51):
I would expect test patients to live in an isolated test environment, so a different base URL.
Brian Postlethwaite (Jun 22 2016 at 20:55):
If tags was the approach selected, using an unrestricted confidentiality security tag would be appropriate
http://hl7-fhir.github.io/v3/ConfidentialityClassification/vs.html
James Agnew (Jun 22 2016 at 21:33):
Many of the production health systems I've encountered in the wild follow a system of "try to use the word test in either the last name or in the first name but if you forget we'll just add a rule in the interface engine".
I guess that's an option as well..... :p
Abbie Watson (Jun 22 2016 at 21:55):
+1 for a Patient extension. Would love to see a isTest boolean field available.
John Moehrke (Jun 23 2016 at 02:17):
Brian has it right. When data has been de-identified, it can then be marked with the _confidentiality code of "L". This by definition tags the data as being de-identified. We don't need an extension, the support is already there.
Brian Postlethwaite (Jun 23 2016 at 02:59):
@John Moehrke , i'm interested in why the difference using the U or the L here for the test data.
Brian Postlethwaite (Jun 23 2016 at 02:59):
I can see it should be L if _was_ real data though.
John Moehrke (Jun 23 2016 at 03:09):
in the _confidentiality scale, it seems L is more appropriate. I guess U could be used. Of course I start with the assumption that the data comes from real-data that is de-identified. This would only move the data to L as there is no way to assure risk has moved completely to zero. However the data might be completely fabricated (random fuzzing). I think the intent of U is more for the completely random, or data like a cafeteria menu.
John Moehrke (Jun 23 2016 at 03:10):
It does seem useful to have a code specifically for 'test data'. I thought that there was one, but can't find it. I think the easiest is to declare a security-label 'compartment' for 'test-data'... seems like an opportunity to improve the vocabulary.
Brian Postlethwaite (Jun 23 2016 at 03:11):
U = good for things like the public directory bits, like Organiziation/Location (usually)
L = better for patient/clinical test data
That makes a lot of sense.
Brian Postlethwaite (Jun 23 2016 at 03:13):
My server will serve up anything with "U" over unsecure connections, everything else has to be on https with an authenticated account.
(not active on my reference deployment though, as want anyone to be able to test content)
John Moehrke (Jun 23 2016 at 03:14):
right. However as I just realized and explained; we are not looking for a _confidentiality scale.. this is needed; but we want an additional tag for 'test-data'... right?
Grahame Grieve (Jun 23 2016 at 07:34):
it's not clear to me that this is a compartment tag. It's not about permission to access. I'm not convinced it's a security label at all. it's a tag. But we've done this before - defined a security label and then said it goes in the tag not the security label
John Moehrke (Jun 23 2016 at 12:41):
The blunt model is to have a completely different system that only contains test-data. aka physical segmentation. I think the discussion is how can we use have both kinds of data combined, this does seem like a security question. Of course if you allow both combined, then you open the 'risk' that someone will accidently or maliciously set the tags to test-data; and thus cause opportunity for inappropriate disclosure. Also combined data doesn't allow you to test the non-test-data access control rules, that is in order to test that the access control system is working you must try to access real patient data, thus if it is not working you have caused a breach... so there are benefits and drawbacks.
Grahame Grieve (Jun 23 2016 at 12:42):
well, typically, the system doesn't know it's test data - that's so it can be tested like it's real patient data
Grahame Grieve (Jun 23 2016 at 12:43):
the company I used to work for, any patient whose surname was the same as the company was automatically a test patient.
Grahame Grieve (Jun 23 2016 at 12:43):
but programmers were not allowed to write any code about that
Grahame Grieve (Jun 23 2016 at 12:44):
in argonaut, we talked about having a manifest of test patients - e.g .a List resource with a magic id, which listed all the test patients.
John Moehrke (Jun 23 2016 at 13:04):
yes, those are ways it could be done.... they all suffer from the same problems... So why is it not a meta tag? And why is it especially not a security-tag? It certainly seems security relevant to me.
Grahame Grieve (Jun 23 2016 at 13:09):
not everytihng that is security relevant goes in a security tag
John Moehrke (Jun 23 2016 at 13:14):
I don't disagree.. I am trying to understand why you indicated that you are not convinced that it is a security tag. So I am looking for the 'why'. I don't disagree that sometimes there are good reasons. I gave one in my last few responses. Just trying to figure out one more reason that you have identified but not communicated.
Grahame Grieve (Jun 23 2016 at 13:15):
well, I don't really have a trite answer to that. I feel as though putting something in a secuity tag that isn't a 'security thing' is a bit of a surprise, and we kind of hate surprises. but that's not a well founded feeling.
John Moehrke (Jun 23 2016 at 13:20):
okay, that is a fair answer... Might bring it up on the security call, so would also like others 'feelings'. I was expecting a tag to already exist, so am looking for why that gap (especially given the plethora of tags we do have). It feels to me like an "Integrity" tag https://www.hl7.org/fhir/v3/SecurityIntegrityObservationValue/index.html
Grahame Grieve (Jun 23 2016 at 13:49):
could be. Though I think people outside security's scope are the ones to consult about the surprise factor
Abbie Watson (Jun 23 2016 at 16:39):
Lets keep in mind that there are multiple testing paradigms... validation, verification, unit, integration, acceptance, end-to-end, smoke, regression, boundary, load, security, just to name a few. In some of those paradigms, it's necessary to test security rules; while in others it's completely irrelevant. Likewise, in some paradigms one wants isolated test databases, whereas in others one doesn't. From a QA perspective, having a test value in a security tag would be great. It adds to the overall testing story. However, it alone is not sufficient. Just as having an isolated test database is desirable, but also insufficient by itself. This isn't a one-size fits all solution.
Peter Bernhardt (Jun 23 2016 at 17:02):
I know our systems have test patients in production. Various use cases for this, deployment verification being the one that first comes to mind. For this use case, at least, a meta tag strikes me as the best solution to identify a test resource.
Abbie Watson (Jun 23 2016 at 17:17):
Agreed. A testing specific meta tag or field would be ideal.
Lloyd McKenzie (Jun 24 2016 at 03:00):
I like the idea of a modifierExtension - reduces the risk of systems that don't recognize the extension using the data. ("This isn't actually a patient" sounds like a modification of meaning to me . . .)
Brian Postlethwaite (Jun 24 2016 at 03:01):
Its a patient, just not a real one.
Lloyd McKenzie (Jun 24 2016 at 03:02):
Right, but isn't that a change of meaning?
Brian Postlethwaite (Jun 24 2016 at 03:02):
I wouldn't have thought so. Anyone else?
Grahame Grieve (Jun 24 2016 at 03:21):
I don't think it's a modifier extension. in fact, it's specifically not - this is a record that is treated identically to other records through out the system . That's why you have test records in the live system
Lloyd McKenzie (Jun 24 2016 at 03:24):
But how do you exclude it from metrics and other things then?
Grahame Grieve (Jun 24 2016 at 03:25):
I don't think you do, or can
Lloyd McKenzie (Jun 24 2016 at 04:05):
Probably something for the CQI folks to think about then . . .
James Agnew (Jun 24 2016 at 13:01):
The issue I have with modifierextension or security tag as a solution to this is that both of these things can/should fundamentally change the way systems treat these resources. Generally test patients in production are there to make sure that end-to-end tests work exactly correctly.
The issue of course is that somewhere along the chain you want to block them, in order to prevent family physicians from receiving faxes about their patient "Testy Tester" (as has happened to us :( ) or financial systems sending invoices about Mr Testy (as has also happened to us.....)
Test data is hard.
John Moehrke (Jun 24 2016 at 14:26):
James, not all meta tags are a 'must understand'... I think this is the critical thing you seek. Because in these cases we want the systems being tested to ignore (gracefully) this tag, while systems that do formal counting to recognize that these are test patients.
James Agnew (Jun 24 2016 at 17:28):
meta tag no, but I think I would probably prefer to design systems to err on the side of caution for security tags they don't understand
David Hay (Jun 24 2016 at 19:23):
It seems a bit odd to use security tags for this. eg suppose the resource is a public Location resource with a confidentiality of 'U'? Both real and test Locations would have the same confidentiality code wouldn't they?
Mark Kramer (Jun 27 2016 at 14:51):
A test patient is simply a fictional being. It doesn't have anything to do with security (a fictional patient could have any security settings; for example, the fictional patient could have security tags allowing access to another organization for interoperability testing). In terms of implementation, tagging is too easy to miss. Therefore, I agree it should be an boolean element in the base Patient resource. Marking it as modifying attribute is a good idea. Appending numbers to names (Smith2423) is good practice, so human viewers could easily pick out fictional beings.
John Moehrke (Jun 27 2016 at 15:19):
Mark... that is not a problem.. the security-tags are infinite in number. So just because one has the tag that indicates it is a test resource, does not mean you cant have other tags that indicate special sensitivity, or have consents with special rules. But by having the tag indicating it as a test resource, then the audit log will clearly show that the disclosure is not a real-world disclosure. having a well-known security tag is the same thing as having a boolean element, without creating special structure.
James Agnew (Jun 27 2016 at 15:20):
Modifier element on Patient seems like a very large hammer for fairly trivial use case... Not to mention that it won't help you for test clinicians, test orders, test locations, etc...
I still think a plain old tag is probably the most workable solution here.
Chris Grenz (Jun 27 2016 at 15:21):
+1 for tag - modifierExtension is very much wrong for the scenarios where test data would be in the production system.
James Agnew (Jun 27 2016 at 15:21):
I still don't get why it would be a security tag and not a normal tag? What do test patients have to do with security?
Chris Grenz (Jun 27 2016 at 15:23):
Right? Test patients should be subject to the same security considerations as real patients, especially when testing security!
John Moehrke (Jun 27 2016 at 15:26):
I just explained that the fact that it is a security-tag that indicates it is a test patient does NOT CHANGE any other security tag or operation. IF you look back further in the thread you will see where I explain in more detail various scenarios where one would need it to impact securty decisions, and others where you would want security decisions not impacted by the tag. All of this is very easy done with security decision engines. There is nothing special about a security-tag or a tag; except that sometimes the security engine does look at it. Keep in mind that security decisions are often made on things that you might think are purely clinical knowledge. There is nothing all that special.
John Moehrke (Jun 27 2016 at 15:29):
The reason why I do think it is a security tag is because distinguishing test data from real data is indeed very important to Privacy and Security. It is indeed fundamental to the very fact that Privacy is concerned with the rights an desires of the patient, and a privacy breach is exposure of real patient data. The first concern when the IT organization finds that a database is breached is to determine what classification of data was in that database. If it is the cafeteria menu, they worry very little; if it is business secrets they worry very much; if it is real patient data they know there are disclosure rules that must be followed. how is this NOT clear?
Chris Grenz (Jun 27 2016 at 15:31):
I think the confusion is that the "cafeteria menu" identifier also doesn't show up in the security tag section - there's lots of information that is relevant to security that isn't explicitly tagged as "SECURITY".
Chris Grenz (Jun 27 2016 at 15:32):
And "test-patient" isn't explicitly a information protection instruction....
John Moehrke (Jun 27 2016 at 15:32):
how is it not?
Chris Grenz (Jun 27 2016 at 15:35):
Can you make a universally applicable statement about how the "test-patient" designation affects privacy and security handling of data taking into account legal, organizational, and other relevant policies? In other words, in some organizations, test patients may be considered to be as sensitive as real data (and may be used as a smoke test in that capacity). Contrast this to "CONFIDENTIAL" which is explicit.
John Moehrke (Jun 27 2016 at 15:36):
I have argued that it is sometimes not part of the security policy, but sometimes it is... so it would seem we are arguing the same thing... However you end up at the conclusion that this means it should not be a security-tag, and I end up that it should be a security-tag. Please explain your perspective, inform me...
John Moehrke (Jun 27 2016 at 15:37):
are you arguing that test-patient as a security-tag would somehow not be allowed to be tagged "CONFIDENTIAL"? Where do you see that rule?
Chris Grenz (Jun 27 2016 at 15:38):
No, not at all. I'm simply saying that they "test" doesn't imply anything from a security standpoint (in some scenarios).
John Moehrke (Jun 27 2016 at 15:38):
fine. I have said that too.
John Moehrke (Jun 27 2016 at 15:38):
but somehow you are saying that this very argument means that the tag should not be a security-tag.
John Moehrke (Jun 27 2016 at 15:39):
that is what I am confused by.
Chris Grenz (Jun 27 2016 at 15:39):
I guess I'd err on the side of limiting security tags to those things that are explicitly security instructions and leave everything else outside that list (since there are *many* other things that *may* have security implications).
Chris Grenz (Jun 27 2016 at 15:39):
Otherwise, the criteria for inclusion in the security tag section is going to become...difficult.
John Moehrke (Jun 27 2016 at 15:40):
Yet I have explained a very useful attribute of putting this in as a security tag... it will automatically end up in the audit log... if it is not a security-tag, then one must investigate object-by-object to determine which audit events are damaging vs which were on test-data.
John Moehrke (Jun 27 2016 at 15:40):
it is 'security relevant'... that is the evalation.
John Moehrke (Jun 27 2016 at 15:42):
besides security never asked that meta be separated into security and non-security tags... we were happy with tags, as long as there was clear rules on why a change to the meta tags were clear. It was FHIR-I that decided that this requred that the tags be separated.
Chris Grenz (Jun 27 2016 at 15:42):
Fair enough....my history doesn't go back that far!
John Moehrke (Jun 27 2016 at 15:44):
but now I understand that you percieved the separation of meta into security and non-security as an indicattion that anything in security tags WILL be part of security access control rules... This is not true.(unusual, but not fact).
James Agnew (Jun 27 2016 at 17:12):
I guess the perspective I have is tainted by the way we've used test patients here at our hospital. To us, they aren't treated any different from a security or access control perspective. Access to their records is audited like any other, they have consent directives and break-the-glass and privacy officer notification and all the same security stuff that any other record has.
To us, marking test patients is all about 2 things: avoiding communication outside our walls (i.e. don't send faxes about test patients), and avoiding any of the data ending up in finance/billing systems (since these apparently do not have any way of ignoring test data)
Yet I have explained a very useful attribute of putting this in as a security tag... it will automatically end up in the audit log...
Is this documented somewhere? I've never heard of that attribute of security tags, but that does seem like a good reason to use one for this.
John Moehrke (Jun 27 2016 at 17:21):
@James Agnew See AuditEvent.entity.securityLablel http://hl7-fhir.github.io/auditevent-definitions.html#AuditEvent.entity.securityLabel
John Moehrke (Jun 27 2016 at 17:24):
Even if the audit logging system doesn't reflect the meta tag (leaving the auditevent.entity.securityLabel empty), one can always just look it up by dereferencing the auditevent.entiy.reference. It is an explicit element (.securityLabel) as not all entities that one might ever access are FHIR objects (might be DICOM, hl7 v2, CDA, openehr, xds, etc).
Elliot Silver (Jun 27 2016 at 18:46):
@James Agnew , just to be clear, are you suggesting that the "test" tag should be applicable to any (or at least some number of) resource? Test-tagged Patient is scheduled for a test-tagged Appointment, recieves a test-tagged MedicationAdministration by a test-tagged Practitioner?
James Agnew (Jun 27 2016 at 20:57):
Well, I mean I haven't given it a ton of thought.. Certainly in the v2 world we had test patients and test clinicians in our production system and the interface engine had routing rules based on messages using either of those things. Individual ORUs (for example) were understood to be tests because the patient in the PID segment had test in their name. In the FHIR world you're more likely to have (say) an Observation that just has a reference to to the patient, so it would probably be best for it to have the tag/security tag too.
Grahame Grieve (Jun 27 2016 at 21:11):
@John Moehrke interesting point about the differentiation between tags and security labels. We primarily made it because of meta data considerations.
Grahame Grieve (Jun 27 2016 at 21:12):
(different value sets, different constraints when used in practice)
Grahame Grieve (Jun 27 2016 at 21:13):
I think the issue here is just one of surprise - for people not focused on security, it's a surprise that this would be a thing defined in the security labels, because it's semantics about the data, not about it's security
Brian Postlethwaite (Jun 28 2016 at 01:12):
Funny that I've implemented the AuditEvent stuff in my server, and I didn't notice that entity.securityProperty and I'm not filling it out.
Maybe should clarify the wording to indicate that the securityLabels were copied from the object when the event occurred.
(And in the details descriptions indicate that these may have changed since the AuditEvent was recorded)
Tracker for it?
John Moehrke (Jun 28 2016 at 02:14):
@Brian Postlethwaite As I indicated, if the resource is a FHIR resource then it is not critical to copy the security labels into the AuditEvent, as we can always query the resource (version) and find them. The element is there to give a place for the tags to be stored if they need to be. Given the volume of auditevent instances, we would rather recommend that only data that can't be discovered be put into the auditevent instance.
Brian Postlethwaite (Jun 28 2016 at 02:15):
But they can be modified in history, so could be different to what was there at the time.
(Which is why I thought it now made sense)
John Moehrke (Jun 28 2016 at 02:16):
I am not surprised at peoples surprise about auditEvent... unfortunately... I try to be as helpful and as verbose as I can be. I also don't know what people don't understand until they ask.
Brian Postlethwaite (Jun 28 2016 at 02:17):
Very true, http://hl7-fhir.github.io/resource-operations.html#meta-add says historical versions can be modified
John Moehrke (Jun 28 2016 at 02:17):
well, yes and no... the modification of meta tags without versioning is a contended topic.
Brian Postlethwaite (Jun 28 2016 at 02:18):
Given where it is now, copying the "current" values into the AuditEvent makes sense in that at least the history is somewhere for that info.
John Moehrke (Jun 28 2016 at 02:18):
I would prefer FHIR never have said that meta is special... any element might be made special by policy... this is an operational characteristic involving access control policy and operational policy.
John Moehrke (Jun 28 2016 at 02:19):
it only hurts to copy the values into auditevent in that this makes the auditevent larger... but bits are cheap, so we shouldn't worry about a few characters to store tas.
Grahame Grieve (Jun 28 2016 at 02:22):
other elements cannot be made special by policy. Servers do not need to make the meta special in that regard, but they can
John Moehrke (Jun 28 2016 at 02:30):
everything is possible...
Last updated: Apr 12 2022 at 19:14 UTC
