Stream: dotnet
Topic: API Validation Query
Richard Kavanagh (Sep 24 2017 at 19:05):
I am using the.NET API to validate a STU3 message bundle.
The results of the validation are mixed and the errors shown look to be valid in that we can understand why they have been identified.
Looking at the output the following is seen.
It appears as if the API is failing on aspects of the terminology validation but this is not making it to the validation result.
Is this an error in the .NET API code?
Yunwei Wang (Sep 24 2017 at 19:36):
.NeE FHIR Client nugget package can validate extensional ValueSet specified in spec. The error says that the valueset it is trying to validate is too complex for the client package to handle. To validate against such complex ValueSet, you have use a terminology server.
Richard Kavanagh (Sep 24 2017 at 19:48):
@Yunwei Wang I appreciate that that's the case. My issue is that the validation result returned was positive - surely if the terminology validation failed then the validation result should take this into account.
Yunwei Wang (Sep 24 2017 at 21:04):
@Richard Kavanagh Are you using a FHIR server to do the validation or Fhir Client library?
Richard Kavanagh (Sep 24 2017 at 21:48):
It the FHIR Client library. All the work is local no FHIR servers are involved.
Yunwei Wang (Sep 24 2017 at 21:56):
The problem is the library doesn't know how to validate that value set.
Richard Kavanagh (Sep 24 2017 at 21:59):
Yes, I'm fine with that. The issue is that as the valuesets cannot be expanded and hence the codes validated - the validation of the instance should fail. Yet the failed terminology validations appears to be silent and does not impact the overall validation status.
Yunwei Wang (Sep 25 2017 at 02:09):
@Richard Kavanagh do you know which valueset validation failed?
Grahame Grieve (Sep 25 2017 at 04:57):
@Richard Kavanagh this is a common argument with regard to terminology validation. Some of us thing that it should fail to error, and some of us think it should fail to no error.
Grahame Grieve (Sep 25 2017 at 04:58):
On the java validator, this is a setting
Brian Postlethwaite (Sep 25 2017 at 05:59):
This is up to the terminology service class that you pass as a reference to the validator in .net
Then it depends what you do in that class that counts.
(My server has a local implementation that fails to the dotnet one that Ewout ported)
Brian Postlethwaite (Sep 25 2017 at 06:00):
You can of course pass in a reference to the ExternalTerminologyService class to use too.
(and make it a problem for a real terminology service implementation)
Richard Kavanagh (Sep 25 2017 at 10:05):
to me the question is not really about where the terminology service occurs. It's what happens when it fails. If an intrinsic part of a validation process can fail and not be signalled then that's worrying. I had hoped the.NET API would cater for this. It appears that the JAVA equivalent controls this through config which appears sensible. Is there a reason why the .NET version did not take the same approach?
Michel Rutten (Sep 25 2017 at 10:12):
@Richard Kavanagh reason being it is still a work in progress. IMHO validation needs to use three-way logic (correct | incorrect | undetermined) everywhere. Then clients can determine how to deal with indeterminate validation results (i.e. accept or reject).
Richard Kavanagh (Sep 25 2017 at 12:06):
@Michel Rutten I agree but unfortunately at this point in time that's not on the table. From an implementation perspective it's looking like we will need to move certain processes towards JAVA which is not really where I'd planned to be at this point.
Michel Rutten (Sep 25 2017 at 12:21):
Ouch, did you just mention the J-word in this stream...?
I think that with some small temporary tweaks and/or additional config settings, we should be able to let the .NET validator fail in case of missing (ValueSet) dependencies, so you can safely deploy it within your production environment.
Then we can take some time to discuss proper validator improvements (i.e. 3-way logic) within the community and harmonize API behavior.
Richard Kavanagh (Sep 25 2017 at 12:28):
@Michel Rutten it hurts me a lot but it's the obvious way forward at the moment. It's going to be a tricky manoeuvre getting out of this which runs the risk of baking in a direction of travel for some time. It's not really where I hoped we'd be.
Yunwei Wang (Sep 25 2017 at 14:34):
I read through the source code for .NET validator. Interesting, I noticed the error is suppressed for non required bindings. And for required binding, my understanding about the source code is that the validator should return validator error in OperationOutcome.
Yunwei Wang (Sep 25 2017 at 14:34):
@Richard Kavanagh that is why I want to know if you know which value set failed. If it is non-required binding. Maybe that is the reason.
Ewout Kramer (Sep 26 2017 at 07:51):
Yes, I'm fine with that. The issue is that as the valuesets cannot be expanded and hence the codes validated - the validation of the instance should fail. Yet the failed terminology validations appears to be silent and does not impact the overall validation status.
It's not obvious to me that this should be a failure - I've flip-flopped a few times, since there are arguments both ways, so I can confirm what Grahame says. As Yunwei said, I have settled on an approach where required bindings report an error, and others a warning.
That said, it's preferable that the .NET and Java API handle things the same way. I am not a fan of a setting - although it's easy to do and you could simply add a suggestion for enhancement to the .NET API Github issue list. The idea that -depending on the setting- some servers will validate your instance as correct and others as incorrect feels wrong.
I feel that the problem lies deeper: the current validator (and probably the Java one too) either determines something is valid or invalid - but there are cases (not just this one BTW), where the outcome is "undetermined". I initially did not choose to go this way - since it might be unclear to the user, but it's issues like these that maybe proof we do need to introduce "undetermined" outcomes.
Ewout Kramer (Sep 26 2017 at 07:53):
@Michel Rutten it hurts me a lot but it's the obvious way forward at the moment. It's going to be a tricky manoeuvre getting out of this which runs the risk of baking in a direction of travel for some time. It's not really where I hoped we'd be.
If you'd need it as a setting to sooth the pain - feel free to ask!
Last updated: Apr 12 2022 at 19:14 UTC
