Stream: terminology / utg
Topic: Errata? V2 Table LIst
Clayton Daley (Jun 08 2020 at 03:07):
I was digging around this table (https://build.fhir.org/ig/HL7/UTG/branches/master/CodeSystem-v2-tables.html) when looking for a clean way to identify v2 fields where a Code System has been defined (so I can enrich them if they exist). When perusing, I noted a variety of likely issues:
- 0288 shows usage in XAD.109 (XAD.10 is the right value)
- 0298 specifically says "datatype" (instead of implicitly by syntax)
- 0008 refers to MSA-1.1 where other ID fields (e.g. 0155) refer to just MSH-15 and MSH-16.
- 0003, 0076, and 0354 are similar to 0008 but for a more obvious reason i.e. that MSH-9 is a nested type (or at least represented that way in many places). For consistency, this should probably be MSG.1 through MSG.3 (plus just MSH-9, but see my note below about mixing segment fields and datatype fields below).
- 0163 refers to usage in "CH7". 0206 lists "CH2". 0398 says "CH02". Maybe these are placeholders.
- 0113 through 0116 refer to DLD.1. (with a trailing period). Looks like this is only valid for 0113 (with no trailing dot)
- 0548 uses a colon (CON:25) see also 0496 through 0502.
- 0317 says OBX ANO. Looks odd but don't know the context well enough.
- It looks like XXX-# is used for fields and XXX.# is used for data types. It took me a second to sort it out. Not strictly errata, but I'd certainly prefer separate columns for the two types of usage.
- In most cases, either a segment field or datatype field are listed. In some cases (see 0200), however, both the datatype location (e.g. XPN.8) and fields using that datatype (e.g. PID-5) are identified. It seems to me that a field should not be listed if it's an instance of a type that's listed. See also 0347 maybe 0440, 0444.
- 0399 is an example of a place where both "columns" must be populated. MSH-17 is an ID that uses the table and is not subsumed by any of the datatype fields.
I'm not sure if there's a place that cross-links types and field usage, but it would be very useful to out-link datatype columns (e.g. XPN.8) to a place where all uses of XPN may be found. In theory the values could be inlined (especially in a 2-column layout) but a lot of these are going to collapse into "numerous" which undermines the effort.
Frank Oemig (Jun 08 2020 at 06:33):
Thanks for the detailed list. Some of the problems are known and worked on for years. What we currently have here is created by indirection of a database. There is no simple way to create such a list. Source of truth is Word. So tgat must be parsed and extracted. First to mention is the missing integrity in the documents resulting in dublication of information. Second, originally, ie. before v2.4 we did not have a component model, so that references from datatyoea where added to the field. Here we dailed to remove that simply for convenience. (Readers do not want to browse.)
That explains some of them. In v2+ we hopefully can fix some.
Clayton Daley (Jun 08 2020 at 15:52):
FWIW along the way we've manually extracted a non-trivial fraction of this data for our internal sanity. I suspect the entire datatype list (it looks like 2.6 and 2.7.1)...
... and segments for several of the core sections/versions
I'm not sure what your intermediate database looks like, but we can certainly pull e.g. all XTN fields in our dataset (this is an Excel pivot table for expedience but could easily be converted to a SQL talbe and query)...
I think we were basically able to cut-paste the tables from the newer documents (I think 2.2 and maybe 2.3 had a legacy format that was less friendly).
Clayton Daley (Jun 08 2020 at 15:56):
Dunno if anything we have could be pushed back into an intermediate database (or be used as a template for an intermediate source of truth). These were all forward constructed from the segment and datatype tables which may avoid errata in the tables/usage list in the actual tables section.
Frank Oemig (Jun 08 2020 at 16:03):
The challenge is the gap between nice to have consistency and correctness on the side of source of truth. The more we adjust the greater the gap will be.
I am working on that for 25 years now, and failed to clarify a lot of those issues due to pushback. But I would highly appreciate appropriate change requests.
Clayton Daley (Jun 08 2020 at 16:27):
Our only contact with HL7 (besides userland questions/feedback in here) has been processing and using the standards documents for a v3-based product designed in 2014 (when that seemed to make sense) and a v2 to v3 converter to support that product. So we have no concept of what's required to create assets and what invites pushback.
So let me ask the simplest question... is there a publicly available database with all of the v2 segment/datatype tables? If so is it rendered into tables/JSON/XML anywhere on the internet? A competent programmer could derive what I'm suggesting from them with relatively little friction. Even better if they were merged and versioned like we do, but that only reduces the amount of work to process the input (if someone even cares about multiple versions).
Frank Oemig (Jun 08 2020 at 16:49):
There is a db, but its not publicly available. There are also other sources.
It is not an issue of finding those points, but placing and defending appropriate CRs.
Frank Oemig (Jun 08 2020 at 16:51):
Quite a lot of those queries produces inconsistency lists that editors have to work on. But the willingness to work on outdated v2 is deminishing.
We hope to raise that interest again once v2+ is available.
Clayton Daley (Jun 10 2020 at 01:15):
I was just thinking that you can add value by exposing the raw data for userland to process/edit even if just for their own use -- i.e. if they lack the knowledge or ability to make official CRs. I'm also not clear how a list of "where XPN is used" that is derived from the segments tables can include errata that wasn't fixed (by necessity) in those original tables. Or are you saying the intermediate DB representation may not be accurate due to extraction issues?
Frank Oemig (Jun 10 2020 at 06:41):
I cannot garantee correctness, only my willingness to work and solve identified issues.
2nd, due to relational integrity of the db currently there cannot be 100% correctness.
But we are working on that for v2+. We have to fix all issues and therefore appreciate hints.
Frank Oemig (Jun 10 2020 at 06:43):
(The db is made available for WG use for free. But requires a request per HQ.)
Last updated: Apr 12 2022 at 19:14 UTC
