Stream: terminology / utg

Topic: Publisher issues for UTG

Ted Klein (May 17 2020 at 21:10):

@Grahame Grieve I am starting this thread for the items still outstanding with the Publisher relative to UTG pages at terminology.hl7.org.

Ted Klein (May 17 2020 at 21:11):

1. UTG has a local extension for NamingSystem.title, but this does not show up with the Publisher. The missing <title> for NamingSytsem has been addressed in R5 as well, but the Published pages have no title them at all for the NamingSystem entries, which is also an issue in the Table or Contents. Needs to be addressed in some way.

Ted Klein (May 17 2020 at 21:13):

2. There seems to be some problem with multiple entries for uniqueID for NamingSystem. The OID and two entries (true and false for preferred) for URI are rendering but an additional one of type 'other' is not. Also it appears that having additional URI entries for uniqueID where preferree=falso do not work either.

Ted Klein (May 17 2020 at 21:16):

3. This is a 'nice to have' for perhaps down the road someday - it would be nice to flag the List entries that drive the tab contents so that visual hints could be provided for some of the entries which are special in some way. Initially, the though is to flag the ballot-bound code systems and value sets, as these will not be subject to the community maintenance process in UTG, but must be updated through the ballot process. Ideally, something is List.entry.display to do this would be great.

Grahame Grieve (May 18 2020 at 20:59):

1. where do you think it should be appearing?
2. where can I see an example?
3. Is this a property of the list, or of the resource? I think it's a property of the resource.

Grahame Grieve (May 18 2020 at 22:23):

1. ToC and page title

Grahame Grieve (May 18 2020 at 22:27):

proper extension for NamingSystem.title is http://hl7.org/fhir/5.0/StructureDefinition/extension-NamingSystem.title

Grahame Grieve (May 18 2020 at 22:30):

example for #2 is https://terminology.hl7.org/NamingSystem-icd10.html

Grahame Grieve (May 18 2020 at 22:34):

discussion for #3 moved to IG creation to see what the general case is

Grahame Grieve (May 19 2020 at 00:04):

@Lloyd McKenzie it looks like both missing titles in #1 - both table of contents text and also the page title - come from the template. It's the template that will have to look for the Naming system extension not the IG publisher

Lloyd McKenzie (May 19 2020 at 01:09):

The publisher puts the titles into the IG from the resource information. The template just grabs this information to create the artifact list (which we're not using for UTG). The ToC is generated by publisher too. So I'm not sure where in the template I need to make changes?

Lloyd McKenzie (May 19 2020 at 01:09):

(@Grahame Grieve )

Grahame Grieve (May 19 2020 at 01:24):

@Ted Klein I can't reproduce #2

Grahame Grieve (May 19 2020 at 04:59):

@Lloyd McKenzie looking further at this: at least for UTG, but I think for all IGs, the template should insert the resource type in the page title. e.g. check out these 2 pages:

• https://terminology.hl7.org/CodeSystem-v3-ActClass.html
• https://terminology.hl7.org/ValueSet-v3-ActClass.html

Ted Klein (May 19 2020 at 13:09):

Hmmm....odd. With the latest test #2 does seem to work properly; maybe I was confused earlier. You can take a look at the ICD10 Naming System resource in build 1.0.4, seems good.

Lloyd McKenzie (May 20 2020 at 00:05):

Fixed

Ted Klein (May 26 2020 at 17:36):

Publisher 96 seems to be broken on a branch build...checking into it...can't find snowed any more? Check for supported code systems for http://snomed.info/sct (01:31.0387)
Publishing Content Failed: null

Ted Klein (May 27 2020 at 01:25):

I think I found what was a causing the issue: some items inserted into a naming system resource from the editor were legal xml but the publisher chokes on them. Like a subtle bug in the editor not the publisher. Will experiment with 97 now.

Ted Klein (May 27 2020 at 13:12):

Publisher 97 builds but still has the 60000+ broken link error on the FHIR R4 items references by the value sets in UTG.

Ted Klein (Aug 11 2020 at 17:29):

Publisher 1.0.10 looks pretty good. One question: if a ValueSet.compose.include references a value set that is newly created in the IG (and is created without error in the UTG IG) but the value set did NOT exist in the last published version - ie the value set canonical URL does not resolve and is de novo in the current build - the referencing value set appears to not be able to be built although its other components are all ok and the referenced value set builds fine in the current build. @Grahame Grieve is there anything that can be done about this other than waiting until a new release is published so the referenced objects have valid redirects?

Grahame Grieve (Aug 13 2020 at 11:39):

I don't know why it won't build

Grahame Grieve (Aug 13 2020 at 11:39):

what's an example?

Ted Klein (Jul 22 2021 at 16:16):

Had an odd error surface when the update to Publisher 1.1.77 made to get rid of the 600 or so UTG errors on the NamingSystem extension URL issue. BUT - a new error seems to have been introduced.

Ted Klein (Jul 22 2021 at 16:17):

On the V2 code system resources, we have defined a property in the THO/UTG conceptproperties code system for a v2 thing - 'v2-concComment'

Ted Klein (Jul 22 2021 at 16:18):

It seems that the string value in the valueString is being treated as a URL when the string begins with one word followed by a colon then a space then the rest of the string value

Ted Klein (Jul 22 2021 at 16:18):

The error given is:

Ted Klein (Jul 22 2021 at 16:20):

The link 'Class: Insurance<p>Used for the UK NHS national identifier.<p>In the US, the Assigning Authority for this value is typically CMS, but it may be used by all providers and insurance companies in HIPAA related transactions.' for "Class: Insurance<p>Used for the UK NHS national identifier.<p>In the US, the Assigning Authority for this value is typically CMS, but it may be used by all providers and insurance companies in HIPAA related transactions." cannot be resolved

Ted Klein (Jul 22 2021 at 16:20):

another error of the same ilk is:

Ted Klein (Jul 22 2021 at 16:20):

The link 'HCPCS: contains codes for medical equipment, injectable drugs, transportation services, and other services

Ted Klein (Jul 22 2021 at 16:20):

The one I used to find the issue was in v2 table 0717 code system, a concept for a security item

Ted Klein (Jul 22 2021 at 16:21):

The element that causes the error is:

Ted Klein (Jul 22 2021 at 16:22):

<property>
  <code value="v2-concComment"/>
  <valueString value="Map: An &#x201C;implied&#x201D; consent directive maps to ISO/TS 17975:2015(E) definition for &#x201C;Implied: Consent to Collect, Use and Disclose personal health information is implied by the actions or inactions of the individual and the circumstances under which it was implied"/>
</property>

Ted Klein (Jul 22 2021 at 16:23):

The change that made the error go away was to remove the colon after the word Map

Ted Klein (Jul 22 2021 at 16:23):

<property>
  <code value="v2-concComment"/>
  <valueString value="The Map - An &#x201C;implied&#x201D; consent directive maps to ISO/TS 17975:2015(E) definition for &#x201C;Implied: Consent to Collect, Use and Disclose personal health information is implied by the actions or inactions of the individual and the circumstances under which it was implied&#x201D;"/>
</property>

Ted Klein (Jul 22 2021 at 16:24):

@Mark Iantorno @Grahame Grieve Mark I - methinks this is something in the changes you made in the course of going from 1.1.73 (no error) to 1.1.77

Ted Klein (Jul 22 2021 at 16:24):

@Rob Hausam I think this summarizes it, Rob - yes?

Rob Hausam (Jul 22 2021 at 16:27):

@Ted Klein Yes, I think that's a good summary and demonstrates the issue clearly enough - and should direct to the right place to get it fixed.

Ted Klein (Jul 22 2021 at 22:12):

hopefully it is an easy to find and fix problem. Seems to be about 70 or so places this is happening in the build, mostly in v2 tables but also in other places.

Ted Klein (Jul 28 2021 at 17:01):

@Mark Iantorno I am pinging you again in case you didn't see this...

Mark Iantorno (Jul 28 2021 at 17:04):

Let me take a look

Mark Iantorno (Jul 28 2021 at 17:16):

So, these are difficult for me to know where to start looking exactly. But, it would help me out a lot if you did the following for me:

Mark Iantorno (Jul 28 2021 at 17:16):

Try with 1.1.74 (https://github.com/HL7/fhir-ig-publisher/releases/download/1.1.74/publisher.jar), and let me know if it works.

Mark Iantorno (Jul 28 2021 at 17:16):

Try with 1.1.76 (https://github.com/HL7/fhir-ig-publisher/releases/download/1.1.76/publisher.jar) and let me know if it works.

Mark Iantorno (Jul 28 2021 at 17:17):

Tell me the exact files I need and command you run to make this happen, so I can reproduce on my machine.

Mark Iantorno (Jul 28 2021 at 17:17):

@Ted Klein

Ted Klein (Jul 28 2021 at 23:27):

so if you want to run the build to see what is going on, the Git UTG is the build source files:

Ted Klein (Jul 28 2021 at 23:27):

https://github.com/HL7/UTG

Ted Klein (Jul 28 2021 at 23:29):

there are a raft of branches in there; the THO/UTG ci build (https://build.fhir.org/ig/HL7/UTG/index.html ) is auto built (Josh M's web hook stuff) from the Git UTG master branch.

Ted Klein (Jul 28 2021 at 23:29):

the UTG build is such a beast, I need to use the following arguments for the java command:

Ted Klein (Jul 28 2021 at 23:30):

java -Xmx16G -Xss1G -jar /...path to where I keep my latest release of the publisher.../publisher.jar -ig ig.ini

Ted Klein (Jul 28 2021 at 23:31):

the UTG git master branch image has the ig.ini file in the usual place

Ted Klein (Jul 28 2021 at 23:32):

let me know if any problem. the QA report of the error log for the output shows the errors I detail above - about 70 or so of them. I suppress a lot of warnings, and there are errors that are well expected for value sets build on external code systems that tx.fhir.org has no access to the content for.

Ted Klein (Jul 28 2021 at 23:33):

the errors can be viewed in the ci build, QA report at https://build.fhir.org/ig/HL7/UTG/qa.html

Ted Klein (Jul 28 2021 at 23:34):

let me know if you want me to try with 74 and 76; I skipped those releases and went straight fro 73 to 77; if needed I can download and install then and give it a whirl.

Mark Iantorno (Jul 29 2021 at 13:28):

Can you please try with both 74 and 76. @Ted Klein

Mark Iantorno (Jul 29 2021 at 13:29):

Then let me know. It will help me track down the exact change that is causing this.

Ted Klein (Jul 29 2021 at 13:35):

I will do it this afternoon, on my way out the door for an appointment this morning.

Mark Iantorno (Jul 29 2021 at 13:35):

no rush

Mark Iantorno (Jul 29 2021 at 13:35):

thank you

Ted Klein (Jul 29 2021 at 22:32):

ok tested with 74, same 610 or so errors as 73. Testing with 76 now...

Ted Klein (Jul 29 2021 at 23:38):

ok, with 76 we still have the 610 errors, all in the form of:

Ted Klein (Jul 29 2021 at 23:39):

NamingSystem.extension[1][url='http://hl7.org/fhir/5.0/StructureDefinition/extension-NamingSystem.url'] error Extension url 'http://hl7.org/fhir/5.0/StructureDefinition/extension-NamingSystem.url' is not valid (invalid Version '5.0')
NamingSystem.extension[1] error The extension http://hl7.org/fhir/5.0/StructureDefinition/extension-NamingSystem.url is unknown, and not allowed here

Ted Klein (Jul 29 2021 at 23:40):

I will run with 77 on the same content I just ran 74 and 76 on to double check that the change is from 76 to 77...

Ted Klein (Jul 30 2021 at 03:04):

The 600 errors went away with 77, but the 70 or so new errors (original problem description at start of this thread) is introduced with 77.

Ted Klein (Jul 30 2021 at 03:04):

@Mark Iantorno Does this help narrow it down? Let me know if you need more information.

David Pyke (Jul 30 2021 at 19:31):

So, any time the publisher finds a word with a colon at either the beginning of a valueString or after a CRLF, it changes that string to a link. Workaround is to global change colons in valueString elements to the html equivalent. That will workaround it in the short term and get it published.

David Pyke (Jul 30 2021 at 19:31):

sed -i -e "/valueString/s/:/\:/" *xml is the command that will do it

Ted Klein (Jul 30 2021 at 20:38):

UTG/THO has over 4000 resources in a score of different directories. This is a bug that should be fixed. Or perhaps you know someone who wants to run sed on 4000+ files scattered around? AND - we have a vocabulary WG policy that voted content from Harmonization that underwent formal discussion, vote, vetting, QA, and publication is published AS VOTED; tooling issues are to be resolved by the tool smiths. That is what we did with the coremif issues back when we did the old Harmonization. I see no reason for that policy to be changed. We have been dealing with THO/UTG with 600+ hard errors from the tooling for six months already as Grahame and Mark I. have had higher priorities from TSC and CTO in the intervening time. I am unwilling to change approved balloted content as a workaround for a tooling snafu.

David Pyke (Jul 30 2021 at 20:48):

It was just a suggestion. I apologize for bringing it up

Ted Klein (Jul 30 2021 at 23:53):

no need for apology = there are good reasons I am such a pain in the butt on this stuff LOL...Mark will get it fixed I am quite certain.

Grahame Grieve (Aug 17 2021 at 04:28):

any time the publisher finds a word with a colon at either the beginning of a valueString or after a CRLF, it changes that string to a link

What's an example? Where an I see this?

Jessica Bota (Aug 17 2021 at 13:58):

@Grahame Grieve there are about 80 examples in our current build. http://build.fhir.org/ig/HL7/UTG/qa.html. Search on "Class:" and you should find some good examples.

Ted Klein (Aug 17 2021 at 20:24):

Grahame, most of these (but not all) are in the concept description for concepts in V2 tables, sometimes in Usage Notes, etc. An example from v2 table 0357 where the string causing the issue is "Rejection: " in a text element is:

Ted Klein (Aug 17 2021 at 20:24):

<concept id="3578">
<code value="200"/>
<display value="Unsupported message type"/>
<definition value="Unsupported message type"/>
<designation>
<language value="de"/>
<use>
<system value="http://terminology.hl7.org/CodeSystem/hl7TermMaintInfra"/>
<code value="preferredForLanguage"/>
</use>
<value value="Nachrichtentyp wird nicht unterstützt"/>
</designation>
<property>
<code value="v2-concComment"/>
<valueString value="Rejection: The Message Type is not supported."/>
</property>
<property>
<code value="v2-concCommentAsPub"/>
<valueString value="Rejection: The Message Type is not supported."/>
</property>
<property>
<code value="status"/>
<valueCode value="A"/>
</property>
</concept>

Ted Klein (Aug 18 2021 at 21:33):

@Grahame Grieve @Mark Iantorno The error manifests on all of the resources that have various elements or properties where the valueString begins with one word followed by a colon (:) followed by a space. Change it (for instance the colon at the end of the second word, or replace the colon with a hyphen) and the problem disappears.

Grahame Grieve (Aug 19 2021 at 07:51):

fixed next release

Mark Iantorno (Aug 24 2021 at 12:44):

What was the PR for this Grahame?

Grahame Grieve (Aug 24 2021 at 12:55):

https://github.com/hapifhir/org.hl7.fhir.core/pull/581

Grahame Grieve (Aug 24 2021 at 12:55):

the changes to isAbsoluteUrl

Ted Klein (Aug 25 2021 at 20:54):

Any ETA on the release which will include the fix for this?

Grahame Grieve (Aug 25 2021 at 21:41):

sometime in the next few days. Working hard on it

Ted Klein (Aug 27 2021 at 17:40):

ok thanks for letting me know. I'll look into that other question you had on the OMOP stuff...

Ted Klein (Sep 01 2021 at 18:00):

@Mark Iantorno @Grahame Grieve Just rebuilt UTG with the new publisher 1.1.78 - the error count went from 76 errors (in publisher 1.1.77) to 347. WTF?

Mark Iantorno (Sep 01 2021 at 18:00):

What are the errors

Ted Klein (Sep 01 2021 at 18:09):

Ugh - they are all scattered around the log file, intermixed with the original 76, 68 or so of which were the original substring followed by a colon in a commend or string value being processed as if it was a URL. It will take some research to see what the newly introduced errors are.

Ted Klein (Sep 01 2021 at 18:10):

you can do a pull from Git UTG and do a build and see what is going on and debug if that will help...

Ted Klein (Sep 01 2021 at 18:11):

I think the last ci build was with 1.1.77; I did a local build on 1.1.78 and saw the errors; did not push to Git (yet)

Mark Iantorno (Sep 01 2021 at 18:14):

This probably is related to the changes that went into the last release, @Grahame Grieve

Ted Klein (Sep 01 2021 at 18:15):

I am running a local build of the ci image so can compare the errors in the local with the one from Git which last ran with 1.1.77

Ted Klein (Sep 01 2021 at 19:37):

OK, a quite large number of the new errors were triggered by the code system 'v3-AmericanIndianAlaskaNativeLanuages.xml. Here are two exemplars of the scores (perhaps hundreds?) of errors that are new with 1.1.78:

Ted Klein (Sep 01 2021 at 19:37):

CodeSystem.concept[0].concept[0].concept[0].concept[0].concept[0].designation[0].use error Error performing operation 'validate-code: null' (parameters = "") for 'http://snomed.info/sct#900000000000013009' (see Tx log)
CodeSystem.concept[0].concept[0].concept[0].concept[0].concept[1].designation[0].use error Error performing operation 'validate-code: null' (parameters = "") for 'http://snomed.info/sct#900000000000013009' (see Tx log)

Ted Klein (Sep 01 2021 at 19:40):

The value used for designationUse in CodeSystem.concept in the source is the same value we have been using for a few years, and can be viewed at http://hl7.org/fhir/valueset-designation-use.html

Ted Klein (Sep 01 2021 at 19:41):

but may be a validation issue with some gnarly bit of tx.fhir.org; now you are outside of my limited knowledge of how this stuff is validated during Publisher validation. I leave it in you fellows' capable hands from here.

Ted Klein (Sep 01 2021 at 23:14):

Did someone - @Grahame Grieve or @Lloyd McKenzie change something with the UTG template in the package library? I can build locally, but the main build (triggered by the git push webhook) fails with:

Ted Klein (Sep 01 2021 at 23:14):

Package Cache: var/lib/.fhir/packages (00:00.0011)
Load Template from hl7.utg.template (00:00.0856)
2021-09-01 22:49:06.304 [main] INFO o.h.f.u.npm.BasePackageCacheManager [BasePackageCacheManager.java:80] Failed to resolve package hl7.utg.template#null from server: http://packages.fhir.org
2021-09-01 22:49:06.844 [main] INFO o.h.f.u.npm.BasePackageCacheManager [BasePackageCacheManager.java:80] Failed to resolve package hl7.utg.template#null from server: https://packages2.fhir.org/packages
2021-09-01 22:49:07.100 [main] INFO o.h.f.u.npm.BasePackageCacheManager [BasePackageCacheManager.java:80] Failed to resolve package hl7.utg.template#null.x from server: http://packages.fhir.org
2021-09-01 22:49:07.299 [main] INFO o.h.f.u.npm.BasePackageCacheManager [BasePackageCacheManager.java:80] Failed to resolve package hl7.utg.template#null.x from server: https://packages2.fhir.org/packages
Publishing Content Failed: Error loading template hl7.utg.template: null (00:02.0549)

Grahame Grieve (Sep 02 2021 at 00:08):

oh right

Grahame Grieve (Sep 02 2021 at 00:09):

update and try again

Ted Klein (Sep 02 2021 at 02:17):

tsk tsk. ok, I'll tickle it but won't check until tomorrow, getting late-ish here and I need to arise early tomorrow...

Ted Klein (Sep 02 2021 at 02:21):

LOL just failed again instantly this time:

Ted Klein (Sep 02 2021 at 02:21):

API keys loaded from /etc/ig.builder.keyfile.ini (00:00.0008)
Package Cache: var/lib/.fhir/packages (00:00.0011)
Load Template from hl7.utg.template (00:00.0896)
2021-09-02 02:19:17.774 [main] INFO o.h.f.u.npm.BasePackageCacheManager [BasePackageCacheManager.java:80] Failed to resolve package hl7.utg.template#null from server: http://packages.fhir.org
2021-09-02 02:19:18.410 [main] INFO o.h.f.u.npm.BasePackageCacheManager [BasePackageCacheManager.java:80] Failed to resolve package hl7.utg.template#null from server: https://packages2.fhir.org/packages
2021-09-02 02:19:19.022 [main] INFO o.h.f.u.npm.BasePackageCacheManager [BasePackageCacheManager.java:80] Failed to resolve package hl7.utg.template#null.x from server: http://packages.fhir.org
2021-09-02 02:19:19.224 [main] INFO o.h.f.u.npm.BasePackageCacheManager [BasePackageCacheManager.java:80] Failed to resolve package hl7.utg.template#null.x from server: https://packages2.fhir.org/packages
Publishing Content Failed: Error loading template hl7.utg.template: null (00:03.0408)

Grahame Grieve (Sep 02 2021 at 02:21):

you didn't update

Ted Klein (Sep 02 2021 at 02:22):

Update...ummm...new publisher? update what? The templates in the source?

Grahame Grieve (Sep 02 2021 at 02:23):

get the latest version of UTG

Ted Klein (Sep 02 2021 at 02:23):

oh you pushed source to git?

Grahame Grieve (Sep 02 2021 at 02:23):

y

Ted Klein (Sep 02 2021 at 02:24):

ugh...hard to update the branches with source changes due to horrendous merge conflicts...lemme see...I can update my master...what file(s) did you update so I can just pull those from master into the branches in question?

Ted Klein (Sep 02 2021 at 02:26):

Also - when we update the full source for UTG in Git...we must also update the BitBucket copy as that is what the unwashed public uses as a clone point for the branches that they create for changes...I do that as part of my process whenever I update the git master

Grahame Grieve (Sep 02 2021 at 02:26):

it's a simple fix:

Grahame Grieve (Sep 02 2021 at 02:26):

https://github.com/HL7/UTG/commit/614a2e00e23860cf4a48ad9c8241e30d6b91a4e8

Ted Klein (Sep 02 2021 at 02:30):

thanks - kicked it off again...I don't get any notifications when something like this is done to the source in areas that I do not generally touch...sigh...

Grahame Grieve (Sep 02 2021 at 02:30):

no indeed

Ted Klein (Oct 25 2021 at 13:39):

@Grahame Grieve Something changed over the weekend and I can no longer do any builds of UTG/THO - I cannot find any notification of something needed...error is after all the publisher operations are done. On the automated one running at build.fhir.org, I get:

Ted Klein (Oct 25 2021 at 13:39):

Generating Narratives (01:50.0114)
Publishing Content Failed: An error occurred trying to process this transaction request (13:26.0128)
(13:26.0128)
Use -? to get command line help (13:26.0129)
(13:26.0129)
Stack Dump (for debugging): (13:26.0129)
org.hl7.fhir.r5.utils.client.EFhirClientException: An error occurred trying to process this transaction request
at org.hl7.fhir.r5.utils.client.FHIRToolingClient.handleException(FHIRToolingClient.java:373)

Ted Klein (Oct 25 2021 at 13:40):

On my local builds, all fail even those that worked previously just fine. All like:

Ted Klein (Oct 25 2021 at 13:40):

Publishing Content Failed: An error occurred trying to process this transaction request (05:10.0189)
(05:10.0190)
Use -? to get command line help (05:10.0190)
(05:10.0191)
Stack Dump (for debugging): (05:10.0192)
org.hl7.fhir.r5.utils.client.EFhirClientException: An error occurred trying to process this transaction request
at org.hl7.fhir.r5.utils.client.FHIRToolingClient.handleException(FHIRToolingClient.java:373)
at org.hl7.fhir.r5.utils.client.FHIRToolingClient.transaction(FHIRToolingClient.java:341)

Ted Klein (Oct 25 2021 at 13:41):

Does some change need to be done to the ig.ini file? If so, where are such things usually announced so I can make sure I am subscribed?

John Moehrke (Oct 25 2021 at 13:54):

my experience shows that it is likely the new IG publisher. I have reported details on the #IG creation

Ted Klein (Oct 25 2021 at 14:07):

seems to fail with my old version 82 publisher locally...hmmm...

John Moehrke (Oct 25 2021 at 14:27):

okay, then not the same problem... or not as likely the same problem

Ted Klein (Oct 26 2021 at 16:37):

@Grahame Grieve verified that the issue is the same local builds and git-push-triggered builds of UTG out at build.fhir.org; several minutes into validating conformance resources it fails with java errors. Here is a snippet of the log; all branches fail the same way (trying to build things that build without error last week). Same failure on publisher 1.1.82 and 1.1.84

Ted Klein (Oct 26 2021 at 16:37):

Validating Conformance Resources (01:37.0997)
Publishing Content Failed: An error occurred trying to process this transaction request (03:58.0420)
(03:58.0420)
Use -? to get command line help (03:58.0421)
(03:58.0422)
Stack Dump (for debugging): (03:58.0422)
org.hl7.fhir.r5.utils.client.EFhirClientException: An error occurred trying to process this transaction request
at org.hl7.fhir.r5.utils.client.FHIRToolingClient.handleException(FHIRToolingClient.java:373)
at org.hl7.fhir.r5.utils.client.FHIRToolingClient.transaction(FHIRToolingClient.java:341)
at org.hl7.fhir.convertors.txClient.TerminologyClientR5.validateBatch(TerminologyClientR5.java:119)
at org.hl7.fhir.r5.context.BaseWorkerContext.validateCodeBatch(BaseWorkerContext.java:830)
at org.hl7.fhir.validation.instance.type.ValueSetValidator.validateValueSetInclude(ValueSetValidator.java:123)
at org.hl7.fhir.validation.instance.type.ValueSetValidator.validateValueSetCompose(ValueSetValidator.java:68)
at org.hl7.fhir.validation.instance.type.ValueSetValidator.validateValueSet(ValueSetValidator.java:58)
at org.hl7.fhir.validation.instance.InstanceValidator.checkSpecials(InstanceValidator.java:4207)
at org.hl7.fhir.validation.instance.InstanceValidator.startInner(InstanceValidator.java:4179)
at org.hl7.fhir.validation.instance.InstanceValidator.start(InstanceValidator.java:4033)
at org.hl7.fhir.validation.instance.InstanceValidator.validateResource(InstanceValidator.java:5270)
at org.hl7.fhir.validation.instance.InstanceValidator.validate(InstanceValidator.java:746)
at org.hl7.fhir.validation.instance.InstanceValidator.validate(InstanceValidator.java:719)
at org.hl7.fhir.igtools.publisher.Publisher.validate(Publisher.java:5071)
at org.hl7.fhir.igtools.publisher.Publisher.validate(Publisher.java:4003)
at org.hl7.fhir.igtools.publisher.Publisher.loadConformance(Publisher.java:3982)
at org.hl7.fhir.igtools.publisher.Publisher.createIg(Publisher.java:901)
at org.hl7.fhir.igtools.publisher.Publisher.execute(Publisher.java:753)
at org.hl7.fhir.igtools.publisher.Publisher.main(Publisher.java:8766)
Caused by: java.net.SocketException: Broken pipe (Write failed)
at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)

Ted Klein (Oct 26 2021 at 16:37):

Validated that others can reproduce this error as well. So THO/UTG builds are dead in the water now.

Ted Klein (Oct 26 2021 at 16:38):

started also with a clean clone of the branch with empty txcache and temp folders. same problem.

Ted Klein (Oct 26 2021 at 16:39):

I see in the log it is validating against r5 items...do I need to change the package reference in the ig.ini file? Something as simple as that?

Grahame Grieve (Oct 26 2021 at 19:36):

no. everything is R5 internally in the tool. I had my head down yesterday working through supporting infrastructure issues. Now I'm looking at UTG directly

Ted Klein (Oct 26 2021 at 20:02):

new piece of data just in last minutes:

Ted Klein (Oct 26 2021 at 20:03):

an old branch 2.1.9 is building without the error; something may have been added which was always not correct and error not caught, but some change triggered a failure

Ted Klein (Oct 26 2021 at 20:03):

starting the UTG call now, ttyl

Grahame Grieve (Oct 26 2021 at 20:42):

ok it builds fine. I had to fix a configuration issue on the server (UTG really does hammer the system)

Ted Klein (Oct 26 2021 at 20:43):

THANK YOU!!!!!

Ted Klein (Nov 12 2021 at 17:29):

New problem just introduced: I was in the middle of doing the implementation of a proposal (UP-251) and it ran perfectly locally, but a new error I have never seen crashes Jekyll on the Git-triggered main build at build.fhir.org - noticed that the identical source builds perfectly with publisher 1.1.84 but fails in this way locally also now - updated my local version from 1.1.84 to 1.1.85

Ted Klein (Nov 12 2021 at 17:29):

So something was introduced that is breaking THO builds @Grahame Grieve

Ted Klein (Nov 12 2021 at 17:30):

The error now locally as well as on the web build is:

Ted Klein (Nov 12 2021 at 17:30):

Jekyll: Generating... (26:25.0729)
Jekyll has failed. Complete output from running Jekyll: Liquid Exception: Could not locate the included file 'CodeSystem-measure-type-xref.xhtml' in any of ["/Users/tedklein/Documents/tedhl7/vocabulary/Projects/UTG/gitImages/in-flight/UP-251/temp/pages/_includes"]. Ensure it exists in one of those direct (27:26.0856)
Note: Check that Jekyll is installed correctly (27:26.0858)

Ted Klein (Nov 12 2021 at 17:31):

This resource it is complaining about 'measure-type-xref' does NOT exist in the input source material

Ted Klein (Nov 12 2021 at 17:31):

Please advise

Ted Klein (Nov 12 2021 at 18:15):

@Grahame Grieve same error on local rebuild of the THO ci build which builds flawlessly with Publisher 1.1.84; perhaps @Mark Iantorno you can shed some light on what is going on here?

Grahame Grieve (Nov 15 2021 at 03:56):

hmm. I'll have a look

Ted Klein (Nov 15 2021 at 15:08):

on the last ci build with Publisher 1.1.84, the file measure-type-xref does not exist anywhere, in any xml or html output or bundle or anywhere. so something very odd going on.

Ted Klein (Nov 16 2021 at 02:01):

tested a local build with Publisher 1.1.86 just now @Grahame Grieve and it fails the same way with the same error:

Ted Klein (Nov 16 2021 at 02:01):

Reclaiming memory...

Memory (MB): Use = 2107, Free = 4988, Total = 7096, Max =16384

Jekyll: Source: /Users/tedklein/Documents/tedhl7/vocabulary/Projects/UTG/gitImages/UTG/temp/pages (30:59.0288)
Jekyll: Generating... (30:59.0289)
Jekyll has failed. Complete output from running Jekyll: Liquid Exception: Could not locate the included file 'CodeSystem-measure-type-xref.xhtml' in any of ["/Users/tedklein/Documents/tedhl7/vocabulary/Projects/UTG/gitImages/UTG/temp/pages/_includes"]. Ensure it exists in one of those directories and, if (31:56.0812)
Note: Check that Jekyll is installed correctly (31:56.0814)
Publishing Content Failed: Process exited with an error: 1 (Exit value: 1) (31:56.0816)
(31:56.0817)
Use -? to get command line help (31:56.0818)
(31:56.0819)
Stack Dump (for debugging): (31:56.0819)
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153)
at org.hl7.fhir.igtools.publisher.Publisher.runJekyll(Publisher.java:5879)
at org.hl7.fhir.igtools.publisher.Publisher.runTool(Publisher.java:5801)
at org.hl7.fhir.igtools.publisher.Publisher.generate(Publisher.java:5186)
at org.hl7.fhir.igtools.publisher.Publisher.createIg(Publisher.java:916)
at org.hl7.fhir.igtools.publisher.Publisher.execute(Publisher.java:755)
at org.hl7.fhir.igtools.publisher.Publisher.main(Publisher.java:8777)
teds-third-mbp:FHIR_tool_stuff tedklein$

Ted Klein (Nov 16 2021 at 02:07):

IG publisher 1.1.84 and earlier actually created this in the .../temp/pages/_includes folder; from the contents of it this looks like some kinda of hosement in the combination of the UTG/THO content with the FHIR R4 content...the files that is created contains:

Ted Klein (Nov 16 2021 at 02:07):

{% raw %}
<ul>
<li><a href="http://hl7.org/fhir/R4/valueset-measure-type.html">MeasureType</a></li>
<li><a href="ValueSet-measure-type.html">MeasureType</a></li>
</ul>
{% %}

Ted Klein (Nov 16 2021 at 02:08):

But this file is no longer being created during the Jekyll processing apparently...

Ted Klein (Nov 16 2021 at 02:09):

This may have been one item that was updated (the value set ValueSet-measure-type) in UTG, or perhaps it is a version processing issue?

Ted Klein (Nov 16 2021 at 02:15):

The value set source in UTG is <version value="0.2.0"/>; in the FHIR core it is <version value="4.0.1"/> . The original version that you assigned to the value sets brought into THO from FHIR R4.0.1 was "0.1.0". Did this change mess up your processing in some way @Grahame Grieve ? Were we supposed to do something with these values in <version> for the FHIR value sets before R5?

Grahame Grieve (Nov 16 2021 at 04:22):

it's a bit mystifying why this is version dependent on the IG publisher - none of the relevent code has been changed for months.

Grahame Grieve (Nov 16 2021 at 04:22):

in this case, there was a bug checking for deprecation if not all the codes in a code system have a property (?)

Grahame Grieve (Nov 16 2021 at 04:30):

there does seem to be a case consistency problem around v3-hc-AIC

Grahame Grieve (Nov 16 2021 at 06:17):

anyway, fixed. O

Grahame Grieve (Nov 16 2021 at 06:17):

I'll try and get a new version out in the next 12 hours

Ted Klein (Nov 18 2021 at 02:03):

ok thanks, back home I will see what the possible case issue is with v3-hc-AIC. Once the build is clean, you and I need to schedule when to do the THO release 3.0.0; TSC has approved the Publication requests and it is now on my docket.

Grahame Grieve (Nov 18 2021 at 02:14):

and the new issues?

Ted Klein (Nov 18 2021 at 14:48):

Good morning just back from NYC...so as soon as you release IGP 1.1.87 I will check a build with it...or you can try a local THO build with either git master (the ci build) or the content in git branch UP-251 (where I first discovered the error). Or we can see if it fails even on the last release set 2.1.0; let me know how to help because the UTG proposal processing, the release, everything THO/UTG is dead in the water until this gets addressed. THEN we can look at the fixes for the 100s of warning from the new validation you are introducing on hierarchy meaning...

Grahame Grieve (Nov 18 2021 at 19:49):

well, I got the publisher out at last

Ted Klein (Nov 18 2021 at 23:16):

building locally now to check it - I have a question on the case mismatch error - the validation log seems to imply a mismatch between the ci build and release 2.1.0 on v3-hc-aic...but the submitter of a change a month or two ago explicitly changed the case of it from "v3-hc-AIC" in 2.1.0 to "v3-hc-aic" in the ci build (git master) now. Is this causing the problem? Is this supposed to be not doable?

Grahame Grieve (Nov 19 2021 at 18:50):

it's not consistently changed. The case renaming may cause problems in the previous version comparison, but I wouldn't comment about that; the code systems list was not updated

Ted Klein (Nov 23 2021 at 15:55):

I see in the temp folders you have the release cache I guess for check against previous release...there are a whole bunch of resources (mostly from Canada Health Infoway) which had their case changed.

Abdullah Rafiqi (Dec 03 2021 at 19:57):

(deleted)

Marc Duteau (Feb 14 2022 at 22:15):

Running the latest version of the master through version 1.1.99 of the publisher it builds fine, but now with 1.1.102 It fails to build after reaching the 'generate summary outputs' stage with this message:

Exception in thread "main" java.lang.Error: The resource type "NamingSystem" does not implement the property "experimental"
at org.hl7.fhir.r5.model.NamingSystem.getExperimental(NamingSystem.java:1768)
at org.hl7.fhir.igtools.publisher.Publisher.populateResourceEntry(Publisher.java:6926)
at org.hl7.fhir.igtools.publisher.Publisher.generateSummaryOutputs(Publisher.java:6828)
at org.hl7.fhir.igtools.publisher.Publisher.generate(Publisher.java:5862)
at org.hl7.fhir.igtools.publisher.Publisher.createIg(Publisher.java:1018)
at org.hl7.fhir.igtools.publisher.Publisher.execute(Publisher.java:856)
at org.hl7.fhir.igtools.publisher.Publisher.main(Publisher.java:10007)

Grahame Grieve (Feb 14 2022 at 22:24):

oh

Grahame Grieve (Feb 14 2022 at 22:24):

yay @Lloyd McKenzie

Lloyd McKenzie (Feb 15 2022 at 03:31):

I don't think I ever argued against experimental, just url.

Ted Klein (Feb 15 2022 at 03:31):

still fails the same way with publisher 1.1.102 - note that I do not see (or find) any such property in any of the NamingSystem resources in the UTG source in the UTG Git master set

Ted Klein (Feb 15 2022 at 03:32):

@Lloyd McKenzie can you give me some hint as to how to find this issue? Or is it something funky in the infrastructure?

Lloyd McKenzie (Feb 15 2022 at 03:33):

Did you intend to have experimental namingsystems?

Ted Klein (Feb 15 2022 at 03:33):

Absolutely not

Lloyd McKenzie (Feb 15 2022 at 03:33):

If that wasn't deliberate, fixing that is the fastest thing to change. Though the code will have to be adjusted to handle it where they're legitimate

Ted Klein (Feb 15 2022 at 03:34):

the only place we use experimental is in CodeSystem resources

Ted Klein (Feb 15 2022 at 03:34):

and I think that is only in the FHIR ones that were imported last year from the old R4 content leftover from Connectathons

Ted Klein (Feb 15 2022 at 03:39):

THO has 9 instance where the CodeSystem.experimental element is set to 'true' - all FHIR stuff. Sorry, 5 instances in CodeSystem, 4 more in ValueSet. All of them FHIR imported items. All of the other instances of it in THO master content is for CodeSystem.experimental value="false". I see it nowhere in any NamingSystem. Is this some oddball thing in the templates?

Lloyd McKenzie (Feb 15 2022 at 03:40):

Actually, the issue is with the CanonicalResource code. It doesn't handle the fact that, despite NamingSystem being a canonical resource, it doesn't have an experimental element.

Lloyd McKenzie (Feb 15 2022 at 03:40):

(And that bit's not my code @Grahame Grieve ...)

Grahame Grieve (Feb 15 2022 at 03:44):

Courtesy of resistance in committee from .. someone… naming system is a canonical resource that doesn’t have a ‘experimental’. So you need to consult the resource metadata using ‘supportsX’ to see if it does before assuming it does. But you didn’t.

Ted Klein (Feb 15 2022 at 03:45):

ROTFL. ok I see. And I totally understand it, since NamingSystem as always been a kind of incomplete red-headed stepchild of a resource anyway. LOL. If the element is added, we will need to add its default value to hundreds of them in the THO source...I don't really care either way, but it is yet another giant heap of clerical work - btw @Grahame Grieve I need to discuss with you doing the <caseSensitive> update...

Lloyd McKenzie (Feb 15 2022 at 03:46):

I never objected to experimental. Why doesn't the interface just automatically return 'false' if experimental doesn't exist? That seems like the logical thing to do... In any case, do you need me to make a pull request or will you just take care of it?

Grahame Grieve (Feb 15 2022 at 03:56):

I took care of it

Ted Klein (Feb 15 2022 at 03:56):

I do need a fix pretty quick...cannot do any builds with any publisher post 1.1.98, local or otherwise...

Ted Klein (Feb 15 2022 at 03:57):

ah ok thanks, I'll kick off another build now then (local of the UTG Git master) with 1.1.102...

Grahame Grieve (Feb 15 2022 at 04:00):

I’ll get another release out when I can

Ted Klein (Feb 15 2022 at 04:02):

ah so it needs to wait until 1.1.103 then? I'll tell Marc D to stand down on attempting the implementation of another ticket that was approved to be integrated into master until it is released then...

Ted Klein (Feb 15 2022 at 14:53):

ok 1.1.103 gets it back to working shape, thank you very much @Grahame Grieve !

Ted Klein (Feb 16 2022 at 01:37):

ok something new with the publisher in building THO (and the UTG branches for changes):

Ted Klein (Feb 16 2022 at 01:37):

rg.hl7.fhir.exceptions.FHIRException: Error loading epsg-crs.cache: begin 0, end -1, length 13 entry 1
at org.hl7.fhir.r5.context.TerminologyCache.loadNamedCache(TerminologyCache.java:603)
at org.hl7.fhir.r5.context.TerminologyCache.load(TerminologyCache.java:614)
at org.hl7.fhir.r5.context.TerminologyCache.<init>(TerminologyCache.java:184)
at org.hl7.fhir.r5.context.BaseWorkerContext.initTS(BaseWorkerContext.java:1260)
at org.hl7.fhir.r5.context.SimpleWorkerContext$SimpleWorkerContextBuilder.build(SimpleWorkerContext.java:228)
at org.hl7.fhir.r5.context.SimpleWorkerContext$SimpleWorkerContextBuilder.fromPackage(SimpleWorkerContext.java:246)
at org.hl7.fhir.igtools.publisher.PreviousVersionComparator.startChecks(PreviousVersionComparator.java:203)
at org.hl7.fhir.igtools.publisher.Publisher.checkConformanceResources(Publisher.java:4766)
at org.hl7.fhir.igtools.publisher.Publisher.loadConformance(Publisher.java:4680)
at org.hl7.fhir.igtools.publisher.Publisher.createIg(Publisher.java:1005)
at org.hl7.fhir.igtools.publisher.Publisher.execute(Publisher.java:856)
at org.hl7.fhir.igtools.publisher.Publisher.main(Publisher.java:10001)
Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end -1, length 13
at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3410)
at java.base/java.lang.String.substring(String.java:1883)
at org.hl7.fhir.r5.context.TerminologyCache.loadNamedCache(TerminologyCache.java:592)
... 11 more
Validating Resources (18:33.0927)

Ted Klein (Feb 16 2022 at 01:38):

this does not go away with removing the cache; is it related to the 3.0.0 folder of cache files? ie does that need to be removed also to clear this error?

Grahame Grieve (Feb 16 2022 at 01:58):

certainly try

Ted Klein (Feb 16 2022 at 12:54):

well then...now a new problem @Grahame Grieve - it appears that publisher 1.1.104 has broken THO/UTG builds:

Ted Klein (Feb 16 2022 at 12:54):

Generating Summary Outputs (01:02:08.0576)
Sending Usage Stats to Server (01:04:36.0438)

Memory (MB): Use = 3888, Free = 14478, Total = 18366, Max =18366

Reclaiming memory...

Memory (MB): Use = 2478, Free = 15888, Total = 18366, Max =18366

Run jekyll: jekyll build --destination "/scratch/ig-build-temp-37POM7/repo/output" (in folder /scratch/ig-build-temp-37POM7/repo/temp/pages) (01:04:42.0958)
Jekyll: Source: /scratch/ig-build-temp-37POM7/repo/temp/pages (01:04:43.0328)
Jekyll: Generating... (01:04:43.0329)
Jekyll has failed. Complete output from running Jekyll:  Liquid Exception: Could not locate the included file 'fragment-simpletable.html' in any of ["/scratch/ig-build-temp-37POM7/repo/temp/pages/_includes"]. Ensure it exists in one of those directories and, if it is a symlink, does not point outside your (01:04:48.0207)
Note: Check that Jekyll is installed correctly (01:04:48.0207)
Publishing Content Failed: Process exited with an error: 1 (Exit value: 1) (01:04:48.0207)
(01:04:48.0207)
Use -? to get command line help (01:04:48.0207)
(01:04:48.0207)
Stack Dump (for debugging): (01:04:48.0208)
org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153)
at org.hl7.fhir.igtools.publisher.Publisher.runJekyll(Publisher.java:6649)
at org.hl7.fhir.igtools.publisher.Publisher.runTool(Publisher.java:6561)
at org.hl7.fhir.igtools.publisher.Publisher.generate(Publisher.java:5914)
at org.hl7.fhir.igtools.publisher.Publisher.createIg(Publisher.java:1018)
at org.hl7.fhir.igtools.publisher.Publisher.execute(Publisher.java:856)
at org.hl7.fhir.igtools.publisher.Publisher.main(Publisher.java:10001)

Ted Klein (Feb 16 2022 at 12:55):

do you and Mark I actually run a THO build to verify before you do a new publisher release?

Ted Klein (Feb 16 2022 at 12:56):

it seems to be able to be built just fine with publisher 1.1.103

John Moehrke (Feb 16 2022 at 12:56):

watch the #IG creation stream, many others have reported this

John Moehrke (Feb 16 2022 at 12:56):

I think...

Ted Klein (Feb 16 2022 at 12:57):

oh geez...seems have to watch like 100 streams...sigh...thanks John.

John Moehrke (Feb 16 2022 at 13:00):

i watch them all...

Ted Klein (Feb 16 2022 at 13:06):

so...it looks like maybe a template update issue with 1.1.104...sigh...we have a raft of UTG change proposals in flight which all use the old template...this is just craziness...

Ted Klein (Feb 16 2022 at 13:09):

in my THO ig.ini file I use:

Ted Klein (Feb 16 2022 at 13:09):

[IG]
ig = input/utg.xml
template = hl7.utg.template#current

John Moehrke (Feb 16 2022 at 13:10):

yup, get rid of that "#current"

Ted Klein (Feb 16 2022 at 13:10):

oh geez thank you John lemme try again...

John Moehrke (Feb 16 2022 at 13:11):

we all have learned over the past month that the use of #current is almost always wrong, as that tells the build to use the very latest build even if that build is an experimental build by the template gods.

Grahame Grieve (Feb 16 2022 at 13:12):

no i think THO has to use current, but it should be fixed now anyway

Ted Klein (Feb 16 2022 at 13:13):

ah ok lemme try again both ways...yes Grahame I thought UTG needed to use current

John Moehrke (Feb 16 2022 at 13:13):

okay,, but was my understanding right? (Note that I looked at the various derivation templates and they all seem to use #current of their previous

Ted Klein (Feb 16 2022 at 13:14):

is it fixed in the upcoming 105? or was the current template case error fixed?

Ted Klein (Feb 16 2022 at 13:15):

ie. do I need to pull a new template for the UTG branches...

Ted Klein (Feb 16 2022 at 14:35):

so it worked with 104 with the current removed; I will see if it is fixed with it put back.

John Moehrke (Feb 16 2022 at 15:27):

it is not an IG builder issue... it is a base template issue... In theory, it is fixed now.

Grahame Grieve (Feb 16 2022 at 16:23):

well, one of my challenges is that I have not, as yet, figured out how to get THO to run on my mac. It's that javascript / java issue. And I have so many other issues with my mac right now that this is down the list

Grahame Grieve (Feb 16 2022 at 16:24):

but THO takes too long to run for me to routinely run it - the test suite for the IG publisher already takes 45minutes as it is

Ted Klein (Feb 16 2022 at 16:55):

hmmm...that is very very odd. I run the THO builds in OSX on my Mac all the time...but it is an older one, two revs back. Only 15 minutes? Hahahahahaha...every UTG branch build for a change proposal takes me 23 minutes to do the build. And I do a LOT of them...

Ted Klein (Feb 16 2022 at 16:57):

and also John, the last run of THO I kicked off at build.fhir.org had the current added back into the ig.ini file - and it failed again 10 minutes ago with 1.1.104. So I reverted it to the removal of current and kicked it off again. We are bleeding from the eyes trying to get a whole bunch of updates done before the THO content freeze on Sunday...

Grahame Grieve (Feb 16 2022 at 17:18):

well, we're all struggling with deadlines right now

Rob Hausam (Feb 16 2022 at 17:28):

I've never had problems with running THO on my Mac (2020 Macbook Air M1 on latest Monterey). I'll try it again now and see if there are any new issues.

John Moehrke (Feb 16 2022 at 17:56):

is that related to sun/oracle java vs open java? Because that is MY problem. Open java does not have a javascript executable.

Rob Hausam (Feb 16 2022 at 17:57):

Built UTG successfully on my Mac with .104 Publisher in 19:40.

Rob Hausam (Feb 16 2022 at 17:58):

I don't know. I'm using the liberica-11.0.12+7 JDK (native M1).

Ted Klein (Feb 16 2022 at 20:54):

yes it is working locally I committed the template update in the ig file earlier

Ted Klein (Feb 16 2022 at 21:19):

so there seems to be some issue with the current template and publisher 1.1.104; I can get things to build with no misbehavior with

Ted Klein (Feb 16 2022 at 21:19):

[IG]
ig = input/utg.xml
template = hl7.utg.template

Ted Klein (Feb 16 2022 at 21:20):

but different kinds of problems seem to occur with

Ted Klein (Feb 16 2022 at 21:20):

[IG]
ig = input/utg.xml
template = hl7.utg.template#current

Ted Klein (Feb 16 2022 at 21:20):

these problems do not occur with publisher 1.1.103

Ted Klein (Feb 16 2022 at 21:20):

running more tests in the meantime

Grahame Grieve (Feb 16 2022 at 22:35):

well, UTG builds for me as is

Grahame Grieve (Feb 16 2022 at 22:35):

using 1.1.104

Rob Hausam (Feb 16 2022 at 23:11):

same

Ted Klein (Feb 17 2022 at 13:56):

whatever it was seems to have cleared all are building fine now. I am a bit mystified.

Ted Klein (Feb 17 2022 at 17:49):

I continue to have difficulties getting warnings of various kinds suppressed using the ignore warnings.txt file - it seems that only certain kinds of warnings can be effectively suppressed, not any warning. Anyone have an insight on this? Just a quirk in the publisher?

Jean Duteau (Feb 17 2022 at 18:13):

any kind of warning can be suppressed as long as it is an actual warning. The easiest way is to look in the qa.txt and see the exact text of the warning to suppress.

Ted Klein (Feb 17 2022 at 18:43):

I did that and copied the text from the qa.txt file directly into ignorewarnings.txt, but the warnings are still being generated - and the count of number of warnings does not go down

Ted Klein (Feb 17 2022 at 18:43):

hence my question

Ted Klein (Feb 17 2022 at 18:43):

for example:

Ted Klein (Feb 17 2022 at 18:45):

lines 4389-4391 in the qa.txt file:

Ted Klein (Feb 17 2022 at 18:45):

== input/sourceOfTruth/external/v3/codeSystems/cs-v3-ndc.xml ==
WARNING: CodeSystem/v3-ndc: CodeSystem: HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly
== inpu...

Ted Klein (Feb 17 2022 at 18:45):

what I have un the ingorrewarnings.txt file:

Ted Klein (Feb 17 2022 at 18:46):

WARNING: CodeSystem/v3-ndc: CodeSystem: HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly
WARNI...

Ted Klein (Feb 17 2022 at 18:46):

and in the generated output it still has the warning, rendered as:

Ted Klein (Feb 17 2022 at 18:47):

input/sourceOfTruth/external/v3/codeSystems/cs-v3-ndc.xml Show Validation Information (1)
Path Severity Message
CodeSystem warning HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly

Ted Klein (Feb 17 2022 at 18:48):

so...what am I doing wrong? this is using publisher 1.1.104

Ted Klein (Feb 17 2022 at 18:48):

other warnings are suppressed just fine, but these ones (have a few hundred of them) I cannot seem to be able to suppress them

Jean Duteau (Feb 17 2022 at 19:01):

You should have:
HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly

It's the text of the warning that is important and that should remove all of the caseSensitive warnings that you get.

Ted Klein (Feb 17 2022 at 19:03):

ok so the header "WARNING: " should not be there, eh? I'll give that a try and build again. thx.

Ted Klein (Feb 17 2022 at 19:05):

I don't want to suppress all of them, just specific ones. if a submitter of a new code system resource forgets to put it in, I do want the warning to appear

Lloyd McKenzie (Feb 17 2022 at 20:22):

You either include the fully line with WARNING: and the location or you include only the text following the colon after the location. Which you use depends on whether you want to suppress all occurrences, regardless of location or want to suppress only those specific locations you've validated are not a problem.

Lloyd McKenzie (Feb 17 2022 at 20:23):

It looks like what you had in your original ignorewarnings.txt file was right - possible there was an issue with CR/LF or some non-visible character or something else weird

Ted Klein (Feb 17 2022 at 20:44):

well I just ran it both ways and neither works as you describe, Lloyd. So...guess again.

Ted Klein (Feb 17 2022 at 20:45):

been trying every combination and permutation since this morning, all to no avail.

Grahame Grieve (Feb 17 2022 at 20:47):

actually, this is my oversight, to not mention this. You don't need to suppress these issues because they'll go away in the next release

Ted Klein (Feb 17 2022 at 20:50):

so...when someone adds a CodeSystrem resource and it does not have it, we definitely want it to show up. I cannot get a decision of what to do about the metadata resources for non-HL7 code systems...what I THINK should happen is that HTA should be documenting if true or false in their master record, and we implement it in our CS metadata record accurately. In the meantime, I'm trying to suppress the ones that we have not completed for the release, but undo the suppression afterwards as a reminder to them to get the information.

Ted Klein (Feb 17 2022 at 20:51):

so the changes to the source in ticket UP-286 will take care of the ones for the HL7 systems, but for the externals...are you going to make them go away in the publisher for those CS not anchored in THO?

Ted Klein (Feb 17 2022 at 20:52):

which I think would be perfectly fine...

Grahame Grieve (Feb 17 2022 at 20:58):

oh no - sorry. I'm confused here. The warning will stay - it's still important to tell people whether it's case sensitive or not, but we just didn't know how to automatically populate it

Grahame Grieve (Feb 17 2022 at 20:59):

I was considering removing it, but decided not to late last night. grrr. So yes, it's a warning suppression thing. But that should work

Ted Klein (Feb 17 2022 at 21:05):

Well...let me make sure that I do not misunderstand - we want the warning (good), but we need to be able to suppress either one at a time or the whole shebang for a 'clean' release if possible. One at a time is more manual work, but it does not hide stuff that way.

Ted Klein (Feb 17 2022 at 21:51):

so if I cannot suppress the errors individually, then as just decided on the Vocab call we will need to add the element with the surmise of 'true' to the external terminology metadata record CodeSystem resources.

Ted Klein (Feb 18 2022 at 03:13):

The warning suppression of that particular warning does not seem to work. I'

Ted Klein (Feb 18 2022 at 03:13):

I'd really like it to

Grahame Grieve (Feb 18 2022 at 13:23):

I'll investigate

Ted Klein (Feb 18 2022 at 18:00):

I tried all of the following permutations, and none suppressed the error. Am I doing something wrong?

Ted Klein (Feb 18 2022 at 18:00):

== input/sourceOfTruth/v3/codeSystems/v3-SubstitutionCondition.xml ==
WARNING: CodeSystem/v3-SubstitutionCondition: CodeSystem: HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly

Ted Klein (Feb 18 2022 at 18:01):

WARNING: CodeSystem/v3-SubstitutionCondition: CodeSystem: HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly

Ted Klein (Feb 18 2022 at 18:01):

CodeSystem/v3-SubstitutionCondition: CodeSystem: HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly

Ted Klein (Feb 18 2022 at 18:01):

CodeSystem: HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly

Ted Klein (Feb 18 2022 at 18:01):

HL7 Defined CodeSystems SHOULD have a stated value for the caseSensitive element so that users know the status and meaning of the code system clearly

Lloyd McKenzie (Feb 18 2022 at 18:22):

I don't see a "# Explanation of suppression reason" in front of your messages - presume you have one?

Ted Klein (Feb 18 2022 at 19:05):

I'm sorry Lloyd, I do not understand. The only thing I have ever been told is to copy the warning from the qa.txt file into the ignorewarnings.txt file. I already have a whole bunch of ignored warnings in that tile the work just fine. Things such as:

Ted Klein (Feb 18 2022 at 19:06):

We don't care about any of these - and they create noise

Validate resource against profile http://hl7.org/fhir/StructureDefinition/Bundle
Validate resource against profile http://hl7.org/fhir/StructureDefinition/C...

Ted Klein (Feb 18 2022 at 19:06):

and

Ted Klein (Feb 18 2022 at 19:06):

value sets based on external code systems where we do not have access to the content

WARNING: v2-notAllCodes-0347: Error from server: Unable to provide support for code system urn:iso:std:iso:3166:-2
WARNING: v3-ActInjuryCodeCSA: Error from server: Unable to provide support for ...

Ted Klein (Feb 18 2022 at 19:07):

ah I see that the zulip editor here converted the pound sign to bold. Sigh.

Ted Klein (Feb 18 2022 at 19:08):

Do I need a special section with a comment (first character a pound sign) for each type of warning? If so - I didn't know that - then I'll put it in and try again and it is a simple user error as I do not have a BNF description of what this file needs to contain and how it should be syntactically formatted.

Ted Klein (Feb 18 2022 at 19:08):

Please advise.

Ted Klein (Feb 18 2022 at 20:08):

tried again with the warning the the case sensitivity thing with the pound character header and it again did not suppress the warning. Please advise.

Grahame Grieve (Feb 18 2022 at 21:57):

it's a whitespace problem. The source message ends in a ' ', and the input from the suppressed message file is trimmed, so it can never match

Grahame Grieve (Feb 18 2022 at 21:57):

I'll release a new IG publisher shortly

Grahame Grieve (Feb 20 2022 at 11:13):

released

Last updated: Apr 12 2022 at 19:14 UTC