Stream: IG creation
Topic: CI not working
Brian Reinhold (Oct 15 2021 at 09:38):
I have pushed some changes to GitHub for my IG and previously this has always led to a new IG being generated for build.fhir.org/ig/HL7/PHD. However, that is no longer happening. I pushed some changes yesterday and the IG is still from back in July. Is something down or has the process changed?
Lloyd McKenzie (Oct 15 2021 at 13:10):
Did you monitor your build on the #committers/notification stream?
Brian Reinhold (Oct 15 2021 at 20:18):
@Lloyd McKenzie DId not know it existed. However, there is a HUGE HUGE difference between the result shown in the CI and the same thing done locally. Locally I get 5 errors and I know why (MDC codes not yet in RTMSS). In the CI build I get 89 errors, most of which make no sense. How could the results with the same source files be so different? How can we fix errors if we cannot reproduce them?
Lloyd McKenzie (Oct 15 2021 at 20:28):
Are you using the current version of the publisher locally? Only other possible difference is terminology caching differences
Brian Reinhold (Oct 15 2021 at 20:32):
@Lloyd McKenzie
I did an update yesterday and got the latest and greatest. I'm still only getting 5 errors locally. Could it be the Jira file (FHIR-phd.xml)? THere is a generated one locally but it's not even close to that in the JIRA project. I checked yesterday
Jean Duteau (Oct 15 2021 at 20:32):
no, the JIRA file has nothing to do with the errors you get either locally or in the CI-build. I've just cloned your repository and am running the publisher locally to see what I get
Brian Reinhold (Oct 15 2021 at 20:34):
@Jean Duteau Thanks. I only get 5 errors locally but the CI build seems to somehow have mixed up profiles as if some resource is corrupted. I just did another push and am waiting for the build.
Jean Duteau (Oct 15 2021 at 20:35):
when I do that, I get 89 errors, 18 warnings, and 8 information messages. and I just did a git pull and didn't get anything so I'm not sure that your push actually went through.
John Moehrke (Oct 15 2021 at 20:36):
Your canonical URI does not match the path... in case...
John Moehrke (Oct 15 2021 at 20:36):
I guess you build on WINDOWS... the CI build is linux.. so case matters
Jean Duteau (Oct 15 2021 at 20:37):
and I'm building on a Mac (if case ends up being the problem)
Jean Duteau (Oct 15 2021 at 20:39):
John Moehrke said:
Your canonical URI does not match the path... in case...
In the latest code, that isn't true, in the package-list.json and the IG resource, the canonical appears to match the path.
John Moehrke (Oct 15 2021 at 20:40):
okay
John Moehrke (Oct 15 2021 at 20:40):
didn't seem to match on the build site
Brian Reinhold (Oct 15 2021 at 20:49):
@John Moehrke @Jean Duteau @Lloyd McKenzie
That case issue was supposed to have been fixed. It should have been all lower case. It may not have totally completed. It required changes in places I do not have access to so Lloyd or Grahame or someone else with admin permissions had to do it. I think Lloyd tried to fix it.
Assuming that is the problem. ... and it probably is. I know I have to use upper case in my browser to get the IG. How can this be fixed? I am running on WIndows.
Lloyd McKenzie (Oct 15 2021 at 20:52):
The only place you wouldn't have permission to fix directly would be JiraSpecArtifacts, and that can only trigger a single warning (that your JiraSpecArtifacts file doesn't line up). All the files in Git for your IG you should have full commit rights to...
Lloyd McKenzie (Oct 15 2021 at 20:52):
It may be simplest to just fix it via Git's web interface (if that's the problem).
Chris Moesel (Oct 15 2021 at 20:59):
Jean Duteau said:
and I'm building on a Mac (if case ends up being the problem)
Hmm... by default, the Mac file system is also case-insensitive (like Windows). So it is interesting that you're also getting the 89 errors (if case really is the culprit). Or did you intentionally switch your file system to the case-sensitive format?
John Moehrke (Oct 15 2021 at 21:03):
is it legal to use urn in a code system?
{
"system" : "urn:iso:std:iso:11073:10101",
"code" : "150020"
},
Brian Reinhold (Oct 15 2021 at 21:05):
@John Moehrke That's the official system for 10101 - its right in the FHIR spec. It's been that way for years.
John Moehrke (Oct 15 2021 at 21:06):
I have many cases where historic systems are URN... but I had to move to a FHIR friendly URL... because... no idea, but I am told that programmers don't like URNs, especially OID urns
Brian Reinhold (Oct 15 2021 at 21:13):
@John Moehrke I believe the intent is to change that as the MDC code system gets integrated into FHIR. That will break a lot of uses when it happens; especially my IG and everything based upon it.
John Moehrke (Oct 15 2021 at 21:14):
it just jumped out at me... I have been yelled at for not using urls
John Moehrke (Oct 15 2021 at 21:14):
no idea why that would work locally and fail on ci
Brian Reinhold (Oct 15 2021 at 21:18):
@John Moehrke Do you think the urn: is the problem? It has not been a problem before. I did have the IG down to 5 errors not too long ago but there have been lots of changes that I have not kept up with - I didn't even know about the new stream for the IG builds. Its been a while since I did a build (on GitHub). Have done mostly local after publisher updates
John Moehrke (Oct 15 2021 at 21:18):
Im poking to see what @Lloyd McKenzie might say
John Moehrke (Oct 15 2021 at 21:19):
@Chris Moesel how did your local build go?
Jean Duteau (Oct 15 2021 at 21:19):
to me, the main culprit is that when I dl what is in Git and build it (on my mac and on my windows machine), I get the same as what is in CI, so that says that you haven't committed all of the changes you have on your local system to CI
John Moehrke (Oct 15 2021 at 21:22):
I was wondering that too
Jean Duteau (Oct 15 2021 at 21:24):
okay, just did a pull and now I get 5 errors and 1 warning, so that appears to have been the culprit
John Moehrke (Oct 15 2021 at 21:24):
why is there a ./resources and also ./input/resources ?
John Moehrke (Oct 15 2021 at 21:26):
does the build pull from different pathways local vs ci?
Jean Duteau (Oct 15 2021 at 21:30):
does anyone remember the page that lists all of the IGs and their status?
Jean Duteau (Oct 15 2021 at 21:34):
okay, interesting caching thing going on. when I check out a clean build and run the publisher, I get the 89 errors and 18 warnings. when I run it a 2nd time, I only get 5 errors and 1 warning.
John Moehrke (Oct 15 2021 at 21:35):
https://fhir.github.io/auto-ig-builder/builds.html
John Moehrke (Oct 15 2021 at 21:35):
I get 89 locally
John Moehrke (Oct 15 2021 at 21:40):
I confirm, second build changes to 5
Jean Duteau (Oct 15 2021 at 22:06):
@Grahame Grieve the first time that this IG is run (https://github.com/HL7/phd/) there is no iso1107310101.cache and the validator basically can't determine what codes belong to the various value sets that use the 10101 code system. The 2nd time through the cache file does exist and it then does know what codes exist. Here is the cache file that got created the first run though: iso1107310101.cache
Jean Duteau (Oct 15 2021 at 22:09):
when I delete just that cache file and re-run, I get the 89 errors where it doesn't know what codes are in the code system.
Grahame Grieve (Oct 15 2021 at 22:11):
that doesn't make sense. the cache is built from previous answers
Jean Duteau (Oct 15 2021 at 22:12):
i agree that it doesn't make sense, but it's reproduceable :(
Jean Duteau (Oct 15 2021 at 22:15):
with no cache file - 89 errors/18 warnings
with cache file - 5 errors/1 warning
Brian Reinhold (Oct 15 2021 at 22:20):
@Jean Duteau
Good debug work ! :+1:
Lloyd McKenzie (Oct 15 2021 at 22:29):
@John Moehrke urn's are ok, though resolvable URLs are preferred. OIDs are evil and UUIDs are just as bad when it comes to representing content that developers and IG designers must type and recognize.
Brian Reinhold (Oct 15 2021 at 22:29):
@Jean Duteau I reproduced the error as well. I noted that all the directories and files in the input-cache are not generated at the start - just the txcache. Could it be that a cache file needed it generated later than when it is needed?
Jean Duteau (Oct 15 2021 at 22:30):
no. as grahame said, it shouldn't matter as the cache files are supposed to be just caching the vocabulary calls. we'll need him to figure out why there is different validation output depending on the presence of this one cache file.
Brian Reinhold (Oct 18 2021 at 15:14):
@Grahame Grieve Any news on the cache issue?
Grahame Grieve (Oct 18 2021 at 18:58):
I spent several hours on it yesterday. Right now, I don't know what I'm looking for
Grahame Grieve (Oct 19 2021 at 01:39):
well, that wasn't much fun. I took me an entire day but I tracked the problem down. fixed next release
Grahame Grieve (Oct 19 2021 at 01:49):
next release will blow everyone's terminology cache away
Sarah Gaunt (Oct 19 2021 at 02:52):
So which is the correct validation? No cache: 89 errors or Cache: 5 errors? (Really hoping it's the latter!)
Grahame Grieve (Oct 19 2021 at 06:34):
no cache
Brian Reinhold (Oct 19 2021 at 13:05):
@Grahame Grieve Thanks for your work. I certainly could not have done it! The down side I am still getting the same 89 errors and they don't make sense. It's like it can't find my code systems and value sets or the codes used by them. Then again my publisher is still 6 days old - which is the 13th of October. Has an update been pushed yet?
Jean Duteau (Oct 19 2021 at 15:45):
it does find the code systems, it's just that some of the codes you are specifying don't exist in the version of the code system that tx.fhir.org knows about. Codes 68193, 528401, and 8452096 aren't valid according to tx.fhir.org. And since you reference those codes in your profiles, that causes a lot of profile validation errors.
Brian Reinhold (Oct 19 2021 at 16:01):
@Jean Duteau Are you able to ascertain that its only those codes? When I get the 5-error result the 5-errors are still due to missing codes. Those codes do not yet exist in the RTMSS for a variety of reasons, even though they have been around for more than 10 years.
Jean Duteau (Oct 19 2021 at 16:02):
well, i'd start with fixing the error with those codes. NOTE: the "5-error result" is just wrong because the cache was being used incorrectly, so I wouldn't even look at it. I'd reach out to @Rob Hausam to see if he can update the 10101 code system to a version that has those codes.
Rob Hausam (Oct 19 2021 at 16:54):
The version of 11073-10101 (MDC) that we have on tx.fhir.org now is 2020-06-18. If there is an updated version available, it can be updated on the server.
Grahame Grieve (Oct 19 2021 at 18:49):
I haven't done the release yet - later this week, probably
Brian Reinhold (Oct 26 2021 at 18:31):
@Grahame Grieve Dropped down to 6 errors with update. Missing MDC codes. These are known. Thanks for the fix.
Rob Hausam (Oct 26 2021 at 19:24):
@Brian Reinhold What's the current release (date) of MDC? Am I recalling correctly that you (or maybe John Rhoads) provided it for me originally?
Brian Reinhold (Oct 26 2021 at 19:29):
@Rob Hausam Unfortunately, I don't know. THe problem with the missing codes (I believe) is on our end as they have not been submitted into the MDC Rosetta. These missing codes are only used in V2 PCD-01 messages and now FHIR. They are not used by 11073 devices. That may explain their lack of inclusion but I am not sure.
These are the set of codes that are used only in V2 and FHIR:
531969 MDC_ID_MODEL_NUMBER
531970 MDC_ID_MODEL_MANUFACTURER
531971 MDC_ID_PROD_SPEC_UNSPECIFIED
531972 MDC_ID_PROD_SPEC_SERIAL
531973 MDC_ID_PROD_SPEC_PART
531974 MDC_ID_PROD_SPEC_HW
531975 MDC_ID_PROD_SPEC_SW
531976 MDC_ID_PROD_SPEC_FW
531977 MDC_ID_PROD_SPEC_PROTOCOL
531978 MDC_ID_PROD_SPEC_GMDN
532352 MDC_REG_CERT_DATA_CONTINUA_VERSION
532353 MDC_REG_CERT_DATA_CONTINUA_CERT_DEV_LIST
532354 MDC_REG_CERT_DATA_CONTINUA_REG_STATUS
532355 MDC_REG_CERT_DATA_CONTINUA_AHD_CERT_LIST
68220 MDC_TIME_SYNC_PROTOCOL
68219 MDC_TIME_CAP_STATE
68224 MDC_TIME_RES_REL_HI_RES
68223 MDC_TIME_RES_REL
68222 MDC_TIME_RES_ABS
68226 MDC_TIME_RES_BO
68221 MDC_TIME_SYNC_ACCURACY
68229 MDC_ATTR_TICK_RES
Last updated: Apr 12 2022 at 19:14 UTC
