Stream: IG creation
Topic: source markdown copied to output
John Moehrke (Aug 02 2021 at 19:11):
I am noticing that my source markdown files (./input/pagecontent) are ending up in the IG build output folder (./output). This likely is not doing any serious harm. The staging system for the IHE publication web site is a github repo, and thus github is trying to convert these markdown files and complaining about missing links and such. These complaints are not too worrisome.
I am more worried that there is code copying these markdown files with no good reason, and this code might be either not intended or potentially going to cause problems in the future if it is unknown behavior.
I do know there are some folders that we are expecting the build to copy the contents, (e.g. ./input/images). There are other folders that are NOT copied (e.g. ./input/fsh). So someone put in the code to copy pagecontent, and we all might not know this is being done.
Is this a problem I should create an issue about? (Presume it would be an IG builder issue, not template.)
Jean Duteau (Aug 02 2021 at 19:14):
i just checked two of my IGs that I built recently and the *.md files aren't in the output folder.
John Moehrke (Aug 02 2021 at 19:39):
This might be something the IHE template is doing? @Jose Costa Teixeira
Jose Costa Teixeira (Aug 02 2021 at 20:07):
There goes my evil plot to expose markdown files.
#MarkdownLeaks
Jose Costa Teixeira (Aug 02 2021 at 20:08):
I have no idea. I think there was a similar issue before, could be the IHE template has some leftovers, but I doubt. I'll check anyway
John Moehrke (Aug 02 2021 at 20:14):
seems to copy everything from input\pagecontent
Eric Haas (Aug 02 2021 at 20:16):
This is a known issue and you mentioned it before. The publisher pollutes the output with markdown files. This screws up publishing on Github pages for example. I wrote a bash script to scrub out the markdown since there was no interest when I brought this issue it up here previously, perhaps you will have more luck.
John Moehrke (Aug 02 2021 at 20:16):
the generic template does not seem to do it.
John Moehrke (Aug 02 2021 at 20:16):
I don't have any HL7 IGs.. do they do it?
Eric Haas (Aug 02 2021 at 20:17):
it does for me .... let me confirm
John Moehrke (Aug 02 2021 at 20:24):
HotBeverage builds clean. It is using fhir.base.template
John Moehrke (Aug 02 2021 at 20:28):
could it be an IG parameter? Like path-resources or path-pages? -- they are not defined as having a copying behavior.
Eric Haas (Aug 02 2021 at 20:32):
yes it still passes the markdown pages into the output ( 'docs' folder in this case):
image.png
John Moehrke (Aug 02 2021 at 20:33):
so, I have a different IG that is using base template and it does copy... so I don't think it is template
John Moehrke (Aug 02 2021 at 20:40):
I think it is the IG publisher parameter path-pages. Given that my markdown are in pagecontent, I don't need to specify this, but I do because I saw it recommended somewhere.
I am also using sushi-config.yaml and no IG xml...
John Moehrke (Aug 02 2021 at 20:46):
@Eric Haas do you use path-pages?
Eric Haas (Aug 03 2021 at 01:10):
yes I do:
path-pages:
- input/pagecontent
- fsh-generated/includes
Lloyd McKenzie (Aug 03 2021 at 01:55):
I just ran davinci-crd which uses the current template and current publisher. It doesn't propagate any of the .md files to the output, so it's not an issue with the base template or the publisher
John Moehrke (Aug 03 2021 at 12:55):
@Lloyd McKenzie I think you need to use the path-pages parameter . Can you confirm?
Lloyd McKenzie (Aug 03 2021 at 13:28):
I can confirm that you don't want path-pages. path-pages will be copied directly to the output folder. The processing of the pagecontent folder is handled by the config.json in the template - specifically this part:
{
"folder": "input/pagecontent",
"relativePath": "_includes",
"transform": "template/scripts/processPages.xslt"
},
John Moehrke (Aug 03 2021 at 14:33):
so it would seem that when using sushi-config.yaml (as the driver), we should NOT include the path-pages that Eric and I have used.
Lloyd McKenzie (Aug 03 2021 at 14:42):
Right.
John Moehrke (Aug 03 2021 at 14:49):
I just removed those lines from three projects with no apparent negative. I am not sure where I got these recommendations, so not clear how to keep future projects from using these in a way not intended. I do think that the hl7 confluence page does not explain this fully.
path-pages:
- input/pagecontent
- fsh-generated/includes
Eric Haas (Aug 03 2021 at 16:47):
sureley, explicitvs implicit statement of where the pages live should not lead to different behavior, additionally if you pages live outside these folders how will the publisher know to grab them? I think we need to:
- either say you can have content only here and here or...
- fix the bug
Lloyd McKenzie (Aug 03 2021 at 17:26):
path-pages is not used to indicate where pages are. I get that that's confusing, but that's how the templates work. If we change it now, it will break all downstream templates.
John Moehrke (Aug 03 2021 at 17:27):
I don't think the template is the problem. I think the definition in the confluence page seems to imply that it would be needed to specify
Lloyd McKenzie (Aug 03 2021 at 17:27):
The only way to specify where to look for pages is in the xslt that produces the config.json file.
Eric Haas (Aug 03 2021 at 17:43):
Lloyd McKenzie said:
The only way to specify where to look for pages is in the xslt that produces the config.json file.
this is published on the parameters page in confluence:
Eric Haas (Aug 03 2021 at 17:45):
Having secret rules by the xlst that do the opposite of what is published is not helpful. The templates are prohibiting an author who want to use a different folder for page content and publish on github pages?
Lloyd McKenzie (Aug 03 2021 at 18:02):
The only way to use a different folder is by it being specified in the config.json file and that's generated by an XSLT. We can provide a mechanism to add additional source folders if the designers of non-HL7 templates need that as a capability. Agree the documentation needs to be fixed. path-pages indicates folders whose contents are to be copied directly to output.
Jose Costa Teixeira (Aug 03 2021 at 18:11):
How about
"path-pages | yes | path | A relative path that contains files that will be directly copied to the output folder in the IG. Default = "\pages". Note that this is not the path of the source for the normal pages (in markdown or xhtml format) - those are handled by the template"
Jose Costa Teixeira (Aug 03 2021 at 18:12):
(nevermind the formatting, markdown is not my friend today)
Lloyd McKenzie (Aug 03 2021 at 20:21):
page-paths is not (and will not) be the element to identify where pages are located unless we're happy to break the existing derived templates
Lloyd McKenzie (Aug 03 2021 at 20:22):
We can add something new, but we can't repurpose that parameter which is used for other purposes.
John Moehrke (Aug 03 2021 at 20:58):
I don't think I want to repurpose. I simply want to understand the intent. I had no idea that the intent of the parameter was to identify other folders that should also be "copied" to the output. If this is indeed the meaning, then the confluence page simply needs to be updated to state that. I am happy to update the confluence page, I just want to update it correctly.
Last updated: Apr 12 2022 at 19:14 UTC
