Stream: committers
Topic: Reworking the FHIR Build
Grahame Grieve (Oct 06 2020 at 12:23):
I'm reworking the main spec build process so that spreadsheets are no longer the major source material for the specification.
If anyone is interested, here's a copy of the source directory for a few resources showing the new files (as well as the old files) that are going to replace the existing spreadsheets. Comments are welcome.
Grahame Grieve (Oct 06 2020 at 12:23):
Jose Costa Teixeira (Oct 06 2020 at 13:01):
- I presume the spreadsheet files are the old files, and the SVGs are generated (is that right? When Confluence is updated, I presume we'll identify which are the files the author needs to create.
David Pyke (Oct 06 2020 at 13:03):
The spreadsheets are a great way for those who are new to FHIR to understand the structures before trying to create new resources. Can you explain the need to be rid of them?
Jose Costa Teixeira (Oct 06 2020 at 13:04):
- consent-extensions - both a List and an ImplementationGuide?
Jose Costa Teixeira (Oct 06 2020 at 13:05):
@David Pyke I understood (maybe I'm wrong) the intention is not to replace but to complement.
Rob Hausam (Oct 06 2020 at 13:12):
Yes, it would be good to clarify what are the source of truth files that need to be created/maintained.
Jose Costa Teixeira (Oct 06 2020 at 13:36):
- Is it the same to use Bundle (like for SearchParameters) and List (like extensions)? Or searchparams always go into Bundle and the rest has a List?
Jose Costa Teixeira (Oct 06 2020 at 13:37):
3.1 I would like to have the same possibility to use Lists in IGs (e.g. to tell the vocabulary guys to work on their stuff and give us a List)
Lloyd McKenzie (Oct 06 2020 at 14:33):
@David Pyke The objective is to have source in Git that can actually reasonably be merged across branches. Spreadsheets aren't viable for that.
David Pyke (Oct 06 2020 at 14:34):
So, will we have something equally simple for the semi-technical who want to create resources?
Lloyd McKenzie (Oct 06 2020 at 14:36):
My understanding is that you can still use spreadsheets, but you'll be required to commit the accompanying XML file
Jose Costa Teixeira (Oct 06 2020 at 14:37):
If we keep spreadsheet editing (which is a good thing to do) I'd hope that spreadsheets would be a preliminary step to generate all these resources. That is what I recall hearing - you can go from spreadsheet to the proper resources, and iirc the resources could also update the spreadsheet
David Pyke (Oct 06 2020 at 14:43):
So, we would create using the spreadsheet, run it though some utility to create the XML and then check in both?
Lloyd McKenzie (Oct 06 2020 at 14:45):
I believe so
David Pyke (Oct 06 2020 at 14:47):
That works then. Just need to have the utility available.
Melva Peters (Oct 06 2020 at 14:55):
What ever the changes are, I hope there will be clear documentation on how to do the new process and that it has been tested across all operating systems - MacOs and Windows. And I expect that it won't just be thrown out there with an expectation to "go for it".
Chris Moesel (Oct 06 2020 at 15:01):
So, will we have something equally simple for the semi-technical who want to create resources?
I dream of a day when you can author brand new resources in FHIR Shorthand, but that day has not yet arrived. Still some other things to do in FSH before we get there.
Gino Canessa (Oct 06 2020 at 15:21):
How hard would the penalty be to change to/allow markdown instead of xhtml for the docs (e.g., the introduction, notes, etc.)? It's far kinder in source control and less prone to causing build errors (e.g., unclosed paragraph, div nested where it's not allowed, etc.).
Gino Canessa (Oct 06 2020 at 15:22):
David Pyke said:
The spreadsheets are a great way for those who are new to FHIR to understand the structures before trying to create new resources. Can you explain the need to be rid of them?
If the spreadsheets are more functional than the generated documentation of the spec, we should probably put some work into figuring out why.
David Pyke (Oct 06 2020 at 15:23):
Purely the simple interface. Most people are familiar with using Excel, fewer are familiar with XML/JSON editing
Lloyd McKenzie (Oct 06 2020 at 15:26):
The fact XHTML raises errors when markdown doesn't is a value-add in my opinion. You can break things in markdown and nothing will tell you. If you screw up a hyperlink or fail to close a 'bold' tag or something in XHTML, you'll be told. I've never had issues with XHTML and source control.
Gino Canessa (Oct 06 2020 at 15:31):
David: fair enough.
Vassil Peytchev (Oct 06 2020 at 16:12):
Looking at Task, the following files are part of the source (listed at the end). Out of those, how difficult would be to a have an Electron-based app (or similar off-line available tool) that would provide editing capabilities and guardrails the Page fragments, Resource Definition, Search Parameters, list of examples and extensions? Or is that not worth pursuing?
- Page fragments
* task-introduction.xml
* task-notes.xml - Resource definition
* structuredefinition-Task.xml - Search Parameters
* bundle-Task-search-params.xml - List of examples
* list-Task-examples.xml - List of Extensions (do these need to be kept in sync?)
* list-task-extensions.xml
* implementationguide-task-extensions.xml - Code systems defined as part of the resource (?)
* codesystem-*.xml - Value sets defined as part of the resource (?)
* valueset-tack-*.xml - Examples
* task-example*.xml
Gino Canessa (Oct 06 2020 at 16:20):
Lloyd, difference of preference it is. I've had issues in source control around exactly things I mentioned (e.g., merging tags missing the close or causing incorrect nesting, etc.). I will also admit that it's annoying, but not difficult to fix.
Overall, I feel that markdown is more user-friendly (e.g., previews in editors, etc.). I am also thinking about things like embedding mermaid diagrams, which markdown processors are already doing. You can get the same output using XHTML, but it's more steps.
Lloyd McKenzie (Oct 06 2020 at 17:24):
I think we're moving towards PlantUML rather than mermaid? (Would really rather not introduce yet more diagramming tools...)
John Moehrke (Oct 06 2020 at 17:27):
hmm, mermaid seems very plantUMl like... but not exactly... the markdown community is out of control..
Gino Canessa (Oct 06 2020 at 17:38):
I'm less concerned with the tool package and more asking about the ability to inline data (separate artifacts tend towards getting out of sync)
David Pyke (Oct 06 2020 at 17:40):
Anything that inlines the files needed (PlantUML is fine) is a great idea. We just need to give guidance on the use of the various flavours of markdown (and other markups) so that we get a reasonably consistent feel to IGs.
Grahame Grieve (Oct 06 2020 at 19:00):
I very much prefer mermaid over plantUML for several reasons
Grahame Grieve (Oct 06 2020 at 19:17):
more comments on this thread:
- @Vassil Peytchev not so difficult technically but quite infeasible, in my view. but if you want to take it on and commit to it... knock yourself out
- @Gino Canessa I too strongly prefer markdown but it's not quite as precise as HTML. I focused on markdown for guides.
- @David Pyke the resources will be the master format. the build will generate a spreadsheet you can edit, though the format will be a little different than what we have now. But the spreadsheet will be a purely local artifact
- @Chris Moesel I'm totally unsold on the idea of a DSL somewhat like FSH for this particular purpose; the vast bulk of the content is documentation
- @Melva Peters of course you're right that documentation etc will be needed. But the timeline and available resources limits my ability to do a perfectly prepared roll out with extensive testing. Of course, any testers will be welcome
Chris Moesel (Oct 06 2020 at 19:37):
@Chris Moesel I'm totally unsold on the idea of a DSL somewhat like FSH for this particular purpose; the vast bulk of the content is documentation
I agree that a DSL isn't ideal for tons of documentation. I'd like to find a way for something like FSH to be able to define the structural components, but defer to better approaches for the documentation components. FSH is not up to the task today, but FSH only just had its first ballot; there's lots of room for growth. And I expect we'll learn some things as we attempt the design and implementation of logical models.
Josh Mandel (Oct 06 2020 at 21:11):
Looking at (for example) structuredefinition-Task.xml in Grahame's zip file at the top of this thread: how robustly do we expect changes to be mergeable in these sorts of XML files? Better than spreadsheets certainly, but do we have a practical sense of how much better?
Grahame Grieve (Oct 06 2020 at 21:12):
you want a number?
Grahame Grieve (Oct 06 2020 at 21:12):
my answer is: good enough to expect editors to be able to merge
Josh Mandel (Oct 06 2020 at 21:12):
No, but like... has anyone tried it?
Josh Mandel (Oct 06 2020 at 21:13):
And how did it feel? Your answer suggests 'yes' and 'workable' which is all I was asking.
Josh Mandel (Oct 06 2020 at 21:13):
(That wasn't obvious to me!)
Grahame Grieve (Oct 06 2020 at 21:19):
well, I think it is. I think a better way to put this is: the problem with the spreadsheets is that you have to deal with merging the format as well as the content. The resources will represent - mostly - simply merging the content. The non-content boilerplate stuff will mostly get out of the way.
If I was defining a format purely to support merging the content, I might do things a little differently but not that much
Gino Canessa (Oct 06 2020 at 22:11):
Yeah, there's a tradeoff there - markdown would be less precise, but easier to work with (e.g., live previews (even if not in the correct template), inline diagrams, etc.). Wanted to have a discussion around it at least.
Grahame Grieve (Oct 07 2020 at 00:06):
you can get live previews from html too.
Gino Canessa (Oct 07 2020 at 14:28):
Yes, but not of xml files without messing up my settings for xml. =/
Last updated: Apr 12 2022 at 19:14 UTC
