Stream: fmg
Topic: QA
David Hay (Feb 24 2018 at 19:05):
Hi Guys - been thinking about the upcoming ballot QA - in particular QA of the ValueSets. Rather than creating a document of the contents of selected ValueSets, I've written a small app to display VS contents and collect comments - http://conman.fhir.org/qa.html
You select a Resource, then get a list of all ValueSets in that resource. Select a ValueSet and then you can expand the contents.
Finally you can enter a comment about that VS which is stored in a mongo db.
I'm thinking that each resource gets assigned a reviewer/s, and part of that review is to look at the ValueSets (they also need to look at the resource contents and associated material as well of course).
What do you think? If it is worth pursuing, then I'll add a page to allow a committee to view all the comments for 'their' resources and mark them as having been reviewed.
Lloyd McKenzie (Feb 24 2018 at 19:07):
One would almost think that you liked writing tools... :)
Lloyd McKenzie (Feb 24 2018 at 19:08):
So next weekend we should expect tooling to support reviewing everything else?...
David Hay (Feb 24 2018 at 19:15):
who me? :)
David Hay (Feb 24 2018 at 19:20):
Slightly more seriously, I was thinking about how we could do that. For reviewing the contents of a resource, I could imagine something similar - tree to the left, details of selected node in the middle, comments to the right. While it would be feasible to display the textual stuff as well, it's hard to beat change tracking in a document...
Lloyd McKenzie (Feb 24 2018 at 21:14):
The main reason for use of MS-Word was to take advantage of its spelling and grammar checking capabilities. It's hard to replicate that.
Grahame Grieve (Feb 24 2018 at 21:56):
how useful was that? did we we get much grammar related changes? Spelling is lots easier
Lloyd McKenzie (Feb 24 2018 at 21:58):
Yes
Lloyd McKenzie (Feb 24 2018 at 21:58):
Quite a bit of grammar stuff (tense agreement, missing commas and other punctuation, etc.)
David Hay (Feb 28 2018 at 19:12):
Here's the most recent set of instructions: https://docs.google.com/document/d/1-kMS-S0dgxa5pyK_87g_DiZonSS7FRuTvqwPUmDH6FQ/edit
I've moved away from trying to develop tooling - for what we're doing the Word Document is hard to beat. However, I've re-organized the files a bit so that it's simpler (I hope) both for reviewers and for those applying the changes.
Grahame Grieve (Jul 19 2018 at 22:03):
When we do QA for the this ballot cycle, instead of asking people to look at spelling/grammar, we should ask people to review:
- unresolved todo items
- link correctness
- look for inconsistencies in definitions
- look for extensions that have been replaced by core elements but are still defined
- notes to implementers - should link to a wiki landing page that should say something useful
- check that pages labelled informative do not include 'SHALL' or 'SHOULD'
Grahame Grieve (Jul 19 2018 at 22:14):
- check page status (FMG members)
Lloyd McKenzie (Jul 19 2018 at 22:17):
Unresolved todos are ok for some FMM levels. "Inconsistencies in definitions" is a hard thing - inconsistent with what? how?
Lloyd McKenzie (Jul 19 2018 at 22:18):
Extensions that have been replaced is a task we can assign to a single person
Lloyd McKenzie (Jul 19 2018 at 22:18):
Pages labeled informative - can't we do that using tooling?
Lloyd McKenzie (Jul 19 2018 at 22:18):
Not sure what's meant by a "wiki landing page"
Grahame Grieve (Jul 19 2018 at 22:19):
I was particularly thinking about normative pages
Grahame Grieve (Jul 19 2018 at 22:19):
inconsistencies is hard, but we can at least check
Grahame Grieve (Jul 19 2018 at 22:19):
I'm not sure about the tooling - I just found something missed because it's included late
Lloyd McKenzie (Jul 19 2018 at 22:21):
I think we might assign specific QA tasks to us that provide deeper checks on definitions and other things that might be unreasonable to expect of the regular QA folks.
Grahame Grieve (Jul 19 2018 at 22:22):
well, with regard to the normative pages, I do not want QA people proposing grammar fixes. That's not on anymore. Because they're often times wrong, and the stakes around errors are higher now
John Moehrke (Jul 19 2018 at 22:23):
we likely do need to classify the pool of volnteers. If someone wold be best to point at spelling, they should get that task.
Lloyd McKenzie (Jul 19 2018 at 22:27):
I do want grammar fixes. We need to carefully vet what we apply - but we will have made lots of edits and there will be places where someone will have the commas wrong or "it's" where it should be "its" and it's still good to fix those.
Grahame Grieve (Jul 19 2018 at 22:34):
you might want them but have you been making the change proposed? It's hard work because the changes fit into 4 categories:
- fixing something really broken
- trying to import some style consistency that differs from reviewer to reviewer
- totally misunderstanding what is being said, so grammar does need fixing
- totally misunderstanding what is said with no excuse at all
about 25% each....
Grahame Grieve (Jul 19 2018 at 22:35):
also
- checking that all substantiative changes between this ballot and the last for normative have been listed in the ballot intro
John Moehrke (Jul 20 2018 at 01:08):
we should be able to look at SVN to see what narrative changed... right? Thus we would no need to QA for spelling those pages that didn't change (or put them lower priority)
Grahame Grieve (Jul 20 2018 at 01:12):
better to look at the diffs - svn is too sparse
Last updated: Apr 12 2022 at 19:14 UTC
