Stream: fmg
Topic: STU3 finalization
Grahame Grieve (Mar 17 2017 at 04:16):
ok, So just checking. The following things in STU3 are draft, not STU:
- GraphDefinition
- StructureMap + the mapping language content
- ChargeItem
- TestReport
- RequestGroup
- ResearchStudy
- ResearchSubject
- MessageDefinition
- AdverseEvent
Grahame Grieve (Mar 17 2017 at 04:16):
does anyone have any advance on that list @Lloyd McKenzie
Grahame Grieve (Mar 17 2017 at 04:18):
Also, how do we assign FMM levels to pages?
Grahame Grieve (Mar 17 2017 at 04:19):
here's my proposal: I'm going to do it by horse sense, and all of you can all criticise them
Grahame Grieve (Mar 17 2017 at 04:26):
note that there's plenty of pages without any fmm level - nothing to implement. And some are already fixed by committee. Namely, XML and JSON. I don't know of others that committees have ruled on
Lloyd McKenzie (Mar 17 2017 at 05:02):
Not sure what you mean by "advance on that list"
Lloyd McKenzie (Mar 17 2017 at 05:03):
Workflow would be FMM2 I think - tested at connectathon, but probably not meeting our quality criteria in terms of fully complete
Grahame Grieve (Mar 17 2017 at 05:04):
well, is the list correct? have I missed anything?
Grahame Grieve (Mar 17 2017 at 05:06):
according to FHIR.ini: Linkage, EnrollmentRequest, EnrollmentResponse, DeviceUseStatement too
Grahame Grieve (Mar 17 2017 at 06:29):
so, here's the rules I worked to:
- pages must be have a FMM level, or be informative (or normative, but we have none of that yet)
- mappings, examples are informative
- advice pages are informative unles they include the words 'should' or 'shall' as applied to implementers.
- content cannot be less than 4 if content that depends on it is >= 4
- basic infrastructure is now 5 (references, narrative, conformance rules etc)
- left committee explicit values in place except for FHIR-I values where I bumped a few of them up
John Moehrke (Mar 17 2017 at 13:07):
so Task was FMM0 in the STU3 ballot.. so Task is still draft, right?
John Moehrke (Mar 17 2017 at 13:08):
If I am correct and the Sep 2016 is the STU3 ballot http://hl7.org/fhir/2016Sep/resourcelist.html
Grahame Grieve (Mar 17 2017 at 13:08):
not neccesarily. if it wasn't balloted, or it was marked as draft at ballot, then it must be draft when we publish. If it was balloted, then it is STU3, and can be whatever FMM is appropriate
John Moehrke (Mar 17 2017 at 13:09):
that is not what the FMG meeting discussion said. It was if the FMM was ZERO or it didn't exist; then it must not go higher than Draft in STU3 release
Grahame Grieve (Mar 17 2017 at 13:09):
if the FMM is 0 now.
Grahame Grieve (Mar 17 2017 at 13:10):
not if it was 0 then
John Moehrke (Mar 17 2017 at 13:10):
Lloyd argued strongly that FMM ZERO in STU3 ballot; that was an indication of draft
Grahame Grieve (Mar 17 2017 at 13:11):
that wasn't at all what I understood him to mean. I believed that he was talking about FMM0 now. And I would have strongly objected to the idea that FMM0 then meant draft now
John Moehrke (Mar 17 2017 at 13:11):
I tried to argue in favor of ANY existence in the STU3 ballot was sufficient; but I think I lost that argument
Grahame Grieve (Mar 17 2017 at 13:11):
it has to have progressed past FMM0 since then
John Moehrke (Mar 17 2017 at 13:12):
yes, I argued that point
Grahame Grieve (Mar 17 2017 at 13:12):
indeed. and I think it's perfectly reasonable that if a resource hasn't progressed to at least level 1 since ballot, we call it draft still. It's a pretty low bar to get over indeed
John Moehrke (Mar 17 2017 at 13:13):
It needed to exist in the STU3 ballot.
John Moehrke (Mar 17 2017 at 13:13):
right?
Grahame Grieve (Mar 17 2017 at 13:13):
yes
Grahame Grieve (Mar 17 2017 at 13:14):
the list above is my list of things that didn't exist, plus those that were marked as draft
John Moehrke (Mar 17 2017 at 13:14):
then that is what I was arguing. Especially since our FMM evaluation criteria said nothing about this ballot-worthyness point
Grahame Grieve (Mar 17 2017 at 13:14):
good
John Moehrke (Mar 17 2017 at 13:14):
How was something 'marked draft'? Do you mean 'currently marked draft'?
John Moehrke (Mar 17 2017 at 13:15):
because the argument said that the only way something "was marked draft" in the STU3 ballot, was with FMM 0 indicator
Grahame Grieve (Mar 17 2017 at 13:15):
there were a few resources where they were marked as 'draft not for ballot' in the introduction
John Moehrke (Mar 17 2017 at 13:15):
that got us into the conclusion that anything marked FMM 0 in the STU3 ballot; can't go higher than 0 in the STU3 final pub
Grahame Grieve (Mar 17 2017 at 13:16):
that was with text in the introduction, and they also had fmm = 0 but it was the text that was important
John Moehrke (Mar 17 2017 at 13:16):
so, text seems logical to hold it back... too. This point was not brought up in FMG discussion, but surely it would also be recognized.
Grahame Grieve (Mar 17 2017 at 13:17):
I just committed a fairly big overhaul, marking most pages with both a status and a maturity.
Grahame Grieve (Mar 17 2017 at 13:17):
I'm going to sleep now.
Grahame Grieve (Mar 17 2017 at 13:18):
the actual levels are not correct except for the manually edited pages - those are for commenting on. the other levels, Lloyd is updating them.
Grahame Grieve (Mar 17 2017 at 20:30):
I'm going to assign Fmm=1 to all profiles and extensions
Grahame Grieve (Mar 17 2017 at 20:32):
by default
Lloyd McKenzie (Mar 17 2017 at 21:42):
If something was FMM 0 in the ballot, we agreed it can be up to FMM2 in the publication, but must have a ballot status of "Draft". Anything that was draft in the ballot MUST be draft in the published specification. (Because the status at the ballot is essentially "draft candidate" or "stu candidate" and some balloters choose what they're going to review on the basis of whether it's considered draft or not.
Lloyd McKenzie (Mar 17 2017 at 21:43):
CG has their profiles listed on the QA spreadsheet with proper FMM levels. Agree the remainder should be 1 for now.
Grahame Grieve (Mar 17 2017 at 22:35):
I don't understand why something that was FMM0 but balloted has to be draft
Grahame Grieve (Mar 17 2017 at 22:37):
in fact, looking at the list of things that were FMM0 in the ballot, I passionately disagree that things that were FMM0 in the ballot must be draft
Grahame Grieve (Mar 17 2017 at 22:37):
I feel *extremely* strongly about this
Lloyd McKenzie (Mar 17 2017 at 23:41):
If something is labeled as draft when it goes to ballot, then that means it's not candidate for STU. I *know* there are things that shouldn't have been labeled as draft in the ballot. But if the ballot doesn't say "this is a candidate for STU", then we can't arbitrarily elevate them to STU after the fact. (Just like we couldn't elevate something from STU to Normative if it was labeled as STU candidate in the R4 ballot.) It sucks. But I from a governance perspective, I can't see how we can take something that was clearly marked as "this is draft and not ready for STU consideration" in the ballot and then say "the balloters want this to be STU"
Lloyd McKenzie (Mar 17 2017 at 23:42):
The ballot has to declare whether the content is "in scope" for the ballot or not. Marking something as FMM 0 is FHIR's way of saying "this isn't ready yet". We tell work groups that they don't need to formally vote on changes to FMM 0 artifacts
Grahame Grieve (Mar 18 2017 at 06:58):
we didn't clearly label them as draft. And no one treated them as draft. I agree that if we explicitly said that 'they're draft', then they're draft. But we didn't say that, they were just labelled as 'fmm 0'. I treated that as transient and I don't believe that we had any advice to balloters that 'fmm 0' = not for ballot, and we got plenty of ballot comment on things that were FMM 0
Grahame Grieve (Mar 18 2017 at 06:58):
nor am I prepared to release STU 3 with Questionnaire and CodeSystem marked as 'draft', which would mean that they could not be normative next time. and if code system is not normative, then what can be?
Grahame Grieve (Mar 18 2017 at 06:59):
if you can show that we told balloters that fmm0 = don't comment because it's draft, I might have to reconciser, but we certainly didn't tell editors that
Grahame Grieve (Mar 18 2017 at 06:59):
and I will hold up the ballot if I have to get this resolved. Because I am not releasing a publication with CodeSystem as a draft
Brian Postlethwaite (Mar 18 2017 at 08:22):
CodeSystem has been in the spec in DSTU2 also, just as a sub section of ValueSet is how I would argue this.
Grahame Grieve (Mar 18 2017 at 08:39):
indeed, that is part of the rationale for it to be level 5. But procedurally, it was at FMM 0 for the ballot
John Moehrke (Mar 18 2017 at 15:07):
Are we going to change our broad decision? I think the broad decision should stand. I also think that specific resources can be discussed, like CodeSystem. So, what are the compelling reasons why CodeSystem should be considered having been significantly reviewed in ballot? Post ballot can't really be considered, for which I agree CodeSystem has seen significant post ballot focus.
Lloyd McKenzie (Mar 18 2017 at 18:09):
Questionnaire was STU in DSTU2. CareTeam, PractitionerRole and CodeSystem were part of existing resources that were STU in DSTU2. So there's some foundation there for treating those resources differently.
Lloyd McKenzie (Mar 18 2017 at 18:11):
In STU2 ballot, we clearly marked draft resources as "marked as draft". We didn't do that in STU 3 ballot, even though some resources clearly were considered draft. We did however change the FMM descriptions to clearly say that FMM 0 was equivalent to "Draft". Those two things combined would have led at least some balloters to treat FMM 0 artifacts in the same way they would have treated "marked as draft" resources in the STU 2 ballot. Thus the issue.
Lloyd McKenzie (Mar 18 2017 at 20:21):
Turns out I was wrong. (Because I'd looked in the one place I was right :(). We *did* explicitly mark "Draft" resources in the STU 3 ballot as draft. E.g. http://hl7.org/fhir/2016Sep/enrollmentrequest.html
Lloyd McKenzie (Mar 18 2017 at 20:23):
We just happened to not do so for ClinicalImpression for some reason. So most of those things that should have been identified as draft explicitly were. As such, I think we're safe elevating FMM 0 resources that weren't marked as draft in the ballot and if they weren't marked as draft in the ballot, they don't have to be marked as draft in the published spec.
Grahame Grieve (Mar 20 2017 at 19:00):
Note: SDC and DAF-research will not be published today along side the formal publication of FHIR and the US-Core specification because TSC has not approved their publication
Grahame Grieve (Mar 21 2017 at 04:20):
fyi: http://wiki.hl7.org/index.php?title=Posting_the_FHIR_specification_on_hl7.org
Paul Knapp (Mar 21 2017 at 12:02):
I think on August 13 2015 as part of the Countdown to DSTU Lloyd sent out an email stating that if the content was not to be balloted as DSTU it could be included as a draft for comment with FMM set to 0.
And on August 12 "Note that the spreadsheet doesn't account for warning messages, so right now there are a lot more FMM0 resources than is indicated here. (For a summary of warning messages in the build, you can look here: http://hl7-fhir.github.io/qa.html) Any resource with warnings is constrained to FMM0. FMM0 resources are expected to be marked as Draft in the next DSTU release."
Paul Knapp (Mar 21 2017 at 12:17):
FMM didn't mark resources as draft if they were to be considered as draft, they were marked as FMM 0 - I marked EnrollmentRequest and EnrollmentResponse as stubs and said they were draft in the hope that people would not send in copious ballot comments on the obvious missing content as they had in the DSTU2 ballot.
Paul Knapp (Mar 21 2017 at 12:19):
If the managing committee hadn't agreed that the resource in the ballot was fit for trial use then I don't see how the ballotters would consider them as anything but draft.
Paul Knapp (Mar 21 2017 at 12:42):
In some cases the editors marked their draft resources as draft, and each in their own style, for example: EnrolmentRequest, Linkage, DeviceUseRequest, but in other cases such as Account and Consent they were not marked but were clearly not STU ready.
Paul Knapp (Mar 21 2017 at 12:45):
I'll raise at SGB tomorrow.
Grahame Grieve (Mar 21 2017 at 13:27):
what will you raise?
Paul Knapp (Mar 21 2017 at 15:23):
The issue above. The posted FMM Maturity Model states: "FMM1 and higher will be published as STU level rather than draft".
Grahame Grieve (Mar 21 2017 at 15:25):
so there is no governance issue here. The governance situation is clear:
- it should be clear what is balloted for draft, and what is balloted for STU
- if something was balloted as draft, then it can't be published as STU
Grahame Grieve (Mar 21 2017 at 15:26):
there is a management issue here in that we were not clear what was draft or not. we had two different methods, neither clearly documented
Paul Knapp (Mar 21 2017 at 15:27):
Was trying to find you some latitude.
And I expect the posted FMM rule trump (no pun intended) the ballot content.
Grahame Grieve (Mar 21 2017 at 15:28):
well, if you want to report to SGB, make sure the report is clear: the governance principles are clear, and FMG is aware that the management of this was not exemplary, and it will be next time
Grahame Grieve (Mar 21 2017 at 15:28):
the product manager is aware of that too.
Paul Knapp (Mar 21 2017 at 15:31):
We may need to have another ballot, perhaps just for select resources, before next May - FMM5 requires publication 2 formal ballots (DSTU1, DSTU2, DSTU3) at FMM1+, some of these one you want to go normative may not currently qualify next May.
Grahame Grieve (Mar 21 2017 at 15:34):
we think they do because they were broken out of resources that were already published, and that should count
Grahame Grieve (Mar 21 2017 at 15:35):
something for FMG to consider
Last updated: Apr 12 2022 at 19:14 UTC
