Stream: ig publishing requirements
Topic: Highlighting unchanged stuff in differential
Eric Haas (Oct 19 2021 at 00:31):
I have a tracker that complains that my differential contains stuff that is unchanged from the base. These are mostly min max and type that is repeated. I am reluctant to go through dozens of profiles to scrub them out partly since I think is a waste of my time and partly because I think its useful for me when I edit the diff to see these elements. I had assumed that the publisher dims out the element if they are unchanged from the base. In other words if min = 0 and max ='1' in both the profile diff and the base the tree view min max would be dimmed out. Has this been discussed before? Anyway I think it should be dimmed -mostly so I don't get any more trackers :-)
@Lloyd McKenzie @Grahame Grieve?
Lloyd McKenzie (Oct 19 2021 at 01:38):
Right now, the validator highlights whatever is in the differential. It doesn't check against the base to see if it's different. (Making things even more fun, if authoring using spreadsheets, sometimes declaring the type is mandatory even if unchanged to allow other elements (e.g. pattern) to be declared.
Eric Haas (Oct 19 2021 at 02:52):
I assume you meant the publisher's narrative generator and not the validator.... I don't expect the publisher to change the contents for the differential.
Lloyd McKenzie (Oct 19 2021 at 04:20):
Yeah, we probably can't change the differential - because sometimes you want to force something to be a particular way, even if the parent changes.
Grahame Grieve (Oct 19 2021 at 18:48):
what's the actual complaint?
Lloyd McKenzie (Oct 19 2021 at 18:51):
It's highlighting things as 'changed' because they're in the differential - even though they haven't actually changed.
Grahame Grieve (Oct 19 2021 at 18:54):
if you mean, it's not making them less opaque, then that's what it does. It's a statement of what's in the differential. That's not a statement of what hasn't changed
Lloyd McKenzie (Oct 19 2021 at 18:56):
What people want to see is "what's changed". What's in the differential doesn't typically matter if it hasn't changed.
Elliot Silver (Oct 19 2021 at 21:03):
Lloyd McKenzie said:
What people want to see is "what's changed". What's in the differential doesn't typically matter if it hasn't changed.
From my point of view as a consumer, the differential is what's changed, and if there's something in there that hasn't changed, well, it shouldn't be in there. I understand if that's not actually works, but as a reader of an IG, that's my expectation.
Eric Haas (Oct 19 2021 at 21:12):
the issue is that profile authors are lazy and not always careful about including only what is changed.
Elliot Silver (Oct 19 2021 at 21:18):
Agree. And that's why I'd like to see the publisher strip out content that isn't changed. But again, I recognize that's not consistent with historical use.
Eric Haas (Oct 19 2021 at 21:19):
the other option is to make a rule that the diffs need to be pure and validate that they are
Grahame Grieve (Oct 19 2021 at 21:20):
you know, if I'm going to do that work, I might as well just make the things that haven't changed less opaque. It's the same underlying work
Grahame Grieve (Oct 19 2021 at 21:21):
right now, I know what isn't in the differential, because I copied it across. In order to change the rendering, I'd have to add logic to decide whether the content that is present is different. Some times, for things like strings, that'll be easy. for other things... it's tricky.
Grahame Grieve (Oct 19 2021 at 21:22):
does order of choice types matter? Does forgetting profiles in types in the differential matter? Does missing invariants matter?
Eric Haas (Oct 19 2021 at 21:23):
I agree it gets complicated quickly. What is currently added as greyed in content? min, max , type
Grahame Grieve (Oct 19 2021 at 21:23):
the renderer, because the content isn't in the differential, so it falls through to the base
Eric Haas (Oct 19 2021 at 21:35):
looking at US Core diffs the min max and type stick out. Is it worth doing if only captures the 90%?
Grahame Grieve (Oct 19 2021 at 21:45):
is what worth doing?
Eric Haas (Oct 19 2021 at 22:02):
dimming the unchanged propertied - if most (the easy stuff) but not 100% of the unchanged items are dimmed
Grahame Grieve (Oct 19 2021 at 22:09):
well, I figure that once I start, people will bitch about the corner cases until it's 100% right. Well, at least Lloyd, if not normal peoples
Chris Moesel (Oct 19 2021 at 22:39):
If people use SUSHI, it automatically includes only the changed things in the differential by default. ;-)
Eric Haas (Oct 19 2021 at 22:46):
@Chris Moesel I figured that much out from the tracker :-). But I use sushi and it doesn't do that to my json files. So I assume it only works on the fsh ones.
Eric Haas (Oct 19 2021 at 22:53):
if it did that on json and yaml files would save me a little time writing my own script to do the same.
Chris Moesel (Oct 20 2021 at 01:56):
Yeah. I guess I should have been more specific. SUSHI includes only the things that truly changed when creating differentials from FSH files only. Sorry about that!
Chris Moesel (Oct 20 2021 at 01:58):
Eric Haas said:
Chris Moesel I figured that much out from the tracker :-).
Oh. Is this my tracker? Oops. Thanks for chasing it down!
Lloyd McKenzie (Oct 20 2021 at 04:05):
There are two issues here:
- Is a differential only things that have changed?
-
Do we need to highlight what's in the differential or what's changed?
-
Differentials contain content for 4 reasons:
a) it's changed (obvious & most common)
b) it hasn't changed - but the profile author wants to ensure a constraint is enforced even if the underlying resource/profile changes. E.g. current max is 1, but I want to enforce a max of 1 even if the parent changes
c) some authoring tools (e.g. spreadsheet authoring) mandate certain elements are present even if unchanged
d) some authors include content in their differential as reminders or because they forget to remove stuff that's unchanged from the base, or the base has evolved to match the differential and differential hasn't changed to reflect that -
Of those reasons, we don't really want to show stuff that's in category (c) or (d). Ideally, we'd like these things to not appear in the differential at all. Auto-stripping them would sort of be nice. However, we can't really distinguish them from (b)
So, I think the first question we need to ask is - is (b) a real thing? It's a theoretical possibility. But I'm not aware of any situation where anyone's actually done this. If it's not a practical reality, then "normalizing" the differential seems like a good thing to do.
On Grahame's other point, should we normalize other information - like ensuring that types are listed in the same order? (Obviously only works where types are unchanged - if you start introducing profiles and have no idea what the underlying types are, retaining order isn't really feasible.)
Eric Haas (Oct 20 2021 at 04:56):
One other display issue when experimenting with this. I noticed that the MS flags are not displayed if inherited in the diff. I think that is important to expose just like cardinality, and type, even if inherited. Which is why I add em in if not showing up in the diff.
Grahame Grieve (Oct 20 2021 at 10:13):
well, I think that's a bug, that last bit
Grahame Grieve (Oct 20 2021 at 10:13):
you could make a github issue for that
Eric Haas (Oct 23 2021 at 01:49):
For the other bits discussed above (dimming out the repeated stuff), do we have enough discussion here to move it to github issue to hash it out more? next steps? ( I am basing a tracker on this outcome so I need to push for a resolution and timeframe)
Lloyd McKenzie (Oct 23 2021 at 02:08):
I don't think we have. I think the first question is whether constraints that aren't changes should be removed from the differential.
Eric Haas (Oct 23 2021 at 02:15):
well that could be part of the WG discussion of if and how to implement when review possible options :
1) the ig-pub changes the diff to be unique then the renders
2) the ig-pub checks if diff uniquie when rendering
I don't have an opinion except that the source should be unchanged.
Eric Haas (Oct 23 2021 at 02:16):
...but I do think we need an overide, because I have a few diff that I want the diff to be rendered with redundant element s for display purposes.
Eric Haas (Oct 23 2021 at 02:20):
or maybe is easier to just leave it and exhort editors to not repeat in the diff...
Eric Haas (Oct 23 2021 at 02:21):
(I've already scripted a solution to sanitized my profiles so I am good personally)
Lloyd McKenzie (Oct 23 2021 at 02:38):
Diff should be rendered consistently - I'm definitely opposed to overrides. If something should be displayed, it should always be displayed - it shouldn't be up to author whim.
Chris Moesel (Oct 25 2021 at 12:41):
Lloyd McKenzie said:
I think the first question is whether constraints that aren't changes should be removed from the differential.
As noted above, this is the way SUSHI operates, but it's also worth noting that there are some exceptions required by the IG Publisher. For example, the sliceName element must be repeated in the differential if you constrain an inherited slice (even though you're not changing the sliceName). And we just recently learned that although a slice "inherits" the constraints from its base, you must repeat constraints from the base in every slice in order for the validator to work correctly.
So... if we were to suggest that differentials should only contain true differences, then we should also document the exceptions that require inclusion of non-different data due to limitations in the tooling. (Actually, we should probably document those anyway).
Matthijs van der Wielen (Nov 03 2021 at 14:35):
Should the tooling not do it's best to create a pure differential? As SUSHI is doing now. I think that the eld-1, that is incorrectly triggered, should be silenced here.
Last updated: Apr 12 2022 at 19:14 UTC
