Stream: implementers
Topic: Representing array of data with variable timing
Vadim Peretokin (Nov 25 2021 at 08:39):
I need to represent two arrays of data coming from a device - representing head movement (in degrees) and the amount of time between measurements (in seconds). SampledData does not work here because it assumes the sampling period is consistent, which is not the case here.
What's the best way to go?
1) SampledData + extension on the 1..1 period to say that the element isn't applicable?
2) string extension with space-separated values
3) any others?
Grahame Grieve (Nov 25 2021 at 09:06):
well, if period didn't apply, how do you use SampledData?
Vadim Peretokin (Nov 25 2021 at 09:08):
The problem with period in SampledData is that it assumes a fixed interval, but I've got a variable interval here which is represented by another array
Vadim Peretokin (Nov 25 2021 at 09:09):
One way could be to populate with period with an extension that has this array of timings...
Grahame Grieve (Nov 25 2021 at 10:38):
that's quite the extension.
Vadim Peretokin (Nov 25 2021 at 10:41):
yep... any better approaches?
Grahame Grieve (Nov 25 2021 at 10:41):
another way to do it is to set period to 0 - this is not data that has a regular period, and use the first dimension as the time offset
Grahame Grieve (Nov 25 2021 at 10:41):
but... no, I don't like that
Vadim Peretokin (Nov 25 2021 at 10:43):
the choices are massacring sampleddata or creating a custom extension with semantics that fit variable timings, maybe the latter is better?
Grahame Grieve (Nov 25 2021 at 10:54):
maybe. But you have to massacre it whatever.
Grahame Grieve (Nov 25 2021 at 10:55):
it's really grounds for a new data type, since the definition of SampleData is quite clear.
Grahame Grieve (Nov 25 2021 at 10:55):
but that doesn't help you either
Grahame Grieve (Nov 25 2021 at 10:56):
how much data are we talking about?
Vadim Peretokin (Nov 25 2021 at 12:30):
I'll find out the max, in the sample I got - one array has 90 data points (degrees of range) for every exercise rep + corresponding array of timings (seconds taken per rep). There's also a 2D array of degrees of range that measures two degrees instead of one
Vadim Peretokin (Nov 25 2021 at 14:56):
Around a hundred would be max
John Moehrke (Nov 25 2021 at 15:27):
that may seem large, but could these be recorded as independent Observation? There are some kinds of data that are today recorded multiple times a day continuously (continuous blood-sugar, body weight, activity status).
Vadim Peretokin (Nov 25 2021 at 15:59):
Yeah, I think that would be too much... even separate Observation.component's would be big and not be really putting FHIR in a good light
Grahame Grieve (Nov 25 2021 at 17:56):
what's the semantics around repeated measurements? Do they ever have 'failed to record' fro some of the data?
Elliot Silver (Nov 25 2021 at 18:06):
@Vadim Peretokin If you do decide to use an extension, I think it needs to be a modifier extension. It totally changes how to interpret the data.
Vadim Peretokin (Nov 26 2021 at 14:47):
Agreed for sure.
Eric Haas (Nov 27 2021 at 23:37):
create your own structure ( json, yaml, csv, whatever ) and send as binary using the valueAttachment extension ( note this is a R5 pre-adoption element : http://hl7.org/fhir/5.0/StructureDefinition/extension-Observation.valueAttachment)
Grahame Grieve (Nov 29 2021 at 01:00):
ping on
Do they ever have 'failed to record' fro some of the data?
Vadim Peretokin (Dec 09 2021 at 09:28):
I'm told that would not happen (perhaps data wouldn't be sent at all). Either way there wouldn't be 'holes' in the array
Vadim Peretokin (Dec 09 2021 at 09:30):
Given how sampleddata works, I am thinking of going with solution (2) string modifier extension with space-separated values
Vadim Peretokin (Dec 09 2021 at 09:31):
that would be simpler to work with than a custom structure and follow an existing pattern already in the standard
Grahame Grieve (Dec 09 2021 at 09:56):
how long between sampling? is it in milliseconds or seconds? or longer than that
Grahame Grieve (Dec 09 2021 at 09:59):
we could add a new element offsets : string that has the same format as data, and specifies the instants at which measurements were made. In this scheme, period is redefined as the default offset increment for any data point that runs off the end of the array of offsets. That's procedurally possible, and the least worst way to handle sampled data that doesn't have regular time intervals
Vadim Peretokin (Dec 21 2021 at 12:45):
it's in seconds
Vadim Peretokin (Dec 21 2021 at 12:49):
That suggestion sounds great to me! I'll verify it & file a JIRA ticket.
Vadim Peretokin (Feb 02 2022 at 09:36):
I take it offsets should also take on the same time unit that period works with, so ms. Example:
"valueSampledData": {
"origin": {
"value": 0,
"code": "deg",
"unit": "degree",
"system": "http://unitsofmeasure.org",
},
"period": 1000,
"dimensions": 1,
"data": "47.53885435017747 90.00021045865756 90.00021045865756",
"offsets": "3378.787878787879 42.26542688081149 2500.000000000009"
}
data is degrees, offsets is when the measurement was made in ms, and logically period doesn't mean anything because the data and offsets are aligned in the amount of measurements
Vadim Peretokin (Feb 02 2022 at 09:47):
r4 version -
"valueSampledData": {
"origin": {
"value": 0,
"code": "deg",
"unit": "degree",
"system": "http://unitsofmeasure.org"
},
"period": 1000,
"dimensions": 1,
"data": "47.53885435017747 90.00021045865756 90.00021045865756",
"extension" : [{
"url" : "http://example.org/fhir/StructureDefinition/offsets",
"valueString" : "3378.787878787879 42.26542688081149 2500.000000000009"
}]
},
Grahame Grieve (Feb 03 2022 at 00:14):
well, yes, though I don't understand this because the offsets aren't ordered. Surely they'd have to be in order...
Vadim Peretokin (Feb 03 2022 at 06:52):
Right. Fixed - it was previously doing intervals not offsets -
"valueSampledData": {
"origin": {
"value": 0,
"code": "deg",
"unit": "degree",
"system": "http://unitsofmeasure.org",
},
"period": 1000,
"dimensions": 1,
"data": "47.53885435017747 90.00021045865756 90.00021045865756",
"offsets": "3378.787878787879 3421.053306 5921.053306"
}
Vadim Peretokin (Feb 03 2022 at 12:20):
Bigger example -
"code": {
"coding": [{
"system": "http://snomed.info/sct",
"code": "404979006",
"display": "Neck joint - range of movement"
}],
"text": "Range of head turns in yaw plane (Left-Right)"
},
"valueSampledData": {
"data": "90.537109375 60.652740478515625 26.10601806640625 27.388641357421875 29.38006591796875 29.771392822265625 36.079193115234375 36.02728271484375 39.947265625 39.11334228515625 36.36187744140625 39.76837158203125 35.002655029296875 37.106689453125 37.569183349609375 39.219970703125 36.10723876953125 36.199798583984375 38.24993896484375 40.15887451171875 30.865631103515625 30.717041015625 35.72735595703125 34.045623779296875 32.52325439453125 35.46441650390625 39.65191650390625 40.89208984375 38.74102783203125 38.68255615234375 34.196502685546875 33.570281982421875 33.662353515625 36.50860595703125 36.3001708984375 34.88165283203125 29.999114990234375 32.201751708984375 38.925994873046875 38.350677490234375 35.208740234375 36.309783935546875 35.1832275390625 36.083221435546875 35.595458984375 32.995849609375 32.04583740234375 36.79669189453125 31.879791259765625 32.9957275390625 37.14227294921875 31.359344482421875 30.902557373046875 28.116241455078125 35.347015380859375 39.050323486328125 35.38067626953125 37.117340087890625 38.162139892578125 36.11517333984375 34.652191162109375 39.923004150390625 45.19793701171875 44.25115966796875 35.553466796875 36.464996337890625 39.7689208984375 39.8714599609375 27.531036376953125 26.58917236328125 41.603363037109375 42.4051513671875 35.62786865234375 34.12286376953125 37.003570556640625 36.39971923828125 33.043701171875 37.58795166015625 40.32501220703125 37.0455322265625 33.568450927734375 37.696990966796875 37.075408935546875 37.482086181640625 35.437286376953125 38.6829833984375 30.6162109375 31.314422607421875 37.43499755859375",
"origin": {
"unit": "degree",
"system": "http://unitofmeasure.org",
"code": "deg"
},
"dimensions": 1,
"period": 1000,
"modifierExtension": [{
"url": "http://example.org/fhir/StructureDefinition/offsets",
"valueString": "3378 3921 4672 5921 6789 7239 7674 8109 8544 8979 9414 9849 10284 10719 11154 11589 12024 12459 12894 13329 13764 14199 14634 15069 15504 15939 16374 16809 17244 17679 18114 18549 18984 19419 19854 20289 20724 21159 21594 22029 22464 22899 23334 23769 24204 24639 25074 25509 25944 26379 26814 27249 27684 28119 28554 28989 29424 30159 30894 31629 32364 33099 33834 34569 35304 36039 36774 37509 38244 38979 39714 40449 41184 41919 42654 43389 44124 44859 45594 46329 47064 47799 48534 49269 50004 50739 51474 52209 52944"
}]
}
Grahame Grieve (Feb 03 2022 at 21:07):
do we have a task for this?
Vadim Peretokin (Feb 04 2022 at 08:24):
yup, here it is https://jira.hl7.org/browse/FHIR-35948
JIRA corrupted the json example and I can't seem to edit it
Last updated: Apr 12 2022 at 19:14 UTC
