Stream: implementers
Topic: StructureMap for FHIR to/from CSV
John Timm (Mar 12 2020 at 20:43):
Can a StructureMap be used to define mappings to/from a CSV file? If so, does anyone have an example of what it would look like?
Grahame Grieve (Mar 12 2020 at 20:55):
yes, in principle. Though it requires a class model for the csv file - two classes - a container that has a list of row classes, and then a row class that has a set of named string properties. Then you write a against that class model like any other class model
John Timm (Mar 12 2020 at 21:01):
And we would want to define said class model in a structure definition I assume?
Grahame Grieve (Mar 12 2020 at 21:05):
you could. yes.
Grahame Grieve (Mar 12 2020 at 21:06):
to actually make it work, you'd need some software that knew how to turn a csv into instances described by the structure definition.
Grahame Grieve (Mar 12 2020 at 21:06):
and maybe some kind of extension on the structure definition to say that it's expected format is CSV
John Timm (Mar 12 2020 at 21:12):
could be a neat extension to the work being done here:
https://build.fhir.org/ig/HL7/cdmh/profiles.html
Alexander Zautke (Mar 12 2020 at 21:54):
The .NET Implementation for executing StructureMaps does exactly that. You'll only need to pass in a CSV file to $transform, if the CSV contains contains a header row it will become available as source elements in the StructureMap. The underlying StructureDefinition is generated dynamically.
@John Timm Let me know if you want to try it out, I can give you some pointers on how to start.
Grahame Grieve (Mar 12 2020 at 21:55):
it would be helpful if the java one did the same - can you describe the algorithm?
Grahame Grieve (Mar 12 2020 at 21:55):
(the non obivous parts like how to handle non-valid names)
Abel Stolz (Mar 13 2020 at 06:42):
@Alexander Zautke I am working recently on exctly the same thing as John Timm. I would be very thankful if you could give us some pointers on how to start.
Alexander Zautke (Apr 25 2020 at 16:55):
Grahame Grieve said:
it would be helpful if the java one did the same - can you describe the algorithm?
Sounds great, I can prepare this for the Mapping Language Track at the connectathon
Alexander Zautke (Apr 25 2020 at 17:04):
Abel Stolz said:
Alexander Zautke I am working recently on exctly the same thing as John Timm. I would be very thankful if you could give us some pointers on how to start.
Sorry about the long delay! You can find the documentation for CSV support using the .NET FHIR Mapper here: http://docs.simplifier.net/mappingengine/featureslist/supportedformats.html#csv
Alexander Zautke (Apr 25 2020 at 17:05):
I just noticed that the public Vonk endpoint is currently not running the latest version of the FHIR Mapper, I will try to fix this in the next week so you can try it out without setting up your own Vonk server.
Amrit (Jun 29 2020 at 06:47):
With regard to StructureMap - is the StructureDefinition required for either or both of the source and target types? If I wish to map elements from a FHIR JSON object to a non-FHIR JSON object, and I define the mapping via a StructureMap, could the transform operation on a FHIR server perform the transformation even if I do not provide a StructureDefinition for the target structure?
Grahame Grieve (Jun 29 2020 at 07:12):
it's possible, though we haven't defined a standard way to process csv files. I think @Alexander Zautke was going to propose something?
Amrit (Jun 29 2020 at 07:21):
Since my question was similar to the one on this thread, I asked the question in the same thread - sorry - as such, I am primarily interested in going from FHIR to V2 and back (and V2 message can easily be represented as a JSON object).
Amrit (Jun 29 2020 at 07:23):
And I asked the question because I did not want to get into the tedious job of creating a StructureDefinition for the V2 JSON.
Grahame Grieve (Jun 29 2020 at 07:24):
oh it this same thread !
Grahame Grieve (Jun 29 2020 at 07:25):
@Alexander Zautke it now says "cannot display embedded content" ...
Alexander Zautke (Jun 29 2020 at 07:39):
Oh, the rendering seems to be broken. Will look into it. It should have rendered: https://simplifier.net/fhirmapperr4/csvtransport
Alexander Zautke (Jun 29 2020 at 07:43):
The main idea behind the CSV transformation is that the StructureDefinition for the CSV file itself is generated in the background for the user. The .NET implementation does currently not allow for untyped transformations. I think it would make sense to harmonize the way $transform is called for different content-types. The CSV_Transport logical model would be a suggestion on how to represent CSV content.
Alexander Zautke (Jun 29 2020 at 07:45):
Our implementation parses the CSV file, then create the StructureDefinition and then allows you use that in your StructureMap. I haven't worked on serializing back to CSV
Alexander Zautke (Jun 29 2020 at 07:46):
I still have to write down what the implementation does if it encounters a name in CSV which is not allowed in ElementDefinition.path
Grahame Grieve (Jun 29 2020 at 07:47):
so string, with the name of the column. that's straight forward. Questions for me:
- why 1..1?
- what to with with column names that are not regular tokens?
- do we have any support for implicit repeats (e.g. fields with commas in them?)?
- why the header layer? that can't matter for transforms, right?
Alexander Zautke (Jun 29 2020 at 12:24):
I choose string because I don't see the need of attempting to infer the type the content. I think it should be the task of the mapping using the cast function.
- The implementation verifies which elements are present in all rows. Depending on that it 0..1 or 1..1 is chosen.
- Two options are implemented in our solution: a) reject the CSV b) specify a replacement character for characters which are not allowed in the path
- I like the idea, though for now it's being handled in the mapping. In most cases you need to simply use FHIRPath to split the strings.
- If no header row is indicated by the Content-Type parameters as defined in RFC 4180 the elements are named record.field1 ... record.fieldN. Based on the hasHeader element you can dynamically adjust the mapping
Alexander Zautke (Jun 29 2020 at 12:31):
Amrit said:
And I asked the question because I did not want to get into the tedious job of creating a StructureDefinition for the V2 JSON.
Sorry, I don't know any workaround at the moment. I think many people are currently struggling with this issue. It would be awesome if we could start a project to create StructureDefinitions like we have for CCDA.
René Spronk (Jun 29 2020 at 12:34):
@Frank Oemig ?
Frank Oemig (Jun 30 2020 at 16:15):
We do have structure definitions for v2, at least as a draft. We are working on that in the v2+ tooling project.
Ivan Dubrov (Jul 01 2020 at 04:19):
Another approach (which is what we do) is to define standardized fixed mapping between CSV and Basic resource (each field would become an extension), then map Basic resource. Not as convenient as having dedicated data model for CSV, but can use existing tooling.
Frank Oemig (Jul 01 2020 at 08:24):
Ok, but who is capable to understand this massive set of extensions?
An extension is good for solving a single local problem, but each is in principle a new one...
Last updated: Apr 12 2022 at 19:14 UTC
