Stream: bulk data
Topic: zip
Brian Kolowitz (Aug 30 2019 at 11:24):
we have a client that would like the bulk data zipped. would it re reasonable for my server to support _outputFormat=application/zip and return a .zip file or should i write a small app that consumes the .ndjson from the server and packages it for the client?
Jeffrey Taylor (Aug 30 2019 at 12:51):
I like the idea of a the application/zip value for _outputFormat.
An alternative could be a request that looks something like:
/Patient/$export?_type=DocumentManifest&_typeFilter =.... This would result in an .ndjson file containing 1 document manifest that has 1 Attachment that contains a url pointing to the zip file.
I think Brian's approach is simpler, but it'd be helpful to think and discuss alternatives to fulfill this use case.
Brian Kolowitz (Aug 30 2019 at 12:53):
@Jeffrey Taylor I like your suggestion, my question regards how do we specify that the DocumentManifest contains a zip as opposed to a pointer to the ndjson
Jenni Syed (Aug 30 2019 at 14:27):
I thought we did this at a connectathon using standard HTTP headers... let me look
Jenni Syed (Aug 30 2019 at 14:27):
Accept-Encoding
Jenni Syed (Aug 30 2019 at 14:28):
Because if you replaced the output format, you wouldn't know what was in the zip (fhir JSON, ND JSON, etc)
Jenni Syed (Aug 30 2019 at 14:28):
I could be remembering incorrectly
Josh Mandel (Aug 30 2019 at 14:29):
Our experience in connectathon led us to encourage/prefer clients to request encodings, as Jenni says here. This way we re-use all the content type negotiations.
Josh Mandel (Aug 30 2019 at 14:30):
It doesn't quite give you "a zip" at the end of the process, but it allows a client to limit bandwidth required, and the client can pipe the file to a zip or whatever its compressed format of choice might be.
Brian Kolowitz (Aug 30 2019 at 14:31):
I'm considering returning an ndjson with a single row containing a documentmanifest that contains a link to a zip file... not sure if that makes sense
Josh Mandel (Aug 30 2019 at 14:39):
I don't think it makes a ton of sense. That is to say: you'd seem to lose most of the API capabilities and you wouldn't get much in the way of interop.
Brian Kolowitz (Aug 30 2019 at 14:44):
@Josh Mandel thanks for the gut check.
Grahame Grieve (Aug 31 2019 at 13:11):
zip is tricky because it's both a compression and a folder format.
Last updated: Apr 12 2022 at 19:14 UTC
