Stream: bulk data
Topic: empty line
Yunwei Wang (May 07 2020 at 21:49):
Does anywhere in Bulk Data IG specify is empty line is NDJSON is valid?
NDJSON spec says "The parser MAY silently ignore empty lines, e.g. \n\n."
1) If a NDJSON file has empty line, is that valid?
2) If the NDJSON file is empty, is that valid?
@Dan Gottlieb @Josh Mandel
Josh Mandel (May 07 2020 at 23:46):
I don't know why we would establish anything different from the spec. Are you encountering some sort of issue?
Yunwei Wang (May 08 2020 at 13:52):
Yes. I received an empty export file.
Josh Mandel (May 08 2020 at 14:07):
Gotcha. And is it your understanding that The server had no resources for you and chose to inform you in this fashion?
Yunwei Wang (May 08 2020 at 14:27):
It could be either the server does not that resource or the file export failed. IMO either case, server should not list that resource in output result. I am still thinking how to deal with empty line in the middle of a file. I may just ignore the empty line.
Josh Mandel (May 08 2020 at 15:08):
Ignoring empty lines in the middle of a file seems like the right thing. Ignoring invalid JSON lines in the middle of a file might also be the right thing ;-)
Yunwei Wang (May 11 2020 at 16:31):
Each JSON text MUST conform to the [RFC7159] standard I don't think I can "ignore" an invalid line.
Vladimir Ignatov (May 11 2020 at 17:10):
On the client side (when consuming/parsing) I would ignore every kind of new lines - leading, trailing or inside. On the back-end, when generating NDJSON, I believe one should avoid them all if possible.
Last updated: Apr 12 2022 at 19:14 UTC
