I'd like my program to write to its stderr a log that is simultaneously human and machine readable.
Requirements:
- A log is a stream of messages. I.e. I can't write just one big xml or json document. Every message has to be parseable separately without requiring an incremental parser or generator.
- tty detection and log format selection switches are considered cheating for the purposes of this question. I want the same exact output to be simultaneously human and machine readable.
- No postprocessing for the same reasons as 2
- No ad hoc formats. I don't want the consumer to have to write a parser. Not even a trivial one.
- No formats that are too obscure. There must be a library to parse this format in the top 10 most popular general purpose programming languages and the library must be able to parse the entire log into a stream of messages out of the box without requiring the consumer to massage the data.
Pretty JSON fails 5 - most JSON parsing APIs cannot parse multiple concatenated JSON documents.
JSON Lines is not human readable especially if it contains nested data because the entire log entry ends up on a single line.
It appears that application/json-seq (RFC 7464) does allow for the JSON texts to be pretty printed (human readable) while only requiring very simple parsing on top of a regular JSON decoder. This is the closest one yet.
Post-mortem
I ended up rethinking my approach: log jsonlines - easiest to generate and consume, post-process with en external pretty-printer such as jq .
for human consumption.
YAML seems to be a fine choice. You can emit the log as stand-alone YAML documents, separated by
---
[1]. Having multiple documents is standard YAML and this is at least supported by the YAML parser I know about. Basic structured data can be printed in one line in YAML, while still being pretty human readable.https://yaml.org/ has a list of implementations for different programming languages. There is an implementation for all the top ten programming languages in the stackoverflow 2021 developer survey[2].
Example Log
[1] ok, that means you have to emit
---
every second line or you have to insert it before parsing the document between all lines. If you just emit a---
in every second line, YAML satisfies your fifth requirement.[2] except for Kotlin, but Kotlin users may use the java library as far as I can see