JSON API

[gsnw-sebast]

Introduction

The new JSON API for fping will be discussed in this section.

There are currently 3 versions, which are partly based on each other

• JSON output support with --json/-J option #160
• New option -j / --json for the JSON output in the functions print_per_system_stats() and print_global_stats() #379
• Issue/164-alternate - JSON functionality #380
• New option -J / --json for the JSON output #386 (NEW)

The best version is #380 on which the discussion should be based.

Topics

@schweikert

1. We might want to differentiate between output lines that contain summaries and those that contain details.
2. The output should be readable but as compact as possible (this is probably already fine)

More content to follow

[schweikert]

I suggest the following:

• The JSON output would be streamable and one "event" per line (except when pretty-printing is on).
• Every event can be of different types, with different fields. The event line would look like this:

  { "event type": { "field1": value1, "field2": value2, ... }}
  

• Unknown fields should be ignored by clients (so that we can extend events)
• In general every "event" represents a line in the current output.

Here are a couple of examples of the current output, and how it would look like in JSON:

1. fping -c3 1.1.1.1

   1.1.1.1 : [0], 64 bytes, 13.5 ms (13.5 avg, 0% loss)
   1.1.1.1 : [1], 64 bytes, 13.5 ms (13.5 avg, 0% loss)
   1.1.1.1 : [2], 64 bytes, 13.5 ms (13.5 avg, 0% loss)
   
   1.1.1.1 : xmt/rcv/%loss = 3/3/0%, min/avg/max = 13.5/13.5/13.5
   

   JSON:

   {"resp": {"host": "1.1.1.1", "seq": 0, "size": 64, "rtt": 13.5, "rttAvg": 13.5, "loss": 0}}
   {"resp": {"host": "1.1.1.1", "seq": 1, "size": 64, "rtt": 13.5, "rttAvg": 13.5, "loss": 0}}
   {"resp": {"host": "1.1.1.1", "seq": 2, "size": 64, "rtt": 13.5, "rttAvg": 13.5, "loss": 0}}
   {"summary": {"host": "1.1.1.1", "xmt": 3, "rcv": 3, "loss"=0, "rttMin": 13.5, "rttAvg": 13.5, "rttMax": 13.5}}
   

2. fping -Q1 -C2 1.1.1.1 8.8.8.8 -p1500

    [15:57:35]
    1.1.1.1 : xmt/rcv/%loss = 1/1/0%, min/avg/max = 13.5/13.5/13.5
    8.8.8.8 : xmt/rcv/%loss = 1/1/0%, min/avg/max = 3.61/3.61/3.61
    1.1.1.1 : 13.5 13.5
    8.8.8.8 : 3.61 3.64
   

   JSON:

   {"intSum": { "time": 1739631700, "host": "1.1.1.1", "xmt": 1, "rcv": 1, "loss"=0, "rttMin": 13.5, "rttAvg": 13.5, "rttMax": 13.5}}
   {"intSum": { "time": 1739631700, "host": "8.8.8.8", "xmt": 1, "rcv": 1, "loss"=0, "rttMin": 3.61, "rttAvg": 3.61, "rttMax": 3.61}}
   {"vSum": {"host": "1.1.1.1", "values": [ 13.5, 13.5]}}
   {"vSum": {"host": "8.8.8.8", "values": [3.61, 3.64]}}
   

   Here "intSum" would be for intermediate host summary, and "vSum" would be for "verbose host summary".

[auerswal]

I'd prefer if unsupported combinations of options with --json would result in an error exit of fping. Thus adding support would not break existing scripts.

[gsnw-sebast]

I would currently keep the JSON format as small as possible and agree with @schweikert.
The reason for this is the current implementation in the code, which is very bloated due to the many individual outputs.

It already looks quite confusing. Especially as I also remember that JSON is only one format and after JSON someone else will want to use XML or other formats, for example.

[gsnw-sebast]

What do you think about simply duplicating print_netdata() and modifying it for the JSON output?
print_netdata() has the same purpose, which might be considered for json first.

[schweikert]

Yes. I think that's a good idea. Maybe we can have a lightweight abstraction of the output function, which calls the appropriate function based on the wanted format. Something like:

void print_per_system_stats(void) {
    if (netdata_flag) {
            print_per_system_stats_netdata()
    }
    else if(json_flag) {
            print_per_system_stats_json()
    }
    else {
            print_per_system_stats_legacy()
    }
}

[auerswal]

Especially as I also remember that JSON is only one format and after JSON someone else will want to use XML or other formats, for example.

I do not think that additional generic output formats should be added after JSON. The JSON output can be used to generate other formats using external tools that work with JSON (e.g., jq or Python with the json standard library module).

[auerswal]

I'd think it could be helpful to have some use case descriptions. How would you like to use fping --json? (I do not (yet) have a use case for JSON output from fping. That makes it hard for me to review and evaluate pull requests for this feature.)

[schweikert]

The main use case I think is integration into tools that want to do some probing using fping.

[schweikert]

By the way another use case is errors. Those could also be surfaced as "error" events.

[auerswal]

I'd think that it might be simpler for tools using a new JSON output format from fping to only have to read one data stream, i.e., if errors were sent on standard output with a key called error.

[auerswal]

To look at another tool, iproute2 creates non-JSON error messages even when normal output is JSON:

$ ip --json a s dev lo | jq .
[
  {
    "ifindex": 1,
    "ifname": "lo",
    "flags": [
      "LOOPBACK",
      "UP",
      "LOWER_UP"
    ],
    "mtu": 65536,
    "qdisc": "noqueue",
    "operstate": "UNKNOWN",
    "group": "default",
    "txqlen": 1000,
    "link_type": "loopback",
    "address": "00:00:00:00:00:00",
    "broadcast": "00:00:00:00:00:00",
    "addr_info": [
      {
        "family": "inet",
        "local": "127.0.0.1",
        "prefixlen": 8,
        "scope": "host",
        "label": "lo",
        "valid_life_time": 4294967295,
        "preferred_life_time": 4294967295
      },
      {
        "family": "inet6",
        "local": "::1",
        "prefixlen": 128,
        "scope": "host",
        "valid_life_time": 4294967295,
        "preferred_life_time": 4294967295
      }
    ]
  }
]

$ ip --json a s dev lo0 | jq .
Device "lo0" does not exist.

I do not have a strong opinion regarding the format of error messages, i.e., JSON or not.

[schweikert]

I also don't think that having json-formatted errors is important, but what would be good is a JSON output format that:

1. Is streaming-compatible
2. Is future-proof (JSON consumers will be required to ignore unknown fields)

This would work with the format that I suggested above:

{"resp": {"host": "1.1.1.1", "seq": 0, "size": 64, "rtt": 13.5, "rttAvg": 13.5, "loss": 0}}
{"resp": {"host": "1.1.1.1", "seq": 1, "size": 64, "rtt": 13.5, "rttAvg": 13.5, "loss": 0}}
{"resp": {"host": "1.1.1.1", "seq": 2, "size": 64, "rtt": 13.5, "rttAvg": 13.5, "loss": 0}}
{"summary": {"host": "1.1.1.1", "xmt": 3, "rcv": 3, "loss"=0, "rttMin": 13.5, "rttAvg": 13.5, "rttMax": 13.5}}

We just need to make sure that stdout has JSON when the json option is used. stderr could still have non-JSON output for errors for example (and we might move that output to stdout/JSON at some point later).

[auerswal]

Pull request #386 has shown that we need to specify how to handle missing values. This pertains to at least "verbose host summary" lines if there have been timeouts, and failure to read TTL/Hop Count and/or TOS/TC byte. I would suggest to use the JSON null value.

fping uses NaN to indicate computed values that are not a number (e.g., because of division by zero). I'd suggest to use the JSON null value instead of the JSON string "NaN" for those cases, too.

What would work best for potential users of JSON output from fping?

[gsnw-sebast]

The JSON output is now available in the develop branch.
Next, I'm going to swap it into a function print_recv()