Check BookStack

[schoettl]

Sounds like a dumb question...

I want to scrape and check our BookStack (wiki) instance. I need to pass authentication cookies for authorization.

I can't find docs on file format expected by --cookie-jar . I also can't get the --header option working with a Cookie header.

[nix-shell:~]$ lychee --header 'Cookie: XSRF-TOKEN=base64encodedstring=' https://bookstack.example.de/
Error: Header value must be of the form key=value, got Cookie: XSRF-TOKEN=base64encodedstring=

[nix-shell:~]$ lychee --header 'Cookie: XSRF-TOKEN=base64encodedstring%3D' https://bookstack.example.de/
Error: invalid HTTP header name

The = and %3D is the last fill character of the base64 encoded auth key.

Lychee version 0.18.1

[mre]

Can you update to lychee 0.19.1? We made a lot of changes in the recent versions regarding headers. You can find more at https://lychee.cli.rs/troubleshooting/custom-headers/.
Basically, the separator is now :.

[schoettl]

Thanks for your reply! BTW, I listened to the Engineering Kiosk episode with you today, very nice! :)

I just cloned the repo and tested with current master.

Now (without --header) it says Network error. Why does it print http://lychee/ as URL in the error message?

I'm on NixOS, I'm in WireGuard VPN and the domain I'm testing is in my /etc/hosts file. In the browser I can access it.

$ cargo run lychee https://bookstack.mydomain.de/
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.36s
     Running `target/debug/lychee lychee 'https://bookstack.mydomain.de/'`

thread 'tokio-runtime-worker' panicked at lychee-bin/src/commands/check.rs:256:18:
cannot send response to queue: SendError { .. }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: Network error

Caused by:
    0: error sending request for url (http://lychee/)
    1: client error (Connect)
    2: dns error: failed to lookup address information: Name or service not known
    3: failed to lookup address information: Name or service not known

[mre]

It takes lychee as an argument to the cli. The error is a bit misleading because it assumes that non-existent paths are URLs, which is why it prepends the http schema. We might change that.

Can you try to add two dashes like so?
cargo run -- and after that the paths/urls you're trying to check.

[schoettl]

Oh, got it. Now it works with the Cookie header. I'm looking forward to the #78 recursion feature! At the moment I just don't have a list of all BookStack page URLs I need to check.

BTW I've opened an PR for NixOS update NixOS/nixpkgs#423986

[mre]

Perfect! Thanks.