Implement classic AD benchmarks/examples with horde-ad

[Mikolaj]

Here's a popular set of benchmarks: https://github.com/gradbench/gradbench. An older and defunct one may provide inspiration as well: https://github.com/microsoft/ADBench. Probably, horde-ad doesn't include any of these examples nor anything related. There is a longer document by Andrew, among others, at https://arxiv.org/abs/1807.10129 about the older set of benchmarks, which may be worth skimming, too.

[Edit: horde-ad is already published] If publishing horde-ad on Hackage is needed or convenient for this task, we will publish it. If extending horde-ad turns out necessary for some of the examples (mostly, I image, to get better performance), this is a very useful thing to learn and horde-ad may get extended as a result.

More ideas for such diverse and relatively small benchmarks/examples are very welcome. For big examples, e.g., transformers, see #50 (comment).

[athas]

I'm willing to help out with implementing the GradBench benchmarks in horde-ad. There is no real reason to use ADBench, as GradBench contains everything from ADBench (and with better documentation).

GradBench already contains some Haskell implementations written using ad. They're pretty slow, but they can help illuminate the purposes of the benchmarks when the documentation is insufficient.

[Mikolaj]

Thank you very much, @athas! I've just read your excellent

https://sigkill.dk/blog/2025-04-03-gradbench.html

which already answers a lot of questions. Let me also add the rather obvious link to

https://github.com/gradbench/gradbench/blob/main/CONTRIBUTING.md

and a link to

https://futhark-lang.org/docs/prelude/doc/prelude/ad.html

The latter may be useful if we look at the Futhark benchmarks/tools for inspiration, e.g., because the Haskell ad library does not implement all benchmarks.

[athas]

I suggest taking the protocol implementation from the existing haskell tool. There's not much to it - just a controlling loop and some Aeson instances. In fact, a horde-ad tool can probably be written simply by copying the current tool and replacing the eval-specific modules, like this one.