Customizable / Extensible movers

[Xnyle]

Make Volsync more modular/customizable/extensible

In search for a non bloat backup solution to replace my own I landed here.
Volsync looks very promising but is missing some features I'd need.

Currently I'm running my own Admission Controller plus Sidecar based approach:
Admission Controller looks at Pod Annotations and adds Backup Sidecars and Init Containers that work on the Volumes or Service endpoints configured.
Sidecars API is basically just a cron def and an additional ConfigMap for the actual backup config.
There are simple file backups/restores but also sidecars for MariaDB / Postgres, Mongo etc.
The destination/backup source is not inside a cluster, it's just a very minimalist ssh+rsync box (for a reason).

You may ask, if I have a working solution, why look further:
Cause I don't like the Admission Controller/Sidecar Idea, I'd rather have an Operator plus CRDs.
And Volsync already has that part plus it kind of already has a concept of replaceable Movers.
They just only seem to cover the topic how to move and not what and partially where to move.

What could be a starting point

The rsync Mover currently has hardcoded logik how to rsync. It assumes the other end is also a Volsync managed resource and it also does not bother with retention cause that's the Job ob the other side which in my case is just a dumb ssh+rsync host.
It would work, because basically it's just an rsync call, but I cant' configure it.
It does rsync -aAhHSxz --delete ... where I would need rsync -av --delete --link-dest=../$HISTDIR ... which would also cover hardlink based snapshots on the backup which is very neat.

In order to do that with Volsycnc The Source CRD would need an additional rsyncCommandTemplate Option.
I think that's very easy to implement.
But it doesn't cover Restores and Non File Backups.

Next steps

Making an existing Mover move configurable doesn't make the concept extensible. Why not decouple the Operator/Resources Definition Logik from the Mover Logik:

Movers are just a few Shell Script as they currently are. (And that's a good thing)
But they become Container Images that just get invoked.
The CRD gets moverImage and moverConfig or moverConfigMap instead of the rsync / restic / etc subnodes.
moverConfig or moverConfigMap has no fixed schema, it's just a Hashmap that gets added as ENV for the movers.
The Operator still provides Snapshots or Temp Volumes but only if asked to do so (in case of a MariaDB Mover not needed)

Benefits
Volsync does everything it currently does.
Additionally it provides a very simple but extensible "API" to create your own Movers.
Just copy a Mover Container Template that already exists, it only has 2 Shell scripts.
Modify the two scripts to suit your needs
Build the Image and use it with volsync.
Volsync handles all the heavy lifting of invoking Moves and attaching PVs.

Additional
Rename Volsync to Anysync after done.

[tesshuflower]

@Xnyle Right now VolSync has targeted scenarios like you said to perform specific actions.

I don't want to modify to make it completely generic at this point, there are always security implications when allowing scripts to be run, particularly with any parameters. That being said, it would be possible to make a generic mover of some sort (rather than replacing the current movers).

Maybe first off, have you had a chance to look at restic? We have the restic mover already for backup/restore scenarios where the rsync mover is more for a replication use-case. Restic is written for these backup/restore scenarios and will do snapshotting.