Cache Distribution

[siennathesane]

Adds io.Writer interface to flush cache on eviction and/or manually via new bigcache.Flush() API to an arbitrary destination.

Tests to match.

[coveralls]

Coverage decreased (-1.3%) to 97.588% when pulling f07a8a0 on mxplusb:buffer-flush into b6bd76a on allegro:master.

[coveralls]

Coverage decreased (-1.9%) to 96.957% when pulling 42b1cc1 on mxplusb:buffer-flush into b6bd76a on allegro:master.

[siennathesane]

So to use gob encoding in Go, you have to have a fixed-width data type, so no slices. Due to the sharding design, without doing an entire rewrite of the whole sharding subsystem I won't be able to use gob encoding for flushing data to a writer. I found this out once I started looking at where coveralls said the test coverage was lower. I followed a crazy rabbit hole of good and bad ideas, and decided encoding/gob wasn't he right solution.

Once gob was determined to not be the right solution (spent like two hours on it), JSON seemed to be the next appropriate encoder. In order to have the proper schema for exporting the cache, I had a couple options. I could either continue with unexported field off the primary struct types and implement a custom marshaler, which would copy the relevant data into a new unexported struct with exported fields, or I could export the necessary fields on the current structs. The latter seemed less invasive and indirect.

Now that this aspect is done, I'm going to work on cache importing via io.Reader. With both of those interfaces in place, you'd then be able to export the cache and/or import a pre-built cache without having to repopulate it.

This is useful for distributed cache design across multiple machines, especially if you have a cache cluster, with a master -> slaves relationship for HA/Failover designs.

[janisz]

Thanks for PR.

I don't understand this PR. Title is misleading. What feature you really want to add? Can you start with documentation? It will review if we all understand what it should do.

JSON seemed to be the next appropriate encoder.

JSON is the worst data format you can use. It's designed for people not machines.

[janisz]

Why do you changed visibility of this fields? This makes PR unreadable and cover main purpose of this PR.

[wendigo]

And breaks API.

[janisz]

Closing due to lack of activity. Please reopen if you want to work on reported issues.