Welcome to the New World Order
Project Illuminati is an absurdly multithreaded application designed for fast and flexible versioning of on-disk and live system configurations. It's suitable for versioning any device that runs an SSHd. Rsync is supported too (via the command line).
Backed by a Git repository, it provides a fast and easy means of comparing and maintaining configuration changes on your routers, switches, Linux, BSD boxen and whatever else! Illuminati was designed to be completly agnostic of what it's versioning, you simply tell it what command(s) it should execute on the remote host and it'll do them, grab the result and commit it. You can also have Illuminati create a git tag at the end of processing so that you can see what's changed between runs.
Illuminati also supports running output through a filter before committing it, so if you're versioning something that can get arbitrarily reordered on the host, you can invoke any Ruby functions you like to mutate it, you can even temporarily convert it to other datatypes (like an array, for example) but you must ensure that the final operation will make it a string. There's an example of this in the sample config that does a cleanup of iptables-save.
We highly recommend uploading your configuration files into a GitLab repository for beautiful web-based diffs, but if you prefer the Git commandline, that's cool too. RANCID is Rancid.
Open p2.sample.yml
in your favorite YAML editor (that's Notepad++, right?) for documentation on the configuration file.
The key points to take away from the example config is that you must have a root-level key for Illuminati to use. This is so that you can, if you wish, store multiple repository configs in a single file since the root-level key is considered to be an individual git repo.
Illuminati itself only cares about the hosts array and the defaults, the additional fluff exists in the example as a means of demonstrating how you can leverage the capabilities of YAML to avoid repeating yourself.
bundle install --deployment
bundle exec ruby p2.rb -f [configuration file] -n [YAML root section name]
Currently, there is a potential race condition with SSH hosts that rudely close the channel, resulting in only a partial capture of data. this typically only applies to older Cisco devices that you've configured with an autocommand.
Rsync is also not optimised right now and each rsync job will fetch into an empty temp dir rather than checking out from the current version and then letting rsync do all it's delta work, so bear this in mind if you intend to do versioning of something very large.