When monitoring systems, as in automagically checking all business processes and systems are ok, I used to use a Master/Slave setup. This was built with plain Nagios installs and a ssh-tunnel over the Internet.
And then the responsability grows. Another slave? More (manual) labor? right.
I started a search to find the best option which would suit my needs. A master server locally in the office and either separate masters per remote location, or slaves running reporting to my master. I run monitoring as a service. Although the configuration of users and the hostgroups they can see can be configured in Nagios, I do not need the remote users to see my systems, however I do need to see these..
Running slaves means a proper connection to the master needs to exist and remain. Failing communications between slave and master means false positives, as the failing services will start to yell. And adding services and eventhandlers to restart tunnels is not fun.
Running separate masters is better since the dependancy between slave and master is gone. No master server available means no visibility for me, however both the remote customer and myself still see the results. (I would get the emails anyhow)
But how would I connect the 3 masters (1 main, 2 remote) together? 3 browser windows continuously open? 3 profiles on my Unagi app?
I bumped into Thruk. From their website:
“Thruk is an independent multibackend monitoring webinterface which
currently supports Nagios, Icinga and Shinken as backend using the Livestatus addon.
It is designed to be a “dropin” replacement. The target is to
cover 100% of the original features plus additional enhancements for large
installations.”
And that’s what I needed. Same as Nagios, more funtionality, drop in replacement. So each master still can be accessed using the regular webinterface and for myself, I have the total overview. Wicked.
Thruk was a piece of cake for the installation. On their website they offer the *.deb package for my linux distribution, which includes all required packages. A bit more work was the installation of the livestatus broker for all masters. Different distributions, so different actions. Was it hard? no. easy.
So now running thruk on my main machine, having the total picture, while each remote site has its own installation. All installs are maintained from NagiosQL, a MySQL driven php web application for easy configuration. In NagiosQL, I have each master in its own nagiosql-domain. This gives complete separation beyond the borders of application security. Done.