Changes between Initial Version and Version 1 of Ticket #29864


Ignore:
Timestamp:
Mar 23, 2019, 6:50:45 PM (7 months ago)
Author:
anarcat
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #29864 – Description

    initial v1  
    1515To do this progressively while allowing us to make new alerts on Prometheus instead of Nagios, I suggest to proceed the same way Cloudflare did, which is to establish a "Nagios to Prometheus" bridge, by which Nagios doesn't send the alerts on its own and instead forwards them to the Prometheus server, a plugin they called [https://github.com/cloudflare/promsaint Promsaint].
    1616
    17 With the bridge in place, Nagios checks can be migrated into Prometheus alerts progressively without disruption. Note that Cloudflare documented their experience with Prometheus in [https://promcon.io/2017-munich/talks/monitoring-cloudflares-planet-scale-edge-network-with-prometheus/ this 2017 promcon talk]. Cloudflare also made a [https://github.com/cloudflare/unsee alert dashboard] and [https://github.com/cloudflare/alertmanager2es elasticsearch integration] which might be good to investigate further.
     17With the bridge in place, Nagios checks can be migrated into Prometheus alerts progressively without disruption. Note that Cloudflare documented their experience with Prometheus in [https://promcon.io/2017-munich/talks/monitoring-cloudflares-planet-scale-edge-network-with-prometheus/ this 2017 promcon talk]. Cloudflare also made an alert dashboard called [https://github.com/cloudflare/unsee unsee] (see also the fork called [https://github.com/prymitive/karma karma]) and [https://github.com/cloudflare/alertmanager2es elasticsearch integration] which might be good to investigate further.
    1818
    1919Another useful piece is this [https://www.robustperception.io/nagios-nrpe-prometheus-exporter NRPE to Prometheus exporter], which allows Prometheus to directly scrape NRPE targets. It doesn't include Prometheus alerts and instead relies on a Grafana dashboard to show possible problems so, as such, I don't think it's that useful an alternative.