Monitoring systems with munin 

http://www.debian-administration.org/articles/229

by chris on Fri 26 Aug 2005

*Tags*: monitoring, munin

From the package description: Munin is a highly flexible and powerful solution used to create graphs of virtually everything imaginable throughout your network, while still maintaining a rattling ease of installation and configuration.

Getting munin 

Munin is in the Debian archive in two parts - munin - the part that creates the monitoring graphs (you can think of this as the munin server), and munin-node - the munin client program.

So - if monitoring one machine installation should be a simple:

apt-get install munin munin-node

Configuration 

Configuration is conducted via configuration files inside the directory /etc/munin:

munin.conf 

The configuration of the grapher/collector.

The default configuration is set up to monitor the local machine. Things that can be set here (defaults shown below)

# The data storage area
dbdir   /var/lib/munin
# The place to put the generated web pages
htmldir /var/www/munin
# The log files
logdir  /var/log/munin
# Temporary run files (pid file etc)
rundir  /var/run/munin
# Where to look for the HTML templates
tmpldir /etc/munin/templates

You can also setup hosts to check here - the default is for localhost:

[localhost.localdomain]
   address 127.0.0.1
   use_node_name yes

Examples are given in this file for adding other hosts.

munin-node.conf 

Setup of the node which server addresses can connect. For monitoring just your local machine the default is fine.

plugin-conf.d/munin-node 

Configuration of the plugins for this node. The syntax is simple:

[pluginname]
param val
param val

You can set user, group, etc for each plugin.

Allowed parameters are described at the top of the file.

plugins 

A directory in which each file is a symlink to a real plugin in /usr/share/munin/plugins.

Any plugin linked in here will be checked for and displayed in the resulting web pages.

Add the plugins you want (e.g. if running exim4 then I'd add exim_mailqueue and exim_mailstats).

You'll need to set user/group rights in the munin-node conf file.

Most plugins can be run from the command line with the autoconf param to check if they can run - e.g.

./exim_mailstats autoconf yes

Running munin 

Munin sets up a cron job via the file /etc/cron.d/munin which will run /usr/bin/munin-cron.

Running this file will poll each of the nodes - and then will create the graphs in /var/www/html which you can then browse under http://hostname.example.com/munin.

Examples can be found here: http://www.linpro.no/projects/munin/example/.

Keeping an eye on these graphs will help you to keep your server running healthily - and can give advance warning of problems to come.

Re: Monitoring systems with munin 

here is the complete and step by step guide for munin configuration in debian

step by step guide for munin configuration in debian
http://www.debianhelp.co.uk/munin.htm

Re: Monitoring systems with munin 

Two things that I find useful

As root

munin-node-configure --suggest

which will show you what you could be running and if munin reckons it would work

and

munin-node-configure --suggest --shell

which will generate a list of ln commands that can be run in a shell to install the services that munin said "suggest: yes" to in the above output

documented on: 27 Mar 2006, chris

Re: Monitoring systems with munin 

Personally, I've been using Cacti for some time now. I'd say it's prettier than Munin (which is important when the graphs are being shown to customers), and the built-in user authentication system makes it easy to gives access to specific graphs or collections of graphs.

I'd definitely recommend it.

Sam Bashton

Re: Monitoring systems with munin 

Yes - cacti does look nice. Seems a little heavier than munin though.

I think that for my use (two small PCs that serve websites for small groups of people) that munin will probably do. If I was going to monitor larger networks with SNMP enabled hardware etc - then cacti would probably be better.

documented on: 27 Aug 2005, chris

Re: Monitoring systems with munin 

Cacti can read the RRDtool files created by Munin. I did this with version 0.8.4 of cacti, so please take this with a grain of salt for newer versions.

  1. Create a data template. One data source item per Munin RRD file. E.g. for a CPU graph, define cpu_idle, cpu_nice, cpu_system and cpu_user. Enter the values as used by Munin, e.g. step is 300.
  2. Create a host template and associate the data template.
  3. Create a graph template. For each value (idle, nice, system, user) define an AREA and three GRPINTs, LAST, AVERAGE, and MAX.
  4. Define data sources. Enter the correct Data Source Pathes to the Munin RRD files. Again, do this for each value, idle, nice, system, user.
  5. Define a graph using the data sources and the graph template.

I wrote this while inspecting the unused but still existing Cacti test installation. So this probably won't work. Please post your recipe when you got thi working.

documented on: 24 Aug 2006, Lupe

Re: adding new hosts to munin 

OK - here's a sample 2 machine config (it's an anonymized copy of what I have running):

On the server /etc/munin.conf contains

dbdir   /var/lib/munin
htmldir /var/www/munin
logdir  /var/log/munin
rundir  /var/run/munin

tmpldir /etc/munin/templates

[host1.domain.tld]
       address xxx.xxx.xxx.xxx

[host2.domain.tld]
       address yyy.yyy.yyy.yyy

And the munin-node.conf contains

log_level 4
log_file /var/log/munin/munin-node.log
port 4949
pid_file /var/run/munin/munin-node.pid
background 1
setseid 1

host *
user root
group root
setsid yes

ignore_file ~$
ignore_file \.bak$
ignore_file %$
ignore_file \.dpkg-(tmp|new|old|dist)$
ignore_file \.rpm(save|new)$

host_name host1.domain.tld

allow ^127\.0\.0\.1$
allow ^xxx\.xxx\.xxx\.xxx$

On the second machine (client only) munin-node.conf contains

log_level 4
log_file /var/log/munin/munin-node.log
port 4949
pid_file /var/run/munin/munin-node.pid
background 1
setseid 1

host *
user root
group root
setsid yes

ignore_file ~$
ignore_file \.bak$
ignore_file %$
ignore_file \.dpkg-(tmp|new|old|dist)$
ignore_file \.rpm(save|new)$

host_name host2.domain.tld

allow ^127\.0\.0\.1$
allow ^xxx\.xxx\.xxx\.xxx$
allow ^yyy\.yyy\.yyy\.yyy$

If you're not [getting graph] I'd check the /var/log/munin files on both server and client.

documented on: 24 Jan 2006, chris

munin: alerts? 

Well, I managed to install munin on a server and one client machine — it works quite nicely it seems.

I then proceeded to hook it up with hddtemp so that i could monitor the drive temperatures of my boxen (as per this great d-a article). http://www.debian-administration.org/articles/327

This required me to do three things:

  1. apt-get install smartmontool which is apparently needed by munin's hddtemp_smartctl plugin ;
  2. and ln -s /usr/share/munin/plugins/hddtemp_smartctl /etc/munin/plugins/hddtemp_smartctl

    1. which 'activates' the plugin ;
  3. and /etc/init.d/munin-node restart to apply these changes.

Now that this is done, I have temperature graphs for all my sensor-enabled hard drives, which is pretty great. But ultimately staring at a graph all day isn't all that efficient, so what I'd like to do now is have munin send me an e-mail when the temperature of any drive hits, say, 42^C.

Unfortunately the official documentation pages are kinda incomplete at the moment, so I'm not sure what to do.. there seems to be a page specifically about sending alerts, but it's not too clear..

Am I supposed to just dive in and start modifying the actual /etc/munin/plugins/hddtemp_smartctl plugin file (it's Perl) to stick in the warning trigger values?

Or is there a cleaner, more global way of doing this somewhere? Can anyone here provide an example?

munin: alerts? 

> Many munin plugins have thresholds configured, and munin can send alerts to
> Nagios. See http://munin.projects.linpro.no/wiki/HowToContact[]

Yes, thank you, I am aware of this, as I have also visited that page. Installing Nagios is not an option for now — I'm trying to keep the root partition as small as possible for reasons I won't go into here — but I did try to follow the non-Nagios instructions at the bottom of the page, specifically this snippet:

contact.email.command mail -s "Munin-notification for ${var:group} :: ${var:host}" your@email.address.here

The problem is, I'm still not getting any e-mail alerts even though two of my hard drives have reached 'critical' capacity levels, and my Munin-generated pages recognize this fact by marking said values with an orange background.

Does anyone have a clue as to what it is that I am missing?

(and while I do appreciate all advice from d-a users, I'd like to stress that Nagios is not an answer in my opinion — given the fact that Munin should already be able to send a simple mail by itself, I think installing Nagios only for this purpose would be like installing Windows 98 for its screensavers..)

munin: alerts? 

I tend to use munin for monitoring of history/trends over time and nagios for current status/warning.

Take a look at http://www.debian-administration.org/articles/299

documented on: 24 Jan 2006, chris