User Tools

Site Tools


icinga:icinga_configuration_page3

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

icinga:icinga_configuration_page3 [2020/04/10 17:38] (current)
Line 1: Line 1:
 +===== 3. Adding A Remote Server To Icinga =====
  
 +
 +|[[:​icinga:​icinga_confgiguration|Page 1]] |[[:​icinga:​icinga_configuration_page2|Page 2]] |[[:​icinga:​icinga_configuration_page3|Page 3]] |[[:​icinga:​icinga_configuration_page4|Page 4]] |
 +
 +Monitoring ​ localhost is nice, but of  course, it would be even better if we could monitor all of our servers ​ in one location. This is possible with Icinga, and this chapter ​ describes how we can add our second Ubuntu 11.10 server to the setup.
 +
 +
 +To do this, we need to install the Nagios NRPE (Nagios Remote Plugin Executor) server on the server to be monitored, and the Nagios NRPE plugin on Icinga server (monitoring server). The NRPE server will listen on server to be monitored; and Icinga server will connect to it using the NRPE plugin and pass commands to it that the NRPE server will execute on the monitored server; finally, it will pass back the results to Icinga server.
 +
 +
 +First we install the  nagios-nrpe-plugin package on Icinga server:
 +
 +
 +''​Icinga server''​
 +<​code>​
 +apt-get install nagios-nrpe-plugin
 +</​code>​
 +
 +
 +Nagios web administration password:<​– nagiosadmin_password \\
 +Password confirmation:<​– nagiosadmin_password
 +
 +
 +''​Monitored server''​
 +
 +
 +Now we go to monitored server:
 +
 +
 +Install the  nagios-nrpe-server package :
 +
 +
 +<​code>​
 +apt-get install nagios-nrpe-server
 +</​code>​
 +
 +
 +Now open  /​etc/​nagios/​nrpe.cfg :
 +
 +
 +<​code>​
 +vi /​etc/​nagios/​nrpe.cfg
 +</​code>​
 +
 +
 +We must configure the NRPE server to allow Icinga server to connect, therefore we add IP-ADDRESS Icinga server to the  allowed_hosts line:
 +
 +
 +<​code>​
 +[...]
 +# ALLOWED HOST ADDRESSES
 +# This is an optional comma-delimited list of IP address or hostnames
 +# that are allowed to talk to the NRPE daemon.
 +#
 +# Note: The daemon only does rudimentary checking of the client'​s IP
 +# address. ​ I would highly recommend adding entries in your /​etc/​hosts.allow
 +# file to allow only the specified host to connect to the port
 +# you are running this daemon on.
 +#
 +# NOTE: This option is ignored if NRPE is running under either inetd or xinetd
 +
 +allowed_hosts=127.0.0.1,​192.168.0.100
 +[...]
 +</​file>​
 +
 +(If you don't do this, you will get the following error when you run
 +
 +/​usr/​lib/​nagios/​plugins/​check_nrpe -H 192.168.0.101 on Icinga server:
 +
 +root@Icinga_server:/​etc/​nagios-plugins/​config#​ /​usr/​lib/​nagios/​plugins/​check_nrpe -H IP_ADDRESS_SERVER...........here it is IP address of monitored server\
 +
 +CHECK_NRPE: Error - Could not complete SSL handshake.\
 +
 +root@Icinga_server:/​etc/​nagios-plugins/​config# ​ )
 +
 +Also, Icinga server needs to be allowed to pass command line arguments to the NRPE server, so still in the same file we set
 +dont_blame_nrpe
 +to
 +1
 +:
 +
 +<​code>​[...]
 +# COMMAND ARGUMENT PROCESSING
 +# This option determines whether or not the NRPE daemon will allow clients
 +# to specify arguments to commands that are executed. ​ This option only works
 +# if the daemon was configured with the --enable-command-args configure script
 +# option.
 +#
 +#<​nowiki>​ *</​nowiki><​nowiki>​*</​nowiki><​nowiki>​*</​nowiki>​ ENABLING THIS OPTION IS A SECURITY RISK!<​nowiki>​ *</​nowiki><​nowiki>​*</​nowiki><​nowiki>​*</​nowiki>​
 +# Read the SECURITY file for information on some of the security implications
 +# of enabling this variable.
 +#
 +# Values: 0=do not allow arguments, 1=allow command arguments
 +
 +dont_blame_nrpe=1
 +[...]
 +</​file>​
 +
 +(If you don't do this, you will see the error
 +
 +CHECK_NRPE: Received 0 bytes from daemon. Check the remote server logs for error messages.
 +
 +for lots of remote service checks in the Icinga web interface, and in
 +/​var/​log/​syslog
 +on monitored server you will see these errors:
 +
 +Aug 23 14:20:20 monitored_server nrpe[11496]:​ Error: Request contained command arguments, but argument option is not enabled!\
 +
 +Aug 23 14:20:20 monitored_server nrpe[11496]:​ Client request was invalid, bailing out...
 +
 +)
 +
 +Finally we must add command definitions for each service check we want to run on monitored server and that is not already defined. I want to run the the
 +check_procs
 +,
 +check_all_disks
 +, and
 +check_mysql_cmdlinecred
 +checks on monitored server; these are not defined in
 +/​etc/​nagios/​nrpe.cfg
 +, so I add them now (I also want to run the
 +check_users
 +and
 +check_load
 +checks, but these are already defined):
 +
 +<​code>​[...]
 +command[check_procs]=/​usr/​lib/​nagios/​plugins/​check_procs -w 250 -c 400
 +command[check_all_disks]=/​usr/​lib/​nagios/​plugins/​check_disk -w '​20%'​ -c '​10%'​ -e
 +command[check_mysql_cmdlinecred]=/​usr/​lib/​nagios/​plugins/​check_mysql -H localhost -u '​nagios'​ -p '​howtoforge'​
 +[...]
 +</​code>​
 +
 +
 +(If you don't do this, you will get errors like
 +
 +
 +NRPE: Command '​check_all_disks'​ not defined\\
 +NRPE: Command '​check_mysql_cmdlinecred'​ not defined\\
 +NRPE: Command '​check_procs'​ not defined
 +
 +
 +in the Icinga web interface.)
 +
 +
 +As you see I have hardcoded the command line arguments because using variables like  command[check_procs]=/​usr/​lib/​nagios/​plugins/​check_procs -w
 +
 +
 +$ARG1$
 +
 +
 +-c
 +
 +
 +$ARG2$ did not work for me. But still, when we configure the service checks for monitored server on Icinga server, we will have to pass command line arguments to these checks; monitored server will ignore these because I have hardcoded the comand line arguments into  /​etc/​nagios/​nrpe.cfg , but if you leave them out, you will get errors like  /​usr/​lib/​nagios/​plugins/​check_nrpe:​ option requires an argument – '​a'​ in the Icinga web interface.
 +
 +
 +Now save the file and restart the NRPE server:
 +
 +
 +/​etc/​init.d/​nagios-nrpe-server restart
 +
 +
 +Now check if the NRPE server is listening:
 +
 +
 +''​Icinga server''​
 +<​code>​
 +netstat -tap | grep nrpe
 +
 +root@monitored_server:​~#​ netstat -tap | grep nrpe
 +tcp        0      0 *:​nrpe ​                 *:*                     ​LISTEN ​      ​23668/​nrpe
 +root@monitored_serv:​~#​
 +</​code>​
 +
 +
 +Now go back to Icinga server and configure to check if it can connect to the NRPE server on monitored server:
 +
 +
 +<​code>​
 +/​usr/​lib/​nagios/​plugins/​check_nrpe -H IP_ADDRESS ​ ;this time IP address of monitored server
 +</​code>​
 +
 +
 +output should be as follows in case of success:
 +
 +
 +<​code>​
 +root@Icinga_server:​~#​ /​usr/​lib/​nagios/​plugins/​check_nrpe -H IP_ADDRESS ​  ;IP address of monitored server
 +NRPE v2.12
 +root@Icinga_server:​~#​
 +</​code>​
 +
 +
 +''​Monitored server''​
 +
 +
 +We want to check MySQL on monitored server; because we use the NRPE daemon, we can run the check locallyon monitored server, i.e., we don't have to open MySQL to the outside to allow Icinga server to run the check. Therefore I create the MySQL user  nagios for  localhost and  localhost.localdomain instead of for Ip addreress of Icinga sever and  server1.example.com :
 +
 +
 +<​code>​
 +mysql -u root -p
 +
 +GRANT USAGE ON *.* TO nagios@localhost IDENTIFIED BY '​howtoforge';​
 +GRANT USAGE ON *.* TO nagios@localhost.localdomain IDENTIFIED BY '​howtoforge';​ FLUSH PRIVILEGES;
 +quit;
 +</​code>​
 +
 +
 +Now we go back to Icinga server…
 +
 +
 +<​code>​
 +Icinga server
 +</​code>​
 +
 +
 +.. and create the Icinga configuration for monitored_server:​
 +
 +
 +<​code>​
 +vi /​etc/​icinga/​objects/​server2_icinga.cfg
 +</​code>​
 +
 +
 +<​code>​
 +use                             ​generic-service ​        ; Name of service template to use
 +       ​host_name ​                      ​server2.example.com
 +       ​service_description ​            Disk Space
 +       ​check_command ​                  ​check_nrpe!check_all_disks!20%!10%
 +}
 +define service{
 +       ​use ​                            ​generic-service
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +       ​service_description ​            ​Current Users
 +       ​check_command ​                  ​check_nrpe!check_users!20!50
 +}
 +define service{
 +       ​use ​                            ​generic-service
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +       ​service_description ​            Total Processes
 +       ​check_command ​                  ​check_nrpe!check_procs!250!400
 +}
 +define service{
 +        use                             ​generic-service ​        ; Name of service template to use
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +        service_description ​            ​Current Load
 +        check_command ​                  ​check_nrpe!check_load!5.0!4.0!3.0!10.0!6.0!4.0
 +}
 +define service{
 +       ​use ​                            ​generic-service
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +       ​service_description ​            MySQL
 +       ​check_command ​                  ​check_nrpe!check_mysql_cmdlinecred!nagios!howtoforge
 +}
 +define service{
 +       ​use ​                            ​generic-service
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +       ​service_description ​            SMTP
 +       ​check_command ​                  ​check_smtp
 +}
 +define service{
 +       ​use ​                            ​generic-service
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +       ​service_description ​            POP3
 +       ​check_command ​                  ​check_pop
 +}
 +define service{
 +       ​use ​                            ​generic-service
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +       ​service_description ​            IMAP
 +       ​check_command ​                  ​check_imap
 +}
 +</​code>​
 +
 +
 +(As I've mentioned before, although I have hardcoded the command line arguments for some commands into  /​etc/​nagios/​nrpe.cfg on monitored_server,​ we still need to add command line arguments to certain these checks here.)
 +
 +
 +As you see, I use  check_nrpe for some checks and pass the actual check (like  check_all_disks ) as a command line argument to  check_nrpe . These are the checks that will be executed locally by the NRPE server on monitored_server
 +
 +
 +.  check_nrpe is not needed for all checks. Checks that test a connection from the outside like  check_ping or  check_smtp can be run from  server1 .
 +
 +
 +To check the SSH and HTTP services on monitored_server
 +
 +
 +, we can EITHERadd the following stanzas to   /​etc/​icinga/​objects/​server2_icinga.cfg …
 +
 +
 +<​code>​
 +[...]
 +define service {
 +   ​use ​                            ​generic-service
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +        service_description ​            SSH
 +        check_command ​                  ​check_ssh
 +}
 +define service {
 +        use                             ​generic-service
 +       ​host_name ​                      ​monitored_server.example.com ; OR IP_ADDRESS
 +        service_description ​            HTTP
 +        check_command ​                  ​check_http
 +}
 +</​code>​
 +
 +
 +… ORwe add
 +
 +
 +monitored_server .example.com to the  http-servers and  ssh-servers hostgroups in  /​etc/​icinga/​objects/​hostgroups_icinga.cfg :
 +
 +
 +<​code>​
 +vi /​etc/​icinga/​objects/​hostgroups_icinga.cfg
 +</​code>​
 +
 +
 +<​code>​
 +# Some generic hostgroup definitions
 +
 +# A simple wildcard hostgroup
 +define hostgroup {
 +   ​hostgroup_name ​ all
 +                alias           All Servers
 +                members<​nowiki> ​        ​*</​nowiki>​
 +        }
 +
 +# A list of your Debian GNU/Linux servers
 +define hostgroup {
 +        hostgroup_name ​ debian-servers
 +                alias           ​Debian GNU/Linux Servers
 +                members ​        ​localhost,​monitored_server.example.com
 +        }
 +
 +# A list of your web servers
 +define hostgroup {
 +        hostgroup_name ​ http-servers
 +                alias           HTTP servers
 +                members ​        ​localhost,​monitored_server.example.com
 +        }
 +
 +# A list of your ssh-accessible servers
 +define hostgroup {
 +        hostgroup_name ​ ssh-servers
 +                alias           SSH servers
 +                members ​        ​localhost,​monitored_server.example.com
 +        }
 +</​code>​
 +
 +
 +Restart Icinga:
 +
 +
 +<​code>​
 +/​etc/​init.d/​icinga restart
 +</​code>​
 +
 +
 +Afterwards you should find  server2 in the Icinga web interface:
 +
 +
 +[[http://​static.howtoforge.com/​images/​icinga_monitoring_ubuntu_11.10/​big/​6.png|{{http://​static.howtoforge.com/​images/​icinga_monitoring_ubuntu_11.10/​6.png?​nolink&​409x296}}]]
 +
 +
 +[[http://​static.howtoforge.com/​images/​icinga_monitoring_ubuntu_11.10/​big/​8.png|{{http://​static.howtoforge.com/​images/​icinga_monitoring_ubuntu_11.10/​8.png?​nolink&​409x296}}]]
 +
 +
 +|[[:​icinga:​icinga_confgiguration|Page 1]] |[[:​icinga:​icinga_configuration_page2|Page 2]] |[[:​icinga:​icinga_configuration_page3|Page 3]] |[[:​icinga:​icinga_configuration_page4|Page 4]] |
icinga/icinga_configuration_page3.txt · Last modified: 2020/04/10 17:38 (external edit)