Débogage Centreon Broker sur un Central
![Stacks Image 500297](files/stacks-image-9398564.jpg)
La page suivante est consacrée au bon fonctionnement d'un poller distant dans une architecture distribuée.
Les différents points de contrôles
![Stacks Image 500278](files/stacks-image-aad490a-1200x896.jpg)
Fonctionnement de Centreon-Broker
1 - Vérification du fonctionnement du plugin
tcpdump -i lo icmp
![Stacks Image 5432](files/stacks-image-76b99fb.png)
tcpdump -i eth0 src [IP Centreon] and dst [IP hote distant] and icmp
![Stacks Image 5435](files/stacks-image-95ee496.png)
tcpdump -i eth0 host [IP Centreon] and host [IP hote distant] and icmp
![Stacks Image 5438](files/stacks-image-a3f5b62.png)
![Stacks Image 500182](files/stacks-image-ccd2e11.png)
2 - Vérification du fonctionnement de Centreon-Engine
[root@vmtechcentreon ~]# service centengine status centengine status: running [ OK ] [root@vmtechcentreon ~]# ps aux | grep centengine 494 6629 0.5 0.8 650928 8552 ? Sl Feb05 96:45 /usr/sbin/centengine /etc/centreon-engine/centengine.cfg root 24405 0.0 0.0 103344 856 pts/0 S+ 17:17 0:00 grep centengine
[root@centreon35 ~]# /usr/sbin/centenginestats Centreon Engine Statistics Utility 1.8.1 Copyright 2003-2008 Ethan Galstad Copyright 2011-2013,2016 Centreon License: GPLv2 CURRENT STATUS DATA ------------------------------------------------------ Status File: /var/log/centreon-engine/status.dat Status File Age: 0d 0h 0m 12s Status File Version: (null) Program Running Time: 0d 1h 14m 12s Centreon Engine PID: 13588 Used/High/Total Command Buffers: 0 / 0 / 4096 Total Services: 12 Services Checked: 12 Services Scheduled: 12 Services Actively Checked: 12 Services Passively Checked: 0 Total Service State Change: 0.000 / 0.000 / 0.000 % Active Service Latency: 0.009 / 0.198 / 0.093 sec Active Service Execution Time: 0.131 / 1.875 / 0.354 sec Active Service State Change: 0.000 / 0.000 / 0.000 % Active Services Last 1/5/15/60 min: 2 / 5 / 12 / 12 Passive Service Latency: 0.000 / 0.000 / 0.000 sec Passive Service State Change: 0.000 / 0.000 / 0.000 % Passive Services Last 1/5/15/60 min: 0 / 0 / 0 / 0 Services Ok/Warn/Unk/Crit: 12 / 0 / 0 / 0 Services Flapping: 0 Services In Downtime: 0 Total Hosts: 1 Hosts Checked: 1 Hosts Scheduled: 1 Hosts Actively Checked: 1 Host Passively Checked: 0 Total Host State Change: 0.000 / 0.000 / 0.000 % Active Host Latency: 0.161 / 0.161 / 0.161 sec Active Host Execution Time: 0.087 / 0.087 / 0.087 sec Active Host State Change: 0.000 / 0.000 / 0.000 % Active Hosts Last 1/5/15/60 min: 0 / 1 / 1 / 1 Passive Host Latency: 0.000 / 0.000 / 0.000 sec Passive Host State Change: 0.000 / 0.000 / 0.000 % Passive Hosts Last 1/5/15/60 min: 0 / 0 / 0 / 0 Hosts Up/Down/Unreach: 1 / 0 / 0 Hosts Flapping: 0 Hosts In Downtime: 0 Active Host Checks Last 1/5/15 min: 0 / 1 / 1 Scheduled: 0 / 1 / 1 On-demand: 0 / 0 / 0 Parallel: 0 / 1 / 1 Serial: 0 / 0 / 0 Cached: 0 / 0 / 0 Passive Host Checks Last 1/5/15 min: 0 / 0 / 0 Active Service Checks Last 1/5/15 min: 2 / 9 / 9 Scheduled: 2 / 9 / 9 On-demand: 0 / 0 / 0 Cached: 0 / 0 / 0 Passive Service Checks Last 1/5/15 min: 0 / 0 / 0 External Commands Last 1/5/15 min: 0 / 0 / 0
![Stacks Image 5502](files/stacks-image-795c7a3-800x346.png)
![Stacks Image 500170](files/stacks-image-dbebc15-800x384.png)
![Stacks Image 500193](files/stacks-image-03fde67-798x222.png)
tail -f /var/log/centreon-engine/centengine.log
3 - Vérification des processus des brokers
3.1 - Centreon >= 2.8.x vérification du fonctionnement du watchdog cbwd
ps aux | grep "cb[wd|d]"résultat
496 5992 0.0 0.1 237528 1568 ? Sl Feb05 0:00 /usr/sbin/cbwd /etc/centreon-broker/watchdog.xml 496 5995 1.2 0.8 948992 8940 ? Sl Feb05 242:32 /usr/sbin/cbd /etc/centreon-broker/central-broker.xml 496 5996 0.3 0.8 561984 8872 ? Sl Feb05 66:24 /usr/sbin/cbd /etc/centreon-broker/central-rrd.xml
tail -f /var/log/centreon-broker/watchdog.log
3.2 - Centreon < 2.8.x vérification des process cbd
ps aux | grep cbdrésultat
6002 17389 0.1 1.9 105740 10148 ? Sl 04:29 0:13 /usr/local/centreon-broker/bin/cbd /usr/local/centreon-broker/etc/central-broker.xml 6002 17412 0.1 1.6 54640 8404 ? Sl 04:29 0:11 /usr/local/centreon-broker/bin/cbd /usr/local/centreon-broker/etc/central-rrd.xml root 20879 0.0 0.1 3324 828 pts/1 S+ 07:26 0:00 grep cbd
cat /usr/local/centreon-broker/etc/master.run ## This file define the Centreon Broker daemons to start # Example : # central-cbd central-cbd.xml y y # central-rrd central-rdd.xml y n # # The first line the daemon central-cbd is started with start action from # /etc/init.d/cbd and it's reload by action reload of init script # # The second line the daemon central-rrd is started but it is not reload by # action in /etc/cbd reload # # # daemon_name daemon_config running reload central-broker central-broker.xml y y central-rrd central-rrd.xml y y
rechargement de la configuration
tail -f /var/log/auth.logAprès un rechargement de la configuration, vous devriez voir ces lignes.
4 - Vérification du chargement du module cbmod
service centengine restartDeuxième terminal
tail -f /var/log/centreon-engine/centengine.log
4.1 - CentOS
[1518975609] [27197] Event broker module '/usr/lib64/centreon-engine/externalcmd.so' initialized successfully
![Stacks Image 5441](files/stacks-image-a31c9bf-800x260.png)
4.2 - Debian
[1518976002] [24235] Event broker module '/usr/lib/centreon-broker/cbmod.so' initialized successfully
![Stacks Image 500247](files/stacks-image-f3ae096-800x212.png)
5 - Vérification des logs de Centreon-Broker
tail -f /var/log/centreon-broker/central-module-master.logmodule central-broker-master (événements)
tail -f /var/log/centreon-broker/central-broker-master.logmodule central-rrd-master (graphe de parformances)
tail -f /var/log/centreon-broker/central-rrd-master.log
5.1 - exemples du module central-module-master (cbmod)
![Stacks Image 5408](files/stacks-image-0460261.png)
Base | Detailled | Very detailled | |
---|---|---|---|
Configuration messages | Informations essentielles liées à la configuration | Plus de détail sur la configuration statistique et sur "endpoint applier" | idem |
Debug messages | affichage création output, input | logs très verbeux, chargement des modules, activités du module NDO | logs très verbeux, chargement des modules, activités du module NDO, taille fichier retention |
Error messages | affichage erreurs | affichages erreurs callbacks | idem |
Informational messages | information création fichier rétention | très verbeux, affichage information chargement modules, activités NDO | idem, affichage création des macros personnalisées |
6 - Vérification des flux TCP-IP de Centreon-Broker
netstat -an | grep ^tcp | grep 56
tcp 0 0 0.0.0.0:5669 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:5670 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:5670 127.0.0.1:50993 ESTABLISHED tcp 0 0 127.0.0.1:50993 127.0.0.1:5670 ESTABLISHED tcp 0 0 127.0.0.1:5669 127.0.0.1:40067 ESTABLISHED tcp 0 0 127.0.0.1:40067 127.0.0.1:5669 ESTABLISHED
tcpdump -i lo tcp dst port 5669
tcpdump -i lo tcp dst port 5669 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernelliste le flux sortant, trame vide (length = 0)
tcpdump -i lo tcp src port 5662 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes 17:00:41.678100 IP localhost.5662 > localhost.53443: Flags [R.], seq 0, ack 538189214, win 0, length 0 17:00:41.678256 IP localhost.5662 > localhost.53444: Flags [R.], seq 0, ack 4201357894, win 0, length 0 17:01:11.705533 IP localhost.5662 > localhost.53445: Flags [R.], seq 0, ack 527048862, win 0, length 0 17:01:11.705653 IP localhost.5662 > localhost.53446: Flags [R.], seq 0, ack 1344875028, win 0, length 0
7 - Vérification du chargement dans la base - événements
tail -f /var/log/centreon-broker/central-broker-master.logCréation de l'hôte et de son service
[1384340300] [0x9946b40] info: SQL: processing host event (instance: 1, id: 25, name: central) ..... [1384340300] [0x9946b40] info: SQL: processing service event (host id: 25, service_id: 46, description: Ping)Lancement de la vérification du service
[1384340323] [0x9946b40] info: SQL: processing service check event (host: 25, service: 46, command: /usr/local/plugins/libexec/check_icmp -H 172.16.209.221 -n 3 -w 200,20% -c 400,50%)
Ajout des données de statut à la suite d'une vérification
[1384340324] [0x9946b40] info: SQL: processing service status event (host: 25, service: 46, last check: 1384340323, state (4, 1))
![Stacks Image 5691](files/stacks-image-9565553.jpg)
8 - Vérification du chargement dans la base - données de performance
tail -f /var/log/centreon-broker/central-broker-master.logCréation de la relation hôtes/services - métrique
[1384340323] [0x99a22c8] debug: storage: processing service status event [1384340323] [0x99a22c8] debug: storage: found index 13 of (25, 46) in cache [1384340323] [0x99a22c8] debug: storage: generating status event for (25, 46) of index 13Création des métriques
[1384340324] [0x99a22c8] debug: storage: processing service status event
[1384340324] [0x99a22c8] debug: storage: found index 13 of (25, 46) in cache
[1384340324] [0x99a22c8] debug: storage: generating status event for (25, 46) of index 13
[1384340324] [0x99a22c8] debug: storage: parsing perfdata string 'rta=0.019ms;200.000;400.000;0; pl=0%;20;50;; rtmax=0.041ms;;;; rtmin=0.007ms;;;;'
[1384340324] [0x99a22c8] debug: storage: got new perfdata (name=rta, value=0.019000, unit=ms, warning=200.000000, critical=400.000000, min=0.000000, max=nan)
[1384340324] [0x99a22c8] debug: storage: parsing perfdata string 'pl=0%;20;50;; rtmax=0.041ms;;;; rtmin=0.007ms;;;;'
[1384340324] [0x99a22c8] debug: storage: got new perfdata (name=pl, value=0.000000, unit=%, warning=20.000000, critical=50.000000, min=nan, max=nan)
[1384340324] [0x99a22c8] debug: storage: parsing perfdata string 'rtmax=0.041ms;;;; rtmin=0.007ms;;;;'
[1384340324] [0x99a22c8] debug: storage: got new perfdata (name=rtmax, value=0.041000, unit=ms, warning=nan, critical=nan, min=nan, max=nan)
[1384340324] [0x99a22c8] debug: storage: parsing perfdata string 'rtmin=0.007ms;;;;'
[1384340324] [0x99a22c8] debug: storage: got new perfdata (name=rtmin, value=0.007000, unit=ms, warning=nan, critical=nan, min=nan, max=nan)
[1384340324] [0x99a22c8] debug: storage: creating new metric for (13, rta)
..........
[1384340324] [0x99a22c8] info: storage: new metric 52 for (13, rta)
[1384340324] [0x99a22c8] debug: storage: inserting perfdata in data_bin (metric: 52, ctime: 1384340323, value: 0.019000, status: 4)
[1384340324] [0x99a22c8] debug: storage: generating perfdata event
[1384340324] [0x99a22c8] debug: storage: creating new metric for (13, pl)
..........
[1384340324] [0x99a22c8] info: storage: new metric 53 for (13, pl)
[1384340324] [0x99a22c8] debug: storage: inserting perfdata in data_bin (metric: 53, ctime: 1384340323, value: 0.000000, status: 4)
[1384340324] [0x99a22c8] debug: storage: generating perfdata event
[1384340324] [0x99a22c8] debug: storage: creating new metric for (13, rtmax)
.........
[1384340324] [0x99a22c8] info: storage: new metric 54 for (13, rtmax)
[1384340324] [0x99a22c8] debug: storage: inserting perfdata in data_bin (metric: 54, ctime: 1384340323, value: 0.041000, status: 4)
[1384340324] [0x99a22c8] debug: storage: generating perfdata event
[1384340324] [0x99a22c8] debug: storage: creating new metric for (13, rtmin)
........
[1384340324] [0x99a22c8] info: storage: new metric 55 for (13, rtmin)
[1384340324] [0x99a22c8] debug: storage: inserting perfdata in data_bin (metric: 55, ctime: 1384340323, value: 0.007000, status: 4)
[1384340324] [0x99a22c8] debug: storage: generating perfdata event
[1384340324] [0x99a22c8] debug: storage: current transaction has 2 pending queries
Mise à jour des métriques[1384340324] [0x99a22c8] debug: storage: found metric 52 of (13, rta) in cache
[1384340324] [0x99a22c8] info: storage: updating metric 52 of (13, rta) (unit: ms, warning: 200.000000, critical: 400.000000, min: 0.000000, max: nan)
[1384340324] [0x99a22c8] debug: storage: inserting perfdata in data_bin (metric: 52, ctime: 1384340323, value: 0.019000, status: 0)
[1384340324] [0x99a22c8] debug: storage: generating perfdata event
[1384340324] [0x99a22c8] debug: storage: found metric 53 of (13, pl) in cache
[1384340324] [0x99a22c8] info: storage: updating metric 53 of (13, pl) (unit: %, warning: 20.000000, critical: 50.000000, min: nan, max: nan)
[1384340324] [0x99a22c8] debug: storage: inserting perfdata in data_bin (metric: 53, ctime: 1384340323, value: 0.000000, status: 0)
[1384340324] [0x99a22c8] debug: storage: generating perfdata event
[1384340324] [0x99a22c8] debug: storage: found metric 54 of (13, rtmax) in cache
[1384340324] [0x99a22c8] info: storage: updating metric 54 of (13, rtmax) (unit: ms, warning: nan, critical: nan, min: nan, max: nan)
[1384340324] [0x99a22c8] debug: storage: inserting perfdata in data_bin (metric: 54, ctime: 1384340323, value: 0.041000, status: 0)
[1384340324] [0x99a22c8] debug: storage: generating perfdata event
[1384340324] [0x99a22c8] debug: storage: found metric 55 of (13, rtmin) in cache
[1384340324] [0x99a22c8] info: storage: updating metric 55 of (13, rtmin) (unit: ms, warning: nan, critical: nan, min: nan, max: nan)
[1384340324] [0x99a22c8] debug: storage: inserting perfdata in data_bin (metric: 55, ctime: 1384340323, value: 0.007000, status: 0)
[1384340324] [0x99a22c8] debug: storage: generating perfdata event
![Stacks Image 5694](files/stacks-image-abd46dc.jpg)
9 - Vérification des fichiers RRD
tail -f /var/log/centreon-broker/central-master-rrd.logUn extrait des logs, création d'un graphe et ses métriques
[1383720951] debug: RRD: new status data for index 39 (0) [1383720951] debug: RRD: updating file '/var/lib/centreon/status/39.rrd' (metric 'status', 1383720948:100) .... [1383720951] debug: RRD: new data for metric 48 (time 1383720948) [1383720951] debug: RRD: opening file '/var/lib/centreon/metrics/48.rrd' (DS:rta:GAUGE:300:U:U, RRA:AVERAGE:0.5:1:51841, RRA:AVERAGE:0.5:12:4320, interval 300, from 0) [1383720951] debug: RRD: updating file '/var/lib/centreon/metrics/48.rrd' (metric 'rta', 1383720948:0.159000) [1383720952] debug: RRD: new data for metric 49 (time 1383720948) [1383720952] debug: RRD: opening file '/var/lib/centreon/metrics/49.rrd' (DS:pl:GAUGE:300:U:U, RRA:AVERAGE:0.5:1:51841, RRA:AVERAGE:0.5:12:4320, interval 300, from 0) [1383720952] debug: RRD: updating file '/var/lib/centreon/metrics/49.rrd' (metric 'pl', 1383720948:0.000000) [1383720952] debug: RRD: new data for metric 50 (time 1383720948) [1383720952] debug: RRD: opening file '/var/lib/centreon/metrics/50.rrd' (DS:rtmax:GAUGE:300:U:U, RRA:AVERAGE:0.5:1:51841, RRA:AVERAGE:0.5:12:4320, interval 300, from 0) [1383720952] debug: RRD: updating file '/var/lib/centreon/metrics/50.rrd' (metric 'rtmax', 1383720948:0.217000) [1383720952] debug: RRD: new data for metric 51 (time 1383720948) [1383720952] debug: RRD: opening file '/var/lib/centreon/metrics/51.rrd' (DS:rtmin:GAUGE:300:U:U, RRA:AVERAGE:0.5:1:51841, RRA:AVERAGE:0.5:12:4320, interval 300, from 0) [1383720952] debug: RRD: updating file '/var/lib/centreon/metrics/51.rrd' (metric 'rtmin', 1383720948:0.093000)
[1383720951] debug: RRD: new status data for index 39 (0) [1383720951] debug: RRD: updating file '/var/lib/centreon/status/39.rrd' (metric 'status', 1383720948:100) ....... [1383720895] debug: RRD: new data for metric 36 (time 1383720893) [1383720895] debug: RRD: updating file '/var/lib/centreon/metrics/36.rrd' (metric 'rta', 1383720893:0.015000) [1383720895] debug: RRD: new data for metric 37 (time 1383720893) [1383720895] debug: RRD: updating file '/var/lib/centreon/metrics/37.rrd' (metric 'pl', 1383720893:0.000000) [1383720895] debug: RRD: new data for metric 38 (time 1383720893) [1383720895] debug: RRD: updating file '/var/lib/centreon/metrics/38.rrd' (metric 'rtmax', 1383720893:0.033000) [1383720895] debug: RRD: new data for metric 39 (time 1383720893) [1383720895] debug: RRD: updating file '/var/lib/centreon/metrics/39.rrd' (metric 'rtmin', 1383720893:0.006000)
10 - Vérification des flux Mysql
mysqladmin -u root -p[mot de passe] status processlistRésultat
Uptime: 1077123 Threads: 11 Questions: 2398234 Slow queries: 0 Opens: 750 Flush tables: 2 Open tables: 400 Queries per second avg: 2.226 +-------+----------+-----------+----------+---------+------+-------+------------------+----------+ | Id | User | Host | db | Command | Time | State | Info | Progress | +-------+----------+-----------+----------+---------+------+-------+------------------+----------+ | 61042 | centreon | localhost | centreon | Sleep | 2 | | | 0.000 | | 61063 | centreon | localhost | centreon | Sleep | 210 | | | 0.000 | | 61068 | centreon | localhost | centreon | Sleep | 170 | | | 0.000 | | 61115 | centreon | localhost | centreon | Sleep | 101 | | | 0.000 | | 61145 | centreon | localhost | centreon | Sleep | 49 | | | 0.000 | | 61148 | centreon | localhost | centreon | Sleep | 118 | | | 0.000 | | 61156 | centreon | localhost | centreon | Sleep | 136 | | | 0.000 | | 61172 | centreon | localhost | centreon | Sleep | 12 | | | 0.000 | | 61279 | centreon | localhost | centreon | Sleep | 194 | | | 0.000 | | 61304 | centreon | localhost | centreon | Sleep | 30 | | | 0.000 | | 62179 | root | localhost | | Query | 0 | | show processlist | 0.000 | +-------+----------+-----------+----------+---------+------+-------+------------------+----------+Centreon-Broker fonctionne
mysqladmin -u root -p[mot de passe] status processlistRésultat
Uptime: 1077133 Threads: 13 Questions: 2398424 Slow queries: 0 Opens: 750 Flush tables: 2 Open tables: 400 Queries per second avg: 2.226 +-------+----------+-----------+------------------+---------+------+-------+------------------+----------+ | Id | User | Host | db | Command | Time | State | Info | Progress | +-------+----------+-----------+------------------+---------+------+-------+------------------+----------+ | 61042 | centreon | localhost | centreon | Sleep | 12 | | | 0.000 | | 61063 | centreon | localhost | centreon | Sleep | 7 | | | 0.000 | | 61068 | centreon | localhost | centreon | Sleep | 180 | | | 0.000 | | 61115 | centreon | localhost | centreon | Sleep | 111 | | | 0.000 | | 61145 | centreon | localhost | centreon | Sleep | 59 | | | 0.000 | | 61148 | centreon | localhost | centreon | Sleep | 128 | | | 0.000 | | 61156 | centreon | localhost | centreon | Sleep | 146 | | | 0.000 | | 61172 | centreon | localhost | centreon | Sleep | 22 | | | 0.000 | | 61279 | centreon | localhost | centreon | Sleep | 204 | | | 0.000 | | 61304 | centreon | localhost | centreon | Sleep | 40 | | | 0.000 | | 62182 | centreon | localhost | centreon_storage | Sleep | 1 | | | 0.000 | | 62183 | centreon | localhost | centreon_storage | Sleep | 1 | | | 0.000 | | 62188 | root | localhost | | Query | 0 | | show processlist | 0.000 | +-------+----------+-----------+------------------+---------+------+-------+------------------+----------+Il y a deux process sur la base centreon_storage
![Stacks Image 5543](files/stacks-image-8c72677.png)
11 - Vérification des stats de Centreon-Broker
![Stacks Image 5527](files/stacks-image-5a43d0b.png)
![Stacks Image 5530](files/stacks-image-183d7f4.png)