Full monitoring system: Graphite, collectd and StatsD – Part 2

Introduction

In the first part of this series we installed and configured Graphite on an Ubuntu 16.04 server.
Remember that

is software for visualizing data, so it’s almost useless if there isn’t a way to collect data and pass it on. This is were 

become a fundamental part of the full monitoring system: it is a daemon which gathers metrics and makes the information available to the system admin.

In this tutorial we will explore the steps for installing and configuring 

 to pass data to 

.

Collectd Installation

is available in repositories, so you can install it with 

:

# apt install collectd collectd-utils

That’s it! Now, let’s configure.

Configuring collectd

Edit 

configuration file:

# $EDITOR /etc/collectd/collectd.conf

In the first lines there is the hostname section. Here, the default file looks like this:

#Hostname "localhost" 
FQDNLookup true

This works fine if there is a real domain name configured. It’s the DNS which gets the proper domain.

Since, in this tutorial,

and 

will run on the same machine, uncomment the first line and set a hostname.

On line 42, there is a parameter called Interval which is the time that 

waits before querying data. Uncomment that line, and change the value from 10 to the shortest interval value chosen in Graphite configuration. In the previous tutorial, it was 20 seconds, so the line will look like this:

Interval 20

The matching of these two values is fundamental. If they do not match some data could be lost!

Now it’s time to choose the services that 

will gather information about. To do this, collectd uses plugins. Starting on line 88, there is a very long list. Most of the lines are commented, but you’ll want to uncomment apache and write_graphite. After this, the following plugins will be enabled

LoadPlugin apache

LoadPlugin battery

LoadPlugin cpu

LoadPlugin df

LoadPlugin disk

LoadPlugin entropy

LoadPlugin interface

LoadPlugin irq

LoadPlugin load

LoadPlugin memory

LoadPlugin processes

LoadPlugin rrdtool

LoadPlugin swap

LoadPlugin users

LoadPlugin write_graphite

Further down in the file, there are plugins configuration blocks. Uncomment the Apache block and add an Instance sub-block; now the block will look like this:

<Plugin apache> 
#       <Instance "foo"> 
#               URL "http://localhost/server-status?auto" 
#               User "www-user" 
#               Password "secret" 
#               VerifyPeer false 
#               VerifyHost false 
#               CACert "/etc/ssl/ca.crt" 
#               Server "apache" 
#       </Instance> 
# 
#       <Instance "bar"> 
#               URL "http://some.domain.tld/status?auto" 
#               Host "some.domain.tld" 
#               Server "lighttpd" 
#       </Instance> 
 
        <Instance "Graphite"> 
                URL "http://mydomain/server-status?auto" 
                Server "apache" 
        </Instance> 
</Plugin>

Next, configure the 

block, which gives information about space occupied on the disk. For example:

<Plugin df> 
        Device "/dev/sda3" 
#       Device "192.168.0.2:/mnt/nfs" 
        MountPoint "/" 
        # FSType "btrfs" 
 
        # ignore rootfs; else, the root file-system would appear twice, causing 
        # one of the updates to fail and spam the log 
        # FSType rootfs 
        # ignore the usual virtual / temporary file-systems 
        # FSType sysfs 
        # FSType proc 
        # FSType devtmpfs 
        # FSType devpts 
        # FSType tmpfs 
        # FSType fusectl 
        # FSType cgroup 
        # IgnoreSelected true 
 
#       ReportByDevice false 
#       ReportInodes false 
 
#       ValuesAbsolute true 
#       ValuesPercentage false 
</Plugin>

Finally, go to the 

section and uncomment it:

<Plugin write_graphite> 
        <Node "graphite"> 
                Host "localhost" 
                Port "2003" 
                Protocol "tcp" 
                LogSendErrors true 
                Prefix "collectd" 
                Postfix "collectd" 
                StoreRates true 
                AlwaysAppendDS false 
                EscapeCharacter "_" 
        </Node> 
</Plugin>

Note: remember that “2003” is the port number on which Carbon will listen.

Save and exit.

Configure Apache

Configure the ‘Apache virtual hosts’ file enabled for Graphite:

# $EDITOR /etc/apache2/sites-available/apache2-graphite.conf

Below the content block, add the following lines:

Alias /content/ /usr/share/graphite-web/static/
    <Location "/content/">
        SetHandler None
    </Location>

    <Location "/server-stats">
        SetHandler server-status
        Require all granted
    </Location>

    ErrorLog ${APACHE_LOG_DIR}/graphite-web_error.log

This means that Apache will serve statistics on the /server-stats page.

Save, exit and reload Apache:

# systemctl reload apache2

Check that everything is well configured by accessing http://localhost/server-stats with a web browser.

Configure storage schemas

To configure the storage schemas correctly, edit the following configuration file:

# $EDITOR/etc/carbon/storage-schemas.conf

In that file, above the default block, insert a new block. For example:

[collectd]
pattern = ^collectd.*
retentions = 20s:1d, 5m:7d, 10m:1y

Save, exit and restart the services:

# systemctl restart carbon-cache
# systemctl restart collectd

Conclusion

Now

should be up and running, recording all the server activities and flushing these records to

.
It’s possible to configure

with additional plugins, or modify the configuration files to accomplish new/other tasks.

In the last part of this series of tutorials, we will talk about installing and configuring

, a program for caching data collected by

. Stay tuned!