Monitor - Elastic APM

Tip

I think we need to install X-Pack to get the APM dashboards. (from elastic, Doubling Down on Open, X-Pack features - … dedicated APM UIs)

Server Configuration

To create a server acting as a monitor using Elastic APM, add the following to the top.sls file of your pillar:

base:
  '*':
    - global.users
  'elastic-*':
    - config.firewall
    - config.kibana
    - config.search

Tip

My monitor servers all start with a hostname of elastic-.

Re-start services (it might be a good idea to tail the log files first):

service elasticsearch restart
service kibana restart
service apm-server restart

Check the log files:

tail -f /var/log/syslog
tail -f /var/log/elasticsearch/elasticsearch.log
tail -f /var/log/apm-server/apm-server

Warning

If you re-start apm-server, the log file will be re-created, so you will need to re-start the tail.

Tips

To run the APM server in the foreground:

service apm-server stop
apm-server -e

To view the web interface of the monitor server on your workstation:

ssh -N -L 8888:localhost:5601 monitor.yourbiz.co.uk

Note

This command tunnels the HTTP service from port 5601 on the monitor server (Kibana) to your local workstation on port 8888.

Browse to: http://localhost:8888/ to view the dashboards.

Site Configuration

Add the following to requirements/base.txt:

elastic-apm==

Tip

See Requirements for the current version.

In settings/base.py:

MIDDLEWARE = (
    'elasticapm.contrib.django.middleware.TracingMiddleware',

THIRD_PARTY_APPS = (
    'elasticapm.contrib.django',

Note

Make sure that TracingMiddleware is the first MIDDLEWARE in the list.

In settings/production.py:

DOMAIN = get_env_variable('DOMAIN')
DATABASE = DOMAIN.replace('.', '_').replace('-', '_')

ELASTIC_APM = {
    'DISABLE_SEND': get_env_variable_bool('MONITOR_DISABLE_SEND'),
    'SERVER_URL': get_env_variable('MONITOR_SERVER_URL'),
    'SERVICE_NAME': DATABASE,
}

Note

Refer to your company Checklist and replace monitor.yourbiz.co.uk with the name of your server.

Note

DOMAIN and DATABASE should already exist in production.py.

Deploy

Find the IP address of your server. Add it to config/firewall.sls in your pillar e.g:

firewall:
  monitor:
    # server
    - 139.162.245.14
    # yourbiz
    - 91.85.167.37

Tip

Please add a comment to show which server / workstation has the IP address.

Copy the pillar to your Salt master and run a highstate to update the firewall on your monitor server e.g:

salt 'elastic-monitor' state.highstate --state-verbose=False

Check you have included config/monitor.sls in your pillar e.g:

monitor:
  server_url: http://monitor.yourbiz.co.uk:8200

To check your configuration, run:

python manage.py elasticapm check

To send a test exception using the current settings, run:

python manage.py elasticapm test

To disable monitoring for your test (or temporary) site:

# sites/my.sls
sites:
  hatherleigh_info:
    monitor: False

Old Notes