Monitor business metrics: boat data in Bleemeo

Published on 16 April 2017 by Lionel Porcheron

This year, and for the second time in a row, Bleemeo has supported a team of INP Toulouse (the engineering school of two of our founders) during the EDHEC Sailing Race. This year the race was in Arzon in French Britany.

During the race, we provided them a tracker composed of a Raspberry Pi and a GPS antenna. Then all along the race, it was reporting data in Bleemeo and we were displaying the boat data on a map.

This is an example of how to report business metrics in your monitoring system with Bleemeo.

The Tracker

As you can see on the picture below, our tracker is pretty simple and cost us less than 150€:

We just setup Ubuntu 16.04 for Raspberry Pi and install the agent (yes, we do support ARM architecture on agent!).

Our Custom Web Application

We have created a very basic Flask application which uses the Python GPS module and simply associate a URL to each metric.

#!/usr/bin/python
from flask import Flask, render_template
import gps


app = Flask(__name__)


def get_gps_report():
    session = gps.gps()
    session.stream(gps.WATCH_ENABLE)

    report = None
    while True:
        if not session.waiting(timeout=5):
            break
        data = session.next()
        if hasattr(data, 'lat'):
            report = data
            break
    session.close()
    return report


@app.route('/')
def home():
    return render_template('index.html')


@app.route('/latitude')
def latitude():
    report = get_gps_report()
    return str(report.lat)


@app.route('/longitude')
def longitude():
    report = get_gps_report()
    return str(report.lon)


@app.route('/speed')
def speed():
    report = get_gps_report()
    return str(report.speed)


@app.route('/heading')
def heading():
    report = get_gps_report()
    return str(report.track)


def main():
    app.run(threaded=True, port=8016)

Getting metrics in Bleemeo

In a previous article, we demonstrated how to use statsd for monitoring a Minecraft server. Here we used another capability of the agent: poll metrics by http.

We are followed the configuration detailed in the agent documentation and added a new configuration file for the agent /etc/bleemeo/agent.conf.d/99-gps.conf containing:

metric:
  pull:
    latitude:
        url: http://localhost:8016/latitude
        interval: 60
    longitude:
        url: http://localhost:8016/longitude
        interval: 60
    speed:
        url: http://localhost:8016/speed
        interval: 60
    heading:
        url: http://localhost:8016/heading
        interval: 60

We just restarted the agent, and voilà, new metrics are now uploaded to Bleemeo. All metrics were available for graphs and alerts could be configured. For example, we could add a custom threshold on the speed to know if the boat stopped or on position to ensure the boat is not going to an unexpected place :)

The Map

For the map, we are used Mapbox (based on OpenStreetMap) and JavaScript to display the boat position on it. We used a cron script to generate a static json file using Bleemeo API. The script is a bit long as it is translating some metrics unit.

# -*- encoding: utf8 -*-
"""
Use Bleemeo API to retrieve value (position, speed and heading) and
generate a values.json.
"""

import datetime
import json
import os

import requests
from six.moves import urllib_parse


def main():
    if ('BLEEMEO_USER' not in os.environ or
            'BLEEMEO_PASSWORD' not in os.environ):
        print('Missing BLEEMEO_USER and/or BLEEMEO_PASSWORD in environment')
        return
    if 'BLEEMEO_AGENT_UUID' not in os.environ:
        print('Mising BLEEMEO_AGENT_UUID in environment')
        return

    api_config = {
        'base_url': os.environ.get(
            'BLEEMEO_BASE_URL', 'https://api.bleemeo.com'
        ),
        'user': os.environ['BLEEMEO_USER'],
        'password': os.environ['BLEEMEO_PASSWORD'],
        'agent_uuid': os.environ['BLEEMEO_AGENT_UUID'],
    }
    latitude = get_metric('latitude', api_config)
    longitude = get_metric('longitude', api_config)
    speed = get_metric('speed', api_config)
    heading = get_metric('heading', api_config)

    # speed is in meter/second
    # 1 knot == 1.852 km/hour
    speed_kn = speed / 1000. * 3600. / 1.852

    if longitude < 0:
        longitude_side = 'O'
    else:
        longitude_side = 'E'
    longitude_minute = (abs(longitude) * 60) % 60
    longitude_second = (longitude_minute * 60) % 60
    longitude_text = '%d°%d\'%d" %s' % (
        abs(longitude), longitude_minute, longitude_second, longitude_side
    )

    if latitude < 0:
        latitude_side = 'S'
    else:
        latitude_side = 'N'
    latitude_minute = (abs(latitude) * 60) % 60
    latitude_second = (latitude_minute * 60) % 60
    latitude_text = '%d°%d\'%d" %s' % (
        abs(latitude), latitude_minute, latitude_second, latitude_side
    )

    result = {
        'lastUpdate': datetime.datetime.now().strftime('%Y-%m-%d %H:%M'),
        'latitude': latitude,
        'longitude': longitude,
        'latitude_text': latitude_text,
        'longitude_text': longitude_text,
        'speed': round(speed_kn, 1),
        'direction': int(heading),
        'temperature': 22,
        'pressure': 1013.5,
    }
    with open('values.json', 'w') as fileobj:
        json.dump([result], fileobj)


def get_metric(metric_name, api_config):
    metric_url = urllib_parse.urljoin(api_config['base_url'], '/v1/metric/')
    response = requests.get(
        metric_url,
        params={'agent': api_config['agent_uuid'], 'label': metric_name},
        auth=(api_config['user'], api_config['password']),
    )
    metric_uuid = response.json()['results'][0]['id']

    data_url = metric_url + metric_uuid + '/data/'
    start_date = datetime.datetime.utcnow() - datetime.timedelta(minutes=15)
    response = requests.get(
        data_url,
        params={'start': start_date.isoformat() + 'Z'},
        auth=(api_config['user'], api_config['password']),
    )
    values = response.json()['values']
    if len(values) == 0:
        raise KeyError('No recent value for %s' % metric_name)
    return values[-1]['value']


if __name__ == '__main__':
    main()

Conclusion

While you may not be interested in monitoring a boat, this shows you the simplicity of adding custom metrics in Bleemeo. You could have a dedicated view on your application for counting the number of registered users, the number of active users, today's revenue, etc. Then you can integrate in a few lines of code & configuration these metrics in your Bleemeo dashboards and improve your monitoring experience by bringing facing them to your technical metrics (load, CPU / Memory usage). It will be easier to understand and analyze issues on your platform, determine if a slow-down is due to a higher number of active users or to a bad release.

Start Monitoring your servers, containers, and applications in 30 seconds without credit card.Start your Free Trial