Skip to main content

Collecting and graphing metrics via Apache Camel, Statsd, and Graphite

If you're curious about Apache Camel's metrics reporting via Dropwizard, this guide will help you get Graphite running in a Docker container. Then you will run an instrumented Camel application and view its metrics via Graphite's browser UI.

(1) Install Docker

There are many ways to do this depending on your platform. Google it.

(2) Install and run the Docker container for Statsd and Graphite

Refer to https://github.com/hopsoft/docker-graphite-statsd

The instructions infer that you can run the container without specifying port mappings. This is plain wrong. So use the following command to start the container:

docker run -d --name graphite --restart=always -p 80:80 -p 8125:8125/udp hopsoft/graphite-statsd
It is not necessary to enable Graphite's UDP listener. Statsd receives metrics via UDP and forwards them to Graphite over TCP.
(3) On Windows and OSX, determine the IP address of the graphite Docker container

On Linux, you can use localhost.

On Windows and OSX, there are several ways to determine the IP address of the Docker container. The easiest way is to run Kitematic, click on the container named graphite, click on Settings, and then Ports. This document assumes its IP address is 192.168.99.100.

(4) Install Apache Camel

I used Camel version 2.17.2, but the latest version will likely change by the time you read this. Refer to http://camel.apache.org/download.html

(5) Modify Camel's spring-boot-metrics example

Because the Docker container uses Statsd for UDP message collection, we will need to alter Camel's metrics example to send messages to Statsd. We will also add an additional counter to demonstrate a custom metric collected by the route named fastRoute.

(5a) Go to the following directory under the Camel installation

examples/camel-example-spring-boot-metrics

(5b) Overwrite pom.xml

You may need change package versions to the latest (highlighted in yellow).

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

  <modelVersion>4.0.0</modelVersion>

  <parent>
    <groupId>org.apache.camel</groupId>
    <artifactId>examples</artifactId>
    <version>2.17.2</version>
  </parent>

  <artifactId>camel-example-spring-boot-metrics</artifactId>
  <name>Camel :: Example :: Spring Boot Metrics</name>
  <description>An example showing how to work with Camel and Spring Boot and report metrics to statsd</description>

  <properties>
    <spring.boot-version>${spring-boot-version}</spring.boot-version>
  </properties>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-dependencies</artifactId>
        <version>${spring.boot-version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <dependencies>
    <dependency>
      <groupId>org.apache.camel</groupId>
      <artifactId>camel-spring-boot</artifactId>
    </dependency>

    <dependency>
      <groupId>org.apache.camel</groupId>
      <artifactId>camel-metrics</artifactId>
    </dependency>

    <!-- web -->
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <!-- operations -->
    <dependency>
      <groupId>org.jolokia</groupId>
      <artifactId>jolokia-core</artifactId>
    </dependency>

    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>

    <!-- metrics -->
    <dependency>
      <groupId>io.dropwizard.metrics</groupId>
      <artifactId>metrics-core</artifactId>
      <version>3.1.2</version>
    </dependency>

    <dependency>
      <groupId>com.basistech</groupId>
      <artifactId>metrics-statsd</artifactId>
      <version>3.0.0</version>
    </dependency>
  </dependencies>

  <build>
    <!-- we do not want version in the JAR name -->
    <finalName>${project.artifactId}</finalName>

    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <version>${spring.boot-version}</version>
        <configuration>
          <mainClass>org.apache.camel.example.springboot.metrics.Application</mainClass>
        </configuration>
        <executions>
          <execution>
            <goals>
              <goal>repackage</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

(5c) Overwrite src/main/java/org/apache/camel/springboot/metrics/Application.java

Specify the IP address of the Docker container (highlighted in yellow).

package org.apache.camel.example.springboot.metrics;

import java.net.InetSocketAddress;
import java.net.InetAddress;
import java.util.concurrent.TimeUnit;

import com.codahale.metrics.MetricFilter;
import com.codahale.metrics.MetricRegistry;
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.metrics.routepolicy.MetricsRoutePolicyFactory;
import org.apache.camel.spring.boot.CamelContextConfiguration;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import com.basistech.metrics.reporting.Statsd;
import com.basistech.metrics.reporting.StatsdReporter;

/**
 * A simple Spring Boot application, with a couple of timed camel routes
 * configured with camel-metrics. Reports metrics to Statsd via
 * dropwizard-metrics Statsd sender. Has standard spring-actuator endpoints
 * such as /beans, /autoconfig, /metrics
 */
@SpringBootApplication
public class Application {

    private static final Logger LOG = LoggerFactory.getLogger(Application.class);

    @Autowired
    private MetricRegistry metricRegistry;

    /**
     * @param args no command line args required
     */
    public static void main(String[] args) {
        LOG.info(" *** Starting Camel Metrics Example Application ***");
        SpringApplication.run(Application.class, args);
    }

    /**
     * Create reporter bean and tell Spring to call stop() when shutting down.
     * 
     * @return StatsdReporter
     */
    @Bean(destroyMethod = "stop")
    public 
    StatsdReporter statsdReporter() throws Exception {
        final Statsd statsd = new Statsd("192.168.99.100", 8125);

        final StatsdReporter reporter =
            StatsdReporter
            .forRegistry(metricRegistry)
            .convertRatesTo(TimeUnit.SECONDS)
            .convertDurationsTo(TimeUnit.MILLISECONDS)
            .filter(MetricFilter.ALL)
            .build(statsd);

        reporter.start(5, TimeUnit.SECONDS);

        return reporter;
    }

    /**
     * @return timed route that logs output every 6 seconds
     */
    @Bean
    public RouteBuilder slowRoute() {
        return new RouteBuilder() {
            @Override
            public void configure() throws Exception {
                from("timer://foo?period=6000").routeId("slow-route").setBody().constant("Slow hello world!").log("${body}");
            }
        };
    }

    /**
     * @return timed route that logs output every 2 seconds
     */
    @Bean
    public RouteBuilder fastRoute() {
        return new RouteBuilder() {
            @Override
            public void configure() throws Exception {
                from("timer://foo?period=2000").routeId("fast-route").setBody().constant("Fast hello world!").log("${body}")
                 .to("metrics:counter:counters.camel-example-spring-boot-metrics.fast-route.executions");
            }
        };
    }

    @Bean
    CamelContextConfiguration contextConfiguration() {
        return new CamelContextConfiguration() {
            @Override
            public void beforeApplicationStart(CamelContext context) {
                LOG.info("Configuring Camel metrics on all routes");
                MetricsRoutePolicyFactory fac = new MetricsRoutePolicyFactory();
                fac.setMetricsRegistry(metricRegistry);
                context.addRoutePolicyFactory(fac);
            }

            @Override
            public void afterApplicationStart(CamelContext camelContext) {
                // noop
            }
        };
    }
}

(6) Install JDK8

If you need help with that, you probably shouldn't be reading this.

(7) Install Maven 3.3 or later

Refer to https://maven.apache.org/download.cgi

(8) Run the example

mvn spring-boot:run

(9) Finally, view the metrics

Go to http://192.168.99.100/dashboard. Use the IP address of the graphite Docker container.

View the metrics under stats.gauges, stats.timers, and stats.counters

Comments

Popular posts from this blog

Running 2560x1080 on LG 29UM57-P on an old MacBook Pro OSX El Capitan

Yes. A mid-2011 MacBook Pro running OSX El Capitan can drive a 2560x1080 monitor via HDMI. I was not able to get 60 Hz working. I am settling with 53 Hz. I tried many settings before finding something that works. The refresh rate seems fine to me. Maybe the text could be clearer but for a $200 monitor running this resolution, I'll take it. Apple MacBook 2015 retina resolution is 2560 x 1800. Consider buying a better monitor that may literally give you fewer headaches. Install  SwitchResX Follow the instructions for disabling SIP After rebooting, run SwitchResX via System Preferences Select your LG monitor Click Custom Resolutions Click the + icon  Enter these settings  ( source ) Exit SwitchResX and save Reboot Run SwitchResX via System Preferences Select your LG monitor Click Current Resolutions Select the 2560x1080, 53 Hz setting Enable SIP If you discover better settings, please leave a comment.

MySQL: This stored procedure inserts 'empty' rows into a specified table

This stored procedure accepts two arguments: a table name and a row count. It inserts the specified number of rows into the specified table. Of course, all columns in the table must either have default values or allow NULL. DROP PROCEDURE InsertDimension; DELIMITER $$ CREATE PROCEDURE InsertDimension(IN TableName VARCHAR(50), IN NumRows INT) BEGIN DECLARE i INT; SET i = 1; SET @sql_text = concat('INSERT INTO ', TableName, ' VALUES ()' ); PREPARE stmt FROM @sql_text; START TRANSACTION; WHILE i <= NumRows DO EXECUTE stmt; SET i = i + 1; END WHILE; COMMIT; DEALLOCATE PREPARE stmt; END$$ DELIMITER ;

Terris’ Easy Spicy Vegan Black Beans

Black beans are full of protein and smoky flavor. This easy vegan spicy black bean dish is delish solo and in Mexican classics. Serves: 6 Prep: 10 minutes (if you use canned beans) Cook: 15 minutes Ingredients 2 Tbsp olive oil 1 small yellow onion, diced  1 tsp cumin 2 cloves garlic, pressed 1 14.5oz can diced tomatoes or fresh alternative. Tomatoes are the key ingredient. Do not decrease the amount. 1 or 2  🌶🌶  peppers with 1 tsp sauce from a can of chipotle peppers in adobo sauce (such as from La Costeña). These peppers supply the heat. Try with one pepper first and upgrade if you dare (seriously though, they are pretty mild). Do not use the whole can! 2 15oz cans black beans without spices, drained (or start from dry - see you tomorrow!) 1 tsp black pepper 1 tsp salt, or to taste Instructions Heat the oil, onions, cumin, and garlic over low heat for 10 minutes  While heating, purée tomatoes, peppers, and adobo sauce in a blender or food processor Add tomato purée, beans, pepper, a