Sending GraphOS Router Traces and Metrics to APM Tools Using OpenTelemetry

Application performance monitoring (APM) tools are critical for many organizations, and having direct insight into how your graph is performing is an important piece of overall observability into your systems' stability and performance. The GraphOS Router and subgraphs aren't any different. While you may currently instrument your existing subgraphs with your APM's tooling, you may need to connect router as well to get a clear view of your graph's health.

Thankfully, there's an answer on connecting the two pieces of software together: OpenTelemetry. We've previously discussed utilizing OpenTelemetry previously to connect to Prometheus, and much of the same will apply to this use-case as well.

There are two ways to connect the router to your APM using OpenTelemetry:

• Using an OpenTelemetry Collector
• Connecting directly

For our examples, we'll be connecting to New Relic, however this would apply to any APM that supports OpenTelemetry as an ingest option, such as Honeycomb or Dynatrace.

Using OpenTelemetry Collector (recommended)

We recommend using the OpenTelemetry Collector to send router metrics to your APM tool for the following reasons:

• The collector provides a centralized reporter when running multiple router instances.
• The collector enables you to processing and send metrics to multiple locations beyond your APM tool, such as a local Prometheus instance.

You'll need an instance of the OpenTelemetry Collector to export to your APM tool. A basic configuration requires both an OpenTelemetry Protocol (OTLP) receiver (for the router) and an exporter (to forward to your APM tool).

Additionally, we recommend adding the batch processor to enable batch requests to each listed exporter.

Here's an example collector configuration for New Relic:

1
receivers:
2
  otlp:
3
    protocols:
4
      grpc:
5
      http:
6
        cors:
7
          allowed_origins:
8
            - http://*
9
            - https://*
10

11
exporters:
12
  otlp:
13
    endpoint: ${OTEL_EXPORTER_OTLP_ENDPOINT}
14
    headers:
15
      api-key: ${NEW_RELIC_LICENSE_KEY}
16

17
processors:
18
  batch:
19

20
service:
21
  pipelines:
22
    traces:
23
      receivers: [otlp]
24
      exporters: [otlp]
25
      processors: [batch]
26
    metrics:
27
      receivers: [otlp]
28
      exporters: [otlp]
29
      processors: [batch]

The router then needs to connect directly to the collector instance. Here's an example router configuration for that:

1
cors:
2
  allow_any_origin: true
3
supergraph:
4
  listen: 0.0.0.0:4000
5
telemetry:
6
  metrics:
7
    otlp:
8
      endpoint: http://COLLECTOR_ADDRESS:4317
9
  tracing:
10
    trace_config:
11
      service_name: 'router'
12
      service_namespace: 'apollo'
13
    otlp:
14
      endpoint: http://COLLECTOR_ADDRESS:4317

For more information on configuration options, see the documentation on both capturing metrics and capturing traces using OpenTelemetry Collector.

Connecting directly

Instead of using the OpenTelemetry Collector, you can configure your GraphOS Router instances to send data directly to your APM tool. To do so, you modify the router's YAML configuration file to set the export to use a header (or in this case, metadata), along with the traces/metrics for passing the token.

Here's an example router configuration for connecting directly to New Relic:

1
cors:
2
  allow_any_origin: true
3
supergraph:
4
  listen: 0.0.0.0:4000
5
telemetry:
6
  metrics:
7
    otlp:
8
      endpoint: https://otlp.nr-data.net
9
      grpc:
10
        metadata:
11
          'api-key': ['${env.NEW_RELIC_LICENSE_KEY}']
12
  tracing:
13
    trace_config:
14
      service_name: 'router'
15
      service_namespace: 'apollo'
16
    otlp:
17
      endpoint: https://otlp.nr-data.net
18
      grpc:
19
        metadata:
20
          'api-key': ['${env.NEW_RELIC_LICENSE_KEY}']

This uses the NEW_RELIC_LICENSE_KEY environment variable for passing the key using the router's variable expansion feature to avoid including sensitive tokens in the configuration.