Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

As of Agent 6.0.0, the Trace Agent is enabled by default. If it has been turned off, you can re-enable it in the gcr.io/datadoghq/agent container by passing DD_APM_ENABLED=true as an environment variable.

The CLI commands on this page are for the Docker runtime. Replace docker with nerdctl for the containerd runtime, or podman for the Podman runtime.

If you are collecting traces from a containerized app (your Agent and app running in separate containers), as an alternative to the following instructions, you can automatically inject the tracing library into your application. Read Injecting Libraries for instructions.

Tracing from the host

Tracing is available on port 8126/tcp from your host only by adding the option -p 127.0.0.1:8126:8126/tcp to the docker run command.

To make it available from any host, use -p 8126:8126/tcp instead.

For example, the following command allows the Agent to receive traces from your host only:

docker run -d --cgroupns host \
              --pid host \
              -v /var/run/docker.sock:/var/run/docker.sock:ro \
              -v /proc/:/host/proc/:ro \
              -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
              -p 127.0.0.1:8126:8126/tcp \
              -e DD_API_KEY=<DATADOG_API_KEY> \
              -e DD_APM_ENABLED=true \
              -e DD_SITE=<DATADOG_SITE> \
              gcr.io/datadoghq/agent:latest

Where your <DATADOG_SITE> is (defaults to datadoghq.com).

docker run -d -p 127.0.0.1:8126:8126/tcp \
              -e DD_API_KEY=<DATADOG_API_KEY> \
              -e DD_APM_ENABLED=true \
              -e DD_SITE=<DATADOG_SITE> \
              gcr.io/datadoghq/agent:latest

Where your <DATADOG_SITE> is (defaults to datadoghq.com).

Docker APM Agent environment variables

Use the following environment variables to configure tracing for the Docker Agent. See the sample config_template.yaml file for more details.

DD_API_KEY
required - string
Your Datadog API key.
DD_SITE
optional - string
Your Datadog site. Set this to .
Default: datadoghq.com
DD_APM_ENABLED
optional - Boolean - default: true
When set to true (default), the Datadog Agent accepts traces and trace metrics.
DD_APM_RECEIVER_PORT
optional - integer - default: 8126
Sets the port on which the Datadog Agent’s trace receiver listens. Set to 0 to disable the HTTP receiver.
DD_APM_RECEIVER_SOCKET
optional - string
To collect your traces through UNIX Domain Sockets, provide the path to the UNIX socket. If set, this takes priority over hostname and port configuration, and must point to a valid socket file.
DD_APM_NON_LOCAL_TRAFFIC
optional - Boolean - default: false
When set to true, the Datadog Agent listens to non-local traffic. If you are tracing from other containers, set this environment variable to true.
DD_APM_DD_URL
optional - string
To use a proxy for APM, provide the endpoint and port as <ENDPOINT>:<PORT>. The proxy must be able to handle TCP connections.
DD_APM_CONNECTION_LIMIT
required - integer - default: 2000
Sets the maximum APM connections for a 30 second time window. See Agent Rate Limits for more details.
DD_APM_IGNORE_RESOURCES
optional - [string]
Provides an exclusion list of resources for the Datadog Agent to ignore. If a trace’s resource name matches one or more of the regular expressions on this list, the trace is not sent to Datadog.
Example: "GET /ignore-me","(GET\|POST) and-also-me".
DD_APM_FILTER_TAGS_REQUIRE
optional - object
Defines rules for tag-based trace filtering. To be sent to Datadog, traces must have these tags. See Ignoring Unwanted Resources in APM.
DD_APM_FILTER_TAGS_REGEX_REQUIRE
optional - object
Supported in Agent 7.49+. Defines rules for tag-based trace filtering with regular expressions. To be sent to Datadog, traces must have tags that match these regex patterns.
DD_APM_FILTER_TAGS_REJECT
optional - object
Defines rules for tag-based trace filtering. If a trace has these tags, it is not sent to Datadog. See Ignoring Unwanted Resources in APM for more details.
DD_APM_FILTER_TAGS_REGEX_REJECT
optional - object
Supported in Agent 7.49+. Defines rules for tag-based trace filtering with regular expressions. If a trace has tags that match these regex patterns, the trace is not sent to Datadog.
DD_APM_REPLACE_TAGS
optional - [object]
Defines a set of rules to replace or remove tags that contain potentially sensitive information.
DD_HOSTNAME
optional - string - default: automatically detected
Sets the hostname to use for metrics if automatic hostname detection fails, or when running the Datadog Cluster Agent.
DD_DOGSTATSD_PORT
optional - integer - default: 8125
Sets the DogStatsD port.
DD_PROXY_HTTPS
optional - string
To use a proxy to connect to the internet, provide the URL.
DD_BIND_HOST
optional - string - default: localhost
Sets the host to listen on for DogStatsD and traces.
DD_LOG_LEVEL
optional - string - default: info
Sets the minimum logging level. Valid options: trace, debug, info, warn, error, critical, and off.

Tracing from other containers

As with DogStatsD, traces can be submitted to the Agent from other containers either using Docker networks or with the Docker host IP.

Docker network

As a first step, create a user-defined bridge network:

docker network create <NETWORK_NAME>

The CLI commands on this page are for the Docker runtime. Replace docker with nerdctl for the containerd runtime, or podman for the Podman runtime.

Then start the Agent and the application container, connected to the network previously created:

# Datadog Agent
docker run -d --name datadog-agent \
              --network <NETWORK_NAME> \
              --cgroupns host \
              --pid host \
              -v /var/run/docker.sock:/var/run/docker.sock:ro \
              -v /proc/:/host/proc/:ro \
              -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
              -e DD_API_KEY=<DATADOG_API_KEY> \
              -e DD_APM_ENABLED=true \
              -e DD_SITE=<DATADOG_SITE> \
              -e DD_APM_NON_LOCAL_TRAFFIC=true \
              gcr.io/datadoghq/agent:latest
# Application
docker run -d --name app \
              --network <NETWORK_NAME> \
              -e DD_AGENT_HOST=datadog-agent \
              company/app:latest

Where your <DATADOG_SITE> is (defaults to datadoghq.com).

# Datadog Agent
docker run -d --name datadog-agent \
              --cgroupns host \
              --pid host \
              --network "<NETWORK_NAME>" \
              -e DD_API_KEY=<DATADOG_API_KEY> \
              -e DD_APM_ENABLED=true \
              -e DD_SITE=<DATADOG_SITE> \
              -e DD_APM_NON_LOCAL_TRAFFIC=true \
              gcr.io/datadoghq/agent:latest
# Application
docker run -d --name app \
              --network "<NETWORK_NAME>" \
              -e DD_AGENT_HOST=datadog-agent \
              company/app:latest

Where your <DATADOG_SITE> is (defaults to datadoghq.com).

This exposes the hostname datadog-agent in your app container. If you’re using docker-compose, <NETWORK_NAME> parameters are the ones defined under the networks section of your docker-compose.yml.

Your application tracers must be configured to submit traces to this address. Set environment variables with the DD_AGENT_HOST as the Agent container name, and DD_TRACE_AGENT_PORT as the Agent Trace port in your application containers. The example above uses host datadog-agent and port 8126 (the default value so you don’t have to set it).

Alternately, see the examples below to set the Agent host manually in each supported language:

Either update the Java Agent configuration with environment variables:

DD_AGENT_HOST=datadog-agent \
DD_TRACE_AGENT_PORT=8126 \
java -javaagent:/path/to/the/dd-java-agent.jar -jar /your/app.jar

or through system properties:

java -javaagent:/path/to/the/dd-java-agent.jar \
     -Ddd.agent.host=datadog-agent \
     -Ddd.agent.port=8126 \
     -jar /your/app.jar
from ddtrace import tracer

tracer.configure(
    hostname='datadog-agent',
    port=8126,
)
Datadog.configure do |c|
  c.agent.host = 'datadog-agent'
  c.agent.port = 8126
end
package main

import "gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"

func main() {
    tracer.Start(tracer.WithAgentAddr("datadog-agent:8126"))
    defer tracer.Stop()
}
const tracer = require('dd-trace').init({
    hostname: 'datadog-agent',
    port: 8126
});

Set the environment variables before running your instrumented app:

# Environment variables
export CORECLR_ENABLE_PROFILING=1
export CORECLR_PROFILER={846F5F1C-F9AE-4B07-969E-05C26BC060D8}
export CORECLR_PROFILER_PATH=<SYSTEM_DEPENDENT_PATH>
export DD_DOTNET_TRACER_HOME=/opt/datadog

# For containers
export DD_AGENT_HOST=datadog-agent
export DD_TRACE_AGENT_PORT=8126

# Start your application
dotnet example.dll

The value for the CORECLR_PROFILER_PATH environment variable varies based on the system where the application is running:

Operating System and Process ArchitectureCORECLR_PROFILER_PATH Value
Alpine Linux x64<APP_DIRECTORY>/datadog/linux-musl-x64/Datadog.Trace.ClrProfiler.Native.so
Linux x64<APP_DIRECTORY>/datadog/linux-x64/Datadog.Trace.ClrProfiler.Native.so
Linux ARM64<APP_DIRECTORY>/datadog/linux-arm64/Datadog.Trace.ClrProfiler.Native.so
Windows x64<APP_DIRECTORY>\datadog\win-x64\Datadog.Trace.ClrProfiler.Native.dll
Windows x86<APP_DIRECTORY>\datadog\win-x86\Datadog.Trace.ClrProfiler.Native.dll

In the table above, <APP_DIRECTORY> refers to the directory containing the application’s .dll files.

Docker host IP

Agent container port 8126 should be linked to the host directly. Configure your application tracer to report to the default route of this container (determine this using the ip route command).

The following is an example for the Python Tracer, assuming 172.17.0.1 is the default route:

from ddtrace import tracer

tracer.configure(hostname='172.17.0.1', port=8126)

Unix Domain Socket (UDS)

To submit traces via socket, the socket should be mounted to the Agent container and your application container.

# Datadog Agent
docker run -d --name datadog-agent \
              --network <NETWORK_NAME> \
              --cgroupns host \
              --pid host \
              -v /var/run/docker.sock:/var/run/docker.sock:ro \
              -v /proc/:/host/proc/:ro \
              -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
              -v /var/run/datadog/:/var/run/datadog/ \
              -e DD_API_KEY=<DATADOG_API_KEY> \
              -e DD_APM_ENABLED=true \
              -e DD_SITE=<DATADOG_SITE> \
              -e DD_APM_NON_LOCAL_TRAFFIC=true \
              -e DD_APM_RECEIVER_SOCKET=/var/run/datadog/apm.socket \
              gcr.io/datadoghq/agent:latest
# Application
docker run -d --name app \
              --network <NETWORK_NAME> \
              -v /var/run/datadog/:/var/run/datadog/ \
              -e DD_TRACE_AGENT_URL=unix:///var/run/datadog/apm.socket \
              company/app:latest

Refer to the language-specific APM instrumentation docs for tracer settings.

Further Reading