telemetry-essentials

Telemetry Essentials

Use this skill before writing ANY telemetry, logging, or metrics code.

RULES — Follow these with no exceptions

1. Use structured logging (Logger.info("action", key: value)) — never string interpolation in log messages 2. Attach telemetry handlers in Application.start/2 — not in modules that may restart, and never in GenServer init 3. Use Ecto.Repo telemetry events for query monitoring — Ecto already emits events; don't manually instrument queries 4. Use Phoenix.LiveDashboard in dev/staging — free observability with zero code 5. Tag telemetry events with metadata (user_id, request_id) — without correlation IDs, traces are useless 6. Never log at :debug level in production — it includes query parameters and PII; use :info level instead

Structured Logging

❌ Bad — unsearchable:

Logger.info("User #{user.id} created order #{order.id} for $#{order.total}")

✅ Good — searchable, parseable:

Logger.info("Order created", user_id: user.id, order_id: order.id, total: order.total)

Logger Metadata

# In a Plug
defmodule MyAppWeb.Plugs.RequestMetadata do
  import Plug.Conn

  def call(conn, _opts) do
    Logger.metadata(
      request_id: conn.assigns[:request_id] || Ecto.UUID.generate(),
      remote_ip: to_string(:inet.ntoa(conn.remote_ip))
    )
    conn
  end
end

:telemetry Basics

# Emit a custom event
:telemetry.execute(
  [:my_app, :orders, :created],     # event name
  %{count: 1, total_cents: 4999},    # measurements
  %{user_id: user.id, source: :web}  # metadata
)

Attaching Handlers

See assets/telemetry_handler_snippets.ex for a copy-paste template that attaches handlers in start/2, detaches them in stop/1, and implements handle_event/4.

✅ Good — in application.ex:

defmodule MyApp.Application do
  use Application

  @impl true
  def start(_type, _args) do
    MyApp.Telemetry.attach_handlers()

    children = [MyApp.Repo, MyAppWeb.Endpoint]
    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

defmodule MyApp.Telemetry do
  require Logger

  def attach_handlers do
    :telemetry.attach_many("my-app-handlers", [
      [:my_app, :orders, :created],
      [:my_app, :payments, :processed]
    ], &handle_event/4, nil)
  end

  def handle_event([:my_app, :orders, :created], measurements, metadata, _config) do
    Logger.info("Order created",
      total_cents: measurements.total_cents,
      user_id: metadata.user_id
    )
  end
end

Verification

After attaching handlers, confirm everything is wired up in iex:

# In iex -S mix
:telemetry.execute([:my_app, :orders, :created], %{count: 1, total_cents: 4999}, %{user_id: 42, source: :web})
# Expected: Logger output like — [info] Order created total_cents=1 user_id=42

If no log line appears, verify attach_handlers/0 was called before the event and that the event name matches exactly.

Telemetry Spans

def process_order(order) do
  :telemetry.span([:my_app, :orders, :process], %{order_id: order.id}, fn ->
    result = do_process(order)
    {result, %{order_id: order.id, status: :completed}}
  end)
end

Monitoring Slow Queries

def handle_slow_query(_event, measurements, metadata, %{threshold_ms: threshold}) do
  duration_ms = System.convert_time_unit(measurements.total_time, :native, :millisecond)

  if duration_ms > threshold do
    Logger.warning("Slow query",
      duration_ms: duration_ms,
      source: metadata.source,
      query: metadata.query
    )
  end
end

LiveDashboard Setup

# router.ex
import Phoenix.LiveDashboard.Router

scope "/" do
  pipe_through :browser

  live_dashboard "/dashboard",
    metrics: MyAppWeb.Telemetry,
    ecto_repos: [MyApp.Repo]
end

Deeper topics: For performance profiling and benchmarking, see the benchee-profiling skill. For exporting metrics to external tools (Prometheus, Datadog) and production deployment considerations, see the deployment-gotchas skill. For low-level metric aggregation and reporter configuration, consult the Telemetry.Metrics and TelemetryMetricsPrometheus library docs.

Common Pitfalls

❌ Don't	✅ Do
Interpolate values into the log message string	Pass structured metadata: Logger.info("event", key: value)
Attach handlers inside a module init / GenServer that restarts	Attach once in Application.start/2, detach in stop/1
Run heavy work inside handle_event/4 (blocks the caller)	Keep handlers fast; offload slow work to a process/queue
Manually instrument every Ecto query	Consume Ecto's built-in [:my_app, :repo, :query] events
Emit events without correlation metadata	Tag events with user_id / request_id for traceable logs
Leave :debug logging on in production	Use :info; :debug leaks query params and PII

---

Integration

Predecessor	This Skill	Successor
otp-essentials	telemetry-essentials	deployment-gotchas
otp-essentials	telemetry-essentials	benchee-profiling
security-essentials	telemetry-essentials	deployment-gotchas