Insider.Collector.Otel (Insider v0.1.0)

Copy Markdown View Source

Bounded capture of finished OpenTelemetry spans as otel.span events.

It runs only in a collector session that names the :otel component, and only where the OpenTelemetry SDK is present. It never adds, replaces, or reconfigures a span processor, exporter, tracer, or sampler: it observes the SDK's span end (Insider.Collector.Otel.Tracing), so the node's own processors and exporters, including traces_exporter: :none, behave exactly as before, and removing the capture leaves nothing behind.

Each finished span reaches this process as one trace message. The record-time gate decides first (origin and minimum duration); the span is then built into an event (Insider.Collector.Otel.Span) and offered to the Recorder.

The capture is throttled by count, not admission-controlled: a trace message is in this process's queue before anything here can refuse it. A valve process checks the queue every millisecond and pauses the trace while more than :max_waiting messages wait, resuming at a quarter of that; spans that end while paused are not observed. pauses counts pause starts and paused says whether one is ongoing, so every read that overlaps a pause can report a loss of unknown size. A span received while more than :max_waiting wait is counted as shed without being built, so memory is released as soon as a span is received. Each span's size is bounded when its event is built (Insider.Collector.Otel.Span). Every received span is counted once: recorded, recorder_dropped, invalid, shed, or filtered by the gate.

ingest/1 adds spans from outside the node's SDK, such as a frontend's spans, through the same build, profile, and accounting. With capture: false the collector observes nothing and records only those; it then needs neither the SDK nor OTP 28.

Summary

Functions

Returns a specification to start this module under a supervisor.

Processes the spans that ended before the call, for at most one second, then returns the status; pending counts the span messages still waiting. The call waits behind span messages already queued, so a collector with more queued than it can process within the two-second call bound returns {:error, :unavailable} (a live session then reports otel_accounting_unavailable). With the default :max_waiting, spans received above it are shed cheaply, which keeps that wait short.

Records spans built outside this node's SDK (span maps, see Insider.Collector.Otel.Span) in the running session, under its profile, gate, and accounting. At most ingest_max/0 spans per call. Returns the "accepted", "rejected" (invalid or refused by the Recorder), "filtered", "shed", and "duplicate" counts, or {:error, :not_running} when no session runs the otel component.

ingest/1 with a deadline (monotonic milliseconds). A call the collector takes after its deadline is refused before any span or counter is touched; the caller waits up to ingest_reply_ms/0 past the deadline.

The most spans one ingest/1 call takes.

How long a taken ingest/2 call may still take to record its spans.

The services and routes each recorded trace's spans named, newest 4096 traces: %{trace_id => %{services: MapSet, routes: MapSet}}. Only spans this collector recorded count, from the SDK and from ingest/1 alike, each credential-masked as Insider.Redaction.Credentials masks event text; a trace keeps at most eight of each. A live view scoped to services or routes resolves its traces from it (Insider.Live.View).

Functions

child_spec(init_arg)

Returns a specification to start this module under a supervisor.

See Supervisor.

flush(server)

@spec flush(GenServer.server()) :: {:ok, map()} | {:error, :unavailable}

Processes the spans that ended before the call, for at most one second, then returns the status; pending counts the span messages still waiting. The call waits behind span messages already queued, so a collector with more queued than it can process within the two-second call bound returns {:error, :unavailable} (a live session then reports otel_accounting_unavailable). With the default :max_waiting, spans received above it are shed cheaply, which keeps that wait short.

ingest(spans)

@spec ingest([map()]) ::
  {:ok, map()} | {:error, :not_running | :invalid_options | :timeout}

Records spans built outside this node's SDK (span maps, see Insider.Collector.Otel.Span) in the running session, under its profile, gate, and accounting. At most ingest_max/0 spans per call. Returns the "accepted", "rejected" (invalid or refused by the Recorder), "filtered", "shed", and "duplicate" counts, or {:error, :not_running} when no session runs the otel component.

Ingest is idempotent per {trace_id, span_id}: a span this collector already recorded from an ingest is counted as "duplicate" and not recorded again (within the last :ingest_seen_max recorded ids, default 16,384; older ids are forgotten and counted as seen_evicted).

ingest(spans, deadline)

@spec ingest([map()], integer()) ::
  {:ok, map()} | {:error, :not_running | :invalid_options | :timeout}

ingest/1 with a deadline (monotonic milliseconds). A call the collector takes after its deadline is refused before any span or counter is touched; the caller waits up to ingest_reply_ms/0 past the deadline.

{:error, :timeout} means outcome unknown: a call taken before the deadline may still be recording when the caller stops waiting, so some, all, or none of its spans may be recorded. Retrying the same spans does not duplicate a span while its id is among the last :ingest_seen_max ingested ids this collector recorded; a forgotten id (counted as seen_evicted) is recorded again.

ingest_max()

@spec ingest_max() :: pos_integer()

The most spans one ingest/1 call takes.

ingest_reply_ms()

@spec ingest_reply_ms() :: pos_integer()

How long a taken ingest/2 call may still take to record its spans.

start_link(recorder, options)

@spec start_link(Insider.Recorder.Handle.t(), keyword()) ::
  GenServer.on_start() | {:error, atom()}

status(server)

@spec status(GenServer.server()) :: {:ok, map()} | {:error, :unavailable}

stop(server)

@spec stop(GenServer.server()) :: {:ok, map()} | {:error, :unavailable}

trace_index(server)

@spec trace_index(GenServer.server()) :: {:ok, map()} | {:error, :unavailable}

The services and routes each recorded trace's spans named, newest 4096 traces: %{trace_id => %{services: MapSet, routes: MapSet}}. Only spans this collector recorded count, from the SDK and from ingest/1 alike, each credential-masked as Insider.Redaction.Credentials masks event text; a trace keeps at most eight of each. A live view scoped to services or routes resolves its traces from it (Insider.Live.View).