# `Insider.Live`
[🔗](https://github.com/hiivemarkets/insider/blob/v0.1.0/lib/insider/live.ex#L1)

Transport-agnostic live observation of the node it runs in.

One live collector session records bounded Logger, Telemetry, and Poller
evidence. A supervised owner under `Insider.Supervisor`, never the caller,
owns the session, so short-lived transport processes such as a Tidewave
`project_eval` or an `:erpc` call from an observer node can start, read, and
stop it across separate calls. A renewable lease stops an abandoned session
through the normal collector cleanup path.

Every argument and return value is a distribution-safe plain term:
string-keyed maps, binaries, integers, booleans, `nil`, and lists. The only
process identifier the API accepts is a subscriber pid; none is returned.
Errors are `{:error, map}` with a closed `"code"` vocabulary documented in
[Error codes](docs/live/live-welcome-and-reports.md#error-codes); the
[documentation index](docs/README.md) lists the rest of the reference.
`Insider.Live.JSON` returns the same
results as canonical JSON.

An agent should start with `welcome/1`: by default an agent quick start
(the steps, the core observer commands, and symptom recipes), and with
`%{"full" => true}` a document generated from this code that lists every
operation, option, default, bound, and error code, the transports with
example invocations, and how to report Insider bugs and behaviour that
differs from the document with `report/1`.

Live observation is available only in development, test, and ci, as
resolved from host configuration by the node itself; callers cannot assert
an environment. It is not a production collector.

# `result`

```elixir
@type result() :: {:ok, map()} | {:error, map()}
```

# `alloc`

```elixir
@spec alloc(map()) :: result()
```

Reports the VM's memory allocators once (`insider.live-alloc/v1`) with
`recon_alloc`: allocated and used bytes, the share in use, bytes by
allocator type, and the fragmentation of the `"top"` allocator instances
with the largest multiblock carriers. Without recon on the node it is
`component_unavailable` (component `recon`); the observer's `--load`
pushes recon with Insider.

# `amend`

```elixir
@spec amend(String.t(), map()) :: result()
```

Changes a running session's recording options without losing its
history: every start option given replaces the session's value (a list
replaces the whole list) and the rest keep theirs. `"reader"` names who
amends.

The session keeps its id, cursors, readers, and subscribers. Its events so
far stay readable, and the next receipt is a `live.session_amended` event
that every reader sees through any view: what changed, from and to, and
`"uncollected_ms"`, the pause in collection while the collectors
restarted. Returns `insider.live-amendment/v1`; `"amend_failed"` when the
new options could not start (`"rolled_back"` says whether the previous
ones run again).

A `"telemetry_events"` prefix entry such as `["shop", "*"]` is expanded
as `start/1` expands it, in this process before the owner is asked, and
the result lists each prefix's names under `"telemetry_event_prefixes"`.

# `baseline`

```elixir
@spec baseline(String.t(), map()) :: result()
```

Marks the anomalies of a digest window normal for this session, as
`insider.live-baseline/v1`: every candidate's kind, hashed identity, and
count over its window (never its line, names, or values), with the
earlier entries the window does not have. Later digests and `status` fold them into one
line and lead with what is new; one that reaches ten times its baseline
comes back, saying so. `"since"` is the window (default `latest-5m`),
read through the reader's view; `"show"` returns the baseline, `"clear"`
removes it. The session keeps it for its life and its read-only window.
A window without anomalies leaves the baseline as it was.

# `bin_leak`

```elixir
@spec bin_leak(map()) :: result()
```

Finds binary leaks once (`insider.live-bin-leak/v1`), after
`recon:bin_leak/1`: every unprotected process's binaries are measured,
it is garbage collected, and measured again. `"by_retained_bytes"` ranks
what each process still references after the collection (a leak keeps
it); `"by_freed_bytes"` ranks what the collection released. Insider's
own processes are summed apart. Option `"top"`: entries per ranking.

# `capabilities`

```elixir
@spec capabilities() :: map()
```

Reports what this node can observe, without starting anything.

`"welcome"` carries the welcome document's hash and how to read it, and
`"quiet_default_telemetry_events"` the Telemetry events a session without
options records on this node.

It includes the host's disclosure profile and the credential masking
policy id.

# `changes`

```elixir
@spec changes(String.t(), map()) :: result()
```

Returns who changed watched in-memory values (`"watch_writes"`) as
`insider.live-changes/v1`: each write, oldest first, with its key, its
value (rich only), and its writer (the process, and the GraphQL
operation, route, user agent, user id, and trace id of the request it
belonged to, from that process's events, else from the session's kept
mutation record). `"key"` keeps one key's writes; `"human"` adds `"text"`.
It reads the session newest first, within the correlated budget, through
the reader's view, and renews the lease.

# `digest`

```elixir
@spec digest(String.t(), map()) :: result()
```

Summarises a window of the session's events as `insider.live-digest/v1`,
and renews the lease.

The window starts at `"since"`: by default `"latest-60s"`, the first
event the Recorder admitted in the last minute (`"latest-DURATION"` in
whole `ms`, `s`, `m`, or `h`, 1 s to 24 h, measured on the Recorder's
admission clock, not the events' own wall times); an integer cursor
(0 is the session start); `"latest-N"` for the newest N receipts; or
`"mark"` (`"mark:LABEL"`), after the newest pairing mark (with that label,
see `mark/2`), with `"window"` `"mark"` naming it. It
ends at `"upto"`, the newest receipt when the digest starts; events that arrive
while it reads are left for the next digest. It is read with
`events_since/3`, through the same disclosure boundary, and returns
`"counts"` by kind, severity, source, and supervisor offender,
`"crash_loops"` with their period and reason fingerprint, the top
`"error_signatures"`, the `"latency"` tables from summaries and recorded
durations with the summaries' exemplar receipts, `"trends"` across the
summary windows (as `trends/2`), the Logger `"logger_bursts"`, the longest
mailboxes (`"processes"`), `"vm"` deltas, and the window's `"filtered"`
and `"gaps"`. One budget,
`bounds.digest_budget_ms`, covers the `"latest-N"` lookup or the time
window's start search and every read; a search it cannot finish is a
`"timeout"`, and a start it could not narrow exactly has
`"start_exact" => false`. The `"window"` says what was asked for
(`"since"`), its start (`"from_wall_time"`), when the digest was made
(`"as_of"`), and the age of its newest event (`"age_ms"`), and each hint
names the `"window"` it was worked out over;
an unfinished `"window"` has `"complete" => false` and the
`"next_cursor"` to continue from. `"human" => true` adds `"text"`, a
short plain-text summary for the developer.

# `discover`

```elixir
@spec discover(map()) :: result()
```

Lists the Telemetry event names this node can emit, without starting or
running anything: names with an attached handler (`:telemetry.list_handlers/1`,
except Insider's own) and literal event lists passed to
`:telemetry.execute/2,3` and `:telemetry.span/3` (expanded to `:start`,
`:stop`, and `:exception`) in loaded application modules, read from their
debug info. Each name carries its `"sources"`: the source (`"handler"`,
`"execute"`, or `"span"`) and the module.

Option `"prefix"`, a list of segments such as `["shop"]`, keeps only the
names under it. The scan is bounded in modules, names, and time, and
`"truncated_by"` names what bounded it. At most two discoveries run at
once (`busy`).

# `events_since`

```elixir
@spec events_since(String.t(), non_neg_integer() | String.t(), map()) :: result()
```

Returns redacted events with a receipt greater than `cursor`, in receipt
order, with explicit `"gaps"`, and renews the lease.

`cursor` is an integer of at least 0, or `"latest"`: the newest receipt
once the read has drained the collector inboxes, so the batch is empty and
its `"next_cursor"` starts a tail; or `"mark"` (`"mark:LABEL"`): after the
newest pairing mark (with that label, see `mark/2`), with the batch's
`"mark"` naming it, else `mark_not_found`.

Options: `"limit"` (events returned), `"max_bytes"` (canonical JSON bytes
of the events returned), `"scan_limit"` (retained events examined),
`"upto"` (the newest receipt the read may return, from the starting
cursor, after `"latest"` resolves, to the newest receipt, else
`invalid_options`; losses are counted only up to it), and `"view"`, a
read-time filter that narrows only this reader's events.
`"truncated_by"` names the bound that ended a truncated read. A read is
complete when it reaches its snapshot boundary: `"upto"` when given, else
the newest receipt when it began. Every batch reports `"filtered"` counts.
Pass the returned `"next_cursor"` to the next call.

# `handoff_record`

```elixir
@spec handoff_record(String.t()) :: result()
```

A handoff record (`insider.live-handoff/v1`) for passing a session to
another agent: the node, the welcome hash, the session id and state, its
recording configuration (which every reader shares), its named readers
with their cursors, the newest receipt, and the commands that resume it.
It reads the session's status and changes nothing. The observer command
is `handoff`. (`handoff/1` is the name the production-artifact boundary
reserves for a host delivery client, so this is `handoff_record/1`.)

# `ingest_spans`

```elixir
@spec ingest_spans(String.t(), [map()]) :: result()
```

Records spans exported by another process, such as a frontend server, in
the running session as `otel.span` events.

`spans` is a list of at most `Insider.Collector.Otel.ingest_max/0` plain
span maps (`"trace_id"`, `"span_id"`, `"parent_span_id"`, `"name"`,
`"span_kind"`, `"status"`, `"status_message"`, `"service"`, `"scope"`,
`"start_time_us"`, `"duration_us"`, `"attributes"`, and `"pid"`). The
session's otel component applies the span contract, its disclosure
profile, and credential masking to each one, exactly as to the node's own
spans, and records it with `"capture"` `"ingest"`. The result counts the
spans `"accepted"`, `"rejected"` by the contract, `"filtered"`, `"shed"`
under load, and `"duplicate"`.

Ingest is idempotent per `{trace_id, span_id}` within one otel collector
of the session: a span already recorded from an ingest is counted as
`"duplicate"` and not recorded again (the collector remembers the last
16,384 recorded ids; one it forgets is counted as `seen_evicted` in the
otel component's status). Every amendment replaces the collector, even
one rolled back into a replacement, and so resets dedupe: a span ingested
before it is recorded again after it, and `seen_evicted` does not count
that reset.

A `timeout` means the outcome is **unknown**: the collector may have
taken the spans and still be recording them when the call stops waiting,
so some, all, or none may be recorded. A retry of the same spans does not
duplicate a span while its id is among the last 16,384 recorded; a
forgotten id is recorded again, and `seen_evicted` counts the forgotten
ones; a retry across an amendment is recorded again.

An ended session is `session_terminal`, and a session started without the
`"otel"` component `component_unavailable`. The Lab's `otlp-receive`
calls this for each OTLP/HTTP trace export it receives.

# `mark`

```elixir
@spec mark(String.t(), map()) :: result()
```

Marks the moment before an action as `insider.live-mark/v1`, for pairing
with a programmer who acts in the app while the agent watches: mark
("about to click Confirm"), ask them to act, then read since the mark
(`digest/2`, `trends/2` and `timeline/2` with `"since" => "mark"`,
`events_since/3` with cursor `"mark"`; `"mark:LABEL"` for the newest mark
with that label) instead of guessing a time or a cursor.

Options: `"label"` (required; one line, credential-masked in every
profile, at most `bounds.mark_label_max_bytes` bytes) and `"reader"`. The
mark is the newest receipt now (`"cursor"`: a read since it starts after
it), its time, label and reader, and is recorded as a `live.mark` event
(`"recorded"` says whether; views never hide it). A session keeps its
newest `bounds.marks_max` marks, through amendments and after a stop.
`"next"` is the command to read since it. It renews the lease.

# `marks`

```elixir
@spec marks(String.t(), map()) :: result()
```

Lists a running or stopped session's pairing marks as
`insider.live-marks/v1`, oldest first, each with its `"age_ms"`, and how
many older marks it let go (`"dropped"`). `"human"` adds `"text"`, one
line per mark.

# `mutations`

```elixir
@spec mutations(String.t(), map()) :: result()
```

Lists the session's kept mutation records (`Insider.Live.Mutations`) as
`insider.live-mutations/v1`, oldest first, through the reader's view:
each mutation's operation, result, response typenames, time, trace,
process, the client's user agent, and the record ids among its
arguments (rich only), with the ring's accounting. `"id"` keeps the
records whose arguments name that id; `"human"` adds `"text"`. The
records outlive the Recorder's eviction. It renews the lease.

# `ports`

```elixir
@spec ports(map()) :: result()
```

Lists the node's ports once (`insider.live-ports/v1`): counts by type
(the first word of the port's name, such as `cat`) with their OS
processes, then the ports matching `"type"` and `"owner"` (a registered
name or pid) with their owner, OS pid, and I/O counters, and the owners
holding the most of them, all from one read of at most
`bounds.inspect.ports_scanned_max` ports. Option `"top"` bounds each
list. A port's full name is returned only in the rich profile; when its
arguments name a credential anywhere or hold anything a shell may rewrite
(an escape, a control character, `$`, a backtick), the whole argument
list is masked.

# `process`

```elixir
@spec process(map()) :: result()
```

Inspects one process once (`insider.live-process/v1`): a `process_info`
subset (status, initial and current function, queue length, memory,
reductions, links by kind, monitors), the current stacktrace's top
frames with file and line, its binaries' count and bytes, and a mailbox
summary: queued messages counted by shape and size class, the shapes of
the first `"messages"`, and how many queued `GenServer.call`s come from
callers that already exited. A shape is a message's structure with types
in place of values, such as `{:"$gen_call", {pid, list}, {:price, binary}}`;
no message value, dictionary entry, or state is ever read. The mailbox is
read only up to `bounds.inspect.mailbox_read_max` messages.

Options: `"name"` (a registered name such as `"MyApp.Worker"`) or
`"pid"` (a local pid such as `"<0.200.0>"`), `"frames"`, and
`"messages"`. A protected process is `protected_process`; an absent one
`process_not_found`.

Like every inspection, the result passes the disclosure profile and
credential masking (`"redactions"`), and when a session runs it is also
recorded in it as an `inspect.process` event; `"session"` says whether.

# `renew`

```elixir
@spec renew(String.t()) :: result()
```

Renews the session lease.

# `report`

```elixir
@spec report(map()) :: result()
```

Queues a report about Insider itself on this node.

Report Insider bugs and any behaviour that differs from `welcome/1`.
Options: `"category"`, one of `"bug"`, `"doc_mismatch"`,
`"missing_capability"`, or `"confusing_output"`, and `"message"`, a
printable text of at most 2,048 bytes. Insider adds the context itself: its
version and commit, the welcome hash, the node, the environment, and the
last operation and error code, never event payloads or option values.

The queue is bounded memory in this node: a report whose fingerprint
(category and normalised message) is already queued increments its
`"count"`, and the oldest report is evicted beyond 100. Nothing leaves the
node; the Lab observer's `reports` command copies the queue and its
`reports submit` files GitHub issues only when the operator or host allows.

# `reports`

```elixir
@spec reports() :: result()
```

Lists the reports queued on this node, oldest first.

# `start`

```elixir
@spec start(map()) :: result()
```

Starts the live session.

Options: `"components"` (a non-empty subset of `"calls"`, `"logger"`,
`"otel"`, `"poller"`, and `"telemetry"`; `"otel"` records finished
OpenTelemetry spans and needs the SDK on the node; default `["logger", "telemetry"]` when the node has quiet
default Telemetry events, else `["logger"]`), `"telemetry_events"` (event
names as lists of existing-atom strings, recorded with phase `:event`;
default: the quiet default events), `"telemetry_measurements"`
(measurement names to project), `"disclosure"` (`"rich"` or `"strict"`, no richer than the host
profile; default the host profile), `"max_events"`, and `"lease_ms"`.
Logger message text and metadata are collected only in the rich profile.

A `"telemetry_events"` entry whose last segment is `"*"`, such as
`["shop", "*"]`, is a prefix: it stands for every event name `discover/1`
finds below it now, counted against the event bound after expansion. The
result lists each prefix's names under `"telemetry_event_prefixes"`.

Record-time filters, applied in the emitting process before anything is
recorded: `"logger_level"` (default `"warning"`), `"logger_include"`,
`"logger_exclude"`, `"exclude_origins"` (default `["oban"]`),
`"telemetry_min_duration_ms"` (default 100),
`"telemetry_event_min_duration_ms"`, `"telemetry_summaries"`,
`"telemetry_summary_interval_ms"`, `"telemetry_duration_scopes"` (a map of
summarised kind to `"handler"` or `"caller"`, how its duration was
measured; an undeclared kind's scope is unknown), `"logger_summaries"`
(Logger rules whose messages are counted per message template, call site,
and level, a burst, instead of recorded; at most 64 bursts per interval,
the rest folded into one overflow row per level), and
`"logger_summary_interval_ms"`. See
[Filters, summaries, and views](docs/live/live-digest-filters-and-trends.md#filters-summaries-and-views).

Poller options, with the poller component: `"poller_interval_ms"` (default
5,000), `"top_processes"` and `"top_ets_tables"` (entries in each process
and ETS ranking; default 5), and `"watch_processes"` (registered names to
sample every interval). See
[Process and ETS evidence](docs/live/live-runtime-evidence.md#process-and-ets-evidence).

Call-trace options, with the calls component (OTP 28 and later): `"calls"`
(functions to trace as `"MOD.FUN/ARITY"`, required), `"calls_results"`
(`"exceptions"`, `"all"`, or `"none"`), `"calls_max"`, `"calls_rate_max"`,
`"calls_rate_ms"`, and `"calls_max_bytes"`. See
[Call traces](docs/live/live-runtime-evidence.md#call-traces).

# `status`

```elixir
@spec status(String.t()) :: result()
```

Reports a running session, or the retained record of an ended one.

A running session's record carries `"next_receipt"`, the newest receipt;
`"recorder"`: `"oldest_receipt"`, `"retained_events"`, `"retained_bytes"`,
`"max_events"`, `"max_bytes"`, `"archived_events"` (events kept from
before the last amendment), and `"sources"`, each share group's
`"share_percent"`, `"reserved_events"`, `"reserved_bytes"`,
`"retained_events"`, `"retained_bytes"`, and `"evicted"` (conservative:
after a collector is killed mid-write it can exceed the events really
evicted, never fall short; a batch's `"evicted"` gap stays exact); and
`"insider_memory"`: `"processes"` (bytes per Insider process serving the
session), `"processes_bytes"`, `"recorder_bytes"` (the Recorder's tables
and event binaries), and `"total_bytes"`. It lists `"readers"`, each
named reader with its `"cursor"`, `"behind"`, `"last_seen_ms_ago"`,
`"last_seen_at"`, `"subscribed"`, and `"active"`, and the latest
`"amendments"`. An ended session still readable after a stop carries
`"readable_remaining_ms"` and its readers.

# `stop`

```elixir
@spec stop(String.t(), map()) :: result()
```

Stops the session and returns its record with the cleanup receipt.

It refuses with `"readers_active"` while another named reader read within
`reader_active_ms` or is subscribed, unless `"force"` is `true`. Unnamed
readers never hold a stop. `"reader"` names who stops, so that reader's own reads do not
count. A stopped session stays readable with `events_since/3` for
`stopped_readable_ms` (its record's `"readable_until"`), so peers can
drain it. Stopping an ended session returns its record again.

# `subscribe`

```elixir
@spec subscribe(String.t(), pid(), map()) :: result()
```

Pushes `{:insider_live_events, session_id, batch}` messages to `pid`.

A batch has the `events_since/3` shape, and each batch starts at the
`"next_cursor"` of the one before it. The owner monitors `pid` and drops
it when it exits or its node disconnects. The subscription does not renew
the lease and ends with one batch whose `"state"` is terminal and which
carries the `"receipt"`. Options: `"cursor"` (default 0; an integer no
newer than the newest receipt, or `"latest"`), `"max_events"` (the events
pushed in total; then nothing more is sent but the terminal batch), and
`"view"`, as for `events_since/3`. Subscribing again replaces the view and
`"max_events"`, and moves the cursor only when one is given.

# `timeline`

```elixir
@spec timeline(String.t(), map()) :: result()
```

Returns the correlated timeline of one process (`"pid"`, as events name it
in `subject.value`) or one request (`"request_id"`, Plug's request id) as
`insider.live-timeline/v1`, across every source, and renews the lease.

A request's events are those carrying its id, the events of its
processes within its time window, and the events and spans of its
traces. Options as for `trace/3`, plus exactly one of `"pid"` and
`"request_id"`.

# `trace`

```elixir
@spec trace(String.t(), String.t(), map()) :: result()
```

Returns one trace as `insider.live-trace/v1`: the spans the session
recorded with this W3C `trace_id` (32 lowercase hex digits) merged in time
order with the Logger and Telemetry events of the same trace, and renews
the lease.

An event joins by its trace id; an event without one joins when the
process that emitted it ended one of the trace's spans and its time falls
inside that span. Each item has `"at"`, `"depth"` in the span tree,
`"span_id"` (the span it belongs to), `"joined_by"`, and the disclosed
`"event"`. `"unjoined"` says what could not be joined (parent spans that
were not captured, events without a time), `"scan"` how much of the
session was read (`"complete"` is false when the read budget ran out),
and `"gaps"` what the reads lost. Options: `"limit"` (items, default
200), `"max_bytes"` (default 262,144), and `"human"`, which adds
`"text"`, an indented plain-text rendering.

# `trends`

```elixir
@spec trends(String.t(), map()) :: result()
```

Trends across a window's summary windows, as `insider.live-trends/v1`, and
renews the lease.

The window is a digest's (`"since"`, read through `events_since/3` within
`bounds.digest_budget_ms`). For each summarised Telemetry kind,
`"telemetry"` lists its windows (`started_at`, `ended_at`, `count`,
`rate_per_min`, `errors`, `error_pct`, `p50_us`, `p95_us`, `max_us`, the
exemplar receipts, `exemplars_lost`, and `first_error_retained`) and the
change of rate, p95, max, errors and error share from the first window to
the last, with its direction. Its `"duration_scope"` is the summaries':
`"measured_in"` is `"handler"` or `"caller"` only when the session
declared it (`"telemetry_duration_scopes"`), else `"unknown"`, and
`"emitter"` is only the context the exemplars were emitted in. A note
compares a declared handler series with a declared caller series only
conditionally: Insider does not establish that they describe the same
operation. `"logger"` lists each Logger burst's windows, one series per
template fingerprint, call site fingerprint, and level, and each level's
overflow row. `"poller"` lists Poller series, one point per poller cycle:
`binary_bytes` and `port_count` per `vm_sample`, ports per type per
`port_top` (a zero only at a complete census), and `message_queue_len`
per ranked or watched process (`"coverage"` `ranked`, `watched`, or
`mixed`), each with `"unit"`, `"change"`, `"per_min"`, `"max"`, and its
points; `"poller_omitted"` and `"poller_untracked"` count what the bounds
left out. `"units"` names every value's fixed unit, and `"notes"` what
the series show together. Options: `"since"`, `"kinds"` (the Telemetry
kinds to show, `"logger_summary"` for the bursts, and `"vm_sample"`,
`"port_top"`, `"process_top"` or `"process_watch"` for the Poller
series), `"windows"` (the most recent windows per series), and
`"human"`, which adds `"text"`.

# `unsubscribe`

```elixir
@spec unsubscribe(String.t(), pid()) :: result()
```

Removes a subscriber; removing an absent one succeeds.

# `welcome`

```elixir
@spec welcome(map()) :: map() | {:error, map()}
```

Returns the welcome document for an agent.

By default it is the agent quick start (`insider.live-welcome-quick-start/v1`):
the steps, the core observer commands with their flags, the symptom
recipes, where to read the full reference, and `"runtime"`. With
`%{"full" => true}` it is the full reference (`insider.live-welcome/v2`),
generated from the code's own tables: the operations with their
options, defaults, and bounds, the target's and the observer's error codes,
components, allowed measurement names, the transports with example
invocations and the observer's value syntax for every flag, a short guide
(what to do first, the act-then-read loop, how to stop, and the duty to
report), and `"runtime"`: this node, its environment, the active session,
and this node's quiet default Telemetry events. `"hash"` covers every key
of the full reference except `"hash"` and `"runtime"`; both forms carry
it. Any other option is `{:error, map}` with code `"invalid_options"`.

---

*Consult [api-reference.md](api-reference.md) for complete listing*
