dial9-trace-loading

Loading and Parsing Traces

ParsedTrace structure

parseTrace(path) yields ParsedTrace objects:

{
  magic: string,                   // "D9TF" format identifier
  version: number,                 // trace format version
  events: TraceEvent[],          // PollStart, PollEnd, WorkerPark, WorkerUnpark, QueueSample, WakeEvent
  minTs: number|null,            // earliest event timestamp (ns), null if no events
  maxTs: number|null,            // latest event timestamp (ns), null if no events
  recordMinTs: number|null,      // earliest sliceable timestamped record (ns), null if none
  recordMaxTs: number|null,      // latest sliceable timestamped record (ns), null if none
  cpuSamples: CpuSample[],      // Periodic stack traces from perf/eBPF
  customEvents: CustomEvent[],   // SpanEnter/SpanExit events from tracing layer (requires dial9-tokio-telemetry tracing-layer feature)
  spawnLocations: Map<string, string>,    // spawn location ID → source location
  taskSpawnLocs: Map<number, string|null>,// task ID → spawn location (null if unknown)
  taskSpawnTimes: Map<number, number>,    // task ID → spawn timestamp (ns)
  taskTerminateTimes: Map<number, number>,// task ID → terminate timestamp (ns)
  callframeSymbols: Map<string, {symbol, location}|[{symbol, location}]>, // address → resolved symbol (array for inlined frames)
  threadNames: Map<number, string>,       // thread ID → name
  clockOffsetNs: number|null,            // monotonic-to-wall-clock offset
  clockSyncAnchors: [{monotonicNs, realtimeNs}],
  runtimeWorkers: Map<string, number[]>, // runtime name → worker IDs
  segmentMetadata: Map<string, string>,  // latest segment metadata key → value
  truncated: boolean,
  timeFiltered: boolean,
  filterStartTime: number|null,          // start of time range filter (ns), null if unfiltered
  filterEndTime: number|null,            // end of time range filter (ns), null if unfiltered
  hasCpuTime: boolean,                   // trace includes CPU time data
  hasSchedWait: boolean,                 // trace includes kernel scheduling wait data
  hasTaskTracking: boolean,              // trace includes task spawn/terminate events
  taskInstrumented: Map<number, boolean>, // task ID → whether task has tracing instrumentation
  taskDumps: Map<number, [{timestamp, callchain}]>, // task ID → async stack captures (sorted by timestamp); see dial9-tokio-telemetry `taskdump` feature
  allocEvents: AllocEvent[],     // Sampled memory allocations (requires dial9-tokio-telemetry memory-profiling feature)
  freeEvents: FreeEvent[],       // Deallocations paired with sampled allocs (requires `track_liveset`)
  memoryOverflows: [{timestamp, droppedAllocs, droppedFrees}], // Ring buffer overflow events (dropped samples per flush period)
  blockInPlaceGaps: [{workerId, fromTid, toTid, startNs, endNs}], // Detected block_in_place handoff intervals (worker attribution unknowable during gap)
  tidToWorker: Map<number, number>,  // thread ID → worker ID mapping (derived from park/unpark events)
}

Event types

const EVENT_TYPES = {
  PollStart: 0,   // Worker begins polling a task
  PollEnd: 1,     // Worker finishes polling a task
  WorkerPark: 2,  // Worker goes to sleep (no work available)
  WorkerUnpark: 3,// Worker wakes up
  QueueSample: 4, // Global queue depth sample
  WakeEvent: 9,   // One task wakes another
};

TraceEvent fields

CpuSample fields

AllocEvent fields

FreeEvent fields

Parse options

const trace = await parseTrace('/path/to/trace.bin', {
  maxEvents: 100000,        // Cap event count (metadata/symbols always parsed)
  startTime: 1000000000,    // Filter events to time range (absolute ns)
  endTime:   2000000000,
});

Converting timestamps

Trace timestamps are monotonic nanoseconds. To convert to wall clock:

if (trace.clockOffsetNs != null) {
  const wallNs = event.timestamp + trace.clockOffsetNs;
  const wallDate = new Date(wallNs / 1e6);
}

To get relative time from trace start:

const relativeMs = (event.timestamp - trace.minTs) / 1e6;

Symbol resolution

const { formatFrame, symbolizeChain, deduplicateSamples } = require('./trace_parser.js');

// Resolve a full callchain to frame objects
const frames = symbolizeChain(sample.callchain, trace.callframeSymbols);
// [{symbol: "hyper::proto::h1::dispatch::Dispatcher<...>::poll_inner", location: "hyper-0.14.28/src/proto/h1/dispatch.rs:174"}, ...]

// Format a single frame for display
const { text, docsUrl } = formatFrame(frames[0]);
// text: "Dispatcher::poll_inner dispatch.rs:174"

// Deduplicate samples by stack trace
const groups = deduplicateSamples(trace.cpuSamples, trace.callframeSymbols);
// [{count: 8932, leaf: "__schedule", frames: [...]}, ...]

Handling gzip

parseTrace automatically decompresses gzip input. Pass .bin.gz files directly.

Fetching traces from S3

Start the viewer (dial9-viewer --bucket BUCKET, default port 3000), then fetch traces:

// List traces in a time range
const resp = await fetch('http://localhost:3000/api/browse?bucket=BUCKET&prefix=2026-04-09/19&from=0&to=' + Math.floor(Date.now()/1000));
const objects = (await resp.json()).objects; // [{key, size, last_modified}, ...]

// Single file: fetch and parse one trace. /api/object serves the file's raw
// (still-gzipped) bytes; parseTrace decompresses transparently.
const traceResp = await fetch(`http://localhost:3000/api/object?bucket=BUCKET&key=${encodeURIComponent(objects[0].key)}`);
const buf = Buffer.from(await traceResp.arrayBuffer());
const trace = await parseTrace(buf);

// Multiple files: download to a local directory, then analyze
const fs = require('fs');
const dir = '/tmp/traces';
fs.mkdirSync(dir, { recursive: true });
// Download in parallel (20 at a time)
const limit = 20;
for (let i = 0; i < objects.length; i += limit) {
  await Promise.all(objects.slice(i, i + limit).map(async (obj) => {
    const r = await fetch(`http://localhost:3000/api/object?bucket=BUCKET&key=${encodeURIComponent(obj.key)}`);
    fs.writeFileSync(`${dir}/${obj.key.split('/').pop()}`, Buffer.from(await r.arrayBuffer()));
  }));
}
for await (const trace of parseTrace(dir)) {
  // analyze each trace
}

Merging multiple trace files

Trace files can be concatenated back-to-back. The parser handles multiple concatenated segments transparently (headers, string pools, and schemas are re-read at each boundary).

gunzip -k segment-001.bin.gz segment-002.bin.gz segment-003.bin.gz
cat segment-001.bin segment-002.bin segment-003.bin > combined.bin