screenize-explorer

Package Structure

MyProject.screenize/
├── project.json               # ScreenizeProject v5
└── recording/
    ├── recording.mp4          # Video file
    ├── metadata.json          # PolyRecordingMetadata
    ├── mousemoves-0.json      # [PolyMouseMoveEvent]
    ├── mouseclicks-0.json     # [PolyMouseClickEvent]
    ├── keystrokes-0.json      # [PolyKeystrokeEvent]
    └── uistates-0.json        # [PolyUIStateEvent]

project.json Schema (v5)

Top-level fields:

• id (UUID), version (int), name, createdAt, modifiedAt (ISO 8601)
• media: { videoPath, mouseDataPath, pixelSize: {width, height}, frameRate, duration }
• captureMeta: { displayID?, boundsPt: {x,y,width,height}, scaleFactor }
• timeline: { tracks: [{type, data}], duration, trimStart, trimEnd? }
• renderSettings: codec, quality, resolution, background, corners, shadow, padding, motionBlur
• interop: { sourceKind, eventBundleVersion, recordingMetadataPath, streams: {mouseMoves, mouseClicks, keystrokes, uiStates}, primaryVideoPath }
• frameAnalysisCache (optional array), frameAnalysisVersion

Timeline Track Types

Each track is wrapped as { type: "transform"|"cursor"|"keystroke", data: {...} }:

Camera track (type: "transform"): segments with startTime, endTime, startTransform: {zoom, center: {x, y}}, endTransform, interpolation, mode ("manual"/"followCursor"), cursorFollow, transitionToNext

Cursor track (type: "cursor"): useSmoothCursor, segments with startTime, endTime, style, visible, scale, clickFeedback, transitionToNext

Keystroke track (type: "keystroke"): segments with startTime, endTime, displayText, position, fadeInDuration, fadeOutDuration

Event Stream Formats

metadata.json: { formatVersion, recorderName, recorderVersion, createdAt, processTimeStartMs, processTimeEndMs, unixTimeStartMs, display: { widthPx, heightPx, scaleFactor } }

mousemoves-0.json: Array of { type, processTimeMs, unixTimeMs, x, y, cursorId?, activeModifiers, button? }. Coordinates: display-local pixels, top-left origin.

mouseclicks-0.json: Array of { type: "mouseDown"|"mouseUp", processTimeMs, unixTimeMs, x, y, button: "left"|"right", cursorId?, activeModifiers }.

keystrokes-0.json: Array of { type: "keyDown"|"keyUp", processTimeMs, unixTimeMs, character?, isARepeat, activeModifiers }.

uistates-0.json: Array of { processTimeMs, unixTimeMs, cursorX, cursorY, elementRole?, elementSubrole?, elementTitle?, elementAppName?, elementFrameX/Y/W/H?, elementIsClickable?, caretX/Y/W/H? }.

Computing Timeline Seconds

timelineSec = (event.processTimeMs - metadata.processTimeStartMs) / 1000.0

Step 1: Find Project

If the user provides a specific .screenize path as an argument, use it directly.

Otherwise, search for projects in these locations (in order):

1. ./projects/ relative to the current working directory
2. ~/Movies/Screenize/

Use Glob to find .screenize packages:

pattern: projects/**/*.screenize

If multiple projects are found, list them with names and ask the user which one to inspect. If only one is found, use it automatically. If none are found, inform the user.

Step 2: Package Overview

Read project.json from the package root. Present:

### Project: [name]
- **Version**: [version] | **ID**: [id]
- **Created**: [date] | **Modified**: [date]
- **Video**: [width]x[height] @ [fps] fps, [MM:SS duration]
- **Capture**: Display [id], [w]x[h] pt, scale [factor]x
- **Export**: [codec] / [quality] / [resolution]
- **Background**: [enabled/disabled]
- **Trim**: [start] - [end] (or "None")

Step 3: Timeline Summary

For each track in the timeline:

Camera Track ([N] segments)

Table with columns: #, Time Range, Start Zoom, End Zoom, Center, Mode, Easing

Cursor Track ([N] segments, smooth: [yes/no])

Table with columns: #, Time Range, Style, Visible, Scale

Keystroke Track ([N] segments)

Table with columns: #, Time Range, Text

Step 4: Event Stream Statistics

Read recording/metadata.json first for processTimeStartMs.

For each event stream file that exists, read it (use Bash wc -l for very large files, or Read for moderate ones) and report:

Recording Metadata

• Recorder: [name] v[version]
• Display: [w]x[h] px @ [scale]x
• Duration: [computed] sec
• Created: [date]

Mouse Moves

• Total: [count] events
• Time range: [first] - [last] sec
• Avg frequency: [count/duration] Hz

Mouse Clicks

• Total: [count] ([downs] mouseDown, [ups] mouseUp)
• Left: [count] | Right: [count]

Keystrokes

• Total: [count] ([downs] keyDown, [ups] keyUp)
• Characters: [first 20 unique non-nil characters]

UI States

• Total: [count] samples
• Unique apps: [list]
• Unique roles: [list]

Step 5: Deep Dive (Optional)

After showing the summary, ask the user if they want to:

1. View raw JSON of a specific file
2. Inspect events in a specific time range (e.g., "show clicks between 5.0-10.0 sec")
3. See detailed camera segment transitions with easing curves
4. List all UI element interactions in chronological order

Only proceed if the user requests one of these.