From 5f0d5354edcbbb25501592d101d98a5e46400dc9 Mon Sep 17 00:00:00 2001 From: Skyler Lehmkuhl Date: Sun, 21 Jun 2026 23:06:23 -0400 Subject: [PATCH] Document the .beam SQLite format and port the inspector The .beam container moved from ZIP to SQLite, but the docs/tooling still described the old ZIP format. - Rewrite BEAM_FILE_FORMAT.md as a normative spec of the current SQLite container: file-magic identification, the four-table schema, the two version numbers, the media model (kinds, packed/referenced storage, 4 MiB chunking), derived media-id formulas, project.json top-level + key entities, save/load semantics, the legacy-ZIP format + migration, large-media policy, conformance and security sections, and known quirks. - Port beam_inspector.py to read SQLite (auto-detecting container by magic, with the legacy ZIP path preserved): a --media store view, packed/referenced storage detection, and chunk-reassembling media extraction. - Update beam_inspector_README.md to match. --- BEAM_FILE_FORMAT.md | 484 +++++++++++++++++++++++++++ beam_inspector.py | 701 +++++++++++++++++++++++++++++++++++++++ beam_inspector_README.md | 210 ++++++++++++ 3 files changed, 1395 insertions(+) create mode 100644 BEAM_FILE_FORMAT.md create mode 100755 beam_inspector.py create mode 100644 beam_inspector_README.md diff --git a/BEAM_FILE_FORMAT.md b/BEAM_FILE_FORMAT.md new file mode 100644 index 0000000..91a0e75 --- /dev/null +++ b/BEAM_FILE_FORMAT.md @@ -0,0 +1,484 @@ +# `.beam` File Format Specification + +**Status:** Normative. +**Container schema version:** `1` (`meta.schema_version`). +**Project payload version:** `1.0.0` (`BeamProject.version` / `meta.version`). +**Last updated:** 2026-06-21. + +This document specifies the on-disk format of Lightningbeam `.beam` project files +precisely enough that an independent implementation can read and write them. It +describes the **current** format — a SQLite database — and the **legacy** format +(a ZIP archive) that current readers still accept and migrate. + +> An earlier revision of this document described the ZIP container as the current +> format. That is now historical; see [§11 Legacy ZIP format](#11-legacy-zip-format-pre-sqlite). + +## 1. Notational conventions + +The key words **MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, and **MAY** are +to be interpreted as described in RFC 2119. + +Byte sizes use binary units: `KiB = 1024`, `MiB = 1024²`, `GiB = 1024³`. + +"UUID" means an RFC 4122 128-bit identifier. Unless stated otherwise, a UUID is +serialized in JSON as its canonical hyphenated lowercase string, and stored in +SQLite as its 16 raw bytes in **network (big-endian) byte order** — i.e. the byte +sequence returned by `Uuid::as_bytes`, most significant byte first. + +## 2. Overview + +A `.beam` file is a **single SQLite 3 database** (default rollback journal; no WAL, +no special pragmas) containing: + +- one JSON document, `project.json`, holding all project state that is small and + structural (the document/scene tree, audio project, asset metadata); and +- zero or more **media** items (audio, raster frames, image assets, and derived + blobs such as waveforms and thumbnails), each either **packed** into the database + in chunks or **referenced** by an external file path. + +The design goals are: a single beginner-friendly file (no project folder); +streaming reads of large media via SQLite blob I/O; transactional, crash-safe, +**in-place** saves that do not rewrite unchanged media; and inspectability with the +ordinary `sqlite3` CLI. + +## 3. File identification + +A reader MUST determine the container type from the first 16 bytes of the file: + +- If the first 16 bytes equal the ASCII string `SQLite format 3\0` (i.e. + `53 51 4C 69 74 65 20 66 6F 72 6D 61 74 20 33 00`), the file is a SQLite + `.beam` and MUST be read per [§4](#4-container-the-sqlite-schema)–[§10](#10-load-semantics). +- Otherwise the file is treated as a [legacy ZIP `.beam`](#11-legacy-zip-format-pre-sqlite) + (ZIP local-file-header magic `50 4B 03 04`). + +The `.beam` extension is used for both container types; the magic bytes — not the +extension — are authoritative. + +## 4. Container: the SQLite schema + +A conforming SQLite `.beam` MUST contain exactly these four tables (DDL as created; +column order and types are normative): + +```sql +CREATE TABLE media ( + id BLOB PRIMARY KEY, -- 16-byte UUID (big-endian) + kind INTEGER NOT NULL, -- MediaKind (§6.1) + codec TEXT NOT NULL, -- original codec/container, e.g. "flac","mp3","png" + storage INTEGER NOT NULL, -- MediaStorage (§6.2): 0=Packed, 1=Referenced + ext_path TEXT, -- external path, set iff storage=Referenced + total_len INTEGER NOT NULL DEFAULT 0, -- payload length in bytes (packed); 0 if referenced + channels INTEGER, -- audio: channel count (nullable) + sample_rate INTEGER, -- audio: sample rate in Hz (nullable) + width INTEGER, -- visual media: pixel width (nullable) + height INTEGER -- visual media: pixel height (nullable) +); + +CREATE TABLE media_chunk ( + id INTEGER PRIMARY KEY, -- rowid (used as the handle for blob streaming) + media_id BLOB NOT NULL, -- → media.id + chunk_index INTEGER NOT NULL, -- 0-based ordinal of this chunk + bytes BLOB NOT NULL, -- chunk payload + UNIQUE (media_id, chunk_index) +); + +CREATE TABLE project_json ( + id INTEGER PRIMARY KEY CHECK (id = 0), -- exactly one row, id 0 + data TEXT NOT NULL -- the serialized BeamProject (§8) +); + +CREATE TABLE meta ( + key TEXT PRIMARY KEY, + value TEXT NOT NULL +); +``` + +There is no foreign-key constraint between `media_chunk.media_id` and `media.id`; +referential integrity is maintained by the writer. A reader MUST NOT assume SQLite +enforces it. + +### 4.1 The `meta` table + +Defined keys (all values are strings): + +| Key | Value | Meaning | +|------------------|--------------------|---------| +| `schema_version` | `"1"` | Container schema version. Set once at creation. | +| `version` | `"1.0.0"` | Project payload version (mirrors `project.json`'s `version`). | +| `created` | RFC 3339 timestamp | Creation time. Preserved across in-place re-saves. | +| `modified` | RFC 3339 timestamp | Last save time. Updated on every save. | + +Writers MAY add additional keys; readers MUST ignore unknown keys. + +### 4.2 Two independent version numbers + +The format carries **two** version numbers with different compatibility rules: + +- **`meta.schema_version`** (`INTEGER`, currently `1`) — versions the SQLite + container/table layout. A reader MUST reject a file whose `schema_version` is + **greater** than the highest it supports, and SHOULD accept any value less than + or equal to its maximum (forward-compatible for older files). +- **`BeamProject.version`** / `meta.version` (currently `"1.0.0"`) — versions the + `project.json` payload. The current implementation requires **exact string + equality** and rejects any other value. A future revision MAY relax this to a + semantic-version range; until then, writers MUST emit exactly `"1.0.0"` and + readers reject anything else. + +These two numbers are orthogonal and MUST both be checked. + +## 5. UUIDs and identity + +Every media item is identified by a UUID. In `project.json` UUIDs appear as +canonical strings; in the `media`/`media_chunk` tables they appear as 16 raw +big-endian bytes (`media.id`, `media_chunk.media_id`). A reader MUST treat the two +representations as equal by their 128-bit value. + +`media_chunk.id` is the SQLite rowid and is the handle a reader uses to open a blob +for streaming (`sqlite3_blob_open` on table `media_chunk`, column `bytes`). It has +no meaning beyond row identity and MUST NOT be relied upon to be stable across +re-saves. + +## 6. Media model + +### 6.1 `MediaKind` (`media.kind`) + +| Value | Kind | `codec` examples | Notes | +|-------|---------------|------------------|-------| +| `0` | `Audio` | `flac`, `mp3`, `wav`, `ogg`, `opus`, `aac`, `m4a`, `alac`, `caf`, `aiff` | Source audio for a pool entry. | +| `1` | `Video` | — | **Reserved/unused.** The current writer never emits video rows; video bytes live in an external file referenced by `VideoClip.file_path`. | +| `2` | `Raster` | `png` | Full-resolution pixels of a raster keyframe (PNG-encoded RGBA). | +| `3` | `ImageAsset` | `png`, `jpg`, … | An imported image asset's original bytes. | +| `4` | `Waveform` | `lbwf` | Precomputed waveform LOD pyramid for an audio item. Opaque blob owned by `daw_backend::audio::waveform_pyramid`. | +| `5` | `Thumbnail` | `lbtn` | A pack of precomputed video thumbnails for a clip. Opaque blob owned by the editor. | +| `6` | `RasterProxy` | `png` | Low-resolution PNG proxy of a raster keyframe, shown during cold scrubs while full-res pages in. | + +A reader MUST reject a `kind` value it does not recognise only if it needs that +item; unknown kinds MAY otherwise be ignored. + +### 6.2 `MediaStorage` (`media.storage`) + +| Value | Storage | Bytes location | `total_len` | `ext_path` | +|-------|--------------|----------------|-------------|------------| +| `0` | `Packed` | `media_chunk` rows in this DB | payload length | `NULL` | +| `1` | `Referenced` | external file at `ext_path` | `0` | path string | + +### 6.3 Packed storage and chunking + +Packed bytes are split into chunks of **`CHUNK_SIZE = 4 MiB`** and stored one chunk +per `media_chunk` row, ordered by `chunk_index` ascending starting at `0`. The +writer MUST: + +1. set `media.total_len` to the exact total byte length of the payload; +2. write `ceil(total_len / CHUNK_SIZE)` chunk rows (zero rows iff `total_len == 0`); +3. make every chunk exactly `CHUNK_SIZE` bytes **except the last**, which holds the + remainder `total_len − (n−1)·CHUNK_SIZE`. + +Because chunk lengths are fully determined by `total_len` and `CHUNK_SIZE`, a reader +performing random access MUST compute, for a byte offset `pos < total_len`: + +``` +chunk_index = pos / CHUNK_SIZE +offset_in_chunk = pos % CHUNK_SIZE +chunk_len = min(CHUNK_SIZE, total_len − chunk_index·CHUNK_SIZE) +``` + +and read from the row with that `chunk_index`. A reader MAY stream the whole payload +by concatenating chunk `bytes` in `chunk_index` order. The chunk size MAY differ in +files written by other tools; a robust reader SHOULD derive chunk boundaries from +actual row lengths rather than assuming 4 MiB. (The reference reader assumes uniform +`CHUNK_SIZE` except for the last chunk; writers targeting it MUST keep chunks +uniform.) + +### 6.4 Referenced storage + +A referenced item stores only `ext_path` (with `total_len = 0` and no chunk rows). +The path is resolved relative to the directory containing the `.beam` file unless it +is absolute. If the file is absent at load time, the item is reported as a *missing +file* ([§10.4](#104-missing-referenced-files)) — this is non-fatal. + +## 7. Derived media IDs + +Three media kinds are keyed by UUIDs **derived** from another id rather than by an +independent random UUID. A reader/writer MUST compute them exactly as follows +(`from_u128` constructs a UUID from a 128-bit big-endian integer): + +| Kind | Derived from | Formula | +|---------------|---------------------|---------| +| `Waveform` | audio pool **index** `i` (`usize`) | `from_u128((0x4C42_5746 << 96) \| (i as u128))` — high 32 bits = `0x4C425746` ("LBWF"), low 96 bits = the pool index. | +| `Thumbnail` | video clip UUID `c` | `from_u128(c.as_u128() ^ 0x4C42_544E_4C42_544E_4C42_544E_4C42_544E)` — full-width XOR with "LBTN" repeated 4×. | +| `RasterProxy` | raster keyframe UUID `k` | `from_u128(k.as_u128() ^ 0x4C42_5058_4C42_5058_4C42_5058_4C42_5058)` — full-width XOR with "LBPX" repeated 4×. | + +The full-resolution `Raster` row is keyed by the keyframe's **own** UUID (no +derivation); an `ImageAsset` row is keyed by the asset's own UUID; a packed `Audio` +row is keyed by the pool entry's `media_id`. + +> Note the asymmetry: the waveform id ORs the pool index into the high bits, while +> thumbnail/proxy ids XOR a 128-bit sentinel with a source UUID. This is intentional +> (waveforms are keyed by *index*, not by a UUID) but is a sharp edge for +> implementers. + +## 8. `project.json` + +`project_json.data` is a UTF-8 JSON document: the serde serialization of a +`BeamProject`. This section specifies the top-level structure and the entities a +reader needs to resolve media; the deep UI/scene tree is defined by the +corresponding Rust types (serde field names match struct field names unless a +`#[serde(rename)]` is noted) and is intentionally **not** enumerated exhaustively +here. + +### 8.1 `BeamProject` (root) + +| Field | Type | Notes | +|-----------------|---------------------------|-------| +| `version` | string | MUST be `"1.0.0"`. | +| `created` | string (RFC 3339) | | +| `modified` | string (RFC 3339) | | +| `ui_state` | `Document` | The scene/document tree (§8.2). | +| `audio_backend` | `SerializedAudioBackend` | Audio engine state (§8.3). | + +### 8.2 `Document` (top-level fields) + +Collections that carry media linkage are **bold**: + +- `id: Uuid`, `name: string`, `background_color`, `width: f64`, `height: f64`, + `framerate: f64`, `duration: f64` +- `time_signature`, `master_layer`, `timeline_mode` — all `#[serde(default)]` +- `root: GraphicsObject` — the layer tree; raster keyframes live here and inside + nested group/clip layers +- **`image_assets: map`** — keyed by asset UUID (= the media id) +- **`video_clips: map`** — `VideoClip.file_path` is the external + video file; thumbnails are derived media (§7) +- `vector_clips`, `audio_clips`, `instance_groups`, `effect_definitions`, + `script_definitions`, the various `*_folders` asset trees — structural, no direct + media-row linkage +- `ui_layout`, `ui_layout_base` — `Option`, skipped when `None` +- `current_time`, `layer_to_clip_map` — `#[serde(skip)]` (not persisted) + +`RasterKeyframe` (within the layer tree) has `id: Uuid` (= its full-res `Raster` +media id and the seed for its `RasterProxy` id), `time`, `width`, `height`, +`tween_after`, `stroke_log`, and a `media_path` string that is **vestigial** in +SQLite files (it names the legacy ZIP entry path; the SQLite path keys on `id` and +ignores it). Pixel buffers are `#[serde(skip)]` and faulted in from media rows. + +### 8.3 `SerializedAudioBackend` + +| Field | Type | Notes | +|----------------------|---------------------|-------| +| `sample_rate` | `u32` | **Unreliable:** the current writer hardcodes `48000` here; authoritative rates are on `AudioProject.sample_rate` and each `AudioPoolEntry.sample_rate`. | +| `project` | `AudioProject` | Tracks + MIDI clip pool (§8.4). | +| `audio_pool_entries` | `[AudioPoolEntry]` | Audio source registry (§8.5). | +| `layer_to_track_map` | `map` | UI layer UUID → engine track id. `#[serde(default)]`. | + +### 8.4 `AudioProject` (top-level fields) + +`tracks: map`, `root_tracks: [TrackId]`, `next_track_id`, +`sample_rate: u32`, `midi_clip_pool`, `next_midi_clip_instance_id`. DSP graphs are +not serialized; they are rebuilt on load. + +### 8.5 `AudioPoolEntry` (media linkage) + +| Field | Type | Role | +|-----------------|----------------------------|------| +| `pool_index` | `usize` | Stable index; seeds the `Waveform` media id (§7). | +| `name` | string | | +| `media_id` | `Option` (UUID) | Set ⇔ audio bytes are **packed** in this DB under that id. `#[serde(default, skip_serializing_if=None)]`. | +| `relative_path` | `Option` | Set ⇔ audio is **referenced** (external file) or missing. | +| `embedded_data` | `Option`| Legacy/inline: `{ data_base64, format }`. Used when bytes are neither packed nor referenced. | +| `sample_rate` | `u32` | Authoritative sample rate. | +| `channels` | `u32` | Channel count. | +| `duration` | `f64` | Seconds. | +| `is_video_audio`| `bool` | If set, the entry is the audio track of a video; always stored **referenced**. `#[serde(default, skip_if=false)]`. | +| `waveform_blob` | (transient) | `#[serde(skip)]`; carries waveform bytes in memory only. | + +### 8.6 Media linkage summary + +| Media kind | Referenced from `project.json` by | Media-row id | +|--------------------|-----------------------------------|--------------| +| Audio (packed) | `AudioPoolEntry.media_id` | that UUID | +| Audio (referenced) | `AudioPoolEntry.relative_path` | — (external) | +| Audio (embedded) | `AudioPoolEntry.embedded_data` | — (inline base64) | +| Raster (full) | `RasterKeyframe.id` | that UUID | +| Raster proxy | derived from `RasterKeyframe.id` | §7 | +| Image asset | `Document.image_assets` key / `ImageAsset.id` | that UUID | +| Video bytes | `VideoClip.file_path` | — (always external) | +| Video thumbnails | derived from `video_clips` key | §7 | +| Waveform | derived from `AudioPoolEntry.pool_index` | §7 | + +## 9. Save semantics + +A conforming writer MUST perform all media, `project.json`, and `meta` writes for a +save inside **one** SQLite transaction. + +### 9.1 In-place vs. create + +- If the target path exists **and** is already a SQLite `.beam`, the writer opens it + and writes in place. Unchanged packed media MUST NOT be rewritten (§9.3); this is + the central performance/crash-safety property, and writers MUST NOT use a + copy-to-temp-and-rename flow for in-place saves. +- Otherwise (new file, or migrating a legacy ZIP), the writer creates a fresh DB at + `.beam.tmp`, writes everything, commits, then atomically `rename`s it over + the target. + +### 9.2 Order of operations within the transaction + +1. Audio pool entries → `Audio` (+ `Waveform`) media rows. +2. Raster keyframes → `Raster` (+ `RasterProxy`) media rows. +3. Video thumbnail packs → `Thumbnail` media rows. +4. Image assets → `ImageAsset` media rows. +5. **Garbage-collect**: delete every `media` row (and its chunks) whose id is **not** + in the set of live ids accumulated in steps 1–4. +6. Write `project.json` and the `meta` keys (`version`, `created`, `modified`). +7. Commit; then (create path only) rename the temp file over the target. + +### 9.3 Packed vs. referenced decision (audio) + +For each audio pool entry, in order: + +- If a packed row for its `media_id` already exists in the archive, keep it untouched + (do not rewrite the bytes) and keep the entry packed. +- Else if its external file resolves and exists: store **referenced** if the entry is + video audio, *or* the file is `≥ LARGE_MEDIA_THRESHOLD` (2 GiB) and the large-media + mode is not `Pack`; otherwise **pack** it (streamed from disk chunk-by-chunk). +- Else if it has `embedded_data`: base64-decode and pack it. +- Else: leave its references as-is (it will be reported missing on load). + +Raster, image, and thumbnail media follow the same "write if present, else keep the +existing row" rule so that paged-out content survives a save without being held in +memory. + +## 10. Load semantics + +### 10.1 Dispatch and version checks + +Open the file, read `project.json`, parse `BeamProject`. Reject unless +`version == "1.0.0"` (§4.2). The container's `schema_version` is verified on open +(`≤` the reader's maximum). + +### 10.2 Audio resolution + +Per pool entry with a `media_id`: look up the media row. If its `codec` is a +**streamable** audio codec (`mp3`, `flac`, `ogg`/`oga`, `wav`/`wave`, `aiff`/`aif`, +`aac`, `m4a`, `opus`, `alac`, `caf`), the bytes are **streamed lazily** from the blob +at playback time (the entry keeps `media_id`, with `embedded_data`/`path` cleared). +Otherwise the bytes are read eagerly into `embedded_data`. A precomputed `Waveform` +blob, if present, is read into the entry. + +### 10.3 Visual media paging + +- **Raster** full-res pixels are **not** eagerly decoded; each keyframe is flagged + for fault-in and its pixels are paged from its `Raster` row on demand. Its + `RasterProxy` PNG **is** read and decoded eagerly so cold scrubs show a low-res + frame immediately. +- **Image assets** are **not** eagerly read; they page from their `ImageAsset` rows + on demand. +- **Thumbnail** packs are read eagerly per video clip. + +### 10.4 Missing referenced files + +A pool entry that is neither packed nor embedded, whose `relative_path` resolves to a +non-existent file, MUST be reported as a missing file (non-fatal). The host +application prompts the user to relocate it. + +## 11. Legacy ZIP format (pre-SQLite) + +Files whose magic is not `SQLite format 3\0` are read as a ZIP archive with this +internal layout: + +- `project.json` — the same serialized `BeamProject` (`version` must be `"1.0.0"`). +- `media/audio/.` — audio source files. The codec is taken from the + extension. **FLAC entries are decoded to PCM `f32` and re-encoded in memory as + IEEE-float WAV (WAV format tag 3)** before being base64-embedded; other formats are + embedded as-is. The entry's `relative_path` is cleared after extraction. +- `media/raster/.png` — raster keyframe pixels, named by + `RasterKeyframe.media_path`. + +A reader MUST NOT modify a legacy ZIP on open; it loads into memory only. **The next +save migrates the project to SQLite** (the save sees a non-SQLite target, so it takes +the create-and-rename path and writes a fresh `.beam` database). + +> Known limitation of the legacy reader: raster pixels are loaded only for +> **top-level** layers; raster keyframes nested inside groups/clips are not populated +> from a ZIP. The SQLite path recurses correctly. Re-saving to SQLite is the remedy. + +## 12. Large-media policy + +- `LARGE_MEDIA_THRESHOLD = 2 GiB`. Files at or above it trigger the packed-vs- + referenced decision below; smaller files are always packed (unless video audio, + which is always referenced). +- `LargeMediaMode` ∈ `{ Ask, Pack, Reference }`: + - `Pack` — pack large files into the database. + - `Reference` — store large files by external path. + - `Ask` (default) — prompt the user on the first large import, then persist their + choice; treated as `Reference` at save time until answered. +- **The chosen mode is an application preference, not part of the `.beam` file.** It + is stored in the editor's `config.json` (`AppConfig.large_media_default`) under the + platform config directory, not in the project. Readers/writers of the format need + not consult it except to drive the packed/referenced save decision. + +## 13. Conformance summary + +A conforming **reader** MUST: + +1. dispatch on the 16-byte magic (§3); +2. reject `schema_version` greater than supported and `version != "1.0.0"` (§4.2); +3. resolve packed media via `media`/`media_chunk` (§6.3) and referenced media + relative to the file (§6.4), reporting missing referenced files non-fatally; +4. compute derived media ids exactly as in §7. + +A conforming **writer** MUST: + +1. produce the four tables of §4 with `schema_version="1"` and `version="1.0.0"`; +2. store UUIDs as 16 big-endian bytes and chunk packed media at a uniform size with a + remainder final chunk, setting `total_len` correctly (§6.3); +3. perform each save in a single transaction and garbage-collect orphaned media (§9), + and prefer in-place writes that do not rewrite unchanged packed media. + +## 14. Security considerations + +- **Path traversal:** referenced media and legacy ZIP entries carry filesystem paths. + Readers SHOULD resolve relative paths against the project directory and SHOULD + reject or sandbox paths that escape it (`..`, absolute paths) when opening + untrusted files. +- **Resource limits:** `project.json` and packed blobs are attacker-controllable in + an untrusted file. Readers SHOULD bound JSON size and avoid loading entire large + blobs into memory (stream via §6.3) to resist memory-exhaustion. +- **Decoder safety:** audio/image bytes are handed to media decoders (Symphonia, + image, FFmpeg); keep those dependencies current, as they parse untrusted input. +- **Legacy ZIP bombs:** the ZIP path SHOULD enforce sane decompression-ratio/size + limits. + +## 15. Non-normative notes / known quirks + +These reflect the current reference implementation and are flagged for implementers +and future spec revisions: + +- `SerializedAudioBackend.sample_rate` is hardcoded to `48000` on save and SHOULD be + ignored in favour of `AudioProject.sample_rate` / `AudioPoolEntry.sample_rate`. +- `MediaKind::Video` (`1`) is defined but never written; video media is always an + external file via `VideoClip.file_path`. +- `RasterKeyframe.media_path` is serialized but unused by the SQLite path (vestigial + legacy-ZIP linkage). +- The reference writer's `SaveSettings` fields `auto_embed_threshold_bytes`, + `force_embed_all`, and `force_link_all` are not consulted by the current save path; + only the large-media mode is. +- The project `version` check is exact-match with no compatibility window; a future + revision should define semantic-version tolerance before bumping it. + +## 16. Inspecting a `.beam` file + +Because the container is plain SQLite, any `.beam` (SQLite variant) can be inspected +with standard tooling, e.g.: + +```sh +sqlite3 project.beam '.tables' +sqlite3 project.beam 'SELECT key,value FROM meta;' +sqlite3 project.beam 'SELECT hex(id),kind,codec,storage,total_len FROM media;' +sqlite3 project.beam 'SELECT data FROM project_json;' | jq . +``` + +## 17. References + +- Container: `lightningbeam-ui/lightningbeam-core/src/beam_archive.rs` +- Save/load orchestration & `BeamProject`: `lightningbeam-ui/lightningbeam-core/src/file_io.rs` +- Audio pool entry: `daw-backend/src/audio/pool.rs` +- Document / scene tree: `lightningbeam-ui/lightningbeam-core/src/document.rs` +- RFC 2119 (requirement keywords), RFC 4122 (UUID), RFC 3339 (timestamps) diff --git a/beam_inspector.py b/beam_inspector.py new file mode 100755 index 0000000..1ca5cfc --- /dev/null +++ b/beam_inspector.py @@ -0,0 +1,701 @@ +#!/usr/bin/env python3 +""" +Lightningbeam .beam File Inspector + +A command-line tool to inspect .beam project files. +""" + +import argparse +import json +import sqlite3 +import sys +import uuid as uuidlib +import zipfile +from datetime import datetime +from pathlib import Path +from typing import Any, Dict, List, Optional + + +# First 16 bytes of any SQLite 3 database. +SQLITE_MAGIC = b"SQLite format 3\x00" + +# media.kind values (see BEAM_FILE_FORMAT.md §6.1). +MEDIA_KIND_NAMES = { + 0: "Audio", 1: "Video", 2: "Raster", 3: "ImageAsset", + 4: "Waveform", 5: "Thumbnail", 6: "RasterProxy", +} +# media.storage values (§6.2). +MEDIA_STORAGE_NAMES = {0: "Packed", 1: "Referenced"} + + +def _is_sqlite(path: Path) -> bool: + """True if the file begins with the SQLite 3 header magic.""" + try: + with open(path, "rb") as f: + return f.read(16) == SQLITE_MAGIC + except OSError: + return False + + +def _human_size(n: int) -> str: + f = float(n) + for unit in ("B", "KiB", "MiB", "GiB", "TiB"): + if f < 1024 or unit == "TiB": + return f"{f:.0f} {unit}" if unit == "B" else f"{f:.1f} {unit}" + f /= 1024 + return f"{n} B" + + +class BeamInspector: + """Inspector for .beam project files (SQLite container, or legacy ZIP).""" + + def __init__(self, beam_file: Path): + self.beam_file = beam_file + self.project_data: Optional[Dict[str, Any]] = None + # SQLite (current) vs ZIP (legacy) container. + self.is_sqlite = _is_sqlite(beam_file) + # Populated for SQLite files by load(): + self.meta: Dict[str, str] = {} + self.media: List[Dict[str, Any]] = [] # rows from the media table + self.media_by_id: Dict[str, Dict[str, Any]] = {} # uuid-string -> row + + def _connect(self) -> sqlite3.Connection: + """Open the SQLite container read-only.""" + uri = self.beam_file.resolve().as_uri() + "?mode=ro" + return sqlite3.connect(uri, uri=True) + + def load(self) -> bool: + """Load and parse the .beam file (container-agnostic).""" + try: + if self.is_sqlite: + return self._load_sqlite() + return self._load_zip() + except Exception as e: + print(f"Error loading .beam file: {e}", file=sys.stderr) + return False + + def _load_zip(self) -> bool: + with zipfile.ZipFile(self.beam_file, 'r') as zip_ref: + with zip_ref.open('project.json') as f: + self.project_data = json.load(f) + return True + + def _load_sqlite(self) -> bool: + con = self._connect() + try: + cur = con.cursor() + row = cur.execute("SELECT data FROM project_json WHERE id = 0").fetchone() + if not row: + raise ValueError("archive has no project.json row") + self.project_data = json.loads(row[0]) + + self.meta = {k: v for (k, v) in cur.execute("SELECT key, value FROM meta")} + + for (idb, kind, codec, storage, ext_path, total_len, + channels, sample_rate, width, height) in cur.execute( + "SELECT id, kind, codec, storage, ext_path, total_len, " + "channels, sample_rate, width, height FROM media" + ): + uid = str(uuidlib.UUID(bytes=bytes(idb))) + info = { + "uuid": uid, + "kind": kind, + "codec": codec, + "storage": storage, + "ext_path": ext_path, + "total_len": total_len or 0, + "channels": channels, + "sample_rate": sample_rate, + "width": width, + "height": height, + } + self.media.append(info) + self.media_by_id[uid] = info + finally: + con.close() + return True + + def show_info(self): + """Display basic project information.""" + if not self.project_data: + return + + print("=" * 60) + print("PROJECT INFORMATION") + print("=" * 60) + + print(f"Container: {'SQLite' if self.is_sqlite else 'Legacy ZIP'}") + if self.is_sqlite: + print(f"Schema Ver: {self.meta.get('schema_version', 'Unknown')}") + print(f"Version: {self.project_data.get('version', 'Unknown')}") + print(f"Created: {self.project_data.get('created', 'Unknown')}") + print(f"Modified: {self.project_data.get('modified', 'Unknown')}") + + ui_state = self.project_data.get('ui_state', {}) + print(f"\nProject Name: {ui_state.get('name', 'Unnamed')}") + print(f"ID: {ui_state.get('id', 'Unknown')}") + print(f"Dimensions: {ui_state.get('width', 0):.0f} x {ui_state.get('height', 0):.0f}") + print(f"Framerate: {ui_state.get('framerate', 0):.1f} fps") + print(f"Duration: {ui_state.get('duration', 0):.2f} seconds") + + bg = ui_state.get('background_color', {}) + print(f"Background: rgba({bg.get('r', 0)}, {bg.get('g', 0)}, {bg.get('b', 0)}, {bg.get('a', 255)})") + + audio_backend = self.project_data.get('audio_backend', {}) + print(f"\nSample Rate: {audio_backend.get('sample_rate', 0)} Hz") + + def show_clips(self): + """Display clips information.""" + if not self.project_data: + return + + ui_state = self.project_data.get('ui_state', {}) + + vector_clips = ui_state.get('vector_clips', {}) + video_clips = ui_state.get('video_clips', {}) + audio_clips = ui_state.get('audio_clips', {}) + + print("\n" + "=" * 60) + print("CLIPS") + print("=" * 60) + + print(f"Vector Clips: {len(vector_clips)}") + print(f"Video Clips: {len(video_clips)}") + print(f"Audio Clips: {len(audio_clips)}") + + if vector_clips: + print("\nVector Clips:") + for clip_id, clip in vector_clips.items(): + print(f" - {clip.get('name', 'Unnamed')} (ID: {clip_id[:8]}...)") + + if video_clips: + print("\nVideo Clips:") + for clip_id, clip in video_clips.items(): + print(f" - {clip.get('name', 'Unnamed')} (ID: {clip_id[:8]}...)") + + if audio_clips: + print("\nAudio Clips:") + for clip_id, clip in audio_clips.items(): + print(f" - {clip.get('name', 'Unnamed')} (ID: {clip_id[:8]}...)") + + def show_layers(self): + """Display layer hierarchy.""" + if not self.project_data: + return + + ui_state = self.project_data.get('ui_state', {}) + root = ui_state.get('root', {}) + + print("\n" + "=" * 60) + print("LAYER HIERARCHY") + print("=" * 60) + + def print_layer(layer: Dict[str, Any], indent: int = 0): + # Handle case where layer might not be a dictionary + if not isinstance(layer, dict): + prefix = " " * indent + print(f"{prefix}- ERROR: Layer is {type(layer).__name__}, not a dict: {repr(layer)[:50]}") + return + + layer_type = layer.get('type', 'Unknown') + layer_name = layer.get('name', 'Unnamed') + layer_id = layer.get('id', 'Unknown') + + prefix = " " * indent + # Handle layer_id that might not be a string + id_str = layer_id[:8] + "..." if isinstance(layer_id, str) and len(layer_id) > 8 else str(layer_id) + print(f"{prefix}- [{layer_type}] {layer_name} (ID: {id_str})") + + # Recursively print children + children = layer.get('children', []) + for child in children: + print_layer(child, indent + 1) + + print_layer(root) + + def show_tracks(self): + """Display audio tracks information.""" + if not self.project_data: + return + + audio_backend = self.project_data.get('audio_backend', {}) + project = audio_backend.get('project', {}) + tracks_dict = project.get('tracks', {}) + root_tracks = project.get('root_tracks', []) + master_track = project.get('master_track', {}) + + print("\n" + "=" * 60) + print("AUDIO TRACKS") + print("=" * 60) + + print(f"Total Tracks: {len(tracks_dict)}") + + # Iterate through root_tracks in order + for track_id in root_tracks: + track_id_str = str(track_id) + if track_id_str not in tracks_dict: + print(f"\n Track {track_id}: ERROR - Track ID not found in tracks dict") + continue + + track_node = tracks_dict[track_id_str] + + # TrackNode is an enum with variants like {"Audio": {...}} or {"Midi": {...}} + if not isinstance(track_node, dict): + print(f"\n Track {track_id}: ERROR - Track node is {type(track_node).__name__}, not a dict") + continue + + # Extract the variant (Audio, Midi, or Group) + if len(track_node) != 1: + print(f"\n Track {track_id}: ERROR - Track node has unexpected structure") + continue + + track_type, track_data = list(track_node.items())[0] + + if not isinstance(track_data, dict): + print(f"\n Track {track_id}: ERROR - Track data is {type(track_data).__name__}, not a dict") + continue + + track_name = track_data.get('name', f'Track {track_id}') + muted = track_data.get('muted', False) + solo = track_data.get('solo', False) + volume = track_data.get('volume', 1.0) + pan = track_data.get('pan', 0.0) + + status = [] + if muted: + status.append('MUTED') + if solo: + status.append('SOLO') + status_str = f" [{', '.join(status)}]" if status else "" + + print(f"\n Track {track_id}: {track_name}{status_str}") + print(f" Type: {track_type}") + print(f" Volume: {volume:.2f}") + print(f" Pan: {pan:.2f}") + + if track_type == "Midi": + print(f" Instrument: {track_data.get('instrument', 'Unknown')}") + notes = track_data.get('notes', []) + print(f" Notes: {len(notes)}") + preset = track_data.get('instrument_graph_preset') + if preset: + nodes = preset.get('nodes', []) + conns = preset.get('connections', []) + print(f" Graph: {len(nodes)} nodes, {len(conns)} connections") + else: + print(f" Graph: (no preset saved)") + elif track_type == "Audio": + clips = track_data.get('clips', []) + print(f" Clips: {len(clips)}") + preset = track_data.get('effects_graph_preset') + if preset: + nodes = preset.get('nodes', []) + conns = preset.get('connections', []) + print(f" Graph: {len(nodes)} nodes, {len(conns)} connections") + else: + print(f" Graph: (no preset saved)") + + print(f"\nMaster Track:") + print(f" Volume: {master_track.get('volume', 1.0):.2f}") + + def show_graphs(self): + """Display detailed node graph information for all tracks.""" + if not self.project_data: + return + + audio_backend = self.project_data.get('audio_backend', {}) + project = audio_backend.get('project', {}) + tracks_dict = project.get('tracks', {}) + + print("\n" + "=" * 60) + print("NODE GRAPHS") + print("=" * 60) + + for track_id_str, track_node in tracks_dict.items(): + if not isinstance(track_node, dict) or len(track_node) != 1: + continue + + track_type, track_data = list(track_node.items())[0] + track_name = track_data.get('name', f'Track {track_id_str}') + + # Get the appropriate preset + if track_type == "Audio": + preset = track_data.get('effects_graph_preset') + graph_label = "Effects Graph" + elif track_type == "Midi": + preset = track_data.get('instrument_graph_preset') + graph_label = "Instrument Graph" + else: + continue + + print(f"\n Track {track_id_str}: {track_name} ({track_type}) - {graph_label}") + + if not preset: + print(f" ** NO PRESET SAVED **") + continue + + nodes = preset.get('nodes', []) + connections = preset.get('connections', []) + output_node = preset.get('output_node') + midi_targets = preset.get('midi_targets', []) + metadata = preset.get('metadata', {}) + + if metadata: + print(f" Preset Name: {metadata.get('name', 'Unknown')}") + + print(f" Nodes ({len(nodes)}):") + for i, node in enumerate(nodes): + node_type = node.get('node_type', '?') + node_name = node.get('name', node_type) + params = node.get('parameters', {}) + pos_x = node.get('position_x', 0) + pos_y = node.get('position_y', 0) + + markers = [] + if output_node is not None and i == output_node: + markers.append("OUTPUT") + if i in midi_targets: + markers.append("MIDI TARGET") + marker_str = f" [{', '.join(markers)}]" if markers else "" + + print(f" [{i}] {node_type}{marker_str} (name={node_name}, pos={pos_x:.0f},{pos_y:.0f})") + + if params: + for param_name, param_val in params.items(): + print(f" {param_name} = {param_val}") + + print(f" Connections ({len(connections)}):") + for conn in connections: + src = conn.get('from_node', '?') + src_port = conn.get('from_output', '?') + dst = conn.get('to_node', '?') + dst_port = conn.get('to_input', '?') + print(f" [{src}]:{src_port} -> [{dst}]:{dst_port}") + + # Also show layer_to_track_map if present + track_map = audio_backend.get('layer_to_track_map', {}) + if track_map: + print(f"\n Layer-to-Track Mapping ({len(track_map)} entries):") + for layer_id, track_id in track_map.items(): + print(f" {layer_id[:8]}... -> Track {track_id}") + else: + print(f"\n Layer-to-Track Mapping: (none saved)") + + def show_audio_pool(self): + """Display audio pool entries.""" + if not self.project_data: + return + + audio_backend = self.project_data.get('audio_backend', {}) + pool_entries = audio_backend.get('audio_pool_entries', []) + + print("\n" + "=" * 60) + print("AUDIO POOL") + print("=" * 60) + + print(f"Total Entries: {len(pool_entries)}") + + for entry in pool_entries: + pool_index = entry.get('pool_index', '?') + name = entry.get('name', 'Unnamed') + relative_path = entry.get('relative_path') + media_id = entry.get('media_id') + channels = entry.get('channels', 0) + sample_rate = entry.get('sample_rate', 0) + has_embedded = entry.get('embedded_data') is not None + + # Storage precedence matches the loader (§8.5/§9.3): + # packed (media_id) > external (relative_path) > embedded > unresolved. + row = self.media_by_id.get(media_id) if media_id else None + if media_id: + size = f", {_human_size(row['total_len'])}" if row else "" + codec = f", {row['codec']}" if row else "" + storage_type = f"Packed in DB{codec}{size}" + elif relative_path and not self.is_sqlite and relative_path.startswith("media/audio/"): + # Legacy ZIP: media/audio/* lives inside the archive, not external. + storage_type = "Embedded (in ZIP)" + elif relative_path: + storage_type = "External reference" + elif has_embedded: + storage_type = "Embedded (inline base64)" + else: + storage_type = "Unresolved (missing)" + + print(f"\n [{pool_index}] {name}") + if media_id: + print(f" Media ID: {media_id}") + print(f" Path: {relative_path if relative_path else 'N/A'}") + print(f" Storage: {storage_type}") + print(f" Channels: {channels}") + print(f" Sample Rate: {sample_rate} Hz") + + def show_media(self): + """Display the SQLite media store (the `media` table).""" + if not self.is_sqlite: + print("\n(media table view applies to SQLite .beam files; " + "use --zip for legacy archives)", file=sys.stderr) + return + + print("\n" + "=" * 60) + print("MEDIA STORE") + print("=" * 60) + + print(f"Total media rows: {len(self.media)}") + + # Summary by kind. + by_kind: Dict[str, List[Dict[str, Any]]] = {} + for m in self.media: + by_kind.setdefault(MEDIA_KIND_NAMES.get(m["kind"], f"Kind {m['kind']}"), []).append(m) + if by_kind: + print("\nBy kind:") + for kind_name, rows in sorted(by_kind.items()): + packed = sum(r["total_len"] for r in rows if r["storage"] == 0) + print(f" {kind_name:<12} {len(rows):>4} ({_human_size(packed)} packed)") + + print(f"\n{'Kind':<12} {'Storage':<11} {'Codec':<6} {'Size':>10} UUID / details") + print("-" * 78) + for m in self.media: + kind = MEDIA_KIND_NAMES.get(m["kind"], f"Kind {m['kind']}") + storage = MEDIA_STORAGE_NAMES.get(m["storage"], f"Stor {m['storage']}") + size = _human_size(m["total_len"]) if m["storage"] == 0 else "-" + detail = m["uuid"] + extra = [] + if m["channels"] is not None: + extra.append(f"{m['channels']}ch@{m['sample_rate']}Hz") + if m["width"] is not None: + extra.append(f"{m['width']}x{m['height']}") + if m["storage"] == 1 and m["ext_path"]: + extra.append(f"-> {m['ext_path']}") + if extra: + detail += " " + " ".join(extra) + print(f"{kind:<12} {storage:<11} {m['codec']:<6} {size:>10} {detail}") + + def show_container(self): + """Show whichever container structure applies to this file.""" + if self.is_sqlite: + self.show_media() + else: + self.show_zip_structure() + + def show_zip_structure(self): + """Display the ZIP file structure.""" + if self.is_sqlite: + print("\n(this is a SQLite .beam; use --media for the media store)", + file=sys.stderr) + return + + print("\n" + "=" * 60) + print("ZIP ARCHIVE STRUCTURE") + print("=" * 60) + + try: + with zipfile.ZipFile(self.beam_file, 'r') as zip_ref: + total_size = 0 + compressed_size = 0 + + print(f"{'File':<40} {'Size':>12} {'Compressed':>12} {'Method':<10}") + print("-" * 80) + + for info in zip_ref.infolist(): + size = info.file_size + comp_size = info.compress_size + method = "DEFLATE" if info.compress_type == 8 else "STORED" if info.compress_type == 0 else f"Type {info.compress_type}" + + total_size += size + compressed_size += comp_size + + print(f"{info.filename:<40} {size:>12,} {comp_size:>12,} {method:<10}") + + print("-" * 80) + print(f"{'TOTAL':<40} {total_size:>12,} {compressed_size:>12,}") + + if total_size > 0: + ratio = (1 - compressed_size / total_size) * 100 + print(f"\nCompression Ratio: {ratio:.1f}%") + + except Exception as e: + print(f"Error reading ZIP structure: {e}", file=sys.stderr) + + def extract_json(self, output_path: Optional[Path] = None): + """Extract project.json to a file or stdout.""" + try: + if self.is_sqlite: + con = self._connect() + try: + row = con.execute("SELECT data FROM project_json WHERE id = 0").fetchone() + finally: + con.close() + if not row: + raise ValueError("archive has no project.json row") + json_data = row[0].encode("utf-8") if isinstance(row[0], str) else row[0] + else: + with zipfile.ZipFile(self.beam_file, 'r') as zip_ref: + json_data = zip_ref.read('project.json') + + if output_path: + output_path.write_bytes(json_data) + print(f"Extracted project.json to: {output_path}") + else: + data = json.loads(json_data) + print(json.dumps(data, indent=2)) + + except Exception as e: + print(f"Error extracting project.json: {e}", file=sys.stderr) + + def extract_media(self, output_dir: Path): + """Extract all media to a directory. + + SQLite: each packed media row is reassembled from its chunks and written + as ``.``; referenced rows are reported, not copied. + ZIP (legacy): the ``media/`` entries are extracted preserving paths. + """ + try: + if self.is_sqlite: + self._extract_media_sqlite(output_dir) + else: + self._extract_media_zip(output_dir) + except Exception as e: + print(f"Error extracting media: {e}", file=sys.stderr) + + def _extract_media_sqlite(self, output_dir: Path): + con = self._connect() + try: + rows = con.execute( + "SELECT id, kind, codec, storage, ext_path, total_len FROM media" + ).fetchall() + if not rows: + print("No media rows found in archive.") + return + output_dir.mkdir(parents=True, exist_ok=True) + written = 0 + for (idb, kind, codec, storage, ext_path, total_len) in rows: + uid = str(uuidlib.UUID(bytes=bytes(idb))) + kind_name = MEDIA_KIND_NAMES.get(kind, f"kind{kind}") + if storage == 1: # Referenced + print(f"Referenced (not copied): {uid} [{kind_name}] -> {ext_path}") + continue + chunks = con.execute( + "SELECT bytes FROM media_chunk WHERE media_id = ? ORDER BY chunk_index", + (idb,), + ).fetchall() + data = b"".join(bytes(c[0]) for c in chunks) + out = output_dir / f"{uid}.{codec}" + out.write_bytes(data) + print(f"Extracted: {out.name} [{kind_name}] {_human_size(len(data))}") + written += 1 + print(f"\nExtracted {written} packed media item(s) to: {output_dir}") + finally: + con.close() + + def _extract_media_zip(self, output_dir: Path): + with zipfile.ZipFile(self.beam_file, 'r') as zip_ref: + media_files = [f for f in zip_ref.namelist() if f.startswith('media/')] + if not media_files: + print("No media files found in archive.") + return + output_dir.mkdir(parents=True, exist_ok=True) + for media_file in media_files: + output_path = output_dir / media_file + output_path.parent.mkdir(parents=True, exist_ok=True) + with zip_ref.open(media_file) as source: + output_path.write_bytes(source.read()) + print(f"Extracted: {media_file}") + print(f"\nExtracted {len(media_files)} media file(s) to: {output_dir}") + + +def main(): + parser = argparse.ArgumentParser( + description="Inspect Lightningbeam .beam project files", + formatter_class=argparse.RawDescriptionHelpFormatter, + epilog=""" +Examples: + %(prog)s project.beam # Show all information + %(prog)s project.beam --info # Show only basic info + %(prog)s project.beam --tracks # Show only tracks + %(prog)s project.beam --media # Show the SQLite media store + %(prog)s project.beam --zip # Show legacy ZIP structure + %(prog)s project.beam --extract-json # Print project.json to stdout + %(prog)s project.beam --extract-json out.json # Save project.json + %(prog)s project.beam --extract-media ./media # Extract media files + +Handles both current SQLite .beam files and legacy ZIP .beam files +(detected automatically by the file's magic bytes). + """ + ) + + parser.add_argument('beam_file', type=Path, help='.beam file to inspect') + parser.add_argument('--info', action='store_true', help='Show project information') + parser.add_argument('--clips', action='store_true', help='Show clips') + parser.add_argument('--layers', action='store_true', help='Show layer hierarchy') + parser.add_argument('--tracks', action='store_true', help='Show audio tracks') + parser.add_argument('--graphs', action='store_true', help='Show node graph details for all tracks') + parser.add_argument('--pool', action='store_true', help='Show audio pool') + parser.add_argument('--media', action='store_true', help='Show the SQLite media store') + parser.add_argument('--zip', action='store_true', help='Show legacy ZIP structure') + parser.add_argument('--extract-json', nargs='?', const=True, metavar='OUTPUT', + help='Extract project.json (to file or stdout)') + parser.add_argument('--extract-media', type=Path, metavar='DIR', + help='Extract media files to directory') + + args = parser.parse_args() + + # Validate input file + if not args.beam_file.exists(): + print(f"Error: File not found: {args.beam_file}", file=sys.stderr) + sys.exit(1) + + if not args.beam_file.suffix == '.beam': + print(f"Warning: File does not have .beam extension: {args.beam_file}", file=sys.stderr) + + # Create inspector + inspector = BeamInspector(args.beam_file) + + # Handle extract operations (don't need to load project.json) + if args.extract_json: + if args.extract_json is True: + inspector.extract_json() + else: + inspector.extract_json(Path(args.extract_json)) + return + + if args.extract_media: + inspector.extract_media(args.extract_media) + return + + # Load the beam file + if not inspector.load(): + sys.exit(1) + + # If no specific flags, show everything + show_all = not any([args.info, args.clips, args.layers, args.tracks, + args.graphs, args.pool, args.media, args.zip]) + + if show_all or args.info: + inspector.show_info() + + if show_all or args.clips: + inspector.show_clips() + + if show_all or args.layers: + inspector.show_layers() + + if show_all or args.tracks: + inspector.show_tracks() + + if show_all or args.graphs: + inspector.show_graphs() + + if show_all or args.pool: + inspector.show_audio_pool() + + if show_all: + # Show whichever container structure applies to this file. + inspector.show_container() + elif args.media: + inspector.show_media() + elif args.zip: + inspector.show_zip_structure() + + +if __name__ == '__main__': + main() diff --git a/beam_inspector_README.md b/beam_inspector_README.md new file mode 100644 index 0000000..339ad4c --- /dev/null +++ b/beam_inspector_README.md @@ -0,0 +1,210 @@ +# Lightningbeam .beam File Inspector + +A Python command-line tool to inspect and analyze `.beam` project files. + +Handles **both** container formats automatically (detected by the file's magic +bytes): the current **SQLite** `.beam` and the **legacy ZIP** `.beam`. See +[BEAM_FILE_FORMAT.md](./BEAM_FILE_FORMAT.md) for the format specification. + +## Features + +- **Project Information**: Container type, schema version, metadata, dimensions, framerate, duration, background color +- **Clips Analysis**: List all vector, video, and audio clips +- **Layer Hierarchy**: Display the complete layer tree structure +- **Audio Tracks**: Show all audio/MIDI tracks with their settings +- **Audio Pool**: List all audio files and their storage details +- **Media Store** (SQLite): List the `media` table — kinds, storage, codecs, sizes +- **ZIP Structure** (legacy): Examine the internal ZIP archive structure and compression +- **Extraction**: Extract `project.json` or media files + +## Installation + +The tool requires Python 3.6+ with no external dependencies (uses only the standard +library; `sqlite3` ships with Python). + +```bash +chmod +x beam_inspector.py +``` + +## Usage + +### Show All Information +```bash +./beam_inspector.py project.beam +``` + +### Show Specific Sections +```bash +# Basic project info only +./beam_inspector.py project.beam --info + +# Audio tracks only +./beam_inspector.py project.beam --tracks + +# Audio pool entries +./beam_inspector.py project.beam --pool + +# Layer hierarchy +./beam_inspector.py project.beam --layers + +# Clips summary +./beam_inspector.py project.beam --clips + +# Media store (SQLite files): the media table — kinds, storage, codecs, sizes +./beam_inspector.py project.beam --media + +# ZIP archive structure (legacy ZIP files only) +./beam_inspector.py project.beam --zip +``` + +When run with no section flags, the tool shows everything and automatically +picks the media-store view (SQLite) or ZIP-structure view (legacy) for the file. + +### Extract Files +```bash +# Print project.json to stdout (pretty-printed) +./beam_inspector.py project.beam --extract-json + +# Save project.json to a file +./beam_inspector.py project.beam --extract-json output.json + +# Extract all media files to a directory +./beam_inspector.py project.beam --extract-media ./extracted_media +``` + +## Example Output + +``` +============================================================ +PROJECT INFORMATION +============================================================ +Container: SQLite +Schema Ver: 1 +Version: 1.0.0 +Created: 2025-12-01T12:00:00Z +Modified: 2025-12-01T12:30:00Z + +Project Name: My Animation +ID: 550e8400-e29b-41d4-a716-446655440000 +Dimensions: 1920 x 1080 +Framerate: 60.0 fps +Duration: 10.00 seconds +Background: rgba(255, 255, 255, 255) + +Sample Rate: 48000 Hz + +============================================================ +AUDIO TRACKS +============================================================ +Total Tracks: 2 + + Track 0: Piano [SOLO] + Type: Midi + Volume: 0.80 + Pan: 0.00 + Instrument: Piano + Notes: 24 + + Track 1: Background Music + Type: Audio + Volume: 0.60 + Pan: 0.00 + Clips: 3 + +Master Track: + Volume: 1.00 + +============================================================ +AUDIO POOL +============================================================ +Total Entries: 2 + + [0] C2.mp3 + Media ID: e7e555a6-85f1-4faa-bfd4-c6e4b790986a + Path: N/A + Storage: Packed in DB, mp3, 412.0 KiB + Channels: 2 + Sample Rate: 44100 Hz + + [1] Background.flac + Media ID: a18c0e22-1d3b-4c77-9f0a-2b5e6c4d8e10 + Path: N/A + Storage: Packed in DB, flac, 6.4 MiB + Channels: 2 + Sample Rate: 48000 Hz +``` + +## Understanding the Output + +### Storage Types +- **Packed in DB**: Audio bytes are chunked into the SQLite `media` table (the entry's `media_id` resolves to a packed row). Current default for most audio. +- **External reference**: File is referenced from the filesystem by `relative_path` (e.g. large media or video audio). +- **Embedded (inline base64)**: Bytes stored directly in `project.json` (`embedded_data`) — legacy/fallback. +- **Embedded (in ZIP)**: Legacy ZIP files only — bytes stored inside the archive under `media/audio/`. +- **Unresolved (missing)**: No packed row, external file, or embedded data — reported as a missing file on load. + +### Track Types +- **Audio**: Traditional audio track with clips from the audio pool +- **Midi**: MIDI track with note events and virtual instrument + +### Layer Types +Based on the `AnyLayer` enum: +- **Group**: Container for other layers +- **Vector**: Vector graphics layer +- **Video**: Video clip layer +- **Audio**: Audio waveform layer +- **Image**: Raster image layer +- **Text**: Text layer + +## Advanced Usage + +### Combine with Other Tools +```bash +# Pretty-print and page through JSON +./beam_inspector.py project.beam --extract-json | less + +# Search for specific content in JSON +./beam_inspector.py project.beam --extract-json | grep "sample_rate" + +# Count total audio files +./beam_inspector.py project.beam --pool | grep "^\[" | wc -l + +# Extract and process media files +# SQLite: writes flat . files; ZIP: preserves media/ paths +./beam_inspector.py project.beam --extract-media /tmp/media +ls -lh /tmp/media/ +``` + +### Scripting +```python +from beam_inspector import BeamInspector +from pathlib import Path + +inspector = BeamInspector(Path("project.beam")) +if inspector.load(): + # Access parsed data + version = inspector.project_data['version'] + tracks = inspector.project_data['audio_backend']['project']['tracks'] + print(f"Found {len(tracks)} tracks in version {version}") +``` + +## Troubleshooting + +### "Error loading .beam file" +- Ensure the file is a valid SQLite database (current) or ZIP archive (legacy) +- Check that `project.json` exists (the `project_json` table for SQLite, or a top-level entry for ZIP) +- Verify the JSON is well-formed + +### "File not found" +- Provide the full path to the .beam file +- Check file permissions + +### Missing media files +- External references may point to files that don't exist +- Use `--extract-media` to see what's actually in the archive +- Check the `relative_path` values in `--pool` output + +## See Also + +- [BEAM_FILE_FORMAT.md](./BEAM_FILE_FORMAT.md) - Complete .beam file format specification +- [Lightningbeam Documentation](./README.md) - Main project documentation