Compare commits

..

No commits in common. "main" and "feat/mobile-ui" have entirely different histories.

84 changed files with 4201 additions and 9177 deletions

View File

@ -54,27 +54,6 @@ jobs:
with:
targets: ${{ matrix.target != '' && matrix.target || '' }}
# A [patch.crates-io] whose version doesn't satisfy the dependency requirement is silently
# IGNORED by cargo: it warns "patch ... was not used in the crate graph" and builds against the
# real crates.io crate instead. That's how a stale egui fork got us a `cannot find type
# CursorImage` error four minutes into the build, in a file that had nothing to do with the
# cause. Fail here instead, with the actual reason.
- name: Verify egui fork patch applies
shell: bash
working-directory: ./lightningbeam-ui
run: |
unused=$(cargo metadata --format-version 1 2>&1 >/dev/null | grep "was not used in the crate graph" || true)
if [ -n "$unused" ]; then
echo "::error::The egui fork's [patch.crates-io] entries are being ignored — cargo is building against stock crates.io egui, not the fork."
echo "$unused"
echo ""
echo "The fork's version must SATISFY the requirement in lightningbeam-ui/Cargo.toml."
echo "Fork workspace version: $(grep -m1 '^version' ../egui-fork/Cargo.toml)"
echo "Required by us: $(grep -m1 '^eframe' Cargo.toml)"
exit 1
fi
echo "egui fork patch applies cleanly."
- name: Rust cache
uses: swatinem/rust-cache@v2
with:

5
.gitignore vendored
View File

@ -38,7 +38,4 @@ packaging/AppDir/
packaging/squashfs-root/
# Wrapper script (generated, not needed with static FFmpeg)
lightningbeam-ui/lightningbeam-editor/debian/
# Personal working notes (not version-controlled)
TODO.md
lightningbeam-ui/lightningbeam-editor/debian/

View File

@ -1,53 +1,3 @@
# 1.0.10-alpha:
Changes:
- Cycle recording: highlight a range on the timeline ruler (the cycle strip, shown when looping is armed) and the transport loops it. Recording round the loop turns each pass into a separate take you can choose between afterwards, and recording again over the same region adds more takes rather than stacking a second clip on top. Starting playback from outside the region jumps to its start
- Takes: a "Take N/M" badge on the clip picks which take plays. Splitting a clip lets each half play a different take, so you can comp a part together from the best bits of several passes. Right-click a clip to delete the take that's playing or to delete all the unused ones; double-click a take in the list to rename it
- MIDI cycle recording can either merge every pass into one clip (the default — earlier passes play back as you record, so you can layer a hi-hat over a kick) or keep each pass as a separate take, like audio does. Set it in Preferences → Audio
- Painting: every colour-using tool (brush, paint bucket, eyedropper) now shares one Foreground/Background swatch row, and the eyedropper has an explicit toggle for which of the two it fills
- Painting: the effect brushes (dodge/burn, sponge, blur, smudge, clone stamp, healing brush, pattern stamp) were stuck on a single fixed brush shape. They now each have their own size, strength, hardness and spacing, and can use any brush from the library
- Tablet: stylus barrel buttons now work, with actions you can bind in Preferences (Pan, Eyedropper, or Eraser). Middle-mouse panning added too
- Timeline: raster and text layers get an opacity slider; each layer row now shows only the controls that mean anything for it (volume for audio, opacity for raster/text, both for vector and video), and layers gain a visibility (eye) toggle
- The stage gains undo/redo buttons, sized for touch and tablet use
Bugfixes:
- Splitting an audio clip made it play back at the wrong length (a clip split at 1 second cut off after half a second at 120 BPM)
- Trimmed MIDI clips were the wrong length on the timeline at any tempo other than 60 BPM
- The paint bucket always filled with the background colour, whatever the brush was set to
- Where an eyedropper sample landed (foreground or background) depended on which colour picker you had opened last
- Stylus barrel buttons were silently ignored on Wayland
- The opacity slider on a raster layer actually changed its volume, which means nothing on a raster layer
- Tool cursors lagged behind the pointer while drawing; they're now drawn by the system rather than into the canvas
- Colour picker popups wouldn't close when you dragged on the stage, so drawing left one hanging open, and clicking inside a picker's own hue slider closed it
- The toolbar could not be scrolled when the pane was too short to show every tool
# 1.0.9-alpha:
Bugfixes:
- Fix audio recording placement: a second recording landed at the wrong spot (and clicking/dragging clips was similarly off) at any tempo other than 60 BPM — recordings now start exactly at the playhead at any tempo
- While recording a second audio clip, the live bar showed as a zero-length clip until you stopped; it now grows as you record
- MIDI clips were drawn with the wrong length (their end grew too fast) at tempos other than 120 BPM
- Recording a MIDI clip didn't mark the project as having unsaved changes, so closing or starting a new file didn't prompt to save
- Audio and MIDI recording can now be undone (and redone)
# 1.0.8-alpha:
Changes:
- Mobile/touch UI (experimental, testing only — not built or packaged for mobile yet; enabled on desktop with the LB_MOBILE_UI environment variable): work-in-progress phone-friendly interface with a vertical sliding-window pane stack you drag to reveal panes, a new-file intent picker, a selection inspector sheet, a keyboard-primary music surface, a Focus/Patch node editor, long-press context menus, a command palette, and landscape/orientation support
- Text layers: add and edit text with a chosen font; non-bundled fonts are embedded in the project so it renders on machines that lack them
- Animated GIF export (parallel palette encoding)
- Audio tag metadata (title, artist, album, genre, year, track, comment) is written into exports — ID3v2 for MP3, iTunes/MP4 atoms for AAC, Vorbis comments for FLAC, RIFF INFO for WAV — with sensible defaults (year, artist/album remembered between exports)
- Lossy WebP image export, so the quality control now actually applies
- SVG export now includes text layers, as real font-independent glyph outlines
- Crash recovery: the editor autosaves your work to a recovery file in the background and offers to restore it after an unclean shutdown
- Prompt to save unsaved changes before starting a new file, opening another, or quitting
- Faster saves on painting projects: unchanged raster frames are no longer re-encoded every save
Bugfixes:
- FLAC export previously wrote a WAV file with a .flac extension; it now encodes real (compressed) FLAC
- ProRes 422 export always failed; it now encodes 10-bit 4:2:2 correctly
- Fix VP8 video export with audio (muxed into WebM instead of an incompatible container)
- The WebP "Quality" slider had no effect
- Starting a new file now fully resets the audio engine, so instruments and voices from the previous project no longer linger
- Fix oscillator/synth phase drift over long playback
# 1.0.7-alpha:
Changes:
- HDR video support: PQ/HLG/BT.2020 video is now read correctly (decoded to scene-linear), with a per-document output mode (clip vs highlight rolloff) and 10-bit HDR export (HEVC Main10, PQ or HLG)

View File

@ -58,21 +58,13 @@ Lightningbeam is developed on the `main` branch. The project has been rewritten
- ✅ Audio engine with node graph processing
- ✅ GPU waveform rendering with mipmaps
- ✅ Video decoding integration
- ✅ Export system (video, image, audio, animated GIF, SVG)
- ✅ Node editor UI
- 🚧 Piano roll editor (in progress)
- 🚧 Export system (in progress)
- 🚧 Node editor UI (planned)
- 🚧 Piano roll editor (planned)
## Getting Started
### Download a prebuilt release
If you don't want to build from source, prebuilt binaries for each release are on the
[GitHub releases page](https://github.com/skykooler/lightningbeam/releases). Download the build for
your platform and run it directly.
### Building from source
#### Prerequisites
### Prerequisites
- Rust (stable toolchain via [rustup](https://rustup.rs/))
- System dependencies:
@ -82,7 +74,7 @@ your platform and run it directly.
See [docs/BUILDING.md](docs/BUILDING.md) for detailed setup instructions.
#### Building and Running
### Building and Running
```bash
# Clone the repository

859
STREAMING_TO_DISK_PLAN.md Normal file
View File

@ -0,0 +1,859 @@
# Streaming Media To/From Disk — Plan
**Goal:** Lightningbeam must handle audio and video files (and raster animation, and
image assets) of *arbitrary length/size*. Anywhere we touch media we should stream from
and to disk when the data is too large to fit comfortably in memory, rather than loading
the entire file regardless of size.
**Scope of this document:** audio, video, raster frames, image-asset paging, **and the
`.beam` container format** — these turned out to be one problem, not two. Streaming on load
is impossible while the container forces a full decode, so the container decision (below)
is now part of this plan.
## Deferred bugs (do at the end)
- [x] **Timeline thumbnail scroll (FIXED):** the strip tiled from the *clamped* visible-left of the
clip, so when a clip was scrolled partly off the left it showed the clip's start content at the
viewport edge. Now tiled from the clip's **true (unclamped) origin** over its full width, drawing
only the tiles intersecting the visible rect (`draw_video_thumbnail_strip` in timeline.rs). Both
render sites (collapsed-group + expanded-track) share the helper. *(Compiles; needs in-app check.)*
- [x] **Clip thumbnails stop updating (FIXED):** the GPU texture cache was keyed by the *requested*
content time, so once a tile cached the first (often far-off) thumbnail it never refreshed as
closer ones loaded. `VideoManager::get_thumbnail_at` now also returns the **actual** thumbnail
timestamp, and the cache keys on that — so a tile picks up a new texture when a closer thumbnail
finishes generating. Existing `retain`-by-visible-clip cleanup keeps it bounded. *(Needs in-app check.)*
## Raster-keyframe-UI bugs — **[DONE]** (built the raster keyframe timeline UI, 2026-06-20)
Both resolved by the raster-keyframe-timeline-UI work: timeline now draws a diamond per
`RasterKeyframe` (mirrors vector), `K`/New Keyframe inserts a blank cel via `AddRasterKeyframeAction`
(canvas refreshes), paint tools edit the active keyframe instead of lazily creating, diamonds are
click-to-seek (pointing-hand cursor), playback prefetches frames, and onion skinning (raster+vector,
tinted, Info-Panel settings) is in. (a) canvas-refresh-on-new-keyframe and (b) keyframes-on-timeline
are both fixed.
## Noted enhancements (later, after the phases)
- [x] **Surround → stereo downmix (DONE).** Done uniformly in `render_from_file` (`pool.rs`) so it
covers every storage type (PCM/InMemory, compressed via symphonia, video-audio via ffmpeg — all
flow through this mixer with the source kept multichannel in the read-ahead buffer). New
`stereo_downmix_matrix(src_channels)` gives `[L][src]`/`[R][src]` coefficients for the conventional
interleave order (FL FR FC LFE BL BR SL SR…) for 3/4/5/5.1/6.1/7.1: full level for the matching
front, `1/√2` for centre + each surround, LFE dropped; each row normalized so |coef| sum ≤ 1 to
prevent clipping (matches ffmpeg's default). Applied in both the direct-copy and sinc-resample
paths (only when `dst==2 && src>2`; unknown layouts fall back to front L/R). Compiles clean.
*(Needs in-app check: a 5.1 file now has centre/dialog present and isn't thin; not distorted/clipping.)*
Native multichannel support remains a separate, larger project.
- **Export speed (audited 2026-06-21):** a 1:14 1080p MP4 took ~9:06 (~7.4x realtime, ~135 ms/frame).
Audit **refuted** the per-frame-seek theory — export decodes the source *sequentially*
(`video.rs` `need_seek` is false once advancing forward), and readback is already async +
triple-buffered. Real hotspots:
- **[DONE] #1 — per-frame renderer rebuild.** The export pump built a fresh `vello::Renderer`
(full wgpu pipeline init) + empty `ImageCache` *every egui repaint* (`main.rs` ~6218). Now built
once per export and reused; also fixed lazy-image export (the throwaway cache had no container
path). **Expected the dominant win.**
- **[DONE] #2a — encode swscale rebuilt per frame.** `CpuYuvConverter::convert` now caches the
RGBA→YUV420p `scaling::Context` + frames in `new()` instead of per call.
- **[TODO] #2b — decode swscale + stride-repack** per frame in `video.rs:294-320` (shared with
scrubbing; cache the YUV→RGBA scaler on the decoder). Small win, modest risk.
- **Result of #1+#2a (measured):** ~7.4x → **~1.74x realtime** (130.7 s for 4488 frames @ 60 fps;
34 fps). Per-stage avg: Render(CPU build) 15 ms, **Readback(GPU latency) 42 ms**, Extract 1.3 ms,
Convert 5.7 ms.
- **Now GPU-bound.** Per ~87 ms poll cycle the CPU does ~66 ms (3× build 45 + convert 17 + extract 4)
but the GPU does ~87 ms (3 × ~29 ms composite) → GPU saturated at ~29 ms/frame; "Readback 42 ms" is
queue latency, not transfer (8 MB is sub-ms).
- **[SKIP] #3 GPU YUV / #5 pacing** — both only trim the CPU side, which is already *under* the GPU.
Won't move a GPU-bound throughput.
- **[TODO, big] Reduce the GPU composite (~29 ms/frame).** The per-layer HDR pipeline (Vello render →
linear → composite, ×layers) is the wall, shared with live rendering. Options: batch composite
passes; a fast-path skipping HDR compositing for simple single-layer/no-blend docs; cache unchanged
layers' scenes (CPU-side, only helps if it later becomes CPU-bound). Render-architecture project.
- Non-issues: per-frame seek, blocking readback, audio. (`video.rs:237` container-reopen-on-seek is
a latent cost but doesn't fire on forward export.)
- **AAC export NaN guard (done):** `convert_chunk_to_planar_f32` now sanitizes non-finite samples
(NaN/Inf → 0, finite clamped to [-1,1]) like the integer paths, with a one-time warning — a stray
non-finite render sample no longer fails the whole export. Upstream NaN source (effect/automation/
decode) still worth chasing if it recurs.
- [x] **Persist video thumbnails (DONE).** Mirrors waveform persistence: each clip's thumbnails are
PNG-encoded + packed into one opaque `LBTN` blob (editor owns the format; `encode/decode_thumbnail_blob`
in main.rs), stored as a `MediaKind::Thumbnail` row keyed by `thumbnail_media_id(clip_id)` (clip id XOR
a fixed sentinel). Save: a cheap Arc-clone snapshot (`VideoManager::snapshot_all_thumbnails`) rides the
`FileCommand::Save`, PNG-encoded off the UI thread in the worker, written by `save_beam` (kept in place
on re-save). Load: `load_beam_sqlite` reads the packs into `LoadedProject.thumbnail_blobs`; the editor
decodes + `insert_thumbnail`s them on a background thread and **gates regeneration** (`register_loaded_videos`
skips clips with persisted thumbnails). Bonus: thumbnails show even if the source video file is missing.
**Partial sets are persisted and resumed** (not thrown away): the `LBTN` blob (v2) carries a `complete`
flag (`VideoManager.thumbnails_complete`, marked when the keyframe pass finishes). On load, complete
packs are restored + skip regeneration; *partial* packs are restored AND generation is resumed —
`generate_keyframe_thumbnails` takes a `should_skip` predicate (`has_thumbnail_near`) so it only decodes
the keyframes not already covered. `insert_thumbnail` is now sorted + idempotent (fixes a latent
unsorted-`binary_search` bug and makes concurrent restore + resume race-safe). So a save 50 min into a
2 h video keeps that work and continues from there on reload.
Container tests still green; all crates compile. *(Needs in-app check: reload = instant thumbnails for
complete clips; a mid-generation save resumes from where it left off on reload.)*
**Size assessment (done):** thumbnails are 128px wide, height by aspect (72px at 16:9 →
128×72×4 ≈ **36 KB raw** each; 4:3 ≈ 49 KB), generated **one per ~5 s** (capped `interval_secs`,
at keyframes — so ~12/min). Raw: ~0.5 MB per 1:14 clip, ~26 MB/hour, ~52 MB/2 h. Compressed for
on-disk: JPEG ~36 KB/thumb → **~6 MB/2 h**; PNG ~815 KB → ~14 MB/2 h. So persistence is cheap
(≤ the waveform's ~36 MB/2 h), especially as JPEG. Plan: encode each clip's thumbnails (JPEG) +
their timestamps into one blob, a new `MediaKind::Thumbnail` row keyed by the clip/media id (mirror
the waveform persistence: write on save, restore via `insert_thumbnail` on load, regenerate if
absent). The 5 s interval already bounds count; no extra budget needed.
- **Progressive waveform on first import:** generation streams the whole file before the
waveform appears (several seconds for large files). Since `build_waveform_pyramid` already
streams, emit partial floors as it advances (e.g. flush every N seconds of decoded audio via
the existing `waveform_result` channel + chunked GPU upload) so the overview fills in across
the clip left-to-right instead of appearing all at once. Persistence saves only the final
complete pyramid.
## Guiding principle
Three subsystems already have the right streaming primitive; most of the work is wiring,
bounding caches, and adding a residency window. The recurring pattern:
> Keep tiny metadata always-resident, fault the heavy payload in on demand keyed by a
> stable ID, and evict everything outside a window around the playhead.
---
## Audit summary (where we stand today)
### Correctly streaming / bounded
- Video frame decode/seek/playback (`lightningbeam-core/src/video.rs:191` `get_frame`
keyframe-index seek + decode-until-target, one frame resident).
- WAV/AIFF import via mmap (`daw-backend/src/audio/engine.rs:2328`).
- Webcam capture encodes directly to disk (`lightningbeam-core/src/webcam.rs`).
- `WaveformCache` (100MB cap), decoder `LruCache` (20 frames), export render loop (≤3
frames in flight).
- The compressed-audio disk reader `daw-backend/src/audio/disk_reader.rs`
(`CompressedReader` + 3s `ReadAheadBuffer`) — **correct but never activated** (Phase 1a).
### Fully-loaded, unbounded by file length (the problems)
| Site | Issue |
|---|---|
| `daw-backend/src/io/audio_file.rs:344` `decode_progressive` | Decodes whole compressed file into a `Vec<f32>`; de-facto playback source. |
| `daw-backend/src/audio/pool.rs:1071` `load_file_into_pool` | Every audio file in a saved project fully decoded to `InMemory` on open. |
| `lightningbeam-core/src/video.rs:711` `extract_audio_from_video` | Whole video audio track into one `Vec<f32>`. |
| `lightningbeam-core/src/video.rs:412` `VideoManager.frame_cache` | Unbounded `HashMap` of full-res RGBA frames; grows while scrubbing. |
| `export/mod.rs:388-400` | Mux step buffers all compressed packets into `Vec`s; O(duration). |
| `lightningbeam-core/src/raster_layer.rs:115` `RasterKeyframe.raw_pixels` | ~8MB/frame at 1080p; all keyframes decoded from PNG at load (`file_io.rs:611-640`), never evicted. |
| `lightningbeam-editor/src/gpu_brush.rs:1051` `raster_layer_cache` | Unbounded GPU texture `HashMap`. |
| `lightningbeam-core/src/renderer.rs:25` `ImageCache` | Unbounded decoded image cache (asset textures). |
| `Document.image_assets` (`document.rs:206`) | Every image asset's compressed bytes resident for document life. |
---
## Container format decision: `.beam` → SQLite *(DECIDED)*
The `.beam` container moves from a **ZIP archive** to a **SQLite database file** (same
`.beam` extension). This is the foundation the rest of the plan builds on.
### Why
ZIP can stream `Stored` entries in place (via `data_start()`), but it has **no in-place
mutation** — every save and every raster frame write-back rewrites the whole archive — and
embedded PCM is rarely mmap-aligned. The current load path is even worse: it reads each
ZIP audio entry fully, decodes FLAC → re-encodes WAV → base64 → base64-decodes → temp file
→ full Symphonia decode → resident `Vec<f32>` (`file_io.rs:513-604`, `pool.rs:1071`).
SQLite dissolves the single-file-vs-performance tension:
- **Single file** — beginner-friendly, behaves like a file on every OS (no package-folder
confusion; we have no bundle magic on Linux/Windows).
- **Streaming reads**`sqlite3_blob_open` / `blob_read(offset, len)` gives seekable,
chunked reads through the pager (mmap mode for the DB). For chunked streaming the
pager-copy is negligible vs. decode cost, so the lack of zero-copy mmap doesn't matter.
- **Cheap, crash-safe mutation** — raster frame write-back is a transactional `UPDATE`;
save is a metadata write + dirty-blob updates. **ACID** means a force-quit / power loss /
crash mid-save can't corrupt the project (ZIP and package-dirs both have to hand-roll
atomicity).
- **Inspectable / scriptable**`sqlite3` CLI; `beam_inspector.py` can read it directly.
**Net effect: there is no scratch directory anywhere in this plan.** Media stream via blob
reads (or external paths); raster frames live in blob rows and write back transactionally.
### Large-media policy: packed OR referenced
Two storage modes per media item, both supported:
- **Packed** — bytes live in the DB. To stay under SQLite's ~2GB per-blob ceiling (and to
make reads naturally chunked), large media is split into **multiple blob-chunk rows**
(e.g. 64 MB/chunk); streaming reads address `(chunk_index, offset)`.
- **Referenced** — the DB stores only a path; bytes stay on disk (useful for shared media
on a network drive, or media too large/volatile to pack).
**Default-mode preference for files over the per-blob limit (~2GB):**
- A user preference `large_media_default: Pack | Reference` controls what happens to
imports above the threshold.
- The **first time** the user imports a media file over the limit, **prompt** them
(Pack vs Reference), apply it, and **persist the choice** as the preference for future
large imports (changeable later in settings).
- Files under the limit are packed by default (chunked only if needed).
### Schema sketch
```
media(
id BLOB PRIMARY KEY, -- stable Uuid
kind INTEGER, -- audio | video | raster | image-asset
codec TEXT, -- "flac","mp3","png",... (original, lossless-preserving)
storage INTEGER, -- 0 = packed, 1 = referenced
ext_path TEXT, -- set when storage = referenced
total_len INTEGER, -- bytes (packed) for chunk math
channels INTEGER, sample_rate INTEGER, width INTEGER, height INTEGER -- kind-specific meta
)
media_chunk(
media_id BLOB, chunk_index INTEGER, bytes BLOB,
PRIMARY KEY (media_id, chunk_index)
)
project_json(id INTEGER PRIMARY KEY CHECK (id = 0), data TEXT) -- existing project.json, verbatim
meta(key TEXT PRIMARY KEY, value TEXT) -- version, created, modified
```
`project.json` stays the same serialized `BeamProject` for now — only its container and the
media storage change. A migration reads a legacy ZIP `.beam` and writes the SQLite form on
first open/save.
### Streaming reads from packed media
A `BlobReader` implementing `Read + Seek` over `media_chunk` rows feeds the existing
streaming consumers unchanged: `CompressedReader` (audio) decodes from it instead of a
`File`; the video decoder seeks within it; raster `UPDATE`s a chunk. Referenced media uses a
plain `File` exactly as `do_import_audio` already does for originals today.
---
## Phase 1 — Audio: activate what already exists *(highest impact, lowest effort)*
### 1a. Turn on the compressed-audio disk reader
The `CompressedReader` + 3-second `ReadAheadBuffer` in `disk_reader.rs` is complete but
never invoked (`DiskReaderCommand::ActivateFile` / `DiskReader::create_buffer` are never
called; `AudioClip::read_ahead` at `clip.rs:63` is hard-wired to `None`).
- On compressed import (`engine.rs:2381`) and during playback setup, activate the file and
assign `AudioClip::read_ahead`.
- Change `decode_progressive` (`io/audio_file.rs:344`) to produce only the downsampled
waveform overview (min/max peaks) the UI needs, then drop decoded PCM. Playback comes
from the ring buffer, not RAM.
- Verify `render_from_file` (`pool.rs:449`) reads from `read_ahead` when `data()` is empty.
**Risk:** the real-time thread must never block on disk. The ring buffer prefetches ~2s
ahead; underruns degrade to silence (live) or block-wait (export), which `disk_reader.rs`
already distinguishes.
### 1b. Stream on project load *(depends on the SQLite container)*
Three coupled changes (none works alone):
1. Replace `load_file_into_pool`'s full decode (`pool.rs:1071`) with the same branching as
`do_import_audio`: PCM → mmap (referenced) or in-memory for tiny packed PCM; compressed
(incl. FLAC) → `from_compressed` placeholder backed by a `BlobReader` (packed) or `File`
(referenced). The claxon FLAC→WAV→base64 round-trip in `file_io.rs:533-591` is deleted.
2. **Bulk read-ahead activation:** loaded clips are deserialized directly
(`audio_backend.project`), bypassing `AddAudioClip`, so the Phase 1a wiring never fires
for them. After the engine installs the project, walk all audio clips and
`create_buffer` + `ActivateFile` + set `read_ahead` for every clip referencing a
`Compressed` pool entry. (`CompressedReader::open` needs a variant that takes a
`BlobReader` instead of a path for packed media.)
3. Pool entries carry storage mode (packed-chunks vs referenced path) from the `media`
table instead of base64 `embedded_data`.
### 1c. Video's embedded audio track — stream from the video via ffmpeg
**Interim stopgap (shipped):** `extract_audio_from_video_to_wav` streams the decoded audio to
a temp WAV, imported via `import_audio_sync` (mmap). Fixes the RAM OOM but writes the whole
uncompressed track to `/tmp` (fills small temp partitions) and the temp path doesn't survive
save/reload. **Superseded by the design below.**
**Proper design — stream the video's audio track on demand, never materialized.**
*Enabler:* `daw-backend` already depends on `ffmpeg-next` (used for MP3/AAC encoding), so the
ffmpeg audio decoder lives beside `CompressedReader` in `daw-backend/src/audio/`. No
cross-crate work (`core → daw-backend` is one-way). `CompressedReader` already has the needed
interface.
1. **`VideoAudioReader` (ffmpeg)** — mirrors `CompressedReader`:
`open(path)`, `decode_next(&mut Vec<f32>) -> frames` (resample → interleaved f32 at native
rate; reuse the old extraction resampler), `seek(target_frame) -> actual`,
`sample_rate`/`channels`/`total_frames`.
2. **Source dispatch:** `enum StreamSource { Compressed(CompressedReader), Video(VideoAudioReader) }`
(or a small `trait AudioFrameSource`) held by the reader thread; ring buffer / prefetch /
export-blocking unchanged. `DiskReaderCommand::ActivateFile` gains a `kind: SourceKind`.
3. **Pool model:** `AudioStorage::VideoAudio { video_path, decoded_for_waveform, decoded_frames,
total_frames }` (near-copy of `Compressed`); `data()` empty, playback via `read_ahead`. Pool
entry `path` = the video file.
4. **Engine API:** `EngineController::add_video_audio_sync(video_path) -> usize` — ffmpeg-probe
the audio track (rate/channels/frames/duration, no decode), build the pool entry, return index.
5. **Clip activation:** extend the Phase 1a `AddAudioClip` wiring — if entry is `VideoAudio`,
make the buffer + `ActivateFile{kind:VideoAudio, path:video_path}` + set `clip.read_ahead`.
One ffmpeg context + 3 s buffer per active clip instance.
6. **Import flow:** `import_video` calls `add_video_audio_sync(video_path)`
`AudioClip::new_sampled`. **Remove** `extract_audio_from_video_to_wav`, the temp-WAV
handling, and the now-dead `add_audio_file_sync`. No WAV / `/tmp` / RAM.
7. **Save/load:** the `VideoAudio` entry serializes as a path reference to the video (no media
bytes — the video is already referenced by its `VideoClip`); reconstruct on load by
re-probing. Fixes the stopgap's reload fragility (nothing to persist).
8. **Waveform overview:** background ffmpeg pass emitting **downsampled peaks only** (bounded
memory) into the existing waveform path — shared with the Phase 1a `decode_progressive`
cleanup.
**Sample accuracy (required — video audio must stay frame-synced with other clips):**
Coarse ffmpeg seeks are NOT sufficient. `VideoAudioReader::seek(target_frame)` must:
- coarse-seek to a point ≤ target, then **decode-and-discard** to land exactly on
`target_frame`, tracking the absolute sample position from decoded-frame PTS (discard whole
frames before target; for the frame straddling target, drop its leading samples). After
`seek`, `decode_next` yields samples starting at exactly `target_frame`.
- This makes frame N of the video-audio pool entry correspond to the exact timeline position,
so it mixes sample-aligned with mmap/InMemory clips. Continuous decode advances frame-exact.
- *Consistency note:* `CompressedReader` should get the same decode-discard alignment (its
current coarse-seek-then-write-at-target can misalign by up to a GOP after a seek). Fold in
while here, or at least flag.
*Model decision (confirmed):* the video's audio stays a **separate, editable `AudioClip`** on
an audio track, backed by the `VideoAudio` pool entry — users can move/trim/mute/detach it.
*Build order:* `VideoAudioReader` + `StreamSource` → pool `VideoAudio` variant →
`add_video_audio_sync` + activation → swap `import_video` (remove WAV path) → sample-accurate
seek (both readers) → waveform-peaks pass.
---
## Phase 2 — Video: bound the caches *(small, isolated)*
### 2a. Bound `VideoManager.frame_cache`
`video.rs:412` — convert the unbounded `HashMap<(Uuid,i64), Arc<VideoFrame>>` to an LRU
mirroring the decoder-level cache (`video.rs:34`). Frame-count or byte budget.
### 2b. Stream the export mux
`export/mod.rs:388-400` — interleave-write packets to the output as produced (compare PTS,
write the earlier stream) instead of collecting all then writing. O(duration) → O(1).
---
## Phase 3 — Raster: disk-backed keyframe paging *(the heavy one)* **[locked design]**
Today `load_beam_sqlite` (`file_io.rs:564`) eagerly `decode_png`s **every** raster keyframe's
`Raster` media row into `RasterKeyframe.raw_pixels` (`raster_layer.rs:115`, `w·h·4` ≈ 8 MB @
1080p, `#[serde(skip)]`), never evicts, has an unbounded GPU texture cache, and holds full-frame
undo snapshots. `raw_pixels` is the working rep (edits write it, save reads it, render reads it),
`has_pixels()` = `!raw_pixels.is_empty()`, `keyframe_at` is a `partition_point` binary search, and
the container is opened only at load/save (no live handle).
**Design (confirmed with user):** keep `raw_pixels` as the working rep; make residency explicit
via a `RasterStore` + an editor-run fault-in/evict pass *before* the immutable render. Async
fault-in (no scrub hitch), with a **low-res image proxy** shown until the full frame lands.
Decisions: small window (±~2 keyframes); **dirty (edited-unsaved) frames stay fully resident**
(spill-to-scratch deferred); fault-in is **async**; proxy is a **per-keyframe low-res RGBA image**
(PNG/WebP, correct alpha), NOT a video (VP9-alpha was rejected as finicky for negligible disk win).
### Drive-by (Arc pixels): DROPPED
Investigated and rejected: `raw_pixels` has ~64 access sites, and most `.clone()`s genuinely need
an owned `Vec<u8>` (undo buffers, export, GPU readback) so `Arc<Vec<u8>>` would force `(*p).clone()`
and still copy. The only beneficiary, the per-frame `renderer.rs:550` Vello clone, is on the
**legacy/dead** path — the live HDR canvas renders raster as `RenderedLayerType::Raster` → GPU
upload in `stage.rs` which passes a `&[u8]` slice and uploads only on cache-miss (no per-frame
clone). Not worth 64 edits. Start at 3a.
### 3a. Lazy async fault-in + image proxy
- **[DONE 3a-1]** Lazy load: full-decode removed; `raw_pixels` empty on load, `needs_fault_in`
armed recursively; canvas records misses → App pages in via `RasterStore.load_pixels`.
- **[DONE 3a-2]** Async: page-in runs on a background thread (deduped via `raster_loads_inflight`);
results applied at top of `update()`. No UI block on cold scrub.
- **[DONE 3a-3]** Image proxy: `MediaKind::RasterProxy` (≤192px PNG, derived id), written
beside each resident full PNG on save + eager-decoded on load into `RasterKeyframe::proxy`.
Separate `proxy_layer_cache` (own LRU, budget 64); the raster render blits the proxy mapped to
the keyframe's FULL logical dims (upscales via sampler) when the full texture isn't resident.
*(Proxies exist only after a save+reload; eager decode → lazy/paged is a refinement for huge
paint projects.)*
- **`RasterStore`** (core): current `.beam` path + a read-only connection; `load_pixels(kf_id,w,h)`
reads the `Raster` row and `decode_png`s it. Set/cleared by the editor on load + save-as.
- **Save:** alongside the full PNG, write a low-res RGBA proxy per resident keyframe
(`MediaKind::RasterProxy`, ≤~480px long edge, keyed by `kf.id`).
- **Load:** stop eager full-decode; decode **proxies** eagerly (cheap → instant scrub everywhere);
leave full `raw_pixels` empty.
- **Fault-in pass** (editor, `&mut document` + store, each frame before render): for each raster
layer ensure the active keyframe ±N is requested; load full PNGs on a **background thread pool**;
on arrival, set `raw_pixels` + `texture_dirty`. Render uses full `raw_pixels` if resident, else the
upscaled proxy. Reused by the exporter (already frame-by-frame).
### 3b. Residency window + eviction **[DONE]**
- Added `#[serde(skip)] dirty: bool` (edited-since-persist; distinct from `texture_dirty`). Set on
stroke/fill/paint-bucket/floating-lift commits + undo/redo; cleared on save (which re-arms the LRU).
- Implemented as a fault-in-recency **LRU** (`RASTER_RESIDENT_MAX = 12`), not a strict ±N window:
evict the oldest **clean** frame (drop `raw_pixels`, re-arm `needs_fault_in`); the shown frame is
always most-recent so it's protected; **dirty frames never evicted**. Save preserves evicted frames'
rows via `media_exists` (no data loss) and walks all layers to match load.
*(Refinement deferred: count budget → byte budget for 4K resolution-robustness.)*
### 3c. Bound the GPU cache **[DONE for raster_layer_cache]**
`raster_layer_cache` (`gpu_brush.rs`, `HashMap<Uuid,CanvasPair>`, Rgba16Float ping-pong
`w·h·16`/entry, was **unbounded**) → recency LRU (`RASTER_LAYER_CACHE_MAX = 12`) in
`ensure_layer_texture`: bump-to-most-recent + evict oldest; shown frames protected. F3 overlay
now shows tracked VRAM (raster cache MB + count). *(Refinements: count→byte budget; raise/headroom
if >12 raster layers are visible at once. Export `raster_cache` lives one export — fine. Vello
`ImageCache` is image *assets* → Phase 4.)*
### 3d. Undo memory **[DONE]**
`RasterStrokeAction`/`RasterFillAction` stored `buffer_before`+`buffer_after` full frames.
Now store a `RasterDiff` (`actions/raster_diff.rs`) — changed bbox before/after only, computed in
`new()`, full buffers dropped. Undo/redo apply onto the keyframe's resident pixels; the editor
faults the target frame in first (`Action::raster_resident_hint` + `peek_undo/redo_raster_hint`),
correct because a clean evicted frame's container bytes == its logical state. Non-resident base ⇒
skip (no corruption). Unit-tested round-trip. *(Refinement: compress full-canvas-fill diffs, whose
bbox is the whole frame.)*
### 3e. Prefetch frames **[DONE for playback]**
Implemented for playback: each update during playback, page in the next `PREFETCH_AHEAD=4`
upcoming keyframes per raster layer (reusing the async worker + `raster_loads_inflight` dedup), so
full frames are resident before the playhead arrives — fixes "proxy on every frame"/flicker during
playback. *(Caveat: with many simultaneous raster layers the 12-frame resident budget may evict a
prefetched frame before it's shown — raise budget or scale prefetch if that surfaces. Scrub-direction
prefetch still TODO.)*
Original note: *(future, after 3d — pure latency win, no correctness need)*
Fault-in is reactive (page in only on a render miss), so a never-visited frame still shows the
proxy for a beat before the full lands. **Prefetch the full pixels for frames about to be shown**:
on scrub/playback, dispatch background page-ins for the active keyframe ±N in the direction of
playhead motion (and during playback, the next K keyframes), reusing the 3a-2 async worker +
`raster_loads_inflight` dedup. Keep prefetched frames in the 3b LRU so they're still bounded; cap
concurrent prefetch loads so scrubbing fast doesn't thrash the disk. Optional: also prewarm the GPU
texture (3c cache) for the immediate next frame. Net effect: cold scrubbing/playback shows full-res
frames with no proxy flicker. Proxy stays as the instant fallback when prefetch can't keep up.
### Build order & tests
1. Arc drive-by — COW make_mut test. 2. 3a fault-in + store + proxy — load→empty-until-faulted,
PNG round-trip, proxy-then-swap. 3. 3b window/evict/dirty — residency ≤ window while scrubbing,
dirty never evicted. 4. 3c GPU bound. 5. 3d undo diffs reproduce pre-stroke buffer exactly.
---
## Phase 3.5 — Image textures in vector scenes **[DONE 2026-06-21]** *(prereq for Phase 4; fixed DCEL-broken image import)*
**Done:** 3.5a — import/drop places an image as a borderless image-filled rectangle
(`AddShapeAction::image_rect`), centered (direct import) or at the drop point (library drag);
renderer now maps the image brush onto the fill's bounding box (was anchored at world origin →
only a corner showed); `SetImageFillAction` + an **Image** fill-type tab (None|Solid|Gradient|Image)
with an asset picker in the Info Panel. 3.5b — image bytes persist as `MediaKind::ImageAsset` rows in
the `.beam` (kept-in-place; `ImageAsset.data` is `skip_serializing` + container-backed; old base64
projects migrate on re-save); eager-read on load. *(ImageCache still unbounded — Phase 4 adds the
usage-based LRU/lazy paging.)*
### (original plan below)
## Phase 3.5 — Image textures in vector scenes *(prereq for testing Phase 4; fixes DCEL-broken image import)*
**Why:** Phase 4 pages *image assets*, but there's currently no way to get an image asset into a
vector scene — so nothing to page. This also repairs image import, half-broken since the DCEL switch.
**Current state (audited 2026-06-21):**
- *Works:* `import_image` (`main.rs`) decodes dims + creates an `ImageAsset` (raw bytes embedded in
`Document::image_assets`, serialized as **base64 in project JSON**). The renderer's image-fill paths
are **complete** — GPU/Vello (`renderer.rs:~1160`, `ImageBrush` via `ImageCache.get_or_decode`) and
CPU/tiny-skia (`renderer.rs:~1486`). `Fill::image_fill` (`vector_graph/mod.rs:110`) and
`Face::image_fill` (`dcel2/mod.rs:117`) fields exist and render when set.
- *Broken/missing (the workflow):*
1. **Drop image → canvas is stubbed:** `stage.rs:~11782` and `main.rs:~4924` both just print
"Image drag to stage not yet supported with DCEL backend". Nothing is added to the scene.
2. **No way to assign an image fill:** no `SetImageFillAction` (only `SetFillPaintAction` for
color/gradient); no Info-Panel picker. `Fill`/`Face.image_fill` are never populated.
3. **DCEL faces never get `image_fill`** (`dcel2/import.rs:275` always `None`; topology copies from
parent which is also `None`).
4. **Not in the container:** `MediaKind::ImageAsset` exists but is **dead** — image bytes live only
as base64 in project JSON. Not chunked, not pageable (so Phase 4 can't page them).
**Tasks:**
- **3.5a — Place + assign.** Replace the two drop stubs: dropping an image onto a vector layer creates
a rectangle face sized to the image at the drop point with `image_fill = asset_id`. Add
`SetImageFillAction` (set/clear an image fill on the selected face/shape; mirrors `SetFillPaintAction`)
+ an Info-Panel image-asset picker for the selected shape's fill. Populate `Face.image_fill` in DCEL
(and keep it through topology ops — already copied from parent).
- **3.5b — Persist in the container.** Write image assets as `MediaKind::ImageAsset` rows in the `.beam`
SQLite (like raster/audio: write on save kept-in-place on re-save; read on load), keyed by asset id;
drop the base64-in-JSON embedding (or keep a tiny ref). This is the storage Phase 4 pages from.
- **3.5c — Lazy decode hook.** Image bytes load from the container into `ImageCache` on first render
(decode → `ImageBrush`/`Pixmap`). Leave `ImageCache` **unbounded for now**; Phase 4 adds the
usage-based LRU/eviction (this phase just makes there *be* real, container-backed image assets to page).
- **Tests:** import→drop→render round-trip; save/reload preserves the image fill + reads bytes from the
container (not JSON); CPU and GPU render paths both show the image.
---
## Phase 4 — Asset paging by usage + LRU *(vector's real cost is assets, not geometry)*
Vector geometry is compact flat POD (tens of KB/frame, no cached tessellation/DCEL) — leave
it resident. The heavy, evictable thing is the **image assets** referenced by fills.
**Data model.**
- `ImageAsset` (`clip.rs:250`): `path: PathBuf` + `data: Option<Vec<u8>>` (whole compressed
file bytes) + dims. Imported fully into `data` at `main.rs:3936`.
- All assets resident in `Document.image_assets: HashMap<Uuid, ImageAsset>` (`document.rs:206`).
- Decoded form in `ImageCache` (`renderer.rs:25`): `HashMap<Uuid, Arc<ImageBrush>>` + CPU
`Pixmap` map, keyed by asset id, **unbounded**.
- A `Fill` references an asset by `image_fill: Option<Uuid>` (`vector_graph/mod.rs:110`).
Same UUID may appear in many fills/keyframes/layers and recursively through clip instances.
**No asset→frame or frame→asset index exists today.**
**Two evictable tiers:** Tier 1 = compressed bytes (`ImageAsset.data`, droppable, reload
from blob row or external `path`); Tier 2 = decoded pixels (`ImageCache` + GPU textures —
the heavy one).
**Progress (2026-06-21):**
- **[DONE] Tier 2 — bound the decoded `ImageCache`.** 256 MB **usage-LRU**: every
`get_or_decode`/`_cpu` bumps the asset's recency; inserts past budget evict the least-recently-used
(a miss re-decodes from `asset.data`). Achieves usage-based eviction via render-access recency
(simpler than the frame→asset enumeration below; that enumeration is only needed for *prefetch*).
- **[DONE] Tier 1 — lazy compressed bytes.** `ImageCache` holds the container path (threaded
App.current_file_path → SharedPaneState → VelloRenderContext) and pages bytes on a decode miss via
`read_packed_media_readonly`; `load_beam_sqlite` no longer eager-reads → instant load, compressed
bytes don't accumulate. `asset.data` is still used when resident (fresh import / old base64 project).
*(Refinement: persistent read connection vs open-per-miss.)*
- **[DONE] Prefetch.** `assets_needed_at(document, time)` enumerates image ids in the visible vector
layers' active keyframes; during playback the stage decodes the ~0.5s-ahead set into the cache.
*(Refinements: nested clip-instance recursion; background-thread decode.)*
**Phase 4 = DONE** (image asset paging by usage + LRU).
### 4a. Frame→asset enumeration (incl. nested clips — see note below)
A function `assets_needed_at(time) -> HashSet<Uuid>`: walk each visible vector layer's active
`ShapeKeyframe`, collect `fill.image_fill` across its `VectorGraph.fills`, **recursing into
clip instances** with the outer→inner local-time mapping. This is "needed now". Scanning
upcoming keyframes (and upcoming nested-clip keyframes) gives "needed soon" for prefetch.
### 4b. Usage bookkeeping (the multi-frame problem)
Maintain a reverse index `asset_id → usage count` (fills referencing it across the whole
document), updated incrementally as edits add/remove `image_fill`s (hook the fill-mutation
paths in `vector_graph` and the relevant actions).
- count 0 → dead, fully evictable / GC candidate.
- count > 0 → keep metadata; residency of `data`/decoded pixels driven by **proximity to
playhead**, not by count (a high-count asset far from the playhead is still evicted).
Residency decision: `resident = needed-now needed-soon`; beyond that, an **LRU with a byte
budget** for referenced-but-distant assets (covers scrubbing back without a reload).
Eviction never touches an asset in needed-now.
### 4c. Bound the decoded tier
Convert `ImageCache`'s two maps to LRU/byte-budgeted (`renderer.rs:25`) and bound the GPU
image-texture cache the same way, keyed to the residency window.
### Nested-clip prefetch (important)
A clip instance placed on an outer frame has its **own internal timeline of keyframes**,
each of which can reference its own image assets. Prefetch must therefore:
- Recurse through clip instances when computing both needed-now and needed-soon.
- Map outer playhead time → each nested clip's local time, and look ahead along the
**nested** timeline (not just the outer one) so assets used by an upcoming *inner*
keyframe are loaded before the nested clip reaches it.
- Deduplicate across the whole recursion (an asset shared by outer and inner frames counts
once); the usage index handles refcounting.
---
## Cross-cutting: a shared residency abstraction
A generic **`PagedStore<Id, Payload>`** with three consumers — always-resident metadata,
disk backing, residency = window/needed-set around playhead + LRU byte budget:
| Consumer | Metadata kept | Paged payload | Backing | "Needed now" key |
|---|---|---|---|---|
| Raster keyframes (Ph 3) | id, dims, time | `raw_pixels` + GPU texture | SQLite blob row (`UPDATE` on write-back) | active keyframe per layer |
| Image assets (Ph 4) | id, dims, storage | `data` bytes + decoded pixels/texture | SQLite blob row or external path | fills' `image_fill` set at time (recursive) |
| Video frames (Ph 2a) | — | RGBA frame | source via ffmpeg seek | requested timestamps |
Audio stays separate (real-time ring buffer, different constraints). The frame→asset
enumeration + usage index is unique to Phase 4.
---
## Sequencing
1. **Phase 1a** — done; independent of the container, works with the current ZIP loader.
2. **Phase 2** — small, isolated, independently shippable; container-independent.
3. **Phase 0 (container)**`.beam` ZIP → SQLite + `BlobReader` + large-media policy +
legacy-ZIP migration. Prerequisite for 1b/1c/3/4.
4. **Phase 1b** — streaming pool loader + bulk read-ahead activation (on the SQLite store).
5. **Phase 1c** — depends on 1b's pool path.
6. **Phase 3** — the substantial build; implement `PagedStore` over blob rows.
7. **Phase 4** — thin layer on the same abstraction + the frame→asset/usage index.
Phase 1a and Phase 2 can ship now; everything else waits on Phase 0 (the container).
---
## Status
- [~] Phase 1a — activate compressed-audio disk reader ← **in progress**
- [x] Wire `ActivateFile` + assign `clip.read_ahead` on `AddAudioClip` for compressed
pool files (`engine.rs:909`). Per-clip reader keyed by `clip_id`; matches the
existing `DeactivateFile` convention in `RemoveAudioClip`. Compiles clean.
- [ ] Stop `decode_progressive` (`io/audio_file.rs:344`) from accumulating/streaming the
full PCM; emit only the downsampled waveform overview. (Crosses into the UI
waveform pipeline — `AudioDecodeProgress` consumer — so handled as its own step.)
- [ ] Runtime verification: confirm a compressed clip actually plays from the ring
buffer (was effectively silent before, since `read_ahead` was always `None`).
- [~] **Phase 0 — container migration `.beam` ZIP → SQLite** ← **in progress**
- [x] SQLite schema (`media`, `media_chunk`, `project_json`, `meta`) + `rusqlite` dep
(bundled) — `lightningbeam-core/src/beam_archive.rs`
- [x] `BlobReader` (`Read + Seek` over `media_chunk`, owns its own read-only connection,
opens a blob handle per read with rowids resolved once) — for `CompressedReader` /
video decoder in 1b. 5 integration tests pass (`tests/beam_archive.rs`): json
round-trip, packed full read, streaming reads + seeks across chunk boundaries,
referenced-path, overwrite-replaces-chunks.
- [x] Packed (chunked) + referenced media write/read API; `is_sqlite()` format detection;
`MediaKind`/`MediaStorage`/`MediaMeta`/`MediaInfo`.
- [x] `BeamArchive::transaction()` / `BeamTxn` — in-place transactional save (only
changed rows written; unchanged large media never rewritten); orphan cleanup via
`retain_media`. 7 archive tests pass (added txn-grouping + rollback). Per user: save
must NOT copy+rename for existing SQLite files.
- [x] Wire `save_beam` to `BeamArchive` — in-place txn for existing SQLite, temp+rename
only for new/migrated files. Audio → packed (or referenced ≥2GB) `media` rows;
raster → PNG `media` rows keyed by keyframe id. FLAC→WAV→base64 save round-trip
deleted (now packs original bytes with their codec).
- [x] Wire `load_beam` — format dispatch: SQLite (`load_beam_sqlite`) vs legacy ZIP
(`load_beam_zip_legacy`, kept verbatim). SQLite load reconstitutes packed audio into
`embedded_data` so the existing pool loader is unchanged (streaming = Phase 1b).
- [x] Legacy ZIP `.beam` → SQLite migration: `is_sqlite()` routes load; saving a
ZIP-loaded project writes SQLite (migrates on save). Editor compiles end-to-end.
- [x] Large-media policy: packed (chunked) vs referenced — `LargeMediaMode {Ask,Pack,
Reference}`; save honors it for files ≥`LARGE_MEDIA_THRESHOLD`. Packing streams from
disk via `put_media_packed_from_path` (chunk-by-chunk, never loads the whole file).
`Ask` behaves as `Reference` at save time.
- [x] `large_media_default` user preference: persisted in `AppConfig`, editable in
Preferences → Advanced (incl. resetting to `Ask` to re-trigger the prompt).
- [x] First-import-over-threshold prompt: `note_possible_large_media` (hooked into
import_audio/video/image) queues a one-time modal; choice persists to config.
Threshold shown in the modal is derived from the constant.
- [ ] Runtime verification: save a real project, reopen it, confirm audio + raster survive
round-trip; confirm an old ZIP `.beam` still opens and migrates on save.
- [ ] (Optimization, later) FLAC-compress packed PCM/WAV audio; raster disk-dirty flag to
skip unchanged frames on in-place save (Phase 3).
> Note: the crate's internal `#[cfg(test)]` modules (`clip.rs`, `effect_layer.rs`) have
> pre-existing compile breakage (old `Beats`/`TempoMap` API) unrelated to this work; it
> blocks `cargo test --lib`, so `beam_archive` tests live in `tests/` (integration) which
> build the lib in normal mode. Worth fixing separately.
- [x] Phase 1b — stream on project load (PACKED audio path complete & user-verified: streams on load,
waveform generates + persists, sample-accurate seeking). Referenced-path streaming + MP3 seek index
+ proper video-audio reload remain as noted follow-ups.
- **Decision (user):** cross-crate packed streaming via an **inversion-of-control factory**
daw-backend defines the interface, core implements it over `BlobReader`. Keeps the audio
engine container-agnostic. (Alternatives rejected: daw-backend owning rusqlite = layering
violation; referenced-only-first = leaves packed <2GB in RAM.)
- **Current load reality (why this is needed):** *nothing* streams on load today — every entry
is fully decoded to a PCM `Vec<f32>`. Packed audio is base64-reconstituted into `embedded_data`
(`load_beam_sqlite`) → written to a temp file → `load_file_into_pool` full-decodes; referenced
audio also full-decodes via `load_file_into_pool`; and the Phase 1a/1c disk-reader activation
never fires for loaded clips (they bypass `AddAudioClip`).
- [x] **B1/B2 foundation (DONE, headless-tested):** in `disk_reader.rs` — `trait MediaByteSource:
Read+Seek+Send+Sync { byte_len }` + `trait AudioBlobSourceFactory: Send+Sync { open(media_id)
-> Box<dyn MediaByteSource> }`; `SymphoniaByteSource` adapter (impl `MediaSource`,
is_seekable/byte_len); `CompressedReader::open_source(src, ext)` sharing probe via a
refactored `from_mss`; `enum StreamOpen { Path, Source{src,ext} }`; `StreamSource::open` and
`DiskReaderCommand::ActivateFile` now take `StreamOpen` (engine site wraps `Path`); re-exported
`AudioBlobSourceFactory`/`MediaByteSource` at `daw_backend::audio`. Test
`tests/compressed_source_stream.rs` decodes an in-memory WAV through a `Cursor`-backed
`MediaByteSource` (proves probe+decode+seek over a byte stream). daw-backend compiles clean.
- [x] **B3 (engine, DONE):** `Engine.blob_source_factory: Option<Arc<dyn AudioBlobSourceFactory>>` +
`EngineController::set_blob_source_factory` (via `Query::SetBlobSourceFactory`, ordered before
`SetProject` on the same queue). `AudioFile.packed_media_id: Option<String>` (Some ⇒ open via
factory using `original_format` as the ext hint; None ⇒ `StreamOpen::Path`). Activation factored
into `Engine::activate_streaming_for(reader_id, pool_index)`, used by `AddAudioClip` and bulk.
- [x] **C (core factory, DONE):** `file_io::blob_source_factory(beam_path)``BeamBlobFactory`
implementing `AudioBlobSourceFactory` over `BeamArchive::open_blob_reader`. `BlobReader` holds a
`!Sync` rusqlite `Connection`, so it's wrapped in `SyncBlobReader` (a `Mutex` used via `get_mut`
on the hot path — no runtime locking) to satisfy Symphonia's `MediaSource: Send + Sync`. Installed
by the editor between `load_audio_pool` and `set_project`.
- [x] **D (load-path, DONE — packed audio):** `load_beam_sqlite` now streams packed audio whose codec
is recognized (`is_streamable_audio_codec`) — leaves `embedded_data` empty so the pool builds a
Compressed placeholder with `packed_media_id`; no base64, no temp file, no decode. `serialize`
round-trips packed entries by media id (so in-place re-save keeps the row). Non-audio codecs
(video-container audio tracks) keep the legacy reconstitution path → **no regression**.
- [x] **E (bulk activation, DONE):** `SetProject` calls `Engine::activate_all_streaming_clips`
walks every loaded audio clip and `activate_streaming_for` (create_buffer + `ActivateFile` + set
`read_ahead`), the loaded-clip equivalent of the Phase 1a wiring.
- [x] **Waveform-on-load for streamed audio (DONE):** streaming broke the old waveform path (it came
from the full in-RAM decode, which no longer happens). Added
`disk_reader::build_waveform_pyramid_from_source(Box<dyn MediaByteSource>, ext, B)` (load-time
counterpart of the path-based builder). On load, the editor background-generates a pyramid for any
streamed entry lacking a persisted one (opens the packed blob via a local factory), sending the
floor through the same `waveform_result` channel `update()` drains; the next save persists it.
Verified in-app: packed MP3 **streams + plays** (`Activated reader=0, kind=CompressedAudio`); the
overview now fills in shortly after load.
- **Headless tests pass** (compressed_source_stream, video_audio_stream, waveform_pyramid); all three
crates compile clean. **Needs in-app verification:** the waveform appears after load (background gen),
then instantly on subsequent loads once saved; RAM stays flat on a big project.
- [x] **Seek alignment fix (DONE):** streamed compressed audio was ~1.2s off *after seeking*
(fine from the start). `CompressedReader::seek` used `SeekMode::Coarse`, which for MP3
byte-estimates the position and seeds the timestamp from that estimate — wrong for VBR / files
whose header padding the estimate ignores, so `actual_ts` (and thus the buffer's frame labels)
landed ~1.2s early. Switched to `SeekMode::Accurate`: Symphonia counts frame *headers* (no
decode) from a true anchor (current pos, or rewind-to-0 for backward seeks) → exact `actual_ts`;
the existing sub-frame `pending_discard` finishes the job. FLAC/OGG seek cheaply (seek tables);
a long MP3 backward seek walks headers from 0 (I/O, not decode). Tests still green.
- [ ] **Deferred (follow-up):** per-file **seek index** for elementary streams (MP3) — a one-time
header scan (ts↔byte map) to make far seeks O(1) instead of an Accurate header-walk from the
anchor. Matters for multi-hour MP3s; song-length files are fine as-is.
- [x] **Proper video-audio reload (DONE):** a video's audio is now stored as a **path reference** to
the video (never packed/embedded as audio media) and **re-probed via FFmpeg** on load into a
streaming `VideoAudio` entry — `AudioPoolEntry.is_video_audio` flag drives both `serialize`
(reference, not pack), `save_beam` (`reference_it |= is_video_audio`), and `load_from_serialized`
(`VideoAudioReader::open` → `from_video_audio`). Fixes 5.1 audio losing its channels on reload
(the old Symphonia reconstitution collapsed it); also no more decode-whole-video-to-RAM / temp
files on load. Old saves (video mis-packed as audio) self-heal on the next save.
- [ ] **Deferred (follow-up):** stream *referenced* (external-path) **audio** on load too — replace
`load_file_into_pool`'s full decode with the `do_import_audio` branching (PCM → mmap, compressed
`from_compressed` placeholder). Higher risk (touches the working referenced path); packed
covers the common <2GB case first.
- [x] **DONE: packed video streaming.** Small videos pack into the `.beam`
(a `MediaKind::Video` blob at the clip id, `VideoClip.media_id` referencing it) and stream **both
frames and audio** from the DB blob via FFmpeg. The `AVIOContext`-over-`Read+Seek` shim lives in
the new `ffmpeg-blob-io` crate (`BlobInput`, version-pinned `=8.0.0`/`=8.0.1`), isolating the
unsafe + ABI coupling. Frames: `video.rs` `VideoSource{Path,Packed}` opens a fresh `BlobReader`
per decoder/seek/scan. Audio: `VideoAudioReader::open_source` over the same blob (the
`disk_reader.rs` `StreamSource` blocker is removed); save points the linked video-audio pool
entry's `media_id` at the video row so it streams from the same blob. Tests: ffmpeg-blob-io AVIO
unit tests (WAV via Cursor + seek + open/drop loop), core `packed_video_stream` (blob→AVIO→Input),
`beam_archive` packed-video round-trip, daw-backend `open_source` (compiles; can't link in the
container — user runtime-verifies actual A/V playback).
- [~] Phase 1c — video embedded-audio track ← **stopgap shipped; proper design next**
- [x] Stopgap: `extract_audio_from_video_to_wav` streams to a temp WAV → `import_audio_sync`
(mmap). Fixed the ~2.8GB-`Vec<f32>` OOM. But writes the whole WAV to `/tmp` (fills
small temp partitions) and the temp path doesn't survive reload.
- [~] **Proper design** (see "Phase 1c" body): stream the video's audio on demand via a new
ffmpeg `VideoAudioReader` in the disk reader — no extraction, no `/tmp`, no RAM; path
reference survives save/load.
- [x] **Step 1 (DONE):** `VideoAudioReader` (ffmpeg) + `StreamSource` enum + `SourceKind`
in `disk_reader.rs`. Sample-accurate seek (coarse seek + decode-discard to exact
frame via PTS). 2 integration tests pass (`daw-backend/tests/video_audio_stream.rs`):
in-order decode + sample-exact seek at several targets. (Found: mono frames have an
empty channel layout → must `set_channel_layout` before resampling, else swr returns
AVERROR_INPUT_CHANGED.) Lib compiles clean; `StreamSource` `#[allow(dead_code)]`
until wired. `VideoAudioReader` made `pub` for the integration test.
- [x] **Step 2 (DONE):** `AudioStorage::VideoAudio { decoded_for_waveform, decoded_frames,
total_frames }` + `AudioFile::from_video_audio` (path = the video file). `data()`
empty / `read_samples()` 0 (streamed). `Query::AddVideoAudioSync` +
`do_add_video_audio` (probes via `VideoAudioReader::open`, no decode) +
`EngineController::add_video_audio_sync`. `GetPoolAudioSamples` surfaces VideoAudio's
waveform overview too. daw-backend compiles clean; probe `total_frames` test passes.
- [x] **Step 3 (DONE):** reader thread now holds `StreamSource` (opens via
`StreamSource::open(path, kind)`, dispatches `sample_rate()/channels()/seek/decode_next`);
`ActivateFile` carries `kind: SourceKind`; `#[allow(dead_code)]` removed. `AddAudioClip`
activation maps `Compressed`→`CompressedAudio`, `VideoAudio`→`VideoAudio`, creates the
read-ahead buffer + `ActivateFile{kind}` + sets `clip.read_ahead`. Compressed path is
behaviorally identical (StreamSource::Compressed wraps the same CompressedReader).
daw-backend + editor compile clean; VideoAudioReader tests still pass.
⚠️ Not runtime-verified — needs in-app check that compressed audio still plays (no
regression) and that an activated VideoAudio clip produces sound.
- [x] **Step 4 (DONE):** `import_video` now calls `add_video_audio_sync(video_path)`
pool index, fetches channels/sample_rate via `get_pool_file_info`, makes the
`AudioClip` with the video's duration. **No WAV / /tmp / RAM.** Removed the stopgap
(`extract_audio_from_video_to_wav` + WAV helpers + `ExtractedAudioInfo`), dead
`add_audio_file_sync` (+ `Query::AddAudioFileSync` / `QueryResponse::AudioFileAddedSync`
/ handler), and the now-unreachable `AudioExtractionResult::NoAudio`. Kept
`import_audio_sync` (still used by normal audio import). daw-backend + editor clean.
**→ Feature is live end-to-end; ready for in-app testing.**
- [x] **Step 5 (DONE):** `CompressedReader` now seeks sample-accurately too — coarse
symphonia seek + decode-discard (`pending_discard` set from `seeked.actual_ts` in
`seek`, applied in `decode_next`, which continues rather than reporting EOF when a
whole packet is discarded). So compressed clips no longer drift vs video audio after
a seek. Test `compressed_reader_seek_is_sample_accurate` passes (the WAV coarse seek
lands pre-target, exercising the discard). `CompressedReader` made `pub` for the test.
- [~] Step 6: **bounded waveform overview** — replaces today's full-resolution
`raw_audio_cache`/GPU waveform (which doesn't scale: it stores every sample at mip 0,
so a long file is multi-GB on GPU + RAM — the same memory issue, and the Phase 1a
`decode_progressive` leftover). Design below. Slices: (1a) streaming pyramid builder
+ (1b) persistence + (1c) min/max GPU upload, then (2) LRU tile cache + re-decode floor.
- [x] **Slice 1a (DONE):** `daw-backend/src/audio/waveform_pyramid.rs`
`WaveformPyramidBuilder` streams interleaved samples, accumulates the floor, and
reduces `BRANCH(4):1` at `finish` into a root-first pyramid (convention B:
`levels[0]`=root envelope, `levels.last()`=floor, `.root()`/`.floor()` accessors).
Ragged last buckets reduce over available children (no value padding). Bounded
(~22 MB/2 h @ B=256). 7 integration tests pass (`tests/waveform_pyramid.rs`):
bucket min/max, partial flush, multi-level envelope == global min/max, root-first
ordering, stereo channels, size bound, chunk-agnostic.
- [~] **Slice 1b (data layer DONE; orchestration folded into 1c):**
- [x] Generation bridge `disk_reader::build_waveform_pyramid(path, kind, B)` — streams
a decode (`StreamSource` over symphonia/ffmpeg) into the builder; bounded
memory (one chunk + the pyramid). Test: envelope matches the signal through
both backends.
- [x] Serialization `WaveformPyramid::to_bytes`/`from_bytes` (LBWF blob; f32 texels —
f16 a later size optimization). Round-trip test + rejects truncated/garbage.
- [x] `MediaKind::Waveform` in the SQLite container (keyed by the audio item's id).
- [ ] Orchestration (with 1c).
- [~] **Slice 1c (in-memory floor overview DONE; persistence next):**
- [x] `waveform_gpu`: `PendingUpload.minmax` flag + `pack_texel` helper; `upload_audio`
threads `minmax` (frame_stride 4, packs `(Lmin,Lmax,Rmin,Rmax)` directly).
The texture is already Rgba16Float and the GPU mipgen builds zoom-out levels, so
only the texel-packing differs. Render the floor at **effective rate `sr/B`** (so
time→texel maps B samples/texel) and `total_frames = floor_texel_count`.
- [x] `AppConfig.waveform_floor_samples_per_texel` (default 256, user-configurable).
- [x] App: `waveform_minmax_pools: HashMap<usize, u32>` (pool → `B`, carries the floor rate
with full float precision) + a `(pool, packed_floor, sr, channels, B)` results channel;
drained in `update()``raw_audio_cache.insert(floor)` + flag pool + `waveform_gpu_dirty`.
- [x] Generation: on video-audio import Success, the same bg thread streams
`disk_reader::build_waveform_pyramid(path, VideoAudio, B)` once and sends the packed
`floor()`. (Video-audio has no in-RAM samples, so this is what makes its waveform appear.)
- [x] Threaded `waveform_minmax_pools` through the pane-context (`panes/mod.rs` +
main.rs construction) → `render_layers`**both** render sites (collapsed-group
~timeline.rs:3048 AND expanded-track ~3613): compute `total_frames = len/4`,
`eff_sr = sr/B`, set `minmax`. Compiles clean (editor `cargo check` = 0 errors).
- [x] Shader fix: `waveform.wgsl` now reads the **nearest integer LOD via `textureLoad`**
instead of sampling a fractional mip. Trilinear blends two levels whose row-major
linearizations differ → horizontal shift that flips each 0.5 of `mip_f` (= each 2x
zoom step), the "every other zoom level is offset" artifact. **User-confirmed fixed:**
features hold position at every zoom and line up with playback.
See memory `waveform-shader-fractional-mip-offset`.
- [x] **Persistence (done):** the full pyramid is serialized (`to_bytes`) on generation and
kept in `App.waveform_pyramid_blobs`. `save_beam` writes it as a `MediaKind::Waveform`
row keyed by a **deterministic id derived from the pool index** (`file_io::waveform_media_id`,
"LBWF" sentinel in the high 32 bits) — independent of how the audio bytes are stored, so
it works for packed/referenced/video-audio alike, and an in-place re-save reuses the row.
Carried in/out via a transient `#[serde(skip)] AudioPoolEntry.waveform_blob` and a
`waveform_blobs` field on `FileCommand::Save`. `load_beam_sqlite` reads the row back;
the editor restores `raw_audio_cache`/`waveform_minmax_pools`/`waveform_pyramid_blobs`
+ flags `waveform_gpu_dirty` after the backend loads the pool (using each entry's
`sample_rate` for `eff_sr`, the stored `B` for the rate). No re-decode on load.
`register_loaded_videos` only loads frames (not audio), so there is no redundant
regeneration to suppress. Compiles clean across all three crates.
### Waveform LOD pyramid design (step 6)
A min/max LOD pyramid (tree of zoom-level textures): fully zoomed out → envelope; fully zoomed
in → per-sample; seamless between.
- **One streaming decode pass** builds the whole pyramid down to a configurable **floor**
`B` samples/texel (default 256), via a hierarchical reduction (each sample updates a running
per-level min/max accumulator; a filled bucket emits a texel and folds into its parent —
`branch` 4:1). Bounded memory: holds only the pyramid (~`N/B·4/3` texels ≈ **~14 MB / 2 h
stereo @ B=256**), never the full samples. Full-res (B=1 ≈ 2.7 GB) is the only level NOT
stored.
- **Persist the pyramid** in the `.beam` SQLite container (a `waveform` media kind; session
temp before first save). `B` is stored with it (preference is just the default for new gen).
Persistence is load-bearing: it makes mid-zoom a cheap **disk read**, not a re-decode.
- **Runtime = LRU tile cache** (GPU textures) loaded from the persisted pyramid on demand.
Eviction is **ancestor-closed**: only evict an LRU node with no resident children ("a node is
cleared only after its children") — so rendering can always walk up to a resident ancestor;
detail sharpens in, never blanks. Root is tiny/hot → effectively pinned for free.
- **Re-decode only below the floor** (texel < `B` samples): by then the visible window spans a
tiny time range, so decoding it (via the sample-accurate seekable readers from steps 15 —
the payoff) for true per-sample detail is cheap. This removes the large-span re-decode gap:
above the floor it's a disk read; below it the span is already small.
- **Why a deep floor (not a coarse cutoff):** a coarse-only pinned set would force the first
on-demand level to re-reduce a huge time span per tile. Persisting deep makes every level a
disk read; `B` is a size-vs-crossover knob (smaller B = bigger pyramid, cheaper re-decode).
- `waveform_gpu` needs a **min/max texel upload** (`Lmin,Lmax,Rmin,Rmax` per texel) instead of
min=max-per-sample; the existing compute mipgen still builds the mip chain *within* a tile.
**Decisions (locked):** branch 4:1; floor `B≈256` samples/texel, **user-configurable**
(`AppConfig.waveform_floor_samples_per_texel`, stored per-pyramid); 8192-wide tiles; LRU ~4
viewports of fine tiles; persist pyramid in `.beam`.
- [x] Video decoder concurrency (movie-length lag/freeze): keyframe-index scan now runs
holding no VideoManager/decoder lock (brief locks only bracket it) → no more multi-second
UI freeze on import/load; thumbnail generation uses a **dedicated** decoder and samples
at keyframes (≈1 frame each vs whole-GOP) → no playback contention. Removed dead
`VideoManager::build_keyframe_index`, `build_and_set_keyframe_index`, `downsample_rgba*`.
- [x] Phase 2a — bound video frame cache. `VideoManager.frame_cache` (was an unbounded
`HashMap<(Uuid,i64), Arc<VideoFrame>>` that grew per distinct frame during playback) is now an
`LruCache` evicted by a **byte budget** (`FRAME_CACHE_BYTE_BUDGET` = 256 MB) rather than a frame
count — robust across resolutions (a 4K frame is ~33 MB vs ~2 MB at 800×600). Byte total tracked
on insert/evict/remove; `unload_video` pops per-clip keys (LruCache has no `retain`). Decoder-level
cache was already LRU. Editor compiles clean. *(Not yet runtime-verified.)*
- [x] Phase 2b — stream export mux. `export/mod.rs::mux_video_and_audio` no longer collects every
packet into two `Vec`s before interleaving; it stream-merges the two inputs by PTS with one pending
packet per stream (O(1) memory vs O(duration)). Same tie-break (`v_us <= a_us`) and drain-on-EOF
behavior; output is byte-identical. Editor compiles clean. *(Not yet runtime-verified — needs an
in-app export to confirm A/V sync.)*
- [x] Phase 3a — lazy + async raster fault-in (`RasterStore` + background thread + image proxy)
- [x] Phase 3b — raster residency LRU + eviction (dirty-flag data-loss safety)
- [x] Phase 3c — bound raster GPU texture cache (recency LRU + F3 VRAM readout)
- [x] Phase 3d — raster undo dirty-rect diffs (+ fault-in-before-undo)
- [x] Phase 3.5 — image textures in vector scenes (fixed DCEL-broken image import; image-fill tab + picker; container-persisted)
- [x] Phase 4 — image asset paging: Tier 2 decoded-cache byte-LRU, Tier 1 lazy container bytes, playback prefetch
- [x] Phase 5 — fixed the broken `#[cfg(test)]` unit tests; **`cargo test --lib` green again**
(daw-backend 17 passed, lightningbeam-core 264 passed). Wrapped stale raw-`f64` time literals
in `Beats(...)` / passed `&TempoMap` to changed signatures (automation.rs, clip.rs,
effect_layer.rs); fixed stale test setup (register a vector clip so `get_clip_duration` resolves)
and a stale default expectation (shape `fill_color` defaults `None`). Surfaced + fixed one **real
undo bug**: `DeleteFolderAction(MoveToParent)` reparented child subfolders but never restored them
on rollback (orphaned them) — now tracked and restored. Production code otherwise untouched.

305
TODO.md Normal file
View File

@ -0,0 +1,305 @@
# Lightningbeam TODO
> ⚠️ **Stale entries:** Lightningbeam was rewritten from JavaScript to Rust. Any entry below
> that cites `src/*.js` / `main.js` / `animation.js` predates that migration — the *issue* may
> or may not still exist in the Rust codebase, but the file/line references are obsolete.
> **Re-verify against the current Rust code before acting** (this covers the "Animation System
> Refactoring" section and the JS-referencing "Known Issues" entries — node editor, default
> interpolation, etc.). Items with no `.js` references are current.
## Animation System Refactoring *(STALE — JS-era migration notes; superseded by the Rust DCEL/keyframe system)*
### Completed
- ✅ Implement AnimationData curve-based system (Keyframe, AnimationCurve, AnimationData classes)
- ✅ Add GraphicsObject.currentTime property
- ✅ Migrate shape rendering to use AnimationData curves (exists, zOrder)
- ✅ Binary search optimization for keyframe lookups
### In Progress
- Migrating from Frame-based to AnimationData curve-based system throughout codebase
### Pending Features
#### Animation Curve Enhancements
- [ ] Implement extrapolation modes (separate for start vs end):
- "hold" (default) - hold value at first/last keyframe
- "extend" - linearly extend the curve beyond keyframes
- "repeat" - repeat the animation
- "decay" - exponential decay to a target value
- [ ] Add position, scale, rotation animation curves for shapes
- [ ] Add shape morphing/tweening between keyframes
#### Keyframing Behavior
- [ ] Add user preference for keyframing behavior when editing objects:
- Auto-keyframe (current default): create/update keyframe at current time
- Edit previous (Flash-style): update most recent keyframe before current time
- Ephemeral (Blender-style): changes don't persist without manual keyframe
- Optional: Add modifier key (e.g. Shift) to toggle between modes
#### Shape Ordering
- [ ] Add "Bring Forward" menu option (swap zOrder with shape in front)
- [ ] Add "Send Backward" menu option (swap zOrder with shape behind)
- [ ] Add "Bring to Front" menu option (set zOrder to max + 1)
- [ ] Add "Send to Back" menu option (set zOrder to min - 1)
#### Code Cleanup
- [ ] Remove all remaining references to Frame-based system
- [ ] Remove legacy Frame class once migration is complete
- [ ] Clean up GraphicsObject.shapes[] array (shapes should only live in Layers)
## Known Issues / Platform Limitations
### Animation: Tweens are broken (Rust codebase) — LOW PRIORITY
- **Issue**: Animation tweening between keyframes (shape/vector interpolation, and the
`tween_after` behavior on keyframes) does not work correctly in the current Rust app.
Needs investigation + fix. Not urgent — revisit later.
- (Older JS-codebase animation entries below reference `src/*.js` and are stale.)
### Audio: Oscillator Timbre Drift (Phase Accumulation Error)
- **Issue**: Oscillators exhibit timbre changes over time due to floating-point phase accumulation errors
- **Affected Files**:
- `daw-backend/src/effects/synth.rs:117-120` (SimpleSynth)
- `daw-backend/src/audio/node_graph/nodes/oscillator.rs:167-170` (OscillatorNode)
- **Root Cause**: Current phase wrapping uses conditional subtraction (`if phase >= 1.0 { phase -= 1.0 }`), which accumulates f32 rounding errors over time, especially for long-playing notes
- **Current Code**:
```rust
self.phase += frequency / sample_rate;
if self.phase >= 1.0 {
self.phase -= 1.0;
}
```
- **Recommended Fix**: Replace with `.fract()` for numerically stable wraparound:
```rust
self.phase += frequency / sample_rate;
self.phase = self.phase.fract();
```
- **Impact**: Medium - affects audio quality for sustained notes, becomes noticeable after several seconds
- **Priority**: Medium - should be addressed before production use
### UI: Node Connections Render Behind VoiceAllocator Child Nodes
- **Issue**: Connection lines (SVG paths) inside expanded VoiceAllocator nodes render behind child nodes due to z-index stacking
- **Affected File**: `src/styles.css:1128`
- **Root Cause**: Child nodes have `z-index: 10` while connection SVG paths have default/lower z-index
- **Current Code**:
```css
.drawflow .drawflow-node.child-node {
opacity: 0.9;
border: 1px solid #5a5aaa !important;
box-shadow: 0 2px 8px rgba(90, 90, 170, 0.3);
z-index: 10;
}
```
- **Recommended Fix**: Either:
1. Remove `z-index: 10` from `.child-node` (simplest), or
2. Add higher z-index to connection SVG paths, or
3. Use CSS `isolation: isolate` on the VoiceAllocator contents area to create a new stacking context
- **Impact**: Low - visual issue only, connections still function but appear to go "behind" nodes
- **Priority**: Low - cosmetic issue that doesn't affect functionality
### UI: VoiceAllocator Child Nodes Don't Move with Parent
- **Issue**: When a VoiceAllocator node is moved, its child nodes remain in their original positions instead of moving with the parent
- **Affected File**: `src/main.js:6202-6207`
- **Root Cause**: The `nodeMoved` event handler only handles the case where a child node is moved (resizes parent), but doesn't handle when the VoiceAllocator itself is moved
- **Current Code**:
```javascript
editor.on("nodeMoved", (nodeId) => {
const node = editor.getNodeFromId(nodeId);
if (node && node.data.parentNodeId) {
resizeVoiceAllocatorToFit(node.data.parentNodeId);
}
});
```
- **Recommended Fix**: Add logic to detect when a VoiceAllocator is moved and update all child node positions:
```javascript
editor.on("nodeMoved", (nodeId) => {
const node = editor.getNodeFromId(nodeId);
// Case 1: A child node was moved - resize parent
if (node && node.data.parentNodeId) {
resizeVoiceAllocatorToFit(node.data.parentNodeId);
}
// Case 2: A VoiceAllocator was moved - move all children
if (node && node.data.nodeType === 'VoiceAllocator') {
// Calculate delta from previous position (need to track)
// Update all child node positions by the delta
// Call editor.updateConnectionNodes() for parent and all children
}
});
```
- **Impact**: High - child nodes become disconnected from parent visually
- **Priority**: High - breaks expected behavior of grouped nodes
### UI: VoiceAllocator Expansion Doesn't Update Connection Positions
- **Issue**: When expanding/collapsing a VoiceAllocator, connection endpoints don't update to match the new port positions
- **Affected File**: `src/main.js:6496-6555` (handleNodeDoubleClick function)
- **Root Cause**: The expand/collapse logic shows/hides child nodes and resizes the container, but never calls `editor.updateConnectionNodes()` to refresh connection positions
- **Current Code**: In `handleNodeDoubleClick()`, after expanding or collapsing:
```javascript
// Expand
expandedNodes.add(nodeId);
nodeElement.classList.add('expanded');
nodeElement.style.width = '600px';
nodeElement.style.height = '400px';
// ... shows child nodes ...
// Missing: editor.updateConnectionNodes(`node-${nodeId}`)
```
- **Recommended Fix**: Call `editor.updateConnectionNodes()` after resizing:
```javascript
// After expanding
expandedNodes.add(nodeId);
nodeElement.classList.add('expanded');
// ... resize and show children ...
// Update connection positions for VoiceAllocator and all children
editor.updateConnectionNodes(`node-${nodeId}`);
for (const [childId, parentId] of nodeParents.entries()) {
if (parentId === nodeId) {
editor.updateConnectionNodes(`node-${childId}`);
}
}
```
- **Impact**: Medium - connections appear in wrong positions until manually moved
- **Priority**: Medium - visual issue that affects usability
### UI: Node Editor Allows Editing Without MIDI Layer Selected
- **Issue**: The node editor pane allows adding/editing instrument nodes even when no MIDI layer is selected, and always uses hardcoded `trackId: 0`
- **Affected File**: `src/main.js:6045-6920` (nodeEditor function)
- **Root Cause**: The node editor never checks if `context.activeObject.activeLayer` exists or is a MIDI track, and all backend commands use hardcoded `trackId: 0`
- **Current Code**: All graph commands hardcode track 0:
```javascript
const commandArgs = parentNodeId
? {
trackId: 0, // HARDCODED!
voiceAllocatorId: editor.getNodeFromId(parentNodeId).data.backendId,
nodeType: nodeType,
x: x,
y: y
}
: {
trackId: 0, // HARDCODED!
nodeType: nodeType,
x: x,
y: y
};
```
- **Recommended Fix**:
1. Check if activeLayer is a MIDI track before allowing edits:
```javascript
function getSelectedMidiTrack() {
const activeLayer = context.activeObject?.activeLayer;
if (!activeLayer || activeLayer.type !== 'midi') {
return null;
}
return activeLayer;
}
```
2. Show placeholder when no MIDI track selected:
```javascript
function nodeEditor() {
const container = document.createElement("div");
const midiTrack = getSelectedMidiTrack();
if (!midiTrack) {
container.innerHTML = '<div class="placeholder">Select a MIDI layer to edit instruments</div>';
return container;
}
// ... rest of node editor code ...
}
```
3. Use actual track ID instead of hardcoded 0:
```javascript
const trackId = midiTrack.audioTrackId || 0;
const commandArgs = { trackId, nodeType, x, y };
```
4. Add listener to refresh node editor when layer selection changes
- **Impact**: High - allows editing wrong track's instrument graph, data corruption risk
- **Priority**: High - can cause confusion and data loss
### Animation: Wrong Default Interpolation for Shape and Object Keyframes
- **Issue**: Shape index and object transform keyframes default to "linear" interpolation but should default to "hold" (step function), and there's no UI to change interpolation after creation
- **Affected Files**:
- `src/models/animation.js:124` (Keyframe constructor defaults to "linear")
- `src/main.js:2161` (shapeIndex keyframes default to "linear")
- `src/main.js:2198` (object position/rotation/scale keyframes default to "linear")
- `src/main.js:5910` (Timeline menu - missing tween options)
- **Root Cause**:
1. The Keyframe constructor defaults interpolation to "linear"
2. Shape index keyframes preserve existing interpolation or default to "linear"
3. Object transform keyframes explicitly use "linear"
4. No menu options exist to change interpolation mode after keyframe creation
- **Current Code**:
- Keyframe constructor (animation.js:124):
```javascript
constructor(time, value, interpolation = "linear", uuid = undefined) {
```
- Shape index keyframes (main.js:2161):
```javascript
const interpolationType = existingShapeIndexKf ? existingShapeIndexKf.interpolation : 'linear';
const shapeIndexKeyframe = new Keyframe(currentTime, newShapeIndex, interpolationType);
```
- Object keyframes (main.js:2198):
```javascript
const newKeyframe = new Keyframe(
currentTime,
currentValue,
'linear' // Default to linear interpolation
);
```
- **Expected Behavior**:
- Shape index keyframes should default to "hold" (shapes shouldn't morph between versions)
- Object transforms should default to "hold" (objects shouldn't move/rotate/scale between keyframes unless explicitly tweened)
- Timeline menu should have options to convert between interpolation modes
- **Recommended Fix**:
1. Change shapeIndex default to "hold" (main.js:2161):
```javascript
const interpolationType = existingShapeIndexKf ? existingShapeIndexKf.interpolation : 'hold';
```
2. Change object keyframe default to "hold" (main.js:2198):
```javascript
const newKeyframe = new Keyframe(currentTime, currentValue, 'hold');
```
3. Add Timeline menu options (main.js:5910, in timelineSubmenu):
```javascript
{
text: "Add Shape Tween",
enabled: /* check if shape is selected and has keyframes */,
action: () => {
// Find shapeIndex curve for selected shape
// Change interpolation between keyframes to "linear"
}
},
{
text: "Add Motion Tween",
enabled: /* check if object is selected and has transform keyframes */,
action: () => {
// Find position/rotation/scale curves for selected object
// Change interpolation between keyframes to "linear" or "bezier"
}
}
```
- **Note**: exists and zOrder keyframes already correctly use "hold" (main.js:2139, 2150)
- **Impact**: High - causes unwanted interpolation, shapes morph unexpectedly, objects move when they shouldn't
- **Priority**: High - fundamental animation behavior is incorrect
### Tauri Pinch-Zoom on Linux
- **Issue**: Two-finger pinch gestures zoom the entire Tauri window instead of individual canvases
- **Status**: Known Tauri limitation on Linux/GTK with no cross-platform solution
- **Tracking**: https://github.com/tauri-apps/tauri/discussions/3843
- **Workaround attempts**: Tried `zoomHotkeysEnabled: false`, `touch-action: none`, viewport meta tags - none worked
- **Resolution**: Monitor Tauri releases for official fix
## Notes
### Architecture
- **GraphicsObject** contains Layers and has `currentTime` (continuous time)
- **Layer** contains `shapes[]` array and `animationData` (AnimationData instance)
- **AnimationData** contains curves dictionary, each curve identified by parameter name
- Shape curves: `shape.{uuid}.exists`, `shape.{uuid}.zOrder`
- Future: `shape.{uuid}.x`, `shape.{uuid}.y`, `shape.{uuid}.rotation`, etc.
- **Shapes render based on curves**: Layer.draw checks exists > 0, sorts by zOrder, draws in order
### Interpolation Types
- `linear` - Linear interpolation between keyframes
- `bezier` - Cubic Bezier with easing control points
- `step`/`hold` - Step function (jumps to next value)

10
daw-backend/Cargo.lock generated
View File

@ -586,7 +586,6 @@ dependencies = [
"dasp_rms",
"dasp_sample",
"dasp_signal",
"ffmpeg-blob-io",
"ffmpeg-next",
"hound",
"memmap2",
@ -662,15 +661,6 @@ version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af9673d8203fcb076b19dfd17e38b3d4ae9f44959416ea532ce72415a6020365"
[[package]]
name = "ffmpeg-blob-io"
version = "0.1.0"
dependencies = [
"ffmpeg-next",
"ffmpeg-sys-next",
"libc",
]
[[package]]
name = "ffmpeg-next"
version = "8.0.0"

File diff suppressed because it is too large Load Diff

View File

@ -50,10 +50,6 @@ pub struct ExportSettings {
pub end_time: Seconds,
/// Tempo map for beat-position scheduling
pub tempo_map: TempoMap,
/// Tag metadata as (ffmpeg-key, value) pairs (e.g. ("title", "…"), ("artist", "…")). Written to
/// the container's native tags: ID3v2 (MP3), MP4 atoms (M4A), Vorbis comments (FLAC), RIFF INFO
/// (WAV). Empty = no tags.
pub metadata: Vec<(String, String)>,
}
impl Default for ExportSettings {
@ -67,26 +63,10 @@ impl Default for ExportSettings {
start_time: Seconds::ZERO,
end_time: Seconds(60.0),
tempo_map: TempoMap::constant(120.0),
metadata: Vec::new(),
}
}
}
/// Set tag metadata on an ffmpeg output context (before `write_header`). FFmpeg maps the standard
/// keys to each container's native tags.
fn apply_metadata(output: &mut ffmpeg_next::format::context::Output, metadata: &[(String, String)]) {
if metadata.is_empty() {
return;
}
let mut dict = ffmpeg_next::Dictionary::new();
for (k, v) in metadata {
if !v.is_empty() {
dict.set(k, v);
}
}
output.set_metadata(dict);
}
/// Export the project to an audio file
///
/// This performs offline rendering, processing the entire timeline
@ -128,21 +108,17 @@ pub fn export_audio<P: AsRef<Path>>(
// Route to appropriate export implementation based on format.
// Ensure export mode is disabled even if an error occurs.
let result = match settings.format {
ExportFormat::Wav => {
ExportFormat::Wav | ExportFormat::Flac => {
let samples = render_to_memory(project, pool, settings, event_tx.as_mut().map(|tx| &mut **tx))?;
// Signal that rendering is done and we're now writing the file
if let Some(ref mut tx) = event_tx {
let _ = tx.push(AudioEvent::ExportFinalizing);
}
write_wav(&samples, settings, &output_path)
// hound writes no metadata; append a RIFF INFO chunk for tags.
.and_then(|_| append_wav_info_chunk(output_path.as_ref(), &settings.metadata))
}
ExportFormat::Flac => {
let samples = render_to_memory(project, pool, settings, event_tx.as_mut().map(|tx| &mut **tx))?;
if let Some(ref mut tx) = event_tx {
let _ = tx.push(AudioEvent::ExportFinalizing);
match settings.format {
ExportFormat::Wav => write_wav(&samples, settings, &output_path),
ExportFormat::Flac => write_flac(&samples, settings, &output_path),
_ => unreachable!(),
}
export_flac(&samples, settings, &output_path)
}
ExportFormat::Mp3 => {
export_mp3(project, pool, settings, output_path, event_tx)
@ -211,7 +187,6 @@ pub fn render_to_memory(
settings.sample_rate,
settings.channels,
false,
None, // export never runs with a recording in flight
);
// Calculate how many samples we actually need from this chunk
@ -298,174 +273,48 @@ fn write_wav<P: AsRef<Path>>(
Ok(())
}
/// Export real FLAC via ffmpeg from already-rendered interleaved f32 samples (Vorbis-comment
/// metadata). Replaces the former `write_flac`, which wrote WAV bytes to a `.flac` file. 16-bit
/// uses S16; 24-bit uses S32 (ffmpeg's flac encoder emits `bits_per_raw_sample = 24` for S32,
/// taking the top 24 bits).
fn export_flac<P: AsRef<Path>>(
/// Write FLAC file using hound (FLAC is essentially lossless WAV)
fn write_flac<P: AsRef<Path>>(
samples: &[f32],
settings: &ExportSettings,
output_path: P,
) -> Result<(), String> {
use ffmpeg_next as ffmpeg;
ffmpeg::init().map_err(|e| format!("Failed to initialize FFmpeg: {}", e))?;
let codec = ffmpeg::encoder::find(ffmpeg::codec::Id::FLAC)
.ok_or("FLAC encoder not found in this ffmpeg build")?;
let mut output = ffmpeg::format::output(&output_path)
.map_err(|e| format!("Failed to create output file: {}", e))?;
let channel_layout = match settings.channels {
1 => ffmpeg::channel_layout::ChannelLayout::MONO,
2 => ffmpeg::channel_layout::ChannelLayout::STEREO,
_ => return Err(format!("Unsupported channel count: {}", settings.channels)),
// For now, we'll use hound to write a WAV-like FLAC file
// In the future, we could use a dedicated FLAC encoder
let spec = hound::WavSpec {
channels: settings.channels as u16,
sample_rate: settings.sample_rate,
bits_per_sample: settings.bit_depth,
sample_format: hound::SampleFormat::Int,
};
// FLAC accepts packed S16 or S32; S32 → 24-bit output.
let use_24 = settings.bit_depth >= 24;
let sample_fmt = if use_24 {
ffmpeg::format::Sample::I32(ffmpeg::format::sample::Type::Packed)
} else {
ffmpeg::format::Sample::I16(ffmpeg::format::sample::Type::Packed)
};
let mut writer = hound::WavWriter::create(output_path, spec)
.map_err(|e| format!("Failed to create FLAC file: {}", e))?;
let mut encoder = ffmpeg::codec::Context::new_with_codec(codec)
.encoder()
.audio()
.map_err(|e| format!("Failed to create FLAC encoder: {}", e))?;
encoder.set_rate(settings.sample_rate as i32);
encoder.set_channel_layout(channel_layout);
encoder.set_format(sample_fmt);
encoder.set_time_base(ffmpeg::Rational(1, settings.sample_rate as i32));
let mut encoder = encoder.open_as(codec)
.map_err(|e| format!("Failed to open FLAC encoder: {}", e))?;
{
let mut stream = output.add_stream(codec)
.map_err(|e| format!("Failed to add stream: {}", e))?;
stream.set_parameters(&encoder);
}
apply_metadata(&mut output, &settings.metadata);
output.write_header()
.map_err(|e| format!("Failed to write FLAC header: {}", e))?;
let channels = settings.channels as usize;
let num_frames = samples.len() / channels;
let frame_size = if encoder.frame_size() > 0 { encoder.frame_size() as usize } else { 4096 };
let mut done = 0usize;
while done < num_frames {
let n = (num_frames - done).min(frame_size);
let mut frame = ffmpeg::frame::Audio::new(sample_fmt, n, channel_layout);
frame.set_rate(settings.sample_rate);
frame.set_pts(Some(done as i64)); // samples; the FLAC muxer requires PTS
let buf = frame.data_mut(0); // packed interleaved → plane 0
let base = done * channels;
if use_24 {
for i in 0..n * channels {
let s = samples[base + i].clamp(-1.0, 1.0);
let v = (s as f64 * 2_147_483_647.0) as i32; // full-scale S32; encoder takes top 24
buf[i * 4..i * 4 + 4].copy_from_slice(&v.to_le_bytes());
}
} else {
for i in 0..n * channels {
let s = samples[base + i].clamp(-1.0, 1.0);
let v = (s * 32767.0) as i16;
buf[i * 2..i * 2 + 2].copy_from_slice(&v.to_le_bytes());
// Write samples (same as WAV for now)
match settings.bit_depth {
16 => {
for &sample in samples {
let clamped = sample.max(-1.0).min(1.0);
let pcm_value = (clamped * 32767.0) as i16;
writer.write_sample(pcm_value)
.map_err(|e| format!("Failed to write sample: {}", e))?;
}
}
encoder.send_frame(&frame).map_err(|e| format!("Failed to send FLAC frame: {}", e))?;
flac_write_packets(&mut encoder, &mut output)?;
done += n;
}
encoder.send_eof().map_err(|e| format!("Failed to flush FLAC encoder: {}", e))?;
flac_write_packets(&mut encoder, &mut output)?;
output.write_trailer().map_err(|e| format!("Failed to finalize FLAC: {}", e))?;
Ok(())
}
/// Drain encoded FLAC packets and write them (non-interleaved). Skips the trailing empty flush
/// packet, which the FLAC muxer otherwise rejects as "Invalid data". Rescales packet ts from the
/// encoder time base to the stream's.
fn flac_write_packets(
encoder: &mut ffmpeg_next::encoder::Audio,
output: &mut ffmpeg_next::format::context::Output,
) -> Result<(), String> {
let mut pkt = ffmpeg_next::Packet::empty();
let enc_tb = encoder.time_base();
let stream_tb = output.stream(0).map(|s| s.time_base()).unwrap_or(enc_tb);
while encoder.receive_packet(&mut pkt).is_ok() {
if pkt.size() == 0 {
continue;
24 => {
for &sample in samples {
let clamped = sample.max(-1.0).min(1.0);
let pcm_value = (clamped * 8388607.0) as i32;
writer.write_sample(pcm_value)
.map_err(|e| format!("Failed to write sample: {}", e))?;
}
}
pkt.set_stream(0);
pkt.rescale_ts(enc_tb, stream_tb);
pkt.write(output).map_err(|e| format!("Failed to write FLAC packet: {}", e))?;
}
Ok(())
}
/// Append a RIFF `LIST`/`INFO` metadata chunk to a finished WAV file (hound writes no tags), then
/// fix up the top-level RIFF size. Maps ffmpeg-style keys to RIFF INFO sub-chunk IDs. Trailing INFO
/// chunks are ignored by players that don't read them.
fn append_wav_info_chunk(path: &Path, metadata: &[(String, String)]) -> Result<(), String> {
use std::io::{Seek, SeekFrom, Write};
let riff_id = |key: &str| -> Option<&'static [u8; 4]> {
match key {
"title" => Some(b"INAM"),
"artist" => Some(b"IART"),
"album" => Some(b"IPRD"),
"genre" => Some(b"IGNR"),
"comment" => Some(b"ICMT"),
"date" => Some(b"ICRD"),
"track" => Some(b"ITRK"),
_ => None,
}
};
let mut info: Vec<u8> = Vec::new();
info.extend_from_slice(b"INFO");
for (key, val) in metadata {
if val.is_empty() {
continue;
}
let Some(id) = riff_id(key) else { continue };
let mut bytes = val.as_bytes().to_vec();
bytes.push(0); // NUL-terminate
if bytes.len() % 2 == 1 {
bytes.push(0); // pad to even
}
info.extend_from_slice(id);
info.extend_from_slice(&(bytes.len() as u32).to_le_bytes());
info.extend_from_slice(&bytes);
}
if info.len() <= 4 {
return Ok(()); // nothing but the "INFO" tag
_ => return Err(format!("Unsupported bit depth: {}", settings.bit_depth)),
}
let mut list: Vec<u8> = Vec::with_capacity(info.len() + 8);
list.extend_from_slice(b"LIST");
list.extend_from_slice(&(info.len() as u32).to_le_bytes());
list.extend_from_slice(&info);
writer.finalize()
.map_err(|e| format!("Failed to finalize FLAC file: {}", e))?;
let mut f = std::fs::OpenOptions::new()
.read(true)
.write(true)
.open(path)
.map_err(|e| format!("Failed to open WAV for tagging: {}", e))?;
let end = f.seek(SeekFrom::End(0)).map_err(|e| e.to_string())?;
if end % 2 == 1 {
f.write_all(&[0]).map_err(|e| e.to_string())?;
}
f.write_all(&list).map_err(|e| format!("Failed to write WAV tags: {}", e))?;
let new_len = f.seek(SeekFrom::End(0)).map_err(|e| e.to_string())?;
f.seek(SeekFrom::Start(4)).map_err(|e| e.to_string())?;
f.write_all(&((new_len - 8) as u32).to_le_bytes())
.map_err(|e| format!("Failed to update RIFF size: {}", e))?;
Ok(())
}
@ -513,7 +362,6 @@ fn export_mp3<P: AsRef<Path>>(
stream.set_parameters(&encoder);
}
apply_metadata(&mut output, &settings.metadata);
output.write_header()
.map_err(|e| format!("Failed to write header: {}", e))?;
@ -558,7 +406,6 @@ fn export_mp3<P: AsRef<Path>>(
settings.sample_rate,
settings.channels,
false,
None, // export never runs with a recording in flight
);
// Calculate how many samples we need from this chunk
@ -684,7 +531,6 @@ fn export_aac<P: AsRef<Path>>(
stream.set_parameters(&encoder);
}
apply_metadata(&mut output, &settings.metadata);
output.write_header()
.map_err(|e| format!("Failed to write header: {}", e))?;
@ -729,7 +575,6 @@ fn export_aac<P: AsRef<Path>>(
settings.sample_rate,
settings.channels,
false,
None, // export never runs with a recording in flight
);
// Calculate how many samples we need from this chunk
@ -993,61 +838,4 @@ mod tests {
assert_eq!(ExportFormat::Wav.extension(), "wav");
assert_eq!(ExportFormat::Flac.extension(), "flac");
}
fn tagged_settings(format: ExportFormat) -> ExportSettings {
ExportSettings {
format,
sample_rate: 48000,
channels: 2,
bit_depth: 24,
mp3_bitrate: 192,
start_time: Seconds::ZERO,
end_time: Seconds(0.2), // tiny render
tempo_map: TempoMap::constant(120.0),
metadata: vec![
("title".to_string(), "Test Title".to_string()),
("artist".to_string(), "Test Artist".to_string()),
],
}
}
/// FLAC export must be a real FLAC container (not WAV bytes) carrying Vorbis-comment tags.
#[test]
fn flac_export_is_real_flac_with_tags() {
let settings = tagged_settings(ExportFormat::Flac);
let mut project = Project::new(48000);
let pool = AudioPool::new();
let path = std::env::temp_dir().join("lb_be_flac_test.flac");
export_audio(&mut project, &pool, &settings, &path, None).expect("FLAC export failed");
let bytes = std::fs::read(&path).unwrap();
assert_eq!(&bytes[0..4], b"fLaC", "not real FLAC (got {:?})", &bytes[0..4]);
let s = String::from_utf8_lossy(&bytes);
assert!(s.contains("Test Title"), "title tag missing from FLAC");
assert!(s.contains("Test Artist"), "artist tag missing from FLAC");
std::fs::remove_file(&path).ok();
}
/// WAV export keeps a valid RIFF container and gains a LIST/INFO tag chunk with a fixed-up size.
#[test]
fn wav_export_has_info_chunk() {
let settings = tagged_settings(ExportFormat::Wav);
let mut project = Project::new(48000);
let pool = AudioPool::new();
let path = std::env::temp_dir().join("lb_be_wav_test.wav");
export_audio(&mut project, &pool, &settings, &path, None).expect("WAV export failed");
let bytes = std::fs::read(&path).unwrap();
assert_eq!(&bytes[0..4], b"RIFF");
assert_eq!(&bytes[8..12], b"WAVE");
let riff_size = u32::from_le_bytes([bytes[4], bytes[5], bytes[6], bytes[7]]) as usize;
assert_eq!(riff_size, bytes.len() - 8, "RIFF size not fixed up after tagging");
let s = String::from_utf8_lossy(&bytes);
assert!(s.contains("LIST") && s.contains("INFO") && s.contains("INAM"),
"no RIFF INFO chunk");
assert!(s.contains("Test Title"), "title not in WAV INFO chunk");
std::fs::remove_file(&path).ok();
}
}

View File

@ -383,7 +383,6 @@ impl Project {
sample_rate: u32,
channels: u32,
live_only: bool,
recording_midi: Option<(TrackId, MidiClipId)>,
) {
output.fill(0.0);
@ -392,7 +391,6 @@ impl Project {
// Create initial render context
let ctx = RenderContext {
live_only,
recording_midi,
..RenderContext::new(playhead_seconds, tempo_map, sample_rate, channels, output.len())
};

View File

@ -5,61 +5,6 @@ use crate::time::{Beats, Seconds};
use std::collections::HashMap;
use std::path::PathBuf;
/// Cycle-recording bookkeeping attached to a recording that started with a cycle region armed.
///
/// Takes are sliced **geometrically** at stop, in exact `loop_len_frames` multiples — not at the
/// instant the wrap was detected. The playhead advances before the capture block in `process()`, so
/// the wrap instant isn't sample-exact against the buffer that was just captured, but the geometry
/// is. `wrap_count` therefore only decides *whether* this is a multi-take recording, never where the
/// cuts land.
#[derive(Debug, Clone, Copy)]
pub struct CycleRecordInfo {
/// Where the cycle region starts, in beats. Takes are laid down here, not at the punch-in point.
pub loop_start: Beats,
/// The cycle region's length in beats — what the take folder records as `recorded_loop_beats`.
pub loop_len_beats: Beats,
/// One cycle pass, in frames. The take size.
pub loop_len_frames: usize,
/// Frames between the region start and where capture actually began. Non-zero only for a
/// punch-in (record while already rolling); take 1 gets this much silence prepended so it still
/// spans the whole region.
pub lead_pad_frames: usize,
/// How many times the transport wrapped during this recording. Zero normally means the user
/// stopped before completing a pass, which stays an ordinary single recording — unless
/// `force_takes` says otherwise.
pub wrap_count: usize,
/// Cut takes even if the transport never wrapped.
///
/// Set when the region already holds takes: a further recording there is another take, however
/// short, and it gets padded out to the region like any partial pass. Without this a run that
/// stopped before the loop came round would land as a separate overlapping clip instead of
/// joining the take list.
///
/// The editor decides this at record start, because whether takes already exist is document
/// state the engine can't see.
pub force_takes: bool,
}
/// Min/max waveform peaks for a finished buffer of interleaved samples.
///
/// The live recording path builds its peaks incrementally as samples arrive; cycle takes don't
/// exist until the recording is sliced at stop, so they get theirs in one pass here.
pub fn compute_peaks(samples: &[f32], channels: u32, frames_per_peak: usize) -> Vec<WaveformPeak> {
let samples_per_peak = (frames_per_peak * channels.max(1) as usize).max(1);
samples
.chunks(samples_per_peak)
.map(|chunk| {
let mut min = 0.0f32;
let mut max = 0.0f32;
for s in chunk {
min = min.min(*s);
max = max.max(*s);
}
WaveformPeak { min, max }
})
.collect()
}
/// State of an active recording session
pub struct RecordingState {
/// Track being recorded to
@ -90,8 +35,6 @@ pub struct RecordingState {
pub frames_per_peak: usize,
/// All recorded audio data accumulated in memory (written to disk at finalization)
pub audio_data: Vec<f32>,
/// Cycle-recording bookkeeping, when a cycle region was armed at record start.
pub cycle: Option<CycleRecordInfo>,
}
impl RecordingState {
@ -126,69 +69,9 @@ impl RecordingState {
waveform_buffer: Vec::new(),
frames_per_peak,
audio_data: Vec::new(),
cycle: None,
}
}
/// Slice the recording into cycle takes: one per pass, each spanning the FULL cycle region.
///
/// Partial passes are padded with silence — the head of take 1 for a punch-in, the tail of the
/// last take when the user stops mid-pass — so every take is the same length and aligned to the
/// region. That uniformity is what makes comping-via-split work: take 1 on the left half and
/// take 3 on the right always line up.
///
/// Returns `None` if this wasn't a cycle recording or the transport never wrapped (an ordinary
/// single recording, which keeps the existing path untouched).
pub fn slice_takes(&self) -> Option<Vec<Vec<f32>>> {
let cycle = self.cycle?;
if (cycle.wrap_count == 0 && !cycle.force_takes) || cycle.loop_len_frames == 0 {
return None;
}
let ch = self.channels.max(1) as usize;
let take_len = cycle.loop_len_frames * ch;
let lead = cycle.lead_pad_frames * ch;
// The recording as positioned *within the region*: silence for the gap between the region
// start and the punch-in, then the captured audio. Slicing this at whole-take boundaries is
// the whole trick — take 1 comes out short-by-`lead` at the front, already padded.
let virtual_len = lead + self.audio_data.len();
let take_count = virtual_len.div_ceil(take_len);
let mut takes: Vec<Vec<f32>> = Vec::with_capacity(take_count);
for i in 0..take_count {
let mut take = vec![0.0f32; take_len];
let take_begin = i * take_len;
for slot in 0..take_len {
// Position in the virtual (lead-padded) buffer.
let v = take_begin + slot;
if v < lead {
continue; // still in the punch-in silence
}
match self.audio_data.get(v - lead) {
Some(s) => take[slot] = *s,
None => break, // past the end of capture; the rest stays silent
}
}
takes.push(take);
}
// A final take holding only a sliver of real audio is a stop artifact (the user hit stop a
// moment after the wrap), not a performance. Drop it — but only if it's actually a PARTIAL
// pass, and never the only take. A pass that filled the region is a real take no matter how
// short the region is.
const MIN_TAKE_SECONDS: f64 = 0.05;
if takes.len() > 1 {
let last_real_samples = virtual_len - (takes.len() - 1) * take_len;
let last_real_seconds = (last_real_samples / ch) as f64 / self.sample_rate as f64;
if last_real_samples < take_len && last_real_seconds < MIN_TAKE_SECONDS {
takes.pop();
}
}
Some(takes)
}
/// Add samples to the accumulation buffer
/// Returns true if a flush occurred
pub fn add_samples(&mut self, samples: &[f32]) -> Result<bool, std::io::Error> {
@ -306,26 +189,6 @@ pub struct MidiRecordingState {
active_notes: HashMap<u8, ActiveMidiNote>,
/// Completed notes: (time_offset, note, velocity, duration) — all times in beats
pub completed_notes: Vec<(Beats, u8, u8, Beats)>,
/// The cycle region's length in beats, if one was armed at record start.
///
/// A cycle MIDI recording is anchored at the region start (`start_time == loop_start`), which is
/// what makes MERGE fall out for free: the transport always wraps back into the region, so every
/// note's offset already lands inside `[0, loop_len)` and successive passes overdub onto each
/// other with no folding needed.
pub cycle_loop_len: Option<Beats>,
/// Whether the transport actually came round. Distinct from `cycle_loop_len`, which only says a
/// region was armed: the clip only pins to the full region once a pass has completed, so until
/// then the bar still grows with the playhead.
pub wrapped: bool,
/// Cut takes even if the transport never wrapped — see [`CycleRecordInfo::force_takes`].
pub force_takes: bool,
/// Which cycle pass is currently being recorded (0-based). Bumped at each wrap.
current_pass: usize,
/// The pass each completed note belongs to, parallel to `completed_notes`.
///
/// Only meaningful in "separate takes" mode, where each pass becomes its own MIDI clip. Merge
/// mode ignores it — all passes fold into one clip, which is the whole point.
note_pass: Vec<usize>,
}
impl MidiRecordingState {
@ -336,28 +199,9 @@ impl MidiRecordingState {
start_time,
active_notes: HashMap::new(),
completed_notes: Vec::new(),
cycle_loop_len: None,
wrapped: false,
force_takes: false,
current_pass: 0,
note_pass: Vec::new(),
}
}
/// Record a finished note, tagging it with the pass it was played in.
///
/// Every completion goes through here so `completed_notes` and `note_pass` can't drift apart.
fn push_completed(&mut self, note: &ActiveMidiNote, end_time: Beats) {
let note_start = note.start_time.max(self.start_time);
self.completed_notes.push((
note_start - self.start_time,
note.note,
note.velocity,
end_time - note_start,
));
self.note_pass.push(self.current_pass);
}
pub fn note_on(&mut self, note: u8, velocity: u8, absolute_time: Beats) {
self.active_notes.insert(note, ActiveMidiNote { note, velocity, start_time: absolute_time });
}
@ -367,38 +211,16 @@ impl MidiRecordingState {
if absolute_time <= self.start_time {
return;
}
self.push_completed(&active_note, absolute_time);
let note_start = active_note.start_time.max(self.start_time);
self.completed_notes.push((
note_start - self.start_time,
active_note.note,
active_note.velocity,
absolute_time - note_start,
));
}
}
/// Completed notes grouped by cycle pass — one bucket per pass, in recording order.
///
/// Used by "separate takes" mode, where each pass becomes its own MIDI clip.
///
/// An *interior* pass in which nothing was played still yields an empty take, so take N in the
/// folder is always pass N on the transport — otherwise the numbering would silently shift and
/// "take 3" would stop meaning "the third time round". A *trailing* empty pass is dropped
/// though: that's what you get by hitting stop shortly after a wrap, and it's a stop artifact
/// rather than a take you played. (Same reasoning as the audio path's short-final-take rule.)
pub fn notes_by_pass(&self, passes: usize) -> Vec<Vec<(Beats, u8, u8, Beats)>> {
let mut buckets = vec![Vec::new(); passes.max(1)];
for (note, &pass) in self.completed_notes.iter().zip(self.note_pass.iter()) {
if let Some(bucket) = buckets.get_mut(pass) {
bucket.push(*note);
}
}
// Never drop the only take.
while buckets.len() > 1 && buckets.last().is_some_and(|b| b.is_empty()) {
buckets.pop();
}
buckets
}
/// How many cycle passes this recording covered (1 if the transport never wrapped).
pub fn pass_count(&self) -> usize {
self.current_pass + 1
}
pub fn get_notes(&self) -> &[(Beats, u8, u8, Beats)] {
&self.completed_notes
}
@ -407,28 +229,18 @@ impl MidiRecordingState {
self.completed_notes.len()
}
/// The still-held notes, given a provisional duration running to `current_time`.
///
/// These belong to whatever pass is in progress, so a per-pass view can append them as-is.
pub fn active_notes_with_provisional_end(&self, current_time: Beats) -> Vec<(Beats, u8, u8, Beats)> {
self.active_notes
.values()
.map(|active| {
let note_start = active.start_time.max(self.start_time);
(
note_start - self.start_time,
active.note,
active.velocity,
(current_time - note_start).max(Beats::ZERO),
)
})
.collect()
}
/// Get all completed notes plus currently-held notes with a provisional duration.
pub fn get_notes_with_active(&self, current_time: Beats) -> Vec<(Beats, u8, u8, Beats)> {
let mut notes = self.completed_notes.clone();
notes.extend(self.active_notes_with_provisional_end(current_time));
for active in self.active_notes.values() {
let note_start = active.start_time.max(self.start_time);
notes.push((
note_start - self.start_time,
active.note,
active.velocity,
(current_time - note_start).max(Beats::ZERO),
));
}
notes
}
@ -440,259 +252,13 @@ impl MidiRecordingState {
let active_notes: Vec<_> = self.active_notes.drain().collect();
for (_note_num, active_note) in active_notes {
self.push_completed(&active_note, end_time);
let note_start = active_note.start_time.max(self.start_time);
self.completed_notes.push((
note_start - self.start_time,
active_note.note,
active_note.velocity,
end_time - note_start,
));
}
}
/// Handle a transport cycle wrap during MIDI recording.
///
/// Note times are stored as offsets from `start_time`, and the playhead jumps *backwards* at a
/// wrap — so a note still held across the boundary would otherwise get a nonsensical (negative)
/// duration, or never be closed at all. Write its note-off at `region_end` (exactly as
/// `close_active_notes` does when recording stops), then re-open it at `region_start` so a key
/// the player is still physically holding keeps being captured in the next pass. Mirrors the
/// way `handle_start_midi_recording` re-injects already-held notes at the recording start.
pub fn wrap_at_cycle(&mut self, region_end: Beats, region_start: Beats) {
// Snapshot the held notes (close_active_notes drains them and loses the velocities).
let held: Vec<(u8, u8)> = self
.active_notes
.values()
.map(|n| (n.note, n.velocity))
.collect();
// Close first, so a note held across the boundary has its tail attributed to the pass that's
// ending; then advance, so the re-opened half belongs to the pass that's beginning.
self.close_active_notes(region_end);
self.current_pass += 1;
for (note, velocity) in held {
self.note_on(note, velocity, region_start);
}
// A pass has completed, so from here the clip spans the whole region rather than however long
// the user happens to hold the record button.
self.cycle_loop_len = Some(region_end - region_start);
self.wrapped = true;
}
}
#[cfg(test)]
mod cycle_tests {
use super::*;
/// A recording state holding `audio_data`, armed for cycle recording. Mono, 100 Hz, so a frame
/// is a sample and 5 frames is 50 ms (exactly the min-take threshold).
fn rec_forced(audio: Vec<f32>, loop_len_frames: usize, lead_pad_frames: usize, wraps: usize) -> RecordingState {
let mut r = rec(audio, loop_len_frames, lead_pad_frames, wraps);
if let Some(c) = r.cycle.as_mut() {
c.force_takes = true;
}
r
}
fn rec(audio: Vec<f32>, loop_len_frames: usize, lead_pad_frames: usize, wraps: usize) -> RecordingState {
let mut r = RecordingState::new(
0,
0,
PathBuf::from("/dev/null"),
WavWriter::create(&PathBuf::from("/dev/null"), 100, 1).expect("wav writer"),
100,
1,
Beats(0.0),
1.0,
);
r.audio_data = audio;
r.cycle = Some(CycleRecordInfo {
loop_start: Beats(0.0),
loop_len_beats: Beats(4.0),
loop_len_frames,
lead_pad_frames,
wrap_count: wraps,
force_takes: false,
});
r
}
#[test]
fn no_wrap_is_not_a_cycle_recording() {
// Stopping before the transport ever wraps stays an ordinary single recording — the whole
// point of triggering on the wrap rather than on the cycle region merely existing.
let r = rec(vec![1.0; 10], 4, 0, 0);
assert!(r.slice_takes().is_none());
}
#[test]
fn force_takes_makes_a_partial_pass_a_take() {
// Recording over a region that already holds takes: this run is another take however short
// it ran, so it's cut and padded like any partial pass rather than landing as a separate
// overlapping clip. Two real frames of an 8-frame region -> one take, silence for the rest.
let takes = rec_forced(vec![1.0, 2.0], 8, 0, 0)
.slice_takes()
.expect("forced takes");
assert_eq!(takes.len(), 1);
assert_eq!(takes[0], vec![1.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]);
}
#[test]
fn takes_are_cut_at_exact_loop_multiples() {
// 12 frames of audio, 4-frame loop, started at the region start => 3 clean takes.
let audio: Vec<f32> = (1..=12).map(|i| i as f32).collect();
let takes = rec(audio, 4, 0, 2).slice_takes().expect("cycle takes");
assert_eq!(takes.len(), 3);
assert_eq!(takes[0], vec![1.0, 2.0, 3.0, 4.0]);
assert_eq!(takes[1], vec![5.0, 6.0, 7.0, 8.0]);
assert_eq!(takes[2], vec![9.0, 10.0, 11.0, 12.0]);
}
#[test]
fn punch_in_pads_the_head_of_take_one() {
// Punched in 2 frames into the region: take 1 gets 2 frames of silence at the FRONT so it
// still spans the whole region and lines up with every other take.
let audio: Vec<f32> = (1..=10).map(|i| i as f32).collect();
let takes = rec(audio, 4, 2, 2).slice_takes().expect("cycle takes");
assert_eq!(takes.len(), 3);
assert_eq!(takes[0], vec![0.0, 0.0, 1.0, 2.0]);
assert_eq!(takes[1], vec![3.0, 4.0, 5.0, 6.0]);
assert_eq!(takes[2], vec![7.0, 8.0, 9.0, 10.0]);
}
#[test]
fn stopping_mid_pass_pads_the_tail_of_the_last_take() {
// 13 frames, 8-frame loop => the second take holds 5 real frames (50 ms at 100 Hz, right at
// the keep threshold) and 3 of silence.
let audio: Vec<f32> = (1..=13).map(|i| i as f32).collect();
let takes = rec(audio, 8, 0, 1).slice_takes().expect("cycle takes");
assert_eq!(takes.len(), 2);
assert_eq!(takes[1], vec![9.0, 10.0, 11.0, 12.0, 13.0, 0.0, 0.0, 0.0]);
}
#[test]
fn every_take_is_the_same_length() {
// Uniform length is the invariant comping-via-split depends on.
let audio: Vec<f32> = (1..=23).map(|i| i as f32).collect();
let takes = rec(audio, 8, 3, 3).slice_takes().expect("cycle takes");
assert!(takes.iter().all(|t| t.len() == 8), "takes must be uniform");
}
#[test]
fn a_sliver_of_a_final_take_is_dropped() {
// Stopped 1 frame (10 ms at 100 Hz) after the wrap — below the 50 ms floor, so that stub of
// a take is a stop artifact and goes.
let audio: Vec<f32> = (1..=9).map(|i| i as f32).collect();
let takes = rec(audio, 8, 0, 1).slice_takes().expect("cycle takes");
assert_eq!(takes.len(), 1, "a 10ms tail take should be dropped");
assert_eq!(takes[0].len(), 8);
}
#[test]
fn a_full_final_take_is_never_dropped() {
// Regression: the sliver rule must only fire on a PARTIAL pass. A pass that filled the
// region is a real take however short the region is — an earlier version compared a full
// take's duration to the floor and silently ate it.
let audio: Vec<f32> = (1..=8).map(|i| i as f32).collect();
let takes = rec(audio, 4, 0, 1).slice_takes().expect("cycle takes");
assert_eq!(takes.len(), 2, "both passes filled the region");
assert_eq!(takes[1], vec![5.0, 6.0, 7.0, 8.0]);
}
}
#[cfg(test)]
mod midi_cycle_tests {
use super::*;
/// A MIDI recording anchored at the region start (beat 0), region 4 beats long.
fn rec() -> MidiRecordingState {
MidiRecordingState::new(0, 0, Beats(0.0))
}
/// One pass of the transport around a 4-beat region.
fn wrap(r: &mut MidiRecordingState) {
r.wrap_at_cycle(Beats(4.0), Beats(0.0));
}
#[test]
fn notes_are_bucketed_by_the_pass_they_were_played_in() {
let mut r = rec();
r.note_on(60, 100, Beats(1.0));
r.note_off(60, Beats(2.0)); // pass 0
wrap(&mut r);
r.note_on(62, 100, Beats(1.0));
r.note_off(62, Beats(2.0)); // pass 1
wrap(&mut r);
r.note_on(64, 100, Beats(1.0));
r.note_off(64, Beats(2.0)); // pass 2
assert_eq!(r.pass_count(), 3);
let by_pass = r.notes_by_pass(r.pass_count());
let pitches: Vec<Vec<u8>> = by_pass
.iter()
.map(|p| p.iter().map(|n| n.1).collect())
.collect();
assert_eq!(pitches, vec![vec![60], vec![62], vec![64]]);
}
#[test]
fn a_note_held_across_a_wrap_splits_between_the_two_passes() {
// The key is still down at the boundary: the sounding half belongs to the pass that's
// ending, and the re-opened half to the pass that's beginning. Getting the pass bump on the
// wrong side of close_active_notes would file the whole note under one pass.
let mut r = rec();
r.note_on(60, 100, Beats(3.0));
wrap(&mut r); // still held
r.note_off(60, Beats(1.0)); // released 1 beat into the next pass
assert_eq!(r.pass_count(), 2);
let by_pass = r.notes_by_pass(r.pass_count());
assert_eq!(by_pass[0].len(), 1, "the held half lands in the pass that ended");
assert_eq!(by_pass[1].len(), 1, "the re-opened half lands in the next pass");
// Pass 0's half runs from beat 3 to the region end at 4.
assert_eq!(by_pass[0][0].0, Beats(3.0));
assert_eq!(by_pass[0][0].3, Beats(1.0));
// Pass 1's half starts at the region start and runs to the release.
assert_eq!(by_pass[1][0].0, Beats(0.0));
assert_eq!(by_pass[1][0].3, Beats(1.0));
}
#[test]
fn a_silent_interior_pass_still_yields_an_empty_take() {
// Take N in the folder must be pass N on the transport, even if nothing was played — else
// the take numbering silently shifts under the user.
let mut r = rec();
r.note_on(60, 100, Beats(1.0));
r.note_off(60, Beats(2.0)); // pass 0
wrap(&mut r);
wrap(&mut r); // pass 1: played nothing
r.note_on(64, 100, Beats(1.0));
r.note_off(64, Beats(2.0)); // pass 2
let by_pass = r.notes_by_pass(r.pass_count());
assert_eq!(by_pass.len(), 3);
assert_eq!(by_pass[1].len(), 0, "the silent pass is still take 2");
assert_eq!(by_pass[2][0].1, 64);
}
#[test]
fn a_trailing_empty_pass_is_dropped() {
// Hitting stop shortly after a wrap leaves a pass you never played into. That's a stop
// artifact, not a take — unlike a silent pass in the middle, which was a deliberate rest.
let mut r = rec();
r.note_on(60, 100, Beats(1.0));
r.note_off(60, Beats(2.0)); // pass 0
wrap(&mut r);
r.note_on(62, 100, Beats(1.0));
r.note_off(62, Beats(2.0)); // pass 1
wrap(&mut r); // pass 2 begins... and the user hits stop
assert_eq!(r.pass_count(), 3);
let by_pass = r.notes_by_pass(r.pass_count());
assert_eq!(by_pass.len(), 2, "the empty trailing pass is not a take");
}
#[test]
fn an_empty_recording_still_yields_one_take() {
let mut r = rec();
wrap(&mut r);
wrap(&mut r);
assert_eq!(r.notes_by_pass(r.pass_count()).len(), 1);
}
}

View File

@ -1,6 +1,6 @@
use super::automation::{AutomationLane, AutomationLaneId, ParameterId};
use super::clip::{AudioClipInstance, AudioClipInstanceId};
use super::midi::{MidiClipId, MidiClipInstance, MidiClipInstanceId, MidiEvent};
use super::midi::{MidiClipInstance, MidiClipInstanceId, MidiEvent};
use super::midi_pool::MidiClipPool;
use super::node_graph::AudioGraph;
use super::node_graph::nodes::{AudioInputNode, AudioOutputNode};
@ -43,13 +43,6 @@ pub struct RenderContext<'a> {
/// Used after pause/stop to route note-off tails through the normal group hierarchy
/// without re-triggering notes from clips at the paused position.
pub live_only: bool,
/// The MIDI recording in progress, if any: (track being recorded to, clip being recorded into).
///
/// On that track, every OTHER clip is silenced for the duration of the recording. You're playing
/// a part into this region — hearing what's already there (a previous take, say) fighting with
/// what you're playing now is just noise. The clip being recorded into is exempt, because in
/// merge mode that's exactly what you DO want to hear: the overdub you've been building up.
pub recording_midi: Option<(TrackId, MidiClipId)>,
}
impl<'a> RenderContext<'a> {
@ -68,7 +61,6 @@ impl<'a> RenderContext<'a> {
buffer_size,
time_stretch: 1.0,
live_only: false,
recording_midi: None,
}
}
@ -872,21 +864,9 @@ impl MidiTrack {
let playhead_beats = ctx.playhead_beats();
let buffer_end_beats = ctx.buffer_end_beats();
// While recording into this track, every clip EXCEPT the one being recorded into is
// silenced. Otherwise a take folder already sitting in the cycle region would play its
// active take underneath you on every pass, fighting the part you're trying to record.
// The recording clip itself is exempt: in merge mode that's the overdub monitoring.
let muted_clip = match ctx.recording_midi {
Some((track_id, clip_id)) if track_id == self.id => Some(clip_id),
_ => None,
};
// Collect MIDI events from all clip instances that overlap with current beat range
let mut currently_active = HashSet::new();
for instance in &self.clip_instances {
if muted_clip.is_some_and(|recording| instance.clip_id != recording) {
continue;
}
if instance.overlaps_range(playhead_beats, buffer_end_beats) {
currently_active.insert(instance.id);
}

View File

@ -1,3 +1,3 @@
pub mod types;
pub use types::{AudioEvent, Command, MidiClipData, OscilloscopeData, Query, QueryResponse, TrimRange};
pub use types::{AudioEvent, Command, MidiClipData, OscilloscopeData, Query, QueryResponse};

View File

@ -8,21 +8,6 @@ use crate::audio::node_graph::nodes::LoopMode;
use crate::io::WaveformPeak;
use crate::time::{Beats, Seconds};
/// A clip's internal (content) boundaries, tagged with the domain they're measured in.
///
/// A clip's content time is SECONDS for sampled audio but BEATS for MIDI — the same polymorphism
/// `ClipInstance::trim_start`/`trim_end` carry. Passing these as bare `f64`s meant the caller and
/// the engine could disagree about the unit with nothing to catch it: an audio trim of "1.0" was
/// once stored as `Beats(1.0)` for the clip's external duration, so a 1-second split played back as
/// half a second at 120 BPM. Tagging the domain makes that a type error instead of a bug report.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum TrimRange {
/// Sampled-audio content time.
Seconds { start: Seconds, end: Seconds },
/// MIDI content time.
Beats { start: Beats, end: Beats },
}
/// Commands sent from UI/control thread to audio thread
#[derive(Debug, Clone)]
pub enum Command {
@ -34,7 +19,7 @@ pub enum Command {
/// Pause playback (maintains position)
Pause,
/// Seek to a specific position in seconds
Seek(Seconds),
Seek(f64),
// Track management commands
/// Set track volume (0.0 = silence, 1.0 = unity gain)
@ -46,12 +31,13 @@ pub enum Command {
// Clip management commands
/// Move a clip to a new timeline position (track_id, clip_id, new_external_start)
MoveClip(TrackId, ClipId, Beats),
/// Trim a clip's internal boundaries — which portion of the source content is used.
TrimClip(TrackId, ClipId, TrimRange),
MoveClip(TrackId, ClipId, f64),
/// Trim a clip's internal boundaries (track_id, clip_id, new_internal_start, new_internal_end)
/// This changes which portion of the source content is used
TrimClip(TrackId, ClipId, f64, f64),
/// Extend/shrink a clip's external duration (track_id, clip_id, new_external_duration)
/// If duration > internal duration, the clip will loop
ExtendClip(TrackId, ClipId, Beats),
ExtendClip(TrackId, ClipId, f64),
// Metatrack management commands
/// Create a new metatrack with a name and optional parent group
@ -67,15 +53,15 @@ pub enum Command {
SetTimeStretch(TrackId, f32),
/// Set metatrack time offset in seconds (track_id, offset)
/// Positive = shift content later, negative = shift earlier
SetOffset(TrackId, Seconds),
SetOffset(TrackId, f64),
/// Set metatrack pitch shift in semitones (track_id, semitones) - for future use
SetPitchShift(TrackId, f32),
/// Set metatrack trim start in seconds (track_id, trim_start)
/// Children won't hear content before this point
SetTrimStart(TrackId, Seconds),
SetTrimStart(TrackId, f64),
/// Set metatrack trim end in seconds (track_id, trim_end)
/// None means no end trim
SetTrimEnd(TrackId, Option<Seconds>),
SetTrimEnd(TrackId, Option<f64>),
// Audio track commands
/// Create a new audio track with a name and optional parent group
@ -85,8 +71,7 @@ pub enum Command {
AddAudioFile(String, Vec<f32>, u32, u32),
/// Add a clip to an audio track (track_id, clip_id, pool_index, start_time, duration, offset)
/// The clip_id is pre-assigned by the caller (via EngineController::next_audio_clip_id())
/// (track, clip_id, pool_index, start_time [beats], duration [beats], offset [seconds])
AddAudioClip(TrackId, AudioClipInstanceId, usize, Beats, Beats, Seconds),
AddAudioClip(TrackId, AudioClipInstanceId, usize, f64, f64, f64),
// MIDI commands
/// Create a new MIDI track with a name and optional parent group
@ -94,14 +79,14 @@ pub enum Command {
/// Add a MIDI clip to the pool without placing it on a track
AddMidiClipToPool(MidiClip),
/// Create a new MIDI clip on a track (track_id, start_time, duration)
CreateMidiClip(TrackId, Beats, Beats),
CreateMidiClip(TrackId, f64, f64),
/// Add a MIDI note to a clip (track_id, clip_id, time_offset, note, velocity, duration)
AddMidiNote(TrackId, MidiClipId, Beats, u8, u8, Beats),
AddMidiNote(TrackId, MidiClipId, f64, u8, u8, f64),
/// Add a pre-loaded MIDI clip to a track (track_id, clip, start_time)
AddLoadedMidiClip(TrackId, MidiClip, Beats),
AddLoadedMidiClip(TrackId, MidiClip, f64),
/// Update MIDI clip notes (track_id, clip_id, notes: Vec<(start_time, note, velocity, duration)>)
/// NOTE: May need to switch to individual note operations if this becomes slow on clips with many notes
UpdateMidiClipNotes(TrackId, MidiClipId, Vec<(Beats, u8, u8, Beats)>),
UpdateMidiClipNotes(TrackId, MidiClipId, Vec<(f64, u8, u8, f64)>),
/// Replace all events in a MIDI clip (track_id, clip_id, events). Used for CC/pitch bend editing.
UpdateMidiClipEvents(TrackId, MidiClipId, Vec<MidiEvent>),
/// Remove a MIDI clip instance from a track (track_id, instance_id) - for undo/redo support
@ -117,9 +102,9 @@ pub enum Command {
/// Create a new automation lane on a track (track_id, parameter_id)
CreateAutomationLane(TrackId, ParameterId),
/// Add an automation point to a lane (track_id, lane_id, time, value, curve)
AddAutomationPoint(TrackId, AutomationLaneId, Beats, f32, CurveType),
AddAutomationPoint(TrackId, AutomationLaneId, f64, f32, CurveType),
/// Remove an automation point at a specific time (track_id, lane_id, time, tolerance)
RemoveAutomationPoint(TrackId, AutomationLaneId, Beats, Beats),
RemoveAutomationPoint(TrackId, AutomationLaneId, f64, f64),
/// Clear all automation points from a lane (track_id, lane_id)
ClearAutomationLane(TrackId, AutomationLaneId),
/// Remove an automation lane (track_id, lane_id)
@ -127,24 +112,9 @@ pub enum Command {
/// Enable/disable an automation lane (track_id, lane_id, enabled)
SetAutomationLaneEnabled(TrackId, AutomationLaneId, bool),
// Transport cycle (loop) region
/// Set the cycle region the transport loops over, in beats (None clears it).
/// Authored in beats so it survives tempo changes.
SetLoopRegion(Option<(Beats, Beats)>),
/// Enable/disable wrapping at the cycle region's end.
SetLoopEnabled(bool),
/// How a cycle MIDI recording treats its passes.
///
/// `false` (default) = MERGE: every pass overdubs into one clip. `true` = SEPARATE TAKES: each
/// pass becomes its own MIDI clip, and the editor folds them into a take folder — the same shape
/// audio always gets.
SetCycleMidiSeparateTakes(bool),
// Recording commands
/// Start recording on a track (track_id, start_time)
/// (track, start_time, force_takes — cut takes even if the transport never wraps, because the
/// region already holds takes and this is another one)
StartRecording(TrackId, Beats, bool),
StartRecording(TrackId, Beats),
/// Stop the current recording
StopRecording,
/// Pause the current recording
@ -154,8 +124,7 @@ pub enum Command {
// MIDI Recording commands
/// Start MIDI recording on a track (track_id, clip_id, start_time)
/// (track, clip, start_time, force_takes — see [`Command::StartRecording`])
StartMidiRecording(TrackId, MidiClipId, Beats, bool),
StartMidiRecording(TrackId, MidiClipId, Beats),
/// Stop the current MIDI recording
StopMidiRecording,
@ -267,9 +236,9 @@ pub enum Command {
// Automation Input Node commands
/// Add or update a keyframe on an AutomationInput node (track_id, node_id, time, value, interpolation, ease_out, ease_in)
AutomationAddKeyframe(TrackId, u32, Beats, f32, String, (f32, f32), (f32, f32)),
AutomationAddKeyframe(TrackId, u32, f64, f32, String, (f32, f32), (f32, f32)),
/// Remove a keyframe from an AutomationInput node (track_id, node_id, time)
AutomationRemoveKeyframe(TrackId, u32, Beats),
AutomationRemoveKeyframe(TrackId, u32, f64),
/// Set the display name of an AutomationInput node (track_id, node_id, name)
AutomationSetName(TrackId, u32, String),
@ -301,7 +270,7 @@ pub enum Command {
#[derive(Debug, Clone)]
pub enum AudioEvent {
/// Current playback position in seconds
PlaybackPosition(Seconds),
PlaybackPosition(f64),
/// Playback has stopped (reached end of audio)
PlaybackStopped,
/// Audio buffer underrun detected
@ -322,35 +291,6 @@ pub enum AudioEvent {
RecordingProgress(ClipId, Seconds),
/// Recording stopped (clip_id, pool_index, waveform)
RecordingStopped(ClipId, usize, Vec<WaveformPeak>),
/// A MIDI recording that wrapped the cycle region at least once, in SEPARATE TAKES mode.
///
/// One MIDI clip per pass, in recording order. (Merge mode emits the ordinary
/// `MidiRecordingStopped` instead — all passes are already folded into the one clip.)
MidiCycleRecordingStopped {
track_id: TrackId,
/// One pool MIDI clip per pass. The first is the clip the recording started on.
clip_ids: Vec<MidiClipId>,
/// Where the takes sit on the timeline — the cycle region's start.
loop_start: Beats,
/// The region's length in beats: every take spans exactly this.
loop_len_beats: Beats,
},
/// A recording that wrapped the cycle region at least once, and so became multi-take.
///
/// Each take spans the full region and they're all the same length (partial passes are padded
/// with silence), so the editor can promote the recording clip straight to a take folder.
CycleRecordingStopped {
clip_id: ClipId,
/// One entry per pass: (audio pool index, waveform peaks), in recording order.
takes: Vec<(usize, Vec<WaveformPeak>)>,
/// Where the takes sit on the timeline — the cycle region's start, not the punch-in point.
loop_start: Beats,
/// The region's length in beats (what the take folder stores as `recorded_loop_beats`).
loop_len_beats: Beats,
/// The same length in seconds — the take folder's content duration, which is seconds-domain
/// for audio.
loop_len_seconds: Seconds,
},
/// Recording error (error_message)
RecordingError(String),
/// MIDI recording stopped (track_id, clip_id, note_count)
@ -401,7 +341,7 @@ pub enum AudioEvent {
WaveformChunksReady {
pool_index: usize,
detail_level: u8,
chunks: Vec<(u32, (Seconds, Seconds), Vec<WaveformPeak>)>,
chunks: Vec<(u32, (f64, f64), Vec<WaveformPeak>)>,
},
/// An audio file has been imported and is ready for playback.
@ -412,7 +352,7 @@ pub enum AudioEvent {
path: String,
channels: u32,
sample_rate: u32,
duration: Seconds,
duration: f64,
format: crate::io::audio_file::AudioFormat,
},
@ -486,7 +426,7 @@ pub enum Query {
/// Export audio to file (settings, output_path)
ExportAudio(crate::audio::ExportSettings, std::path::PathBuf),
/// Add a MIDI clip to a track synchronously (track_id, clip, start_time) - returns instance ID
AddMidiClipSync(TrackId, crate::audio::midi::MidiClip, Beats),
AddMidiClipSync(TrackId, crate::audio::midi::MidiClip, f64),
/// Add a MIDI clip instance to a track synchronously (track_id, instance) - returns instance ID
/// The clip must already exist in the MidiClipPool
AddMidiClipInstanceSync(TrackId, crate::audio::midi::MidiClipInstance),
@ -531,21 +471,16 @@ pub struct OscilloscopeData {
}
/// MIDI clip data for serialization
///
/// `Beats`/`Seconds` are `#[serde(transparent)]`, so naming the domain here costs nothing on disk —
/// the `.beam` still holds a plain number.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct MidiClipData {
/// MIDI content length is musical, so beats.
pub duration: Beats,
pub duration: f64,
pub events: Vec<crate::audio::midi::MidiEvent>,
}
/// Automation keyframe data for serialization
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct AutomationKeyframeData {
/// Automation x-axes are all beats.
pub time: Beats,
pub time: f64,
pub value: f32,
pub interpolation: String,
pub ease_out: (f32, f32),
@ -582,7 +517,7 @@ pub enum QueryResponse {
/// Pool waveform data
PoolWaveform(Result<Vec<crate::io::WaveformPeak>, String>),
/// Pool file info (duration, sample_rate, channels)
PoolFileInfo(Result<(Seconds, u32, u32), String>),
PoolFileInfo(Result<(f64, u32, u32), String>),
/// Audio exported
AudioExported(Result<(), String>),
/// MIDI clip instance added (returns instance ID)

View File

@ -20,7 +20,7 @@ pub use audio::{
TrackNode,
};
pub use audio::node_graph::{GraphPreset, AudioGraph, PresetMetadata, SerializedConnection, SerializedNode};
pub use time::{Beats, ContentTime, Seconds};
pub use time::{Beats, Seconds};
pub use tempo_map::{TempoEntry, TempoInterpolation, TempoMap, beats_to_seconds_stack, seconds_to_beats_stack};
pub use command::{AudioEvent, Command, OscilloscopeData};
pub use command::types::AutomationKeyframeData;

View File

@ -16,50 +16,6 @@ pub struct Beats(pub f64);
#[serde(transparent)]
pub struct Seconds(pub f64);
/// A time *inside a clip's own content*, in whatever unit that clip measures content in.
///
/// Clip content time is domain-polymorphic: SECONDS for sampled audio, video and vector, but BEATS
/// for MIDI (musical, so it survives tempo changes). `ClipInstance::trim_start`/`trim_end` are
/// content times, and storing them as bare `f64`s is what let a seconds delta get added to a MIDI
/// clip's beats trim — splitting a MIDI clip at beat 4 landed at beat 2 at 120 BPM.
///
/// This type is deliberately a **dead end**: it has no `.to_seconds()`, no `.to_beats()`, and no
/// arithmetic with `Seconds` or `Beats`. Content times can be compared and combined with each other
/// (that's domain-safe — both operands are in the same clip's domain), but the only way to get a
/// real timeline duration out is to resolve it against the clip that knows the domain, via
/// `AudioClip::resolve_content_time` / `Document::resolve_content_time`. So a passthrough costs
/// nothing, and mixing domains won't compile.
#[derive(Debug, Clone, Copy, PartialEq, PartialOrd, Default, Serialize, Deserialize)]
#[serde(transparent)]
pub struct ContentTime(pub f64);
impl ContentTime {
pub const ZERO: Self = Self(0.0);
pub fn max(self, other: Self) -> Self { Self(self.0.max(other.0)) }
pub fn min(self, other: Self) -> Self { Self(self.0.min(other.0)) }
/// The raw magnitude, with the domain discarded.
///
/// Only for code that is *already* working in this clip's content domain (trim arithmetic,
/// serialization, drawing a waveform whose x-axis is the clip's own content). If you are about
/// to combine this with a timeline position, resolve it against the clip instead.
pub fn raw(self) -> f64 { self.0 }
}
impl Add for ContentTime {
type Output = Self;
fn add(self, rhs: Self) -> Self { Self(self.0 + rhs.0) }
}
impl Sub for ContentTime {
type Output = Self;
fn sub(self, rhs: Self) -> Self { Self(self.0 - rhs.0) }
}
impl Rem for ContentTime {
type Output = Self;
fn rem(self, rhs: Self) -> Self { Self(self.0 % rhs.0) }
}
impl Beats {
pub const ZERO: Self = Self(0.0);

View File

@ -556,7 +556,7 @@ pub fn run_tui(
while let Ok(event) = rx.pop() {
match event {
AudioEvent::PlaybackPosition(pos) => {
app.update_playback_position(pos.seconds_to_f64());
app.update_playback_position(pos);
}
AudioEvent::PlaybackStopped => {
app.set_playing(false);
@ -790,7 +790,7 @@ fn execute_command(
return Err("Usage: seek <seconds>".to_string());
}
let pos: f64 = parts[1].parse().map_err(|_| "Invalid position")?;
controller.seek(crate::Seconds(pos));
controller.seek(pos);
app.set_status(format!("Seeked to {:.2}s", pos));
}
"track" => {
@ -830,7 +830,7 @@ fn execute_command(
app.next_clip_id += 1;
app.add_clip(track_id, clip_id, start_time, duration, format!("Clip {}", clip_id), Vec::new());
controller.create_midi_clip(track_id, crate::Beats(start_time), crate::Beats(duration));
controller.create_midi_clip(track_id, start_time, duration);
app.set_status(format!("Created MIDI clip on track {} at {:.2}s for {:.2}s", track_id, start_time, duration));
}
"loadmidi" => {
@ -882,7 +882,7 @@ fn execute_command(
app.next_clip_id += 1;
// Send to audio engine with the start_time (clip content is separate from timeline position)
controller.add_loaded_midi_clip(track_id, midi_clip, crate::Beats(start_time));
controller.add_loaded_midi_clip(track_id, midi_clip, start_time);
app.set_status(format!("Loaded {} ({} events, {:.2}s) to track {} at {:.2}s",
file_path, event_count, duration, track_id, start_time));

View File

@ -3615,7 +3615,6 @@ dependencies = [
"rusqlite",
"serde",
"serde_json",
"skrifa 0.43.2",
"tiny-skia",
"uuid",
"vello",
@ -3628,7 +3627,7 @@ dependencies = [
[[package]]
name = "lightningbeam-editor"
version = "1.0.9-alpha"
version = "1.0.7-alpha"
dependencies = [
"beamdsp",
"bytemuck",
@ -3642,7 +3641,6 @@ dependencies = [
"egui_extras",
"egui_node_graph2",
"ffmpeg-next",
"gif",
"gpu-video-encoder",
"half",
"image",

View File

@ -83,15 +83,6 @@ opt-level = 2
[profile.dev.package.cpal]
opt-level = 2
# GIF export: NeuQuant palette quantization is tight numeric loops that are punishingly slow
# unoptimized (1030× in debug). Optimize the encoder crates even in dev builds.
[profile.dev.package.gif]
opt-level = 3
[profile.dev.package.color_quant]
opt-level = 3
[profile.dev.package.weezl]
opt-level = 3
# Use local egui fork with ibus/Wayland text input fix
[patch.crates-io]
egui = { path = "../../egui-fork/crates/egui" }

View File

@ -0,0 +1,102 @@
# GPU-resident video decode + dynamic decode resolution
## Context
Profiling the zero-copy H.264 export (single Group[Video, Audio] clip, `LB_RENDER_PROFILE=1`)
broke the per-frame CPU "render" bucket down as:
| Cost (ms/frame) | 1080p | 4K | What it is |
|-------------------------|-------|-------|-----------------------------------------------------|
| decode | 3.1 | 19.0 | software ffmpeg decode (`video.rs::get_frame`) |
| background re-render | 3.6 | 7.5 | static background pushed through Vello *every frame* |
| video upload + blit | 4.1 | 4.2 | per-frame transient texture alloc + `write_texture` |
| srgb | 0.4 | 0.4 | linear→sRGB pass |
The video correctly takes the GPU Video-instance path (not Vello-baked) — `LB_LAYER_DEBUG=1`
shows `Video (1 instance)`. So the cost is **the video frame itself**: software decode, then an
8 MB `write_texture` upload of the decoded RGBA every frame. At 4K, software decode (19 ms)
dominates everything.
### Two correctness problems found alongside the perf issue
1. **Decode resolution is frozen to document size at import.** `load_video(clip, src, doc_w, doc_h)`
(`main.rs:4302`) sizes the decoder's swscale output to the document, capped to never upscale
(`video.rs:149`). Export *reuses that decoder*, so exporting **above** document resolution yields
video that was decoded to ≤document res and then GPU-**up**scaled — real source detail thrown away.
2. **It can't follow the consumer or a document resize.** Preview wants small/fast frames; export
wants full res; changing the document size should re-target the decode. None of that works with a
size frozen at import.
## Goal
Decouple **decode resolution** from import/document size: the renderer requests a frame *at a target
resolution*, and the decode path produces it. Hardware-decode H.264 (and later HEVC/AV1) into a GPU
surface and keep it GPU-resident through composite into the encoder — no CPU frame copy in either
direction. Software decode stays a **first-class** path (codecs/platforms without HW support), decoding
at the requested target res. This fixes the 4K decode wall, the 8 MB upload, *and* the resolution bugs.
## Design principles
- **Decode native, scale to the consumer's target.**
- *Hardware path:* decode into a native VAAPI surface → import as a wgpu texture (reuse the
`gpu-video-encoder` `dmabuf.rs` / `vk_device.rs` plumbing, read direction) → the GPU blit that
already composites the Video instance scales native→target for free. Handles any target res and
document resizes inherently; the cached frame is a native GPU texture.
- *Software path:* decode native → `swscale` to the requested target (the reusable scaler is keyed
on input format/size **and** output size — rebuilt when the target changes). Preview requests
preview res (cheap); export requests export res (full quality).
- **`VideoManager::get_frame` takes a target `(w, h)`** instead of relying on a frozen output size.
The frame cache is keyed to handle multiple live targets (preview + export) — either cache native
frames and scale on demand, or key by `(clip, ts, target)`; decide in Stage 2 by measuring cache
hit/scale tradeoff.
- **Software is not optional.** Hardware decode is an acceleration of the same `get_frame` contract,
selected per source when the codec/driver supports it; everything falls back to software cleanly.
## Approach (staged; each stage compiles + is independently useful)
### Stage 0 — independent quick wins (not blocked on decode)
- **Cache the static background** (`composite_document_to_hdr`): render once, reuse via a persistent
HDR texture (copy-in each frame) instead of a full Vello render + 2 passes/submits every frame.
Recovers ~3.6 ms (1080p) / ~7.5 ms (4K) per frame on *every* export. (In flight.)
### Stage 1 — software: decode at the requested target res (testable; fixes the quality bug now)
- Change `VideoManager::get_frame(clip, ts)``get_frame(clip, ts, target_w, target_h)`; thread the
target from the renderer (preview = current doc/preview res, export = export res). Cap at native.
- Rework `VideoDecoder` so output size is per-request, not frozen at construction; cache the swscale
context per output size (already cached per stream — extend the key). Adjust the frame cache key.
- Result: software exports are full-quality at any export res, and document resizes re-target decode.
No hardware needed; this is the correctness fix for the codecs HW can't handle anyway.
### Stage 2 — hardware decode primitive (headless-testable here, like the 8 encode tests)
- In `gpu-video-encoder` (rename → `gpu-video-codec`): `h264_vaapi`-style **decode** → VAAPI surface →
export DMA-BUF → import as a wgpu texture. Hardware test: decode a known file, verify dims/contents.
### Stage 3 — wire hardware decode into `get_frame` (blind; user-verifies)
- When the source codec/driver is HW-decodable, `get_frame` returns a **GPU texture** (native res)
instead of `Arc<Vec<u8>>`; the compositor uses it directly (no `write_texture`), GPU-scaling to the
target. For the zero-copy export the frame never leaves the GPU: **decode → composite → encode** on
one device. Software path is the fallback for everything else.
## Critical files
- `lightningbeam-core/src/video.rs``VideoDecoder` (per-request output size, scaler cache),
`VideoManager::get_frame` (target param, cache key).
- `lightningbeam-core/src/renderer.rs` — pass the render target res into the video-instance build.
- `lightningbeam-editor/src/export/video_exporter.rs` — background cache (Stage 0); consume a GPU
texture instead of uploading RGBA (Stage 3).
- `gpu-video-encoder/` (→ `gpu-video-codec`) — `dmabuf.rs`/`vk_device.rs` reused for the decode import.
## Risks
- **Codec coverage** — only some codecs are HW-decodable per GPU/driver; software must stay correct
and well-tested. Selection must probe support per source, not assume.
- **Cache memory** — native-res GPU textures (esp. 4K) are large; the frame cache budget needs revisiting.
- **Colorspace/format** — VAAPI decode surfaces are NV12/tiled; the existing import handles NV12, but
10-bit/HDR sources (P010) need format handling.
- **Preview vs export sharing** — two live targets (preview res + export res) from the same source; the
cache/scaler design must serve both without thrashing.
## Verification
- Stage 0/1: visual — export above document res is now full-quality (not upscaled); profile shows
background ≈ 0 and (Stage 1) software export correct at the chosen res.
- Stage 2: headless hardware test in `gpu-video-codec` (decode → wgpu texture, ffprobe/byte checks).
- Stage 3 (user): 1080p + 4K H.264 export — decode/upload buckets collapse; software fallback for a
non-HW codec (e.g. ProRes) still produces correct full-res output.

View File

@ -20,9 +20,6 @@ vello = { workspace = true }
# Text layout/shaping (text layers)
parley = { workspace = true }
# Glyph outline extraction for lossless text→SVG export. Pinned to the version parley resolves
# (0.43) so glyph IDs / normalized coords from parley layouts match this outliner.
skrifa = "0.43"
# Image decoding for image fills
image = { workspace = true }

View File

@ -47,113 +47,6 @@ pub struct BackendContext<'a> {
// Future: pub video_controller: Option<&'a mut VideoController>,
}
impl BackendContext<'_> {
/// Hand a clip instance to the audio engine and record it in the instance→backend map.
///
/// Take folders are resolved through the instance's `active_take`, so the backend gets whichever
/// take is selected. Returns the backend track and instance IDs, or `None` when there's nothing
/// to sync yet (a recording in progress, or an empty take folder).
///
/// Lives here rather than in any one action because more than one action needs it: adding an
/// instance, and switching a take folder's active take (which is a remove + re-add, there being
/// no in-place pool-swap command). Keeping one implementation keeps the trim/duration
/// conversions — the easy thing to get subtly wrong, since `trim_*` is SECONDS while
/// `timeline_*` is BEATS — from drifting between copies.
pub fn add_clip_instance(
&mut self,
document: &Document,
layer_id: &Uuid,
instance: &crate::clip::ClipInstance,
) -> Result<Option<(daw_backend::TrackId, BackendClipInstanceId)>, String> {
use crate::clip::ResolvedContent;
let clip = document
.get_audio_clip(&instance.clip_id)
.ok_or_else(|| format!("Audio clip {} not found", instance.clip_id))?;
let track_id = *self
.layer_to_track_map
.get(layer_id)
.ok_or_else(|| format!("Layer {} not mapped to backend track", layer_id))?;
let resolved = instance.resolve(clip);
let content = clip.content_duration();
let internal_start = instance.trim_start;
let internal_end = instance
.trim_end
.unwrap_or(daw_backend::ContentTime(content.native()));
let start_time = instance.timeline_start;
// How long the clip occupies the timeline, in BEATS. `effective_duration_beats` resolves the
// content window in the clip's own domain — beats content carries over directly, wall-clock
// content converts at the clip's position — so neither kind can be read as the other here.
let effective_duration = instance.effective_duration_beats(content, document.tempo_map());
let controller = self
.audio_controller
.as_mut()
.ok_or_else(|| "Audio controller not available".to_string())?;
let backend_id = match resolved {
ResolvedContent::Midi { midi_clip_id } => {
use daw_backend::command::{Query, QueryResponse};
// MIDI content time IS beats, so the trims carry straight over.
let midi_instance = daw_backend::MidiClipInstance::new(
0, // assigned by the backend
midi_clip_id,
daw_backend::Beats(internal_start.raw()),
daw_backend::Beats(internal_end.raw()),
start_time,
effective_duration,
);
match controller
.send_query(Query::AddMidiClipInstanceSync(track_id, midi_instance))?
{
QueryResponse::MidiClipInstanceAdded(Ok(id)) => BackendClipInstanceId::Midi(id),
QueryResponse::MidiClipInstanceAdded(Err(e)) => return Err(e),
_ => return Err("Unexpected query response".to_string()),
}
}
ResolvedContent::Audio { audio_pool_index } => {
// Sampled-audio content time is SECONDS; the backend's start/duration are BEATS.
let id = controller.add_audio_clip(
track_id,
audio_pool_index,
start_time,
effective_duration,
daw_backend::Seconds(internal_start.raw()),
);
BackendClipInstanceId::Audio(id)
}
// Nothing to sync until it has content.
ResolvedContent::Recording => return Ok(None),
};
self.clip_instance_to_backend_map
.insert(instance.id, backend_id);
Ok(Some((track_id, backend_id)))
}
/// Remove a clip instance's backend clip and drop it from the instance→backend map.
pub fn remove_clip_instance(
&mut self,
track_id: daw_backend::TrackId,
backend_id: BackendClipInstanceId,
instance_id: Uuid,
) {
if let Some(controller) = self.audio_controller.as_mut() {
match backend_id {
BackendClipInstanceId::Midi(id) => controller.remove_midi_clip(track_id, id),
BackendClipInstanceId::Audio(id) => controller.remove_audio_clip(track_id, id),
}
}
self.clip_instance_to_backend_map.remove(&instance_id);
}
}
/// Action trait for undo/redo operations
///
/// Each action must be able to execute (apply changes) and rollback (undo changes).
@ -343,24 +236,6 @@ impl ActionExecutor {
Ok(())
}
/// Register an action whose effect has **already been applied** to the document (and backend)
/// outside the executor — e.g. a recording, which streams its content into the document live
/// over time and can't be applied by a single synchronous `execute()`.
///
/// Unlike `execute`, this does NOT run `execute()`/`execute_backend()` (the effect is already
/// present). It clears the redo stack, bumps the epoch (so the document reads as modified), and
/// pushes the action so it becomes undoable: undo runs `rollback`/`rollback_backend` to remove
/// the content, redo runs `execute`/`execute_backend` to bring it back. The action must be
/// constructed already in its post-execute state (see e.g. `AddClipInstanceAction::already_applied`).
pub fn push_applied(&mut self, action: Box<dyn Action>) {
self.redo_stack.clear();
self.epoch = self.epoch.wrapping_add(1);
self.undo_stack.push(action);
if self.undo_stack.len() > self.max_undo_depth {
self.undo_stack.remove(0);
}
}
/// Undo the last action
///
/// Returns Ok(true) if an action was undone, Ok(false) if undo stack is empty,
@ -372,7 +247,6 @@ impl ActionExecutor {
Ok(()) => {
// Move to redo stack
self.redo_stack.push(action);
self.epoch = self.epoch.wrapping_add(1);
Ok(true)
}
Err(e) => {
@ -397,7 +271,6 @@ impl ActionExecutor {
Ok(()) => {
// Move back to undo stack
self.undo_stack.push(action);
self.epoch = self.epoch.wrapping_add(1);
Ok(true)
}
Err(e) => {
@ -571,7 +444,6 @@ impl ActionExecutor {
// Move to redo stack
self.redo_stack.push(action);
self.epoch = self.epoch.wrapping_add(1);
Ok(true)
} else {
@ -609,7 +481,6 @@ impl ActionExecutor {
// Move back to undo stack
self.undo_stack.push(action);
self.epoch = self.epoch.wrapping_add(1);
Ok(true)
} else {

View File

@ -47,31 +47,6 @@ impl AddClipInstanceAction {
}
}
/// Construct the action in its **already-applied** state: the clip instance is already in the
/// document and its backend clip already exists (e.g. a finished recording). Pair with
/// `ActionExecutor::push_applied` so the recording becomes undoable without re-adding anything.
/// Undo will `rollback`/`rollback_backend` (removing the clip from doc + backend via the seeded
/// ids); redo re-adds it via the normal `execute`/`execute_backend` path.
pub fn already_applied(
layer_id: Uuid,
clip_instance: ClipInstance,
backend_track_id: daw_backend::TrackId,
backend_id: crate::action::BackendClipInstanceId,
) -> Self {
let (backend_midi_instance_id, backend_audio_instance_id) = match backend_id {
crate::action::BackendClipInstanceId::Midi(id) => (Some(id), None),
crate::action::BackendClipInstanceId::Audio(id) => (None, Some(id)),
};
Self {
layer_id,
clip_instance,
executed: true, // already present in the document
backend_track_id: Some(backend_track_id),
backend_midi_instance_id,
backend_audio_instance_id,
}
}
/// Get the ID of the clip instance that will be/was added
pub fn clip_instance_id(&self) -> Uuid {
self.clip_instance.id
@ -85,15 +60,13 @@ impl AddClipInstanceAction {
impl Action for AddClipInstanceAction {
fn execute(&mut self, document: &mut Document) -> Result<(), String> {
// Calculate the clip's effective duration in BEATS for overlap testing.
// `get_clip_duration` is the content length in seconds; the placement span
// must be beats (the timeline is beats-domain), so convert via the clip's
// typed helper rather than treating the seconds span as beats.
// The clip's content duration in ITS OWN domain, so the trims resolve correctly for MIDI.
let clip_content = document.clip_trim_duration(&self.clip_instance.clip_id)
// Calculate the clip's effective duration
let clip_duration = document.get_clip_duration(&self.clip_instance.clip_id)
.ok_or_else(|| format!("Clip {} not found", self.clip_instance.clip_id))?;
let effective_duration = self.clip_instance
.effective_duration_beats(clip_content, document.tempo_map());
let trim_start = self.clip_instance.trim_start;
let trim_end = self.clip_instance.trim_end.unwrap_or(clip_duration);
let effective_duration = trim_end - trim_start;
// Auto-adjust position for audio/video layers to avoid overlaps
let adjusted_start = document.find_nearest_valid_position(
@ -197,23 +170,112 @@ impl Action for AddClipInstanceAction {
return Ok(());
}
// Add via the shared BackendContext helper — the same one SetActiveTakeAction uses, so
// the trim/duration conversions (and take-folder resolution) live in exactly one place.
if let Some((track_id, backend_id)) =
backend.add_clip_instance(document, &self.layer_id, &self.clip_instance)?
{
self.backend_track_id = Some(track_id);
match backend_id {
crate::action::BackendClipInstanceId::Midi(id) => {
self.backend_midi_instance_id = Some(id)
}
crate::action::BackendClipInstanceId::Audio(id) => {
self.backend_audio_instance_id = Some(id)
// Look up the clip from the document
let clip = document
.get_audio_clip(&self.clip_instance.clip_id)
.ok_or_else(|| "Audio clip not found".to_string())?;
// Look up backend track ID from layer mapping
let backend_track_id = backend
.layer_to_track_map
.get(&self.layer_id)
.ok_or_else(|| format!("Layer {} not mapped to backend track", self.layer_id))?;
// Get audio controller
let controller = backend
.audio_controller
.as_mut()
.ok_or_else(|| "Audio controller not available".to_string())?;
// Handle different clip types
use crate::clip::AudioClipType;
match &clip.clip_type {
AudioClipType::Midi { midi_clip_id } => {
// Create a MIDI clip instance referencing the existing clip in the backend pool
// No need to add to pool again - it was added during MIDI import
use daw_backend::command::{Query, QueryResponse};
// Calculate internal start/end from trim parameters
let internal_start = self.clip_instance.trim_start;
let internal_end = self.clip_instance.trim_end.unwrap_or(clip.duration);
let external_start = self.clip_instance.timeline_start;
// Calculate external duration (for looping if timeline_duration is set)
let external_duration = self.clip_instance.timeline_duration
.unwrap_or(internal_end - internal_start);
// Create MidiClipInstance
let instance = daw_backend::MidiClipInstance::new(
0, // Instance ID will be assigned by backend
*midi_clip_id,
daw_backend::Beats(internal_start),
daw_backend::Beats(internal_end),
daw_backend::Beats(external_start),
daw_backend::Beats(external_duration),
);
// Send query to add instance and get instance ID
let query = Query::AddMidiClipInstanceSync(*backend_track_id, instance);
match controller.send_query(query)? {
QueryResponse::MidiClipInstanceAdded(Ok(instance_id)) => {
self.backend_track_id = Some(*backend_track_id);
self.backend_midi_instance_id = Some(instance_id);
// Add to global clip instance mapping
backend.clip_instance_to_backend_map.insert(
self.clip_instance.id,
crate::action::BackendClipInstanceId::Midi(instance_id)
);
Ok(())
}
QueryResponse::MidiClipInstanceAdded(Err(e)) => Err(e),
_ => Err("Unexpected query response".to_string()),
}
}
}
AudioClipType::Sampled { audio_pool_index } => {
// `trim_*` / `clip.duration` are in SECONDS (audio content time),
// while `timeline_*` and the backend's `duration` are in BEATS.
let internal_start = self.clip_instance.trim_start;
let internal_end = self.clip_instance.trim_end.unwrap_or(clip.duration);
let start_time = self.clip_instance.timeline_start;
// `effective_duration` is in BEATS. When `timeline_duration` is set
// it already is; otherwise the clip occupies its natural content
// length, so convert that seconds-span to beats at the clip's start
// (NOT `internal_end - internal_start`, which is seconds — that was
// the seconds-as-beats bug that made clips stop early off 60 BPM).
let effective_duration = self.clip_instance.timeline_duration.unwrap_or_else(|| {
let tempo_map = document.tempo_map();
let content_secs = internal_end - internal_start;
tempo_map.inverse_transform(tempo_map.transform(start_time) + content_secs)
- start_time
});
Ok(())
let instance_id = controller.add_audio_clip(
*backend_track_id,
*audio_pool_index,
start_time,
effective_duration,
internal_start,
);
self.backend_track_id = Some(*backend_track_id);
self.backend_audio_instance_id = Some(instance_id);
// Add to global clip instance mapping
backend.clip_instance_to_backend_map.insert(
self.clip_instance.id,
crate::action::BackendClipInstanceId::Audio(instance_id)
);
Ok(())
}
AudioClipType::Recording => {
// Recording clips are not synced to backend until finalized
Ok(())
}
}
}
fn rollback_backend(&mut self, backend: &mut BackendContext, _document: &Document) -> Result<(), String> {

View File

@ -159,7 +159,7 @@ mod tests {
fn test_add_effect() {
let (mut document, layer_id, def) = create_test_setup();
let instance = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance = def.create_instance(0.0, 10.0);
let instance_id = instance.id;
let mut action = AddEffectAction::new(layer_id, instance);
@ -181,7 +181,7 @@ mod tests {
fn test_add_effect_rollback() {
let (mut document, layer_id, def) = create_test_setup();
let instance = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance = def.create_instance(0.0, 10.0);
let mut action = AddEffectAction::new(layer_id, instance);
action.execute(&mut document).unwrap();
@ -201,19 +201,19 @@ mod tests {
let (mut document, layer_id, def) = create_test_setup();
// Add first effect
let instance1 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance1 = def.create_instance(0.0, 10.0);
let id1 = instance1.id;
let mut action1 = AddEffectAction::new(layer_id, instance1);
action1.execute(&mut document).unwrap();
// Add second effect
let instance2 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance2 = def.create_instance(0.0, 10.0);
let id2 = instance2.id;
let mut action2 = AddEffectAction::new(layer_id, instance2);
action2.execute(&mut document).unwrap();
// Insert third effect at index 1 (between first and second)
let instance3 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance3 = def.create_instance(0.0, 10.0);
let id3 = instance3.id;
let mut action3 = AddEffectAction::at_index(layer_id, instance3, 1);
action3.execute(&mut document).unwrap();

View File

@ -1,141 +0,0 @@
//! Append freshly-recorded takes to an existing take folder.
//!
//! Cycle-recording over a region that already holds a take folder should *add* to that folder, not
//! drop a second clip on top of it. Otherwise the takes from your second attempt are stranded in a
//! separate, overlapping clip and you can't audition them against the first.
//!
//! The recorded content already exists in the backend pools by the time this runs (the engine put it
//! there at stop), so this action only touches the document — plus the one backend clip the instance
//! plays, which has to be repointed at the newly-active take.
use crate::action::{Action, BackendClipInstanceId, BackendContext};
use crate::clip::AudioTake;
use crate::document::Document;
use crate::layer::AnyLayer;
use uuid::Uuid;
/// Action that appends takes to an instance's take list and selects the last of them.
pub struct AppendTakesAction {
layer_id: Uuid,
/// The instance whose take list is being extended (and whose active take changes).
instance_id: Uuid,
/// The takes to add, in recording order.
new_takes: Vec<AudioTake>,
// Stored during execute for rollback.
old_take_count: usize,
old_active_take: Option<usize>,
executed: bool,
}
impl AppendTakesAction {
pub fn new(layer_id: Uuid, instance_id: Uuid, new_takes: Vec<AudioTake>) -> Self {
Self {
layer_id,
instance_id,
new_takes,
old_take_count: 0,
old_active_take: None,
executed: false,
}
}
/// Swap the instance's backend clip to whatever take the document now says is active.
///
/// Same remove + re-add as `SetActiveTakeAction` — there's no in-place pool-swap command.
fn resync(&self, backend: &mut BackendContext, document: &Document) -> Result<(), String> {
let instance = document
.get_layer(&self.layer_id)
.and_then(|l| match l {
AnyLayer::Audio(al) => al.clip_instances.iter().find(|ci| ci.id == self.instance_id),
_ => None,
})
.cloned()
.ok_or_else(|| format!("Clip instance {} not found", self.instance_id))?;
let existing: Option<BackendClipInstanceId> = backend
.clip_instance_to_backend_map
.get(&self.instance_id)
.copied();
let track_id = backend.layer_to_track_map.get(&self.layer_id).copied();
if let (Some(backend_id), Some(track_id)) = (existing, track_id) {
backend.remove_clip_instance(track_id, backend_id, self.instance_id);
}
backend.add_clip_instance(document, &self.layer_id, &instance)?;
Ok(())
}
}
impl Action for AppendTakesAction {
fn execute(&mut self, document: &mut Document) -> Result<(), String> {
let layer = document
.get_layer_mut(&self.layer_id)
.ok_or_else(|| format!("Layer {} not found", self.layer_id))?;
let AnyLayer::Audio(audio_layer) = layer else {
return Err("Takes only exist on audio layers".to_string());
};
let instance = audio_layer
.clip_instances
.iter_mut()
.find(|ci| ci.id == self.instance_id)
.ok_or_else(|| format!("Clip instance {} not found", self.instance_id))?;
// Only record the pre-state on the first execute; a redo must not overwrite it with the
// post-state left behind by the previous run.
if !self.executed {
self.old_take_count = instance.takes.len();
self.old_active_take = instance.active_take;
}
// Number the new takes on from what's already there. Existing names are left alone — the
// user may well have renamed them, and renumbering would clobber that.
let base = instance.takes.len();
for (i, take) in self.new_takes.iter().enumerate() {
let mut take = take.clone();
if take.name.is_empty() {
take.name = format!("Take {}", base + i + 1);
}
instance.takes.push(take);
}
// Land on the take just recorded, GarageBand-style.
instance.active_take = Some(instance.takes.len() - 1);
self.executed = true;
Ok(())
}
fn rollback(&mut self, document: &mut Document) -> Result<(), String> {
if let Some(AnyLayer::Audio(audio_layer)) = document.get_layer_mut(&self.layer_id) {
if let Some(instance) = audio_layer
.clip_instances
.iter_mut()
.find(|ci| ci.id == self.instance_id)
{
instance.takes.truncate(self.old_take_count);
instance.active_take = self.old_active_take;
}
}
Ok(())
}
fn description(&self) -> String {
format!("Record {} take(s)", self.new_takes.len())
}
fn execute_backend(
&mut self,
backend: &mut BackendContext,
document: &Document,
) -> Result<(), String> {
self.resync(backend, document)
}
fn rollback_backend(
&mut self,
backend: &mut BackendContext,
document: &Document,
) -> Result<(), String> {
self.resync(backend, document)
}
}

View File

@ -9,9 +9,8 @@ use crate::layer::AnyLayer;
use std::collections::HashMap;
use uuid::Uuid;
/// Per-instance loop change: (instance_id, old_timeline_duration, new_timeline_duration, old_loop_before, new_loop_before).
/// All durations/offsets are in beats.
pub type LoopEntry = (Uuid, Option<daw_backend::Beats>, Option<daw_backend::Beats>, Option<daw_backend::Beats>, Option<daw_backend::Beats>);
/// Per-instance loop change: (instance_id, old_timeline_duration, new_timeline_duration, old_loop_before, new_loop_before)
pub type LoopEntry = (Uuid, Option<f64>, Option<f64>, Option<f64>, Option<f64>);
/// Action that changes the loop duration of clip instances
pub struct LoopClipInstancesAction {
@ -91,7 +90,7 @@ impl Action for LoopClipInstancesAction {
impl LoopClipInstancesAction {
fn sync_backend(&self, backend: &mut crate::action::BackendContext, document: &Document, rollback: bool) -> Result<(), String> {
use crate::clip::ResolvedContent;
use crate::clip::AudioClipType;
let controller = match backend.audio_controller.as_mut() {
Some(c) => c,
@ -128,22 +127,19 @@ impl LoopClipInstancesAction {
(new_dur, new_lb)
};
// Natural content length as a beats span (the fallback when no explicit
// timeline_duration is set). Resolved in the clip's own domain, so MIDI's beats
// content carries over directly rather than being read as seconds.
let content_window_beats = instance.effective_duration_beats(
clip.content_duration(),
document.tempo_map(),
);
let right_duration = target_duration.unwrap_or(content_window_beats);
let left_duration = target_loop_before.unwrap_or(daw_backend::Beats::ZERO);
let content_window = {
let trim_end = instance.trim_end.unwrap_or(clip.duration);
(trim_end - instance.trim_start).max(0.0)
};
let right_duration = target_duration.unwrap_or(content_window);
let left_duration = target_loop_before.unwrap_or(0.0);
let external_duration = left_duration + right_duration;
let external_start = instance.timeline_start - left_duration;
let get_backend_clip_id = |inst_id: &Uuid| -> Result<u32, String> {
match &instance.resolve(clip) {
ResolvedContent::Midi { midi_clip_id } => Ok(*midi_clip_id),
ResolvedContent::Audio { .. } => {
match &clip.clip_type {
AudioClipType::Midi { midi_clip_id } => Ok(*midi_clip_id),
AudioClipType::Sampled { .. } => {
let backend_id = backend.clip_instance_to_backend_map.get(inst_id)
.ok_or_else(|| format!("Clip instance {} not mapped to backend", inst_id))?;
match backend_id {
@ -151,7 +147,7 @@ impl LoopClipInstancesAction {
_ => Err("Expected audio instance ID for sampled clip".to_string()),
}
}
ResolvedContent::Recording => Err("Cannot sync recording clip".to_string()),
AudioClipType::Recording => Err("Cannot sync recording clip".to_string()),
}
};

View File

@ -1,419 +0,0 @@
//! Take management: delete and rename the takes on a clip instance.
//!
//! Takes live on the INSTANCE, so both of these are naturally scoped to the one the user clicked —
//! deleting a take from one half of a comped split leaves the other half's list alone.
use crate::action::{Action, BackendClipInstanceId, BackendContext};
use crate::clip::{AudioTake, ClipInstance};
use crate::document::Document;
use crate::layer::AnyLayer;
use uuid::Uuid;
/// The instance a take action targets, looked up mutably.
fn instance_mut<'a>(
document: &'a mut Document,
layer_id: &Uuid,
instance_id: &Uuid,
) -> Result<&'a mut ClipInstance, String> {
let layer = document
.get_layer_mut(layer_id)
.ok_or_else(|| format!("Layer {} not found", layer_id))?;
let AnyLayer::Audio(audio_layer) = layer else {
return Err("Takes only exist on audio layers".to_string());
};
audio_layer
.clip_instances
.iter_mut()
.find(|ci| ci.id == *instance_id)
.ok_or_else(|| format!("Clip instance {} not found", instance_id))
}
/// Swap an instance's backend clip to whatever take the document now says is active.
///
/// The same remove + re-add as `SetActiveTakeAction` — there's no in-place pool-swap command.
fn resync(
backend: &mut BackendContext,
document: &Document,
layer_id: &Uuid,
instance_id: &Uuid,
) -> Result<(), String> {
let instance = document
.get_layer(layer_id)
.and_then(|l| match l {
AnyLayer::Audio(al) => al.clip_instances.iter().find(|ci| ci.id == *instance_id),
_ => None,
})
.cloned()
.ok_or_else(|| format!("Clip instance {} not found", instance_id))?;
let existing: Option<BackendClipInstanceId> = backend
.clip_instance_to_backend_map
.get(instance_id)
.copied();
let track_id = backend.layer_to_track_map.get(layer_id).copied();
if let (Some(backend_id), Some(track_id)) = (existing, track_id) {
backend.remove_clip_instance(track_id, backend_id, *instance_id);
}
backend.add_clip_instance(document, layer_id, &instance)?;
Ok(())
}
/// Remove a take from an instance's take list.
///
/// The take's recorded audio/MIDI stays in the backend pool — undo has to be able to put it back,
/// and other instances (the other half of a split, say) may still be playing it.
pub struct DeleteTakeAction {
layer_id: Uuid,
instance_id: Uuid,
take_index: usize,
// Stored during execute for rollback.
removed: Option<AudioTake>,
old_active_take: Option<usize>,
}
impl DeleteTakeAction {
pub fn new(layer_id: Uuid, instance_id: Uuid, take_index: usize) -> Self {
Self {
layer_id,
instance_id,
take_index,
removed: None,
old_active_take: None,
}
}
}
impl Action for DeleteTakeAction {
fn execute(&mut self, document: &mut Document) -> Result<(), String> {
let instance = instance_mut(document, &self.layer_id, &self.instance_id)?;
if self.take_index >= instance.takes.len() {
return Err(format!("Take {} does not exist", self.take_index + 1));
}
// An instance with no takes at all would fall back to the clip's own content, which for a
// cycle recording is a take we may just have deleted. Refuse rather than strand it.
if instance.takes.len() == 1 {
return Err("Can't delete the only take".to_string());
}
self.old_active_take = instance.active_take;
self.removed = Some(instance.takes.remove(self.take_index));
// Everything above the removed take shifts down one, so the selection has to move with it.
// Deleting the *active* take lands on the one that took its place (or the new last take, if
// it was at the end) — that keeps the clip sounding rather than silently picking take 1.
let active = instance.active_take.unwrap_or(0);
instance.active_take = Some(if active > self.take_index {
active - 1
} else if active == self.take_index {
self.take_index.min(instance.takes.len() - 1)
} else {
active
});
Ok(())
}
fn rollback(&mut self, document: &mut Document) -> Result<(), String> {
let Some(take) = self.removed.take() else {
return Ok(());
};
let instance = instance_mut(document, &self.layer_id, &self.instance_id)?;
let at = self.take_index.min(instance.takes.len());
instance.takes.insert(at, take);
instance.active_take = self.old_active_take;
Ok(())
}
fn description(&self) -> String {
format!("Delete take {}", self.take_index + 1)
}
fn execute_backend(
&mut self,
backend: &mut BackendContext,
document: &Document,
) -> Result<(), String> {
resync(backend, document, &self.layer_id, &self.instance_id)
}
fn rollback_backend(
&mut self,
backend: &mut BackendContext,
document: &Document,
) -> Result<(), String> {
resync(backend, document, &self.layer_id, &self.instance_id)
}
}
/// Throw away every take except the one that's playing.
///
/// The tidy-up once you've picked your keeper. Scoped to this instance, so on a comped split it
/// prunes the half you clicked and leaves the other half's alternatives intact.
pub struct DeleteUnusedTakesAction {
layer_id: Uuid,
instance_id: Uuid,
// Stored during execute for rollback.
old_takes: Vec<AudioTake>,
old_active_take: Option<usize>,
}
impl DeleteUnusedTakesAction {
pub fn new(layer_id: Uuid, instance_id: Uuid) -> Self {
Self {
layer_id,
instance_id,
old_takes: Vec::new(),
old_active_take: None,
}
}
}
impl Action for DeleteUnusedTakesAction {
fn execute(&mut self, document: &mut Document) -> Result<(), String> {
let instance = instance_mut(document, &self.layer_id, &self.instance_id)?;
if instance.takes.len() < 2 {
return Err("Nothing to delete".to_string());
}
let keep = instance.active_take_index();
self.old_takes = instance.takes.clone();
self.old_active_take = instance.active_take;
let kept = instance.takes.remove(keep);
instance.takes.clear();
instance.takes.push(kept);
instance.active_take = Some(0);
Ok(())
}
fn rollback(&mut self, document: &mut Document) -> Result<(), String> {
let instance = instance_mut(document, &self.layer_id, &self.instance_id)?;
instance.takes = std::mem::take(&mut self.old_takes);
instance.active_take = self.old_active_take;
Ok(())
}
fn description(&self) -> String {
"Delete unused takes".to_string()
}
// The take that plays doesn't change, so the backend clip is already correct.
}
/// Rename a take. Document-only — which take *plays* doesn't change, so the backend is untouched.
pub struct RenameTakeAction {
layer_id: Uuid,
instance_id: Uuid,
take_index: usize,
new_name: String,
old_name: String,
}
impl RenameTakeAction {
pub fn new(layer_id: Uuid, instance_id: Uuid, take_index: usize, new_name: String) -> Self {
Self {
layer_id,
instance_id,
take_index,
new_name,
old_name: String::new(),
}
}
}
impl Action for RenameTakeAction {
fn execute(&mut self, document: &mut Document) -> Result<(), String> {
let instance = instance_mut(document, &self.layer_id, &self.instance_id)?;
let take = instance
.takes
.get_mut(self.take_index)
.ok_or_else(|| format!("Take {} does not exist", self.take_index + 1))?;
self.old_name = std::mem::replace(&mut take.name, self.new_name.clone());
Ok(())
}
fn rollback(&mut self, document: &mut Document) -> Result<(), String> {
let instance = instance_mut(document, &self.layer_id, &self.instance_id)?;
if let Some(take) = instance.takes.get_mut(self.take_index) {
take.name = self.old_name.clone();
}
Ok(())
}
fn description(&self) -> String {
format!("Rename take to \"{}\"", self.new_name)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::clip::TakeContent;
use crate::layer::AudioLayer;
/// A document with one audio layer holding one instance with 4 takes (pools 10..13).
fn doc_with_takes() -> (Document, Uuid, Uuid) {
let mut document = Document::new("Test");
let clip = crate::clip::AudioClip::new_sampled("Cycle rec", 10, 2.0);
let clip_id = document.add_audio_clip(clip);
let mut instance = ClipInstance::new(clip_id);
instance.takes = (10..14)
.map(|pool| AudioTake {
name: format!("Take {}", pool - 9),
content: TakeContent::Audio { audio_pool_index: pool },
})
.collect();
let instance_id = instance.id;
let mut layer = AudioLayer::new("Layer");
layer.clip_instances.push(instance);
let layer_id = document.root.add_child(AnyLayer::Audio(layer));
(document, layer_id, instance_id)
}
fn takes_of(document: &Document, layer_id: &Uuid, instance_id: &Uuid) -> ClipInstance {
let AnyLayer::Audio(al) = document.get_layer(layer_id).unwrap() else { panic!() };
al.clip_instances.iter().find(|ci| ci.id == *instance_id).unwrap().clone()
}
#[test]
fn deleting_a_take_below_the_active_one_shifts_the_selection_down() {
// Everything above the removed take shifts down one, so a selection above it has to move
// with it — otherwise the instance silently starts playing a different take.
let (mut document, layer_id, instance_id) = doc_with_takes();
{
let AnyLayer::Audio(al) = document.get_layer_mut(&layer_id).unwrap() else { panic!() };
al.clip_instances[0].active_take = Some(3); // playing pool 13
}
DeleteTakeAction::new(layer_id, instance_id, 1)
.execute(&mut document)
.expect("delete");
let inst = takes_of(&document, &layer_id, &instance_id);
assert_eq!(inst.takes.len(), 3);
assert_eq!(inst.active_take, Some(2), "index shifted down with the take");
assert_eq!(
inst.takes[inst.active_take_index()].content,
TakeContent::Audio { audio_pool_index: 13 },
"still playing the same take it was",
);
}
#[test]
fn deleting_the_active_take_lands_on_its_replacement() {
// Deleting what you're listening to should hand you the take that took its place, not
// silently jump you back to take 1.
let (mut document, layer_id, instance_id) = doc_with_takes();
{
let AnyLayer::Audio(al) = document.get_layer_mut(&layer_id).unwrap() else { panic!() };
al.clip_instances[0].active_take = Some(1); // playing pool 11
}
DeleteTakeAction::new(layer_id, instance_id, 1)
.execute(&mut document)
.expect("delete");
let inst = takes_of(&document, &layer_id, &instance_id);
assert_eq!(inst.active_take, Some(1));
assert_eq!(
inst.takes[1].content,
TakeContent::Audio { audio_pool_index: 12 },
"the take that slid into the deleted one's place",
);
}
#[test]
fn deleting_the_last_take_in_the_list_steps_back() {
let (mut document, layer_id, instance_id) = doc_with_takes();
{
let AnyLayer::Audio(al) = document.get_layer_mut(&layer_id).unwrap() else { panic!() };
al.clip_instances[0].active_take = Some(3);
}
DeleteTakeAction::new(layer_id, instance_id, 3)
.execute(&mut document)
.expect("delete");
let inst = takes_of(&document, &layer_id, &instance_id);
assert_eq!(inst.active_take, Some(2), "there is no take 4 to land on");
}
#[test]
fn the_only_take_cannot_be_deleted() {
let (mut document, layer_id, instance_id) = doc_with_takes();
{
let AnyLayer::Audio(al) = document.get_layer_mut(&layer_id).unwrap() else { panic!() };
al.clip_instances[0].takes.truncate(1);
}
assert!(DeleteTakeAction::new(layer_id, instance_id, 0)
.execute(&mut document)
.is_err());
}
#[test]
fn undoing_a_delete_puts_the_take_back_where_it_was() {
let (mut document, layer_id, instance_id) = doc_with_takes();
{
let AnyLayer::Audio(al) = document.get_layer_mut(&layer_id).unwrap() else { panic!() };
al.clip_instances[0].active_take = Some(2);
}
let mut action = DeleteTakeAction::new(layer_id, instance_id, 1);
action.execute(&mut document).expect("delete");
action.rollback(&mut document).expect("undo");
let inst = takes_of(&document, &layer_id, &instance_id);
assert_eq!(inst.takes.len(), 4);
assert_eq!(
inst.takes[1].content,
TakeContent::Audio { audio_pool_index: 11 },
"restored at its original index",
);
assert_eq!(inst.active_take, Some(2), "and the selection with it");
}
#[test]
fn deleting_unused_takes_keeps_the_one_thats_playing() {
let (mut document, layer_id, instance_id) = doc_with_takes();
{
let AnyLayer::Audio(al) = document.get_layer_mut(&layer_id).unwrap() else { panic!() };
al.clip_instances[0].active_take = Some(2); // pool 12 — the keeper
}
let mut action = DeleteUnusedTakesAction::new(layer_id, instance_id);
action.execute(&mut document).expect("prune");
let inst = takes_of(&document, &layer_id, &instance_id);
assert_eq!(inst.takes.len(), 1);
assert_eq!(inst.active_take, Some(0));
assert_eq!(
inst.takes[0].content,
TakeContent::Audio { audio_pool_index: 12 },
"the take that was playing survives, and nothing else",
);
action.rollback(&mut document).expect("undo");
let inst = takes_of(&document, &layer_id, &instance_id);
assert_eq!(inst.takes.len(), 4);
assert_eq!(inst.active_take, Some(2), "back to what was playing before");
}
#[test]
fn renaming_a_take_round_trips() {
let (mut document, layer_id, instance_id) = doc_with_takes();
let mut action =
RenameTakeAction::new(layer_id, instance_id, 2, "The good one".to_string());
action.execute(&mut document).expect("rename");
assert_eq!(takes_of(&document, &layer_id, &instance_id).takes[2].name, "The good one");
action.rollback(&mut document).expect("undo");
assert_eq!(takes_of(&document, &layer_id, &instance_id).takes[2].name, "Take 3");
}
}

View File

@ -14,10 +14,6 @@ pub mod move_clip_instances;
pub mod reorder_clip_instances;
pub mod paint_bucket;
pub mod remove_effect;
pub mod set_cycle_region;
pub mod append_takes;
pub mod manage_takes;
pub mod set_active_take;
pub mod set_document_properties;
pub mod set_instance_properties;
pub mod set_layer_properties;
@ -54,10 +50,6 @@ pub mod set_text_content;
pub mod resize_text_box;
pub use add_clip_instance::AddClipInstanceAction;
pub use set_cycle_region::SetCycleRegionAction;
pub use append_takes::AppendTakesAction;
pub use manage_takes::{DeleteTakeAction, DeleteUnusedTakesAction, RenameTakeAction};
pub use set_active_take::SetActiveTakeAction;
pub use add_effect::AddEffectAction;
pub use add_layer::AddLayerAction;
pub use add_shape::AddShapeAction;

View File

@ -6,15 +6,13 @@ use crate::action::Action;
use crate::clip::ClipInstance;
use crate::document::Document;
use crate::layer::AnyLayer;
use daw_backend::Beats;
use std::collections::HashMap;
use uuid::Uuid;
/// Action that moves clip instances to new timeline positions
pub struct MoveClipInstancesAction {
/// Map of layer IDs to vectors of (clip_instance_id, old_timeline_start, new_timeline_start).
/// Timeline positions are in beats.
layer_moves: HashMap<Uuid, Vec<(Uuid, Beats, Beats)>>,
/// Map of layer IDs to vectors of (clip_instance_id, old_timeline_start, new_timeline_start)
layer_moves: HashMap<Uuid, Vec<(Uuid, f64, f64)>>,
}
impl MoveClipInstancesAction {
@ -22,8 +20,8 @@ impl MoveClipInstancesAction {
///
/// # Arguments
///
/// * `layer_moves` - Map of layer IDs to vectors of (clip_instance_id, old_timeline_start, new_timeline_start) in beats
pub fn new(layer_moves: HashMap<Uuid, Vec<(Uuid, Beats, Beats)>>) -> Self {
/// * `layer_moves` - Map of layer IDs to vectors of (clip_instance_id, old_timeline_start, new_timeline_start)
pub fn new(layer_moves: HashMap<Uuid, Vec<(Uuid, f64, f64)>>) -> Self {
Self { layer_moves }
}
}
@ -44,7 +42,7 @@ impl Action for MoveClipInstancesAction {
// Check if this instance is in a group
if let Some(group) = document.find_group_for_instance(instance_id) {
let offset = *new_start - *old_start;
let offset = new_start - old_start;
// Add all group members to the move list
for (member_layer_id, member_instance_id) in group.get_members() {
@ -79,7 +77,7 @@ impl Action for MoveClipInstancesAction {
}
// Auto-adjust moves to avoid overlaps
let mut adjusted_moves: HashMap<Uuid, Vec<(Uuid, Beats, Beats)>> = HashMap::new();
let mut adjusted_moves: HashMap<Uuid, Vec<(Uuid, f64, f64)>> = HashMap::new();
for (layer_id, moves) in &expanded_moves {
let layer = document.get_layer(layer_id)
@ -103,10 +101,10 @@ impl Action for MoveClipInstancesAction {
AnyLayer::Text(_) => &[],
};
let group: Vec<(Uuid, Beats, Beats)> = moves.iter().filter_map(|(id, old_start, _)| {
let group: Vec<(Uuid, f64, f64)> = moves.iter().filter_map(|(id, old_start, _)| {
let inst = clip_instances.iter().find(|ci| &ci.id == id)?;
let dur = document.clip_trim_duration(&inst.clip_id)?;
let eff = inst.effective_duration_beats(dur, document.tempo_map());
let dur = document.get_clip_duration(&inst.clip_id)?;
let eff = inst.trim_end.unwrap_or(dur) - inst.trim_start;
Some((*id, *old_start, eff))
}).collect();
@ -114,7 +112,7 @@ impl Action for MoveClipInstancesAction {
let clamped = document.clamp_group_move_offset(layer_id, &group, desired_offset);
for (instance_id, old_start, _) in moves {
adjusted_layer_moves.push((*instance_id, *old_start, (*old_start + clamped).max(Beats::ZERO)));
adjusted_layer_moves.push((*instance_id, *old_start, (*old_start + clamped).max(0.0)));
}
adjusted_moves.insert(*layer_id, adjusted_layer_moves);
@ -190,7 +188,7 @@ impl Action for MoveClipInstancesAction {
fn execute_backend(&mut self, backend: &mut crate::action::BackendContext, document: &Document) -> Result<(), String> {
use crate::layer::AnyLayer;
use crate::clip::ResolvedContent;
use crate::clip::AudioClipType;
// Get audio controller
let controller = match backend.audio_controller.as_mut() {
@ -210,10 +208,9 @@ impl Action for MoveClipInstancesAction {
if let Some(instance) = vl.clip_instances.iter().find(|ci| ci.id == *instance_id) {
// Check if this clip has a metatrack
if let Some(&metatrack_id) = backend.layer_to_track_map.get(&instance.clip_id) {
controller.set_offset(metatrack_id, document.tempo_map().beats_to_seconds(*new_start));
// A vector clip's content is wall-clock, so its content times ARE seconds.
controller.set_trim_start(metatrack_id, daw_backend::Seconds(instance.trim_start.raw()));
controller.set_trim_end(metatrack_id, instance.trim_end.map(|t| daw_backend::Seconds(t.raw())));
controller.set_offset(metatrack_id, *new_start);
controller.set_trim_start(metatrack_id, instance.trim_start);
controller.set_trim_end(metatrack_id, instance.trim_end);
}
}
}
@ -247,12 +244,12 @@ impl Action for MoveClipInstancesAction {
.ok_or_else(|| format!("Audio clip {} not found", instance.clip_id))?;
// Handle move based on clip type
match &instance.resolve(clip) {
ResolvedContent::Midi { midi_clip_id } => {
match &clip.clip_type {
AudioClipType::Midi { midi_clip_id } => {
// For MIDI: move_clip expects the pool clip ID
controller.move_clip(*track_id, *midi_clip_id, *new_start);
}
ResolvedContent::Audio { .. } => {
AudioClipType::Sampled { .. } => {
// For sampled audio: move_clip expects the instance ID
let backend_instance_id = backend.clip_instance_to_backend_map.get(instance_id)
.ok_or_else(|| format!("Clip instance {} not mapped to backend", instance_id))?;
@ -264,7 +261,7 @@ impl Action for MoveClipInstancesAction {
_ => return Err("Expected audio instance ID for sampled clip".to_string()),
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
// Recording clips cannot be moved - skip
}
}
@ -276,7 +273,7 @@ impl Action for MoveClipInstancesAction {
fn rollback_backend(&mut self, backend: &mut crate::action::BackendContext, document: &Document) -> Result<(), String> {
use crate::layer::AnyLayer;
use crate::clip::ResolvedContent;
use crate::clip::AudioClipType;
// Get audio controller
let controller = match backend.audio_controller.as_mut() {
@ -295,10 +292,9 @@ impl Action for MoveClipInstancesAction {
for (instance_id, old_start, _new_start) in moves {
if let Some(instance) = vl.clip_instances.iter().find(|ci| ci.id == *instance_id) {
if let Some(&metatrack_id) = backend.layer_to_track_map.get(&instance.clip_id) {
controller.set_offset(metatrack_id, document.tempo_map().beats_to_seconds(*old_start));
// A vector clip's content is wall-clock, so its content times ARE seconds.
controller.set_trim_start(metatrack_id, daw_backend::Seconds(instance.trim_start.raw()));
controller.set_trim_end(metatrack_id, instance.trim_end.map(|t| daw_backend::Seconds(t.raw())));
controller.set_offset(metatrack_id, *old_start);
controller.set_trim_start(metatrack_id, instance.trim_start);
controller.set_trim_end(metatrack_id, instance.trim_end);
}
}
}
@ -332,12 +328,12 @@ impl Action for MoveClipInstancesAction {
.ok_or_else(|| format!("Audio clip {} not found", instance.clip_id))?;
// Handle move based on clip type (restore old position)
match &instance.resolve(clip) {
ResolvedContent::Midi { midi_clip_id } => {
match &clip.clip_type {
AudioClipType::Midi { midi_clip_id } => {
// For MIDI: move_clip expects the pool clip ID
controller.move_clip(*track_id, *midi_clip_id, *old_start);
}
ResolvedContent::Audio { .. } => {
AudioClipType::Sampled { .. } => {
// For sampled audio: move_clip expects the instance ID
let backend_instance_id = backend.clip_instance_to_backend_map.get(instance_id)
.ok_or_else(|| format!("Clip instance {} not mapped to backend", instance_id))?;
@ -349,7 +345,7 @@ impl Action for MoveClipInstancesAction {
_ => return Err("Expected audio instance ID for sampled clip".to_string()),
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
// Recording clips cannot be moved - skip
}
}
@ -377,15 +373,15 @@ mod tests {
let mut vector_layer = VectorLayer::new("Layer 1");
let mut clip_instance = ClipInstance::new(clip_id);
clip_instance.timeline_start = Beats(1.0); // Start at beat 1
clip_instance.timeline_start = 1.0; // Start at 1 second
let instance_id = clip_instance.id;
vector_layer.clip_instances.push(clip_instance);
let layer_id = document.root.add_child(AnyLayer::Vector(vector_layer));
// Create move action: move from beat 1 to beat 5
// Create move action: move from 1.0 to 5.0 seconds
let mut layer_moves = HashMap::new();
layer_moves.insert(layer_id, vec![(instance_id, Beats(1.0), Beats(5.0))]);
layer_moves.insert(layer_id, vec![(instance_id, 1.0, 5.0)]);
let mut action = MoveClipInstancesAction::new(layer_moves);
@ -399,7 +395,7 @@ mod tests {
.iter()
.find(|ci| ci.id == instance_id)
.unwrap();
assert_eq!(instance.timeline_start, Beats(5.0));
assert_eq!(instance.timeline_start, 5.0);
}
// Rollback
@ -412,7 +408,7 @@ mod tests {
.iter()
.find(|ci| ci.id == instance_id)
.unwrap();
assert_eq!(instance.timeline_start, Beats(1.0));
assert_eq!(instance.timeline_start, 1.0);
}
}
}

View File

@ -138,23 +138,85 @@ impl Action for RemoveClipInstancesAction {
backend: &mut BackendContext,
document: &Document,
) -> Result<(), String> {
if backend.audio_controller.is_none() {
return Ok(());
}
use crate::clip::AudioClipType;
// Re-add the clips that were removed. `BackendContext::add_clip_instance` is the same
// helper the add and split actions use, so the trim/duration conversions (and take-folder
// resolution) stay in exactly one place instead of being copied into every action that has
// to put a clip back.
let saved = std::mem::take(&mut self.saved);
for (layer_id, instance) in &saved {
if !matches!(document.get_layer(layer_id), Some(AnyLayer::Audio(_))) {
let controller = match backend.audio_controller.as_mut() {
Some(c) => c,
None => return Ok(()),
};
// Re-add clips that were removed from backend
for (layer_id, instance) in &self.saved {
let layer = match document.get_layer(layer_id) {
Some(l) => l,
None => continue,
};
if !matches!(layer, AnyLayer::Audio(_)) {
continue;
}
// A missing track/clip just means there's nothing to restore on the backend.
let _ = backend.add_clip_instance(document, layer_id, instance);
let track_id = match backend.layer_to_track_map.get(layer_id) {
Some(id) => *id,
None => continue,
};
let clip = match document.get_audio_clip(&instance.clip_id) {
Some(c) => c,
None => continue,
};
match &clip.clip_type {
AudioClipType::Midi { midi_clip_id } => {
use daw_backend::command::{Query, QueryResponse};
let internal_start = instance.trim_start;
let internal_end = instance.trim_end.unwrap_or(clip.duration);
let external_start = instance.timeline_start;
let external_duration = instance
.timeline_duration
.unwrap_or(internal_end - internal_start);
let midi_instance = daw_backend::MidiClipInstance::new(
0,
*midi_clip_id,
daw_backend::Beats(internal_start),
daw_backend::Beats(internal_end),
daw_backend::Beats(external_start),
daw_backend::Beats(external_duration),
);
let query = Query::AddMidiClipInstanceSync(track_id, midi_instance);
if let Ok(QueryResponse::MidiClipInstanceAdded(Ok(new_id))) =
controller.send_query(query)
{
backend.clip_instance_to_backend_map.insert(
instance.id,
BackendClipInstanceId::Midi(new_id),
);
}
}
AudioClipType::Sampled { audio_pool_index } => {
let internal_start = instance.trim_start;
let internal_end = instance.trim_end.unwrap_or(clip.duration);
let effective_duration = instance.timeline_duration
.unwrap_or(internal_end - internal_start);
let start_time = instance.timeline_start;
let new_id = controller.add_audio_clip(
track_id,
*audio_pool_index,
start_time,
effective_duration,
internal_start,
);
backend.clip_instance_to_backend_map.insert(
instance.id,
BackendClipInstanceId::Audio(new_id),
);
}
AudioClipType::Recording => {}
}
}
self.saved = saved;
// Clear saved backend IDs
self.saved_backend_ids.clear();
@ -176,11 +238,11 @@ mod tests {
let mut vector_layer = VectorLayer::new("Layer 1");
let mut ci1 = ClipInstance::new(clip_id);
ci1.timeline_start = daw_backend::Beats::ZERO;
ci1.timeline_start = 0.0;
let id1 = ci1.id;
let mut ci2 = ClipInstance::new(clip_id);
ci2.timeline_start = daw_backend::Beats(5.0);
ci2.timeline_start = 5.0;
let id2 = ci2.id;
vector_layer.clip_instances.push(ci1);

View File

@ -134,7 +134,7 @@ mod tests {
let (mut document, layer_id, def) = create_test_setup();
// Add an effect first
let instance = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance = def.create_instance(0.0, 10.0);
let instance_id = instance.id;
if let Some(AnyLayer::Effect(el)) = document.get_layer_mut(&layer_id) {
@ -161,7 +161,7 @@ mod tests {
let (mut document, layer_id, def) = create_test_setup();
// Add an effect first
let instance = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance = def.create_instance(0.0, 10.0);
let instance_id = instance.id;
if let Some(AnyLayer::Effect(el)) = document.get_layer_mut(&layer_id) {
@ -185,11 +185,11 @@ mod tests {
let (mut document, layer_id, def) = create_test_setup();
// Add three effects
let instance1 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance1 = def.create_instance(0.0, 10.0);
let id1 = instance1.id;
let instance2 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance2 = def.create_instance(0.0, 10.0);
let id2 = instance2.id;
let instance3 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let instance3 = def.create_instance(0.0, 10.0);
let id3 = instance3.id;
if let Some(AnyLayer::Effect(el)) = document.get_layer_mut(&layer_id) {

View File

@ -1,117 +0,0 @@
//! Choose which take of a take-folder clip an instance plays.
//!
//! The take list lives on the *clip* but the selection lives on the *instance*, so two instances of
//! the same folder can play different takes. Splitting a take-folder instance clones it, which is
//! what makes comping work: set take 1 on the left half and take 3 on the right, and you've comped.
//!
//! There's no in-place pool-swap command in the backend, so switching a take means removing the
//! instance's backend clip and re-adding it against the new take's audio/MIDI resource. Both halves
//! of that go through `BackendContext`, which is also what `AddClipInstanceAction` uses.
use crate::action::{Action, BackendClipInstanceId, BackendContext};
use crate::document::Document;
use crate::layer::AnyLayer;
use uuid::Uuid;
/// Action that points a clip instance at a different take of its take folder.
#[derive(Clone)]
pub struct SetActiveTakeAction {
layer_id: Uuid,
instance_id: Uuid,
new_take: Option<usize>,
old_take: Option<usize>,
/// The backend track/clip the instance was on before we swapped, so rollback can undo it.
backend_track_id: Option<daw_backend::TrackId>,
}
impl SetActiveTakeAction {
pub fn new(layer_id: Uuid, instance_id: Uuid, new_take: usize, old_take: Option<usize>) -> Self {
Self {
layer_id,
instance_id,
new_take: Some(new_take),
old_take,
backend_track_id: None,
}
}
/// Point the instance at `take`, mutating the document. Shared by execute and rollback.
fn apply(&self, document: &mut Document, take: Option<usize>) -> Result<(), String> {
let layer = document
.get_layer_mut(&self.layer_id)
.ok_or_else(|| format!("Layer {} not found", self.layer_id))?;
let AnyLayer::Audio(audio_layer) = layer else {
return Err("Take folders only exist on audio layers".to_string());
};
let instance = audio_layer
.clip_instances
.iter_mut()
.find(|ci| ci.id == self.instance_id)
.ok_or_else(|| format!("Clip instance {} not found", self.instance_id))?;
instance.active_take = take;
Ok(())
}
/// Swap the instance's backend clip to whatever take the document now says is active.
///
/// Called after the document has already been mutated, so re-adding just re-resolves the
/// instance — `BackendContext::add_clip_instance` reads `active_take` itself.
fn resync(&mut self, backend: &mut BackendContext, document: &Document) -> Result<(), String> {
let instance = document
.get_layer(&self.layer_id)
.and_then(|l| match l {
AnyLayer::Audio(al) => al.clip_instances.iter().find(|ci| ci.id == self.instance_id),
_ => None,
})
.cloned()
.ok_or_else(|| format!("Clip instance {} not found", self.instance_id))?;
// Drop the old backend clip first. Its track comes from the map we're about to overwrite,
// so read it before add_clip_instance replaces the entry.
let existing: Option<BackendClipInstanceId> = backend
.clip_instance_to_backend_map
.get(&self.instance_id)
.copied();
let track_id = backend.layer_to_track_map.get(&self.layer_id).copied();
if let (Some(backend_id), Some(track_id)) = (existing, track_id) {
backend.remove_clip_instance(track_id, backend_id, self.instance_id);
}
let added = backend.add_clip_instance(document, &self.layer_id, &instance)?;
self.backend_track_id = added.map(|(track_id, _)| track_id);
Ok(())
}
}
impl Action for SetActiveTakeAction {
fn execute(&mut self, document: &mut Document) -> Result<(), String> {
self.apply(document, self.new_take)
}
fn rollback(&mut self, document: &mut Document) -> Result<(), String> {
self.apply(document, self.old_take)
}
fn description(&self) -> String {
match self.new_take {
Some(i) => format!("Select take {}", i + 1),
None => "Select take".to_string(),
}
}
fn execute_backend(
&mut self,
backend: &mut BackendContext,
document: &Document,
) -> Result<(), String> {
self.resync(backend, document)
}
fn rollback_backend(
&mut self,
backend: &mut BackendContext,
document: &Document,
) -> Result<(), String> {
self.resync(backend, document)
}
}

View File

@ -1,93 +0,0 @@
//! Set the transport cycle (loop) region.
//!
//! The cycle region is document state (it's saved in the `.beam`), so changing it goes through the
//! action system like any other edit: it's undoable and it marks the document modified.
//!
//! The region is stored in **beats** so it stays put musically across tempo changes. Callers commit
//! one action per gesture (e.g. on drag release, or a toggle click) rather than one per frame —
//! the timeline previews the drag from its own local state, exactly like a clip drag does.
use crate::action::{Action, BackendContext};
use crate::document::Document;
use daw_backend::Beats;
/// Action that sets the cycle region and/or whether the transport loops over it.
#[derive(Clone)]
pub struct SetCycleRegionAction {
old_region: Option<(Beats, Beats)>,
old_enabled: bool,
new_region: Option<(Beats, Beats)>,
new_enabled: bool,
}
impl SetCycleRegionAction {
/// Build from the document's current state and the desired new region/enabled flag.
pub fn new(
document: &Document,
new_region: Option<(Beats, Beats)>,
new_enabled: bool,
) -> Self {
Self {
old_region: document.cycle_region,
old_enabled: document.cycle_enabled,
new_region,
new_enabled,
}
}
/// Toggle looping on/off, leaving the region itself alone.
pub fn toggle_enabled(document: &Document) -> Self {
Self::new(document, document.cycle_region, !document.cycle_enabled)
}
/// True if this action would not actually change anything (lets callers skip a no-op undo entry).
pub fn is_noop(&self) -> bool {
self.old_region == self.new_region && self.old_enabled == self.new_enabled
}
}
impl Action for SetCycleRegionAction {
fn execute(&mut self, document: &mut Document) -> Result<(), String> {
document.cycle_region = self.new_region;
document.cycle_enabled = self.new_enabled;
Ok(())
}
fn rollback(&mut self, document: &mut Document) -> Result<(), String> {
document.cycle_region = self.old_region;
document.cycle_enabled = self.old_enabled;
Ok(())
}
fn description(&self) -> String {
"Set cycle region".to_string()
}
fn execute_backend(
&mut self,
backend: &mut BackendContext,
_document: &Document,
) -> Result<(), String> {
let controller = match backend.audio_controller.as_mut() {
Some(c) => c,
None => return Ok(()),
};
controller.set_loop_region(self.new_region);
controller.set_loop_enabled(self.new_enabled);
Ok(())
}
fn rollback_backend(
&mut self,
backend: &mut BackendContext,
_document: &Document,
) -> Result<(), String> {
let controller = match backend.audio_controller.as_mut() {
Some(c) => c,
None => return Ok(()),
};
controller.set_loop_region(self.old_region);
controller.set_loop_enabled(self.old_enabled);
Ok(())
}
}

View File

@ -81,11 +81,9 @@ impl Action for SetKeyframeAction {
if let Some(AnyLayer::Vector(vl)) = document.get_layer(&self.layer_id) {
for clip_id in &self.clip_instance_ids {
if let Some(ci) = vl.clip_instances.iter().find(|c| c.id == *clip_id) {
// `start` is a keyframe time in seconds; group_visibility_start returns seconds,
// so the fallback must convert the clip's beats start to seconds too.
let start = vl
.group_visibility_start(clip_id, self.time)
.unwrap_or_else(|| document.tempo_map().beats_to_seconds(ci.timeline_start).seconds_to_f64());
.unwrap_or(ci.timeline_start);
clip_info.insert(*clip_id, (ci.transform.clone(), ci.opacity, start));
}
}

View File

@ -7,7 +7,6 @@ use crate::action::{Action, BackendContext};
use crate::clip::ClipInstance;
use crate::document::Document;
use crate::layer::AnyLayer;
use daw_backend::ContentTime;
use uuid::Uuid;
/// Action that splits a clip instance at a specific timeline position
@ -18,17 +17,17 @@ pub struct SplitClipInstanceAction {
/// The clip instance to split
instance_id: Uuid,
/// Timeline position where to split (in beats)
split_time: daw_backend::Beats,
/// Timeline time where to split (in seconds)
split_time: f64,
/// Whether the action has been executed (for rollback)
executed: bool,
// Stored during execute for rollback
/// Original trim_end value of the left (original) instance
original_trim_end: Option<ContentTime>,
/// Original timeline_duration value of the left (original) instance (beats)
original_timeline_duration: Option<daw_backend::Beats>,
original_trim_end: Option<f64>,
/// Original timeline_duration value of the left (original) instance
original_timeline_duration: Option<f64>,
/// ID of the new (right) instance created by the split
new_instance_id: Option<Uuid>,
@ -48,8 +47,8 @@ impl SplitClipInstanceAction {
///
/// * `layer_id` - The ID of the layer containing the clip instance
/// * `instance_id` - The ID of the clip instance to split
/// * `split_time` - The timeline position (in beats) where to split
pub fn new(layer_id: Uuid, instance_id: Uuid, split_time: daw_backend::Beats) -> Self {
/// * `split_time` - The timeline time (in seconds) where to split
pub fn new(layer_id: Uuid, instance_id: Uuid, split_time: f64) -> Self {
Self {
layer_id,
instance_id,
@ -73,9 +72,9 @@ impl SplitClipInstanceAction {
///
/// * `layer_id` - The ID of the layer containing the clip instance
/// * `instance_id` - The ID of the clip instance to split
/// * `split_time` - The timeline position (in beats) where to split
/// * `split_time` - The timeline time (in seconds) where to split
/// * `new_instance_id` - The UUID to use for the new (right) clip instance
pub fn with_new_instance_id(layer_id: Uuid, instance_id: Uuid, split_time: daw_backend::Beats, new_instance_id: Uuid) -> Self {
pub fn with_new_instance_id(layer_id: Uuid, instance_id: Uuid, split_time: f64, new_instance_id: Uuid) -> Self {
Self {
layer_id,
instance_id,
@ -123,21 +122,19 @@ impl Action for SplitClipInstanceAction {
.find(|ci| ci.id == self.instance_id)
.ok_or_else(|| format!("Clip instance {} not found", self.instance_id))?;
// The clip's content duration in its OWN domain — seconds for audio/video/vector, beats for
// MIDI. All the content math below is trim-domain, so it has to be done in whichever domain
// this clip uses; a seconds duration would silently be added to a MIDI clip's beats trim.
let trim_duration = document
.clip_trim_duration(&instance.clip_id)
// Get the clip's duration
let clip_duration = document
.get_clip_duration(&instance.clip_id)
.ok_or_else(|| format!("Clip {} not found", instance.clip_id))?;
// Calculate the effective duration and timeline end (both in beats)
let effective_duration = instance.effective_duration(trim_duration, document.tempo_map());
let effective_duration = instance.effective_duration(clip_duration, document.tempo_map());
let timeline_end = instance.timeline_start + effective_duration;
// Validate: split_time must be strictly within the clip's timeline span
let epsilon = daw_backend::Beats(0.001); // ~1ms tolerance
if self.split_time <= instance.timeline_start + epsilon
|| self.split_time >= timeline_end - epsilon
const EPSILON: f64 = 0.001; // 1ms tolerance
if self.split_time <= instance.timeline_start + EPSILON
|| self.split_time >= timeline_end - EPSILON
{
return Err(format!(
"Split time {} must be within clip bounds ({} to {})",
@ -149,30 +146,21 @@ impl Action for SplitClipInstanceAction {
self.original_trim_end = instance.trim_end;
self.original_timeline_duration = instance.timeline_duration;
// Check if this is a looping clip
let is_looping = instance.timeline_duration.is_some();
let content_duration = ContentTime(instance.content_window(trim_duration).native());
let content_duration = instance.trim_end.unwrap_or(clip_duration) - instance.trim_start;
// Timeline split point (beats).
// Calculate the split point
let time_into_clip = self.split_time - instance.timeline_start;
let left_duration = time_into_clip;
let right_duration = effective_duration - left_duration;
// How far the split lands into the clip's *content*, expressed in the content domain: beats
// content takes the beats delta directly, wall-clock content takes the seconds delta.
let tempo_map = document.tempo_map();
let time_into_content = ContentTime(match trim_duration {
crate::clip::ClipDuration::Beats(_) => time_into_clip.beats_to_f64(),
crate::clip::ClipDuration::Seconds(_) => (tempo_map.beats_to_seconds(self.split_time)
- tempo_map.beats_to_seconds(instance.timeline_start))
.seconds_to_f64(),
});
// Calculate the content split point (content domain).
let content_split_time = if is_looping && content_duration > ContentTime::ZERO {
// Calculate content split time
let content_split_time = if is_looping {
// For looping clips, wrap around content
instance.trim_start + (time_into_content % content_duration)
instance.trim_start + (time_into_clip % content_duration)
} else {
instance.trim_start + time_into_content
instance.trim_start + time_into_clip
};
// Clone the instance for the right side
@ -363,63 +351,112 @@ impl Action for SplitClipInstanceAction {
.get_audio_clip(&new_instance.clip_id)
.ok_or_else(|| "Audio clip not found".to_string())?;
use crate::clip::ResolvedContent;
if matches!(original_instance.resolve(clip), ResolvedContent::Recording) {
return Err("Cannot split a clip that is currently recording".to_string());
}
// A split is: shorten the left half's backend clip, then add the right half as a new one.
//
// 1. Trim the left (original) instance. `trim_range` tags the bounds with the clip's own
// content domain, so a MIDI clip's beats trims can't be sent as seconds.
let left_trim = clip.trim_range(
original_instance.trim_start,
original_instance
.trim_end
.unwrap_or(ContentTime(clip.content_duration().native())),
);
let new_instance = new_instance.clone();
let backend_track_id = *backend
// Look up backend track ID from layer mapping
let backend_track_id = backend
.layer_to_track_map
.get(&self.layer_id)
.ok_or_else(|| format!("Layer {} not mapped to backend track", self.layer_id))?;
let left_backend_id = backend
.clip_instance_to_backend_map
.get(&self.instance_id)
.copied();
{
let controller = backend
.audio_controller
.as_mut()
.ok_or_else(|| "Audio controller not available".to_string())?;
match left_backend_id {
Some(crate::action::BackendClipInstanceId::Midi(id))
| Some(crate::action::BackendClipInstanceId::Audio(id)) => {
controller.trim_clip(backend_track_id, id, left_trim);
}
None => {}
}
}
// Get audio controller
let controller = backend
.audio_controller
.as_mut()
.ok_or_else(|| "Audio controller not available".to_string())?;
// 2. Add the right (new) instance via the shared helper — same one AddClipInstanceAction
// uses, so the trim/duration conversions live in exactly one place.
if let Some((track_id, backend_id)) =
backend.add_clip_instance(document, &self.layer_id, &new_instance)?
{
self.backend_track_id = Some(track_id);
match backend_id {
crate::action::BackendClipInstanceId::Midi(id) => {
self.backend_midi_instance_id = Some(id)
// Handle different clip types
use crate::clip::AudioClipType;
match &clip.clip_type {
AudioClipType::Midi { midi_clip_id } => {
use daw_backend::command::{Query, QueryResponse};
// 1. Trim the original (left) instance
let orig_internal_start = original_instance.trim_start;
let orig_internal_end = original_instance.trim_end.unwrap_or(clip.duration);
// Look up the original backend instance ID
if let Some(crate::action::BackendClipInstanceId::Midi(orig_backend_id)) =
backend.clip_instance_to_backend_map.get(&self.instance_id)
{
controller.trim_clip(*backend_track_id, *orig_backend_id, orig_internal_start, orig_internal_end);
}
crate::action::BackendClipInstanceId::Audio(id) => {
self.backend_audio_instance_id = Some(id)
// 2. Add the new (right) instance
let internal_start = new_instance.trim_start;
let internal_end = new_instance.trim_end.unwrap_or(clip.duration);
let external_start = new_instance.timeline_start;
let external_duration = new_instance
.timeline_duration
.unwrap_or(internal_end - internal_start);
let instance = daw_backend::MidiClipInstance::new(
0,
*midi_clip_id,
daw_backend::Beats(internal_start),
daw_backend::Beats(internal_end),
daw_backend::Beats(external_start),
daw_backend::Beats(external_duration),
);
let query = Query::AddMidiClipInstanceSync(*backend_track_id, instance);
match controller.send_query(query)? {
QueryResponse::MidiClipInstanceAdded(Ok(instance_id)) => {
self.backend_track_id = Some(*backend_track_id);
self.backend_midi_instance_id = Some(instance_id);
backend.clip_instance_to_backend_map.insert(
new_instance_id,
crate::action::BackendClipInstanceId::Midi(instance_id),
);
Ok(())
}
QueryResponse::MidiClipInstanceAdded(Err(e)) => Err(e),
_ => Err("Unexpected query response".to_string()),
}
}
}
AudioClipType::Sampled { audio_pool_index } => {
// 1. Trim the original (left) instance
let orig_internal_start = original_instance.trim_start;
let orig_internal_end = original_instance.trim_end.unwrap_or(clip.duration);
Ok(())
// Look up the original backend instance ID
if let Some(crate::action::BackendClipInstanceId::Audio(orig_backend_id)) =
backend.clip_instance_to_backend_map.get(&self.instance_id)
{
controller.trim_clip(*backend_track_id, *orig_backend_id, orig_internal_start, orig_internal_end);
}
// 2. Add the new (right) instance
let internal_start = new_instance.trim_start;
let internal_end = new_instance.trim_end.unwrap_or(clip.duration);
let effective_duration = new_instance.timeline_duration
.unwrap_or(internal_end - internal_start);
let start_time = new_instance.timeline_start;
let instance_id = controller.add_audio_clip(
*backend_track_id,
*audio_pool_index,
start_time,
effective_duration,
internal_start,
);
self.backend_track_id = Some(*backend_track_id);
self.backend_audio_instance_id = Some(instance_id);
backend.clip_instance_to_backend_map.insert(
new_instance_id,
crate::action::BackendClipInstanceId::Audio(instance_id),
);
Ok(())
}
AudioClipType::Recording => {
// Recording clips cannot be split
Err("Cannot split a clip that is currently recording".to_string())
}
}
}
fn rollback_backend(
@ -449,28 +486,26 @@ impl Action for SplitClipInstanceAction {
if let Some(instance) = al.clip_instances.iter().find(|ci| ci.id == self.instance_id) {
if let Some(clip) = document.get_audio_clip(&instance.clip_id) {
let orig_internal_start = instance.trim_start;
let orig_internal_end = self
.original_trim_end
.unwrap_or(ContentTime(clip.content_duration().native()));
let orig_internal_end = self.original_trim_end.unwrap_or(clip.duration);
// Restore based on clip type
use crate::clip::ResolvedContent;
match &instance.resolve(clip) {
ResolvedContent::Midi { .. } => {
use crate::clip::AudioClipType;
match &clip.clip_type {
AudioClipType::Midi { .. } => {
if let Some(crate::action::BackendClipInstanceId::Midi(orig_backend_id)) =
backend.clip_instance_to_backend_map.get(&self.instance_id)
{
controller.trim_clip(track_id, *orig_backend_id, clip.trim_range(orig_internal_start, orig_internal_end));
controller.trim_clip(track_id, *orig_backend_id, orig_internal_start, orig_internal_end);
}
}
ResolvedContent::Audio { .. } => {
AudioClipType::Sampled { .. } => {
if let Some(crate::action::BackendClipInstanceId::Audio(orig_backend_id)) =
backend.clip_instance_to_backend_map.get(&self.instance_id)
{
controller.trim_clip(track_id, *orig_backend_id, clip.trim_range(orig_internal_start, orig_internal_end));
controller.trim_clip(track_id, *orig_backend_id, orig_internal_start, orig_internal_end);
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
// Recording clips - nothing to rollback
}
}
@ -505,16 +540,16 @@ mod tests {
// Create a clip instance at timeline 0, with trim 0-10 (10 seconds)
let mut clip_instance = ClipInstance::new(clip_id);
clip_instance.timeline_start = daw_backend::Beats::ZERO;
clip_instance.trim_start = ContentTime::ZERO;
clip_instance.trim_end = Some(ContentTime(10.0));
clip_instance.timeline_start = 0.0;
clip_instance.trim_start = 0.0;
clip_instance.trim_end = Some(10.0);
let instance_id = clip_instance.id;
vector_layer.clip_instances.push(clip_instance);
let layer_id = document.root.add_child(AnyLayer::Vector(vector_layer));
// Split at timeline 5.0
let mut action = SplitClipInstanceAction::new(layer_id, instance_id, daw_backend::Beats(5.0));
let mut action = SplitClipInstanceAction::new(layer_id, instance_id, 5.0);
// Execute - this will fail because we don't have a real clip in the document
// In a real test, we'd need to add a VectorClip first
@ -524,45 +559,7 @@ mod tests {
#[test]
fn test_split_action_description() {
let action = SplitClipInstanceAction::new(Uuid::new_v4(), Uuid::new_v4(), daw_backend::Beats(5.0));
let action = SplitClipInstanceAction::new(Uuid::new_v4(), Uuid::new_v4(), 5.0);
assert_eq!(action.description(), "Split clip instance");
}
#[test]
fn splitting_a_midi_clip_stays_in_the_beats_domain() {
// Regression: `trim_start`/`trim_end` are domain-polymorphic — SECONDS for audio/video/
// vector, but BEATS for MIDI (the backend takes MIDI trims as `Beats`). Split used to map
// the split point into the clip's content in seconds unconditionally, so on a MIDI clip it
// added a seconds delta to a beats offset. At anything but 60 BPM the right half started at
// the wrong place in the content.
//
// At 120 BPM, beat 4 is 2 SECONDS in. The right half must trim to beat 4, not "4 seconds"
// (= beat 8) and not 2 (the seconds value).
let mut document = Document::new("Test");
document.set_bpm(120.0);
// 8-beat MIDI clip at the timeline origin.
let clip = crate::clip::AudioClip::new_midi("Midi", 1, daw_backend::Beats(8.0));
let clip_id = document.add_audio_clip(clip);
let mut audio_layer = crate::layer::AudioLayer::new("Layer 1");
let mut instance = ClipInstance::new(clip_id);
instance.timeline_start = daw_backend::Beats::ZERO;
instance.trim_start = ContentTime::ZERO;
instance.trim_end = Some(ContentTime(8.0)); // beats
let instance_id = instance.id;
audio_layer.clip_instances.push(instance);
let layer_id = document.root.add_child(AnyLayer::Audio(audio_layer));
let mut action = SplitClipInstanceAction::new(layer_id, instance_id, daw_backend::Beats(4.0));
action.execute(&mut document).expect("split");
let new_id = action.new_instance_id().expect("right instance");
let AnyLayer::Audio(al) = document.get_layer(&layer_id).unwrap() else { panic!() };
let right = al.clip_instances.iter().find(|ci| ci.id == new_id).unwrap();
let left = al.clip_instances.iter().find(|ci| ci.id == instance_id).unwrap();
assert_eq!(right.trim_start, ContentTime(4.0), "right half must start 4 BEATS into the content");
assert_eq!(left.trim_end, Some(ContentTime(4.0)), "left half must end 4 BEATS into the content");
}
}

View File

@ -6,7 +6,6 @@ use crate::action::Action;
use crate::clip::ClipInstance;
use crate::document::Document;
use crate::layer::AnyLayer;
use daw_backend::{Beats, ContentTime, Seconds};
use std::collections::HashMap;
use uuid::Uuid;
@ -32,57 +31,15 @@ pub struct TrimClipInstancesAction {
pub struct TrimData {
/// For TrimLeft: trim_start value
/// For TrimRight: trim_end value (Option because it can be None)
///
/// A content time — measured in the clip's own domain (seconds for audio/video/vector, beats
/// for MIDI), so it must be resolved against the clip before meeting a timeline position.
pub trim_value: Option<ContentTime>,
/// For TrimLeft: timeline_start value (where the clip appears on timeline, beats)
pub trim_value: Option<f64>,
/// For TrimLeft: timeline_start value (where the clip appears on timeline)
/// For TrimRight: unused (None)
pub timeline_start: Option<Beats>,
}
/// A wall-clock gap on the timeline, expressed in a clip's content domain.
///
/// Trim validation clamps how far a clip may be dragged against the empty space next to it, and that
/// space is measured on the timeline (seconds) while the trim lives in the clip's content domain. For
/// wall-clock content they're the same number; for MIDI (beats content) the gap has to be converted
/// at the clip's position, or a seconds gap silently clamps a beats trim.
fn gap_to_content(
gap: Seconds,
clip_content: crate::clip::ClipDuration,
timeline_start: Beats,
tempo_map: &crate::tempo_map::TempoMap,
) -> ContentTime {
match clip_content {
crate::clip::ClipDuration::Seconds(_) => ContentTime(gap.seconds_to_f64()),
crate::clip::ClipDuration::Beats(_) => {
let beats = tempo_map
.seconds_to_beats(tempo_map.beats_to_seconds(timeline_start) + gap)
- timeline_start;
ContentTime(beats.beats_to_f64())
}
}
}
/// The inverse: a content-domain span as wall-clock seconds at the clip's position.
fn content_to_secs(
span: ContentTime,
clip_content: crate::clip::ClipDuration,
timeline_start: Beats,
tempo_map: &crate::tempo_map::TempoMap,
) -> Seconds {
match clip_content {
crate::clip::ClipDuration::Seconds(_) => Seconds(span.raw()),
crate::clip::ClipDuration::Beats(_) => {
tempo_map.beats_to_seconds(timeline_start + Beats(span.raw()))
- tempo_map.beats_to_seconds(timeline_start)
}
}
pub timeline_start: Option<f64>,
}
impl TrimData {
/// Create TrimData for left trim
pub fn left(trim_start: ContentTime, timeline_start: Beats) -> Self {
pub fn left(trim_start: f64, timeline_start: f64) -> Self {
Self {
trim_value: Some(trim_start),
timeline_start: Some(timeline_start),
@ -90,7 +47,7 @@ impl TrimData {
}
/// Create TrimData for right trim
pub fn right(trim_end: Option<ContentTime>) -> Self {
pub fn right(trim_end: Option<f64>) -> Self {
Self {
trim_value: trim_end,
timeline_start: None,
@ -234,8 +191,7 @@ impl Action for TrimClipInstancesAction {
.find(|ci| &ci.id == instance_id)
.ok_or_else(|| format!("Instance {} not found", instance_id))?;
// The clip's content duration in ITS OWN domain, so trims resolve correctly for MIDI.
let clip_content = document.clip_trim_duration(&instance.clip_id)
let clip_duration = document.get_clip_duration(&instance.clip_id)
.ok_or_else(|| format!("Clip {} not found", instance.clip_id))?;
let mut clamped_new = new.clone();
@ -247,74 +203,51 @@ impl Action for TrimClipInstancesAction {
{
// If extending to the left (new_trim < old_trim)
if should_validate && new_trim < old_trim {
let tempo_map = document.tempo_map();
// Max leftward extension: the gap's wall-clock span, converted into
// the clip's content domain so it can clamp a content-domain trim.
let max_extend = gap_to_content(
document.find_max_trim_extend_left(
layer_id,
instance_id,
instance.timeline_start,
),
clip_content,
// Find the maximum we can extend left
let max_extend = document.find_max_trim_extend_left(
layer_id,
instance_id,
instance.timeline_start,
tempo_map,
);
// Calculate how much we want to extend
let desired_extend = old_trim - new_trim;
// Clamp to max allowed
let actual_extend = desired_extend.min(max_extend);
let clamped_trim_start = old_trim - actual_extend;
// Move the timeline left by the same span, as wall-clock seconds.
let shift = content_to_secs(
actual_extend,
clip_content,
instance.timeline_start,
tempo_map,
);
let clamped_timeline_start = tempo_map
.seconds_to_beats(tempo_map.beats_to_seconds(old_timeline) - shift)
.max(Beats::ZERO);
let clamped_timeline_start = (old_timeline - actual_extend).max(0.0);
clamped_new = TrimData::left(clamped_trim_start, clamped_timeline_start);
}
}
}
TrimType::TrimRight => {
let content_end = ContentTime(clip_content.native());
let old_trim_end = old.trim_value.unwrap_or(content_end);
let new_trim_end = new.trim_value.unwrap_or(content_end);
let old_trim_end = old.trim_value.unwrap_or(clip_duration);
let new_trim_end = new.trim_value.unwrap_or(clip_duration);
// If extending to the right (new_trim_end > old_trim_end)
if should_validate && new_trim_end > old_trim_end {
let tempo_map = document.tempo_map();
// Calculate current effective duration
let current_effective_duration = old_trim_end - instance.trim_start;
// How long the clip currently occupies the timeline, in beats. Resolved
// in the clip's own domain, so a MIDI clip's beats content isn't run
// through the seconds→beats conversion a second time.
let current_effective_duration = instance
.effective_duration_beats(clip_content, tempo_map);
// Max rightward extension: the gap's wall-clock span, in content domain.
let max_extend = gap_to_content(
document.find_max_trim_extend_right(
layer_id,
instance_id,
instance.timeline_start,
current_effective_duration,
),
clip_content,
// Find the maximum we can extend right
let max_extend = document.find_max_trim_extend_right(
layer_id,
instance_id,
instance.timeline_start,
tempo_map,
current_effective_duration,
);
// Calculate how much we want to extend
let desired_extend = new_trim_end - old_trim_end;
// Clamp to max allowed
let actual_extend = desired_extend.min(max_extend);
let clamped_trim_end = old_trim_end + actual_extend;
// Don't exceed the clip's content.
let final_trim_end = clamped_trim_end.min(content_end);
// Don't exceed clip duration
let final_trim_end = clamped_trim_end.min(clip_duration);
clamped_new = TrimData::right(Some(final_trim_end));
}
@ -423,7 +356,7 @@ impl Action for TrimClipInstancesAction {
fn execute_backend(&mut self, backend: &mut crate::action::BackendContext, document: &Document) -> Result<(), String> {
use crate::layer::AnyLayer;
use crate::clip::ResolvedContent;
use crate::clip::AudioClipType;
// Get audio controller
let controller = match backend.audio_controller.as_mut() {
@ -443,10 +376,9 @@ impl Action for TrimClipInstancesAction {
if let Some(instance) = vl.clip_instances.iter().find(|ci| ci.id == *instance_id) {
if let Some(&metatrack_id) = backend.layer_to_track_map.get(&instance.clip_id) {
// Instance already has new values after execute()
controller.set_offset(metatrack_id, document.tempo_map().beats_to_seconds(instance.timeline_start));
// A vector clip's content is wall-clock, so its content times ARE seconds.
controller.set_trim_start(metatrack_id, daw_backend::Seconds(instance.trim_start.raw()));
controller.set_trim_end(metatrack_id, instance.trim_end.map(|t| daw_backend::Seconds(t.raw())));
controller.set_offset(metatrack_id, instance.timeline_start);
controller.set_trim_start(metatrack_id, instance.trim_start);
controller.set_trim_end(metatrack_id, instance.trim_end);
}
}
}
@ -482,29 +414,27 @@ impl Action for TrimClipInstancesAction {
// Calculate new internal_start and internal_end for backend
// Note: instance already has the new trim values after execute()
let internal_start = instance.trim_start;
let internal_end = instance
.trim_end
.unwrap_or(ContentTime(clip.content_duration().native()));
let internal_end = instance.trim_end.unwrap_or(clip.duration);
// Handle trim based on clip type
match &instance.resolve(clip) {
ResolvedContent::Midi { midi_clip_id } => {
match &clip.clip_type {
AudioClipType::Midi { midi_clip_id } => {
// For MIDI: trim_clip expects the pool clip ID
controller.trim_clip(*track_id, *midi_clip_id, clip.trim_range(internal_start, internal_end));
controller.trim_clip(*track_id, *midi_clip_id, internal_start, internal_end);
}
ResolvedContent::Audio { .. } => {
AudioClipType::Sampled { .. } => {
// For sampled audio: trim_clip expects the instance ID
let backend_instance_id = backend.clip_instance_to_backend_map.get(instance_id)
.ok_or_else(|| format!("Clip instance {} not mapped to backend", instance_id))?;
match backend_instance_id {
crate::action::BackendClipInstanceId::Audio(audio_id) => {
controller.trim_clip(*track_id, *audio_id, clip.trim_range(internal_start, internal_end));
controller.trim_clip(*track_id, *audio_id, internal_start, internal_end);
}
_ => return Err("Expected audio instance ID for sampled clip".to_string()),
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
// Recording clips cannot be trimmed - skip
}
}
@ -516,7 +446,7 @@ impl Action for TrimClipInstancesAction {
fn rollback_backend(&mut self, backend: &mut crate::action::BackendContext, document: &Document) -> Result<(), String> {
use crate::layer::AnyLayer;
use crate::clip::ResolvedContent;
use crate::clip::AudioClipType;
// Get audio controller
let controller = match backend.audio_controller.as_mut() {
@ -536,10 +466,9 @@ impl Action for TrimClipInstancesAction {
if let Some(instance) = vl.clip_instances.iter().find(|ci| ci.id == *instance_id) {
if let Some(&metatrack_id) = backend.layer_to_track_map.get(&instance.clip_id) {
// Instance already has old values after rollback()
controller.set_offset(metatrack_id, document.tempo_map().beats_to_seconds(instance.timeline_start));
// A vector clip's content is wall-clock, so its content times ARE seconds.
controller.set_trim_start(metatrack_id, daw_backend::Seconds(instance.trim_start.raw()));
controller.set_trim_end(metatrack_id, instance.trim_end.map(|t| daw_backend::Seconds(t.raw())));
controller.set_offset(metatrack_id, instance.timeline_start);
controller.set_trim_start(metatrack_id, instance.trim_start);
controller.set_trim_end(metatrack_id, instance.trim_end);
}
}
}
@ -573,35 +502,34 @@ impl Action for TrimClipInstancesAction {
.ok_or_else(|| format!("Audio clip {} not found", instance.clip_id))?;
// Calculate old internal_start and internal_end for backend
let content_end = ContentTime(clip.content_duration().native());
let internal_start = match trim_type {
TrimType::TrimLeft => old.trim_value.unwrap_or(ContentTime::ZERO),
TrimType::TrimLeft => old.trim_value.unwrap_or(0.0),
TrimType::TrimRight => instance.trim_start, // trim_start wasn't changed
};
let internal_end = match trim_type {
TrimType::TrimLeft => instance.trim_end.unwrap_or(content_end), // trim_end wasn't changed
TrimType::TrimRight => old.trim_value.unwrap_or(content_end),
TrimType::TrimLeft => instance.trim_end.unwrap_or(clip.duration), // trim_end wasn't changed
TrimType::TrimRight => old.trim_value.unwrap_or(clip.duration),
};
// Handle trim based on clip type
match &instance.resolve(clip) {
ResolvedContent::Midi { midi_clip_id } => {
match &clip.clip_type {
AudioClipType::Midi { midi_clip_id } => {
// For MIDI: trim_clip expects the pool clip ID
controller.trim_clip(*track_id, *midi_clip_id, clip.trim_range(internal_start, internal_end));
controller.trim_clip(*track_id, *midi_clip_id, internal_start, internal_end);
}
ResolvedContent::Audio { .. } => {
AudioClipType::Sampled { .. } => {
// For sampled audio: trim_clip expects the instance ID
let backend_instance_id = backend.clip_instance_to_backend_map.get(instance_id)
.ok_or_else(|| format!("Clip instance {} not mapped to backend", instance_id))?;
match backend_instance_id {
crate::action::BackendClipInstanceId::Audio(audio_id) => {
controller.trim_clip(*track_id, *audio_id, clip.trim_range(internal_start, internal_end));
controller.trim_clip(*track_id, *audio_id, internal_start, internal_end);
}
_ => return Err("Expected audio instance ID for sampled clip".to_string()),
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
// Recording clips cannot be trimmed - skip
}
}
@ -630,8 +558,8 @@ mod tests {
let mut vector_layer = VectorLayer::new("Layer 1");
let mut clip_instance = ClipInstance::new(clip_id);
clip_instance.timeline_start = Beats::ZERO;
clip_instance.trim_start = ContentTime::ZERO;
clip_instance.timeline_start = 0.0;
clip_instance.trim_start = 0.0;
let instance_id = clip_instance.id;
vector_layer.clip_instances.push(clip_instance);
@ -644,8 +572,8 @@ mod tests {
vec![(
instance_id,
TrimType::TrimLeft,
TrimData::left(ContentTime::ZERO, Beats::ZERO),
TrimData::left(ContentTime(2.0), Beats(2.0)),
TrimData::left(0.0, 0.0),
TrimData::left(2.0, 2.0),
)],
);
@ -661,8 +589,8 @@ mod tests {
.iter()
.find(|ci| ci.id == instance_id)
.unwrap();
assert_eq!(instance.trim_start, ContentTime(2.0));
assert_eq!(instance.timeline_start, Beats(2.0));
assert_eq!(instance.trim_start, 2.0);
assert_eq!(instance.timeline_start, 2.0);
}
// Rollback
@ -675,8 +603,8 @@ mod tests {
.iter()
.find(|ci| ci.id == instance_id)
.unwrap();
assert_eq!(instance.trim_start, ContentTime::ZERO);
assert_eq!(instance.timeline_start, Beats::ZERO);
assert_eq!(instance.trim_start, 0.0);
assert_eq!(instance.timeline_start, 0.0);
}
}
@ -706,7 +634,7 @@ mod tests {
instance_id,
TrimType::TrimRight,
TrimData::right(None),
TrimData::right(Some(ContentTime(8.0))),
TrimData::right(Some(8.0)),
)],
);
@ -722,7 +650,7 @@ mod tests {
.iter()
.find(|ci| ci.id == instance_id)
.unwrap();
assert_eq!(instance.trim_end, Some(ContentTime(8.0)));
assert_eq!(instance.trim_end, Some(8.0));
}
// Rollback

View File

@ -1,13 +1,7 @@
use crate::action::Action;
use crate::document::Document;
use daw_backend::Beats;
use uuid::Uuid;
/// Convert editor-side beats-domain note tuples to the typed backend form.
fn notes_to_beats(notes: &[(f64, u8, u8, f64)]) -> Vec<(Beats, u8, u8, Beats)> {
notes.iter().map(|&(t, n, v, d)| (Beats(t), n, v, Beats(d))).collect()
}
/// Action to update MIDI notes in a clip (supports undo/redo)
///
/// Stores the before and after note states. MIDI note data lives in the backend,
@ -55,8 +49,7 @@ impl Action for UpdateMidiNotesAction {
.get(&self.layer_id)
.ok_or_else(|| format!("Layer {} not mapped to backend track", self.layer_id))?;
// Note times/durations are beats (MIDI content domain); assert that at the typed boundary.
controller.update_midi_clip_notes(*track_id, self.midi_clip_id, notes_to_beats(&self.new_notes));
controller.update_midi_clip_notes(*track_id, self.midi_clip_id, self.new_notes.clone());
Ok(())
}
@ -75,7 +68,7 @@ impl Action for UpdateMidiNotesAction {
.get(&self.layer_id)
.ok_or_else(|| format!("Layer {} not mapped to backend track", self.layer_id))?;
controller.update_midi_clip_notes(*track_id, self.midi_clip_id, notes_to_beats(&self.old_notes));
controller.update_midi_clip_notes(*track_id, self.midi_clip_id, self.old_notes.clone());
Ok(())
}

View File

@ -14,7 +14,6 @@
use crate::layer::AnyLayer;
use crate::layer_tree::LayerTree;
use crate::object::Transform;
use daw_backend::{Beats, ContentTime, Seconds};
use serde::{Deserialize, Serialize};
use std::path::PathBuf;
use uuid::Uuid;
@ -111,7 +110,7 @@ impl VectorClip {
pub fn content_duration_with(&self, framerate: f64, tempo_map: &crate::tempo_map::TempoMap, clip_duration_fn: impl Fn(&Uuid) -> Option<f64>) -> f64 {
let frame_duration = 1.0 / framerate;
// Work in beats, convert to seconds at the end.
let mut last_beats: Option<Beats> = None;
let mut last_beats: Option<f64> = None;
let mut last_secs: Option<f64> = None;
for layer_node in self.layers.iter() {
@ -127,22 +126,18 @@ impl VectorClip {
};
for ci in clip_instances {
// Compute end position of this clip instance in beats
let end_beats: Beats = if let Some(td_beats) = ci.timeline_duration {
let end_beats = if let Some(td_beats) = ci.timeline_duration {
ci.timeline_start + td_beats
} else if let Some(te) = ci.trim_end {
// `clip_duration_fn` hands back seconds, so this whole path treats nested
// content as wall-clock. That's right for the vector/video/audio clips a vector
// clip actually nests; a nested MIDI clip (beats content) would need resolving
// against its clip, which this callback can't do. Pre-existing limitation.
let secs = (te - ci.trim_start).raw().max(0.0);
tempo_map.seconds_to_beats(tempo_map.beats_to_seconds(ci.timeline_start) + Seconds(secs))
let secs = (te - ci.trim_start).max(0.0);
ci.timeline_start + tempo_map.inverse_transform(tempo_map.transform(ci.timeline_start) + secs) - ci.timeline_start
} else if let Some(clip_dur_secs) = clip_duration_fn(&ci.clip_id) {
let secs = (clip_dur_secs - ci.trim_start.raw()).max(0.0);
tempo_map.seconds_to_beats(tempo_map.beats_to_seconds(ci.timeline_start) + Seconds(secs))
let secs = (clip_dur_secs - ci.trim_start).max(0.0);
ci.timeline_start + tempo_map.inverse_transform(tempo_map.transform(ci.timeline_start) + secs) - ci.timeline_start
} else {
continue;
};
last_beats = Some(last_beats.map_or(end_beats, |t: Beats| t.max(end_beats)));
last_beats = Some(last_beats.map_or(end_beats, |t: f64| t.max(end_beats)));
}
// Vector layer keyframes are in seconds
@ -153,7 +148,7 @@ impl VectorClip {
}
}
let from_clips = last_beats.map(|b| tempo_map.beats_to_seconds(b).seconds_to_f64());
let from_clips = last_beats.map(|b| tempo_map.transform(b));
let combined = match (from_clips, last_secs) {
(Some(a), Some(b)) => Some(a.max(b)),
(Some(a), None) => Some(a),
@ -204,10 +199,8 @@ impl VectorClip {
for clip_instance in &vector_layer.clip_instances {
// Convert parent clip time (seconds) to nested clip local time (seconds).
// timeline_start is in beats; convert to seconds using document BPM.
let start_secs = document.tempo_map().beats_to_seconds(clip_instance.timeline_start).seconds_to_f64();
// Nested clips here are vector clips, whose content is wall-clock seconds.
let nested_clip_time =
((clip_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start.raw();
let start_secs = document.tempo_map().transform(clip_instance.timeline_start);
let nested_clip_time = ((clip_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start;
// Look up the nested clip definition
let nested_bounds = if let Some(nested_clip) = document.get_vector_clip(&clip_instance.clip_id) {
@ -474,67 +467,6 @@ pub enum AudioClipType {
Recording,
}
/// One take of a cycle recording — see [`ClipInstance::takes`].
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct AudioTake {
/// Display name, e.g. "Take 1". User-editable.
pub name: String,
/// The recorded content this take points at.
pub content: TakeContent,
}
/// What a take actually holds. An instance's takes are all the same kind — one cycle-record session
/// captures either audio or MIDI, never a mix.
#[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)]
pub enum TakeContent {
/// Sampled audio: index into the audio pool.
Audio { audio_pool_index: usize },
/// MIDI: backend MIDI clip ID.
Midi { midi_clip_id: u32 },
}
/// A clip's content duration, tagged by its native unit.
///
/// Sampled/recording audio and video measure content in wall-clock **seconds**; MIDI measures
/// it in **beats** (tempo-independent musical length). Carrying the domain in the type means a
/// duration can't be silently read in the wrong unit.
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum ClipDuration {
Seconds(Seconds),
Beats(Beats),
}
impl ClipDuration {
/// Wall-clock seconds. Beats are converted as a length from beat 0 (exact under constant
/// tempo; a reasonable approximation otherwise — durations are position-independent here).
pub fn to_seconds(self, tempo_map: &daw_backend::TempoMap) -> Seconds {
match self {
ClipDuration::Seconds(s) => s,
ClipDuration::Beats(b) => tempo_map.beats_to_seconds(b),
}
}
/// The raw magnitude in the clip's native unit. Use only in code that already works in that
/// domain (e.g. trim math, whose values share the clip's native domain).
pub fn native(self) -> f64 {
match self {
ClipDuration::Seconds(s) => s.seconds_to_f64(),
ClipDuration::Beats(b) => b.beats_to_f64(),
}
}
/// Tag a [`ContentTime`] with *this* duration's domain.
///
/// Handy when you already hold a clip's content duration (so you know the domain) and need to
/// resolve one of its trim bounds, without going back to the clip.
pub fn same_domain(self, t: ContentTime) -> ClipDuration {
match self {
ClipDuration::Seconds(_) => ClipDuration::Seconds(Seconds(t.raw())),
ClipDuration::Beats(_) => ClipDuration::Beats(Beats(t.raw())),
}
}
}
/// Audio clip
///
/// This is compatible with daw-backend's audio system:
@ -548,13 +480,9 @@ pub struct AudioClip {
/// Clip name
pub name: String,
/// Raw content duration in the clip's **native domain** — SECONDS for sampled/recording
/// audio, BEATS for MIDI (musical length, tempo-independent). Private on purpose: the domain
/// depends on `clip_type`, so all access goes through [`AudioClip::content_duration`] /
/// [`AudioClip::set_content_duration`], which keep it type-safe. Stored as a bare `f64`
/// because the `.beam` format serializes it as a plain number (serde derives over private
/// fields fine); a domain-tagged newtype would change the on-disk shape.
duration: f64,
/// Duration in seconds
/// For sampled audio, this can be set to trim the audio shorter than the source file
pub duration: f64,
/// Audio clip type (sampled or MIDI)
pub clip_type: AudioClipType,
@ -565,34 +493,6 @@ pub struct AudioClip {
}
impl AudioClip {
/// The clip's content duration, tagged with its native domain (seconds for sampled/recording,
/// beats for MIDI). This is the only sanctioned way to read the raw `duration` field.
pub fn content_duration(&self) -> ClipDuration {
if self.is_midi_domain() {
ClipDuration::Beats(Beats(self.duration))
} else {
ClipDuration::Seconds(Seconds(self.duration))
}
}
/// Set the content duration. Debug-asserts the value's domain matches the clip type so a
/// beats duration can't be stored on a seconds clip (or vice-versa).
pub fn set_content_duration(&mut self, duration: ClipDuration) {
debug_assert!(
matches!(
(self.is_midi_domain(), duration),
(true, ClipDuration::Beats(_)) | (false, ClipDuration::Seconds(_))
),
"clip duration domain must match clip type",
);
self.duration = duration.native();
}
/// Whether this clip's `duration` is measured in beats (MIDI) rather than seconds.
fn is_midi_domain(&self) -> bool {
matches!(self.clip_type, AudioClipType::Midi { .. })
}
/// Create a new sampled audio clip
///
/// # Arguments
@ -676,73 +576,6 @@ impl AudioClip {
_ => None,
}
}
/// The clip's own content, ignoring takes.
///
/// Callers that are handing content to the backend want [`ClipInstance::resolve`] instead — an
/// instance with takes overrides the clip's content with whichever take is active.
pub fn resolve(&self) -> ResolvedContent {
match &self.clip_type {
AudioClipType::Sampled { audio_pool_index } => ResolvedContent::Audio {
audio_pool_index: *audio_pool_index,
},
AudioClipType::Midi { midi_clip_id } => ResolvedContent::Midi {
midi_clip_id: *midi_clip_id,
},
AudioClipType::Recording => ResolvedContent::Recording,
}
}
/// Resolve a content time against this clip's domain.
///
/// This is the ONLY sanctioned way to turn a [`ContentTime`] into a real duration — the type has
/// no `.to_seconds()` of its own precisely so that the clip, which is the one thing that knows
/// whether its content is measured in seconds or beats, has to be consulted.
pub fn resolve_content_time(&self, t: ContentTime) -> ClipDuration {
if self.is_midi_domain() {
ClipDuration::Beats(Beats(t.raw()))
} else {
ClipDuration::Seconds(Seconds(t.raw()))
}
}
/// Tag a pair of trim bounds with this clip's content domain, ready for the backend.
///
/// Building the [`TrimRange`] from the clip means a caller can't reach for the wrong variant:
/// the clip is the one thing that knows the domain.
pub fn trim_range(&self, start: ContentTime, end: ContentTime) -> daw_backend::command::TrimRange {
if self.is_midi_domain() {
daw_backend::command::TrimRange::Beats {
start: Beats(start.raw()),
end: Beats(end.raw()),
}
} else {
daw_backend::command::TrimRange::Seconds {
start: Seconds(start.raw()),
end: Seconds(end.raw()),
}
}
}
/// Whether this clip's own content is the given audio pool index.
pub fn owns_audio_pool_index(&self, pool_index: usize) -> bool {
self.audio_pool_index() == Some(pool_index)
}
/// Whether this clip's own content is the given backend MIDI clip ID.
pub fn owns_midi_clip_id(&self, id: u32) -> bool {
self.midi_clip_id() == Some(id)
}
}
/// What a clip instance actually plays, once takes are resolved to the active one.
/// Produced by [`ClipInstance::resolve`].
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum ResolvedContent {
Audio { audio_pool_index: usize },
Midi { midi_clip_id: u32 },
/// A recording in progress (or an empty take folder) — no backend content yet.
Recording,
}
/// Unified clip enum for polymorphic handling
@ -814,24 +647,23 @@ pub struct ClipInstance {
/// When this instance starts on the timeline, in **beats**.
/// Default: 0.0
pub timeline_start: Beats,
pub timeline_start: f64,
/// How long this instance appears on the timeline, in **beats**.
/// If set and longer than the trimmed content, the content will loop.
/// Default: None (use trimmed clip duration, no looping)
pub timeline_duration: Option<Beats>,
pub timeline_duration: Option<f64>,
/// Trim start: offset into the clip's internal content.
///
/// A [`ContentTime`] — measured in the CLIP's content domain, which is seconds for sampled
/// audio/video/vector but BEATS for MIDI. Resolve it against the clip
/// ([`Document::resolve_content_time`]) before combining it with anything on the timeline.
/// Trim start: offset into the clip's internal content, in **seconds**.
/// - For audio: byte-offset into the audio file
/// - For video: seek position in the video file
/// - For vector: time offset into the animation
/// Default: 0.0
pub trim_start: ContentTime,
pub trim_start: f64,
/// Trim end: offset into the clip's internal content. See [`Self::trim_start`].
/// Trim end: offset into the clip's internal content, in **seconds**.
/// Default: None (use full clip duration)
pub trim_end: Option<ContentTime>,
pub trim_end: Option<f64>,
/// Playback speed multiplier
/// 1.0 = normal speed, 0.5 = half speed, 2.0 = double speed
@ -847,36 +679,7 @@ pub struct ClipInstance {
/// When set, loop iterations are drawn/played before the content start.
/// Default: None (no pre-loop)
#[serde(default, skip_serializing_if = "Option::is_none")]
pub loop_before: Option<Beats>,
/// Alternate takes from cycle recording. Empty = an ordinary instance with no takes.
///
/// The takes live on the INSTANCE, not the clip, so managing them is per-instance: deleting or
/// renaming a take on one instance leaves every other instance alone. Splitting clones the list
/// along with the rest of the instance, so the two halves get independent take lists — and,
/// since they can each select a different take, comping still falls out for free.
///
/// Every take spans the full cycle region (partial passes are padded with silence at capture
/// time), so they're all the same length as the clip's own content. That uniformity is what lets
/// a take switch leave the instance's geometry untouched.
///
/// When non-empty, these OVERRIDE the clip's own content — see [`Self::resolve`].
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub takes: Vec<AudioTake>,
/// Which of [`Self::takes`] plays. `None` (or a stale index) means take 0.
/// Default: None
#[serde(default, skip_serializing_if = "Option::is_none")]
pub active_take: Option<usize>,
/// The cycle region's length in beats when these takes were recorded.
///
/// Audio takes are cut geometrically (by sample count), so they're only meaningful against the
/// tempo they were recorded at. Keeping the recorded length lets a future time-stretch/conform
/// feature reconcile them if the tempo moves underneath, and lets a new recording tell whether
/// it belongs in this take list or a fresh one.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub recorded_loop_beats: Option<Beats>,
pub loop_before: Option<f64>,
}
/// High 64-bit sentinel used to identify UUIDs that encode a backend audio clip instance ID.
@ -928,16 +731,13 @@ impl ClipInstance {
transform: Transform::default(),
opacity: 1.0,
name: None,
timeline_start: Beats::ZERO,
timeline_start: 0.0,
timeline_duration: None,
trim_start: ContentTime::ZERO,
trim_start: 0.0,
trim_end: None,
playback_speed: 1.0,
gain: 1.0,
loop_before: None,
takes: Vec::new(),
active_take: None,
recorded_loop_beats: None,
}
}
@ -949,16 +749,13 @@ impl ClipInstance {
transform: Transform::default(),
opacity: 1.0,
name: None,
timeline_start: Beats::ZERO,
timeline_start: 0.0,
timeline_duration: None,
trim_start: ContentTime::ZERO,
trim_start: 0.0,
trim_end: None,
playback_speed: 1.0,
gain: 1.0,
loop_before: None,
takes: Vec::new(),
active_take: None,
recorded_loop_beats: None,
}
}
@ -987,14 +784,14 @@ impl ClipInstance {
self
}
/// Set timeline position (beats).
pub fn with_timeline_start(mut self, timeline_start: Beats) -> Self {
/// Set timeline position
pub fn with_timeline_start(mut self, timeline_start: f64) -> Self {
self.timeline_start = timeline_start;
self
}
/// Set trimming (start and end time within the clip's internal content)
pub fn with_trimming(mut self, trim_start: ContentTime, trim_end: Option<ContentTime>) -> Self {
pub fn with_trimming(mut self, trim_start: f64, trim_end: Option<f64>) -> Self {
self.trim_start = trim_start;
self.trim_end = trim_end;
self
@ -1013,149 +810,80 @@ impl ClipInstance {
}
/// Set explicit timeline duration (in beats) by directly setting `timeline_duration`.
pub fn with_timeline_duration(mut self, duration_beats: Beats) -> Self {
pub fn with_timeline_duration(mut self, duration_beats: f64) -> Self {
self.timeline_duration = Some(duration_beats);
self
}
/// The take this instance plays, if it has any.
///
/// A `None`/stale `active_take` falls back to take 0 — an index can go stale (an undo that
/// shrank the list, an old `.beam`), and silently playing the first take beats playing nothing.
pub fn active_take(&self) -> Option<&AudioTake> {
self.takes
.get(self.active_take.unwrap_or(0))
.or_else(|| self.takes.first())
}
/// The index [`Self::active_take`] actually resolves to, clamped into range.
pub fn active_take_index(&self) -> usize {
let i = self.active_take.unwrap_or(0);
if i < self.takes.len() { i } else { 0 }
}
/// What this instance plays: its active take if it has takes, otherwise the clip's own content.
///
/// This is the sanctioned way to ask "what content do I hand the backend for this instance?".
/// Takes are never a distinct *case* at the call site — an instance with takes is just an audio
/// or MIDI instance whose identity depends on which take is live.
pub fn resolve(&self, clip: &AudioClip) -> ResolvedContent {
match self.active_take().map(|t| &t.content) {
Some(TakeContent::Audio { audio_pool_index }) => ResolvedContent::Audio {
audio_pool_index: *audio_pool_index,
},
Some(TakeContent::Midi { midi_clip_id }) => ResolvedContent::Midi {
midi_clip_id: *midi_clip_id,
},
None => clip.resolve(),
}
}
/// The audio pool index this instance plays. See [`Self::resolve`].
pub fn resolved_audio_pool_index(&self, clip: &AudioClip) -> Option<usize> {
match self.resolve(clip) {
ResolvedContent::Audio { audio_pool_index } => Some(audio_pool_index),
_ => None,
}
}
/// The backend MIDI clip ID this instance plays. See [`Self::resolve`].
pub fn resolved_midi_clip_id(&self, clip: &AudioClip) -> Option<u32> {
match self.resolve(clip) {
ResolvedContent::Midi { midi_clip_id } => Some(midi_clip_id),
_ => None,
}
}
/// Content window (`trim_end - trim_start`) in the clip's own content domain.
/// Content window size in seconds: `trim_end - trim_start`.
/// Used for internal looping calculations.
pub fn content_window(&self, clip_content: ClipDuration) -> ClipDuration {
let end = self.trim_end.map_or(clip_content.native(), |t| t.raw());
let window = (end - self.trim_start.raw()).max(0.0);
match clip_content {
ClipDuration::Beats(_) => ClipDuration::Beats(Beats(window)),
ClipDuration::Seconds(_) => ClipDuration::Seconds(Seconds(window)),
}
pub fn content_window_secs(&self, clip_duration_secs: f64) -> f64 {
let end = self.trim_end.unwrap_or(clip_duration_secs);
(end - self.trim_start).max(0.0)
}
/// How long this instance appears on the timeline, in **beats**.
///
/// If `timeline_duration` is set, returns that (enabling content looping). Otherwise the clip
/// occupies its content window — converted to beats *in the clip's own domain*:
///
/// - MIDI content is already beats and is tempo-invariant, so it carries over directly.
/// - Wall-clock content (audio/video/vector) is a seconds span, so it converts at the clip's
/// position on the timeline.
///
/// Taking a `ClipDuration` rather than a bare `Seconds` is what keeps those apart: this used to
/// take seconds and subtract `trim_start` from it, which for a TRIMMED MIDI clip subtracted a
/// beats offset from a seconds duration and got the clip's length wrong.
pub fn effective_duration_beats(&self, clip_content: ClipDuration, tempo_map: &crate::tempo_map::TempoMap) -> Beats {
/// If `timeline_duration` is set, returns that (enabling content looping).
/// Otherwise converts the content window from seconds to beats using the tempo map.
pub fn effective_duration_beats(&self, clip_duration_secs: f64, tempo_map: &crate::tempo_map::TempoMap) -> f64 {
if let Some(td) = self.timeline_duration {
return td;
}
match self.content_window(clip_content) {
ClipDuration::Beats(b) => b,
ClipDuration::Seconds(s) => {
let start_secs = tempo_map.beats_to_seconds(self.timeline_start);
tempo_map.seconds_to_beats(start_secs + s) - self.timeline_start
}
}
let window_secs = self.content_window_secs(clip_duration_secs);
let start_secs = tempo_map.transform(self.timeline_start);
tempo_map.inverse_transform(start_secs + window_secs) - self.timeline_start
}
/// Left edge of the clip's visual extent on the timeline, in **beats**.
pub fn effective_start(&self) -> Beats {
self.timeline_start - self.loop_before.unwrap_or(Beats::ZERO)
pub fn effective_start(&self) -> f64 {
self.timeline_start - self.loop_before.unwrap_or(0.0)
}
/// Total visual duration (loop_before + effective_duration), in **beats**.
pub fn total_duration(&self, clip_content: ClipDuration, tempo_map: &crate::tempo_map::TempoMap) -> Beats {
self.loop_before.unwrap_or(Beats::ZERO) + self.effective_duration_beats(clip_content, tempo_map)
pub fn total_duration(&self, clip_duration_secs: f64, tempo_map: &crate::tempo_map::TempoMap) -> f64 {
self.loop_before.unwrap_or(0.0) + self.effective_duration_beats(clip_duration_secs, tempo_map)
}
/// Map a playback time (in **seconds**) to clip-local content time (in **seconds**).
///
/// The trim bounds are resolved through `clip_content`'s domain first, so a MIDI clip's beats
/// trims are converted rather than read as seconds. Callers are the wall-clock consumers (video
/// seek, vector/raster rendering), which want seconds regardless of how the clip stores content.
///
/// Returns `None` if the clip instance is not active at `time`.
pub fn remap_time_secs(&self, time: Seconds, clip_content: ClipDuration, tempo_map: &crate::tempo_map::TempoMap) -> Option<Seconds> {
let start_secs = tempo_map.beats_to_seconds(self.timeline_start);
let dur_beats = self.effective_duration_beats(clip_content, tempo_map);
let end_secs = tempo_map.beats_to_seconds(self.timeline_start + dur_beats);
/// Returns `None` if the clip instance is not active at `time_secs`.
pub fn remap_time_secs(&self, time_secs: f64, clip_duration_secs: f64, tempo_map: &crate::tempo_map::TempoMap) -> Option<f64> {
let start_secs = tempo_map.transform(self.timeline_start);
let dur_beats = self.effective_duration_beats(clip_duration_secs, tempo_map);
let end_secs = tempo_map.transform(self.timeline_start + dur_beats);
if time < start_secs || time >= end_secs {
if time_secs < start_secs || time_secs >= end_secs {
return None;
}
let trim_start_secs = clip_content.same_domain(self.trim_start).to_seconds(tempo_map);
let content_time = (time - start_secs) * self.playback_speed;
let content_window = self.content_window(clip_content).to_seconds(tempo_map);
let relative_secs = time_secs - start_secs;
let content_time = relative_secs * self.playback_speed;
let content_window = self.content_window_secs(clip_duration_secs);
if content_window == Seconds::ZERO {
return Some(trim_start_secs);
if content_window == 0.0 {
return Some(self.trim_start);
}
let looped = if content_time > content_window {
let looped_time = if content_time > content_window {
content_time % content_window
} else {
content_time
};
Some(trim_start_secs + looped)
Some(self.trim_start + looped_time)
}
/// Alias for `remap_time_secs`.
#[inline]
pub fn remap_time(&self, time: Seconds, clip_content: ClipDuration, tempo_map: &crate::tempo_map::TempoMap) -> Option<Seconds> {
self.remap_time_secs(time, clip_content, tempo_map)
pub fn remap_time(&self, time_secs: f64, clip_duration_secs: f64, tempo_map: &crate::tempo_map::TempoMap) -> Option<f64> {
self.remap_time_secs(time_secs, clip_duration_secs, tempo_map)
}
/// Alias for `effective_duration_beats`.
#[inline]
pub fn effective_duration(&self, clip_content: ClipDuration, tempo_map: &crate::tempo_map::TempoMap) -> Beats {
self.effective_duration_beats(clip_content, tempo_map)
pub fn effective_duration(&self, clip_duration_secs: f64, tempo_map: &crate::tempo_map::TempoMap) -> f64 {
self.effective_duration_beats(clip_duration_secs, tempo_map)
}
/// Convert to affine transform
@ -1231,8 +959,8 @@ mod tests {
assert_eq!(instance.clip_id, clip_id);
assert_eq!(instance.opacity, 1.0);
assert_eq!(instance.timeline_start, Beats::ZERO);
assert_eq!(instance.trim_start, ContentTime::ZERO);
assert_eq!(instance.timeline_start, 0.0);
assert_eq!(instance.trim_start, 0.0);
assert_eq!(instance.trim_end, None);
assert_eq!(instance.playback_speed, 1.0);
assert_eq!(instance.gain, 1.0);
@ -1242,52 +970,27 @@ mod tests {
fn test_clip_instance_trimming() {
let clip_id = Uuid::new_v4();
let instance = ClipInstance::new(clip_id)
.with_trimming(ContentTime(2.0), Some(ContentTime(8.0)));
.with_trimming(2.0, Some(8.0));
assert_eq!(instance.trim_start, ContentTime(2.0));
assert_eq!(instance.trim_end, Some(ContentTime(8.0)));
assert_eq!(instance.trim_start, 2.0);
assert_eq!(instance.trim_end, Some(8.0));
// At 60 BPM the tempo map is identity (1 beat == 1 second), so the
// beats-domain effective duration equals the seconds content window.
let tempo_map = crate::tempo_map::TempoMap::constant(60.0);
let content = ClipDuration::Seconds(Seconds(10.0));
assert_eq!(instance.effective_duration(content, &tempo_map), Beats(6.0));
assert_eq!(instance.effective_duration(10.0, &tempo_map), 6.0);
}
#[test]
fn test_clip_instance_no_end_trim() {
let clip_id = Uuid::new_v4();
let instance = ClipInstance::new(clip_id)
.with_trimming(ContentTime(2.0), None);
.with_trimming(2.0, None);
assert_eq!(instance.trim_start, ContentTime(2.0));
assert_eq!(instance.trim_start, 2.0);
assert_eq!(instance.trim_end, None);
// At 60 BPM the tempo map is identity (1 beat == 1 second).
let tempo_map = crate::tempo_map::TempoMap::constant(60.0);
let content = ClipDuration::Seconds(Seconds(10.0));
assert_eq!(instance.effective_duration(content, &tempo_map), Beats(8.0));
}
#[test]
fn trimmed_midi_clip_keeps_its_beats_length_across_tempo() {
// Regression: `effective_duration_beats` used to take a SECONDS clip duration and subtract
// `trim_start` from it. For a TRIMMED MIDI clip that subtracted a beats offset from a
// seconds duration, so the clip's timeline length came out wrong at any tempo but 60 BPM.
//
// MIDI content is beats and tempo-invariant: a clip trimmed to beats 2..6 is 4 beats long
// whatever the tempo says.
let clip_id = Uuid::new_v4();
let instance = ClipInstance::new(clip_id)
.with_trimming(ContentTime(2.0), Some(ContentTime(6.0)));
let content = ClipDuration::Beats(Beats(8.0));
for bpm in [60.0, 120.0, 90.0] {
let tempo_map = crate::tempo_map::TempoMap::constant(bpm);
assert_eq!(
instance.effective_duration(content, &tempo_map),
Beats(4.0),
"a MIDI clip trimmed to beats 2..6 is 4 beats long at {bpm} BPM",
);
}
assert_eq!(instance.effective_duration(10.0, &tempo_map), 8.0);
}
#[test]
@ -1307,109 +1010,4 @@ mod tests {
assert_eq!(instance.playback_speed, 2.0);
assert_eq!(instance.gain, 0.8);
}
/// A clip whose own content is pool 10, plus an instance carrying takes over the given pools.
fn with_takes(pool_indices: &[usize]) -> (AudioClip, ClipInstance) {
let clip = AudioClip::new_sampled("Cycle rec", pool_indices[0], 2.0);
let mut instance = ClipInstance::new(clip.id);
instance.takes = pool_indices
.iter()
.enumerate()
.map(|(i, &audio_pool_index)| AudioTake {
name: format!("Take {}", i + 1),
content: TakeContent::Audio { audio_pool_index },
})
.collect();
instance.recorded_loop_beats = Some(Beats(8.0));
(clip, instance)
}
#[test]
fn active_take_selects_the_pool_file() {
let (clip, mut instance) = with_takes(&[10, 11, 12]);
instance.active_take = Some(0);
assert_eq!(instance.resolved_audio_pool_index(&clip), Some(10));
instance.active_take = Some(2);
assert_eq!(instance.resolved_audio_pool_index(&clip), Some(12));
// None means take 0.
instance.active_take = None;
assert_eq!(instance.resolved_audio_pool_index(&clip), Some(10));
}
#[test]
fn out_of_range_take_falls_back_to_the_first() {
// An index can go stale (an undo that shrank the list, an old .beam). Playing the first take
// beats playing nothing.
let (clip, mut instance) = with_takes(&[10, 11]);
instance.active_take = Some(99);
assert_eq!(instance.resolved_audio_pool_index(&clip), Some(10));
assert_eq!(instance.active_take_index(), 0);
}
#[test]
fn an_instance_without_takes_plays_the_clips_own_content() {
let clip = AudioClip::new_sampled("Plain", 42, 2.0);
let instance = ClipInstance::new(clip.id);
assert!(instance.takes.is_empty());
assert_eq!(instance.resolved_audio_pool_index(&clip), Some(42));
}
#[test]
fn midi_takes_resolve_to_midi_content() {
let clip = AudioClip::new_midi("Cycle rec", 7, Beats(4.0));
let mut instance = ClipInstance::new(clip.id);
instance.takes = vec![AudioTake {
name: "Take 1".into(),
content: TakeContent::Midi { midi_clip_id: 7 },
}];
assert_eq!(clip.content_duration(), ClipDuration::Beats(Beats(4.0)));
assert_eq!(instance.resolved_midi_clip_id(&clip), Some(7));
assert_eq!(instance.resolved_audio_pool_index(&clip), None);
}
#[test]
fn splitting_gives_each_half_an_independent_take_list() {
// Takes live on the INSTANCE, so a split (which clones the instance) hands each half its own
// list. Two consequences, both wanted: the halves can select different takes (comping), and
// deleting a take from one leaves the other alone.
let (clip, left_src) = with_takes(&[10, 11, 12]);
let mut left = left_src.clone();
let mut right = left_src.clone();
right.id = Uuid::new_v4();
left.active_take = Some(0);
right.active_take = Some(2);
assert_eq!(left.resolved_audio_pool_index(&clip), Some(10));
assert_eq!(right.resolved_audio_pool_index(&clip), Some(12));
// Delete take 2 (pool 11) from the left half only.
left.takes.remove(1);
assert_eq!(left.takes.len(), 2);
assert_eq!(right.takes.len(), 3, "the other half keeps its own takes");
assert_eq!(
right.resolved_audio_pool_index(&clip),
Some(12),
"and its selection still points where it did",
);
}
#[test]
fn clip_instance_without_active_take_deserializes() {
// Back-compat: .beam files written before take folders have no `active_take` field.
let json = r#"{
"id": "550e8400-e29b-41d4-a716-446655440000",
"clip_id": "550e8400-e29b-41d4-a716-446655440001",
"transform": {"x": 0.0, "y": 0.0, "rotation": 0.0, "scale_x": 1.0, "scale_y": 1.0, "skew_x": 0.0, "skew_y": 0.0},
"opacity": 1.0,
"name": null,
"timeline_start": 0.0,
"timeline_duration": null,
"trim_start": 0.0,
"trim_end": null,
"playback_speed": 1.0,
"gain": 1.0
}"#;
let instance: ClipInstance = serde_json::from_str(json).expect("old instances must load");
assert_eq!(instance.active_take, None);
}
}

View File

@ -5,7 +5,6 @@
use crate::asset_folder::AssetFolderTree;
use crate::clip::{AudioClip, ClipInstance, ImageAsset, VideoClip, VectorClip};
use daw_backend::{Beats, Seconds};
use crate::effect::EffectDefinition;
use crate::layer::{AnyLayer, GroupLayer};
use crate::script::ScriptDefinition;
@ -201,17 +200,6 @@ pub struct Document {
#[serde(default)]
pub time_signature: TimeSignature,
/// Transport cycle (loop) region, as `(start, end)` in **beats**.
///
/// Authored in beats so it stays put musically when the tempo changes. `None` = no region set.
/// Saved with the project; `#[serde(default)]` keeps older `.beam` files loading.
#[serde(default, skip_serializing_if = "Option::is_none")]
pub cycle_region: Option<(Beats, Beats)>,
/// Whether the transport loops over `cycle_region`.
#[serde(default)]
pub cycle_enabled: bool,
/// Master track (master bus + tempo automation lane).
/// Stored separately from the root layer tree; shown in timeline when
/// `show_master_track` is enabled in the editor state.
@ -308,8 +296,6 @@ impl Default for Document {
height: 1080.0,
framerate: 60.0,
time_signature: TimeSignature::default(),
cycle_region: None,
cycle_enabled: false,
master_layer: {
let mut ml = GroupLayer::new_master(120.0);
ml.layer.id = uuid::Uuid::new_v4();
@ -452,27 +438,20 @@ impl Document {
/// Returns the end time of the last clip instance across all layers,
/// or the document's duration if no clips are found.
pub fn calculate_timeline_endpoint(&self) -> f64 {
let tempo_map = self.tempo_map();
// Accumulated in **beats** (as f64, to keep the recursive helper's `Fn(_, f64) -> f64`
// signature); converted to seconds once at the return.
let mut max_end_time: f64 = 0.0;
// End position of a clip instance, in **beats**. Its trimmed content window is in seconds
// (scaled by playback speed), so convert that seconds span to beats via the tempo map — the
// old code added a seconds duration straight onto the beats start.
// Helper function to calculate the end time of a clip instance
let calculate_instance_end = |instance: &ClipInstance, clip_duration: f64| -> f64 {
let end_beats: Beats = if let Some(timeline_duration) = instance.timeline_duration {
instance.timeline_start + timeline_duration
let effective_duration = if let Some(timeline_duration) = instance.timeline_duration {
// Explicit timeline duration set (may include looping)
timeline_duration
} else {
// `clip_duration` arrives as seconds (the recursive helper's signature), so this
// path is the wall-clock one; MIDI content would need resolving against its clip.
let trim_end = instance.trim_end.map_or(clip_duration, |t| t.raw());
let trimmed_secs =
((trim_end - instance.trim_start.raw()) / instance.playback_speed).max(0.0);
let start_secs = tempo_map.beats_to_seconds(instance.timeline_start);
tempo_map.seconds_to_beats(start_secs + Seconds(trimmed_secs))
// Calculate from trim points
let trim_end = instance.trim_end.unwrap_or(clip_duration);
let trimmed_duration = trim_end - instance.trim_start;
trimmed_duration / instance.playback_speed // Adjust for playback speed
};
end_beats.beats_to_f64()
instance.timeline_start + effective_duration
};
// Iterate through all layers to find the maximum end time
@ -488,10 +467,8 @@ impl Document {
}
crate::layer::AnyLayer::Audio(audio_layer) => {
for instance in &audio_layer.clip_instances {
// get_clip_duration yields seconds (converting MIDI's beats duration),
// which is what the closure expects.
if let Some(clip_duration) = self.get_clip_duration(&instance.clip_id) {
let end_time = calculate_instance_end(instance, clip_duration.seconds_to_f64());
if let Some(clip) = self.audio_clips.get(&instance.clip_id) {
let end_time = calculate_instance_end(instance, clip.duration);
max_end_time = max_end_time.max(end_time);
}
}
@ -507,7 +484,7 @@ impl Document {
crate::layer::AnyLayer::Effect(effect_layer) => {
for instance in &effect_layer.clip_instances {
if let Some(clip_duration) = self.get_clip_duration(&instance.clip_id) {
let end_time = calculate_instance_end(instance, clip_duration.seconds_to_f64());
let end_time = calculate_instance_end(instance, clip_duration);
max_end_time = max_end_time.max(end_time);
}
}
@ -534,8 +511,8 @@ impl Document {
}
crate::layer::AnyLayer::Audio(al) => {
for inst in &al.clip_instances {
if let Some(clip_duration) = doc.get_clip_duration(&inst.clip_id) {
*max_end = max_end.max(calc_end(inst, clip_duration.seconds_to_f64()));
if let Some(clip) = doc.audio_clips.get(&inst.clip_id) {
*max_end = max_end.max(calc_end(inst, clip.duration));
}
}
}
@ -549,7 +526,7 @@ impl Document {
crate::layer::AnyLayer::Effect(el) => {
for inst in &el.clip_instances {
if let Some(dur) = doc.get_clip_duration(&inst.clip_id) {
*max_end = max_end.max(calc_end(inst, dur.seconds_to_f64()));
*max_end = max_end.max(calc_end(inst, dur));
}
}
}
@ -567,10 +544,9 @@ impl Document {
}
}
// Return the max end (converting the beats accumulator to seconds), or the document
// duration (already seconds) if no clips were found.
// Return the maximum end time, or document duration if no clips found
if max_end_time > 0.0 {
tempo_map.beats_to_seconds(Beats(max_end_time)).seconds_to_f64()
max_end_time
} else {
self.duration
}
@ -783,18 +759,16 @@ impl Document {
}
/// Find the document audio clip (UUID + ref) that owns the given backend pool index.
/// A take folder owns one pool file per take, so any of them maps back to the folder.
pub fn audio_clip_by_pool_index(&self, pool_index: usize) -> Option<(Uuid, &AudioClip)> {
self.audio_clips.iter()
.find(|(_, c)| c.owns_audio_pool_index(pool_index))
.find(|(_, c)| c.audio_pool_index() == Some(pool_index))
.map(|(&id, c)| (id, c))
}
/// Find the document audio clip (UUID + ref) that owns the given backend MIDI clip ID.
/// As above, a take folder owns one MIDI clip per take.
pub fn audio_clip_by_midi_clip_id(&self, midi_clip_id: u32) -> Option<(Uuid, &AudioClip)> {
self.audio_clips.iter()
.find(|(_, c)| c.owns_midi_clip_id(midi_clip_id))
.find(|(_, c)| c.midi_clip_id() == Some(midi_clip_id))
.map(|(&id, c)| (id, c))
}
@ -916,94 +890,19 @@ impl Document {
/// Searches through all clip libraries to find the clip and return its duration.
/// For effect definitions, returns `EFFECT_DURATION` (f64::MAX) since effects
/// have infinite internal duration.
/// A clip's content duration **in the domain its `trim_start`/`trim_end` are measured in**.
///
/// Content time is domain-polymorphic exactly like `AudioClip::duration`: SECONDS for sampled
/// audio, video and vector, but BEATS for MIDI. Anything doing arithmetic against a trim value —
/// mapping a timeline position into the clip's content, say — has to work in that same domain,
/// and [`Self::get_clip_duration`] can't tell it which: that one always converts to seconds.
///
/// Returns `None` for unknown clips.
pub fn clip_trim_duration(&self, clip_id: &Uuid) -> Option<crate::clip::ClipDuration> {
if let Some(clip) = self.audio_clips.get(clip_id) {
return Some(clip.content_duration());
}
// Everything else measures its content in wall-clock seconds.
self.get_clip_duration(clip_id).map(crate::clip::ClipDuration::Seconds)
}
/// Find an existing take folder on `layer_id` that a new cycle recording should be appended to.
///
/// Cycle-recording over a region that already holds a take folder should ADD to that folder
/// rather than drop a second clip on top of it — otherwise the new takes are stranded in a
/// separate, overlapping clip and can't be auditioned against the ones already there.
///
/// "The same region" means the instance starts at `loop_start` and its folder was recorded
/// against the same loop length. Matching the length too means resizing the cycle region starts
/// a fresh folder rather than appending takes of a different length to an existing one, which
/// would break the uniform-take invariant comping depends on.
///
/// `exclude` is the in-progress recording's own instance, which is on the layer but isn't a
/// candidate. Returns the instance id.
pub fn take_folder_at(
&self,
layer_id: &Uuid,
loop_start: Beats,
loop_len: Beats,
exclude: &Uuid,
) -> Option<Uuid> {
let Some(AnyLayer::Audio(audio_layer)) = self.get_layer(layer_id) else {
return None;
};
const EPS: f64 = 1e-6;
audio_layer.clip_instances.iter().find_map(|ci| {
if ci.id == *exclude
|| ci.takes.is_empty()
|| (ci.timeline_start - loop_start).beats_to_f64().abs() > EPS
{
return None;
}
let recorded = ci.recorded_loop_beats?;
((recorded - loop_len).beats_to_f64().abs() < EPS).then_some(ci.id)
})
}
/// Resolve a [`ContentTime`] (a trim bound) against the clip it belongs to.
///
/// The clip is the only thing that knows whether its content is measured in seconds or beats, so
/// this is the sanctioned exit from `ContentTime`. Works for every clip kind, not just audio.
/// Returns `None` for unknown clips.
pub fn resolve_content_time(
&self,
clip_id: &Uuid,
t: daw_backend::ContentTime,
) -> Option<crate::clip::ClipDuration> {
if let Some(clip) = self.audio_clips.get(clip_id) {
return Some(clip.resolve_content_time(t));
}
if self.vector_clips.contains_key(clip_id)
|| self.video_clips.contains_key(clip_id)
|| self.effect_definitions.contains_key(clip_id)
{
// Wall-clock content.
return Some(crate::clip::ClipDuration::Seconds(Seconds(t.raw())));
}
None
}
pub fn get_clip_duration(&self, clip_id: &Uuid) -> Option<Seconds> {
pub fn get_clip_duration(&self, clip_id: &Uuid) -> Option<f64> {
if let Some(clip) = self.vector_clips.get(clip_id) {
if clip.is_group {
Some(Seconds(clip.duration))
Some(clip.duration)
} else {
let tempo_map = self.tempo_map();
Some(Seconds(clip.content_duration_with(self.framerate, tempo_map, |id| {
Some(clip.content_duration_with(self.framerate, tempo_map, |id| {
// Resolve nested clip durations (audio, video, other vector clips)
if let Some(vc) = self.vector_clips.get(id) {
// Avoid deep recursion — use stored duration for nested vector clips
Some(vc.content_duration(self.framerate, tempo_map))
} else if let Some(ac) = self.audio_clips.get(id) {
Some(ac.content_duration().to_seconds(tempo_map).seconds_to_f64())
Some(ac.duration)
} else if let Some(vc) = self.video_clips.get(id) {
Some(vc.duration)
} else if self.effect_definitions.contains_key(id) {
@ -1011,25 +910,23 @@ impl Document {
} else {
None
}
})))
}))
}
} else if let Some(clip) = self.video_clips.get(clip_id) {
Some(Seconds(clip.duration))
Some(clip.duration)
} else if let Some(clip) = self.audio_clips.get(clip_id) {
// Interpret the clip's native-domain duration as wall-clock seconds (MIDI stores
// beats, sampled stores seconds — content_duration keeps that straight).
Some(clip.content_duration().to_seconds(self.tempo_map()))
Some(clip.duration)
} else if self.effect_definitions.contains_key(clip_id) {
// Effects have infinite internal duration - their timeline length
// is controlled by ClipInstance.trim_end
Some(Seconds(crate::effect::EFFECT_DURATION))
Some(crate::effect::EFFECT_DURATION)
} else {
None
}
}
/// Calculate the end position of a clip instance on the timeline, in **beats**.
pub fn get_clip_instance_end_time(&self, layer_id: &Uuid, instance_id: &Uuid) -> Option<Beats> {
/// Calculate the end time of a clip instance on the timeline
pub fn get_clip_instance_end_time(&self, layer_id: &Uuid, instance_id: &Uuid) -> Option<f64> {
let layer = self.get_layer(layer_id)?;
// Find the clip instance
@ -1044,9 +941,13 @@ impl Document {
};
let instance = instances.iter().find(|inst| &inst.id == instance_id)?;
// The clip's content duration in ITS OWN domain, so the trims resolve correctly for MIDI.
let clip_content = self.clip_trim_duration(&instance.clip_id)?;
Some(instance.timeline_start + instance.effective_duration_beats(clip_content, self.tempo_map()))
let clip_duration = self.get_clip_duration(&instance.clip_id)?;
let trim_start = instance.trim_start;
let trim_end = instance.trim_end.unwrap_or(clip_duration);
let effective_duration = trim_end - trim_start;
Some(instance.timeline_start + effective_duration)
}
/// Check if a time range overlaps with any existing clip on the layer
@ -1057,8 +958,8 @@ impl Document {
pub fn check_overlap_on_layer(
&self,
layer_id: &Uuid,
start_time: Beats,
end_time: Beats,
start_time: f64,
end_time: f64,
exclude_instance_ids: &[Uuid],
) -> (bool, Option<Uuid>) {
let Some(layer) = self.get_layer(layer_id) else {
@ -1086,14 +987,13 @@ impl Document {
continue;
}
// Calculate instance extent (accounting for loop_before). Content duration in the clip's
// own domain, so the trims resolve correctly for MIDI.
let Some(clip_content) = self.clip_trim_duration(&instance.clip_id) else {
// Calculate instance extent (accounting for loop_before)
let Some(clip_duration) = self.get_clip_duration(&instance.clip_id) else {
continue;
};
let instance_start = instance.effective_start();
let instance_end = instance.timeline_start + instance.effective_duration(clip_content, self.tempo_map());
let instance_end = instance.timeline_start + instance.effective_duration(clip_duration, self.tempo_map());
// Check overlap: start_a < end_b AND start_b < end_a
if start_time < instance_end && instance_start < end_time {
@ -1112,14 +1012,14 @@ impl Document {
pub fn find_nearest_valid_position(
&self,
layer_id: &Uuid,
desired_start: Beats,
clip_duration: Beats,
desired_start: f64,
clip_duration: f64,
exclude_instance_ids: &[Uuid],
) -> Option<Beats> {
) -> Option<f64> {
let layer = self.get_layer(layer_id)?;
// Clamp to timeline start (can't go before 0)
let desired_start = desired_start.max(Beats::ZERO);
let desired_start = desired_start.max(0.0);
// Vector layers don't need overlap adjustment, but still respect timeline start
if matches!(layer, AnyLayer::Vector(_) | AnyLayer::Group(_)) {
@ -1144,13 +1044,13 @@ impl Document {
AnyLayer::Text(_) => return Some(desired_start), // Text layers don't have own clips
};
let mut occupied_ranges: Vec<(Beats, Beats, Uuid)> = Vec::new();
let mut occupied_ranges: Vec<(f64, f64, Uuid)> = Vec::new();
for instance in instances {
if exclude_instance_ids.contains(&instance.id) {
continue;
}
if let Some(clip_dur) = self.clip_trim_duration(&instance.clip_id) {
if let Some(clip_dur) = self.get_clip_duration(&instance.clip_id) {
let inst_start = instance.effective_start();
let inst_end = instance.timeline_start + instance.effective_duration(clip_dur, self.tempo_map());
occupied_ranges.push((inst_start, inst_end, instance.id));
@ -1163,7 +1063,7 @@ impl Document {
// Find the clip we're overlapping with and try both sides, pick nearest
for (occupied_start, occupied_end, _) in &occupied_ranges {
if desired_start < *occupied_end && *occupied_start < desired_end {
let mut candidates: Vec<Beats> = Vec::new();
let mut candidates: Vec<f64> = Vec::new();
// Try snapping to the right (after this clip)
let snap_right = *occupied_end;
@ -1179,8 +1079,8 @@ impl Document {
}
// Try snapping to the left (before this clip)
let snap_left = *occupied_start - clip_duration;
if snap_left >= Beats::ZERO {
let snap_left = occupied_start - clip_duration;
if snap_left >= 0.0 {
let (overlaps_left, _) = self.check_overlap_on_layer(
layer_id,
snap_left,
@ -1195,8 +1095,8 @@ impl Document {
// Pick the candidate closest to desired_start
if !candidates.is_empty() {
candidates.sort_by(|a, b| {
let dist_a = (*a - desired_start).abs();
let dist_b = (*b - desired_start).abs();
let dist_a = (a - desired_start).abs();
let dist_b = (b - desired_start).abs();
dist_a.partial_cmp(&dist_b).unwrap_or(std::cmp::Ordering::Equal)
});
return Some(candidates[0]);
@ -1206,7 +1106,7 @@ impl Document {
// If no gap found, try placing at timeline start
if occupied_ranges.is_empty() || occupied_ranges[0].0 >= clip_duration {
return Some(Beats::ZERO);
return Some(0.0);
}
// No valid position found
@ -1218,9 +1118,9 @@ impl Document {
pub fn clamp_group_move_offset(
&self,
layer_id: &Uuid,
group: &[(Uuid, Beats, Beats)], // (instance_id, timeline_start, effective_duration) in beats
desired_offset: Beats,
) -> Beats {
group: &[(Uuid, f64, f64)], // (instance_id, timeline_start, effective_duration)
desired_offset: f64,
) -> f64 {
let Some(layer) = self.get_layer(layer_id) else {
return desired_offset;
};
@ -1240,13 +1140,13 @@ impl Document {
AnyLayer::Text(_) => &[],
};
// Collect non-group clip ranges (beats)
let mut non_group: Vec<(Beats, Beats)> = Vec::new();
// Collect non-group clip ranges
let mut non_group: Vec<(f64, f64)> = Vec::new();
for inst in instances {
if group_ids.contains(&inst.id) {
continue;
}
if let Some(dur) = self.clip_trim_duration(&inst.clip_id) {
if let Some(dur) = self.get_clip_duration(&inst.clip_id) {
let start = inst.effective_start();
let end = inst.timeline_start + inst.effective_duration(dur, self.tempo_map());
non_group.push((start, end));
@ -1263,12 +1163,12 @@ impl Document {
// Check against non-group clips
for &(ns, ne) in &non_group {
if clamped < Beats::ZERO {
if clamped < 0.0 {
// Moving left: if non-group clip end is between our destination and current start
if ne <= start && ne > start + clamped {
clamped = clamped.max(ne - start);
}
} else if clamped > Beats::ZERO {
} else if clamped > 0.0 {
// Moving right: if non-group clip start is between our current end and destination
if ns >= end && ns < end + clamped {
clamped = clamped.min(ns - end);
@ -1284,25 +1184,23 @@ impl Document {
///
/// Returns the distance to the nearest clip to the left, or the distance to
/// timeline start (0.0) if no clips exist to the left.
/// Returns the max leftward trim extension as a content-seconds span (the wall-clock
/// length of the timeline gap to the previous clip); the trim domain is seconds.
pub fn find_max_trim_extend_left(
&self,
layer_id: &Uuid,
instance_id: &Uuid,
current_timeline_start: Beats,
) -> Seconds {
current_timeline_start: f64,
) -> f64 {
let Some(layer) = self.get_layer(layer_id) else {
return self.tempo_map().beats_to_seconds(current_timeline_start); // No limit if layer not found
return current_timeline_start; // No limit if layer not found
};
// Only check audio, video, and effect layers
if matches!(layer, AnyLayer::Vector(_) | AnyLayer::Group(_)) {
return self.tempo_map().beats_to_seconds(current_timeline_start); // No limit for vector/group layers
return current_timeline_start; // No limit for vector/group layers
};
// Find the nearest clip to the left
let mut nearest_end = Beats::ZERO; // Can extend to timeline start by default
let mut nearest_end = 0.0; // Can extend to timeline start by default
let instances: &[ClipInstance] = match layer {
AnyLayer::Audio(audio) => &audio.clip_instances,
@ -1320,7 +1218,7 @@ impl Document {
}
// Calculate other clip's extent (accounting for loop_before)
if let Some(clip_duration) = self.clip_trim_duration(&other.clip_id) {
if let Some(clip_duration) = self.get_clip_duration(&other.clip_id) {
let other_end = other.timeline_start + other.effective_duration(clip_duration, self.tempo_map());
// If this clip is to the left and closer than current nearest
@ -1330,27 +1228,27 @@ impl Document {
}
}
self.tempo_map().beats_to_seconds(current_timeline_start) - self.tempo_map().beats_to_seconds(nearest_end)
current_timeline_start - nearest_end
}
/// Find the maximum amount we can extend a clip to the right without overlapping.
/// Find the maximum amount we can extend a clip to the right without overlapping
///
/// Returns the content-seconds span of the timeline gap to the nearest clip on the
/// right, or Seconds(f64::MAX) if none. `current_effective_duration` is beats (timeline).
/// Returns the distance to the nearest clip to the right, or f64::MAX if no
/// clips exist to the right.
pub fn find_max_trim_extend_right(
&self,
layer_id: &Uuid,
instance_id: &Uuid,
current_timeline_start: Beats,
current_effective_duration: Beats,
) -> Seconds {
current_timeline_start: f64,
current_effective_duration: f64,
) -> f64 {
let Some(layer) = self.get_layer(layer_id) else {
return Seconds(f64::MAX); // No limit if layer not found
return f64::MAX; // No limit if layer not found
};
// Only check audio, video, and effect layers
if matches!(layer, AnyLayer::Vector(_) | AnyLayer::Group(_)) {
return Seconds(f64::MAX); // No limit for vector/group layers
return f64::MAX; // No limit for vector/group layers
}
let instances: &[ClipInstance] = match layer {
@ -1363,7 +1261,7 @@ impl Document {
AnyLayer::Text(_) => &[],
};
let mut nearest_start = Beats(f64::MAX);
let mut nearest_start = f64::MAX;
let current_end = current_timeline_start + current_effective_duration;
for other in instances {
@ -1378,11 +1276,10 @@ impl Document {
}
}
if nearest_start == Beats(f64::MAX) {
Seconds(f64::MAX) // No clip to the right, can extend freely
if nearest_start == f64::MAX {
f64::MAX // No clip to the right, can extend freely
} else {
// Gap between our end and next clip's start, as content seconds.
(self.tempo_map().beats_to_seconds(nearest_start) - self.tempo_map().beats_to_seconds(current_end)).max(Seconds::ZERO)
(nearest_start - current_end).max(0.0) // Gap between our end and next clip's start
}
}
/// Find the maximum amount we can extend loop_before to the left without overlapping.
@ -1392,8 +1289,8 @@ impl Document {
&self,
layer_id: &Uuid,
instance_id: &Uuid,
current_effective_start: Beats,
) -> Beats {
current_effective_start: f64,
) -> f64 {
let Some(layer) = self.get_layer(layer_id) else {
return current_effective_start;
};
@ -1412,14 +1309,14 @@ impl Document {
AnyLayer::Text(_) => &[],
};
let mut nearest_end = Beats::ZERO;
let mut nearest_end = 0.0;
for other in instances {
if &other.id == instance_id {
continue;
}
if let Some(clip_duration) = self.clip_trim_duration(&other.clip_id) {
if let Some(clip_duration) = self.get_clip_duration(&other.clip_id) {
let other_end = other.timeline_start + other.effective_duration(clip_duration, self.tempo_map());
if other_end <= current_effective_start && other_end > nearest_end {

View File

@ -351,7 +351,7 @@ impl EffectDefinition {
///
/// * `timeline_start` - When the effect starts on the timeline (seconds)
/// * `duration` - How long the effect appears on the timeline (seconds)
pub fn create_instance(&self, timeline_start: daw_backend::Beats, duration: daw_backend::Beats) -> ClipInstance {
pub fn create_instance(&self, timeline_start: f64, duration: f64) -> ClipInstance {
ClipInstance::new(self.id)
.with_timeline_start(timeline_start)
.with_timeline_duration(duration)

View File

@ -148,16 +148,11 @@ impl EffectLayer {
/// `timeline_start` (beats) to seconds for comparison.
pub fn active_clip_instances_at(&self, time_secs: f64, tempo_map: &crate::tempo_map::TempoMap) -> Vec<&ClipInstance> {
use crate::effect::EFFECT_DURATION;
let time_beats = tempo_map.seconds_to_beats(daw_backend::Seconds(time_secs));
let time_beats = tempo_map.inverse_transform(time_secs);
self.clip_instances
.iter()
.filter(|e| {
// Effects have an "infinite" wall-clock content length.
let end = e.timeline_start
+ e.effective_duration(
crate::clip::ClipDuration::Seconds(daw_backend::Seconds(EFFECT_DURATION)),
tempo_map,
);
let end = e.timeline_start + e.effective_duration(EFFECT_DURATION, tempo_map);
time_beats >= e.timeline_start && time_beats < end
})
.collect()
@ -223,7 +218,7 @@ mod tests {
fn test_add_effect() {
let mut layer = EffectLayer::new("Effects");
let def = create_test_effect_def();
let effect = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let effect = def.create_instance(0.0, 10.0);
let effect_id = effect.id;
let id = layer.add_clip_instance(effect);
@ -238,11 +233,11 @@ mod tests {
let def = create_test_effect_def();
// Effect 1: active from 0 to 5
let effect1 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(5.0));
let effect1 = def.create_instance(0.0, 5.0);
layer.add_clip_instance(effect1);
// Effect 2: active from 3 to 10
let effect2 = def.create_instance(daw_backend::Beats(3.0), daw_backend::Beats(7.0)); // 3.0 + 7.0 = 10.0 end
let effect2 = def.create_instance(3.0, 7.0); // 3.0 + 7.0 = 10.0 end
layer.add_clip_instance(effect2);
// At 60 BPM the tempo map is identity (1 beat == 1 second), so the
@ -264,11 +259,11 @@ mod tests {
let mut layer = EffectLayer::new("Effects");
let def = create_test_effect_def();
let effect1 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let effect1 = def.create_instance(0.0, 10.0);
let id1 = effect1.id;
layer.add_clip_instance(effect1);
let effect2 = def.create_instance(daw_backend::Beats(0.0), daw_backend::Beats(10.0));
let effect2 = def.create_instance(0.0, 10.0);
let id2 = effect2.id;
layer.add_clip_instance(effect2);

View File

@ -51,54 +51,6 @@ impl AudioFormat {
}
}
/// Optional tag metadata written into the exported audio file. Empty fields are omitted. FFmpeg
/// maps these standard keys to each container's native tags: ID3v2 (MP3), iTunes/MP4 atoms (M4A),
/// Vorbis comments (FLAC), and RIFF INFO (WAV).
#[derive(Debug, Clone, Default, Serialize, Deserialize)]
pub struct AudioMetadata {
pub title: String,
pub artist: String,
pub album: String,
pub genre: String,
pub comment: String,
/// Year or full date (written to the `date` tag).
pub year: String,
/// Track number (written to the `track` tag).
pub track: String,
}
impl AudioMetadata {
/// True when no field is set (so no metadata need be written).
pub fn is_empty(&self) -> bool {
self.title.is_empty()
&& self.artist.is_empty()
&& self.album.is_empty()
&& self.genre.is_empty()
&& self.comment.is_empty()
&& self.year.is_empty()
&& self.track.is_empty()
}
/// The set (ffmpeg-key, value) pairs for non-empty fields, in a stable order.
pub fn pairs(&self) -> Vec<(&'static str, &str)> {
let mut v = Vec::new();
for (key, val) in [
("title", &self.title),
("artist", &self.artist),
("album", &self.album),
("genre", &self.genre),
("comment", &self.comment),
("date", &self.year),
("track", &self.track),
] {
if !val.is_empty() {
v.push((key, val.as_str()));
}
}
v
}
}
/// Audio export settings
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AudioExportSettings {
@ -127,10 +79,6 @@ pub struct AudioExportSettings {
/// Project BPM (for beat-position scheduling during export)
pub bpm: f64,
/// Tag metadata (title/artist/…) written into the file. Empty = none.
#[serde(default)]
pub metadata: AudioMetadata,
}
impl Default for AudioExportSettings {
@ -144,7 +92,6 @@ impl Default for AudioExportSettings {
start_time: 0.0,
end_time: 60.0,
bpm: 120.0,
metadata: AudioMetadata::default(),
}
}
}
@ -596,82 +543,6 @@ impl ImageExportSettings {
}
}
// ── Animated GIF export ──────────────────────────────────────────────────────
/// Settings for exporting an animated GIF (multi-frame, palette-quantized, no audio).
///
/// GIF stores a per-frame delay in centiseconds (1/100 s), so effective frame rate is quantized to
/// whole centiseconds — [`Self::frame_delay_ms`] rounds accordingly and the dialog offers sensible
/// GIF rates. Each frame is quantized to a 256-color palette by the encoder.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct GifExportSettings {
/// Output width in pixels (None = use document width).
pub width: Option<u32>,
/// Output height in pixels (None = use document height).
pub height: Option<u32>,
/// Frame rate (fps). Snapped to whole-centisecond delays at encode time.
pub framerate: f64,
/// Loop the animation forever (GIF `NETSCAPE2.0` infinite loop). False = play once.
pub loop_forever: bool,
/// Preserve full alpha as GIF 1-bit transparency (pixels below the alpha threshold become the
/// transparent index). When false, frames are composited onto an opaque background first.
pub transparency: bool,
/// How the document is fit into the output frame when aspect ratios differ (default Letterbox).
#[serde(default)]
pub fit: ExportFitMode,
/// Start time in seconds.
pub start_time: f64,
/// End time in seconds.
pub end_time: f64,
}
impl Default for GifExportSettings {
fn default() -> Self {
Self {
width: None,
height: None,
framerate: 15.0,
loop_forever: true,
transparency: false,
fit: ExportFitMode::Letterbox,
start_time: 0.0,
end_time: 5.0,
}
}
}
impl GifExportSettings {
pub fn validate(&self) -> Result<(), String> {
if let Some(w) = self.width { if w == 0 { return Err("Width must be > 0".into()); } }
if let Some(h) = self.height { if h == 0 { return Err("Height must be > 0".into()); } }
if self.framerate <= 0.0 {
return Err("Framerate must be greater than 0".into());
}
if self.start_time < 0.0 {
return Err("Start time cannot be negative".into());
}
if self.end_time <= self.start_time {
return Err("End time must be greater than start time".into());
}
Ok(())
}
/// Duration in seconds.
pub fn duration(&self) -> f64 { self.end_time - self.start_time }
/// Total number of frames to render.
pub fn total_frames(&self) -> usize {
(self.duration() * self.framerate).ceil().max(1.0) as usize
}
/// Per-frame delay in milliseconds, from the framerate (GIF stores this at centisecond
/// resolution, so the effective rate is snapped to the nearest 10 ms, min 10 ms).
pub fn frame_delay_ms(&self) -> u32 {
let ms = 1000.0 / self.framerate;
((ms / 10.0).round().max(1.0) * 10.0) as u32
}
}
/// Progress updates during export
#[derive(Debug, Clone)]
pub enum ExportProgress {
@ -857,33 +728,6 @@ mod tests {
assert_eq!(settings.total_frames(), 300);
}
#[test]
fn test_gif_frame_delay_and_frames() {
// Frame rates that map to clean centisecond delays.
let mk = |fps: f64| GifExportSettings { framerate: fps, ..Default::default() };
assert_eq!(mk(10.0).frame_delay_ms(), 100); // 100 ms
assert_eq!(mk(20.0).frame_delay_ms(), 50); // 50 ms
assert_eq!(mk(50.0).frame_delay_ms(), 20); // 20 ms
// 15 fps = 66.6 ms rounds to 70 ms (7 cs); 25 fps = 40 ms.
assert_eq!(mk(15.0).frame_delay_ms(), 70);
assert_eq!(mk(25.0).frame_delay_ms(), 40);
// Very high fps clamps to the 10 ms minimum (1 cs).
assert_eq!(mk(1000.0).frame_delay_ms(), 10);
let settings = GifExportSettings { framerate: 20.0, start_time: 0.0, end_time: 3.0, ..Default::default() };
assert_eq!(settings.total_frames(), 60);
}
#[test]
fn test_gif_validate() {
let mut s = GifExportSettings::default();
assert!(s.validate().is_ok());
s.framerate = 0.0;
assert!(s.validate().is_err());
s = GifExportSettings { start_time: 5.0, end_time: 2.0, ..Default::default() };
assert!(s.validate().is_err());
}
#[test]
fn test_export_progress_percentage() {
let progress = ExportProgress::FrameRendered { frame: 50, total: 100 };

View File

@ -399,29 +399,16 @@ pub fn save_beam(
}
// --- raster keyframes -> media rows (PNG), keyed by keyframe id ---
// Incremental: only (re)encode a keyframe whose pixels changed since the last save.
// `kf.dirty` means "current pixels are not yet in the container" (set on any edit,
// cleared on a successful save — see main.rs); a clean frame already stored is kept
// in place, skipping the PNG re-encode of every resident frame on every save.
// (Phase 0 writes all resident frames each save; a disk-dirty flag to skip
// unchanged frames in place is deferred to Phase 3.)
// Walk ALL layers (incl. nested in groups/clips) so nested raster keyframes
// are persisted too, and so `live_media` covers them — matching the load path,
// which arms `needs_fault_in` recursively. Top-level-only projects are unaffected.
let mut raster_count = 0usize;
let mut raster_skipped = 0usize;
for layer in document.all_layers() {
if let crate::layer::AnyLayer::Raster(rl) = layer {
for kf in &rl.keyframes {
if !kf.raw_pixels.is_empty() {
// Clean + already stored → keep the existing full + proxy rows untouched.
if !kf.dirty && txn.media_exists(kf.id)? {
live_media.insert(kf.id);
let proxy_id = raster_proxy_media_id(kf.id);
if txn.media_exists(proxy_id)? {
live_media.insert(proxy_id);
}
raster_skipped += 1;
continue;
}
let img =
crate::brush_engine::image_from_raw(kf.raw_pixels.clone(), kf.width, kf.height);
match crate::brush_engine::encode_png(&img) {
@ -621,10 +608,9 @@ pub fn save_beam(
}
eprintln!(
"📊 [SAVE_BEAM] ✅ Saved {} audio + {} raster media ({} unchanged frames skipped), {} orphans removed, in {:.2}ms",
"📊 [SAVE_BEAM] ✅ Saved {} audio + {} raster media, {} orphans removed, in {:.2}ms",
audio_pool_entries.len(),
raster_count,
raster_skipped,
removed,
fn_start.elapsed().as_secs_f64() * 1000.0
);

View File

@ -3,7 +3,7 @@
//! Provides functions for testing if points or rectangles intersect with
//! vector graph elements and clip instances, taking into account transform hierarchies.
use crate::clip::{ClipDuration, ClipInstance};
use crate::clip::ClipInstance;
use crate::vector_graph::{VertexId, EdgeId, FillId};
use crate::layer::VectorLayer;
use crate::shape::Shape;
@ -260,20 +260,16 @@ pub fn hit_test_clip_instances(
for clip_instance in clip_instances.iter().rev() {
// Check time bounds: skip clip instances not active at this time
// timeline_start/instance_end are in beats; convert timeline_time (seconds) to beats.
// Hit-testing runs on vector/raster content, which is wall-clock seconds.
let clip_duration = ClipDuration::Seconds(
document.get_clip_duration(&clip_instance.clip_id).unwrap_or(daw_backend::Seconds::ZERO),
);
let clip_duration = document.get_clip_duration(&clip_instance.clip_id).unwrap_or(0.0);
let instance_end = clip_instance.timeline_start + clip_instance.effective_duration(clip_duration, tempo_map);
let timeline_beats = tempo_map.seconds_to_beats(daw_backend::Seconds(timeline_time));
let timeline_beats = tempo_map.inverse_transform(timeline_time);
if timeline_beats < clip_instance.timeline_start || timeline_beats >= instance_end {
continue;
}
// clip_time is in seconds; offset from clip start (in seconds) + trim_start (seconds)
let start_secs = tempo_map.beats_to_seconds(clip_instance.timeline_start).seconds_to_f64();
let clip_time =
((timeline_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start.raw();
let start_secs = tempo_map.transform(clip_instance.timeline_start);
let clip_time = ((timeline_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start;
let content_bounds = if let Some(vector_clip) = document.get_vector_clip(&clip_instance.clip_id) {
vector_clip.calculate_content_bounds(document, clip_time)
@ -308,19 +304,15 @@ pub fn hit_test_clip_instances_in_rect(
for clip_instance in clip_instances {
// Check time bounds: skip clip instances not active at this time
// timeline_start/instance_end are in beats; convert timeline_time (seconds) to beats.
// Hit-testing runs on vector/raster content, which is wall-clock seconds.
let clip_duration = ClipDuration::Seconds(
document.get_clip_duration(&clip_instance.clip_id).unwrap_or(daw_backend::Seconds::ZERO),
);
let clip_duration = document.get_clip_duration(&clip_instance.clip_id).unwrap_or(0.0);
let instance_end = clip_instance.timeline_start + clip_instance.effective_duration(clip_duration, tempo_map);
let timeline_beats = tempo_map.seconds_to_beats(daw_backend::Seconds(timeline_time));
let timeline_beats = tempo_map.inverse_transform(timeline_time);
if timeline_beats < clip_instance.timeline_start || timeline_beats >= instance_end {
continue;
}
let start_secs = tempo_map.beats_to_seconds(clip_instance.timeline_start).seconds_to_f64();
let clip_time =
((timeline_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start.raw();
let start_secs = tempo_map.transform(clip_instance.timeline_start);
let clip_time = ((timeline_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start;
let content_bounds = if let Some(vector_clip) = document.get_vector_clip(&clip_instance.clip_id) {
vector_clip.calculate_content_bounds(document, clip_time)

View File

@ -9,9 +9,8 @@
//! The compositing mode enables proper per-layer opacity, blend modes, and effects.
use crate::animation::TransformProperty;
use crate::clip::{ClipDuration, ClipInstance, ImageAsset};
use crate::clip::{ClipInstance, ImageAsset};
use crate::document::Document;
use daw_backend::Seconds;
use crate::gpu::BlendMode;
use crate::layer::{AnyLayer, LayerTrait, VectorLayer};
use kurbo::Affine;
@ -568,8 +567,7 @@ pub fn render_layer_isolated(
let tempo_map = document.tempo_map();
for clip_instance in &video_layer.clip_instances {
let Some(video_clip) = document.video_clips.get(&clip_instance.clip_id) else { continue };
let Some(clip_time) = clip_instance.remap_time(Seconds(time), ClipDuration::Seconds(Seconds(video_clip.duration)), tempo_map) else { continue };
let clip_time = clip_time.seconds_to_f64();
let Some(clip_time) = clip_instance.remap_time(time, video_clip.duration, tempo_map) else { continue };
let Some(frame) = video_mgr.get_frame(&clip_instance.clip_id, clip_time, target_w, target_h) else { continue };
// Evaluate animated transform properties.
@ -977,7 +975,7 @@ pub fn render_single_clip_instance(
.filter(|vc| vc.is_group)
.map(|_| {
let frame_duration = 1.0 / document.framerate;
vector_layer.group_visibility_end(&clip_instance.id, document.tempo_map().beats_to_seconds(clip_instance.timeline_start).seconds_to_f64(), frame_duration)
vector_layer.group_visibility_end(&clip_instance.id, clip_instance.timeline_start, frame_duration)
});
render_clip_instance(
@ -1012,21 +1010,18 @@ fn render_clip_instance(
let clip_time = if vector_clip.is_group {
// Groups are static — visible from timeline_start to the next keyframe boundary.
// timeline_start is in beats; group_end_time is in seconds (render time).
let start_secs = tempo_map.beats_to_seconds(clip_instance.timeline_start).seconds_to_f64();
let start_secs = tempo_map.transform(clip_instance.timeline_start);
let end = group_end_time.unwrap_or(start_secs);
if time < start_secs || time >= end {
return;
}
0.0
} else {
// A vector clip's content is wall-clock seconds.
let clip_dur = ClipDuration::Seconds(
document.get_clip_duration(&vector_clip.id).unwrap_or(Seconds(vector_clip.duration)),
);
let Some(t) = clip_instance.remap_time(Seconds(time), clip_dur, tempo_map) else {
let clip_dur = document.get_clip_duration(&vector_clip.id).unwrap_or(vector_clip.duration);
let Some(t) = clip_instance.remap_time(time, clip_dur, tempo_map) else {
return; // Clip instance not active at this time
};
t.seconds_to_f64()
t
};
// Evaluate animated transform properties
@ -1177,10 +1172,9 @@ fn render_video_layer(
// Remap timeline time to clip's internal time
let tempo_map = document.tempo_map();
let Some(clip_time) = clip_instance.remap_time(Seconds(time), ClipDuration::Seconds(Seconds(video_clip.duration)), tempo_map) else {
let Some(clip_time) = clip_instance.remap_time(time, video_clip.duration, tempo_map) else {
continue; // Clip instance not active at this time
};
let clip_time = clip_time.seconds_to_f64();
// Get video frame from VideoManager at the output (export/preview) resolution.
let (target_w, target_h) = video_decode_target(document, base_transform);
@ -1574,7 +1568,7 @@ fn render_vector_layer(
.filter(|vc| vc.is_group)
.map(|_| {
let frame_duration = 1.0 / document.framerate;
layer.group_visibility_end(&clip_instance.id, document.tempo_map().beats_to_seconds(clip_instance.timeline_start).seconds_to_f64(), frame_duration)
layer.group_visibility_end(&clip_instance.id, clip_instance.timeline_start, frame_duration)
});
render_clip_instance(document, time, clip_instance, layer_opacity, scene, base_transform, &layer.layer.animation_data, image_cache, video_manager, group_end_time, extract.as_deref_mut());
}
@ -1883,7 +1877,7 @@ fn render_vector_layer_cpu(
.filter(|vc| vc.is_group)
.map(|_| {
let frame_duration = 1.0 / document.framerate;
layer.group_visibility_end(&clip_instance.id, document.tempo_map().beats_to_seconds(clip_instance.timeline_start).seconds_to_f64(), frame_duration)
layer.group_visibility_end(&clip_instance.id, clip_instance.timeline_start, frame_duration)
});
render_clip_instance_cpu(
document, time, clip_instance, layer_opacity, pixmap, base_transform,
@ -1908,17 +1902,14 @@ fn render_clip_instance_cpu(
let tempo_map = document.tempo_map();
let clip_time = if vector_clip.is_group {
let start_secs = tempo_map.beats_to_seconds(clip_instance.timeline_start).seconds_to_f64();
let start_secs = tempo_map.transform(clip_instance.timeline_start);
let end = group_end_time.unwrap_or(start_secs);
if time < start_secs || time >= end { return; }
0.0
} else {
// A vector clip's content is wall-clock seconds.
let clip_dur = ClipDuration::Seconds(
document.get_clip_duration(&vector_clip.id).unwrap_or(Seconds(vector_clip.duration)),
);
let Some(t) = clip_instance.remap_time(Seconds(time), clip_dur, tempo_map) else { return };
t.seconds_to_f64()
let clip_dur = document.get_clip_duration(&vector_clip.id).unwrap_or(vector_clip.duration);
let Some(t) = clip_instance.remap_time(time, clip_dur, tempo_map) else { return };
t
};
let transform = &clip_instance.transform;

View File

@ -232,9 +232,8 @@ use crate::vector_graph::{FillId, VectorGraph};
use kurbo::{BezPath, PathEl, Rect, Shape};
/// Serialize the document's **vector** content to a standalone SVG string, at document time `time`.
/// Vector layers, groups of them, and text layers (as real glyph outlines) — raster/video/audio/
/// effect layers are skipped (a later pass can rasterize them to `<image>`). Animation is a single
/// static frame at `time`.
/// Vector layers (and groups of them) only — raster/video/audio/effect layers are skipped (a later
/// pass can rasterize them to `<image>`). Animation is a single static frame at `time`.
pub fn document_to_svg(document: &Document, time: f64) -> String {
let (w, h) = (document.width, document.height);
let mut defs = String::new();
@ -301,136 +300,11 @@ fn layer_to_svg(layer: &AnyLayer, time: f64, parent_opacity: f64, body: &mut Str
body.push_str("</g>");
}
}
AnyLayer::Text(tl) => text_layer_to_svg(tl, time, parent_opacity, body),
// Raster/Video/Audio/Effect have no lossless vector representation — skipped this pass.
_ => {}
}
}
/// A skrifa outline pen that appends transformed glyph contours to an SVG path `d` string.
///
/// skrifa emits outline points in y-up pixel space (origin at the glyph baseline); this maps each
/// point into document space: `x = gx + px + skew·py`, `y = gy py` (Y flips, `skew` applies any
/// synthetic-italic slant), where `(gx, gy)` is the glyph's document-space pen position.
struct SvgOutlinePen<'a> {
gx: f64,
gy: f64,
skew: f64,
d: &'a mut String,
}
impl<'a> SvgOutlinePen<'a> {
fn map(&self, px: f32, py: f32) -> (f64, f64) {
let (px, py) = (px as f64, py as f64);
(self.gx + px + self.skew * py, self.gy - py)
}
}
impl skrifa::outline::OutlinePen for SvgOutlinePen<'_> {
fn move_to(&mut self, x: f32, y: f32) {
let (x, y) = self.map(x, y);
self.d.push_str(&format!("M{x:.2} {y:.2}"));
}
fn line_to(&mut self, x: f32, y: f32) {
let (x, y) = self.map(x, y);
self.d.push_str(&format!("L{x:.2} {y:.2}"));
}
fn quad_to(&mut self, cx: f32, cy: f32, x: f32, y: f32) {
let (cx, cy) = self.map(cx, cy);
let (x, y) = self.map(x, y);
self.d.push_str(&format!("Q{cx:.2} {cy:.2} {x:.2} {y:.2}"));
}
fn curve_to(&mut self, c0x: f32, c0y: f32, c1x: f32, c1y: f32, x: f32, y: f32) {
let (c0x, c0y) = self.map(c0x, c0y);
let (c1x, c1y) = self.map(c1x, c1y);
let (x, y) = self.map(x, y);
self.d.push_str(&format!("C{c0x:.2} {c0y:.2} {c1x:.2} {c1y:.2} {x:.2} {y:.2}"));
}
fn close(&mut self) {
self.d.push('Z');
}
}
/// Append a text layer's glyphs to `body` as a single filled `<path>` of real glyph outlines
/// (lossless — no font dependency in the SVG). Lays the text out with the same parley path the
/// renderer uses, then extracts each glyph's outline with skrifa. Variable-font axis positions and
/// synthetic-italic skew are honored; synthetic bold is not (rare).
fn text_layer_to_svg(
tl: &crate::text_layer::TextLayer,
time: f64,
parent_opacity: f64,
body: &mut String,
) {
use skrifa::MetadataProvider;
if !tl.layer.visible {
return;
}
let content = tl.content_at(time);
if content.text.is_empty() {
return;
}
let (ox, oy) = (tl.box_origin.x, tl.box_origin.y);
let mut d = String::new();
crate::fonts::with_layout(content, tl.box_width as f32, |layout| {
for line in layout.lines() {
for item in line.items() {
let parley::PositionedLayoutItem::GlyphRun(glyph_run) = item else { continue };
let run = glyph_run.run();
let font = run.font();
let font_size = run.font_size();
let skew = run
.synthesis()
.skew()
.map(|angle| (angle as f64).to_radians().tan())
.unwrap_or(0.0);
let Ok(font_ref) = skrifa::FontRef::from_index(font.data.data(), font.index) else {
continue;
};
let outlines = font_ref.outline_glyphs();
// Variable-font axis position for this run (empty for static fonts).
let coords: Vec<skrifa::instance::NormalizedCoord> = run
.normalized_coords()
.iter()
.map(|&c| skrifa::instance::NormalizedCoord::from_bits(c))
.collect();
let location = skrifa::instance::LocationRef::new(&coords);
let size = skrifa::instance::Size::new(font_size);
for g in glyph_run.positioned_glyphs() {
let Some(glyph) = outlines.get(skrifa::GlyphId::new(g.id as u32)) else {
continue;
};
let mut pen = SvgOutlinePen {
gx: ox + g.x as f64,
gy: oy + g.y as f64,
skew,
d: &mut d,
};
let settings = skrifa::outline::DrawSettings::unhinted(size, location);
let _ = glyph.draw(settings, &mut pen);
}
}
}
});
if d.is_empty() {
return;
}
let [r, g, b, a] = content.color;
let to_u8 = |c: f32| (c.clamp(0.0, 1.0) * 255.0).round() as u8;
let fill_opacity = (a as f64 * parent_opacity * tl.layer.opacity).clamp(0.0, 1.0);
body.push_str(&format!(
r#"<path fill="rgb({},{},{})" fill-opacity="{:.4}" fill-rule="nonzero" d="{}"/>"#,
to_u8(r), to_u8(g), to_u8(b), fill_opacity, d
));
}
/// Emit a vector graph's fills (`<path fill>`) and stroked edges (`<path stroke>`) into `body`,
/// accumulating any gradients into `defs`. Geometry is in document space (no per-layer transform).
fn vector_graph_to_svg(graph: &VectorGraph, body: &mut String, defs: &mut String, grad_n: &mut usize) {
@ -614,58 +488,4 @@ mod export_tests {
// 1 fill path + 3 stroked edges = 4 <path> elements.
assert_eq!(body.matches("<path").count(), 4, "{body}");
}
#[test]
fn outline_pen_maps_yflip_and_skew() {
use skrifa::outline::OutlinePen;
let mut d = String::new();
{
let mut pen = SvgOutlinePen { gx: 10.0, gy: 100.0, skew: 0.0, d: &mut d };
pen.move_to(0.0, 0.0); // baseline origin → (10, 100)
pen.line_to(5.0, 20.0); // 20 up → y = 100 20 = 80
pen.close();
}
assert!(d.contains("M10.00 100.00"), "d={d}");
assert!(d.contains("L15.00 80.00"), "d={d}");
assert!(d.ends_with('Z'));
// Synthetic-italic skew shifts x right in proportion to height.
let mut d2 = String::new();
{
let mut pen = SvgOutlinePen { gx: 0.0, gy: 0.0, skew: 0.5, d: &mut d2 };
pen.move_to(0.0, 10.0); // x = 0 + 0.5·10 = 5, y = 10
}
assert!(d2.contains("M5.00 -10.00"), "d={d2}");
}
#[test]
fn text_layer_emits_real_glyph_outlines() {
use crate::text_layer::TextLayer;
let mut tl = TextLayer::new("t", Point::new(20.0, 60.0));
tl.content.text = "Hi".to_string();
tl.content.font_size = 48.0;
tl.content.color = [1.0, 0.0, 0.0, 1.0];
let mut body = String::new();
text_layer_to_svg(&tl, 0.0, 1.0, &mut body);
// Bundled fonts guarantee glyphs → a filled path with actual outline segments.
assert!(body.contains("<path"), "no path emitted: {body}");
assert!(body.contains(r#"fill="rgb(255,0,0)""#), "wrong fill: {body}");
assert!(
body.contains('C') || body.contains('Q') || body.contains('L'),
"path has no outline segments: {body}"
);
assert!(body.len() > 80, "suspiciously short path: {body}");
}
#[test]
fn empty_text_layer_emits_nothing() {
use crate::text_layer::TextLayer;
let tl = TextLayer::new("t", Point::new(0.0, 0.0)); // no text set
let mut body = String::new();
text_layer_to_svg(&tl, 0.0, 1.0, &mut body);
assert!(body.is_empty(), "empty text should emit nothing: {body}");
}
}

View File

@ -1,6 +1,6 @@
[package]
name = "lightningbeam-editor"
version = "1.0.10-alpha"
version = "1.0.7-alpha"
edition = "2021"
description = "Multimedia editor for audio, video and 2D animation"
license = "GPL-3.0-or-later"
@ -41,10 +41,6 @@ serde_json = { workspace = true }
# Image loading
image = { workspace = true }
# Animated GIF encoding — used directly (not just via `image`) so per-frame NeuQuant palette
# quantization can be parallelized across a worker pool. Pinned to the version `image` already
# resolves (0.13), so this adds no new transitive graph.
gif = "0.13"
resvg = { workspace = true }
tiny-skia = "0.11"

View File

@ -35,18 +35,6 @@ pub struct AppConfig {
#[serde(default = "defaults::audio_buffer_size")]
pub audio_buffer_size: u32,
/// How a cycle MIDI recording treats its passes.
///
/// `false` (default) = **merge**: every pass overdubs into one clip, and earlier passes play back
/// as you record so you can layer against them. `true` = **separate takes**: each pass becomes
/// its own take in a take folder, exactly as audio always does, and earlier passes stay silent
/// (they're alternatives, not layers).
///
/// Only applies when the transport actually wraps; a recording that stops inside the first pass
/// is an ordinary single recording either way.
#[serde(default = "defaults::cycle_midi_separate_takes")]
pub cycle_midi_separate_takes: bool,
/// Reopen last session on startup
#[serde(default = "defaults::reopen_last_session")]
pub reopen_last_session: bool,
@ -82,55 +70,6 @@ pub struct AppConfig {
/// sooner; larger = smaller pyramid, wider re-decode span. Default 256.
#[serde(default = "defaults::waveform_floor_samples_per_texel")]
pub waveform_floor_samples_per_texel: u32,
/// Last-used audio-export "Artist" tag, remembered so it prefills next time.
#[serde(default)]
pub last_audio_artist: String,
/// Last-used audio-export "Album" tag, remembered so it prefills next time.
#[serde(default)]
pub last_audio_album: String,
/// What the stylus's lower barrel button does while held.
#[serde(default = "defaults::tablet_button_lower")]
pub tablet_button_lower: TabletButtonAction,
/// What the stylus's upper barrel button does while held.
#[serde(default = "defaults::tablet_button_upper")]
pub tablet_button_upper: TabletButtonAction,
}
/// What a stylus barrel button does while held down.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)]
#[serde(rename_all = "snake_case")]
pub enum TabletButtonAction {
/// Do nothing.
None,
/// Drag to pan the stage.
#[default]
Pan,
/// Temporarily switch to the eyedropper.
Eyedropper,
/// Temporarily switch to the eraser.
Erase,
}
impl TabletButtonAction {
pub const ALL: [TabletButtonAction; 4] = [
TabletButtonAction::None,
TabletButtonAction::Pan,
TabletButtonAction::Eyedropper,
TabletButtonAction::Erase,
];
pub fn label(self) -> &'static str {
match self {
TabletButtonAction::None => "None",
TabletButtonAction::Pan => "Pan",
TabletButtonAction::Eyedropper => "Eyedropper",
TabletButtonAction::Erase => "Eraser",
}
}
}
impl Default for AppConfig {
@ -143,7 +82,6 @@ impl Default for AppConfig {
file_height: defaults::file_height(),
scroll_speed: defaults::scroll_speed(),
audio_buffer_size: defaults::audio_buffer_size(),
cycle_midi_separate_takes: defaults::cycle_midi_separate_takes(),
reopen_last_session: defaults::reopen_last_session(),
restore_layout_from_file: defaults::restore_layout_from_file(),
debug: defaults::debug(),
@ -152,10 +90,6 @@ impl Default for AppConfig {
keybindings: KeybindingConfig::default(),
large_media_default: LargeMediaMode::default(),
waveform_floor_samples_per_texel: defaults::waveform_floor_samples_per_texel(),
last_audio_artist: String::new(),
last_audio_album: String::new(),
tablet_button_lower: defaults::tablet_button_lower(),
tablet_button_upper: defaults::tablet_button_upper(),
}
}
}
@ -346,16 +280,12 @@ impl AppConfig {
/// Default values for preferences (matches JS implementation)
mod defaults {
use super::TabletButtonAction;
pub fn tablet_button_lower() -> TabletButtonAction { TabletButtonAction::Pan }
pub fn tablet_button_upper() -> TabletButtonAction { TabletButtonAction::Eyedropper }
pub fn bpm() -> u32 { 120 }
pub fn framerate() -> u32 { 24 }
pub fn file_width() -> u32 { 800 }
pub fn file_height() -> u32 { 600 }
pub fn scroll_speed() -> f64 { 1.0 }
pub fn audio_buffer_size() -> u32 { 256 }
pub fn cycle_midi_separate_takes() -> bool { false }
pub fn reopen_last_session() -> bool { false }
pub fn restore_layout_from_file() -> bool { true }
pub fn debug() -> bool { false }

View File

@ -1,380 +1,205 @@
//! Custom cursor system
//!
//! Tool cursors are drawn from the bundled Lucide icon font. A tool falls into one of three
//! kinds, because "draw the tool's icon at the pointer" only works for icons that actually have
//! a point:
//!
//! * [`CursorKind::System`] — the OS cursor already says it better than we can (the arrow for
//! Select, the move cross for Transform). We use the real OS cursor.
//! * [`CursorKind::Precise`] — the glyph has an unambiguous tip (pencil, brush, pipette), so it
//! *is* the cursor: the tip lands exactly on the click point.
//! * [`CursorKind::Badge`] — the glyph has no focus point (stamp, bandage, spray can). Using it
//! alone would leave you guessing where you're clicking, so we pair it with a crosshair: the
//! crosshair marks the click point and the glyph hangs off its bottom-right.
//!
//! Rather than *painting* the cursor into the egui scene, we rasterize it and hand it to the
//! windowing system as a real OS cursor (`egui::CursorImage` → winit `CustomCursor`). A painted
//! cursor is composited with our frame and therefore always trails the pointer by a frame or
//! more; an OS cursor is composited by the window system and doesn't lag at all.
//!
//! The glyphs are rasterized out of egui's own font atlas using the same placement maths as
//! `epaint`'s text tessellator, so a glyph lands exactly where `painter.text` would have put it.
//! Provides SVG-based custom cursors beyond egui's built-in system cursors.
//! When a custom cursor is active, the system cursor is hidden and the SVG
//! cursor image is drawn at the pointer position.
use eframe::egui;
use egui::TextureHandle;
use lightningbeam_core::tool::Tool;
use std::collections::HashMap;
use std::sync::Arc;
use crate::mobile::icons;
/// What kind of cursor a tool gets.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum CursorKind {
/// Use a native OS cursor; build nothing ourselves.
System(egui::CursorIcon),
/// The glyph is the cursor. `hotspot` is the click point in Lucide's 24×24 icon grid.
Precise {
glyph: &'static str,
hotspot: egui::Vec2,
},
/// A crosshair marks the click point; the glyph sits to its bottom-right.
Badge { glyph: &'static str },
}
/// Hotspot for a glyph whose tip is at the bottom-left (pencil, brush, pen, pipette).
const TIP_BOTTOM_LEFT: egui::Vec2 = egui::vec2(3.0, 21.0);
/// Hotspot for a glyph centred on the click point.
const TIP_CENTER: egui::Vec2 = egui::vec2(12.0, 12.0);
/// Which cursor a stage tool uses.
/// Custom cursor identifiers
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum CustomCursor {
Tool(Tool),
/// Timeline: dragging the end of the loop region.
// Stage tool cursors
Select,
Draw,
Transform,
Rectangle,
Ellipse,
PaintBucket,
Eyedropper,
Line,
Polygon,
BezierEdit,
Text,
// Timeline cursors
LoopExtend,
}
impl CustomCursor {
/// Convert a Tool enum to the corresponding custom cursor
pub fn from_tool(tool: Tool) -> Self {
CustomCursor::Tool(tool)
match tool {
Tool::Select => CustomCursor::Select,
Tool::Draw => CustomCursor::Draw,
Tool::Transform => CustomCursor::Transform,
Tool::Rectangle => CustomCursor::Rectangle,
Tool::Ellipse => CustomCursor::Ellipse,
Tool::PaintBucket => CustomCursor::PaintBucket,
Tool::Eyedropper => CustomCursor::Eyedropper,
Tool::Line => CustomCursor::Line,
Tool::Polygon => CustomCursor::Polygon,
Tool::BezierEdit => CustomCursor::BezierEdit,
Tool::Text => CustomCursor::Text,
Tool::RegionSelect => CustomCursor::Select,
Tool::Split => CustomCursor::Select,
Tool::Erase => CustomCursor::Draw,
Tool::Smudge => CustomCursor::Draw,
Tool::SelectLasso => CustomCursor::Select,
// Raster brush tools — use draw cursor until implemented
Tool::Pencil
| Tool::Pen
| Tool::Airbrush
| Tool::CloneStamp
| Tool::HealingBrush
| Tool::PatternStamp
| Tool::DodgeBurn
| Tool::Sponge
| Tool::BlurSharpen => CustomCursor::Draw,
// Selection tools — use select cursor until implemented
Tool::SelectEllipse
| Tool::MagicWand
| Tool::QuickSelect => CustomCursor::Select,
// Other tools — use select cursor until implemented
Tool::Gradient
| Tool::CustomShape
| Tool::Warp
| Tool::Liquify => CustomCursor::Select,
}
}
pub fn kind(&self) -> CursorKind {
use egui::CursorIcon as Sys;
/// Hotspot offset — the "click point" relative to the image top-left
pub fn hotspot(&self) -> egui::Vec2 {
match self {
// Select cursor: pointer tip at top-left
CustomCursor::Select => egui::vec2(3.0, 1.0),
// Drawing tools: tip at bottom-left
CustomCursor::Draw => egui::vec2(1.0, 23.0),
// Transform: center
CustomCursor::Transform => egui::vec2(12.0, 12.0),
// Shape tools: crosshair at center
CustomCursor::Rectangle
| CustomCursor::Ellipse
| CustomCursor::Line
| CustomCursor::Polygon => egui::vec2(12.0, 12.0),
// Paint bucket: tip at bottom-left
CustomCursor::PaintBucket => egui::vec2(2.0, 21.0),
// Eyedropper: tip at bottom
CustomCursor::Eyedropper => egui::vec2(4.0, 22.0),
// Bezier edit: tip at top-left
CustomCursor::BezierEdit => egui::vec2(3.0, 1.0),
// Text: I-beam center
CustomCursor::Text => egui::vec2(12.0, 12.0),
// Loop extend: center of circular arrow
CustomCursor::LoopExtend => egui::vec2(12.0, 12.0),
}
}
let tool = match self {
CustomCursor::LoopExtend => {
return CursorKind::Precise {
glyph: icons::REPEAT,
hotspot: TIP_CENTER,
}
}
CustomCursor::Tool(t) => *t,
};
let precise = |glyph, hotspot| CursorKind::Precise { glyph, hotspot };
let badge = |glyph| CursorKind::Badge { glyph };
match tool {
// The OS knows these better than we do.
Tool::Select => CursorKind::System(Sys::Default),
Tool::Transform => CursorKind::System(Sys::Move),
// Glyphs with a real tip: the icon is the cursor.
Tool::Draw => precise(icons::BRUSH, TIP_BOTTOM_LEFT),
Tool::Pencil => precise(icons::PENCIL, TIP_BOTTOM_LEFT),
Tool::Pen => precise(icons::PEN_TOOL, TIP_BOTTOM_LEFT),
Tool::Eyedropper => precise(icons::PIPETTE, TIP_BOTTOM_LEFT),
Tool::PaintBucket => precise(icons::PAINT_BUCKET, TIP_BOTTOM_LEFT),
Tool::Text => precise(icons::TEXT_CURSOR, TIP_CENTER),
// Shape tools: crosshair for precision, glyph to say which shape.
Tool::Rectangle => badge(icons::SQUARE),
Tool::Ellipse => badge(icons::CIRCLE),
Tool::Line => badge(icons::MINUS),
Tool::Polygon => badge(icons::HEXAGON),
Tool::CustomShape => badge(icons::SHAPES),
// Selection tools.
Tool::RegionSelect => badge(icons::SQUARE_DASHED),
Tool::SelectEllipse => badge(icons::CIRCLE_DASHED),
Tool::SelectLasso => badge(icons::LASSO_SELECT),
Tool::MagicWand => badge(icons::WAND_SPARKLES),
Tool::QuickSelect => badge(icons::BRUSH),
Tool::Split => badge(icons::SCISSORS),
// Raster tools whose icons have no focus point. These also draw a brush-size ring on
// the stage, which is the real precision cue.
Tool::Erase => badge(icons::ERASER),
Tool::Airbrush => badge(icons::SPRAY_CAN),
Tool::Smudge => badge(icons::POINTER),
Tool::CloneStamp => badge(icons::STAMP),
Tool::PatternStamp => badge(icons::SHAPES),
Tool::HealingBrush => badge(icons::BANDAGE),
Tool::DodgeBurn => badge(icons::SUN_MOON),
Tool::Sponge => badge(icons::DROPLETS),
Tool::BlurSharpen => badge(icons::CONTRAST),
Tool::Gradient => badge(icons::BLEND),
Tool::Warp => badge(icons::SCALING),
Tool::Liquify => badge(icons::DROPLET),
// Vertex editing: crosshair to place the point, glyph to say we're in bezier mode.
Tool::BezierEdit => badge(icons::SPLINE),
/// Get the embedded SVG data for this cursor
fn svg_data(&self) -> &'static [u8] {
match self {
CustomCursor::Select => include_bytes!("../../../src/assets/select.svg"),
CustomCursor::Draw => include_bytes!("../../../src/assets/draw.svg"),
CustomCursor::Transform => include_bytes!("../../../src/assets/transform.svg"),
CustomCursor::Rectangle => include_bytes!("../../../src/assets/rectangle.svg"),
CustomCursor::Ellipse => include_bytes!("../../../src/assets/ellipse.svg"),
CustomCursor::PaintBucket => include_bytes!("../../../src/assets/paint_bucket.svg"),
CustomCursor::Eyedropper => include_bytes!("../../../src/assets/eyedropper.svg"),
CustomCursor::Line => include_bytes!("../../../src/assets/line.svg"),
CustomCursor::Polygon => include_bytes!("../../../src/assets/polygon.svg"),
CustomCursor::BezierEdit => include_bytes!("../../../src/assets/bezier_edit.svg"),
CustomCursor::Text => include_bytes!("../../../src/assets/text.svg"),
CustomCursor::LoopExtend => include_bytes!("../../../src/assets/arrow-counterclockwise.svg"),
}
}
}
// ---------------------------------------------------------------------------
// Geometry (logical points; scaled by pixels_per_point when rasterized)
// ---------------------------------------------------------------------------
/// Rendered size of a cursor glyph.
const GLYPH_SIZE: f32 = 18.0;
/// Lucide icons are authored on a 24×24 grid; hotspots are given in those units.
const ICON_GRID: f32 = 24.0;
/// Where a badge glyph's top-left sits relative to the crosshair centre.
const BADGE_OFFSET: egui::Vec2 = egui::vec2(6.0, 6.0);
/// Half-length of a crosshair arm.
const CROSSHAIR_ARM: f32 = 7.0;
/// Gap between the crosshair centre and the start of each arm, so the click point stays visible.
const CROSSHAIR_GAP: f32 = 2.0;
/// Margin around the artwork, leaving room for the outline.
const MARGIN: f32 = 1.5;
// ---------------------------------------------------------------------------
// Rasterization
// ---------------------------------------------------------------------------
/// Cache of rasterized cursor images, keyed by cursor and DPI scale.
#[derive(Default)]
/// Cache of rasterized cursor textures (black fill + white outline version)
pub struct CursorCache {
images: HashMap<(CustomCursor, u32), Option<egui::CursorImage>>,
next_id: u64,
/// Black cursor for the main image
textures: HashMap<CustomCursor, TextureHandle>,
/// White cursor for the outline
outline_textures: HashMap<CustomCursor, TextureHandle>,
}
impl CursorCache {
pub fn new() -> Self {
Self::default()
Self {
textures: HashMap::new(),
outline_textures: HashMap::new(),
}
}
fn get_or_build(
&mut self,
ctx: &egui::Context,
cursor: CustomCursor,
ppp: f32,
) -> Option<egui::CursorImage> {
let key = (cursor, ppp.to_bits());
if let Some(cached) = self.images.get(&key) {
return cached.clone();
}
/// Get or lazily load the black (fill) cursor texture
pub fn get_or_load(&mut self, cursor: CustomCursor, ctx: &egui::Context) -> &TextureHandle {
self.textures.entry(cursor).or_insert_with(|| {
let svg_data = cursor.svg_data();
let svg_string = String::from_utf8_lossy(svg_data);
let svg_with_color = svg_string.replace("currentColor", "#000000");
rasterize_cursor_svg(svg_with_color.as_bytes(), &format!("cursor_{:?}", cursor), CURSOR_SIZE, ctx)
.expect("Failed to rasterize cursor SVG")
})
}
let id = self.next_id;
self.next_id += 1;
let built = build_cursor_image(ctx, cursor.kind(), ppp, id);
self.images.insert(key, built.clone());
built
/// Get or lazily load the white (outline) cursor texture
pub fn get_or_load_outline(&mut self, cursor: CustomCursor, ctx: &egui::Context) -> &TextureHandle {
self.outline_textures.entry(cursor).or_insert_with(|| {
let svg_data = cursor.svg_data();
let svg_string = String::from_utf8_lossy(svg_data);
// Replace all colors with white for the outline
let svg_white = svg_string
.replace("currentColor", "#ffffff")
.replace("#000000", "#ffffff")
.replace("#000", "#ffffff");
rasterize_cursor_svg(svg_white.as_bytes(), &format!("cursor_{:?}_outline", cursor), CURSOR_SIZE, ctx)
.expect("Failed to rasterize cursor SVG outline")
})
}
}
/// An alpha coverage mask being composed, in physical pixels.
struct Mask {
w: usize,
h: usize,
a: Vec<f32>,
}
const CURSOR_SIZE: u32 = 24;
const OUTLINE_OFFSET: f32 = 1.0;
impl Mask {
fn new(w: usize, h: usize) -> Self {
Self { w, h, a: vec![0.0; w * h] }
}
fn add(&mut self, x: usize, y: usize, coverage: f32) {
if x < self.w && y < self.h {
let p = &mut self.a[y * self.w + x];
*p = (*p + coverage).min(1.0);
}
}
fn get(&self, x: isize, y: isize) -> f32 {
if x < 0 || y < 0 || x as usize >= self.w || y as usize >= self.h {
0.0
} else {
self.a[y as usize * self.w + x as usize]
}
}
/// Fill an axis-aligned rect (physical px, may be fractional) with full coverage.
fn fill_rect(&mut self, rect: egui::Rect) {
let x0 = rect.min.x.floor().max(0.0) as usize;
let y0 = rect.min.y.floor().max(0.0) as usize;
let x1 = (rect.max.x.ceil() as usize).min(self.w);
let y1 = (rect.max.y.ceil() as usize).min(self.h);
for y in y0..y1 {
for x in x0..x1 {
self.add(x, y, 1.0);
}
}
}
}
/// Rasterize a cursor into a premultiplied-RGBA image: black artwork with a white outline, so it
/// reads against both light and dark backgrounds.
fn build_cursor_image(
/// Rasterize an SVG into an egui texture (same approach as main.rs rasterize_svg)
fn rasterize_cursor_svg(
svg_data: &[u8],
name: &str,
render_size: u32,
ctx: &egui::Context,
kind: CursorKind,
ppp: f32,
id: u64,
) -> Option<egui::CursorImage> {
let (glyph, glyph_origin, hotspot, crosshair) = match kind {
CursorKind::System(_) => return None,
CursorKind::Precise { glyph, hotspot } => {
let scale = GLYPH_SIZE / ICON_GRID;
(
glyph,
egui::vec2(MARGIN, MARGIN),
egui::pos2(MARGIN, MARGIN) + hotspot * scale,
false,
)
}
CursorKind::Badge { glyph } => {
// The crosshair centre is the click point; place it a margin + arm in from the corner.
let centre = egui::pos2(MARGIN + CROSSHAIR_ARM, MARGIN + CROSSHAIR_ARM);
(glyph, centre.to_vec2() + BADGE_OFFSET, centre, true)
}
};
// Lay the glyph out exactly as `painter.text(.., Align2::LEFT_TOP, ..)` would, so we inherit
// epaint's placement rather than re-deriving it from font metrics.
let galley = ctx.fonts_mut(|f| {
f.layout_no_wrap(
glyph.to_owned(),
icons::font(GLYPH_SIZE),
egui::Color32::WHITE,
)
});
// Canvas must cover the glyph, and the crosshair too when there is one.
let mut content = egui::Rect::from_min_size(glyph_origin.to_pos2(), galley.size());
if crosshair {
content = content.union(egui::Rect::from_center_size(
hotspot,
egui::Vec2::splat(2.0 * CROSSHAIR_ARM),
));
}
let canvas = content.expand(MARGIN);
let w = (canvas.width() * ppp).ceil() as usize;
let h = (canvas.height() * ppp).ceil() as usize;
if w == 0 || h == 0 || w > 2048 || h > 2048 {
return None; // winit caps cursors at 2048px.
}
let mut mask = Mask::new(w, h);
// Everything is positioned relative to the canvas origin.
let to_px = |p: egui::Pos2| ((p - canvas.min.to_vec2()).to_vec2() * ppp).to_pos2();
// --- Crosshair ---
if crosshair {
let c = to_px(hotspot);
let thickness = ppp.max(1.0); // one physical pixel, at least
let arm = CROSSHAIR_ARM * ppp;
let gap = CROSSHAIR_GAP * ppp;
// Four arms, leaving a gap at the centre so the exact click point stays visible.
for (dx, dy) in [(-1.0, 0.0), (1.0, 0.0), (0.0, -1.0), (0.0, 1.0)] {
let from = egui::pos2(c.x + dx * gap, c.y + dy * gap);
let to = egui::pos2(c.x + dx * arm, c.y + dy * arm);
let rect = egui::Rect::from_two_pos(from, to).expand2(if dx == 0.0 {
egui::vec2(thickness / 2.0, 0.0)
} else {
egui::vec2(0.0, thickness / 2.0)
});
mask.fill_rect(rect);
}
}
// --- Glyph, copied out of egui's font atlas ---
let atlas = ctx.fonts_mut(|f| f.image());
let atlas_w = atlas.size[0];
for row in &galley.rows {
for g in &row.glyphs {
let uv = g.uv_rect;
if uv.is_nothing() {
continue;
}
// Same maths as epaint's text tessellator: the glyph's top-left in points is
// `glyph.pos + uv_rect.offset`, relative to the galley's top-left.
let left_top = to_px((glyph_origin + (g.pos + uv.offset).to_vec2()).to_pos2());
let src_w = (uv.max[0] - uv.min[0]) as usize;
let src_h = (uv.max[1] - uv.min[1]) as usize;
for sy in 0..src_h {
for sx in 0..src_w {
let src_i = (uv.min[1] as usize + sy) * atlas_w + (uv.min[0] as usize + sx);
let Some(px) = atlas.pixels.get(src_i) else { continue };
let coverage = px.a() as f32 / 255.0;
if coverage <= 0.0 {
continue;
}
let dx = (left_top.x.round() as isize) + sx as isize;
let dy = (left_top.y.round() as isize) + sy as isize;
if dx >= 0 && dy >= 0 {
mask.add(dx as usize, dy as usize, coverage);
}
}
}
}
}
// --- Compose: black fill over a white outline (dilate the mask by one pixel) ---
let mut rgba = vec![0u8; w * h * 4];
for y in 0..h {
for x in 0..w {
let fill = mask.get(x as isize, y as isize);
let mut outline: f32 = 0.0;
for oy in -1..=1_isize {
for ox in -1..=1_isize {
outline = outline.max(mask.get(x as isize + ox, y as isize + oy));
}
}
// White outline underneath, black fill on top; premultiplied.
let white = outline * (1.0 - fill);
let alpha = fill + outline * (1.0 - fill);
let c = (white * 255.0).round().clamp(0.0, 255.0) as u8;
let i = (y * w + x) * 4;
rgba[i] = c;
rgba[i + 1] = c;
rgba[i + 2] = c;
rgba[i + 3] = (alpha * 255.0).round().clamp(0.0, 255.0) as u8;
}
}
let hot = to_px(hotspot);
Some(egui::CursorImage {
id,
rgba: Arc::new(rgba),
size: (w as u16, h as u16),
hotspot: (
hot.x.round().clamp(0.0, w as f32 - 1.0) as u16,
hot.y.round().clamp(0.0, h as f32 - 1.0) as u16,
),
})
) -> Option<TextureHandle> {
let tree = resvg::usvg::Tree::from_data(svg_data, &resvg::usvg::Options::default()).ok()?;
let pixmap_size = tree.size().to_int_size();
let scale_x = render_size as f32 / pixmap_size.width() as f32;
let scale_y = render_size as f32 / pixmap_size.height() as f32;
let mut pixmap = resvg::tiny_skia::Pixmap::new(render_size, render_size)?;
resvg::render(
&tree,
resvg::tiny_skia::Transform::from_scale(scale_x, scale_y),
&mut pixmap.as_mut(),
);
let rgba_data = pixmap.data().to_vec();
let color_image = egui::ColorImage::from_rgba_unmultiplied(
[render_size as usize, render_size as usize],
&rgba_data,
);
Some(ctx.load_texture(name, color_image, egui::TextureOptions::LINEAR))
}
// --- Per-frame cursor slot using egui context data ---
/// Key for storing the active custom cursor in egui's per-frame data
#[derive(Clone, Copy)]
struct ActiveCustomCursor(CustomCursor);
/// Set the custom cursor for this frame. Call from any pane during rendering.
/// This hides the system cursor and draws the SVG cursor at pointer position.
pub fn set(ctx: &egui::Context, cursor: CustomCursor) {
ctx.data_mut(|d| d.insert_temp(egui::Id::new("active_custom_cursor"), ActiveCustomCursor(cursor)));
}
/// Hand the active cursor to the windowing system. Call at the end of the main update loop.
/// Render the custom cursor overlay. Call at the end of the main update loop.
pub fn render_overlay(ctx: &egui::Context, cache: &mut CursorCache) {
// Take and remove the cursor so it doesn't persist to the next frame
let id = egui::Id::new("active_custom_cursor");
@ -384,24 +209,43 @@ pub fn render_overlay(ctx: &egui::Context, cache: &mut CursorCache) {
val
});
let Some(ActiveCustomCursor(cursor)) = cursor else { return };
if let Some(ActiveCustomCursor(cursor)) = cursor {
// If a system cursor was explicitly set (resize handles, text inputs, etc.),
// let it take priority over the custom cursor
let system_cursor = ctx.output(|o| o.cursor_icon);
if system_cursor != egui::CursorIcon::Default {
return;
}
// If a widget explicitly asked for a system cursor (resize handles, text inputs, ...), let it
// win — it knows something about the hover target that we don't.
if ctx.output(|o| o.cursor_icon) != egui::CursorIcon::Default {
return;
}
// Hide the system cursor
ctx.set_cursor_icon(egui::CursorIcon::None);
match cursor.kind() {
CursorKind::System(icon) => ctx.set_cursor_icon(icon),
_ => {
let ppp = ctx.pixels_per_point();
if let Some(image) = cache.get_or_build(ctx, cursor, ppp) {
ctx.set_cursor_image(Some(image));
} else {
// Rasterization failed — better a crosshair than an invisible cursor.
ctx.set_cursor_icon(egui::CursorIcon::Crosshair);
if let Some(pos) = ctx.input(|i| i.pointer.latest_pos()) {
let hotspot = cursor.hotspot();
let size = egui::vec2(CURSOR_SIZE as f32, CURSOR_SIZE as f32);
let uv = egui::Rect::from_min_max(egui::pos2(0.0, 0.0), egui::pos2(1.0, 1.0));
let painter = ctx.debug_painter();
// Draw white outline: render white version offset in 8 directions
let outline_tex = cache.get_or_load_outline(cursor, ctx);
let outline_id = outline_tex.id();
for &(dx, dy) in &[
(-OUTLINE_OFFSET, 0.0), (OUTLINE_OFFSET, 0.0),
(0.0, -OUTLINE_OFFSET), (0.0, OUTLINE_OFFSET),
(-OUTLINE_OFFSET, -OUTLINE_OFFSET), (OUTLINE_OFFSET, -OUTLINE_OFFSET),
(-OUTLINE_OFFSET, OUTLINE_OFFSET), (OUTLINE_OFFSET, OUTLINE_OFFSET),
] {
let offset_rect = egui::Rect::from_min_size(
pos - hotspot + egui::vec2(dx, dy),
size,
);
painter.image(outline_id, offset_rect, uv, egui::Color32::WHITE);
}
// Draw black fill on top
let fill_tex = cache.get_or_load(cursor, ctx);
let cursor_rect = egui::Rect::from_min_size(pos - hotspot, size);
painter.image(fill_tex.id(), cursor_rect, uv, egui::Color32::WHITE);
}
}
}

View File

@ -0,0 +1,475 @@
#![allow(dead_code)]
//! Audio export functionality
//!
//! Exports audio from the timeline to various formats:
//! - WAV and FLAC: Use existing DAW backend export
//! - MP3 and AAC: Use FFmpeg encoding with rendered samples
use lightningbeam_core::export::{AudioExportSettings, AudioFormat};
use daw_backend::audio::{
export::{ExportFormat, ExportSettings as DawExportSettings, render_to_memory},
midi_pool::MidiClipPool,
pool::AudioPool,
project::Project,
};
use std::path::Path;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
/// Export audio to a file
///
/// This function routes to the appropriate export method based on the format:
/// - WAV/FLAC: Use DAW backend export
/// - MP3/AAC: Use FFmpeg encoding (TODO)
pub fn export_audio<P: AsRef<Path>>(
project: &mut Project,
pool: &AudioPool,
midi_pool: &MidiClipPool,
settings: &AudioExportSettings,
output_path: P,
cancel_flag: &Arc<AtomicBool>,
) -> Result<(), String> {
// Validate settings
settings.validate()?;
// Check for cancellation before starting
if cancel_flag.load(Ordering::Relaxed) {
return Err("Export cancelled by user".to_string());
}
match settings.format {
AudioFormat::Wav | AudioFormat::Flac => {
export_audio_daw_backend(project, pool, midi_pool, settings, output_path)
}
AudioFormat::Mp3 => {
export_audio_ffmpeg_mp3(project, pool, midi_pool, settings, output_path, cancel_flag)
}
AudioFormat::Aac => {
export_audio_ffmpeg_aac(project, pool, midi_pool, settings, output_path, cancel_flag)
}
}
}
/// Export audio using the DAW backend (WAV/FLAC)
fn export_audio_daw_backend<P: AsRef<Path>>(
project: &mut Project,
pool: &AudioPool,
_midi_pool: &MidiClipPool,
settings: &AudioExportSettings,
output_path: P,
) -> Result<(), String> {
// Convert our export settings to DAW backend format
let daw_settings = DawExportSettings {
format: match settings.format {
AudioFormat::Wav => ExportFormat::Wav,
AudioFormat::Flac => ExportFormat::Flac,
_ => unreachable!(), // This function only handles WAV/FLAC
},
sample_rate: settings.sample_rate,
channels: settings.channels,
bit_depth: settings.bit_depth,
mp3_bitrate: 320, // Not used for WAV/FLAC
start_time: daw_backend::Seconds(settings.start_time),
end_time: daw_backend::Seconds(settings.end_time),
tempo_map: daw_backend::TempoMap::constant(settings.bpm),
};
// Use the existing DAW backend export function
// No progress reporting for this direct export path
daw_backend::audio::export::export_audio(
project,
pool,
&daw_settings,
output_path,
None,
)
}
/// Export audio as MP3 using FFmpeg
fn export_audio_ffmpeg_mp3<P: AsRef<Path>>(
project: &mut Project,
pool: &AudioPool,
_midi_pool: &MidiClipPool,
settings: &AudioExportSettings,
output_path: P,
cancel_flag: &Arc<AtomicBool>,
) -> Result<(), String> {
use ffmpeg_next as ffmpeg;
// Initialize FFmpeg
ffmpeg::init().map_err(|e| format!("Failed to initialize FFmpeg: {}", e))?;
// Convert settings to DAW backend format
let daw_settings = DawExportSettings {
format: ExportFormat::Wav, // Unused, but required
sample_rate: settings.sample_rate,
channels: settings.channels,
bit_depth: 16, // Unused
mp3_bitrate: settings.bitrate_kbps,
start_time: daw_backend::Seconds(settings.start_time),
end_time: daw_backend::Seconds(settings.end_time),
tempo_map: daw_backend::TempoMap::constant(settings.bpm),
};
// Step 1: Render audio to memory
let pcm_samples = render_to_memory(
project,
pool,
&daw_settings,
None, // No progress events for now
)?;
// Check for cancellation
if cancel_flag.load(Ordering::Relaxed) {
return Err("Export cancelled".to_string());
}
// Step 2: Set up FFmpeg encoder
let encoder_codec = ffmpeg::encoder::find(ffmpeg::codec::Id::MP3)
.ok_or("MP3 encoder (libmp3lame) not found")?;
// Create output file
let mut output = ffmpeg::format::output(&output_path)
.map_err(|e| format!("Failed to create output file: {}", e))?;
// Create encoder
let mut encoder = ffmpeg::codec::Context::new_with_codec(encoder_codec)
.encoder()
.audio()
.map_err(|e| format!("Failed to create encoder: {}", e))?;
// Configure encoder
let channel_layout = match settings.channels {
1 => ffmpeg::channel_layout::ChannelLayout::MONO,
2 => ffmpeg::channel_layout::ChannelLayout::STEREO,
_ => return Err(format!("Unsupported channel count: {}", settings.channels)),
};
encoder.set_rate(settings.sample_rate as i32);
encoder.set_channel_layout(channel_layout);
encoder.set_format(ffmpeg::format::Sample::I16(ffmpeg::format::sample::Type::Planar));
encoder.set_bit_rate((settings.bitrate_kbps * 1000) as usize);
encoder.set_time_base(ffmpeg::Rational(1, settings.sample_rate as i32));
// Open encoder
let mut encoder = encoder.open_as(encoder_codec)
.map_err(|e| format!("Failed to open MP3 encoder: {}", e))?;
// Add stream and set parameters
{
let mut stream = output.add_stream(encoder_codec)
.map_err(|e| format!("Failed to add stream: {}", e))?;
stream.set_parameters(&encoder);
} // Drop stream here to release the borrow
// Write header
output.write_header()
.map_err(|e| format!("Failed to write header: {}", e))?;
// Step 3: Encode frames and write to output
// Convert interleaved f32 samples to planar i16 format
let num_frames = pcm_samples.len() / settings.channels as usize;
let planar_samples = convert_to_planar_i16(&pcm_samples, settings.channels);
// Get encoder frame size
let frame_size = encoder.frame_size();
let samples_per_frame = if frame_size > 0 {
frame_size as usize
} else {
1152 // Default MP3 frame size
};
// Encode in chunks
let mut samples_encoded = 0;
while samples_encoded < num_frames {
if cancel_flag.load(Ordering::Relaxed) {
return Err("Export cancelled".to_string());
}
let samples_remaining = num_frames - samples_encoded;
let chunk_size = samples_remaining.min(samples_per_frame);
// Create audio frame
let mut frame = ffmpeg::frame::Audio::new(
ffmpeg::format::Sample::I16(ffmpeg::format::sample::Type::Planar),
chunk_size,
channel_layout,
);
frame.set_rate(settings.sample_rate);
// Copy planar samples to frame
// Use plane_mut::<i16> instead of data_mut — data_mut(ch) is buggy for planar audio:
// FFmpeg only sets linesize[0], so data_mut returns 0-length slices for ch > 0.
// plane_mut uses self.samples() for the length, which is correct for all planes.
for ch in 0..settings.channels as usize {
let plane = frame.plane_mut::<i16>(ch);
let offset = samples_encoded;
plane.copy_from_slice(&planar_samples[ch][offset..offset + chunk_size]);
}
// Send frame to encoder
encoder.send_frame(&frame)
.map_err(|e| format!("Failed to send frame: {}", e))?;
// Receive and write packets
receive_and_write_packets(&mut encoder, &mut output)?;
samples_encoded += chunk_size;
}
// Flush encoder
encoder.send_eof()
.map_err(|e| format!("Failed to send EOF: {}", e))?;
receive_and_write_packets(&mut encoder, &mut output)?;
// Write trailer
output.write_trailer()
.map_err(|e| format!("Failed to write trailer: {}", e))?;
Ok(())
}
/// Convert interleaved f32 samples to planar i16 format
fn convert_to_planar_i16(interleaved: &[f32], channels: u32) -> Vec<Vec<i16>> {
let num_frames = interleaved.len() / channels as usize;
let mut planar = vec![vec![0i16; num_frames]; channels as usize];
for (i, chunk) in interleaved.chunks(channels as usize).enumerate() {
for (ch, &sample) in chunk.iter().enumerate() {
// Clamp and convert f32 (-1.0 to 1.0) to i16
let clamped = sample.max(-1.0).min(1.0);
planar[ch][i] = (clamped * 32767.0) as i16;
}
}
planar
}
/// Convert interleaved f32 samples to planar f32 format
fn convert_to_planar_f32(interleaved: &[f32], channels: u32) -> Vec<Vec<f32>> {
let num_frames = interleaved.len() / channels as usize;
let mut planar = vec![vec![0.0f32; num_frames]; channels as usize];
for (i, chunk) in interleaved.chunks(channels as usize).enumerate() {
for (ch, &sample) in chunk.iter().enumerate() {
planar[ch][i] = sample;
}
}
planar
}
/// Receive encoded packets and write to output
fn receive_and_write_packets(
encoder: &mut ffmpeg_next::encoder::Audio,
output: &mut ffmpeg_next::format::context::Output,
) -> Result<(), String> {
let mut encoded = ffmpeg_next::Packet::empty();
while encoder.receive_packet(&mut encoded).is_ok() {
encoded.set_stream(0);
encoded.write_interleaved(output)
.map_err(|e| format!("Failed to write packet: {}", e))?;
}
Ok(())
}
/// Export audio as AAC using FFmpeg
fn export_audio_ffmpeg_aac<P: AsRef<Path>>(
project: &mut Project,
pool: &AudioPool,
_midi_pool: &MidiClipPool,
settings: &AudioExportSettings,
output_path: P,
cancel_flag: &Arc<AtomicBool>,
) -> Result<(), String> {
use ffmpeg_next as ffmpeg;
// Initialize FFmpeg
ffmpeg::init().map_err(|e| format!("Failed to initialize FFmpeg: {}", e))?;
// Convert settings to DAW backend format
let daw_settings = DawExportSettings {
format: ExportFormat::Wav, // Unused, but required
sample_rate: settings.sample_rate,
channels: settings.channels,
bit_depth: 16, // Unused
mp3_bitrate: settings.bitrate_kbps,
start_time: daw_backend::Seconds(settings.start_time),
end_time: daw_backend::Seconds(settings.end_time),
tempo_map: daw_backend::TempoMap::constant(settings.bpm),
};
// Step 1: Render audio to memory
let pcm_samples = render_to_memory(
project,
pool,
&daw_settings,
None, // No progress events for now
)?;
// Check for cancellation
if cancel_flag.load(Ordering::Relaxed) {
return Err("Export cancelled".to_string());
}
// Step 2: Set up FFmpeg encoder
let encoder_codec = ffmpeg::encoder::find(ffmpeg::codec::Id::AAC)
.ok_or("AAC encoder not found")?;
// Create output file
let mut output = ffmpeg::format::output(&output_path)
.map_err(|e| format!("Failed to create output file: {}", e))?;
// Create encoder
let mut encoder = ffmpeg::codec::Context::new_with_codec(encoder_codec)
.encoder()
.audio()
.map_err(|e| format!("Failed to create encoder: {}", e))?;
// Configure encoder
let channel_layout = match settings.channels {
1 => ffmpeg::channel_layout::ChannelLayout::MONO,
2 => ffmpeg::channel_layout::ChannelLayout::STEREO,
_ => return Err(format!("Unsupported channel count: {}", settings.channels)),
};
encoder.set_rate(settings.sample_rate as i32);
encoder.set_channel_layout(channel_layout);
// AAC encoder supports FLTP (F32 Planar) format
encoder.set_format(ffmpeg::format::Sample::F32(ffmpeg::format::sample::Type::Planar));
encoder.set_bit_rate((settings.bitrate_kbps * 1000) as usize);
encoder.set_time_base(ffmpeg::Rational(1, settings.sample_rate as i32));
// Open encoder
let mut encoder = encoder.open_as(encoder_codec)
.map_err(|e| format!("Failed to open AAC encoder: {}", e))?;
// Add stream and set parameters
{
let mut stream = output.add_stream(encoder_codec)
.map_err(|e| format!("Failed to add stream: {}", e))?;
stream.set_parameters(&encoder);
} // Drop stream here to release the borrow
// Write header
output.write_header()
.map_err(|e| format!("Failed to write header: {}", e))?;
// Step 3: Encode frames and write to output
// Convert interleaved f32 samples to planar f32 format (no conversion needed, just rearrange)
let num_frames = pcm_samples.len() / settings.channels as usize;
let planar_samples = convert_to_planar_f32(&pcm_samples, settings.channels);
// Get encoder frame size
let frame_size = encoder.frame_size();
let samples_per_frame = if frame_size > 0 {
frame_size as usize
} else {
1024 // Default AAC frame size
};
// Encode in chunks
let mut samples_encoded = 0;
while samples_encoded < num_frames {
if cancel_flag.load(Ordering::Relaxed) {
return Err("Export cancelled".to_string());
}
let samples_remaining = num_frames - samples_encoded;
let chunk_size = samples_remaining.min(samples_per_frame);
// Create audio frame
let mut frame = ffmpeg::frame::Audio::new(
ffmpeg::format::Sample::F32(ffmpeg::format::sample::Type::Planar),
chunk_size,
channel_layout,
);
frame.set_rate(settings.sample_rate);
// Copy planar samples to frame
unsafe {
for ch in 0..settings.channels as usize {
let plane = frame.data_mut(ch);
let offset = samples_encoded;
let src = &planar_samples[ch][offset..offset + chunk_size];
std::ptr::copy_nonoverlapping(
src.as_ptr() as *const u8,
plane.as_mut_ptr(),
chunk_size * std::mem::size_of::<f32>(),
);
}
}
// Send frame to encoder
encoder.send_frame(&frame)
.map_err(|e| format!("Failed to send frame: {}", e))?;
// Receive and write packets
receive_and_write_packets(&mut encoder, &mut output)?;
samples_encoded += chunk_size;
}
// Flush encoder
encoder.send_eof()
.map_err(|e| format!("Failed to send EOF: {}", e))?;
receive_and_write_packets(&mut encoder, &mut output)?;
// Write trailer
output.write_trailer()
.map_err(|e| format!("Failed to write trailer: {}", e))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_export_audio_validation() {
let mut settings = AudioExportSettings::default();
settings.sample_rate = 0; // Invalid
let project = Project::new(44100);
let pool = AudioPool::new();
let midi_pool = MidiClipPool::new();
let cancel_flag = Arc::new(AtomicBool::new(false));
let result = export_audio(
&mut project.clone(),
&pool,
&midi_pool,
&settings,
"/tmp/test.wav",
&cancel_flag,
);
assert!(result.is_err());
assert!(result.unwrap_err().contains("Sample rate"));
}
#[test]
fn test_export_audio_cancellation() {
let settings = AudioExportSettings::default();
let mut project = Project::new(44100);
let pool = AudioPool::new();
let midi_pool = MidiClipPool::new();
let cancel_flag = Arc::new(AtomicBool::new(true)); // Pre-cancelled
let result = export_audio(
&mut project,
&pool,
&midi_pool,
&settings,
"/tmp/test.wav",
&cancel_flag,
);
assert!(result.is_err());
assert!(result.unwrap_err().contains("cancelled"));
}
}

View File

@ -101,94 +101,6 @@ impl CpuYuvConverter {
}
}
/// CPU RGBA→YUV422P10LE converter (10-bit, 4:2:2) via swscale, for ProRes 422 export.
///
/// ProRes (`prores_ks`) requires a 10-bit 4:2:2 input; the SDR pipeline otherwise produces 8-bit
/// 4:2:0. Source is still 8-bit RGBA (bit-depth is promoted, not conjured), which is normal for
/// SDR ProRes. BT.709 with the requested range, matching the encoder's color tags.
pub struct CpuYuv422P10Converter {
width: u32,
height: u32,
scaler: ffmpeg::software::scaling::Context,
rgba_frame: ffmpeg::frame::Video,
yuv_frame: ffmpeg::frame::Video,
}
impl CpuYuv422P10Converter {
pub fn new(width: u32, height: u32, full_range: bool) -> Result<Self, String> {
let mut scaler = ffmpeg::software::scaling::Context::get(
ffmpeg::format::Pixel::RGBA, width, height,
ffmpeg::format::Pixel::YUV422P10LE, width, height,
ffmpeg::software::scaling::Flags::BILINEAR,
)
.map_err(|e| format!("Failed to create YUV422P10 swscale context: {}", e))?;
// BT.709, requested output range (matches setup_video_encoder's SDR tags). No safe
// ffmpeg-next wrapper for sws_setColorspaceDetails, so this is the raw call (as in
// CpuYuvConverter::new above).
unsafe {
let coeffs = ffmpeg::ffi::sws_getCoefficients(ffmpeg::ffi::SWS_CS_ITU709 as i32);
let dst_range = if full_range { 1 } else { 0 };
let one = 1 << 16;
ffmpeg::ffi::sws_setColorspaceDetails(
scaler.as_mut_ptr(),
coeffs, 1,
coeffs, dst_range,
0, one, one,
);
}
let rgba_frame = ffmpeg::frame::Video::new(ffmpeg::format::Pixel::RGBA, width, height);
let yuv_frame = ffmpeg::frame::Video::new(ffmpeg::format::Pixel::YUV422P10LE, width, height);
Ok(Self { width, height, scaler, rgba_frame, yuv_frame })
}
/// Convert packed RGBA (width*height*4) to tight YUV422P10LE planes (little-endian, 2 bytes per
/// sample): Y is width×height, U and V are (width/2)×height. Planes are returned tight (stride
/// padding stripped) to match what `encode_frame` expects.
pub fn convert(&mut self, rgba_data: &[u8]) -> Result<(Vec<u8>, Vec<u8>, Vec<u8>), String> {
let expected = (self.width * self.height * 4) as usize;
assert_eq!(rgba_data.len(), expected,
"RGBA data size mismatch: expected {} bytes, got {}", expected, rgba_data.len());
// Copy RGBA into the source frame honoring its stride (may be padded).
let row_bytes = (self.width * 4) as usize;
let src_stride = self.rgba_frame.stride(0);
{
let dst = self.rgba_frame.data_mut(0);
for row in 0..self.height as usize {
let s = row * row_bytes;
let d = row * src_stride;
dst[d..d + row_bytes].copy_from_slice(&rgba_data[s..s + row_bytes]);
}
}
self.scaler
.run(&self.rgba_frame, &mut self.yuv_frame)
.map_err(|e| format!("YUV422P10 swscale conversion failed: {}", e))?;
// Extract each plane tight (2 bytes/sample). Y: width samples/row × height rows.
// Chroma (4:2:2): width/2 samples/row × height rows.
let extract = |frame: &ffmpeg::frame::Video, idx: usize, samples_w: usize, rows: usize| {
let bytes_per_row = samples_w * 2;
let stride = frame.stride(idx);
let data = frame.data(idx);
let mut out = Vec::with_capacity(bytes_per_row * rows);
for row in 0..rows {
let start = row * stride;
out.extend_from_slice(&data[start..start + bytes_per_row]);
}
out
};
let (w, h) = (self.width as usize, self.height as usize);
let y_plane = extract(&self.yuv_frame, 0, w, h);
let u_plane = extract(&self.yuv_frame, 1, w / 2, h);
let v_plane = extract(&self.yuv_frame, 2, w / 2, h);
Ok((y_plane, u_plane, v_plane))
}
}
#[cfg(test)]
mod tests {
use super::*;
@ -219,20 +131,6 @@ mod tests {
assert_eq!(v.len(), (1920 / 2) * (1080 / 2));
}
#[test]
fn test_yuv422p10_output_sizes() {
// Use a width that forces swscale linesize padding (not a multiple of 32/64) to exercise
// the stride-stripping extraction.
let (w, h) = (1000u32, 720u32);
let mut c = CpuYuv422P10Converter::new(w, h, false).unwrap();
let rgba = vec![0u8; (w * h * 4) as usize];
let (y, u, v) = c.convert(&rgba).unwrap();
// 10-bit → 2 bytes/sample. Y full res; U/V half width, full height (4:2:2).
assert_eq!(y.len(), (w * h * 2) as usize);
assert_eq!(u.len(), ((w / 2) * h * 2) as usize);
assert_eq!(v.len(), ((w / 2) * h * 2) as usize);
}
#[test]
#[should_panic(expected = "RGBA data size mismatch")]
fn test_wrong_input_size_panics() {

View File

@ -5,42 +5,11 @@
use eframe::egui;
use lightningbeam_core::export::{
AudioExportSettings, AudioFormat,
GifExportSettings,
ImageExportSettings, ImageFormat,
VideoExportSettings, VideoCodec, VideoQuality, ColorRange,
};
use std::path::PathBuf;
/// The OS username (`$USER` / `%USERNAME%`), or empty if unavailable. Used as a default Artist tag.
fn os_username() -> String {
std::env::var("USER")
.or_else(|_| std::env::var("USERNAME"))
.unwrap_or_default()
}
/// Current civil year (UTC) computed from the system clock — avoids pulling in a date crate.
fn current_year() -> i64 {
let secs = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs())
.unwrap_or(0) as i64;
year_from_unix_secs(secs)
}
/// Civil year (UTC) for a Unix timestamp, via Howard Hinnant's `civil_from_days` algorithm.
fn year_from_unix_secs(secs: i64) -> i64 {
let days = secs.div_euclid(86_400);
let z = days + 719_468;
let era = if z >= 0 { z } else { z - 146_096 } / 146_097;
let doe = z - era * 146_097; // [0, 146096]
let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365; // [0, 399]
let y = yoe + era * 400;
let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); // [0, 365]
let mp = (5 * doy + 2) / 153; // [0, 11]
let m = if mp < 10 { mp + 3 } else { mp - 9 }; // [1, 12]
y + if m <= 2 { 1 } else { 0 }
}
/// Hint about document content, used to pick a smart default export type.
pub struct DocumentHint {
pub has_video: bool,
@ -56,8 +25,6 @@ pub enum ExportType {
Audio,
Image,
Video,
/// Animated GIF (multi-frame, palette-quantized, no audio).
Gif,
/// Vector-only SVG of the current frame (lossless; raster/video layers skipped).
Svg,
}
@ -69,8 +36,6 @@ pub enum ExportResult {
Image(ImageExportSettings, PathBuf),
VideoOnly(VideoExportSettings, PathBuf),
VideoWithAudio(VideoExportSettings, AudioExportSettings, PathBuf),
/// Animated GIF export.
Gif(GifExportSettings, PathBuf),
/// SVG of vector layers at the given document time.
Svg(f64, PathBuf),
}
@ -92,9 +57,6 @@ pub struct ExportDialog {
/// Video export settings
pub video_settings: VideoExportSettings,
/// Animated GIF export settings
pub gif_settings: GifExportSettings,
/// Include audio with video?
pub include_audio: bool,
@ -142,7 +104,6 @@ impl Default for ExportDialog {
audio_settings: AudioExportSettings::standard_mp3(),
image_settings: ImageExportSettings::default(),
video_settings: VideoExportSettings::default(),
gif_settings: GifExportSettings::default(),
include_audio: true,
output_path: None,
error_message: None,
@ -159,18 +120,10 @@ impl Default for ExportDialog {
impl ExportDialog {
/// Open the dialog with default settings, using `hint` to pick a smart default tab.
pub fn open(
&mut self,
timeline_duration: f64,
project_name: &str,
hint: &DocumentHint,
last_artist: &str,
last_album: &str,
) {
pub fn open(&mut self, timeline_duration: f64, project_name: &str, hint: &DocumentHint) {
self.open = true;
self.audio_settings.end_time = timeline_duration;
self.video_settings.end_time = timeline_duration;
self.gif_settings.end_time = timeline_duration;
self.image_settings.time = hint.current_time;
// Propagate document dimensions as defaults (None means "use doc size").
self.image_settings.width = None;
@ -190,24 +143,6 @@ impl ExportDialog {
else if only_raster { ExportType::Image }
else { self.export_type } // keep current as fallback
};
// Sensible tag defaults, only filled when empty so a user's edits are never clobbered:
// • Title → project name (on a project switch)
// • Year → current year
// • Artist → last-used artist, else the OS username
// • Album → last-used album
let meta = &mut self.audio_settings.metadata;
if meta.title.is_empty() && !same_project {
meta.title = project_name.to_owned();
}
if meta.year.is_empty() {
meta.year = current_year().to_string();
}
if meta.artist.is_empty() {
meta.artist = if !last_artist.is_empty() { last_artist.to_owned() } else { os_username() };
}
if meta.album.is_empty() && !last_album.is_empty() {
meta.album = last_album.to_owned();
}
self.current_project = project_name.to_owned();
// Restore the last exported path if available; otherwise default to project name.
@ -225,7 +160,6 @@ impl ExportDialog {
ExportType::Audio => self.audio_settings.format.extension(),
ExportType::Image => self.image_settings.format.extension(),
ExportType::Video => self.video_settings.codec.container_format(),
ExportType::Gif => "gif",
ExportType::Svg => "svg",
}
}
@ -269,7 +203,6 @@ impl ExportDialog {
ExportType::Audio => "Export Audio",
ExportType::Image => "Export Image",
ExportType::Video => "Export Video",
ExportType::Gif => "Export GIF",
ExportType::Svg => "Export SVG",
};
@ -292,7 +225,6 @@ impl ExportDialog {
(ExportType::Audio, "Audio"),
(ExportType::Image, "Image"),
(ExportType::Video, "Video"),
(ExportType::Gif, "GIF"),
(ExportType::Svg, "SVG"),
] {
if ui.selectable_value(&mut self.export_type, variant, label).clicked() {
@ -310,7 +242,6 @@ impl ExportDialog {
ExportType::Audio => self.render_audio_basic(ui),
ExportType::Image => self.render_image_settings(ui),
ExportType::Video => self.render_video_basic(ui),
ExportType::Gif => self.render_gif_basic(ui),
ExportType::Svg => self.render_svg_settings(ui),
}
@ -330,7 +261,6 @@ impl ExportDialog {
ExportType::Audio => self.render_audio_advanced(ui),
ExportType::Image => self.render_image_advanced(ui),
ExportType::Video => self.render_video_advanced(ui),
ExportType::Gif => self.render_gif_advanced(ui),
ExportType::Svg => {} // SVG has no advanced settings
}
}
@ -530,50 +460,10 @@ impl ExportDialog {
ui.add_space(8.0);
// Tag metadata (ID3 / MP4 / Vorbis / RIFF-INFO depending on format).
self.render_audio_metadata(ui);
ui.add_space(8.0);
// Time range
self.render_time_range(ui);
}
/// Render the audio tag-metadata fields (title/artist/album/…). Written into the file on export.
fn render_audio_metadata(&mut self, ui: &mut egui::Ui) {
let m = &mut self.audio_settings.metadata;
ui.label(egui::RichText::new("Tags").strong());
// Placeholder styling: italic + a clearly faded color so an empty field's hint never reads
// as a real value (the theme's default weak color is too close to the text color). An
// explicit color overrides egui's weak-color fallback for hint text.
let hint_color = {
let t = ui.visuals().text_color();
egui::Color32::from_rgba_unmultiplied(t.r(), t.g(), t.b(), 100)
};
let year_hint = format!("e.g. {}", current_year());
egui::Grid::new("audio_metadata_grid")
.num_columns(2)
.spacing([8.0, 4.0])
.show(ui, |ui| {
let row = |ui: &mut egui::Ui, label: &str, val: &mut String, hint: &str| {
ui.label(label);
ui.add(
egui::TextEdit::singleline(val)
.hint_text(egui::RichText::new(hint).italics().color(hint_color))
.desired_width(260.0),
);
ui.end_row();
};
row(ui, "Title", &mut m.title, "e.g. My Song");
row(ui, "Artist", &mut m.artist, "e.g. Jane Doe");
row(ui, "Album", &mut m.album, "e.g. Greatest Hits");
row(ui, "Genre", &mut m.genre, "e.g. Electronic");
row(ui, "Year", &mut m.year, &year_hint);
row(ui, "Track", &mut m.track, "e.g. 1 or 1/12");
row(ui, "Comment", &mut m.comment, "Optional notes…");
});
}
/// Video presets: (name, codec, quality, width, height, fps)
const VIDEO_PRESETS: &'static [(&'static str, VideoCodec, VideoQuality, u32, u32, f64)] = &[
("1080p H.264 (Standard)", VideoCodec::H264, VideoQuality::High, 1920, 1080, 30.0),
@ -724,65 +614,12 @@ impl ExportDialog {
self.render_time_range(ui);
}
/// GIF frame-rate presets (fps). GIF delays are centisecond-quantized, so these map to clean
/// per-frame delays (10/15/20/25/50 fps → 100/70/50/40/20 ms after rounding).
const GIF_FPS: &'static [f64] = &[10.0, 15.0, 20.0, 25.0, 50.0];
/// Render basic GIF settings (frame rate + loop).
fn render_gif_basic(&mut self, ui: &mut egui::Ui) {
ui.horizontal(|ui| {
ui.label("Frame rate:");
egui::ComboBox::from_id_salt("gif_fps")
.selected_text(format!("{} fps", self.gif_settings.framerate as u32))
.show_ui(ui, |ui| {
for &fps in Self::GIF_FPS {
ui.selectable_value(&mut self.gif_settings.framerate, fps, format!("{} fps", fps as u32));
}
});
});
ui.checkbox(&mut self.gif_settings.loop_forever, "Loop forever");
ui.add_space(8.0);
self.render_time_range(ui);
}
/// Render advanced GIF settings (resolution, fit, transparency).
fn render_gif_advanced(&mut self, ui: &mut egui::Ui) {
ui.horizontal(|ui| {
ui.label("Size:");
let mut w = self.gif_settings.width.unwrap_or(0);
let mut h = self.gif_settings.height.unwrap_or(0);
let changed_w = ui.add(egui::DragValue::new(&mut w).range(0..=u32::MAX).prefix("W ")).changed();
let changed_h = ui.add(egui::DragValue::new(&mut h).range(0..=u32::MAX).prefix("H ")).changed();
if changed_w { self.gif_settings.width = if w == 0 { None } else { Some(w) }; }
if changed_h { self.gif_settings.height = if h == 0 { None } else { Some(h) }; }
ui.weak("(0 = document size)");
});
ui.horizontal(|ui| {
use lightningbeam_core::export::ExportFitMode;
ui.label("Fit:");
egui::ComboBox::from_id_salt("gif_fit_mode")
.selected_text(self.gif_settings.fit.name())
.show_ui(ui, |ui| {
ui.selectable_value(&mut self.gif_settings.fit, ExportFitMode::Letterbox, ExportFitMode::Letterbox.name());
ui.selectable_value(&mut self.gif_settings.fit, ExportFitMode::Crop, ExportFitMode::Crop.name());
ui.selectable_value(&mut self.gif_settings.fit, ExportFitMode::Stretch, ExportFitMode::Stretch.name());
});
});
ui.checkbox(&mut self.gif_settings.transparency, "Preserve transparency (1-bit)");
ui.label(egui::RichText::new("GIF supports only on/off transparency; semi-transparent pixels are keyed out.").weak().small());
}
/// Render time range UI (common to both audio and video)
fn render_time_range(&mut self, ui: &mut egui::Ui) {
let (start_time, end_time) = match self.export_type {
ExportType::Audio => (&mut self.audio_settings.start_time, &mut self.audio_settings.end_time),
ExportType::Image | ExportType::Svg => return, // single time field, not a range
ExportType::Video => (&mut self.video_settings.start_time, &mut self.video_settings.end_time),
ExportType::Gif => (&mut self.gif_settings.start_time, &mut self.gif_settings.end_time),
};
ui.horizontal(|ui| {
@ -856,13 +693,6 @@ impl ExportDialog {
Some(ExportResult::Image(self.image_settings.clone(), output_path))
}
ExportType::Svg => Some(ExportResult::Svg(self.image_settings.time, output_path)),
ExportType::Gif => {
if let Err(err) = self.gif_settings.validate() {
self.error_message = Some(err);
return None;
}
Some(ExportResult::Gif(self.gif_settings.clone(), output_path))
}
ExportType::Audio => {
// Validate audio settings
if let Err(err) = self.audio_settings.validate() {
@ -1031,30 +861,3 @@ impl ExportProgressDialog {
should_cancel
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn year_from_unix_secs_known_values() {
assert_eq!(year_from_unix_secs(0), 1970); // Unix epoch
assert_eq!(year_from_unix_secs(946_684_800), 2000); // 2000-01-01
assert_eq!(year_from_unix_secs(1_735_689_600), 2025); // 2025-01-01
assert_eq!(year_from_unix_secs(1_767_225_599), 2025); // 2025-12-31 23:59:59
assert_eq!(year_from_unix_secs(1_767_225_600), 2026); // 2026-01-01
// Post-2038: these timestamps exceed i32::MAX (2_147_483_647) — and the last exceeds
// u32::MAX — so a 32-bit time_t would wrap here. i64 math handles them correctly.
assert_eq!(year_from_unix_secs(2_148_595_200), 2038); // 2038-02-01 (> i32::MAX)
assert_eq!(year_from_unix_secs(2_223_331_200), 2040); // 2040-06-15
assert_eq!(year_from_unix_secs(4_102_444_800), 2100); // 2100-01-01 (not a leap year)
assert_eq!(year_from_unix_secs(9_214_646_400), 2262); // 2262-01-01 (> u32::MAX)
}
#[test]
fn current_year_is_plausible() {
let y = current_year();
assert!((2020..3000).contains(&y), "implausible year: {y}");
}
}

View File

@ -1,196 +0,0 @@
//! Animated GIF encoding.
//!
//! Palette-quantizes a stream of RGBA8 frames and writes them to a `.gif`. The expensive part —
//! per-frame NeuQuant 256-color quantization — is embarrassingly parallel (each frame gets its own
//! local palette), so it's fanned out across a worker pool. A single writer thread collects the
//! quantized frames, reorders them, and LZW-encodes them to the file in sequence.
//!
//! Pipeline (all off the UI thread):
//! ```text
//! UI render thread ──RGBA──▶ coordinator ──round-robin──▶ N quantizer workers
//! │ (idx, gif::Frame)
//! ▼
//! writer thread ──▶ .gif
//! ```
//! Rendering + readback happen on the UI thread (see `render_next_gif_frame`); this module owns
//! everything after a raw RGBA frame arrives.
use lightningbeam_core::export::ExportProgress;
use std::collections::HashMap;
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{channel, Receiver, Sender};
use std::sync::Arc;
/// Message from the UI (render) thread to the GIF encoder coordinator.
pub enum GifFrameMessage {
/// One RGBA8 frame (top-left origin, tightly packed `width*height*4` bytes).
Frame { frame_num: usize, pixels: Vec<u8> },
/// All frames have been sent.
Done,
}
/// gif crate quantization speed (1 = slowest/best, 30 = fastest/worst). 10 balances palette quality
/// against per-frame cost; the parallelism below is what actually recovers the wall-clock.
const QUANT_SPEED: i32 = 10;
/// Run the GIF encoder pipeline. Receives RGBA8 frames from `frame_rx`, quantizes them in parallel,
/// and writes the ordered result to `output_path`, reporting progress. `transparency == false`
/// composites each frame onto opaque black first (GIF's 1-bit transparency would otherwise key out
/// semi-transparent pixels).
#[allow(clippy::too_many_arguments)]
pub fn run_gif_encoder(
frame_rx: Receiver<GifFrameMessage>,
output_path: PathBuf,
width: u32,
height: u32,
total_frames: usize,
delay_ms: u32,
loop_forever: bool,
transparency: bool,
progress_tx: Sender<ExportProgress>,
cancel_flag: Arc<AtomicBool>,
) {
let _ = progress_tx.send(ExportProgress::Started { total_frames });
let delay_cs = ((delay_ms / 10).max(1)) as u16;
let expected_len = (width as usize) * (height as usize) * 4;
// One quantizer worker per spare core (leave one for the UI render thread), capped so we don't
// spawn absurdly many for short exports.
let n_workers = std::thread::available_parallelism()
.map(|n| n.get().saturating_sub(1))
.unwrap_or(1)
.clamp(1, 8);
// Per-worker input channels (coordinator dispatches round-robin) + one shared result channel.
let mut worker_txs: Vec<Sender<(usize, Vec<u8>)>> = Vec::with_capacity(n_workers);
let (result_tx, result_rx) = channel::<(usize, gif::Frame<'static>)>();
let mut worker_handles = Vec::with_capacity(n_workers);
for _ in 0..n_workers {
let (wtx, wrx) = channel::<(usize, Vec<u8>)>();
worker_txs.push(wtx);
let result_tx = result_tx.clone();
let cancel = Arc::clone(&cancel_flag);
worker_handles.push(std::thread::spawn(move || {
while let Ok((idx, mut pixels)) = wrx.recv() {
if cancel.load(Ordering::Relaxed) {
break;
}
// NeuQuant local-palette quantization (the expensive step). `from_rgba_speed` uses
// the RGBA buffer as scratch, so it's fine that we own `pixels` here.
let mut frame =
gif::Frame::from_rgba_speed(width as u16, height as u16, &mut pixels, QUANT_SPEED);
frame.delay = delay_cs;
if result_tx.send((idx, frame)).is_err() {
break; // writer gone
}
}
}));
}
drop(result_tx); // only the workers hold senders now; writer's rx ends when they all finish
// Writer thread: order frames by index and LZW-encode them sequentially.
let writer_progress = progress_tx.clone();
let writer_cancel = Arc::clone(&cancel_flag);
let writer_output = output_path.clone();
let writer = std::thread::spawn(move || -> Result<(), String> {
let file = std::fs::File::create(&writer_output)
.map_err(|e| format!("Failed to create GIF file: {e}"))?;
let mut buf = std::io::BufWriter::new(file);
let mut encoder = gif::Encoder::new(&mut buf, width as u16, height as u16, &[])
.map_err(|e| format!("GIF encoder init failed: {e}"))?;
if loop_forever {
encoder
.set_repeat(gif::Repeat::Infinite)
.map_err(|e| format!("GIF set_repeat failed: {e}"))?;
}
// Frames may arrive out of order; hold stragglers until their turn.
let mut pending: HashMap<usize, gif::Frame<'static>> = HashMap::new();
let mut next = 0usize;
let mut written = 0usize;
while let Ok((idx, frame)) = result_rx.recv() {
if writer_cancel.load(Ordering::Relaxed) {
break;
}
pending.insert(idx, frame);
while let Some(f) = pending.remove(&next) {
encoder
.write_frame(&f)
.map_err(|e| format!("GIF write_frame failed: {e}"))?;
next += 1;
written += 1;
let _ = writer_progress.send(ExportProgress::FrameRendered {
frame: written,
total: total_frames,
});
}
}
// Encoder/BufWriter flush on drop.
Ok(())
});
// Coordinator: pull RGBA frames from the UI thread and dispatch round-robin to the workers.
let mut dispatched = 0usize;
let mut fatal: Option<String> = None;
loop {
match frame_rx.recv() {
Ok(GifFrameMessage::Frame { frame_num, mut pixels }) => {
if cancel_flag.load(Ordering::Relaxed) {
break;
}
if pixels.len() != expected_len {
fatal = Some("GIF frame size mismatch".into());
break;
}
if !transparency {
// Premultiply onto opaque black, then force alpha opaque.
for px in pixels.chunks_exact_mut(4) {
let a = px[3] as u32;
px[0] = (px[0] as u32 * a / 255) as u8;
px[1] = (px[1] as u32 * a / 255) as u8;
px[2] = (px[2] as u32 * a / 255) as u8;
px[3] = 255;
}
}
let w = dispatched % n_workers;
if worker_txs[w].send((frame_num, pixels)).is_err() {
fatal = Some("GIF quantizer worker died".into());
break;
}
dispatched += 1;
}
Ok(GifFrameMessage::Done) => break,
Err(_) => break, // UI thread dropped the sender
}
}
let _ = progress_tx.send(ExportProgress::Finalizing);
// Close worker inputs → workers finish → their result senders drop → writer's loop ends.
drop(worker_txs);
for h in worker_handles {
let _ = h.join();
}
let writer_result = writer.join().unwrap_or_else(|_| Err("GIF writer thread panicked".into()));
if cancel_flag.load(Ordering::Relaxed) {
std::fs::remove_file(&output_path).ok();
// Emit Complete so the UI poll loop clears its state; the dialog was closed on cancel.
let _ = progress_tx.send(ExportProgress::Complete { output_path });
return;
}
match fatal.or_else(|| writer_result.err()) {
Some(message) => {
std::fs::remove_file(&output_path).ok();
let _ = progress_tx.send(ExportProgress::Error { message });
}
None => {
let _ = progress_tx.send(ExportProgress::Complete { output_path });
}
}
}

View File

@ -175,9 +175,7 @@ impl GpuYuv {
}
/// CPU reference for the exact math/layout the shader produces — used by unit tests so
/// the packing and BT.709 coefficients stay verifiable without a GPU. Test-only, so it isn't
/// compiled into (and flagged as unused by) release builds.
#[cfg(test)]
/// the packing and BT.709 coefficients stay verifiable without a GPU.
fn cpu_reference(rgba: &[u8], width: u32, height: u32, full_range: bool) -> Vec<u8> {
let w = width as usize;
let h = height as usize;

View File

@ -45,179 +45,13 @@ pub fn save_rgba_image(
encoder.encode_image(&rgb_img).map_err(|e| format!("JPEG encode failed: {e}"))
}
ImageFormat::WebP => {
// `image` 0.25's WebP encoder is lossless-only, which ignored the quality slider and
// produced needlessly large files. Encode lossy WebP via ffmpeg's libwebp instead so
// the quality control is real; alpha is preserved (as YUVA420P) when requested.
save_webp_ffmpeg(pixels, width, height, quality, allow_transparency, path)
}
}
}
/// Encode a single frame as lossy WebP via ffmpeg's `libwebp` encoder.
///
/// `quality` is libwebp's 0100 quality factor. When `allow_transparency` is true the source is
/// converted to YUVA420P so libwebp keeps the alpha channel; otherwise it's flattened onto black
/// and converted to YUV420P. Uses swscale's default BT.601 conversion (matching a plain
/// `ffmpeg -i in.png out.webp`).
fn save_webp_ffmpeg(
pixels: &[u8],
width: u32,
height: u32,
quality: u8,
allow_transparency: bool,
path: &Path,
) -> Result<(), String> {
use ffmpeg_next as ffmpeg;
ffmpeg::init().map_err(|e| format!("Failed to initialize ffmpeg: {e}"))?;
let codec = ffmpeg::encoder::find_by_name("libwebp")
.or_else(|| ffmpeg::encoder::find(ffmpeg::codec::Id::WEBP))
.ok_or("libwebp encoder not available in this ffmpeg build")?;
// Flatten onto black up front when alpha isn't wanted, so the source is fully opaque.
let src_rgba: Vec<u8> = if allow_transparency {
pixels.to_vec()
} else {
let mut v = pixels.to_vec();
for px in v.chunks_exact_mut(4) {
let a = px[3] as u32;
px[0] = (px[0] as u32 * a / 255) as u8;
px[1] = (px[1] as u32 * a / 255) as u8;
px[2] = (px[2] as u32 * a / 255) as u8;
px[3] = 255;
}
v
};
let dst_pix = if allow_transparency {
ffmpeg::format::Pixel::YUVA420P
} else {
ffmpeg::format::Pixel::YUV420P
};
// RGBA → YUV(A)420P (swscale defaults: BT.601, limited range — what libwebp expects).
let mut scaler = ffmpeg::software::scaling::Context::get(
ffmpeg::format::Pixel::RGBA, width, height,
dst_pix, width, height,
ffmpeg::software::scaling::Flags::BILINEAR,
)
.map_err(|e| format!("Failed to create swscale context: {e}"))?;
let mut src = ffmpeg::frame::Video::new(ffmpeg::format::Pixel::RGBA, width, height);
// Copy row-by-row honoring the frame's stride (may exceed width*4 due to alignment padding).
let stride = src.stride(0);
let row_bytes = (width * 4) as usize;
{
let dst = src.data_mut(0);
for y in 0..height as usize {
let s = y * row_bytes;
let d = y * stride;
dst[d..d + row_bytes].copy_from_slice(&src_rgba[s..s + row_bytes]);
}
}
let mut yuv = ffmpeg::frame::Video::new(dst_pix, width, height);
scaler.run(&src, &mut yuv).map_err(|e| format!("swscale conversion failed: {e}"))?;
yuv.set_pts(Some(0));
let mut octx = ffmpeg::format::output(&path)
.map_err(|e| format!("Failed to create WebP output: {e}"))?;
let mut enc = ffmpeg::codec::Context::new_with_codec(codec)
.encoder()
.video()
.map_err(|e| format!("Failed to create WebP encoder: {e}"))?;
enc.set_width(width);
enc.set_height(height);
enc.set_format(dst_pix);
enc.set_time_base(ffmpeg::Rational(1, 1));
// libwebp private options: quality 0100, lossy.
let mut opts = ffmpeg::Dictionary::new();
opts.set("quality", &quality.to_string());
opts.set("lossless", "0");
let mut enc = enc
.open_with(opts)
.map_err(|e| format!("Failed to open libwebp encoder: {e}"))?;
{
let mut stream = octx.add_stream(codec)
.map_err(|e| format!("Failed to add WebP stream: {e}"))?;
stream.set_parameters(&enc);
stream.set_time_base(ffmpeg::Rational(1, 1));
}
octx.write_header().map_err(|e| format!("Failed to write WebP header: {e}"))?;
enc.send_frame(&yuv).map_err(|e| format!("Failed to send WebP frame: {e}"))?;
enc.send_eof().map_err(|e| format!("Failed to flush WebP encoder: {e}"))?;
let mut packet = ffmpeg::Packet::empty();
while enc.receive_packet(&mut packet).is_ok() {
packet.set_stream(0);
packet
.write_interleaved(&mut octx)
.map_err(|e| format!("Failed to write WebP packet: {e}"))?;
}
octx.write_trailer().map_err(|e| format!("Failed to finalize WebP: {e}"))?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use lightningbeam_core::export::ImageFormat;
/// A gradient RGBA image so the encoder has real content to quantize/compress.
fn gradient(width: u32, height: u32) -> Vec<u8> {
let mut px = Vec::with_capacity((width * height * 4) as usize);
for y in 0..height {
for x in 0..width {
px.push((x * 255 / width.max(1)) as u8);
px.push((y * 255 / height.max(1)) as u8);
px.push(128);
px.push(255);
if allow_transparency {
img.save(path).map_err(|e| format!("WebP save failed: {e}"))
} else {
let flat = flatten_alpha(img);
flat.save(path).map_err(|e| format!("WebP save failed: {e}"))
}
}
px
}
/// The ffmpeg libwebp path must produce a valid *lossy* WebP (RIFF/WEBP container with a
/// `VP8 ` chunk — lossless would be `VP8L`), and the quality knob must actually change size.
#[test]
fn webp_export_is_real_lossy() {
let (w, h) = (96u32, 64u32);
let px = gradient(w, h);
let dir = std::env::temp_dir();
let lo = dir.join("lb_webp_q10_test.webp");
let hi = dir.join("lb_webp_q95_test.webp");
save_webp_ffmpeg(&px, w, h, 10, false, &lo).expect("low-quality webp encode");
save_webp_ffmpeg(&px, w, h, 95, false, &hi).expect("high-quality webp encode");
let lo_bytes = std::fs::read(&lo).unwrap();
let hi_bytes = std::fs::read(&hi).unwrap();
// RIFF....WEBP container.
assert_eq!(&lo_bytes[0..4], b"RIFF", "not a RIFF container");
assert_eq!(&lo_bytes[8..12], b"WEBP", "not a WEBP file");
// Lossy VP8 chunk (`VP8 ` with trailing space), NOT lossless `VP8L`.
assert_eq!(&lo_bytes[12..16], b"VP8 ", "expected lossy VP8, got {:?}", &lo_bytes[12..16]);
// The quality knob is honored: q10 is meaningfully smaller than q95.
assert!(lo_bytes.len() < hi_bytes.len(),
"quality ignored: q10 {} bytes >= q95 {} bytes", lo_bytes.len(), hi_bytes.len());
std::fs::remove_file(&lo).ok();
std::fs::remove_file(&hi).ok();
}
/// The format enum still advertises a quality control for WebP (now that it works).
#[test]
fn webp_has_quality() {
assert!(ImageFormat::WebP.has_quality());
assert!(ImageFormat::Jpeg.has_quality());
assert!(!ImageFormat::Png.has_quality());
}
}

View File

@ -3,8 +3,8 @@
//! This module provides the export orchestrator and progress tracking
//! for exporting audio and video from the timeline.
pub mod audio_exporter;
pub mod dialog;
pub mod gif_exporter;
pub mod image_exporter;
pub mod video_exporter;
pub mod readback_pipeline;
@ -13,8 +13,7 @@ pub mod cpu_yuv_converter;
pub mod gpu_yuv;
pub mod hdr_frame;
use lightningbeam_core::export::{AudioExportSettings, GifExportSettings, ImageExportSettings, VideoExportSettings, ExportProgress};
use gif_exporter::GifFrameMessage;
use lightningbeam_core::export::{AudioExportSettings, ImageExportSettings, VideoExportSettings, ExportProgress};
use lightningbeam_core::document::Document;
use lightningbeam_core::renderer::ImageCache;
use lightningbeam_core::video::VideoManager;
@ -69,11 +68,6 @@ pub struct VideoExportState {
readback_pipeline: Option<readback_pipeline::ReadbackPipeline>,
/// CPU YUV converter for RGBA→YUV420p conversion
cpu_yuv_converter: Option<cpu_yuv_converter::CpuYuvConverter>,
/// ProRes 422 export: forces the CPU (RGBA) readback path and converts to 10-bit 4:2:2 instead
/// of 8-bit 4:2:0. `true` only for `VideoCodec::ProRes422` (SDR).
prores: bool,
/// CPU RGBA→YUV422P10LE converter, used only on the ProRes path.
cpu_yuv422p10: Option<cpu_yuv_converter::CpuYuv422P10Converter>,
/// Frames that have been submitted to GPU but not yet encoded
frames_in_flight: usize,
/// Next frame number to send to encoder (for ordering)
@ -137,38 +131,6 @@ pub struct ExportOrchestrator {
/// Single-frame image export state
image_state: Option<ImageExportState>,
/// Animated GIF export state (frames rendered on the UI thread, encoded on `thread_handle`).
gif_state: Option<GifExportState>,
}
/// State for an in-progress animated GIF export. Frames are rendered + read back on the UI thread
/// (one per `render_next_gif_frame` call) and streamed to the encoder thread over `frame_tx`.
struct GifExportState {
/// Resolved pixel dimensions (after applying any width/height overrides).
width: u32,
height: u32,
/// Total frames to render.
total_frames: usize,
/// Next frame index to render (0-based).
next_frame: usize,
/// Document time (seconds) of frame 0.
start_time: f64,
/// Seconds between frames (1 / framerate).
frame_step: f64,
/// How the document is fit into the export frame.
fit: lightningbeam_core::export::ExportFitMode,
/// Preserve alpha as GIF transparency (else the encoder flattens onto black).
transparency: bool,
/// GPU resources allocated on the first render call, reused each frame.
gpu_resources: Option<video_exporter::ExportGpuResources>,
/// Output RGBA texture (kept separate from gpu_resources to avoid split-borrow issues).
output_texture: Option<wgpu::Texture>,
output_texture_view: Option<wgpu::TextureView>,
/// Staging buffer for synchronous GPU→CPU readback (reused each frame).
staging_buffer: Option<wgpu::Buffer>,
/// Sender to the encoder thread; dropped after the final frame to signal completion.
frame_tx: Option<Sender<GifFrameMessage>>,
}
/// State for parallel audio+video export
@ -206,7 +168,6 @@ impl ExportOrchestrator {
video_state: None,
parallel_export: None,
image_state: None,
gif_state: None,
}
}
@ -289,7 +250,7 @@ impl ExportOrchestrator {
/// unconsumed terminal message). Used to gate the UI poll loop so it doesn't run every
/// repaint forever after an export finishes.
pub fn has_pending_progress(&self) -> bool {
self.parallel_export.is_some() || self.image_state.is_some() || self.gif_state.is_some() || self.progress_rx.is_some()
self.parallel_export.is_some() || self.image_state.is_some() || self.progress_rx.is_some()
}
/// Poll progress for parallel video+audio export
@ -573,9 +534,6 @@ impl ExportOrchestrator {
}
self.video_state = None;
self.image_state = None;
// Dropping gif_state drops its frame sender, unblocking the encoder thread's recv() so it
// observes the cancel flag, removes the partial file, and exits.
self.gif_state = None;
self.progress_rx = None;
self.thread_handle = None;
}
@ -584,7 +542,6 @@ impl ExportOrchestrator {
pub fn is_exporting(&self) -> bool {
if self.parallel_export.is_some() { return true; }
if self.image_state.is_some() { return true; }
if self.gif_state.is_some() { return true; }
if let Some(handle) = &self.thread_handle {
!handle.is_finished()
} else {
@ -762,196 +719,6 @@ impl ExportOrchestrator {
result.map(|_| true)
}
/// Enqueue an animated GIF export. Spawns the encoder thread now; call `render_next_gif_frame()`
/// from the egui update loop (where the wgpu device/queue are available) to render + stream each
/// frame to it.
pub fn start_gif_export(
&mut self,
settings: GifExportSettings,
output_path: PathBuf,
doc_width: u32,
doc_height: u32,
) {
self.cancel_flag.store(false, Ordering::Relaxed);
let width = settings.width.unwrap_or(doc_width).max(1);
let height = settings.height.unwrap_or(doc_height).max(1);
let total_frames = settings.total_frames();
let delay_ms = settings.frame_delay_ms();
let frame_step = 1.0 / settings.framerate;
let (progress_tx, progress_rx) = channel();
let (frame_tx, frame_rx) = channel();
self.progress_rx = Some(progress_rx);
let cancel_flag = Arc::clone(&self.cancel_flag);
let loop_forever = settings.loop_forever;
let transparency = settings.transparency;
let handle = std::thread::spawn(move || {
gif_exporter::run_gif_encoder(
frame_rx, output_path, width, height, total_frames, delay_ms,
loop_forever, transparency, progress_tx, cancel_flag,
);
});
self.thread_handle = Some(handle);
self.gif_state = Some(GifExportState {
width,
height,
total_frames,
next_frame: 0,
start_time: settings.start_time,
frame_step,
fit: settings.fit,
transparency,
gpu_resources: None,
output_texture: None,
output_texture_view: None,
staging_buffer: None,
frame_tx: Some(frame_tx),
});
}
/// Drive the animated GIF export: render + read back one frame per call and stream it to the
/// encoder thread. Returns `Ok(true)` while more frames remain (call again next egui frame),
/// `Ok(false)` once every frame has been sent (encoding then finishes on the background thread).
pub fn render_next_gif_frame(
&mut self,
document: &mut Document,
device: &wgpu::Device,
queue: &wgpu::Queue,
renderer: &mut vello::Renderer,
image_cache: &mut ImageCache,
video_manager: &Arc<std::sync::Mutex<VideoManager>>,
raster_store: Option<&lightningbeam_core::raster_store::RasterStore>,
) -> Result<bool, String> {
if self.cancel_flag.load(Ordering::Relaxed) {
// Dropping frame_tx unblocks the encoder thread's recv() so it can clean up.
self.gif_state = None;
return Ok(false);
}
let state = match self.gif_state.as_mut() {
Some(s) => s,
None => return Ok(false),
};
// All frames sent → drop the sender (signals the encoder to finalize) and finish.
if state.next_frame >= state.total_frames {
if let Some(tx) = state.frame_tx.take() {
let _ = tx.send(GifFrameMessage::Done);
drop(tx);
}
self.gif_state = None;
return Ok(false);
}
let w = state.width;
let h = state.height;
let fit = state.fit;
let timestamp = state.start_time + state.next_frame as f64 * state.frame_step;
if state.gpu_resources.is_none() {
state.gpu_resources = Some(video_exporter::ExportGpuResources::new(device, w, h));
}
if state.output_texture.is_none() {
let tex = device.create_texture(&wgpu::TextureDescriptor {
label: Some("gif_export_output"),
size: wgpu::Extent3d { width: w, height: h, depth_or_array_layers: 1 },
mip_level_count: 1,
sample_count: 1,
dimension: wgpu::TextureDimension::D2,
format: wgpu::TextureFormat::Rgba8Unorm,
usage: wgpu::TextureUsages::RENDER_ATTACHMENT | wgpu::TextureUsages::COPY_SRC,
view_formats: &[],
});
state.output_texture_view = Some(tex.create_view(&wgpu::TextureViewDescriptor::default()));
state.output_texture = Some(tex);
}
// Render the frame (transparency preserved through readback; the encoder flattens if needed).
{
let gpu = state.gpu_resources.as_mut().unwrap();
let output_view = state.output_texture_view.as_ref().unwrap();
let encoder = video_exporter::render_frame_to_gpu_rgba(
document,
timestamp,
w, h,
device, queue, renderer, image_cache, video_manager,
gpu,
output_view,
None, // no floating raster selection during export
state.transparency,
raster_store,
true, // GIF export composites on the shared device
fit,
)?;
queue.submit(Some(encoder.finish()));
}
// Synchronous readback (wgpu requires bytes_per_row aligned to 256).
let align = wgpu::COPY_BYTES_PER_ROW_ALIGNMENT;
let bytes_per_row = (w * 4 + align - 1) / align * align;
if state.staging_buffer.is_none() {
state.staging_buffer = Some(device.create_buffer(&wgpu::BufferDescriptor {
label: Some("gif_export_staging"),
size: (bytes_per_row * h) as u64,
usage: wgpu::BufferUsages::COPY_DST | wgpu::BufferUsages::MAP_READ,
mapped_at_creation: false,
}));
}
let staging = state.staging_buffer.as_ref().unwrap();
let mut copy_enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor {
label: Some("gif_export_copy"),
});
let output_tex = state.output_texture.as_ref().unwrap();
copy_enc.copy_texture_to_buffer(
wgpu::TexelCopyTextureInfo {
texture: output_tex,
mip_level: 0,
origin: wgpu::Origin3d::ZERO,
aspect: wgpu::TextureAspect::All,
},
wgpu::TexelCopyBufferInfo {
buffer: staging,
layout: wgpu::TexelCopyBufferLayout {
offset: 0,
bytes_per_row: Some(bytes_per_row),
rows_per_image: Some(h),
},
},
wgpu::Extent3d { width: w, height: h, depth_or_array_layers: 1 },
);
queue.submit(Some(copy_enc.finish()));
let slice = staging.slice(..);
slice.map_async(wgpu::MapMode::Read, |_| {});
let _ = device.poll(wgpu::PollType::wait_indefinitely());
let pixels: Vec<u8> = {
let mapped = slice.get_mapped_range();
let mut out = Vec::with_capacity((w * h * 4) as usize);
for row in 0..h {
let start = (row * bytes_per_row) as usize;
out.extend_from_slice(&mapped[start..start + (w * 4) as usize]);
}
out
};
staging.unmap();
let frame_num = state.next_frame;
if let Some(tx) = state.frame_tx.as_ref() {
// If the encoder thread died, stop — its dropped receiver returns an error here.
if tx.send(GifFrameMessage::Frame { frame_num, pixels }).is_err() {
self.gif_state = None;
return Ok(false);
}
}
state.next_frame += 1;
Ok(true)
}
/// Wait for the export to complete
///
/// This blocks until the export thread finishes.
@ -1007,12 +774,6 @@ impl ExportOrchestrator {
start_time: daw_backend::Seconds(settings.start_time),
end_time: daw_backend::Seconds(settings.end_time),
tempo_map: daw_backend::TempoMap::constant(settings.bpm),
metadata: settings
.metadata
.pairs()
.into_iter()
.map(|(k, v)| (k.to_string(), v.to_string()))
.collect(),
};
// Use DAW backend export for all formats
@ -1111,8 +872,6 @@ impl ExportOrchestrator {
let hdr = settings.hdr;
let fit = settings.fit;
let full_range = settings.color_range.is_full();
let prores = matches!(settings.codec, lightningbeam_core::export::VideoCodec::ProRes422)
&& !hdr.is_hdr();
let handle = std::thread::spawn(move || {
Self::run_video_encoder(settings, output_path, frame_rx, progress_tx, cancel_flag, total_frames);
});
@ -1132,8 +891,6 @@ impl ExportOrchestrator {
gpu_resources: None,
readback_pipeline: None,
cpu_yuv_converter: None,
prores,
cpu_yuv422p10: None,
frames_in_flight: 0,
next_frame_to_encode: 0,
perf_metrics: Some(perf_metrics::ExportMetrics::new()),
@ -1364,10 +1121,7 @@ impl ExportOrchestrator {
.unwrap()
.as_secs();
// Use the codec's real container for the temp video, not a hardcoded .mp4 — VP8 isn't a
// valid MP4 codec, so an .mp4 temp made `write_header` fail for any VP8+audio export.
let temp_video_path = temp_dir.join(format!("lightningbeam_video_{}.{}",
timestamp, video_settings.codec.container_format()));
let temp_video_path = temp_dir.join(format!("lightningbeam_video_{}.mp4", timestamp));
let temp_audio_path = temp_dir.join(format!("lightningbeam_audio_{}.{}",
timestamp,
match audio_settings.format {
@ -1577,34 +1331,24 @@ impl ExportOrchestrator {
// Enable GPU YUV only when the encoder's YUV420P planes are tight (no linesize
// padding) — then the packed GPU planes copy in without row misalignment.
// Otherwise fall back to RGBA readback + CPU swscale.
// ProRes needs 10-bit 4:2:2 (built on the CPU from the RGBA readback), so it forces the
// RGBA path — the GPU YUV converter only produces 8-bit 4:2:0.
let gpu_yuv_tight = !state.prores && std::env::var("LB_DISABLE_GPU_YUV").is_err() && {
let gpu_yuv_tight = std::env::var("LB_DISABLE_GPU_YUV").is_err() && {
let probe = ffmpeg_next::frame::Video::new(
ffmpeg_next::format::Pixel::YUV420P, width, height,
);
probe.stride(0) == width as usize && probe.stride(1) == (width / 2) as usize
};
if !gpu_yuv_tight && !state.prores {
if !gpu_yuv_tight {
println!("🎬 [VIDEO EXPORT] YUV planes are padded at {width}x{height}; using CPU YUV path");
}
state.readback_pipeline = Some(readback_pipeline::ReadbackPipeline::new(device, queue, width, height, gpu_yuv_tight, state.full_range));
if state.prores {
state.cpu_yuv422p10 = Some(cpu_yuv_converter::CpuYuv422P10Converter::new(width, height, state.full_range)?);
println!("🎬 [VIDEO EXPORT] ProRes 422: 10-bit 4:2:2 (YUV422P10LE) CPU converter initialized");
} else {
state.cpu_yuv_converter = Some(cpu_yuv_converter::CpuYuvConverter::new(width, height, state.full_range)?);
}
state.cpu_yuv_converter = Some(cpu_yuv_converter::CpuYuvConverter::new(width, height, state.full_range)?);
println!("🚀 [ASYNC PIPELINE] Triple-buffered pipeline initialized");
println!("🚀 [CPU YUV] swscale converter initialized");
}
let pipeline = state.readback_pipeline.as_mut().unwrap();
let gpu_resources = state.gpu_resources.as_mut().unwrap();
// Exactly one of these is present: cpu_yuv422p10 on the ProRes path, cpu_converter on the
// SDR fallback path (or neither is used when the GPU YUV converter is active).
let mut cpu_converter = state.cpu_yuv_converter.as_mut();
let mut cpu_yuv422p10 = state.cpu_yuv422p10.as_mut();
let cpu_converter = state.cpu_yuv_converter.as_mut().unwrap();
let mut metrics = state.perf_metrics.as_mut();
// Poll for completed async readbacks (non-blocking)
@ -1631,17 +1375,12 @@ impl ExportOrchestrator {
let data = pipeline.extract_rgba_data(result.buffer_id);
let extraction_end = Instant::now();
// YUV planes: ProRes 10-bit 4:2:2, else GPU-converted (just slice), else CPU
// swscale 8-bit 4:2:0 fallback (timed).
// YUV planes: GPU-converted (just slice) or CPU swscale fallback (timed).
let conversion_start = Instant::now();
let (y, u, v) = if let Some(conv) = cpu_yuv422p10.as_deref_mut() {
conv.convert(&data)?
} else if pipeline.is_yuv_mode() {
let (y, u, v) = if pipeline.is_yuv_mode() {
pipeline.split_yuv(&data)
} else {
cpu_converter.as_deref_mut()
.ok_or("SDR export missing its CPU YUV converter")?
.convert(&data)?
cpu_converter.convert(&data)?
};
let conversion_end = Instant::now();
@ -1730,7 +1469,6 @@ impl ExportOrchestrator {
state.gpu_resources = None;
state.readback_pipeline = None;
state.cpu_yuv_converter = None;
state.cpu_yuv422p10 = None;
state.perf_metrics = None;
return Ok(false);
}
@ -1980,8 +1718,6 @@ impl ExportOrchestrator {
// Pixel format the encoder frames are built in (matches setup_video_encoder).
let pixel_format = if settings.hdr.is_hdr() {
ffmpeg_next::format::Pixel::YUV420P10LE
} else if matches!(settings.codec, VideoCodec::ProRes422) {
ffmpeg_next::format::Pixel::YUV422P10LE // ProRes 422: 10-bit 4:2:2
} else {
ffmpeg_next::format::Pixel::YUV420P
};
@ -2096,17 +1832,8 @@ impl ExportOrchestrator {
// Copy each plane row-by-row honoring the frame's stride (10-bit / arbitrary widths can have
// row padding that a flat copy would misalign). `bytes_per_row` = samples × sample size.
// Sample size + chroma subsampling depend on the pixel format:
// YUV420P → 8-bit, 4:2:0 (chroma = w/2 × h/2)
// YUV420P10LE → 10-bit, 4:2:0 (chroma = w/2 × h/2)
// YUV422P10LE → 10-bit, 4:2:2 (chroma = w/2 × h, full-height) [ProRes]
use ffmpeg_next::format::Pixel;
let (sample_bytes, chroma_h_div) = match pixel_format {
Pixel::YUV420P => (1usize, 2usize),
Pixel::YUV420P10LE => (2usize, 2usize),
Pixel::YUV422P10LE => (2usize, 1usize),
_ => (1usize, 2usize),
};
let ten_bit = matches!(pixel_format, ffmpeg_next::format::Pixel::YUV420P10LE);
let sample_bytes = if ten_bit { 2usize } else { 1usize };
let copy_plane = |frame: &mut ffmpeg_next::frame::Video, idx: usize, src: &[u8], w: usize, h: usize| {
let bytes_per_row = w * sample_bytes;
let stride = frame.stride(idx);
@ -2121,8 +1848,8 @@ impl ExportOrchestrator {
};
let (w, h) = (width as usize, height as usize);
copy_plane(&mut video_frame, 0, y_plane, w, h);
copy_plane(&mut video_frame, 1, u_plane, w / 2, h / chroma_h_div);
copy_plane(&mut video_frame, 2, v_plane, w / 2, h / chroma_h_div);
copy_plane(&mut video_frame, 1, u_plane, w / 2, h / 2);
copy_plane(&mut video_frame, 2, v_plane, w / 2, h / 2);
// Set PTS (presentation timestamp) in encoder's time base
// Encoder time base is 1/(framerate * 1000), so PTS = timestamp * (framerate * 1000)

View File

@ -616,12 +616,9 @@ pub fn setup_video_encoder(
// Configure encoder parameters BEFORE opening (critical!)
encoder.set_width(aligned_width);
encoder.set_height(aligned_height);
// ProRes needs 10-bit 4:2:2; HDR needs 10-bit 4:2:0 BT.2020; other SDR is 8-bit 4:2:0.
let is_prores = codec_id == ffmpeg::codec::Id::PRORES;
// HDR encodes 10-bit BT.2020 (limited range); SDR keeps 8-bit full-range BT.709.
if hdr.is_hdr() {
encoder.set_format(ffmpeg::format::Pixel::YUV420P10LE);
} else if is_prores {
encoder.set_format(ffmpeg::format::Pixel::YUV422P10LE);
} else {
encoder.set_format(ffmpeg::format::Pixel::YUV420P);
}
@ -653,10 +650,6 @@ pub fn setup_video_encoder(
});
color_opts.set("color_primaries", "bt709");
color_opts.set("color_trc", "bt709");
if is_prores {
// prores_ks profile: 3 = HQ (4:2:2 10-bit). Matches the YUV422P10LE frames we feed.
color_opts.set("profile", "3");
}
}
println!("📐 Video dimensions: {}×{} (aligned to {}×{}){}",
@ -1062,18 +1055,10 @@ fn composite_document_to_hdr(
let success = gpu_resources.effect_processor.compile_effect(device, effect_def);
if !success { eprintln!("Failed to compile effect: {}", effect_def.name); continue; }
}
let tempo_map = document.tempo_map();
let effect_end_beats = effect_instance.timeline_start
+ effect_instance.effective_duration(
lightningbeam_core::clip::ClipDuration::Seconds(daw_backend::Seconds(
lightningbeam_core::effect::EFFECT_DURATION,
)),
tempo_map,
);
let effect_inst = lightningbeam_core::effect::EffectInstance::new(
effect_def,
tempo_map.beats_to_seconds(effect_instance.timeline_start).seconds_to_f64(),
tempo_map.beats_to_seconds(effect_end_beats).seconds_to_f64(),
effect_instance.timeline_start,
effect_instance.timeline_start + effect_instance.effective_duration(lightningbeam_core::effect::EFFECT_DURATION, document.tempo_map()),
);
let effect_output_handle = gpu_resources.buffer_pool.acquire(device, hdr_spec);
if let Some(effect_output_view) = gpu_resources.buffer_pool.get_view(effect_output_handle) {
@ -1448,30 +1433,6 @@ mod tests {
assert!(v[0] > 128, "V value: {}", v[0]);
}
/// ProRes must actually open with the 10-bit 4:2:2 format we now feed it. Before the fix the
/// SDR path handed prores_ks 8-bit YUV420P and `open` failed every time — so this opening
/// successfully is the regression guard for "ProRes export always errored".
#[test]
fn prores_encoder_opens_with_yuv422p10() {
ffmpeg::init().unwrap();
// Skip cleanly if this ffmpeg build lacks a ProRes encoder (rather than false-fail).
if ffmpeg::encoder::find(ffmpeg::codec::Id::PRORES).is_none()
&& ffmpeg::encoder::find_by_name("prores_ks").is_none()
{
eprintln!("prores encoder not present in this ffmpeg build; skipping");
return;
}
let r = setup_video_encoder(
ffmpeg::codec::Id::PRORES,
640, 480, 30.0, 20_000,
lightningbeam_core::export::HdrExportMode::Sdr,
false,
);
assert!(r.is_ok(), "ProRes encoder failed to open: {:?}", r.err());
let (encoder, _codec) = r.unwrap();
assert_eq!(encoder.format(), ffmpeg::format::Pixel::YUV422P10LE);
}
// NOTE: `rgba_to_yuv420p` rounds dimensions up to multiples of 16 (H.264
// macroblock alignment), so its plane lengths are the aligned sizes, not the
// tight input dimensions. The former `test_rgba_to_yuv420p_dimensions` and

File diff suppressed because it is too large Load Diff

View File

@ -42,8 +42,6 @@ pub const GRIP_HORIZONTAL: &str = "\u{e0ea}";
pub const CHEVRONS_UP: &str = "\u{e074}";
pub const PLAY: &str = "\u{e13c}";
pub const PAUSE: &str = "\u{e12e}";
pub const REPEAT: &str = "\u{e146}"; // cycle / loop region toggle
pub const TRASH: &str = "\u{e18d}"; // delete a take
pub const SETTINGS: &str = "\u{e154}";
pub const SEARCH: &str = "\u{e151}";
pub const PLUS: &str = "\u{e13d}";
@ -83,60 +81,3 @@ pub const FILE_PLUS: &str = "\u{e0c9}";
pub const COPY: &str = "\u{e09e}";
pub const LAYERS: &str = "\u{e529}";
pub const ELLIPSIS: &str = "\u{e0b6}";
// Undo / redo (stage header).
pub const UNDO_2: &str = "\u{e2a1}";
pub const REDO_2: &str = "\u{e2a0}";
// Tool cursors (see custom_cursor.rs).
pub const STAMP: &str = "\u{e3bb}";
pub const SPRAY_CAN: &str = "\u{e495}";
pub const BANDAGE: &str = "\u{e61d}";
pub const DROPLET: &str = "\u{e0b4}";
pub const TEXT_CURSOR: &str = "\u{e264}";
pub const CROSSHAIR: &str = "\u{e0ac}";
pub const POINTER: &str = "\u{e1e8}";
pub const CONTRAST: &str = "\u{e09d}";
pub const WIND: &str = "\u{e1b0}";
pub const SUN_MOON: &str = "\u{e2b2}";
pub const SPLINE: &str = "\u{e38b}";
pub const DROPLETS: &str = "\u{e0b5}";
pub const CIRCLE_DASHED: &str = "\u{e4b0}";
pub const PALETTE: &str = "\u{e1dd}";
pub const SHAPES: &str = "\u{e4b3}";
pub const SCALING: &str = "\u{e2ec}";
pub const FRAME: &str = "\u{e291}";
// Timeline layer-row toggles.
pub const VOLUME_2: &str = "\u{e1ab}"; // unmuted
pub const VOLUME_X: &str = "\u{e1ac}"; // muted
pub const HEADPHONES: &str = "\u{e0f1}"; // solo
pub const LOCK: &str = "\u{e10b}";
pub const LOCK_OPEN: &str = "\u{e10c}";
pub const EYE: &str = "\u{e0ba}";
pub const EYE_OFF: &str = "\u{e0bb}";
pub const VIDEO: &str = "\u{e1a5}"; // camera enabled
pub const VIDEO_OFF: &str = "\u{e1a6}"; // camera disabled
/// Lucide glyph for tools that have no bundled SVG icon. `None` means the tool has a real SVG
/// icon in `src/assets/` and the caller should use that instead.
pub fn tool_glyph(tool: lightningbeam_core::tool::Tool) -> Option<&'static str> {
use lightningbeam_core::tool::Tool;
Some(match tool {
Tool::Pencil => PENCIL,
Tool::Pen => PEN_TOOL,
Tool::Airbrush => SPRAY_CAN,
Tool::CloneStamp => STAMP,
Tool::HealingBrush => BANDAGE,
Tool::PatternStamp => SHAPES,
Tool::DodgeBurn => SUN_MOON,
Tool::Sponge => DROPLETS,
Tool::BlurSharpen => CONTRAST,
Tool::Gradient => BLEND,
Tool::CustomShape => HEXAGON,
Tool::SelectEllipse => CIRCLE_DASHED,
Tool::MagicWand => WAND_SPARKLES,
Tool::QuickSelect => BRUSH,
Tool::Warp => SCALING,
Tool::Liquify => DROPLET,
Tool::SelectLasso => LASSO_SELECT,
_ => return None,
})
}

View File

@ -40,7 +40,7 @@ const TOPBAR_H: f32 = 40.0;
/// Clamp a desktop dialog width to fit the current screen (with side margins). A no-op on wide
/// desktop screens (`min` keeps the desired width); on a phone-aspect window it shrinks to fit.
pub fn dialog_width(ctx: &egui::Context, desired: f32) -> f32 {
let avail = ctx.content_rect().width() - 24.0;
let avail = ctx.screen_rect().width() - 24.0;
desired.min(avail.max(200.0))
}

View File

@ -2,7 +2,6 @@
//! project scrub. Wired to the audio controller exactly like `TimelinePane`'s header.
use eframe::egui;
use daw_backend::Seconds;
use super::{icons, Palette};
use crate::panes::SharedPaneState;
@ -33,7 +32,7 @@ pub fn render(ui: &mut egui::Ui, rect: egui::Rect, shared: &mut SharedPaneState,
if let Some(controller_arc) = shared.audio_controller {
let mut controller = controller_arc.lock().unwrap();
if *shared.is_playing {
controller.seek(Seconds(*shared.playback_time));
controller.seek(*shared.playback_time);
controller.play();
} else {
controller.pause();
@ -102,7 +101,7 @@ pub fn render(ui: &mut egui::Ui, rect: egui::Rect, shared: &mut SharedPaneState,
*shared.playback_time = new_time;
if let Some(controller_arc) = shared.audio_controller {
let mut controller = controller_arc.lock().unwrap();
controller.seek(Seconds(new_time));
controller.seek(new_time);
}
}
}

View File

@ -7,7 +7,7 @@
//! - Image Assets (static images)
use eframe::egui;
use lightningbeam_core::clip::{ResolvedContent, VectorClip};
use lightningbeam_core::clip::{AudioClipType, VectorClip};
use lightningbeam_core::document::Document;
use lightningbeam_core::layer::AnyLayer;
use std::collections::{HashMap, HashSet};
@ -918,11 +918,11 @@ impl AssetLibraryPane {
continue;
}
let (extra_info, drag_clip_type) = match &clip.resolve() {
ResolvedContent::Audio { .. } => ("Sampled".to_string(), DragClipType::AudioSampled),
ResolvedContent::Midi { .. } => ("MIDI".to_string(), DragClipType::AudioMidi),
ResolvedContent::Recording => {
// Skip recording-in-progress clips (and empty take folders) from asset library
let (extra_info, drag_clip_type) = match &clip.clip_type {
AudioClipType::Sampled { .. } => ("Sampled".to_string(), DragClipType::AudioSampled),
AudioClipType::Midi { .. } => ("MIDI".to_string(), DragClipType::AudioMidi),
AudioClipType::Recording => {
// Skip recording-in-progress clips from asset library
continue;
}
};
@ -932,7 +932,7 @@ impl AssetLibraryPane {
name: clip.name.clone(),
category: AssetCategory::Audio,
drag_clip_type,
duration: clip.content_duration().native(),
duration: clip.duration,
dimensions: None,
extra_info,
is_builtin: false,
@ -1118,15 +1118,15 @@ impl AssetLibraryPane {
for (id, clip) in &document.audio_clips {
if !linked_audio_ids.contains(id) && clip.folder_id == current_folder {
let (extra_info, drag_clip_type) = match &clip.resolve() {
ResolvedContent::Audio { .. } => {
let (extra_info, drag_clip_type) = match &clip.clip_type {
AudioClipType::Sampled { .. } => {
("Sampled".to_string(), DragClipType::AudioSampled)
}
ResolvedContent::Midi { .. } => {
AudioClipType::Midi { .. } => {
("MIDI".to_string(), DragClipType::AudioMidi)
}
ResolvedContent::Recording => {
// Skip recording-in-progress clips (and empty take folders)
AudioClipType::Recording => {
// Skip recording-in-progress clips
continue;
}
};
@ -1136,7 +1136,7 @@ impl AssetLibraryPane {
name: clip.name.clone(),
category: AssetCategory::Audio,
drag_clip_type,
duration: clip.content_duration().native(),
duration: clip.duration,
dimensions: None,
extra_info,
is_builtin: false,
@ -1765,7 +1765,7 @@ impl AssetLibraryPane {
let prefetched_waveform: Option<Vec<(f32, f32)>> =
if asset_category == AssetCategory::Audio && !self.thumbnail_cache.has(&asset_id) {
if let Some(clip) = document.audio_clips.get(&asset_id) {
if let Some(audio_pool_index) = clip.audio_pool_index().as_ref() {
if let AudioClipType::Sampled { audio_pool_index } = &clip.clip_type {
shared.raw_audio_cache.get(audio_pool_index)
.map(|raw| peaks_from_raw_audio(raw, THUMBNAIL_SIZE as usize))
} else {
@ -1790,8 +1790,8 @@ impl AssetLibraryPane {
AssetCategory::Audio => {
if let Some(clip) = document.audio_clips.get(&asset_id) {
let bg_color = egui::Color32::from_rgba_unmultiplied(40, 40, 40, 200);
match &clip.resolve() {
ResolvedContent::Audio { .. } => {
match &clip.clip_type {
AudioClipType::Sampled { .. } => {
let wave_color = egui::Color32::from_rgb(100, 200, 100);
if let Some(ref peaks) = prefetched_waveform {
Some(generate_waveform_thumbnail(peaks, bg_color, wave_color))
@ -1799,15 +1799,15 @@ impl AssetLibraryPane {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
ResolvedContent::Midi { midi_clip_id } => {
AudioClipType::Midi { midi_clip_id } => {
let note_color = egui::Color32::from_rgb(100, 200, 100);
if let Some(events) = shared.midi_event_cache.get(midi_clip_id) {
Some(generate_midi_thumbnail(events, clip.content_duration().native(), bg_color, note_color))
Some(generate_midi_thumbnail(events, clip.duration, bg_color, note_color))
} else {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
// Recording in progress - show placeholder
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
@ -2344,8 +2344,8 @@ impl AssetLibraryPane {
AssetCategory::Audio => {
if let Some(clip) = document.audio_clips.get(&asset_id) {
let bg_color = egui::Color32::from_rgba_unmultiplied(40, 40, 40, 200);
match &clip.resolve() {
ResolvedContent::Audio { audio_pool_index } => {
match &clip.clip_type {
AudioClipType::Sampled { audio_pool_index } => {
let wave_color = egui::Color32::from_rgb(100, 200, 100);
let waveform: Option<Vec<(f32, f32)>> = shared.raw_audio_cache.get(audio_pool_index)
.map(|raw| peaks_from_raw_audio(raw, THUMBNAIL_SIZE as usize));
@ -2355,15 +2355,15 @@ impl AssetLibraryPane {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
ResolvedContent::Midi { midi_clip_id } => {
AudioClipType::Midi { midi_clip_id } => {
let note_color = egui::Color32::from_rgb(100, 200, 100);
if let Some(events) = shared.midi_event_cache.get(midi_clip_id) {
Some(generate_midi_thumbnail(events, clip.content_duration().native(), bg_color, note_color))
Some(generate_midi_thumbnail(events, clip.duration, bg_color, note_color))
} else {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
@ -2481,8 +2481,8 @@ impl AssetLibraryPane {
AssetCategory::Audio => {
if let Some(clip) = document.audio_clips.get(&asset_id) {
let bg_color = egui::Color32::from_rgba_unmultiplied(40, 40, 40, 200);
match &clip.resolve() {
ResolvedContent::Audio { audio_pool_index } => {
match &clip.clip_type {
AudioClipType::Sampled { audio_pool_index } => {
let wave_color = egui::Color32::from_rgb(100, 200, 100);
let waveform: Option<Vec<(f32, f32)>> = shared.raw_audio_cache.get(audio_pool_index)
.map(|raw| peaks_from_raw_audio(raw, THUMBNAIL_SIZE as usize));
@ -2492,15 +2492,15 @@ impl AssetLibraryPane {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
ResolvedContent::Midi { midi_clip_id } => {
AudioClipType::Midi { midi_clip_id } => {
let note_color = egui::Color32::from_rgb(100, 200, 100);
if let Some(events) = shared.midi_event_cache.get(midi_clip_id) {
Some(generate_midi_thumbnail(events, clip.content_duration().native(), bg_color, note_color))
Some(generate_midi_thumbnail(events, clip.duration, bg_color, note_color))
} else {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
@ -2802,7 +2802,7 @@ impl AssetLibraryPane {
let prefetched_waveform: Option<Vec<(f32, f32)>> =
if asset_category == AssetCategory::Audio && !self.thumbnail_cache.has(&asset_id) {
if let Some(clip) = document.audio_clips.get(&asset_id) {
if let Some(audio_pool_index) = clip.audio_pool_index().as_ref() {
if let AudioClipType::Sampled { audio_pool_index } = &clip.clip_type {
let waveform: Option<Vec<(f32, f32)>> = shared.raw_audio_cache.get(audio_pool_index)
.map(|raw| peaks_from_raw_audio(raw, THUMBNAIL_SIZE as usize));
if waveform.is_some() {
@ -2842,8 +2842,8 @@ impl AssetLibraryPane {
// Check if it's sampled or MIDI
if let Some(clip) = document.audio_clips.get(&asset_id) {
let bg_color = egui::Color32::from_rgba_unmultiplied(40, 40, 40, 200);
match &clip.resolve() {
ResolvedContent::Audio { .. } => {
match &clip.clip_type {
AudioClipType::Sampled { .. } => {
let wave_color = egui::Color32::from_rgb(100, 200, 100);
if let Some(ref peaks) = prefetched_waveform {
println!("✅ Generating waveform thumbnail with {} peaks for asset {}", peaks.len(), asset_id);
@ -2853,17 +2853,17 @@ impl AssetLibraryPane {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
ResolvedContent::Midi { midi_clip_id } => {
AudioClipType::Midi { midi_clip_id } => {
let bg_color = egui::Color32::from_rgba_unmultiplied(40, 40, 40, 200);
let note_color = egui::Color32::from_rgb(100, 200, 100);
if let Some(events) = shared.midi_event_cache.get(midi_clip_id) {
Some(generate_midi_thumbnail(events, clip.content_duration().native(), bg_color, note_color))
Some(generate_midi_thumbnail(events, clip.duration, bg_color, note_color))
} else {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
ResolvedContent::Recording => {
AudioClipType::Recording => {
Some(generate_placeholder_thumbnail(AssetCategory::Audio, 200))
}
}
@ -3187,7 +3187,7 @@ impl PaneRenderer for AssetLibraryPane {
println!("🎨 [ASSET_LIB] Checking for thumbnails to invalidate (pools: {:?})", shared.audio_pools_with_new_waveforms);
let mut invalidated_any = false;
for (asset_id, clip) in &document_arc.audio_clips {
if let Some(audio_pool_index) = clip.audio_pool_index().as_ref() {
if let lightningbeam_core::clip::AudioClipType::Sampled { audio_pool_index } = &clip.clip_type {
if shared.audio_pools_with_new_waveforms.contains(audio_pool_index) {
println!("❌ [ASSET_LIB] Invalidating thumbnail for asset {} (pool {})", asset_id, audio_pool_index);
self.thumbnail_cache.invalidate(asset_id);

View File

@ -160,27 +160,27 @@ pub fn gradient_stop_editor(
delete_idx = Some(i);
}
// Color picker popup — opens on click, closes on a press outside it.
let stop_color = gradient.stops[i].color;
let mut c32 = Color32::from_rgba_unmultiplied(
stop_color.r, stop_color.g, stop_color.b, stop_color.a,
);
if crate::widgets::color_swatch::color_picker_popup(
ui,
resp.id.with("stop_color_popup"),
&resp,
&mut c32,
resp.rect,
) {
// Color32 stores premultiplied RGB; unmultiply before storing as straight-alpha
// ShapeColor to avoid darkening on round-trip.
let [pr, pg, pb, a] = c32.to_array();
let unpm = |c: u8| -> u8 {
if a == 0 { 0 } else { ((c as u32 * 255 + a as u32 / 2) / a as u32).min(255) as u8 }
};
gradient.stops[i].color = ShapeColor::rgba(unpm(pr), unpm(pg), unpm(pb), a);
changed = true;
}
// Color picker popup — opens on click, closes on click-outside.
egui::containers::Popup::from_toggle_button_response(&resp)
.show(|ui| {
ui.spacing_mut().slider_width = 200.0;
let stop = &mut gradient.stops[i];
let mut c32 = Color32::from_rgba_unmultiplied(
stop.color.r, stop.color.g, stop.color.b, stop.color.a,
);
if egui::color_picker::color_picker_color32(
ui, &mut c32, egui::color_picker::Alpha::OnlyBlend,
) {
// Color32 stores premultiplied RGB; unmultiply before storing
// as straight-alpha ShapeColor to avoid darkening on round-trip.
let [pr, pg, pb, a] = c32.to_array();
let unpm = |c: u8| -> u8 {
if a == 0 { 0 } else { ((c as u32 * 255 + a as u32 / 2) / a as u32).min(255) as u8 }
};
stop.color = ShapeColor::rgba(unpm(pr), unpm(pg), unpm(pb), a);
changed = true;
}
});
}
// Apply drag to whichever stop selected_stop points at.

View File

@ -28,9 +28,14 @@ pub struct InfopanelPane {
tool_section_open: bool,
/// Whether the shape properties section is expanded
shape_section_open: bool,
/// Whether each tool's brush picker is expanded. The *selected* preset lives on the tool's
/// `BrushSlot`, since it's part of the document-independent tool state, not panel chrome.
brush_picker_expanded: std::collections::HashMap<crate::tools::BrushKind, bool>,
/// Index of the selected paint brush preset (None = custom / unset)
selected_brush_preset: Option<usize>,
/// Whether the paint brush picker is expanded
brush_picker_expanded: bool,
/// Index of the selected eraser brush preset
selected_eraser_preset: Option<usize>,
/// Whether the eraser brush picker is expanded
eraser_picker_expanded: bool,
/// Cached preview textures, one per preset (populated lazily).
brush_preview_textures: Vec<egui::TextureHandle>,
/// Selected stop index for gradient editor in shape section.
@ -56,10 +61,15 @@ impl InfopanelPane {
// Kick off background loading of the font-picker fonts at startup so the dropdown
// is ready without a hitch by the time the user opens it.
lightningbeam_core::fonts::start_preload();
let presets = bundled_brushes();
let default_eraser_idx = presets.iter().position(|p| p.name == "Brush");
Self {
tool_section_open: true,
shape_section_open: true,
brush_picker_expanded: std::collections::HashMap::new(),
selected_brush_preset: None,
brush_picker_expanded: false,
selected_eraser_preset: default_eraser_idx,
eraser_picker_expanded: false,
brush_preview_textures: Vec::new(),
selected_shape_gradient_stop: None,
selected_tool_gradient_stop: None,
@ -241,7 +251,7 @@ impl InfopanelPane {
|| is_raster_select || is_raster_shape || matches!(
tool,
Tool::PaintBucket | Tool::RegionSelect | Tool::MagicWand | Tool::QuickSelect
| Tool::Warp | Tool::Liquify | Tool::Gradient | Tool::Eyedropper
| Tool::Warp | Tool::Liquify | Tool::Gradient
);
if !has_options {
@ -275,11 +285,12 @@ impl InfopanelPane {
return;
}
// Raster paint tool: tool-unique controls, then the shared brush controls
// (library, color, size/strength/hardness/spacing) that every brush tool gets.
// Raster paint tool: delegate to per-tool impl.
if let Some(def) = raster_tool_def {
def.render_ui(ui, shared.raster_settings);
self.render_raster_tool_options(ui, shared, def);
if def.show_brush_preset_picker() {
self.render_raster_tool_options(ui, shared, def.is_eraser());
}
}
match tool {
@ -333,14 +344,7 @@ impl InfopanelPane {
ui.checkbox(shared.fill_enabled, "Fill Shape");
}
Tool::Eyedropper => {
// Which swatch the sampled color lands in.
render_color_slot_row(ui, &mut shared.raster_settings.eyedropper_use_fg);
}
Tool::PaintBucket => {
render_color_slot_row(ui, &mut shared.raster_settings.fill_use_fg);
if active_is_raster {
use crate::tools::FillThresholdMode;
ui.horizontal(|ui| {
@ -599,66 +603,68 @@ impl InfopanelPane {
});
}
/// Render the shared controls every dab-painting tool has: the brush library, the FG/BG
/// color choice, and size/strength/hardness/spacing/angle. `def` supplies the tool's brush
/// slot and the label for the one slider whose meaning varies (opacity vs. exposure vs.
/// flow vs. strength).
/// Render all options for a raster paint tool (brush picker + sliders).
/// `is_eraser` drives which shared state is read/written.
fn render_raster_tool_options(
&mut self,
ui: &mut Ui,
shared: &mut SharedPaneState,
def: &'static dyn crate::tools::RasterToolDef,
is_eraser: bool,
) {
let kind = def.brush_kind();
self.render_brush_preset_grid(ui, shared, kind);
self.render_brush_preset_grid(ui, shared, is_eraser);
ui.add_space(2.0);
if def.uses_color() {
render_color_slot_row(ui, &mut shared.raster_settings.brush_mut(kind).use_fg);
let rs = &mut shared.raster_settings;
if !is_eraser {
ui.horizontal(|ui| {
ui.label("Color:");
ui.selectable_value(&mut rs.brush_use_fg, true, "FG");
ui.selectable_value(&mut rs.brush_use_fg, false, "BG");
});
}
let slot = shared.raster_settings.brush_mut(kind);
macro_rules! field {
($eraser:ident, $brush:ident) => {
if is_eraser { &mut rs.$eraser } else { &mut rs.$brush }
}
}
ui.horizontal(|ui| {
ui.label("Size:");
ui.add(egui::Slider::new(&mut slot.radius, 1.0_f32..=500.0)
.logarithmic(true)
.suffix(" px"));
ui.add(egui::Slider::new(field!(eraser_radius, brush_radius), 1.0_f32..=200.0).logarithmic(true).suffix(" px"));
});
ui.horizontal(|ui| {
ui.label(format!("{}:", def.strength_label()));
ui.add(egui::Slider::new(&mut slot.strength, 0.0_f32..=1.0)
ui.label("Opacity:");
ui.add(egui::Slider::new(field!(eraser_opacity, brush_opacity), 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Hardness:");
ui.add(egui::Slider::new(&mut slot.hardness, 0.0_f32..=1.0)
ui.add(egui::Slider::new(field!(eraser_hardness, brush_hardness), 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Spacing:");
ui.add(egui::Slider::new(&mut slot.spacing, 0.01_f32..=20.0)
ui.add(egui::Slider::new(field!(eraser_spacing, brush_spacing), 0.01_f32..=1.0)
.logarithmic(true)
.custom_formatter(|v, _| format!("{:.2}", v)));
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
if slot.settings.elliptical_dab_ratio > 1.001 {
let bs = if is_eraser { &rs.active_eraser_settings } else { &rs.active_brush_settings };
if bs.elliptical_dab_ratio > 1.001 {
ui.horizontal(|ui| {
ui.label("Angle:");
ui.add(egui::Slider::new(&mut slot.angle_offset, -180.0_f32..=180.0)
ui.add(egui::Slider::new(&mut rs.brush_angle_offset, -180.0_f32..=180.0)
.suffix("°")
.custom_formatter(|v, _| format!("{:.0}°", v)));
});
}
}
/// Render the brush preset thumbnail grid (collapsible) for one tool's brush slot.
fn render_brush_preset_grid(
&mut self,
ui: &mut Ui,
shared: &mut SharedPaneState,
kind: crate::tools::BrushKind,
) {
/// Render the brush preset thumbnail grid (collapsible).
/// `is_eraser` drives which picker state and which shared settings are updated.
fn render_brush_preset_grid(&mut self, ui: &mut Ui, shared: &mut SharedPaneState, is_eraser: bool) {
let presets = bundled_brushes();
if presets.is_empty() { return; }
@ -684,8 +690,8 @@ impl InfopanelPane {
}
// Read picker state into locals to avoid multiple &mut self borrows.
let mut expanded = self.brush_picker_expanded.get(&kind).copied().unwrap_or(false);
let selected = shared.raster_settings.brush(kind).preset;
let mut expanded = if is_eraser { self.eraser_picker_expanded } else { self.brush_picker_expanded };
let mut selected = if is_eraser { self.selected_eraser_preset } else { self.selected_brush_preset };
let gap = 3.0;
let cols = 2usize;
@ -756,14 +762,25 @@ impl InfopanelPane {
egui::FontId::proportional(9.5),
if is_sel { egui::Color32::from_rgb(140, 190, 255) } else { egui::Color32::from_gray(160) });
if resp.clicked() {
selected = Some(idx);
expanded = false;
shared.raster_settings.brush_mut(kind).apply_preset(idx, &preset.settings);
// If the user was on a preset-backed tool (Pencil/Pen/Airbrush)
// and manually picked a different brush, revert to the generic tool.
if kind == crate::tools::BrushKind::Paint
&& matches!(*shared.selected_tool, Tool::Pencil | Tool::Pen | Tool::Airbrush)
{
*shared.selected_tool = Tool::Draw;
let s = &preset.settings;
let rs = &mut shared.raster_settings;
if is_eraser {
rs.eraser_opacity = s.opaque.clamp(0.0, 1.0);
rs.eraser_hardness = s.hardness.clamp(0.0, 1.0);
rs.eraser_spacing = s.dabs_per_radius;
rs.active_eraser_settings = s.clone();
} else {
rs.brush_opacity = s.opaque.clamp(0.0, 1.0);
rs.brush_hardness = s.hardness.clamp(0.0, 1.0);
rs.brush_spacing = s.dabs_per_radius;
rs.active_brush_settings = s.clone();
// If the user was on a preset-backed tool (Pencil/Pen/Airbrush)
// and manually picked a different brush, revert to the generic tool.
if matches!(*shared.selected_tool, Tool::Pencil | Tool::Pen | Tool::Airbrush) {
*shared.selected_tool = Tool::Draw;
}
}
}
}
@ -772,9 +789,14 @@ impl InfopanelPane {
}
}
// The selected preset lives on the slot itself (written by apply_preset above); only the
// expand/collapse state is panel-local.
self.brush_picker_expanded.insert(kind, expanded);
// Write back picker state.
if is_eraser {
self.eraser_picker_expanded = expanded;
self.selected_eraser_preset = selected;
} else {
self.brush_picker_expanded = expanded;
self.selected_brush_preset = selected;
}
}
// Transform section: deferred to Phase 2 (DCEL elements don't have instance transforms)
@ -1587,33 +1609,21 @@ impl InfopanelPane {
ui.horizontal(|ui| {
ui.label("Start:");
ui.label(format!("{:.2}s", document.tempo_map().beats_to_seconds(ci.effective_start()).seconds_to_f64()));
ui.label(format!("{:.2}s", ci.effective_start()));
});
let clip_dur = document.clip_trim_duration(&ci.clip_id)
.unwrap_or_else(|| lightningbeam_core::clip::ClipDuration::Seconds(
daw_backend::Seconds(
(ci.trim_end.unwrap_or(daw_backend::ContentTime(1.0)) - ci.trim_start).raw(),
),
));
let clip_dur = document.get_clip_duration(&ci.clip_id)
.unwrap_or_else(|| ci.trim_end.unwrap_or(1.0) - ci.trim_start);
let total_dur = ci.total_duration(clip_dur, document.tempo_map());
let total_dur_secs = (document.tempo_map().beats_to_seconds(ci.effective_start() + total_dur)
- document.tempo_map().beats_to_seconds(ci.effective_start())).seconds_to_f64();
ui.horizontal(|ui| {
ui.label("Duration:");
ui.label(format!("{:.2}s", total_dur_secs));
ui.label(format!("{:.2}s", total_dur));
});
if ci.trim_start > daw_backend::ContentTime::ZERO {
if ci.trim_start > 0.0 {
ui.horizontal(|ui| {
ui.label("Trim Start:");
// Content time in the clip's own domain: seconds for sampled
// audio/video/vector, beats for MIDI. Label it accordingly.
let unit = match clip_dur {
lightningbeam_core::clip::ClipDuration::Beats(_) => "beats",
lightningbeam_core::clip::ClipDuration::Seconds(_) => "s",
};
ui.label(format!("{:.2}{}", ci.trim_start.raw(), unit));
ui.label(format!("{:.2}s", ci.trim_start));
});
}
@ -1796,7 +1806,7 @@ impl InfopanelPane {
});
ui.horizontal(|ui| {
ui.label("Duration:");
ui.label(format!("{:.2}s", clip.content_duration().to_seconds(document.tempo_map()).seconds_to_f64()));
ui.label(format!("{:.2}s", clip.duration));
});
} else {
// Could be an image asset or effect — show ID
@ -1921,17 +1931,3 @@ impl PaneRenderer for InfopanelPane {
"Info Panel"
}
}
/// The FG/BG selector shared by every tool that reads or writes a color (brush, paint bucket,
/// eyedropper, …), so the choice is made the same way and in the same place regardless of tool.
/// The wording stays neutral because the eyedropper *writes* to the chosen swatch while the
/// painting tools *read* from it.
fn render_color_slot_row(ui: &mut Ui, use_fg: &mut bool) {
ui.horizontal(|ui| {
ui.label("Color:");
ui.selectable_value(use_fg, true, "FG")
.on_hover_text("Foreground (stroke) color");
ui.selectable_value(use_fg, false, "BG")
.on_hover_text("Background (fill) color");
});
}

View File

@ -189,6 +189,8 @@ pub struct SharedPaneState<'a> {
pub selected_tool: &'a mut Tool,
pub fill_color: &'a mut egui::Color32,
pub stroke_color: &'a mut egui::Color32,
/// Tracks which color (fill or stroke) was last interacted with, for eyedropper tool
pub active_color_mode: &'a mut ColorMode,
pub pending_view_action: &'a mut Option<crate::menu::MenuAction>,
/// Tracks the priority of the best fallback pane for view actions
/// Lower number = higher priority. None = no fallback pane seen yet

View File

@ -36,6 +36,8 @@ pub struct MobileNodeState {
pub mode: NodeViewMode,
/// The module currently shown in Focus (and centred in Patch).
pub focus_node: Option<NodeId>,
/// Armed cable source in Patch: (node, output-port index).
pub patch_source: Option<(NodeId, usize)>,
/// Whether the add-node picker overlay is open.
pub show_add: bool,
/// Search filter in the add-node picker.
@ -49,6 +51,7 @@ impl Default for MobileNodeState {
Self {
mode: NodeViewMode::Focus,
focus_node: None,
patch_source: None,
show_add: false,
add_search: String::new(),
patch_pick: None,

View File

@ -5,7 +5,6 @@
/// When a sampled audio layer is selected, shows a GPU-rendered spectrogram.
use eframe::egui;
use daw_backend::Seconds;
use egui::{pos2, vec2, Align2, Color32, FontId, Rect, Stroke, StrokeKind};
use std::collections::{HashMap, HashSet};
use uuid::Uuid;
@ -465,13 +464,9 @@ impl PianoRollPane {
if let Some(AnyLayer::Audio(audio_layer)) = document.get_layer(&layer_id) {
for instance in &audio_layer.clip_instances {
if let Some(clip) = document.audio_clips.get(&instance.clip_id) {
// Resolve through the instance's active take, so a MIDI take folder edits
// whichever take it's actually playing.
if let Some(midi_clip_id) = instance.resolved_midi_clip_id(clip) {
let duration = instance.effective_duration(clip.content_duration(), document.tempo_map());
// A MIDI clip's content time IS beats, which is what the piano roll's
// x-axis uses.
clip_data.push((midi_clip_id, instance.timeline_start.beats_to_f64(), instance.trim_start.raw(), duration.beats_to_f64(), instance.id));
if let AudioClipType::Midi { midi_clip_id } = clip.clip_type {
let duration = instance.effective_duration(clip.duration, document.tempo_map());
clip_data.push((midi_clip_id, instance.timeline_start, instance.trim_start, duration, instance.id));
}
}
}
@ -655,7 +650,7 @@ impl PianoRollPane {
*shared.playback_time = nt;
if let Some(ctrl) = shared.audio_controller.as_ref() {
if let Ok(mut c) = ctrl.lock() {
c.seek(Seconds(nt));
c.seek(nt);
}
}
}
@ -1768,7 +1763,7 @@ impl PianoRollPane {
let seek_time = snap_to_value(time.max(0.0), self.snap_value, tempo_map);
*shared.playback_time = seek_time;
if let Some(ctrl) = shared.audio_controller.as_ref() {
if let Ok(mut c) = ctrl.lock() { c.seek(Seconds(seek_time)); }
if let Ok(mut c) = ctrl.lock() { c.seek(seek_time); }
}
}
}
@ -2459,16 +2454,10 @@ impl PianoRollPane {
for instance in &audio_layer.clip_instances {
if let Some(clip) = document.audio_clips.get(&instance.clip_id) {
if let AudioClipType::Sampled { audio_pool_index } = clip.clip_type {
// Duration in beats: explicit timeline_duration, else the clip's content
// length converted to beats at the clip's start.
let duration = instance.timeline_duration.unwrap_or_else(|| {
let tmap = document.tempo_map();
tmap.seconds_to_beats(tmap.beats_to_seconds(instance.timeline_start) + clip.content_duration().to_seconds(tmap)) - instance.timeline_start
});
let duration = instance.timeline_duration.unwrap_or(clip.duration);
// Get sample rate from raw_audio_cache
if let Some((_samples, sr, _ch)) = shared.raw_audio_cache.get(&audio_pool_index) {
// A sampled clip's content time is seconds.
clip_infos.push((audio_pool_index, instance.timeline_start.beats_to_f64(), instance.trim_start.raw(), duration.beats_to_f64(), *sr));
clip_infos.push((audio_pool_index, instance.timeline_start, instance.trim_start, duration, *sr));
}
}
}

View File

@ -4,9 +4,8 @@
/// Supports HDR compositing pipeline with per-layer buffers and effects.
use eframe::egui;
use daw_backend::Seconds;
use lightningbeam_core::action::Action;
use lightningbeam_core::clip::{ClipDuration, ClipInstance};
use lightningbeam_core::clip::ClipInstance;
use lightningbeam_core::gpu::{BufferPool, BufferFormat, BufferSpec, Compositor, EffectProcessor, SrgbToLinearConverter};
use lightningbeam_core::layer::{AnyLayer, AudioLayer};
use lightningbeam_core::renderer::RenderedLayerType;
@ -1852,16 +1851,10 @@ impl egui_wgpu::CallbackTrait for VelloCallback {
// Create EffectInstance from ClipInstance for the processor
// For now, create a simple effect instance with default parameters
let tempo_map = self.ctx.document.tempo_map();
let effect_end_beats = effect_instance.timeline_start
+ effect_instance.effective_duration(
ClipDuration::Seconds(Seconds(lightningbeam_core::effect::EFFECT_DURATION)),
tempo_map,
);
let effect_inst = lightningbeam_core::effect::EffectInstance::new(
effect_def,
tempo_map.beats_to_seconds(effect_instance.timeline_start).seconds_to_f64(),
tempo_map.beats_to_seconds(effect_end_beats).seconds_to_f64(),
effect_instance.timeline_start,
effect_instance.timeline_start + effect_instance.effective_duration(lightningbeam_core::effect::EFFECT_DURATION, self.ctx.document.tempo_map()),
);
// Acquire temp buffer for effect output (HDR format)
@ -2211,8 +2204,7 @@ impl egui_wgpu::CallbackTrait for VelloCallback {
let combined_transform = overlay_transform * clip_transform;
// Calculate clip bounds for preview
let start_secs = self.ctx.document.tempo_map().beats_to_seconds(clip_inst.timeline_start).seconds_to_f64();
let clip_time = ((self.ctx.playback_time - start_secs) * clip_inst.playback_speed) + clip_inst.trim_start.raw();
let clip_time = ((self.ctx.playback_time - clip_inst.timeline_start) * clip_inst.playback_speed) + clip_inst.trim_start;
let content_bounds = if let Some(vector_clip) = self.ctx.document.get_vector_clip(&clip_inst.clip_id) {
vector_clip.calculate_content_bounds(&self.ctx.document, clip_time)
} else if let Some(video_clip) = self.ctx.document.get_video_clip(&clip_inst.clip_id) {
@ -2301,21 +2293,15 @@ impl egui_wgpu::CallbackTrait for VelloCallback {
// Also draw selection outlines for clip instances
for &clip_id in self.ctx.selection.clip_instances() {
if let Some(clip_instance) = vector_layer.clip_instances.iter().find(|ci| ci.id == clip_id) {
// Skip clip instances not active at current time (compare in seconds).
let clip_dur = ClipDuration::Seconds(
self.ctx.document.get_clip_duration(&clip_instance.clip_id).unwrap_or(Seconds::ZERO),
);
let tempo_map = self.ctx.document.tempo_map();
let start_secs = tempo_map.beats_to_seconds(clip_instance.timeline_start).seconds_to_f64();
let instance_end = tempo_map.beats_to_seconds(
clip_instance.timeline_start + clip_instance.effective_duration(clip_dur, tempo_map)
).seconds_to_f64();
if self.ctx.playback_time < start_secs || self.ctx.playback_time >= instance_end {
// Skip clip instances not active at current time
let clip_dur = self.ctx.document.get_clip_duration(&clip_instance.clip_id).unwrap_or(0.0);
let instance_end = clip_instance.timeline_start + clip_instance.effective_duration(clip_dur, self.ctx.document.tempo_map());
if self.ctx.playback_time < clip_instance.timeline_start || self.ctx.playback_time >= instance_end {
continue;
}
// Calculate clip-local time
let clip_time = ((self.ctx.playback_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start.raw();
let clip_time = ((self.ctx.playback_time - clip_instance.timeline_start) * clip_instance.playback_speed) + clip_instance.trim_start;
// Get dynamic clip bounds from content at current time
let bbox = if let Some(vector_clip) = self.ctx.document.get_vector_clip(&clip_instance.clip_id) {
@ -2685,13 +2671,9 @@ impl egui_wgpu::CallbackTrait for VelloCallback {
// Find clip instance visible at playback time
let visible_clip = video_layer.clip_instances.iter().find(|inst| {
let clip_duration = ClipDuration::Seconds(
self.ctx.document.get_clip_duration(&inst.clip_id).unwrap_or(Seconds::ZERO),
);
let tempo_map = self.ctx.document.tempo_map();
let start_secs = tempo_map.beats_to_seconds(inst.timeline_start).seconds_to_f64();
let end_secs = tempo_map.beats_to_seconds(inst.timeline_start + inst.effective_duration(clip_duration, tempo_map)).seconds_to_f64();
playback_time >= start_secs && playback_time < end_secs
let clip_duration = self.ctx.document.get_clip_duration(&inst.clip_id).unwrap_or(0.0);
let effective_duration = inst.effective_duration(clip_duration, self.ctx.document.tempo_map());
playback_time >= inst.timeline_start && playback_time < inst.timeline_start + effective_duration
});
if let Some(clip_inst) = visible_clip {
@ -5722,14 +5704,9 @@ impl StagePane {
screen_pos: egui::Pos2,
shared: &mut SharedPaneState,
) {
// On click, store the screen position and which swatch the sample lands in.
// On click, store the screen position and color mode for sampling
if self.rsp_clicked(response) {
let mode = if shared.raster_settings.eyedropper_use_fg {
super::ColorMode::Stroke
} else {
super::ColorMode::Fill
};
self.pending_eyedropper_sample = Some((screen_pos, mode));
self.pending_eyedropper_sample = Some((screen_pos, *shared.active_color_mode));
}
}
@ -6497,16 +6474,15 @@ impl StagePane {
b.dabs_per_radius = spacing;
if matches!(blend_mode, RasterBlendMode::Smudge) {
b.dabs_per_actual_radius = 0.0;
b.smudge_radius_log =
shared.raster_settings.brush(crate::tools::BrushKind::Smudge).strength;
b.smudge_radius_log = shared.raster_settings.smudge_strength;
}
if matches!(blend_mode, RasterBlendMode::BlurSharpen) {
b.dabs_per_actual_radius = 0.0;
}
let color = if !def.uses_color() {
let color = if matches!(blend_mode, RasterBlendMode::Erase) {
[1.0f32, 1.0, 1.0, 1.0]
} else {
let c = if shared.raster_settings.brush(def.brush_kind()).use_fg {
let c = if shared.raster_settings.brush_use_fg {
*shared.stroke_color
} else {
*shared.fill_color
@ -6715,8 +6691,7 @@ impl StagePane {
b.dabs_per_actual_radius = 0.0;
// strength controls how far behind the stroke to sample (smudge_dist multiplier).
// smudge_dist = radius * exp(smudge_radius_log), so log(strength) gives the ratio.
b.smudge_radius_log =
shared.raster_settings.brush(crate::tools::BrushKind::Smudge).strength;
b.smudge_radius_log = shared.raster_settings.smudge_strength; // linear [0,1] strength
}
if matches!(blend_mode, lightningbeam_core::raster_layer::RasterBlendMode::BlurSharpen) {
// Zero dabs_per_actual_radius so the spacing slider is the sole density control.
@ -6725,14 +6700,10 @@ impl StagePane {
b
};
let color = if !def.uses_color() {
let color = if matches!(blend_mode, lightningbeam_core::raster_layer::RasterBlendMode::Erase) {
[1.0f32, 1.0, 1.0, 1.0]
} else {
let c = if shared.raster_settings.brush(def.brush_kind()).use_fg {
*shared.stroke_color
} else {
*shared.fill_color
};
let c = if shared.raster_settings.brush_use_fg { *shared.stroke_color } else { *shared.fill_color };
let s2l = |v: u8| -> f32 {
let f = v as f32 / 255.0;
if f <= 0.04045 { f / 12.92 } else { ((f + 0.055) / 1.055).powf(2.4) }
@ -7447,7 +7418,7 @@ impl StagePane {
use lightningbeam_core::actions::PaintBucketAction;
use vello::kurbo::Point;
let click_point = Point::new(world_pos.x as f64, world_pos.y as f64);
let fill_color = ShapeColor::from_egui(bucket_color(shared));
let fill_color = ShapeColor::from_egui(*shared.fill_color);
let action = PaintBucketAction::new(
active_layer_id,
*shared.playback_time,
@ -7501,7 +7472,7 @@ impl StagePane {
return;
}
let fill_egui = bucket_color(shared);
let fill_egui = *shared.fill_color;
let fill_color = [fill_egui.r(), fill_egui.g(), fill_egui.b(), fill_egui.a()];
let threshold = shared.raster_settings.fill_threshold;
let softness = shared.raster_settings.fill_softness;
@ -10159,8 +10130,7 @@ impl StagePane {
for &clip_id in shared.selection.clip_instances() {
if let Some(clip_instance) = vector_layer.clip_instances.iter().find(|ci| ci.id == clip_id) {
// Calculate clip-local time
let start_secs = shared.action_executor.document().tempo_map().beats_to_seconds(clip_instance.timeline_start).seconds_to_f64();
let clip_time = ((*shared.playback_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start.raw();
let clip_time = ((*shared.playback_time - clip_instance.timeline_start) * clip_instance.playback_speed) + clip_instance.trim_start;
// Get dynamic clip bounds from content at current time
use vello::kurbo::Rect as KurboRect;
@ -10360,8 +10330,7 @@ impl StagePane {
// Try clip instance
if let Some(clip_instance) = vector_layer.clip_instances.iter().find(|ci| ci.id == object_id) {
// Calculate clip-local time
let start_secs = shared.action_executor.document().tempo_map().beats_to_seconds(clip_instance.timeline_start).seconds_to_f64();
let clip_time = ((*shared.playback_time - start_secs) * clip_instance.playback_speed) + clip_instance.trim_start.raw();
let clip_time = ((*shared.playback_time - clip_instance.timeline_start) * clip_instance.playback_speed) + clip_instance.trim_start;
// Get dynamic clip bounds from content at current time
let local_bbox = if let Some(vector_clip) = shared.action_executor.document().get_vector_clip(&clip_instance.clip_id) {
@ -11077,13 +11046,9 @@ impl StagePane {
let document = shared.action_executor.document();
if let Some(AnyLayer::Video(video_layer)) = document.get_layer(layer_id) {
video_layer.clip_instances.iter().find(|inst| {
let clip_duration = ClipDuration::Seconds(
document.get_clip_duration(&inst.clip_id).unwrap_or(Seconds::ZERO),
);
let tempo_map = document.tempo_map();
let start_secs = tempo_map.beats_to_seconds(inst.timeline_start).seconds_to_f64();
let end_secs = tempo_map.beats_to_seconds(inst.timeline_start + inst.effective_duration(clip_duration, tempo_map)).seconds_to_f64();
playback_time >= start_secs && playback_time < end_secs
let clip_duration = document.get_clip_duration(&inst.clip_id).unwrap_or(0.0);
let effective_duration = inst.effective_duration(clip_duration, document.tempo_map());
playback_time >= inst.timeline_start && playback_time < inst.timeline_start + effective_duration
}).map(|inst| inst.id)
} else {
None
@ -11600,11 +11565,7 @@ impl StagePane {
// Handle tool input (only if not using Alt modifier for panning, and not while a
// double-tap-drag marquee is actively dragging so the tool doesn't act during the gesture).
// A pen barrel button bound to Pan suppresses the tool the same way Alt does — the pen
// tip is still down while panning, so without this the brush would paint as you pan.
let pen_panning = crate::tablet::active_button_action()
== Some(crate::config::TabletButtonAction::Pan);
if !alt_held && !pen_panning && !self.marquee_gesture_active {
if !alt_held && !self.marquee_gesture_active {
use lightningbeam_core::tool::Tool;
// On a shape-tween in-between frame the active vector layer's geometry is an
@ -11767,21 +11728,9 @@ impl StagePane {
self.pan_offset.y += scroll_delta.y;
}
// Middle-mouse drag pans, independent of the stage's own drag sense.
let middle_down = ui.input(|i| i.pointer.middle_down());
if middle_down {
let delta = ui.input(|i| i.pointer.delta());
self.pan_offset += delta;
self.is_panning = true;
}
// Holding a pen barrel button bound to Pan turns the pen drag into a pan instead of
// a stroke (the tip is still down, so this is a normal primary drag).
let pen_pan = crate::tablet::active_button_action()
== Some(crate::config::TabletButtonAction::Pan);
// Handle panning with Alt+Drag (or a pan-bound pen button)
if (alt_held || pen_pan) && response.dragged() {
// Handle panning with Alt+Drag
if alt_held && response.dragged() {
// Alt+Click+Drag panning
if let Some(last_pos) = self.last_pan_pos {
if let Some(current_pos) = response.interact_pointer_pos() {
let delta = current_pos - last_pos;
@ -11791,7 +11740,7 @@ impl StagePane {
self.last_pan_pos = response.interact_pointer_pos();
self.is_panning = true;
} else {
if !response.dragged() && !middle_down {
if !response.dragged() {
self.is_panning = false;
self.last_pan_pos = None;
}
@ -12135,22 +12084,26 @@ impl StagePane {
let r = shared.raster_settings.quick_select_radius;
(r, r, 0.0_f32)
} else if let Some(def) = crate::tools::raster_tool_def(shared.selected_tool) {
// Every brush tool can carry an elliptical library brush, so shape the cursor from
// whichever slot the active tool paints with.
let slot = shared.raster_settings.brush(def.brush_kind());
let r = def.cursor_radius(shared.raster_settings);
let bs = &slot.settings;
let ratio = bs.elliptical_dab_ratio.max(1.0);
let expand = 1.0 + bs.offset_by_random;
let angle = (bs.elliptical_dab_angle + slot.angle_offset).to_radians();
(r * expand, r * expand / ratio, angle)
// For the standard paint brush, also account for elliptical shape.
if matches!(*shared.selected_tool,
Tool::Draw | Tool::Pencil | Tool::Pen | Tool::Airbrush)
{
let bs = &shared.raster_settings.active_brush_settings;
let ratio = bs.elliptical_dab_ratio.max(1.0);
let expand = 1.0 + bs.offset_by_random;
let angle = (bs.elliptical_dab_angle + shared.raster_settings.brush_angle_offset).to_radians();
(r * expand, r * expand / ratio, angle)
} else {
(r, r, 0.0_f32)
}
} else {
let slot = shared.raster_settings.brush(crate::tools::BrushKind::Paint);
let bs = &slot.settings;
let bs = &shared.raster_settings.active_brush_settings;
let r = shared.raster_settings.brush_radius;
let ratio = bs.elliptical_dab_ratio.max(1.0);
let expand = 1.0 + bs.offset_by_random;
let angle = (bs.elliptical_dab_angle + slot.angle_offset).to_radians();
(slot.radius * expand, slot.radius * expand / ratio, angle)
let angle = (bs.elliptical_dab_angle + shared.raster_settings.brush_angle_offset).to_radians();
(r * expand, r * expand / ratio, angle)
};
let a = a_world * self.zoom; // major semi-axis in screen pixels
@ -12195,84 +12148,9 @@ impl StagePane {
impl PaneRenderer for StagePane {
fn render_header(&mut self, ui: &mut egui::Ui, shared: &mut SharedPaneState) -> bool {
// Fill most of the 40px header so the buttons are a comfortable tablet-sized target
// rather than the ~20px egui default.
const BTN_H: f32 = 27.0;
let btn_size = egui::vec2(32.0, BTN_H);
// Lay the row out in a full-width band of exactly BTN_H centred in the header, so the
// buttons sit vertically centred instead of dropping to the bottom of the taller header
// rect. Horizontally they still start at the left, as before.
let avail = ui.max_rect();
let band = egui::Rect::from_min_size(
egui::pos2(avail.min.x, avail.center().y - BTN_H / 2.0),
egui::vec2(avail.width(), BTN_H),
);
ui.scope_builder(
egui::UiBuilder::new()
.max_rect(band)
.layout(egui::Layout::left_to_right(egui::Align::Center)),
|ui| {
// Undo / redo — duplicated here so tablet users don't need the keyboard.
let can_undo = shared.action_executor.can_undo();
let can_redo = shared.action_executor.can_redo();
let icon_btn = |glyph: &str| {
egui::Button::new(
egui::RichText::new(glyph).font(crate::mobile::icons::font(16.0)),
)
.min_size(btn_size)
};
if ui
.add_enabled(can_undo, icon_btn(crate::mobile::icons::UNDO_2))
.on_hover_text("Undo")
.clicked()
{
shared.pending_menu_actions.push(crate::menu::MenuAction::Undo);
}
if ui
.add_enabled(can_redo, icon_btn(crate::mobile::icons::REDO_2))
.on_hover_text("Redo")
.clicked()
{
shared.pending_menu_actions.push(crate::menu::MenuAction::Redo);
}
ui.separator();
// Zoom to fit: Lucide "maximize" glyph followed by the label. Two fonts in one
// button means building the layout job by hand.
let fit_label = {
let mut job = egui::text::LayoutJob::default();
let color = ui.visuals().widgets.inactive.fg_stroke.color;
job.append(
crate::mobile::icons::MAXIMIZE,
0.0,
egui::TextFormat {
font_id: crate::mobile::icons::font(15.0),
color,
valign: egui::Align::Center,
..Default::default()
},
);
job.append(
"Fit",
6.0,
egui::TextFormat {
font_id: egui::TextStyle::Button.resolve(ui.style()),
color,
valign: egui::Align::Center,
..Default::default()
},
);
job
};
if ui
.add(egui::Button::new(fit_label).min_size(egui::vec2(0.0, BTN_H)))
.on_hover_text("Zoom to fit canvas in view")
.clicked()
{
ui.horizontal(|ui| {
// Zoom to fit button
if ui.button("⊡ Fit").on_hover_text("Zoom to fit canvas in view").clicked() {
self.zoom_to_fit(shared);
}
@ -12282,8 +12160,7 @@ impl PaneRenderer for StagePane {
let text_style = shared.theme.style(".text-primary", ui.ctx());
let text_color = text_style.text_color.unwrap_or(egui::Color32::from_gray(200));
ui.colored_label(text_color, format!("Zoom: {:.0}%", self.zoom * 100.0));
},
);
});
true
}
@ -12610,14 +12487,8 @@ impl PaneRenderer for StagePane {
let canvas_pos = pointer_pos - rect.min;
let world_pos = (canvas_pos - self.pan_offset) / self.zoom;
// Use playhead time (seconds); the beats placement position for clips.
// Use playhead time
let drop_time = *shared.playback_time;
let drop_beats = shared.action_executor.document().tempo_map().seconds_to_beats(Seconds(drop_time));
// 5-second default effect duration as a beats span at the drop point.
let effect_dur_beats = {
let tmap = shared.action_executor.document().tempo_map();
tmap.seconds_to_beats(tmap.beats_to_seconds(drop_beats) + Seconds(5.0)) - drop_beats
};
// Find or create a compatible layer
let document = shared.action_executor.document();
@ -12688,8 +12559,8 @@ impl PaneRenderer for StagePane {
// Create clip instance for effect with 5 second default duration
let clip_instance = ClipInstance::new(def.id)
.with_timeline_start(drop_beats)
.with_timeline_duration(effect_dur_beats);
.with_timeline_start(drop_time)
.with_timeline_duration(5.0);
// Use AddEffectAction for effect layers
let action = lightningbeam_core::actions::AddEffectAction::new(
@ -12701,7 +12572,7 @@ impl PaneRenderer for StagePane {
} else {
// For clips, create a clip instance
let mut clip_instance = ClipInstance::new(dragging.clip_id)
.with_timeline_start(drop_beats);
.with_timeline_start(drop_time);
// For video clips, scale to fit and center in document
if dragging.clip_type == DragClipType::Video {
@ -12771,7 +12642,7 @@ impl PaneRenderer for StagePane {
// Create audio clip instance at same timeline position
let audio_instance = ClipInstance::new(linked_audio_clip_id)
.with_timeline_start(drop_beats);
.with_timeline_start(drop_time);
let audio_instance_id = audio_instance.id;
eprintln!("DEBUG STAGE: Created audio instance: {} for clip: {}", audio_instance_id, linked_audio_clip_id);
@ -13212,13 +13083,3 @@ impl PaneRenderer for StagePane {
"Stage"
}
}
/// The color the paint bucket fills with. Like the brush, the bucket chooses between the
/// foreground (stroke) and background (fill) swatch — see `RasterToolSettings::fill_use_fg`.
fn bucket_color(shared: &SharedPaneState) -> egui::Color32 {
if shared.raster_settings.fill_use_fg {
*shared.stroke_color
} else {
*shared.fill_color
}
}

File diff suppressed because it is too large Load Diff

View File

@ -29,48 +29,10 @@ impl PaneRenderer for ToolbarPane {
path: &NodePath,
shared: &mut SharedPaneState,
) {
// `ui` spans the whole window, not this pane — bind a child Ui to the pane's content rect
// first, or the ScrollArea would start at the window's top (under the pane header) and
// size itself against the window's height (so it would never need to scroll).
// Salt by path: widget ids inside are auto-generated from the Ui's id, so two toolbar
// panes would otherwise fight over the same ids.
let mut content_ui = ui.new_child(
egui::UiBuilder::new()
.id_salt(("toolbar", path))
.max_rect(rect)
.layout(egui::Layout::top_down(egui::Align::Min)),
);
let button_size = 60.0;
let button_padding = 8.0;
let button_spacing = 4.0;
egui::ScrollArea::vertical()
.id_salt(("toolbar_scroll", path))
.auto_shrink([false; 2])
.show(&mut content_ui, |ui| {
self.render_toolbar(ui, path, shared);
});
}
fn name(&self) -> &str {
"Toolbar"
}
}
const BUTTON_SIZE: f32 = 60.0;
const BUTTON_PADDING: f32 = 8.0;
const BUTTON_SPACING: f32 = 4.0;
const COLOR_BUTTON_SIZE: f32 = 50.0;
const COLOR_LABEL_WIDTH: f32 = 40.0;
impl ToolbarPane {
/// Laid out with real egui widgets (`horizontal_wrapped` for the tool grid) rather than
/// absolute rect math, so the ScrollArea above can measure the content and scroll it when the
/// pane is too short. Buttons are still painted by hand — `allocate_exact_size` reserves the
/// space and gives us the Response, and we draw into the rect egui hands back.
fn render_toolbar(
&mut self,
ui: &mut egui::Ui,
path: &NodePath,
shared: &mut SharedPaneState,
) {
// Determine which tools to show based on the active layer type
let active_layer_type: Option<LayerType> = shared.active_layer_id
.and_then(|id| shared.action_executor.document().get_layer(&id))
@ -90,273 +52,320 @@ impl ToolbarPane {
*shared.selected_tool = Tool::Select;
}
// Calculate how many columns we can fit
let available_width = rect.width() - (button_padding * 2.0);
let columns =
((available_width + button_spacing) / (button_size + button_spacing)).floor() as usize;
let columns = columns.max(1); // At least 1 column
let total_tools = tools.len();
let total_rows = (total_tools + columns - 1) / columns;
let mut y = rect.top() + button_padding;
// Process tools row by row for centered layout
for row in 0..total_rows {
let start_idx = row * columns;
let end_idx = (start_idx + columns).min(total_tools);
let buttons_in_row = end_idx - start_idx;
// Calculate the total width of buttons in this row
let row_width = (buttons_in_row as f32 * button_size)
+ ((buttons_in_row.saturating_sub(1)) as f32 * button_spacing);
// Center the row
let mut x = rect.left() + (rect.width() - row_width) / 2.0;
for tool_idx in start_idx..end_idx {
let tool = &tools[tool_idx];
let button_rect =
egui::Rect::from_min_size(egui::pos2(x, y), egui::vec2(button_size, button_size));
// Check if this is the selected tool
let is_selected = *shared.selected_tool == *tool;
// Button background
let bg_color = if is_selected {
shared.theme.bg_color(&["#toolbar", ".tool-button", ".selected"], ui.ctx(), egui::Color32::from_rgb(70, 100, 150))
} else {
shared.theme.bg_color(&["#toolbar", ".tool-button"], ui.ctx(), egui::Color32::from_rgb(50, 50, 50))
};
ui.painter().rect_filled(button_rect, 4.0, bg_color);
// Load and render tool icon
if let Some(icon) = shared.tool_icon_cache.get_or_load(*tool, ui.ctx()) {
let icon_rect = button_rect.shrink(8.0); // Padding inside button
ui.painter().image(
icon.id(),
icon_rect,
egui::Rect::from_min_max(egui::pos2(0.0, 0.0), egui::pos2(1.0, 1.0)),
egui::Color32::WHITE,
);
}
// Draw sub-tool arrow indicator for tools with modes
let has_sub_tools = matches!(tool, Tool::RegionSelect | Tool::SelectLasso);
if has_sub_tools {
let arrow_size = 6.0;
let margin = 4.0;
let corner = button_rect.right_bottom() - egui::vec2(margin, margin);
let tri = [
corner,
corner - egui::vec2(arrow_size, 0.0),
corner - egui::vec2(0.0, arrow_size),
];
ui.painter().add(egui::Shape::convex_polygon(
tri.to_vec(),
shared.theme.text_color(&["#toolbar", ".tool-button"], ui.ctx(), egui::Color32::from_gray(200)),
egui::Stroke::NONE,
));
}
// Make button interactive (include path to ensure unique IDs across panes)
let button_id = ui.id().with(("tool_button", path, *tool as usize));
let response = ui.interact(button_rect, button_id, egui::Sense::click());
// Check for click first
if response.clicked() {
*shared.selected_tool = *tool;
// Preset-backed tools: auto-select the matching bundled brush.
let preset_name = match tool {
Tool::Pencil => Some("Pencil"),
Tool::Pen => Some("Pen"),
Tool::Airbrush => Some("Airbrush"),
_ => None,
};
if let Some(name) = preset_name {
if let Some(preset) = bundled_brushes().iter().find(|p| p.name == name) {
let s = &preset.settings;
shared.raster_settings.brush_opacity = s.opaque.clamp(0.0, 1.0);
shared.raster_settings.brush_hardness = s.hardness.clamp(0.0, 1.0);
shared.raster_settings.brush_spacing = s.dabs_per_radius;
shared.raster_settings.active_brush_settings = s.clone();
}
}
}
// Right-click context menu for tools with sub-options
if has_sub_tools {
response.context_menu(|ui| {
match tool {
Tool::RegionSelect => {
ui.set_min_width(120.0);
if ui.selectable_label(
*shared.region_select_mode == RegionSelectMode::Rectangle,
"Rectangle",
).clicked() {
*shared.region_select_mode = RegionSelectMode::Rectangle;
*shared.selected_tool = Tool::RegionSelect;
ui.close();
}
if ui.selectable_label(
*shared.region_select_mode == RegionSelectMode::Lasso,
"Lasso",
).clicked() {
*shared.region_select_mode = RegionSelectMode::Lasso;
*shared.selected_tool = Tool::RegionSelect;
ui.close();
}
}
Tool::SelectLasso => {
ui.set_min_width(130.0);
if ui.selectable_label(
*shared.lasso_mode == LassoMode::Freehand,
"Freehand",
).clicked() {
*shared.lasso_mode = LassoMode::Freehand;
*shared.selected_tool = Tool::SelectLasso;
ui.close();
}
if ui.selectable_label(
*shared.lasso_mode == LassoMode::Polygonal,
"Polygonal",
).clicked() {
*shared.lasso_mode = LassoMode::Polygonal;
*shared.selected_tool = Tool::SelectLasso;
ui.close();
}
if ui.selectable_label(
*shared.lasso_mode == LassoMode::Magnetic,
"Magnetic",
).clicked() {
*shared.lasso_mode = LassoMode::Magnetic;
*shared.selected_tool = Tool::SelectLasso;
ui.close();
}
}
_ => {}
}
});
}
if response.hovered() {
ui.painter().rect_stroke(
button_rect,
4.0,
egui::Stroke::new(2.0, shared.theme.border_color(&["#toolbar", ".tool-button", ".hover"], ui.ctx(), egui::Color32::from_gray(180))),
egui::StrokeKind::Middle,
);
}
// Show tooltip with tool name and shortcut (consumes response).
// Hint text is pulled from the live keymap so it reflects user remappings.
let hint = tool_app_action(*tool)
.and_then(|action| shared.keymap.get(action))
.map(|s| format!(" ({})", s.hint_text()))
.unwrap_or_default();
let tooltip = if *tool == Tool::RegionSelect {
let mode = match *shared.region_select_mode {
RegionSelectMode::Rectangle => "Rectangle",
RegionSelectMode::Lasso => "Lasso",
};
format!("{} - {}{}\nRight-click for options", tool.display_name(), mode, hint)
} else if *tool == Tool::SelectLasso {
let mode = match *shared.lasso_mode {
LassoMode::Freehand => "Freehand",
LassoMode::Polygonal => "Polygonal",
LassoMode::Magnetic => "Magnetic",
};
format!("{} - {}{}\nRight-click for options", tool.display_name(), mode, hint)
} else {
format!("{}{}", tool.display_name(), hint)
};
response.on_hover_text(tooltip);
// Draw selection border
if is_selected {
ui.painter().rect_stroke(
button_rect,
4.0,
egui::Stroke::new(2.0, shared.theme.border_color(&["#toolbar", ".tool-button", ".selected"], ui.ctx(), egui::Color32::from_rgb(100, 150, 255))),
egui::StrokeKind::Middle,
);
}
// Move to next column in this row
x += button_size + button_spacing;
}
// Move to next row
y += button_size + button_spacing;
}
let is_raster = matches!(active_layer_type, Some(LayerType::Raster));
let show_colors = matches!(active_layer_type, None | Some(LayerType::Vector) | Some(LayerType::Raster));
ui.spacing_mut().item_spacing = egui::vec2(BUTTON_SPACING, BUTTON_SPACING);
ui.add_space(BUTTON_PADDING);
// Add color pickers below the tool buttons
if show_colors {
y += button_spacing * 2.0; // Extra spacing
// Centre the grid as a block. Work out how many columns fit, then lay the buttons out in
// a band of exactly that width, positioned to centre it. The band has to be an explicit
// max_rect on the child Ui — a `horizontal_wrapped` nested inside a `horizontal` inherits
// the parent's available width and wraps against that, not against the width we set.
let full_width = ui.available_width();
let avail = full_width - BUTTON_PADDING * 2.0;
let columns = (((avail + BUTTON_SPACING) / (BUTTON_SIZE + BUTTON_SPACING)).floor() as usize)
.max(1)
.min(tools.len().max(1));
let grid_width =
columns as f32 * BUTTON_SIZE + (columns.saturating_sub(1)) as f32 * BUTTON_SPACING;
let indent = ((full_width - grid_width) / 2.0).max(0.0);
let rows = (tools.len() + columns - 1) / columns;
let fill_label_width = 40.0;
let color_button_size = 50.0;
let color_row_width = fill_label_width + color_button_size + button_spacing;
let color_x = rect.left() + (rect.width() - color_row_width) / 2.0;
let cursor = ui.cursor().min;
let band = egui::Rect::from_min_size(
egui::pos2(cursor.x + indent, cursor.y),
egui::vec2(grid_width, rows as f32 * (BUTTON_SIZE + BUTTON_SPACING)),
);
ui.scope_builder(
egui::UiBuilder::new().max_rect(band).layout(
egui::Layout::left_to_right(egui::Align::Min).with_main_wrap(true),
),
|ui| {
ui.spacing_mut().item_spacing = egui::vec2(BUTTON_SPACING, BUTTON_SPACING);
for tool in tools.iter() {
self.render_tool_button(ui, tool, shared);
}
},
);
// Colour swatches below the tools.
// Two color swatches:
// Stroke/FG always on top, Fill/BG always on bottom.
// Raster layers label them "FG" / "BG"; vector layers label them "Stroke" / "Fill".
if show_colors {
ui.add_space(BUTTON_SPACING * 2.0);
{
let stroke_label = if is_raster { "FG" } else { "Stroke" };
let label_color = shared.theme.text_color(&["#toolbar", ".text-secondary"], ui.ctx(), egui::Color32::from_gray(200));
ui.painter().text(
egui::pos2(color_x + fill_label_width / 2.0, y + color_button_size / 2.0),
egui::Align2::CENTER_CENTER,
stroke_label,
egui::FontId::proportional(14.0),
label_color,
);
let row_width = COLOR_LABEL_WIDTH + COLOR_BUTTON_SIZE + BUTTON_SPACING;
let color_indent = ((ui.available_width() - row_width) / 2.0).max(0.0);
for (label, is_stroke) in [
(if is_raster { "FG" } else { "Stroke" }, true),
(if is_raster { "BG" } else { "Fill" }, false),
] {
ui.horizontal(|ui| {
ui.add_space(color_indent);
let (label_rect, _) = ui.allocate_exact_size(
egui::vec2(COLOR_LABEL_WIDTH, COLOR_BUTTON_SIZE),
egui::Sense::hover(),
);
let label_color = shared.theme.text_color(
&["#toolbar", ".text-secondary"],
ui.ctx(),
egui::Color32::from_gray(200),
);
ui.painter().text(
label_rect.center(),
egui::Align2::CENTER_CENTER,
label,
egui::FontId::proportional(14.0),
label_color,
);
let (swatch_rect, _) = ui.allocate_exact_size(
egui::vec2(COLOR_BUTTON_SIZE, COLOR_BUTTON_SIZE),
egui::Sense::hover(),
);
let (id_key, color) = if is_stroke {
("stroke_color_button", &mut *shared.stroke_color)
} else {
("fill_color_button", &mut *shared.fill_color)
};
let button_id = ui.id().with((id_key, path));
crate::widgets::color_swatch::color_swatch(ui, button_id, swatch_rect, color);
let stroke_button_rect = egui::Rect::from_min_size(
egui::pos2(color_x + fill_label_width + button_spacing, y),
egui::vec2(color_button_size, color_button_size),
);
let stroke_button_id = ui.id().with(("stroke_color_button", path));
let stroke_response = ui.interact(stroke_button_rect, stroke_button_id, egui::Sense::click());
draw_color_button(ui, stroke_button_rect, *shared.stroke_color);
egui::containers::Popup::from_toggle_button_response(&stroke_response)
.show(|ui| {
ui.spacing_mut().slider_width = 275.0;
let changed = egui::color_picker::color_picker_color32(ui, shared.stroke_color, egui::color_picker::Alpha::OnlyBlend);
if changed {
*shared.active_color_mode = super::ColorMode::Stroke;
}
});
}
y += color_button_size + button_spacing;
}
ui.add_space(BUTTON_PADDING);
// Fill/BG color swatch
{
let fill_label = if is_raster { "BG" } else { "Fill" };
let label_color = shared.theme.text_color(&["#toolbar", ".text-secondary"], ui.ctx(), egui::Color32::from_gray(200));
ui.painter().text(
egui::pos2(color_x + fill_label_width / 2.0, y + color_button_size / 2.0),
egui::Align2::CENTER_CENTER,
fill_label,
egui::FontId::proportional(14.0),
label_color,
);
let fill_button_rect = egui::Rect::from_min_size(
egui::pos2(color_x + fill_label_width + button_spacing, y),
egui::vec2(color_button_size, color_button_size),
);
let fill_button_id = ui.id().with(("fill_color_button", path));
let fill_response = ui.interact(fill_button_rect, fill_button_id, egui::Sense::click());
draw_color_button(ui, fill_button_rect, *shared.fill_color);
egui::containers::Popup::from_toggle_button_response(&fill_response)
.show(|ui| {
ui.spacing_mut().slider_width = 275.0;
let changed = egui::color_picker::color_picker_color32(ui, shared.fill_color, egui::color_picker::Alpha::OnlyBlend);
if changed {
*shared.active_color_mode = super::ColorMode::Fill;
}
});
}
} // end color pickers
}
fn render_tool_button(
&mut self,
ui: &mut egui::Ui,
tool: &Tool,
shared: &mut SharedPaneState,
) {
let (button_rect, response) = ui.allocate_exact_size(
egui::vec2(BUTTON_SIZE, BUTTON_SIZE),
egui::Sense::click(),
);
let is_selected = *shared.selected_tool == *tool;
// Button background
let bg_color = if is_selected {
shared.theme.bg_color(&["#toolbar", ".tool-button", ".selected"], ui.ctx(), egui::Color32::from_rgb(70, 100, 150))
} else {
shared.theme.bg_color(&["#toolbar", ".tool-button"], ui.ctx(), egui::Color32::from_rgb(50, 50, 50))
};
ui.painter().rect_filled(button_rect, 4.0, bg_color);
// Tool icon: tools without a bundled SVG fall back to a Lucide glyph rather than the
// shared TODO placeholder.
if let Some(glyph) = crate::mobile::icons::tool_glyph(*tool) {
ui.painter().text(
button_rect.center(),
egui::Align2::CENTER_CENTER,
glyph,
crate::mobile::icons::font(26.0),
shared.theme.text_color(
&["#toolbar", ".tool-button"],
ui.ctx(),
egui::Color32::from_gray(220),
),
);
} else if let Some(icon) = shared.tool_icon_cache.get_or_load(*tool, ui.ctx()) {
let icon_rect = button_rect.shrink(8.0); // Padding inside button
ui.painter().image(
icon.id(),
icon_rect,
egui::Rect::from_min_max(egui::pos2(0.0, 0.0), egui::pos2(1.0, 1.0)),
egui::Color32::WHITE,
);
}
// Draw sub-tool arrow indicator for tools with modes
let has_sub_tools = matches!(tool, Tool::RegionSelect | Tool::SelectLasso);
if has_sub_tools {
let arrow_size = 6.0;
let margin = 4.0;
let corner = button_rect.right_bottom() - egui::vec2(margin, margin);
let tri = [
corner,
corner - egui::vec2(arrow_size, 0.0),
corner - egui::vec2(0.0, arrow_size),
];
ui.painter().add(egui::Shape::convex_polygon(
tri.to_vec(),
shared.theme.text_color(&["#toolbar", ".tool-button"], ui.ctx(), egui::Color32::from_gray(200)),
egui::Stroke::NONE,
));
}
if response.clicked() {
*shared.selected_tool = *tool;
// Preset-backed tools: auto-select the matching bundled brush.
let preset_name = match tool {
Tool::Pencil => Some("Pencil"),
Tool::Pen => Some("Pen"),
Tool::Airbrush => Some("Airbrush"),
_ => None,
};
if let Some(name) = preset_name {
if let Some(idx) = bundled_brushes().iter().position(|p| p.name == name) {
let settings = bundled_brushes()[idx].settings.clone();
shared
.raster_settings
.brush_mut(crate::tools::BrushKind::Paint)
.apply_preset(idx, &settings);
}
}
}
// Right-click context menu for tools with sub-options
if has_sub_tools {
response.context_menu(|ui| {
match tool {
Tool::RegionSelect => {
ui.set_min_width(120.0);
if ui.selectable_label(
*shared.region_select_mode == RegionSelectMode::Rectangle,
"Rectangle",
).clicked() {
*shared.region_select_mode = RegionSelectMode::Rectangle;
*shared.selected_tool = Tool::RegionSelect;
ui.close();
}
if ui.selectable_label(
*shared.region_select_mode == RegionSelectMode::Lasso,
"Lasso",
).clicked() {
*shared.region_select_mode = RegionSelectMode::Lasso;
*shared.selected_tool = Tool::RegionSelect;
ui.close();
}
}
Tool::SelectLasso => {
ui.set_min_width(130.0);
if ui.selectable_label(
*shared.lasso_mode == LassoMode::Freehand,
"Freehand",
).clicked() {
*shared.lasso_mode = LassoMode::Freehand;
*shared.selected_tool = Tool::SelectLasso;
ui.close();
}
if ui.selectable_label(
*shared.lasso_mode == LassoMode::Polygonal,
"Polygonal",
).clicked() {
*shared.lasso_mode = LassoMode::Polygonal;
*shared.selected_tool = Tool::SelectLasso;
ui.close();
}
if ui.selectable_label(
*shared.lasso_mode == LassoMode::Magnetic,
"Magnetic",
).clicked() {
*shared.lasso_mode = LassoMode::Magnetic;
*shared.selected_tool = Tool::SelectLasso;
ui.close();
}
}
_ => {}
}
});
}
if response.hovered() {
ui.painter().rect_stroke(
button_rect,
4.0,
egui::Stroke::new(2.0, shared.theme.border_color(&["#toolbar", ".tool-button", ".hover"], ui.ctx(), egui::Color32::from_gray(180))),
egui::StrokeKind::Middle,
);
}
// Draw selection border
if is_selected {
ui.painter().rect_stroke(
button_rect,
4.0,
egui::Stroke::new(2.0, shared.theme.border_color(&["#toolbar", ".tool-button", ".selected"], ui.ctx(), egui::Color32::from_rgb(100, 150, 255))),
egui::StrokeKind::Middle,
);
}
// Show tooltip with tool name and shortcut (consumes response).
// Hint text is pulled from the live keymap so it reflects user remappings.
let hint = tool_app_action(*tool)
.and_then(|action| shared.keymap.get(action))
.map(|s| format!(" ({})", s.hint_text()))
.unwrap_or_default();
let tooltip = if *tool == Tool::RegionSelect {
let mode = match *shared.region_select_mode {
RegionSelectMode::Rectangle => "Rectangle",
RegionSelectMode::Lasso => "Lasso",
};
format!("{} - {}{}\nRight-click for options", tool.display_name(), mode, hint)
} else if *tool == Tool::SelectLasso {
let mode = match *shared.lasso_mode {
LassoMode::Freehand => "Freehand",
LassoMode::Polygonal => "Polygonal",
LassoMode::Magnetic => "Magnetic",
};
format!("{} - {}{}\nRight-click for options", tool.display_name(), mode, hint)
} else {
format!("{}{}", tool.display_name(), hint)
};
response.on_hover_text(tooltip);
fn name(&self) -> &str {
"Toolbar"
}
}
/// Draw a color button with checkerboard background for alpha channel
fn draw_color_button(ui: &mut egui::Ui, rect: egui::Rect, color: egui::Color32) {
// Draw checkerboard background
let checker_size = 5.0;
let cols = (rect.width() / checker_size).ceil() as usize;
let rows = (rect.height() / checker_size).ceil() as usize;
for row in 0..rows {
for col in 0..cols {
let is_light = (row + col) % 2 == 0;
let checker_color = if is_light {
egui::Color32::from_gray(180)
} else {
egui::Color32::from_gray(120)
};
let checker_rect = egui::Rect::from_min_size(
egui::pos2(
rect.min.x + col as f32 * checker_size,
rect.min.y + row as f32 * checker_size,
),
egui::vec2(checker_size, checker_size),
).intersect(rect);
ui.painter().rect_filled(checker_rect, 0.0, checker_color);
}
}
// Draw color on top
ui.painter().rect_filled(rect, 2.0, color);
// Draw border
ui.painter().rect_stroke(
rect,
2.0,
egui::Stroke::new(1.0, egui::Color32::from_gray(80)),
egui::StrokeKind::Middle,
);
}

View File

@ -5,7 +5,7 @@
use std::collections::HashMap;
use eframe::egui;
use crate::config::{AppConfig, TabletButtonAction};
use crate::config::AppConfig;
use crate::keymap::{self, AppAction, KeymapManager};
use crate::menu::{MenuSystem, Shortcut, ShortcutKey};
use crate::theme::{Theme, ThemeMode};
@ -55,15 +55,12 @@ struct PreferencesState {
file_height: u32,
scroll_speed: f64,
audio_buffer_size: u32,
cycle_midi_separate_takes: bool,
reopen_last_session: bool,
restore_layout_from_file: bool,
debug: bool,
waveform_stereo: bool,
theme_mode: ThemeMode,
large_media_default: LargeMediaMode,
tablet_button_lower: TabletButtonAction,
tablet_button_upper: TabletButtonAction,
}
impl From<(&AppConfig, &Theme)> for PreferencesState {
@ -75,15 +72,12 @@ impl From<(&AppConfig, &Theme)> for PreferencesState {
file_height: config.file_height,
scroll_speed: config.scroll_speed,
audio_buffer_size: config.audio_buffer_size,
cycle_midi_separate_takes: config.cycle_midi_separate_takes,
reopen_last_session: config.reopen_last_session,
restore_layout_from_file: config.restore_layout_from_file,
debug: config.debug,
waveform_stereo: config.waveform_stereo,
theme_mode: theme.mode(),
large_media_default: config.large_media_default,
tablet_button_lower: config.tablet_button_lower,
tablet_button_upper: config.tablet_button_upper,
}
}
}
@ -97,15 +91,12 @@ impl Default for PreferencesState {
file_height: 600,
scroll_speed: 1.0,
audio_buffer_size: 256,
cycle_midi_separate_takes: false,
reopen_last_session: false,
restore_layout_from_file: true,
debug: false,
waveform_stereo: false,
theme_mode: ThemeMode::System,
large_media_default: LargeMediaMode::default(),
tablet_button_lower: TabletButtonAction::Pan,
tablet_button_upper: TabletButtonAction::Eyedropper,
}
}
}
@ -174,7 +165,7 @@ impl PreferencesDialog {
// mobile modals; on desktop, the familiar draggable window.
let width = crate::mobile::dialog_width(ctx, 550.0);
let scroll_h = if mobile {
(ctx.content_rect().height() - 220.0).clamp(160.0, 400.0)
(ctx.screen_rect().height() - 220.0).clamp(160.0, 400.0)
} else {
400.0
};
@ -248,8 +239,6 @@ impl PreferencesDialog {
ui.add_space(8.0);
self.render_startup_section(ui);
ui.add_space(8.0);
self.render_tablet_section(ui);
ui.add_space(8.0);
self.render_advanced_section(ui);
});
}
@ -554,39 +543,6 @@ impl PreferencesDialog {
});
ui.label("Requires app restart to take effect");
ui.separator();
ui.horizontal(|ui| {
ui.label("Cycle MIDI recording:");
egui::ComboBox::from_id_salt("cycle_midi_mode")
.selected_text(if self.working_prefs.cycle_midi_separate_takes {
"Separate takes"
} else {
"Merge"
})
.show_ui(ui, |ui| {
ui.selectable_value(
&mut self.working_prefs.cycle_midi_separate_takes,
false,
"Merge",
)
.on_hover_text(
"Every pass overdubs into one clip, and earlier passes play back as \
you record so you can layer against them.",
);
ui.selectable_value(
&mut self.working_prefs.cycle_midi_separate_takes,
true,
"Separate takes",
)
.on_hover_text(
"Each pass becomes its own take in a take folder, as audio always \
does. Earlier passes stay silent they're alternatives, not layers.",
);
});
});
});
}
@ -635,41 +591,6 @@ impl PreferencesDialog {
});
}
fn render_tablet_section(&mut self, ui: &mut egui::Ui) {
egui::CollapsingHeader::new("Tablet")
.default_open(false)
.show(ui, |ui| {
ui.label("What each barrel button on the stylus does while held:");
ui.add_space(4.0);
let button_row = |ui: &mut egui::Ui, label: &str, id: &str, value: &mut TabletButtonAction| {
ui.horizontal(|ui| {
ui.label(label);
egui::ComboBox::from_id_salt(id)
.selected_text(value.label())
.show_ui(ui, |ui| {
for action in TabletButtonAction::ALL {
ui.selectable_value(value, action, action.label());
}
});
});
};
button_row(
ui,
"Lower button:",
"tablet_button_lower",
&mut self.working_prefs.tablet_button_lower,
);
button_row(
ui,
"Upper button:",
"tablet_button_upper",
&mut self.working_prefs.tablet_button_upper,
);
});
}
fn render_advanced_section(&mut self, ui: &mut egui::Ui) {
egui::CollapsingHeader::new("Advanced")
.default_open(false)
@ -720,14 +641,11 @@ impl PreferencesDialog {
temp_config.file_height = self.working_prefs.file_height;
temp_config.scroll_speed = self.working_prefs.scroll_speed;
temp_config.audio_buffer_size = self.working_prefs.audio_buffer_size;
temp_config.cycle_midi_separate_takes = self.working_prefs.cycle_midi_separate_takes;
temp_config.reopen_last_session = self.working_prefs.reopen_last_session;
temp_config.restore_layout_from_file = self.working_prefs.restore_layout_from_file;
temp_config.debug = self.working_prefs.debug;
temp_config.waveform_stereo = self.working_prefs.waveform_stereo;
temp_config.theme_mode = self.working_prefs.theme_mode.to_string_lower();
temp_config.tablet_button_lower = self.working_prefs.tablet_button_lower;
temp_config.tablet_button_upper = self.working_prefs.tablet_button_upper;
// Validate
if let Err(err) = temp_config.validate() {
@ -757,23 +675,16 @@ impl PreferencesDialog {
config.file_height = self.working_prefs.file_height;
config.scroll_speed = self.working_prefs.scroll_speed;
config.audio_buffer_size = self.working_prefs.audio_buffer_size;
config.cycle_midi_separate_takes = self.working_prefs.cycle_midi_separate_takes;
config.reopen_last_session = self.working_prefs.reopen_last_session;
config.restore_layout_from_file = self.working_prefs.restore_layout_from_file;
config.debug = self.working_prefs.debug;
config.waveform_stereo = self.working_prefs.waveform_stereo;
config.theme_mode = self.working_prefs.theme_mode.to_string_lower();
config.large_media_default = self.working_prefs.large_media_default;
config.tablet_button_lower = self.working_prefs.tablet_button_lower;
config.tablet_button_upper = self.working_prefs.tablet_button_upper;
config.keybindings = keybinding_config;
// Apply theme immediately
theme.set_mode(self.working_prefs.theme_mode);
crate::tablet::set_button_actions(
self.working_prefs.tablet_button_lower,
self.working_prefs.tablet_button_upper,
);
// Save to disk
config.save();

View File

@ -13,8 +13,6 @@ use std::sync::atomic::{AtomicU32, Ordering};
use eframe::egui;
use lightningbeam_core::tool::Tool;
use crate::config::TabletButtonAction;
// ---------------------------------------------------------------------------
// Global tablet state — read by make_stroke_point() on the UI thread
// ---------------------------------------------------------------------------
@ -23,58 +21,6 @@ static TABLET_PRESSURE_BITS: AtomicU32 = AtomicU32::new(0x3f800000); // 1.0f32
static TABLET_TILT_X_BITS: AtomicU32 = AtomicU32::new(0); // 0.0f32
static TABLET_TILT_Y_BITS: AtomicU32 = AtomicU32::new(0); // 0.0f32
/// Bit 0 = lower barrel button held, bit 1 = upper. Read on the UI thread by the stage to
/// decide whether the pen is currently panning.
static TABLET_BUTTONS: AtomicU32 = AtomicU32::new(0);
fn button_bit(b: TabletButton) -> u32 {
match b {
TabletButton::Lower => 1,
TabletButton::Upper => 2,
}
}
/// Is the given barrel button currently held?
pub fn button_held(b: TabletButton) -> bool {
TABLET_BUTTONS.load(Ordering::Relaxed) & button_bit(b) != 0
}
/// The configured action for each barrel button, mirrored from `AppConfig` so the stage can
/// read it without threading config through every call site.
static BUTTON_ACTIONS: std::sync::Mutex<(TabletButtonAction, TabletButtonAction)> =
std::sync::Mutex::new((TabletButtonAction::Pan, TabletButtonAction::Eyedropper));
/// Mirror the configured barrel-button actions. Call on startup and whenever preferences change.
pub fn set_button_actions(lower: TabletButtonAction, upper: TabletButtonAction) {
if let Ok(mut a) = BUTTON_ACTIONS.lock() {
*a = (lower, upper);
}
}
fn action_for(b: TabletButton) -> TabletButtonAction {
let actions = BUTTON_ACTIONS
.lock()
.map(|a| *a)
.unwrap_or((TabletButtonAction::Pan, TabletButtonAction::Eyedropper));
match b {
TabletButton::Lower => actions.0,
TabletButton::Upper => actions.1,
}
}
/// The action of whichever barrel button is currently held (lower wins if both are).
pub fn active_button_action() -> Option<TabletButtonAction> {
for b in [TabletButton::Lower, TabletButton::Upper] {
if button_held(b) {
let a = action_for(b);
if a != TabletButtonAction::None {
return Some(a);
}
}
}
None
}
/// Current pen pressure (0.01.0). Falls back to 1.0 when no tablet is active.
pub fn current_pressure() -> f32 {
f32::from_bits(TABLET_PRESSURE_BITS.load(Ordering::Relaxed))
@ -111,32 +57,10 @@ pub enum RawTabletEvent {
Tilt { x: f32, y: f32 },
TipDown,
TipUp,
/// A barrel button on the pen was pressed or released.
Button { button: TabletButton, pressed: bool },
/// End of Wayland tablet event group; commit accumulated state.
Frame,
}
/// Which barrel button on the stylus. Most pens have two; some have a third.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum TabletButton {
Lower,
Upper,
}
impl TabletButton {
/// Map a Linux evdev button code (used by both the Wayland tablet protocol and X11's
/// underlying device) to a barrel button.
fn from_evdev(code: u32) -> Option<Self> {
// BTN_STYLUS = 0x14b, BTN_STYLUS2 = 0x14c, BTN_STYLUS3 = 0x149.
match code {
0x14b => Some(TabletButton::Lower),
0x14c => Some(TabletButton::Upper),
_ => None,
}
}
}
#[derive(Debug, Clone, Copy, PartialEq, Default)]
pub enum TabletToolType {
#[default]
@ -167,13 +91,9 @@ pub struct TabletInput {
/// On X11, clicks already come through winit via OS mouse emulation.
inject_buttons: bool,
/// Pending tool switch (eraser in/out, barrel-button override). Consumed by `EditorApp::update()`.
/// Pending tool switch (eraser in/out). Consumed by `EditorApp::update()`.
pub pending_tool_switch: Option<Tool>,
tool_before_eraser: Option<Tool>,
/// Barrel-button state as of last frame, for press/release edge detection.
prev_buttons: u32,
/// Tool to restore when the barrel button driving a tool override is released.
tool_before_button: Option<Tool>,
/// One-shot sender used to hand the egui Context to the background thread
/// on the first poll() call so it can call request_repaint().
@ -205,8 +125,6 @@ impl TabletInput {
inject_buttons,
pending_tool_switch: None,
tool_before_eraser: None,
prev_buttons: 0,
tool_before_button: None,
repaint_sender,
backend,
}
@ -284,9 +202,6 @@ impl TabletInput {
// Handle eraser tool switch.
self.handle_tool_switch(current_tool);
// Handle barrel-button tool overrides (eyedropper / eraser held on the pen).
self.handle_button_tool_override(current_tool);
// Publish globals for make_stroke_point().
set_pressure(self.pressure);
let (tx, ty) = self.tilt;
@ -352,8 +267,6 @@ impl TabletInput {
RawTabletEvent::ProximityOut => {
self.in_proximity = false;
self.pressure = 0.0;
// Don't let a button stay latched if the pen leaves while it's held.
TABLET_BUTTONS.store(0, Ordering::Relaxed);
}
RawTabletEvent::Motion { x, y } => {
// Wayland gives physical pixels; convert to egui logical points.
@ -376,14 +289,6 @@ impl TabletInput {
RawTabletEvent::TipUp => {
self.tip_down = Some(false);
}
RawTabletEvent::Button { button, pressed } => {
let bit = button_bit(button);
if pressed {
TABLET_BUTTONS.fetch_or(bit, Ordering::Relaxed);
} else {
TABLET_BUTTONS.fetch_and(!bit, Ordering::Relaxed);
}
}
RawTabletEvent::Frame => {
// Frame is the commit signal; processing already happened in individual events.
}
@ -413,39 +318,6 @@ impl TabletInput {
}
}
/// Barrel buttons bound to Eyedropper/Erase act as a spring-loaded tool override: switch
/// while held, restore on release. Pan is handled by the stage instead, since it needs the
/// drag delta rather than a tool.
fn handle_button_tool_override(&mut self, current_tool: Tool) {
let buttons = TABLET_BUTTONS.load(Ordering::Relaxed);
if buttons == self.prev_buttons {
return;
}
self.prev_buttons = buttons;
let override_tool = active_button_action().and_then(|a| match a {
TabletButtonAction::Eyedropper => Some(Tool::Eyedropper),
TabletButtonAction::Erase => Some(Tool::Erase),
TabletButtonAction::Pan | TabletButtonAction::None => None,
});
match (override_tool, self.tool_before_button) {
// Button pressed: remember the tool we're overriding, then switch.
(Some(tool), None) => {
if current_tool != tool {
self.tool_before_button = Some(current_tool);
self.pending_tool_switch = Some(tool);
}
}
// Button released: restore.
(None, Some(prev)) => {
self.tool_before_button = None;
self.pending_tool_switch = Some(prev);
}
_ => {}
}
}
fn handle_tool_switch(&mut self, current_tool: Tool) {
let just_entered = self.in_proximity && !self.was_in_proximity;
let just_left = !self.in_proximity && self.was_in_proximity;
@ -471,7 +343,7 @@ impl TabletInput {
#[cfg(target_os = "linux")]
mod wayland {
use super::{RawTabletEvent, TabletButton, TabletToolType};
use super::{RawTabletEvent, TabletToolType};
use eframe::egui;
use std::collections::HashMap;
use std::sync::mpsc;
@ -573,7 +445,6 @@ mod wayland {
pending_motion: bool,
pending_tip: Option<bool>,
pending_proximity: Option<bool>,
pending_buttons: Vec<(TabletButton, bool)>,
}
// -----------------------------------------------------------------------
@ -714,20 +585,7 @@ mod wayland {
Event::Tilt { tilt_x, tilt_y } => {
acc.pending_tilt = (tilt_x as f32, tilt_y as f32);
}
Event::Button { button, state: btn_state, .. } => {
// `button` is a Linux evdev code (BTN_STYLUS / BTN_STYLUS2).
if let Some(b) = TabletButton::from_evdev(button) {
let pressed = matches!(
btn_state.into_result(),
Ok(zwp_tablet_tool_v2::ButtonState::Pressed)
);
acc.pending_buttons.push((b, pressed));
}
}
Event::Frame { .. } => {
for (b, pressed) in acc.pending_buttons.drain(..) {
let _ = state.tx.send(RawTabletEvent::Button { button: b, pressed });
}
// Flush accumulated events to the channel.
if let Some(prox) = acc.pending_proximity.take() {
if prox {
@ -772,7 +630,7 @@ mod wayland {
#[cfg(target_os = "linux")]
mod x11 {
use super::{RawTabletEvent, TabletButton, TabletToolType};
use super::{RawTabletEvent, TabletToolType};
use std::sync::mpsc;
use winit::raw_window_handle::{XcbDisplayHandle, XlibDisplayHandle};
@ -1022,44 +880,17 @@ mod x11 {
}
Event::XinputRawButtonPress(raw) => {
if device_axes.contains_key(&raw.deviceid) {
// X11 reports the tip as button 1 and the barrel buttons as 2 and 3.
match raw.detail {
1 => {
let _ = tx.send(RawTabletEvent::TipDown);
let _ = tx.send(RawTabletEvent::Frame);
}
2 | 3 => {
let button = if raw.detail == 2 {
TabletButton::Lower
} else {
TabletButton::Upper
};
let _ = tx.send(RawTabletEvent::Button { button, pressed: true });
let _ = tx.send(RawTabletEvent::Frame);
}
_ => {}
}
if device_axes.contains_key(&raw.deviceid) && raw.detail == 1 {
let _ = tx.send(RawTabletEvent::TipDown);
let _ = tx.send(RawTabletEvent::Frame);
}
}
Event::XinputRawButtonRelease(raw) => {
if device_axes.contains_key(&raw.deviceid) {
match raw.detail {
1 => {
let _ = tx.send(RawTabletEvent::TipUp);
let _ = tx.send(RawTabletEvent::Frame);
}
2 | 3 => {
let button = if raw.detail == 2 {
TabletButton::Lower
} else {
TabletButton::Upper
};
let _ = tx.send(RawTabletEvent::Button { button, pressed: false });
let _ = tx.send(RawTabletEvent::Frame);
}
_ => {}
if raw.detail == 1 {
let _ = tx.send(RawTabletEvent::TipUp);
let _ = tx.send(RawTabletEvent::Frame);
}
// When all buttons released, synthesise proximity out.
in_proximity.remove(&raw.deviceid);

View File

@ -1,6 +1,6 @@
use super::{BrushKind, RasterToolDef, RasterToolSettings};
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::raster_layer::RasterBlendMode;
use lightningbeam_core::{brush_settings::BrushSettings, raster_layer::RasterBlendMode};
pub struct BlurSharpenTool;
pub static BLUR_SHARPEN: BlurSharpenTool = BlurSharpenTool;
@ -8,11 +8,19 @@ pub static BLUR_SHARPEN: BlurSharpenTool = BlurSharpenTool;
impl RasterToolDef for BlurSharpenTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::BlurSharpen }
fn header_label(&self) -> &'static str { "Blur / Sharpen" }
fn brush_kind(&self) -> BrushKind { BrushKind::BlurSharpen }
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
BrushParams {
base_settings: BrushSettings::default(),
radius: s.blur_sharpen_radius,
opacity: s.blur_sharpen_strength,
hardness: s.blur_sharpen_hardness,
spacing: s.blur_sharpen_spacing,
}
}
fn tool_params(&self, s: &RasterToolSettings) -> [f32; 4] {
[s.blur_sharpen_mode as f32, s.blur_sharpen_kernel, 0.0, 0.0]
}
fn strength_label(&self) -> &'static str { "Strength" }
fn show_brush_preset_picker(&self) -> bool { false }
fn render_ui(&self, ui: &mut egui::Ui, s: &mut RasterToolSettings) {
ui.horizontal(|ui| {
if ui.selectable_label(s.blur_sharpen_mode == 0, "Blur").clicked() {
@ -22,11 +30,31 @@ impl RasterToolDef for BlurSharpenTool {
s.blur_sharpen_mode = 1;
}
});
ui.horizontal(|ui| {
ui.label("Size:");
ui.add(egui::Slider::new(&mut s.blur_sharpen_radius, 1.0_f32..=500.0).logarithmic(true).suffix(" px"));
});
ui.horizontal(|ui| {
ui.label("Strength:");
ui.add(egui::Slider::new(&mut s.blur_sharpen_strength, 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Hardness:");
ui.add(egui::Slider::new(&mut s.blur_sharpen_hardness, 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Kernel:");
ui.add(egui::Slider::new(&mut s.blur_sharpen_kernel, 1.0_f32..=20.0)
.logarithmic(true)
.custom_formatter(|v, _| format!("{:.1} px", v)));
});
ui.horizontal(|ui| {
ui.label("Spacing:");
ui.add(egui::Slider::new(&mut s.blur_sharpen_spacing, 0.5_f32..=20.0)
.logarithmic(true)
.custom_formatter(|v, _| format!("{:.1}", v)));
});
}
}

View File

@ -1,4 +1,4 @@
use super::{BrushKind, RasterToolDef, RasterToolSettings};
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::raster_layer::RasterBlendMode;
@ -8,7 +8,15 @@ pub static CLONE_STAMP: CloneStampTool = CloneStampTool;
impl RasterToolDef for CloneStampTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::CloneStamp }
fn header_label(&self) -> &'static str { "Clone Stamp" }
fn brush_kind(&self) -> BrushKind { BrushKind::CloneStamp }
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
BrushParams {
base_settings: s.active_brush_settings.clone(),
radius: s.brush_radius,
opacity: s.brush_opacity,
hardness: s.brush_hardness,
spacing: s.brush_spacing,
}
}
/// For Clone Stamp, tool_params are filled by stage.rs at stroke-start time
/// (offset = clone_source - stroke_start), not from settings directly.
fn tool_params(&self, _s: &RasterToolSettings) -> [f32; 4] { [0.0; 4] }

View File

@ -1,6 +1,6 @@
use super::{BrushKind, RasterToolDef, RasterToolSettings};
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::raster_layer::RasterBlendMode;
use lightningbeam_core::{brush_settings::BrushSettings, raster_layer::RasterBlendMode};
pub struct DodgeBurnTool;
pub static DODGE_BURN: DodgeBurnTool = DodgeBurnTool;
@ -8,11 +8,19 @@ pub static DODGE_BURN: DodgeBurnTool = DodgeBurnTool;
impl RasterToolDef for DodgeBurnTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::DodgeBurn }
fn header_label(&self) -> &'static str { "Dodge / Burn" }
fn brush_kind(&self) -> BrushKind { BrushKind::DodgeBurn }
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
BrushParams {
base_settings: BrushSettings::default(),
radius: s.dodge_burn_radius,
opacity: s.dodge_burn_exposure,
hardness: s.dodge_burn_hardness,
spacing: s.dodge_burn_spacing,
}
}
fn tool_params(&self, s: &RasterToolSettings) -> [f32; 4] {
[s.dodge_burn_mode as f32, 0.0, 0.0, 0.0]
}
fn strength_label(&self) -> &'static str { "Exposure" }
fn show_brush_preset_picker(&self) -> bool { false }
fn render_ui(&self, ui: &mut egui::Ui, s: &mut RasterToolSettings) {
ui.horizontal(|ui| {
if ui.selectable_label(s.dodge_burn_mode == 0, "Dodge").clicked() {
@ -22,5 +30,25 @@ impl RasterToolDef for DodgeBurnTool {
s.dodge_burn_mode = 1;
}
});
ui.horizontal(|ui| {
ui.label("Size:");
ui.add(egui::Slider::new(&mut s.dodge_burn_radius, 1.0_f32..=500.0).logarithmic(true).suffix(" px"));
});
ui.horizontal(|ui| {
ui.label("Exposure:");
ui.add(egui::Slider::new(&mut s.dodge_burn_exposure, 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Hardness:");
ui.add(egui::Slider::new(&mut s.dodge_burn_hardness, 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Spacing:");
ui.add(egui::Slider::new(&mut s.dodge_burn_spacing, 0.5_f32..=20.0)
.logarithmic(true)
.custom_formatter(|v, _| format!("{:.1}", v)));
});
}
}

View File

@ -1,4 +1,5 @@
use super::{BrushKind, RasterToolDef};
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::raster_layer::RasterBlendMode;
pub struct EraseTool;
@ -7,6 +8,16 @@ pub static ERASE: EraseTool = EraseTool;
impl RasterToolDef for EraseTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::Erase }
fn header_label(&self) -> &'static str { "Eraser" }
fn brush_kind(&self) -> BrushKind { BrushKind::Erase }
fn tool_params(&self, _s: &super::RasterToolSettings) -> [f32; 4] { [0.0; 4] }
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
BrushParams {
base_settings: s.active_eraser_settings.clone(),
radius: s.eraser_radius,
opacity: s.eraser_opacity,
hardness: s.eraser_hardness,
spacing: s.eraser_spacing,
}
}
fn tool_params(&self, _s: &RasterToolSettings) -> [f32; 4] { [0.0; 4] }
fn is_eraser(&self) -> bool { true }
fn render_ui(&self, _ui: &mut egui::Ui, _s: &mut RasterToolSettings) {}
}

View File

@ -1,4 +1,4 @@
use super::{BrushKind, RasterToolDef, RasterToolSettings};
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::raster_layer::RasterBlendMode;
@ -8,7 +8,15 @@ pub static HEALING_BRUSH: HealingBrushTool = HealingBrushTool;
impl RasterToolDef for HealingBrushTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::Healing }
fn header_label(&self) -> &'static str { "Healing Brush" }
fn brush_kind(&self) -> BrushKind { BrushKind::HealingBrush }
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
BrushParams {
base_settings: s.active_brush_settings.clone(),
radius: s.brush_radius,
opacity: s.brush_opacity,
hardness: s.brush_hardness,
spacing: s.brush_spacing,
}
}
/// tool_params are filled by stage.rs at stroke-start time (clone offset).
fn tool_params(&self, _s: &RasterToolSettings) -> [f32; 4] { [0.0; 4] }
fn uses_alt_click(&self) -> bool { true }

View File

@ -2,17 +2,12 @@
///
/// Each tool implements `RasterToolDef`. Adding a new tool requires:
/// 1. A new file in this directory implementing `RasterToolDef`.
/// 2. A `BrushKind` variant (if it paints dabs) and one entry in `raster_tool_def()` below.
/// 2. One entry in `raster_tool_def()` below.
/// 3. Core changes: `RasterBlendMode` variant, `brush_engine.rs` constant, WGSL branch.
///
/// Every dab-painting tool owns a `BrushSlot`: its own size/strength/hardness/spacing, its own
/// brush from the shared `.myb` library, and its own FG/BG color choice. The blend mode is what
/// makes a tool a brush vs. an eraser vs. dodge/burn — the brush shape is orthogonal, so all of
/// them get the same library and the same controls.
use eframe::egui;
use lightningbeam_core::{
brush_settings::{bundled_brushes, BrushSettings},
brush_settings::BrushSettings,
raster_layer::RasterBlendMode,
tool::Tool,
};
@ -27,86 +22,6 @@ pub mod dodge_burn;
pub mod sponge;
pub mod blur_sharpen;
// ---------------------------------------------------------------------------
// Brush slots — one per dab-painting tool
// ---------------------------------------------------------------------------
/// Identifies a tool's brush slot. Each slot remembers its own brush independently, so
/// switching from Dodge to Sponge doesn't clobber either one's size or preset.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub enum BrushKind {
Paint,
Erase,
Smudge,
CloneStamp,
HealingBrush,
PatternStamp,
DodgeBurn,
Sponge,
BlurSharpen,
}
impl BrushKind {
fn index(self) -> usize {
match self {
BrushKind::Paint => 0,
BrushKind::Erase => 1,
BrushKind::Smudge => 2,
BrushKind::CloneStamp => 3,
BrushKind::HealingBrush => 4,
BrushKind::PatternStamp => 5,
BrushKind::DodgeBurn => 6,
BrushKind::Sponge => 7,
BrushKind::BlurSharpen => 8,
}
}
}
/// One tool's brush: the shared controls every dab-painting tool has.
#[derive(Debug, Clone)]
pub struct BrushSlot {
pub radius: f32,
/// What this means is per-tool — opacity, exposure, flow, strength. The label comes from
/// [`RasterToolDef::strength_label`].
pub strength: f32,
pub hardness: f32,
pub spacing: f32,
/// The brush shape, from the bundled `.myb` library.
pub settings: BrushSettings,
/// Index into `bundled_brushes()` of the selected preset, if any.
pub preset: Option<usize>,
/// Added to the preset's `elliptical_dab_angle` (degrees), so stock brushes can be
/// re-oriented without editing the file.
pub angle_offset: f32,
/// true = paint with the FG (stroke) color, false = BG (fill). Ignored when the tool
/// doesn't use a color (see [`RasterToolDef::uses_color`]).
pub use_fg: bool,
}
impl BrushSlot {
fn new(radius: f32, strength: f32, hardness: f32, spacing: f32) -> Self {
Self {
radius,
strength,
hardness,
spacing,
settings: BrushSettings::default(),
preset: None,
angle_offset: 0.0,
use_fg: true,
}
}
/// Adopt a brush preset, pulling its opacity/hardness/spacing along with the shape.
pub fn apply_preset(&mut self, index: usize, settings: &BrushSettings) {
self.preset = Some(index);
self.strength = settings.opaque.clamp(0.0, 1.0);
self.hardness = settings.hardness.clamp(0.0, 1.0);
self.spacing = settings.dabs_per_radius;
self.settings = settings.clone();
}
}
// ---------------------------------------------------------------------------
// Shared settings struct (replaces 20+ individual SharedPaneState / EditorApp fields)
// ---------------------------------------------------------------------------
@ -114,8 +29,25 @@ impl BrushSlot {
/// All per-tool settings for raster painting. Owned by `EditorApp`; borrowed
/// by `SharedPaneState` as a single `&'a mut RasterToolSettings`.
pub struct RasterToolSettings {
/// Per-tool brush state, indexed by `BrushKind`.
brushes: [BrushSlot; 9],
// --- Paint brush ---
pub brush_radius: f32,
pub brush_opacity: f32,
pub brush_hardness: f32,
pub brush_spacing: f32,
/// true = paint with FG (stroke) color, false = BG (fill) color
pub brush_use_fg: bool,
pub active_brush_settings: BrushSettings,
// --- Eraser ---
pub eraser_radius: f32,
pub eraser_opacity: f32,
pub eraser_hardness: f32,
pub eraser_spacing: f32,
pub active_eraser_settings: BrushSettings,
// --- Smudge ---
pub smudge_radius: f32,
pub smudge_hardness: f32,
pub smudge_spacing: f32,
pub smudge_strength: f32,
// --- Clone / Healing ---
/// World-space source point set by Alt+click.
pub clone_source: Option<egui::Vec2>,
@ -123,12 +55,24 @@ pub struct RasterToolSettings {
pub pattern_type: u32,
pub pattern_scale: f32,
// --- Dodge / Burn ---
pub dodge_burn_radius: f32,
pub dodge_burn_hardness: f32,
pub dodge_burn_spacing: f32,
pub dodge_burn_exposure: f32,
/// 0 = dodge (lighten), 1 = burn (darken)
pub dodge_burn_mode: u32,
// --- Sponge ---
pub sponge_radius: f32,
pub sponge_hardness: f32,
pub sponge_spacing: f32,
pub sponge_flow: f32,
/// 0 = saturate, 1 = desaturate
pub sponge_mode: u32,
// --- Blur / Sharpen ---
pub blur_sharpen_radius: f32,
pub blur_sharpen_hardness: f32,
pub blur_sharpen_spacing: f32,
pub blur_sharpen_strength: f32,
/// Neighborhood kernel radius in canvas pixels (120)
pub blur_sharpen_kernel: f32,
/// 0 = blur, 1 = sharpen
@ -143,7 +87,7 @@ pub struct RasterToolSettings {
// --- Quick Select ---
/// Brush radius in canvas pixels for the quick-select tool.
pub quick_select_radius: f32,
// --- Flood fill (Paint Bucket) ---
// --- Flood fill (Paint Bucket, raster) ---
/// Color-distance threshold (Euclidean RGBA, 0510). Pixels within this
/// distance of the comparison color are included in the fill.
pub fill_threshold: f32,
@ -152,11 +96,6 @@ pub struct RasterToolSettings {
/// Whether to compare each pixel to the seed pixel (Absolute) or to its BFS
/// parent pixel (Relative, spreads across gradients).
pub fill_threshold_mode: FillThresholdMode,
/// true = fill with the FG (stroke) color, false = BG (fill). Mirrors `BrushSlot::use_fg`.
pub fill_use_fg: bool,
// --- Eyedropper ---
/// true = sampled color replaces the FG (stroke) swatch, false = the BG (fill) swatch.
pub eyedropper_use_fg: bool,
// --- Marquee select shape ---
/// Whether the rectangular select tool draws a rect or an ellipse.
pub select_shape: SelectionShape,
@ -170,16 +109,10 @@ pub struct RasterToolSettings {
// --- Gradient ---
pub gradient: lightningbeam_core::gradient::ShapeGradient,
pub gradient_opacity: f32,
}
impl RasterToolSettings {
pub fn brush(&self, kind: BrushKind) -> &BrushSlot {
&self.brushes[kind.index()]
}
pub fn brush_mut(&mut self, kind: BrushKind) -> &mut BrushSlot {
&mut self.brushes[kind.index()]
}
// --- Brush rotation offset ---
/// User-controlled angle offset added to the brush's elliptical_dab_angle (degrees).
/// Lets the user re-orient stock .myb brushes without editing the file.
pub brush_angle_offset: f32,
}
/// Brush mode for the Liquify tool.
@ -225,32 +158,43 @@ pub enum FillThresholdMode {
impl Default for RasterToolSettings {
fn default() -> Self {
// The eraser defaults to the bundled "Brush" shape rather than a bare Gaussian.
let mut erase = BrushSlot::new(10.0, 1.0, 0.5, 0.1);
if let Some(idx) = bundled_brushes().iter().position(|p| p.name == "Brush") {
erase.apply_preset(idx, &bundled_brushes()[idx].settings);
// Keep the eraser's own size/opacity rather than the preset's.
erase.radius = 10.0;
erase.strength = 1.0;
}
Self {
brushes: [
/* Paint */ BrushSlot::new(10.0, 1.0, 0.5, 0.1),
/* Erase */ erase,
/* Smudge */ BrushSlot::new(15.0, 1.0, 0.8, 8.0),
/* CloneStamp */ BrushSlot::new(10.0, 1.0, 0.5, 0.1),
/* HealingBrush */ BrushSlot::new(10.0, 1.0, 0.5, 0.1),
/* PatternStamp */ BrushSlot::new(10.0, 1.0, 0.5, 0.1),
/* DodgeBurn */ BrushSlot::new(30.0, 0.5, 0.5, 3.0),
/* Sponge */ BrushSlot::new(30.0, 0.5, 0.5, 3.0),
/* BlurSharpen */ BrushSlot::new(30.0, 0.5, 0.5, 3.0),
],
brush_radius: 10.0,
brush_opacity: 1.0,
brush_hardness: 0.5,
brush_spacing: 0.1,
brush_use_fg: true,
active_brush_settings: BrushSettings::default(),
eraser_radius: 10.0,
eraser_opacity: 1.0,
eraser_hardness: 0.5,
eraser_spacing: 0.1,
active_eraser_settings: lightningbeam_core::brush_settings::bundled_brushes()
.iter()
.find(|p| p.name == "Brush")
.map(|p| p.settings.clone())
.unwrap_or_default(),
smudge_radius: 15.0,
smudge_hardness: 0.8,
smudge_spacing: 8.0,
smudge_strength: 1.0,
clone_source: None,
pattern_type: 0,
pattern_scale: 32.0,
dodge_burn_radius: 30.0,
dodge_burn_hardness: 0.5,
dodge_burn_spacing: 3.0,
dodge_burn_exposure: 0.5,
dodge_burn_mode: 0,
sponge_radius: 30.0,
sponge_hardness: 0.5,
sponge_spacing: 3.0,
sponge_flow: 0.5,
sponge_mode: 0,
blur_sharpen_radius: 30.0,
blur_sharpen_hardness: 0.5,
blur_sharpen_spacing: 3.0,
blur_sharpen_strength: 0.5,
blur_sharpen_kernel: 5.0,
blur_sharpen_mode: 0,
wand_threshold: 15.0,
@ -259,8 +203,6 @@ impl Default for RasterToolSettings {
fill_threshold: 15.0,
fill_softness: 0.0,
fill_threshold_mode: FillThresholdMode::Absolute,
fill_use_fg: true,
eyedropper_use_fg: true,
quick_select_radius: 20.0,
select_shape: SelectionShape::Rect,
warp_grid_cols: 4,
@ -270,6 +212,7 @@ impl Default for RasterToolSettings {
liquify_strength: 0.5,
gradient: lightningbeam_core::gradient::ShapeGradient::default(),
gradient_opacity: 1.0,
brush_angle_offset: 0.0,
}
}
}
@ -286,21 +229,6 @@ pub struct BrushParams {
pub spacing: f32,
}
/// Read a slot straight into `BrushParams`. This is what `RasterToolDef::brush_params` does by
/// default; it's a free function so a tool that overrides `brush_params` can still build on it.
pub fn default_brush_params(kind: BrushKind, s: &RasterToolSettings) -> BrushParams {
let slot = s.brush(kind);
let mut base_settings = slot.settings.clone();
base_settings.elliptical_dab_angle += slot.angle_offset;
BrushParams {
base_settings,
radius: slot.radius,
opacity: slot.strength,
hardness: slot.hardness,
spacing: slot.spacing,
}
}
// ---------------------------------------------------------------------------
// RasterToolDef trait
// ---------------------------------------------------------------------------
@ -308,34 +236,19 @@ pub fn default_brush_params(kind: BrushKind, s: &RasterToolSettings) -> BrushPar
pub trait RasterToolDef: Send + Sync {
fn blend_mode(&self) -> RasterBlendMode;
fn header_label(&self) -> &'static str;
/// Which brush slot this tool paints with.
fn brush_kind(&self) -> BrushKind;
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams;
/// Encode tool-specific state into the 4-float `StrokeRecord::tool_params`.
fn tool_params(&self, s: &RasterToolSettings) -> [f32; 4];
/// Brush shape + size for this stroke. The default pulls everything from the tool's slot;
/// override only if a tool needs to reinterpret one of the fields (see `smudge`).
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
default_brush_params(self.brush_kind(), s)
}
/// Cursor display radius (world pixels).
fn cursor_radius(&self, s: &RasterToolSettings) -> f32 {
self.brush_params(s).radius
}
/// Label for the slot's `strength` slider — the one field whose meaning is tool-specific.
fn strength_label(&self) -> &'static str { "Opacity" }
/// Render tool-specific controls in the infopanel. The shared brush controls (color, size,
/// strength, hardness, spacing, angle, preset library) are rendered around this by the
/// infopanel — only put genuinely tool-unique widgets here.
fn render_ui(&self, _ui: &mut egui::Ui, _s: &mut RasterToolSettings) {}
/// Whether this tool paints with the FG/BG color. False for tools that only transform
/// pixels already on the canvas (erase, smudge, dodge/burn, sponge, blur, clone, heal).
fn uses_color(&self) -> bool { false }
/// Render tool-specific controls in the infopanel (called before preset picker if any).
fn render_ui(&self, ui: &mut egui::Ui, s: &mut RasterToolSettings);
/// Whether to show the brush preset picker after `render_ui`.
fn show_brush_preset_picker(&self) -> bool { true }
/// Whether this tool is the eraser (drives preset picker + color UI visibility).
fn is_eraser(&self) -> bool { false }
/// Whether Alt+click sets a source point for this tool.
fn uses_alt_click(&self) -> bool { false }
}

View File

@ -1,4 +1,5 @@
use super::{BrushKind, RasterToolDef};
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::raster_layer::RasterBlendMode;
pub struct PaintTool;
@ -7,7 +8,17 @@ pub static PAINT: PaintTool = PaintTool;
impl RasterToolDef for PaintTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::Normal }
fn header_label(&self) -> &'static str { "Brush" }
fn brush_kind(&self) -> BrushKind { BrushKind::Paint }
fn tool_params(&self, _s: &super::RasterToolSettings) -> [f32; 4] { [0.0; 4] }
fn uses_color(&self) -> bool { true }
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
let mut base_settings = s.active_brush_settings.clone();
base_settings.elliptical_dab_angle += s.brush_angle_offset;
BrushParams {
base_settings,
radius: s.brush_radius,
opacity: s.brush_opacity,
hardness: s.brush_hardness,
spacing: s.brush_spacing,
}
}
fn tool_params(&self, _s: &RasterToolSettings) -> [f32; 4] { [0.0; 4] }
fn render_ui(&self, _ui: &mut egui::Ui, _s: &mut RasterToolSettings) {}
}

View File

@ -1,4 +1,4 @@
use super::{BrushKind, RasterToolDef, RasterToolSettings};
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::raster_layer::RasterBlendMode;
@ -12,12 +12,18 @@ const PATTERN_NAMES: &[&str] = &[
impl RasterToolDef for PatternStampTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::PatternStamp }
fn header_label(&self) -> &'static str { "Pattern Stamp" }
fn brush_kind(&self) -> BrushKind { BrushKind::PatternStamp }
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
BrushParams {
base_settings: s.active_brush_settings.clone(),
radius: s.brush_radius,
opacity: s.brush_opacity,
hardness: s.brush_hardness,
spacing: s.brush_spacing,
}
}
fn tool_params(&self, s: &RasterToolSettings) -> [f32; 4] {
[s.pattern_type as f32, s.pattern_scale, 0.0, 0.0]
}
/// The pattern is stamped in the brush color (see `brush_dab.wgsl`, blend mode 5).
fn uses_color(&self) -> bool { true }
fn render_ui(&self, ui: &mut egui::Ui, s: &mut RasterToolSettings) {
let selected_name = PATTERN_NAMES
.get(s.pattern_type as usize)
@ -38,5 +44,6 @@ impl RasterToolDef for PatternStampTool {
ui.add(egui::Slider::new(&mut s.pattern_scale, 4.0_f32..=256.0)
.logarithmic(true).suffix(" px"));
});
ui.add_space(4.0);
}
}

View File

@ -1,5 +1,6 @@
use super::{BrushKind, BrushParams, RasterToolDef, RasterToolSettings};
use lightningbeam_core::raster_layer::RasterBlendMode;
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::{brush_settings::BrushSettings, raster_layer::RasterBlendMode};
pub struct SmudgeTool;
pub static SMUDGE: SmudgeTool = SmudgeTool;
@ -7,15 +8,37 @@ pub static SMUDGE: SmudgeTool = SmudgeTool;
impl RasterToolDef for SmudgeTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::Smudge }
fn header_label(&self) -> &'static str { "Smudge" }
fn brush_kind(&self) -> BrushKind { BrushKind::Smudge }
fn tool_params(&self, _s: &RasterToolSettings) -> [f32; 4] { [0.0; 4] }
fn strength_label(&self) -> &'static str { "Strength" }
/// Smudge's slot `strength` drives the smudge distance (applied by the stage as
/// `smudge_radius_log`), not dab opacity — dabs always composite fully.
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
let mut p = super::default_brush_params(self.brush_kind(), s);
p.opacity = 1.0;
p
BrushParams {
base_settings: BrushSettings::default(),
radius: s.smudge_radius,
opacity: 1.0, // strength is a separate smudge_dist multiplier
hardness: s.smudge_hardness,
spacing: s.smudge_spacing,
}
}
fn tool_params(&self, _s: &RasterToolSettings) -> [f32; 4] { [0.0; 4] }
fn show_brush_preset_picker(&self) -> bool { false }
fn render_ui(&self, ui: &mut egui::Ui, s: &mut RasterToolSettings) {
ui.horizontal(|ui| {
ui.label("Size:");
ui.add(egui::Slider::new(&mut s.smudge_radius, 1.0_f32..=200.0).logarithmic(true).suffix(" px"));
});
ui.horizontal(|ui| {
ui.label("Strength:");
ui.add(egui::Slider::new(&mut s.smudge_strength, 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Hardness:");
ui.add(egui::Slider::new(&mut s.smudge_hardness, 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Spacing:");
ui.add(egui::Slider::new(&mut s.smudge_spacing, 0.5_f32..=20.0)
.logarithmic(true)
.custom_formatter(|v, _| format!("{:.1}", v)));
});
}
}

View File

@ -1,6 +1,6 @@
use super::{BrushKind, RasterToolDef, RasterToolSettings};
use super::{BrushParams, RasterToolDef, RasterToolSettings};
use eframe::egui;
use lightningbeam_core::raster_layer::RasterBlendMode;
use lightningbeam_core::{brush_settings::BrushSettings, raster_layer::RasterBlendMode};
pub struct SpongeTool;
pub static SPONGE: SpongeTool = SpongeTool;
@ -8,11 +8,19 @@ pub static SPONGE: SpongeTool = SpongeTool;
impl RasterToolDef for SpongeTool {
fn blend_mode(&self) -> RasterBlendMode { RasterBlendMode::Sponge }
fn header_label(&self) -> &'static str { "Sponge" }
fn brush_kind(&self) -> BrushKind { BrushKind::Sponge }
fn brush_params(&self, s: &RasterToolSettings) -> BrushParams {
BrushParams {
base_settings: BrushSettings::default(),
radius: s.sponge_radius,
opacity: s.sponge_flow,
hardness: s.sponge_hardness,
spacing: s.sponge_spacing,
}
}
fn tool_params(&self, s: &RasterToolSettings) -> [f32; 4] {
[s.sponge_mode as f32, 0.0, 0.0, 0.0]
}
fn strength_label(&self) -> &'static str { "Flow" }
fn show_brush_preset_picker(&self) -> bool { false }
fn render_ui(&self, ui: &mut egui::Ui, s: &mut RasterToolSettings) {
ui.horizontal(|ui| {
if ui.selectable_label(s.sponge_mode == 0, "Saturate").clicked() {
@ -22,5 +30,25 @@ impl RasterToolDef for SpongeTool {
s.sponge_mode = 1;
}
});
ui.horizontal(|ui| {
ui.label("Size:");
ui.add(egui::Slider::new(&mut s.sponge_radius, 1.0_f32..=500.0).logarithmic(true).suffix(" px"));
});
ui.horizontal(|ui| {
ui.label("Flow:");
ui.add(egui::Slider::new(&mut s.sponge_flow, 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Hardness:");
ui.add(egui::Slider::new(&mut s.sponge_hardness, 0.0_f32..=1.0)
.custom_formatter(|v, _| format!("{:.0}%", v * 100.0)));
});
ui.horizontal(|ui| {
ui.label("Spacing:");
ui.add(egui::Slider::new(&mut s.sponge_spacing, 0.5_f32..=20.0)
.logarithmic(true)
.custom_formatter(|v, _| format!("{:.1}", v)));
});
}
}

View File

@ -1,102 +0,0 @@
//! Shared color swatch + picker popup used by the toolbar FG/BG swatches and the
//! gradient-stop editor.
//!
//! egui's default popup close behavior is `CloseOnClick`, which closes on a click *anywhere* —
//! including on the picker's own hue/alpha bars. And `CloseOnClickOutside` only reacts to a
//! click (press+release without movement), so painting a stroke on the stage — a drag — would
//! leave the popup open. Both call sites want the same thing, so it lives here once.
use eframe::egui;
/// Show a color-picker popup anchored to `toggle_response`.
///
/// `swatch_rect` is excluded from the close-on-press check so the press that opens the popup
/// doesn't immediately close it again.
///
/// Returns true if the color changed this frame.
pub fn color_picker_popup(
ui: &mut egui::Ui,
popup_id: egui::Id,
toggle_response: &egui::Response,
color: &mut egui::Color32,
swatch_rect: egui::Rect,
) -> bool {
let mut changed = false;
let popup_response = egui::containers::Popup::from_toggle_button_response(toggle_response)
.id(popup_id)
.close_behavior(egui::PopupCloseBehavior::CloseOnClickOutside)
.show(|ui| {
ui.spacing_mut().slider_width = 275.0;
changed = egui::color_picker::color_picker_color32(
ui,
color,
egui::color_picker::Alpha::OnlyBlend,
);
});
// Close on any pointer *press* outside the popup, not just a click, so that starting a
// brush stroke on the stage dismisses the picker.
if let Some(popup) = &popup_response {
let popup_rect = popup.response.rect;
let pressed_outside = ui.ctx().input(|i| {
i.pointer.any_pressed()
&& i.pointer
.interact_pos()
.is_some_and(|p| !popup_rect.contains(p) && !swatch_rect.contains(p))
});
if pressed_outside {
egui::Popup::close_id(ui.ctx(), popup_id);
}
}
changed
}
/// A color button (checkerboard under the color, so alpha reads correctly) that opens a color
/// picker popup when clicked. Returns true if the color changed this frame.
pub fn color_swatch(
ui: &mut egui::Ui,
id: egui::Id,
rect: egui::Rect,
color: &mut egui::Color32,
) -> bool {
let response = ui.interact(rect, id, egui::Sense::click());
draw_color_button(ui, rect, *color);
color_picker_popup(ui, id.with("picker_popup"), &response, color, rect)
}
/// Draw a color button with a checkerboard background so the alpha channel is visible.
pub fn draw_color_button(ui: &mut egui::Ui, rect: egui::Rect, color: egui::Color32) {
let checker_size = 5.0;
let cols = (rect.width() / checker_size).ceil() as usize;
let rows = (rect.height() / checker_size).ceil() as usize;
for row in 0..rows {
for col in 0..cols {
let is_light = (row + col) % 2 == 0;
let checker_color = if is_light {
egui::Color32::from_gray(180)
} else {
egui::Color32::from_gray(120)
};
let checker_rect = egui::Rect::from_min_size(
egui::pos2(
rect.min.x + col as f32 * checker_size,
rect.min.y + row as f32 * checker_size,
),
egui::vec2(checker_size, checker_size),
)
.intersect(rect);
ui.painter().rect_filled(checker_rect, 0.0, checker_color);
}
}
ui.painter().rect_filled(rect, 2.0, color);
ui.painter().rect_stroke(
rect,
2.0,
egui::Stroke::new(1.0, egui::Color32::from_gray(80)),
egui::StrokeKind::Middle,
);
}

View File

@ -1,7 +1,6 @@
//! Reusable UI widgets for the editor
mod text_field;
pub mod color_swatch;
pub mod dropdown_list;
pub use text_field::ImeTextField;