- OnionSkinSettings (editor view state): enabled, frames_before/after (default 2/2),
opacity (0.35), warm past / cool future tints, linear falloff. Threaded App →
SharedPaneState (gated off during playback) → VelloRenderContext.
- Toggle: View ▸ Onion Skinning + the `O` shortcut (AppAction::ToggleOnionSkin).
- Tint: packed an RGB multiply into the canvas-blit matrix uniform's unused .w slots
(1,1,1 = no tint for all normal blits); BlitTransform::with_tint. Shader multiplies.
- Render: for the ACTIVE raster layer only, blit the N neighbouring keyframes (full
texture if resident, else the low-res proxy — uploaded on demand) tinted + faint,
composited behind the current frame. Off during playback.
Next: a settings panel (frame count/opacity) and vector-layer ghosts.
- Pointing-hand cursor when hovering a clickable keyframe diamond.
- Prefetch (Phase 3e, playback only): each update during playback, page in the next
few upcoming keyframes (PREFETCH_AHEAD=4) per raster layer that aren't resident, via
the existing async worker. Their full pixels land before the playhead reaches them,
so playback shows full frames instead of the low-res proxy on every frame (the
proxy→full pop was the "flicker"). Reactive faults still cover scrubbing.
render_layers now records each drawn keyframe diamond's screen rect + exact time in
`keyframe_diamond_hits`; handle_input hit-tests a click against them and sets the
playhead (and seeks the audio controller) to the keyframe's exact time. Uses the
previous frame's rects — diamonds don't move between frames, so the click lands
right — which sidesteps the input-before-render ordering and the drag/scroll Y math.
Works for both raster and vector keyframes.
Make raster layers behave like vector on the timeline.
- Timeline: draw a diamond per `RasterKeyframe` (mirrors the vector keyframe block).
- New Keyframe (K / menu): on a raster layer, insert a BLANK cel at the playhead via
a new undoable `AddRasterKeyframeAction` (+ `RasterLayer::insert_blank_keyframe_at`
/ `remove_keyframe`). Vector path unchanged.
- Stop lazy creation: paint tools now edit the ACTIVE keyframe (at-or-before the
playhead) instead of creating one. The brush captures the active keyframe's exact
time; `RasterStroke`/`RasterFillAction` resolve via `keyframe_at_mut` (error if
none); the tool-site `ensure_keyframe_at` blocks (brush/fill/bucket/wand/quick-
select/floating-lift) are removed — each read already bails when no keyframe exists.
New layers still seed a keyframe at the playhead, so there's normally one to paint
into; painting before the first keyframe is now a no-op (as intended).
Next: onion skinning.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The brush commits via a GPU-canvas readback and relies on the action's first
execute() to SET raw_pixels from that readback — at that point raw_pixels is empty
(new keyframe) or the pre-stroke state, never something a dirty-rect diff can stamp
onto. My initial diff-only execute skipped (to avoid corruption), so the stroke
disappeared.
Fix: the action keeps the full post-edit buffer ONLY for the first execute (the
commit), assigning raw_pixels outright exactly like the old code; it's taken/dropped
immediately, so the action sitting in the undo stack still retains just the small
diff. Redo replays via the diff onto the now-resident base.
Also harden the diff itself for the blank-base case: `before_blank` lets apply_after
build from a transparent buffer (redo of a first stroke after undo-to-blank) and
apply_before restore to empty; the resident-base skip is kept only for non-blank
bases (faulted in before undo/redo). Tests cover commit/redo from empty.
`RasterStrokeAction`/`RasterFillAction` stored the whole before+after RGBA frame
(~16 MB/action at 1080p → up to ~1.6 GB at the 100-action cap). They now store a
`RasterDiff` — only the changed bounding box's pixels before and after — computed
once in `new()` from the full buffers, which are then dropped. A brush dab shrinks
from ~16 MB to tens of KB; a full-canvas fill is unchanged (its bbox is the frame).
Paging interaction: a diff overwrites just the bbox, so the keyframe's pixels must
be resident when undo/redo applies. A clean evicted frame's container bytes equal
its current logical state, so the editor faults the target frame in (synchronously)
before undo/redo via a new `Action::raster_resident_hint` + `peek_undo/redo_raster_hint`.
Dirty frames are never evicted, so they're already resident. If a base is somehow
not resident the apply is skipped (logged), never resized-and-corrupted.
Unit tests cover exact before/after round-trip, blank-first-stroke, no-op, and the
non-resident-base skip.
The canvas blit used a nearest sampler, so the upscaled low-res proxy looked
blocky. Added a Linear sampler + `CanvasBlitPipeline::blit_smooth`; the raster
render uses it only for the proxy path (full-res canvas stays nearest/crisp). The
bind-group layout already declares the canvas texture filterable, so no layout
change was needed.
Scrubbing onto a paged-out raster keyframe flashed blank for the 1-2 frames its
full pixels took to page in. Now a low-res proxy is shown in that gap.
- core: `MediaKind::RasterProxy` (id derived from the keyframe id via
`raster_proxy_media_id`); `brush_engine::encode_raster_proxy_png` downscales a full
RGBA buffer to a ≤192px-long-edge PNG. Save writes a proxy beside each resident
frame's full PNG (paged-out frames keep their existing proxy row, like the full).
Load eagerly decodes proxies (small) into `RasterKeyframe::proxy`.
- editor: a separate `proxy_layer_cache` in the GPU brush (own recency LRU, budget 64
since each is ~1/100th a full frame) + `ensure_proxy_texture`/`get_proxy_texture`.
The raster render, when the full texture isn't resident, blits the proxy mapped to
the keyframe's FULL logical dims so it upscales via the sampler. F3 VRAM figure now
includes proxy textures.
When the full pixels land (async fault-in), the full path takes over automatically.
Proxies only exist after a save+reload; freshly-painted unsaved frames stay resident
so they need none.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`raster_layer_cache` (one ~w·h·16-byte Rgba16Float CanvasPair per keyframe) had no
size cap — scrubbing a long timeline grew VRAM without bound (~33 MB/frame at 1080p),
the largest unbounded consumer. Added a recency LRU (RASTER_LAYER_CACHE_MAX = 12):
`ensure_layer_texture` bumps the frame to most-recent and evicts the oldest past the
budget; the shown frame (and any rendered this pass) is always most-recent so it's
never the victim. Evicted textures re-upload cheaply from the resident/faulted-in
pixels on revisit. `remove_layer_texture` keeps the LRU in sync.
F3 debug overlay now reports the tracked VRAM (raster cache MB + frame count), pushed
from the GPU brush whenever the cache changes (wgpu exposes no allocator query).
`AudioPool::load_from_serialized` sizes the slot Vec by pool_index and fills gaps
with empty `AudioFile::new(PathBuf::new(), …)` placeholders. Two bugs let a
placeholder reach the next save and abort it with "Is a directory":
- Off-by-one: `entries.max().unwrap_or(0) + 1` made an *empty* pool length 1, so a
project with no audio still got one placeholder. Size by `max(pool_index + 1)`
→ empty entries yield length 0.
- `serialize()` emitted placeholder slots: an empty path round-trips to
`relative_path = Some("")`, which `save_beam` resolves to the project directory
(`join("")`) and tries to read as media. Skip empty-path / no-packed-media slots.
- Defense in `save_beam`: gate referenced-media packing on `full.is_file()` (not
`exists()`), so any blank/dir path falls through to embedded data instead of
reading a directory.
Pre-existing; surfaced by a save → reload → save cycle on a raster-only project.
Scrubbing a large paint project no longer accumulates every visited frame in RAM.
A fault-in-recency LRU keeps the most-recently-paged-in RASTER_RESIDENT_MAX (12)
keyframes resident and drops the pixels of older *clean* ones (re-arming their
fault-in so they re-page on revisit). The shown frame is always the most-recent
fault-in, so it's never evicted.
Data-loss safety: a new `dirty` flag marks any keyframe whose `raw_pixels` were
mutated by editing (stroke/fill/paint-bucket/floating-lift + their undo/redo) and
is NOT yet in the container. Dirty keyframes are NEVER evicted — they're only
unpinned from the LRU. The flag is cleared on a successful save, which also re-arms
the LRU for the now-clean resident frames so the bound still applies to frames
edited this session.
Also: the save loop now walks all layers (incl. nested) to match the load path's
recursive fault-in arming — evicted frames keep their existing container row
(media_exists), and nested raster keyframes are persisted + covered by live_media.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fault-in drain no longer decodes on the UI thread. It now:
- dispatches each newly-requested keyframe's page-in to a background thread
(deduped via an in-flight set, store cloned per request so path changes are
picked up), and
- applies completed results from a channel at the top of update(), keeping the UI
ticking while loads are outstanding.
Cold scrubs no longer freeze. The brief blank gap before a frame lands is removed
by the image proxy (3a-3); eviction to bound RAM while scrubbing is 3b.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Raster keyframes are no longer eagerly decoded at load — `raw_pixels` stays empty
and is paged in on demand from the project container, so a big paint project opens
instantly and only touched frames hit RAM.
- core: `read_packed_media_readonly` (fresh read-only connection, can't conflict
with an in-place save) + `RasterStore` (holds the container path; `load_pixels`
reads+decodes a keyframe's PNG by id). `load_beam_sqlite` stops eager-decoding and
instead marks every raster keyframe `needs_fault_in` (recursively, incl. nested);
a freshly-created keyframe stays false (blank-resident, nothing to page). Added
`Document::all_layers_mut`.
- editor: the canvas records a fault-in request when it needs a paged-out keyframe
(empty pixels && needs_fault_in); the App drains the sink at the top of update(),
pages the pixels in via the store, clears the flag, and repaints. Store path is set
on load and after save. Export faults in synchronously per frame.
Cold-scrub still shows a 1-frame gap and the page-in is synchronous; the image proxy
(3a-2) and async load (3a-3) remove those next.
The lib unit tests had gone stale (time values became newtypes) and no longer
compiled. Updated the test code to the current API and fixed the few real issues
the now-running tests surfaced.
Test-only:
- Wrap raw f64 time literals in Beats(...) where the API now takes Beats
(automation.rs); pass &TempoMap / Beats where signatures changed (clip.rs,
effect_layer.rs).
- shape.rs: assert the documented no-fill default (fill_color None) instead of Some.
- add_clip_instance / trim_clip_instances tests: register a vector clip with the
test's clip_id so the action's get_clip_duration lookup succeeds.
Production fix (delete_folder.rs):
- DeleteFolderAction(MoveToParent) reparented child subfolders to the deleted
folder's parent but never restored them on undo, orphaning them. Track the moved
subfolder ids and restore their parent on rollback.
Result: daw-backend lib 17 passed; lightningbeam-core lib 264 passed.
Surround → stereo downmix:
- render_from_file folds multichannel sources (5.1/7.1/…) down to stereo with
proper coefficients (full level for the matching front channel, 1/√2 for centre
+ each surround, LFE dropped), normalized per row to avoid clipping (matching
ffmpeg's default). Applied uniformly to both the direct-copy and sinc-resample
paths and to every storage type (PCM, compressed, video audio), only when
dst==2 && src>2; unknown layouts fall back to front L/R. Previously it just took
FL/FR, dropping centre dialog + surrounds.
Proper video-audio reload:
- A video's audio track is now stored as a path reference to the video (never
packed/embedded as audio media) and re-probed via FFmpeg on load into a
streaming VideoAudio entry, so multichannel audio survives reload (the old
Symphonia reconstitution collapsed it, breaking the downmix). Driven by a new
AudioPoolEntry.is_video_audio flag across serialize / save_beam / load. Also
removes the decode-whole-video-to-RAM + temp-file path on load.
Fix video scaling:
- Any video with dimensions larger than the stage was being scaled down into the corner incorrectly; we now bake the frame-clip scale into the instance transform.
Thumbnail rendering fixes:
- Strip now tiles from each clip's true (unclamped) origin and draws only the
tiles intersecting the visible rect, so it scrolls correctly and shows the
right frames when a clip is scrolled partly off the left. Both render sites
(collapsed group + expanded track) share one draw_video_thumbnail_strip helper.
- On-clip strip no longer freezes on the first thumbnail: get_thumbnail_at now
returns the actual thumbnail timestamp and the GPU texture cache keys on it, so
tiles refresh as closer thumbnails finish generating.
- Hover preview derives content time from the clip's true origin too (matches the
strip when scrolled off-screen).
- insert_thumbnail keeps the cache sorted + deduped (fixes a latent unsorted
binary_search bug, and makes concurrent restore + resume race-safe).
Thumbnail persistence (mirrors waveform persistence):
- MediaKind::Thumbnail rows, keyed by thumbnail_media_id(clip_id) (clip id XOR a
sentinel). Each clip's thumbnails PNG-encoded into one opaque LBTN blob (editor
owns the format), snapshotted cheaply (Arc clones) and encoded off the UI thread.
- Save writes the packs (kept in place on re-save); load reads them into
LoadedProject.thumbnail_blobs; the editor decodes + inserts them on a background
thread, so reload shows thumbnails instantly with no re-decode (even if the
source video file is missing).
- Partial sets are persisted with a complete flag and RESUMED on load:
generate_keyframe_thumbnails takes a should_skip predicate so a save made
mid-generation continues from where it left off instead of redoing the work.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- VideoManager.frame_cache: unbounded HashMap (grew per distinct frame during
playback) -> LruCache evicted by a 256MB byte budget. Byte-budget rather than
frame count is robust across resolutions (a 4K frame is ~33MB vs ~2MB at
800x600). unload_video pops per-clip keys (LruCache has no retain).
- mux_video_and_audio: stream-merge the two inputs by PTS with one pending
packet per stream (O(1) memory) instead of collecting every packet into Vecs
first (O(duration)). Output is byte-identical.
- export AAC: sanitize the planar-f32 path (non-finite -> 0, finite clamped to
[-1,1]) like the integer paths, with a one-time warning. A stray NaN/Inf
render sample no longer fails the whole export.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migrate the .beam container to SQLite and stream media from it instead of
decoding whole files into RAM on import/load.
Container & large files:
- SQLite .beam container (beam_archive) with in-place transactional saves and an
incremental BlobReader; supports both packed (chunked blobs) and referenced
(external path) media, with a user preference + first-import prompt for files
over the large-media threshold.
Audio streaming:
- Stream packed compressed audio on load via an inversion-of-control blob factory
(AudioBlobSourceFactory): daw-backend defines the trait, core implements it
over BlobReader, so the audio engine stays container-agnostic.
- Bulk-activate disk streaming for all loaded clips after SetProject.
- Sample-accurate compressed seek (SeekMode::Accurate; Coarse mislands on VBR).
Video:
- Video frames decoded/streamed on demand; thumbnails generated asynchronously
on a dedicated decoder so import/load never blocks the UI.
- The video's audio track is streamed on demand via an ffmpeg VideoAudioReader
as a separate editable AudioClip (no /tmp WAV extraction).
Waveform overview:
- Streaming min/max LOD pyramid (waveform_pyramid), bounded memory, configurable
floor B; serialized into the container and restored on load (or generated in
the background from the packed blob when absent), so no re-decode on reload.
- GPU min/max upload path; integer-LOD textureLoad fixes zoom-dependent wobble.
- Scale the document to the selected output resolution. It was rendered
at document size regardless of the export dimensions, so picking a
different resolution didn't scale the stage.
- Run the audio+video mux on a background thread instead of the UI
thread, keeping the app responsive (showing "Finalizing") during the
re-mux pass.
- Send desktop notifications fire-and-forget. notify_rust's show() is a
synchronous D-Bus call that blocked the UI for the full service
activation timeout (~25s) when no notification daemon is running.
Color correctness:
- Unpremultiply before the sRGB OETF on the display and export blits;
encoding premultiplied color corrupted antialiased/transparent edges.
- Tag exported video as full-range BT.709 (matrix/primaries/transfer).
- Run perception effects (invert, brightness/contrast, hue/saturation)
in gamma space to match standard editors.
- Interpolate gradients in gamma space across the raster and vector paths.
- Render effect thumbnails in the same linear space as the live pipeline.
Brush performance:
- Store the raster canvas as Rgba16Float (no shadow banding from 8-bit
linear), with an incremental per-tile ping-pong sync replacing the
per-frame full-canvas copy.
- Do the linear->sRGB readback conversion on the GPU and reuse a cached
scratch texture, dropping a ~110ms-per-stroke CPU decode.
Cleanup:
- Single COLOR_WGSL prelude and shared CPU sRGB scalars instead of ~8
duplicated copies of the transfer functions.
- Shared compute-pipeline builder; smudge folded onto the tile-sync path.
Skyler Lehmkuhl