The MIDI-clip-end-grows-too-fast bug was a units confusion: AudioClip.duration
is documented "seconds" but MIDI clips store their length in BEATS (they share
the AudioClip struct with sampled clips). The timeline's effective_clip_duration
read it as raw seconds, so at 120 BPM a MIDI clip rendered ~2x too long. The old
backend-snapshot display had hidden this by forcing timeline_duration in beats.
Root fix — make the domain explicit and unforgeable:
- `duration` is now a private field. Reading it in the wrong unit is impossible
because access goes through typed accessors: AudioClip::content_duration() ->
ClipDuration (a Seconds|Beats enum tagged by clip_type) and
set_content_duration(). serde still serializes the private field, so the .beam
format is unchanged (bare number).
- ClipDuration::to_seconds(tempo_map) for display/sizing; ::native() for code
that already works in the clip's native domain (trim math shares it).
- get_clip_duration + the timeline-endpoint calc go through to_seconds, so MIDI
is converted correctly; effective_clip_duration delegates to get_clip_duration.
- Recording mirrors (audio/MIDI progress + finalize) write via set_content_duration
(debug-asserts the value's domain matches the clip type).
Every former raw read/write of the field (core actions, timeline, piano roll,
infopanel, asset library, recording handlers) now goes through the accessors.
Whole workspace compiles; 299 core tests pass.
Thread Beats/Seconds through the editor now that lightningbeam-core is typed.
The timeline UI is seconds-domain (pixels_per_second, viewport_start_time,
playback_time, drag_offset), while ClipInstance.timeline_start/duration/
loop_before are beats — the compiler flagged every place the two were mixed.
Fixes the reported bugs and the whole class behind them:
- Recording placement (audio + MIDI + webcam): the new clip's timeline_start
was written from playback_time (seconds) into a beats field, so a second
recording landed at the wrong time. Now converted via the tempo map.
- Drag/move: introduce snapped_move_offset -> Beats (snap in the seconds/pixel
domain, express the anchor's movement in beats) and moved_start(); the group
clamp, live preview, and commit all use one uniform beats offset instead of
adding a seconds delta to a beats position.
- Trim/loop drag preview + commit: overlap limits come from the timeline
(beats) but trims are content seconds — converted at the boundary.
- Drop (asset drag, stage + timeline), paste, duplicate, split-at-playhead:
all convert the seconds drop/playhead position to beats before placing.
- Stage playback gates + clip-local time remap: compared seconds playback_time
against beats timeline_start; now both in seconds.
- Piano roll / infopanel / effect export: clip_dur is Seconds; effect and
waveform data converted at use.
Core API refinement (motivated by the above): find_max_trim_extend_left/right
now return Seconds (the content-seconds gap) instead of raw Beats, since every
trim caller wants seconds; loop-extend callers convert to beats. Added a
beats_to_x() timeline helper.
Whole workspace compiles; 299 core tests pass.
Closing the window with unsaved changes was a silent data-loss path — and worse,
a clean exit deletes the recovery file, removing the safety net too.
Fold quitting into the same unsaved-changes flow as file switches: one
PendingAction enum (NewFile / Open / Quit), one modal, one do_action, one
after_save handler. render_unsaved_prompt now also intercepts the window-close
request (CancelClose + queue a Quit prompt); Save & Quit saves then closes,
Discard & Quit closes now, Cancel keeps the window open. confirmed_close lets the
final programmatic close through instead of re-intercepting it.
Replaces the near-duplicate close-prompt/switch-prompt code with a single path.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
New File / Open / Open Recent previously discarded unsaved work with no warning
(three // TODO: Prompt to save markers). They now route through request_switch,
which shows a Save / Don't Save / Cancel modal when the document is modified:
• Save — saves first (Save As for untitled/recovered docs), then switches once
the background save completes (switch_after_save).
• Don't Save — switches and discards.
• Cancel — stays put.
"Modified" is tracked without hooking every edit site: current epoch vs a
saved_epoch baseline (rebased on manual save / new / load), plus a media_modified
flag for imports/recordings that don't bump the action epoch. Recovered documents
count as modified until given a real home, so switching away from unrecovered work
also prompts.
Recovering a file loaded it through the normal load path, which added it to
Recent Files — so an internal recovered-*.beam in the app data dir showed up in
the Recent list (and could even be auto-reopened as "last session").
Skip add_recent_file for recovery paths on load, and clean any that older builds
already leaked into Recent at startup (before the auto-reopen check reads it).
- LB_AUTOSAVE_SECS overrides the 45s autosave interval (e.g. =5) so the recovery
flow can be exercised without a long wait.
- Log the session's recovery file path at startup so it's easy to find.
Second half of autosave/recovery — the read/restore side.
At startup, scan the recovery dir for session-*.beam files. Since a clean exit
deletes this session's file (on_exit), any leftover means a previous session
crashed or was killed with unsaved work. Offer the newest via a modal:
• Recover — rename it out of the session-* namespace (to recovered-*) and open
it as the current file. It's renamed, not deleted, because recovered raster
keyframes page in from it on demand; renaming also stops it being re-offered.
• Discard — delete it.
• Later — keep it for next launch, stop prompting this session.
Because a recovered file lives in the app data dir, Save behaves as Save As for
it (both the menu action and the Save-As default directory skip recovery paths),
so the work lands where the user wants rather than back in the data dir.
recovered-* files older than a week are garbage-collected at startup.
First half of the autosave/recovery feature — the write side.
Every ~45s while the document is dirty, write the full current state into a
per-session recovery .beam in the app data dir (directories::ProjectDirs data
dir + /recovery/session-<uuid>.beam). Fully background: reuses the existing file
worker, so the pool serialization / encode / DB write all happen off the UI
thread. The only UI-thread cost is one document clone — and build_save_command
now stamps the UI layout onto the *snapshot clone* rather than the live document
(the old prepare_document_for_save mutated live state via Arc::make_mut, which
could deep-clone the whole document mid-frame). Removed that dead helper.
Dirtiness is tracked centrally via ActionExecutor::epoch() (now also bumped on
undo/redo, not just execute) plus a pending_event flag set by non-action changes
(imports, finished recordings). The baseline is rebased on new/load/manual-save
so a freshly-loaded or just-saved project stays quiet. Idle-after-edit still gets
one snapshot via a single request_repaint_after wakeup; completion is polled
lazily (no forced repaints during the write).
on_exit deletes the session's recovery file, so a leftover file on next launch
means an unclean shutdown — the hook the recovery prompt (next commit) keys on.
The prior audio-tags commit put real FLAC + metadata into export/audio_exporter.rs
— which turned out to be dead code (declared, never called; whole file was
EngineController::start_export_audio → daw-backend's export_audio, which still
routed FLAC to the erroring hound stub — hence "not implemented in daw-backend".
Move the work to where export actually happens:
- daw-backend/src/audio/export.rs: real ffmpeg FLAC (16-bit S16 / 24-bit S32,
skipping the trailing empty flush packet the FLAC muxer rejects); apply_metadata
on MP3/AAC/FLAC output; RIFF LIST/INFO chunk appended to WAV. New metadata field
on the backend ExportSettings, threaded from the UI in run_audio_export. Tests
assert real fLaC magic + round-tripped tags, and a valid WAV INFO chunk.
- Delete the dead export/audio_exporter.rs (removes the duplicate FLAC impl).
Smart tag defaults (filled only when empty, never clobbering edits):
- Year → current civil year, computed from the system clock with i64 math (no
date crate; correct past 2038/2106 — tests cover post-i32/u32 timestamps).
- Artist → last-used value, else the OS username ($USER/%USERNAME%).
- Album → last-used value.
Last-used Artist/Album persist in AppConfig and prefill next export.
New export format alongside audio/image/video/SVG. GIF is multi-frame like
video but palette-quantized with no audio, so it reuses the per-frame RGBA
render/readback path (render_frame_to_gpu_rgba) and streams frames to a
background encoder.
- core: GifExportSettings (resolution, framerate, loop, transparency, fit,
time range) with centisecond-quantized frame delay + tests.
- gif_exporter: encoder pipeline. Per-frame NeuQuant quantization is the
dominant cost and is per-frame independent, so it's fanned out across a
worker pool (cores-1, capped 8); a writer thread reorders and LZW-encodes
sequentially. Uses the `gif` crate directly (already resolved via `image`).
- orchestrator: start_gif_export + render_next_gif_frame (one frame per egui
update), wired into is_exporting/has_pending_progress/cancel.
- dialog: GIF tab + settings; main.rs: handle ExportResult::Gif and pump frames.
- Cargo: opt-level=3 for gif/color_quant/weezl in the dev profile so debug
builds aren't crippled by unoptimized NeuQuant loops.
Together these cut a 10s GIF export from ~1:43 to ~3s.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The backend's Command::Reset (full teardown: rebuilds Project, audio/buffer
pools, ID counters) was never invoked from the UI. New File and project loads
only cleared app-side maps, so the previous file's tracks and loaded
instruments stayed resident in the backend and kept getting mixed on every
audio callback.
Add a reset_audio_backend() helper (controller.reset() + clear app-side track
maps and backend-derived caches) and call it in the three teardown paths:
NewFile, create_new_project_with_focus, and apply_loaded_project. Also drops a
duplicate layer_to_track_map.clear() in the NewFile handler.
Race-free: the audio thread drains all command_tx commands before any query_tx
queries each callback, so Reset (a command) runs before the track/pool rebuild
(queries), even though they travel on separate channels.
Fixes from a review of the mobile-UI branch.
- Guard the inspector size clamps so min<=max (f32::clamp panics otherwise) and set a
window min_inner_size; prevents a crash / degenerate layout at small sizes. Same
small-size guards for the piano-roll landscape toggle bar and the intent grid.
- Landscape 2->1 edge reveal: snap to fullscreen at frac>=0.5 (the preview's phase
boundary) instead of COLLAPSE_HI, so releasing mid-drag no longer jumps backward.
Both edges.
- Gate the double-tap-drag marquee to mobile so it can't hijack a desktop Brush/Draw
drag.
- Timeline mobile long-press: request a repaint while counting down so it fires even
when the finger is held perfectly still.
- Landscape folded top bar: lay the filename+search+overflow cluster out within a
reserved center span (gutters for the pane's own label/buttons) and elide the
filename, so it can't overlap or overflow the header.
- Node editor connect/disconnect: assert + document the top-level-graph assumption.
- Palette search field focuses only when unfocused (no per-frame focus stomp); reset
the inspector tap-anchor on orientation change; remove stale main.rs.backup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gate the top menu bar on !mobile_active(). Its commands are all reachable from the
mobile shell (⌕ command palette, ⋯ overflow, and the stage/timeline context menus),
so the desktop menu bar is redundant on mobile and just wastes a strip of vertical
space. Desktop unchanged.
Make the phone shell orientation-aware (portrait vs landscape). Orientation is
aspect-based (rotation = a resize the shell re-lays-out each frame); LB_MOBILE_UI=2
opens a landscape dev window. `SharedPaneState.is_portrait` is published to panes.
- Stack caps at 2 panes in landscape (max_panes through the R1–R6 drag ops), with a
per-frame clamp so rotating while 3 are open drops to 2.
- Edge drags are a continuous full-height reveal instead of stalling at the capped
trigger: 1-pane → split → next-pane-fullscreen, and (landscape 2-pane) a two-phase
slide-then-collapse to 1 pane. Both top and bottom edges, mirrored. Release snaps to
stay / split / fullscreen. Dividers resize and collapse-to-1 at the extremes.
- Inspector becomes a right-side vertical column in landscape (inspector_width_frac,
left-edge horizontal resize); portrait keeps the bottom sheet.
- Piano roll: vertical "Synthesia" view in portrait; landscape shows a Keys/Notes
segmented toggle over the conventional horizontal roll / full keyboard.
- Start screen reflows to 2 rows × 3 with the recent list beside it in landscape.
- Landscape headers are shorter (52→34px) and the top bar folds into the middle of the
top pane header (no separate band), reclaiming vertical space.
All gated to mobile; desktop unchanged (is_portrait defaults true).
Hook up menu-bar actions that lacked a mobile affordance, and implement z-order.
- Stage/object long-press menu (stage.rs): add Paste (on clip, geometry, and
empty stage) and Send to Back / Bring to Front on the selection.
- Timeline long-press menu (timeline.rs): the timeline had no mobile context
menu — add one via manual long-press detection. Clip actions (Split,
Duplicate, Cut, Copy, Paste, Delete) on a clip; animation actions (New/Blank
keyframe, Add keyframe at playhead, Duplicate keyframe, Delete frame, Add
motion/shape tween) on an empty lane. Gate the desktop right-click menu to
!is_mobile so mobile shows only the long-press menu.
- Implement Send to Back / Bring to Front (were // TODO no-ops): new undoable
ReorderClipInstancesAction in lightningbeam-core reorders the selected
instances within their layer's clip_instances Vec (stacking order; last =
on top; geometry stays flattened underneath). Wired via handle_menu_action.
The mobile Piano Roll becomes a unified, keyboard-primary instrument surface
(wireframe Plate 04/08): a playable keyboard as the base, revealing a
Synthesia-style falling-notes roll above it when the pane is tall enough.
- Shared keyboard geometry (panes/keyboard_layout.rs): width-driven, pan-aware
pitch->x, so the roll's note columns stay aligned with the Virtual Piano keys.
Shared keyboard_octave + keyboard_pan_x in SharedPaneState.
- Piano Roll vertical mode (is_mobile): notes as columns falling toward an amber
now-line by the keys, tempo-map converted (beats->seconds) so onsets cross the
line exactly when they sound. Vertical drag scrubs the timeline; horizontal
drag smoothly scrolls the keys (snaps to nearest key on release). Long-press
creates a note (drag to size) or resizes an existing one; pan suppresses it.
The keyboard is embedded (reuses Virtual Piano render + MIDI); the standalone
VirtualPiano stack slot is removed. show_roll is driven by the snapped pane
size-class so the keyboard<->roll reveal lands on a stack snap point.
- Virtual Piano: renders via the shared layout on mobile, colors playback_notes
like pressed keys, gates note-on/glissando to presses that start on the keys
(never gating release), and hides QWERTY hints on mobile.
- Transport formats by document.timeline_mode (Measures->bar.beat.tick,
Frames->MM:SS:FF, Seconds->MM:SS.mmm) — per project type, like desktop.
- In-pane instrument header (name + Presets + REC). Recording is driven from the
app each frame (not the Timeline pane's render) so REC works regardless of
visible panes, and stopping playback stops recording.
- Compose/Record intent opens Timeline + instrument pane; mobile central panel is
full-bleed (no inner margin) so panes sit flush.
- phone-ui-sketches.html: inst-bar moved to the top of the music surface.
- Mobile double-tap is now a single tool-independent priority chain: object →
enter (movie clip or group); empty inside a clip → exit one level; empty at
root → zoom-to-fit. Desktop keeps its own select-tool double-click (gated).
- Inspector "tap outside to dismiss" now only HIDES the sheet (with a dismissed
flag reset on selection change) instead of clearing the selection — the earlier
clear clobbered the selection before menu/omnibutton actions ran, breaking
Group/Convert/Cut/Copy. Delete also clears focus so it dismisses the inspector.
- Breadcrumb shows the full editing path (Scene 1 › … › current) via a new
SharedPaneState.editing_clip_path / EditingContext::clip_path(); each ancestor
crumb jumps to that level (pending_exit_to_depth → EditingContext::exit_to_depth).
- Remove the "Vello Stage" debug overlay; move the breadcrumb up to the top-left.
Gestures (Stage + Timeline):
- Pinch-zoom via zoom_delta() (touch pinch + Ctrl+wheel), unified with the raw-wheel
path so Ctrl+wheel zooms exactly once; plain wheel zoom + trackpad pan preserved.
- Double-tap empty → zoom-to-fit (Stage: artboard via zoom_to_fit; Timeline: new
fit_to_project over the full duration).
- Double-tap-drag on empty → transient marquee regardless of the active tool.
Long-press context menu: a shared SharedPaneState.mobile_context_menu that panes
populate with (label, MenuAction) items on secondary_clicked(); the mobile shell
renders one persistent popup (styled like the timeline menu) and dispatches via
pending_menu_actions. Stage offers Cut/Copy/Duplicate/Delete for clip instances and
Cut/Copy/Convert-to-movie-clip/Delete for geometry.
Fixes surfaced along the way:
- Geometry delete now frees selected fills (free_fill) and GCs isolated vertices
(new VectorGraph::gc_isolated_vertices) — no more orphaned fills / phantom snap
points; applies to the Delete key too.
- clipboard_delete_selection clears focus so deleting dismisses the mobile inspector.
- Mobile inspector: appears on pointer release (not press) so drags aren't
interrupted; reflows only when the tapped point is actually behind the sheet; taps
outside the sheet dismiss it; rounded-corner border no longer detaches.
Add mobile::apply_touch_style(ctx) — applied every frame when mobile is active —
enlarging egui's global spacing/sizing (interact_size, button padding, slider
width/rail, combo width/height, icon sizes, scrollbar, body/button text) so the
standard widgets in panes and dialogs are finger-sized. Desktop is untouched.
Rework the mobile inspector header into a single row using real egui widgets:
title (left, truncating) + Timeline/Nodes jump buttons + ✕ close, laid out with
egui layouts so they inherit the touch sizing and theme visuals instead of being
hand-painted.
Add Theme::var(name, ctx) (CSS custom-property getter) and Theme::apply_to_egui(ctx)
which maps the palette onto egui's global Visuals so standard widgets share the
theme colors. Add a mobile::palette::Palette built from the theme each frame and
replace the duplicated per-file C_* color constants across all mobile modules
(omni, topbar, stack, inspector, transport, intent, shell) with it. Add --scrim
and --accent-* category vars to styles.css. Apply the theme visuals every frame in
update(). Mobile and the main UI now share one CSS-variable-driven palette that
responds to light/dark/user themes.
Add mobile::dialog_width(ctx, desired) — a min()-clamp that's a no-op on wide
desktop screens. Clamp the Export and Export Progress modals and the Sample
Import window to it. Render Preferences as an egui::Modal (dim backdrop, centered,
no title bar) with a screen-fit scroll height when mobile_active(); desktop keeps
its window. Extracted the preferences body into render_body() shared by both paths.
Omnibutton (mobile/omni.rs): a context FAB whose radial holds the active layer's
drawing tools laid out in a staggered inner/outer arc, plus a "+New" petal
(create Vector/Audio/MIDI/Raster/Video layer, Group, Import, and Split/Duplicate
when a clip is selected — dispatched as MenuActions) and a "more" petal that
opens a full-tools grid when the layer has extra tools. Contextual: the tool ring
only appears when the Stage is in the visible window; off-Stage the FAB is a
"+New" create button. Tools/creates use Lucide icons. Pane-internal adds
(nodes/notes/instruments) deferred to P6.
Playback fix: document.current_time (read by the renderer for animation eval and
video-frame decode) was only synced from the audio playback position inside
TimelinePane::render_content. In the mobile shell, panes render on demand, so with
the timeline off-screen current_time went stale and the Stage froze during
playback. Move the sync into the per-frame block in update() so it runs regardless
of which panes are visible.
- Add `is_mobile` to SharedPaneState (set in build_frame). On mobile the timeline
layer-header column collapses to a minimal type-colored swatch (1:2 w:h, ~30px),
brighter/outlined when active/selected; tapping it selects the layer (handled by
the existing header-column input path, which now uses the narrower width).
Desktop headers unchanged.
- Inspector reflow: the sheet overlaps the stack by default, but when the selected
pane would be covered by the sheet, the stack reflows (renders above the sheet)
so it stays visible. Restore on dismiss is automatic since reflow only changes
the render rect; manual divider moves while the sheet is up persist. Adds
stack::pane_bottom_in and inspector::target_slot.
When LB_MOBILE_UI is set, the desktop start screen is replaced by a phone
intent picker: a 2x3 grid of icon cards (Draw / Animate / Compose / Record /
Edit video / Blank) taking the top ~2/3, and a scrollable Recent-projects list
in the bottom ~1/3.
Each intent reuses EditorApp::create_new_project_with_focus and then sets the
initial stack window (e.g. Draw -> fullscreen Stage, Compose -> Timeline +
Piano Roll + Keys). Recent rows load the project and enter the editor. Cards are
icon-only (Lucide glyph + label) for legibility on a phone. Adds the intent
icons to the Lucide set.
Bundle Lucide (ISC license) at assets/fonts/lucide.ttf and register it as the
egui font family "lucide" via mobile/icons.rs::install (called in EditorApp::new).
Icons are drawn as text with icons::font(size) + PUA codepoint constants.
Switch the stack headers (grip, maximize/minimize, node⇄instrument), the footer
chevron, and the transport play/pause to Lucide glyphs instead of relying on
egui's limited built-in emoji-icon-font.
New src/mobile/ module renders a phone-style shell: top surface tabs
(Stage/Time/Nodes/Mixer/Tree), a single full-bleed hero pane reusing the
existing PaneInstance renderers, a resizable timeline ribbon (peek/half/full
snap tiers riding above the floor), and a fixed transport bar wired to the
audio controller (play/pause, timecode, project scrub).
Gated entirely on the LB_MOBILE_UI env var: when set, main() opens a
phone-aspect window and update() routes the central panel through
render_mobile_shell instead of render_layout_node, sharing the same
build_frame context and post-render drain. Desktop is unchanged when unset.
Tabs mirror the pane list; per the wireframe the Toolbar and Infopanel do
not become tabs (they become the omnibutton and inspector in later phases),
and the Nodes surface is a placeholder pending the focus/patch rework.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract the ~80-field SharedPaneState/RenderContext literal from update()
into EditorApp::build_frame, returning a FrameBundle that also carries the
layout-editing borrows (current_layout, drag_state, etc.). Per-frame locals
move into a FrameScratch struct so the bundle can borrow both self and the
scratch under one lifetime.
This is a behavior-preserving refactor that gives the upcoming mobile UI
path a single shared construction site for SharedPaneState instead of
duplicating the literal.
Introduce editable text layers: a resizable text box with editable text,
font size, color, font family, and alignment.
Core:
- New TextLayer/TextContent (text_layer.rs), wired into AnyLayer/LayerType
and all the exhaustive match sites; structured so content can be keyframed
later via content_at().
- fonts.rs: thread-local parley FontContext with three bundled fonts
(Liberation Sans/Serif/Mono, SIL OFL), system-font enumeration consolidated
to base families, document-embedded fonts, glyph/caret/selection geometry,
and a background preloader for the picker fonts.
- Rendering via parley layout + Scene::draw_glyphs (renderer.rs); text
composites through the vector path.
- Actions: CreateTextClipAction (vector-layer branch, undoable),
SetTextContentAction, ResizeTextBoxAction.
- .beam font embedding: MediaKind::Font rows (content-hash dedupe) written on
save and registered on load, with bundled-default fallback.
- VectorClip content bounds include text boxes so text-only clips are
selectable/draggable.
Editor:
- Text tool: click empty/raster/video to create a top-level text layer, or a
vector layer to create+enter a clip containing the text; click an existing
box to edit it.
- Hybrid in-place editing: a hidden egui TextEdit drives input/IME/caret while
the text and caret/selection render in Vello; empty just-created layers are
removed on commit.
- Selection outline + 8 resize handles (re-wrap text) with hover cursors;
factored the corner/edge resize-cursor mapping shared with the Transform tool.
- Info panel: edit text, size, color, alignment, box size, and a font-family
picker that previews each entry in its own font (fonts preloaded in the
background to avoid hitches).
Deps: add parley (git, pinned to match vello's peniko); bundle Liberation
fonts under lightningbeam-core/assets/fonts. Gitignore the local
.cargo/config.toml used to select a machine's ffmpeg.
- Wire MenuAction::ToggleLayerVisibility (was a stub) to flip the active
layer's `visible` flag via SetLayerPropertiesAction (undoable). This is
what the SVG-export visibility fix depends on, and the "Hide/Show Layer"
menu item now works.
- Log the resolved color range in the zero-copy H.264 path to diagnose
whether `full_range` reaches the encoder (the VAAPI driver may still omit
the SPS VUI full_range flag).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Export correctness:
- Honor the user's color-range (Limited/Full) on the software encode path:
thread full_range through gpu_yuv (new shader range uniform), the CPU
swscale fallback (sws_setColorspaceDetails → BT.709 + range, fixing the
BT.601 hue/level shift on odd-width exports), and the encoder color tags.
- Reject HDR + WebM up front with a clear message and log the forced HEVC
override instead of producing an unplayable file.
- Delete dead render_frame_to_rgba_hdr (hardcoded Stretch; live HDR path
already honors the fit mode).
Decode/playback (video.rs):
- Drain the decoder at EOF (send_eof + flush) so the final B-frame-delayed
frames render instead of erroring; per-frame logic extracted to a helper.
- Missing-PTS frames continue monotonically rather than snapping to ts=0.
- Force exact thumbnail width so sub-128px sources aren't shown stretched.
Resource leaks (gpu-video-encoder):
- dmabuf import_raw: RAII guard frees the duped fd + partial VkImages/memory
on every error path.
- vaapi alloc: free device/frames-ctx/AVFrames on the unexpected-DRM path.
Data model / robustness:
- collapse_boundary_spikes requires a full curve reversal (all control
points) so it no longer deletes a real lens/sliver and drops the fill.
- Export audio spin-wait ignores a stale `finished` flag when a forward
seek is pending (was rendering silence over real audio).
- RasterDiff apply_before/after take current dims and skip on a post-resize
mismatch.
- beam_archive read_media_full caps the preallocation from untrusted total_len.
UI/visual:
- SVG export skips hidden layers/empty groups; import folds fill-opacity into
gradients and surfaces failures as a notification.
- Active raster-layer border uses playback_time + overlay_transform.
- gpu_brush remove_layer_texture also evicts the stale low-res proxy.
- ensure_raster_resident_for_undo registers faulted frames in the LRU so
resident RAM stays bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Export (lightningbeam-core/svg_export.rs): document_to_svg walks vector
layers/groups at a given frame and emits <path> per fill (solid or
gradient via <defs>) and per stroked edge. Wired into the export dialog
as an "SVG" tab; written synchronously. Raster/video/effect layers are
skipped (vector-only, lossless), structured for a later rasterize pass.
Import (lightningbeam-editor/svg_import.rs): import_svg parses via usvg,
bakes each path's absolute transform into geometry, converts segments to
cubic edges, and maps solid/linear/radial paint to ShapeColor/
ShapeGradient. .svg is detected in the Ctrl+I Import handler and added as
a new vector layer (keyframe at the playhead). file_types gains
FileType::Vector + VECTOR_EXTENSIONS.
Tests: svg_export::export_tests (core) and svg_import::tests (editor).
A) Imported video had a different aspect ratio depending on how it was placed:
direct import used a uniform scale + center (preserve aspect), while the
asset-library timeline drag used an independent scale_x/scale_y (stretch to fill).
Add Transform::fit_centered (uniform scale, centered, aspect-preserving) and route
both paths through it, so a clip looks identical however it's added.
B) Exported video was stretched when the export resolution's aspect differed from
the document's (base_transform was always scale_non_uniform). Add
ExportFitMode {Stretch, Letterbox (default), Crop} on VideoExportSettings + a "Fit"
dropdown in advanced export settings, and a shared export_base_transform() helper:
Letterbox = uniform fit centered (black bars), Crop = uniform fill centered (trim),
Stretch = the old distort-to-fill. Threaded through the software, HDR, and
zero-copy export render paths (image export is doc-sized → identity).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Makes the common Linux H.264 export fully GPU-resident: decode (HW NV12) →
composite → VAAPI encode all on one device, no CPU round-trips.
- gpu-video-encoder: ZeroCopyEncoder now holds wgpu (device,queue,adapter) handles
instead of owning a DrmDevice. New `new_on_device(device,queue,adapter,...)` runs
the RGBA→NV12 render + DMA-BUF import on a passed device; `new` keeps building its
own. The encoder only *imports* VAAPI surfaces (not export), so the shared
import-capable device works directly.
- editor: stash the shared device handles on EditorApp (set in the creation closure
from the eframe render_state when the shared device is active) and thread them
through start_video_export / start_video_with_audio_export → try_build_zero_copy,
which uses new_on_device when available. ZeroCopyVideo carries on_shared_device →
the export composite uses hardware_ok=true (consuming HW-decoded GPU frames from
pt 1) only then.
Falls back to the own-device encoder (decode downloads to CPU) when no shared
device. Tradeoff: the export now shares the GPU with the UI thread (was a separate
device); acceptable for the GPU-resident-decode win. Compiles; crate tests build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the dormant core HW-decode engine live for the preview path:
- hw_video.rs: editor's HwVideoImporter — maps a decoded VAAPI surface to a
DRM-PRIME DMA-BUF and imports it as wgpu NV12 plane textures on the *shared*
device (the only one with the import extensions). install() creates the VAAPI
device and injects it + the importer into the VideoManager.
- main.rs: track whether the shared device is actually in use; only then (Linux,
not LB_NO_SHARED_DEVICE) install hardware decode, using the CreationContext's
shared device + adapter.
- nv12_blit.rs + nv12_blit.wgsl: NV12 plane textures → BT.709 → sRGB-encoded →
linear, written straight into the Rgba16Float HDR layer (no CPU upload). Colour
math mirrors the software path so HW/SW video match; honours full_range.
- stage.rs: the preview Video arm branches on inst.gpu (NV12 blit) vs rgba_data
(existing upload+blit_straight); sets render_hardware_ok = !cpu_renderer so the
CPU fallback still gets software frames.
- video_exporter.rs: sets render_hardware_ok(false) before both compositing
passes — export composites on the encoder's separate device, so a hardware
decoder downloads to CPU instead (export stays software, correct).
- dmabuf.rs: imported plane textures now also carry SAMPLED/TEXTURE_BINDING so
they can be sampled by the NV12 blit (they were render-target-only); into_planes
hands the textures to the longer-lived GpuVideoFrame.
- video.rs: cache-key the GPU/CPU representation on want_gpu (HW-configured AND
render_hardware_ok) so software-only decode keeps a single cache entry.
Preview only this pass; export GPU-residency is the 3c-export follow-up. Untested
at runtime here (no GPU/display in container) — both crates compile.
Foundation for hardware video decode in BOTH preview and export: wgpu textures
can't cross devices, and a hardware-decoded frame is a DMA-BUF-imported texture
that needs the import extensions (only addable via wgpu-hal device_from_raw). So
eframe + the compositor + decode + encode must share ONE custom device.
- vk_device::create_windowed(): the existing import-capable DrmDevice plus
VK_KHR_swapchain (the WSI surface instance extensions are already enabled by
Instance::init), for use as the editor's main device.
- main.rs: on Linux, build the shared device and inject it into eframe via
WgpuSetup::Existing (the egui fork supports it); fall back to wgpu's normal
device + software decode on any failure, on other platforms, or via
LB_NO_SHARED_DEVICE. The DrmDevice's wgpu handles are cloned into eframe
(Arc-backed), so the VkDevice persists with them.
Runtime-verified: the editor launches and renders identically (canvas/vello,
video, panels) on the shared device, and the env-var fallback works.
Video-only (no-audio) H.264 export now uses the same background-thread zero-copy
VAAPI path as video+audio, instead of the software encoder. It writes the output
.mp4 directly (no mux step) and reports through the single-export progress
channel; completion clears that channel so the dialog closes cleanly.
Factored the encoder/renderer/resources setup shared by both entry points into
ExportOrchestrator::try_build_zero_copy. start_video_export gains the document/
video_manager/raster_store/container_path inputs to seed the off-thread render
from a Document snapshot. Non-H264 / non-VAAPI still falls back to software.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The zero-copy VAAPI export previously rendered one frame per egui repaint on
the UI thread, which pinned throughput to the 60Hz vsync of the present loop
(measured exactly 16ms/frame) -- so the near-free hardware encode bought
nothing. Because the export runs on its own VAAPI device (independent of
eframe), it can run entirely off the UI thread.
- run_zerocopy_video_export: a background thread owning the encoder + its own
vello renderer/device, a Document snapshot (Document is Clone+Send; the UI
keeps the live one), its own ImageCache, and a RasterStore clone. Renders +
hardware-encodes every frame and reports through the same video_progress
channel the software encoder thread uses.
- start_video_with_audio_export takes the document/video_manager/raster_store/
container_path to seed the thread; video_state is None for this path.
- Throttle export-time UI repaints (~6Hz) and the thread's progress sends so
the render thread keeps the cores; the breakdown print stays.
- cancel() tears down parallel_export (detaches threads, removes temp files)
so the progress dialog dismisses; the call site closes the dialog.
- Gate the progress poll loop on has_pending_progress() so it stops once the
export ends instead of polling/logging every repaint forever; the single
export path clears its channel on the terminal event.
Vsync overhead is gone (0.1ms/frame); export is now render-bound (~11ms/frame
Vello scene-build). ~1:50 -> ~56s (~2x) on the validation clip.
Imported video is a Group[Video, Audio] that rendered as a Vello-baked
Vector layer, re-uploading the full frame to Vello's image atlas every
frame (~17ms/frame at 1080p, hitting playback and export alike). Extract
video frames out of the Group/clip scene recursion into
VideoRenderInstances so they composite via the GPU Video path; mixed
video+vector containers fall back to Vello (correct, unaccelerated).
Also route video through hardware sRGB decode: upload raw sRGB bytes to an
Rgba8UnormSrgb texture and blit with a non-unpremultiplying shader variant
(blit_straight), removing the per-frame per-pixel CPU sRGB->linear pass.
Add an F3 GPU-timestamp timer and a per-frame video texture cache.
Drops the live composite of a 1080p video from ~17ms to ~2-3ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Video was the only media type always kept external (VideoClip.file_path),
so a project with video wasn't self-contained. Now video packs into the
SQLite container under the same large-media policy as audio (pack < 2 GB
unless the user chose Reference), and both the frames and the embedded audio
track decode by streaming directly from the blob — no temp files.
- New crate ffmpeg-blob-io: an AVIOContext-over-Read+Seek shim (BlobInput)
that lets ffmpeg demux from an arbitrary byte source. Isolates all the
unsafe FFI + ffmpeg ABI coupling (version-pinned =8.0.0/=8.0.1). Manual
Drop teardown order; AVSEEK_SIZE restores the read position (FFmpeg assumes
a size query doesn't move it — required for MP4 moov-at-end).
- Schema/save/load: VideoClip.media_id; save_beam packs/references video as
MediaKind::Video (keyed by clip id); load resolves packed vs referenced and
reports missing sources. A packed clip points its linked video-audio pool
entry's media_id at the video row so the audio streams from the same blob.
- Frames: video.rs VideoSource{Path,Packed} threaded through new/seek/scan/
probe/thumbnails (a fresh BlobReader per open); editor builds the source
from current_file_path (now set before register_loaded_videos).
- Audio: VideoAudioReader::open_source via BlobInput; the disk_reader
StreamSource block on packed video-audio is removed; the engine's existing
factory activation routes it unchanged.
Tests: ffmpeg-blob-io AVIO unit tests (WAV via Cursor, seek, open/drop loop);
core packed_video_stream (blob->AVIO->Input) and beam_archive video round-trip;
daw-backend open_source test (compiles; links/runs only off-container).
Runtime-verified: a packed video plays frames + audio after the source file
is removed.
Address the code smells flagged in the .beam format spec:
- Write the project's actual sample rate on save instead of a hardcoded 48000
(add AudioProject::sample_rate()).
- Remove the vestigial RasterKeyframe.media_path field (it was only used by the
legacy ZIP loader, which now derives "media/raster/<id>.png" from the keyframe
id) and the dead buffer_path_at_time accessor. Backward-compatible: older files
carrying media_path deserialize fine (the field is ignored).
- Drop the unused SaveSettings fields auto_embed_threshold_bytes / force_embed_all
/ force_link_all; only large_media_mode was ever consulted. Un-prefix the now-used
`settings` parameter.
Update BEAM_FILE_FORMAT.md to match. The remaining notes (reserved MediaKind::Video,
exact-match version check) are design choices, left as-is.
The shipping product is the pure-Rust app in lightningbeam-ui/; the old
JavaScript UI in src/ is browser-only (window.__TAURI__ is shimmed by
src/tauri_polyfill.js), so the src-tauri Tauri backend is dead. It also no
longer compiled (mid-refactor: missing video_server module, handler list
referencing deleted commands), and CI only ever built lightningbeam-ui.
- Delete src-tauri/ entirely (commands, native renderer/render-window,
websocket frame streamers, Tauri config, generated schemas, icons).
- Relocate icon.icns into lightningbeam-ui/lightningbeam-editor/assets/icons/
(the only src-tauri asset still needed) and repoint the editor's window-icon
include_bytes! at assets/icons/256x256.png (was src-tauri/icons/icon.png).
- CI/packaging: drop the redundant icon-copy steps (PNGs are committed in the
editor assets) and point macOS bundling at the relocated icns.
- package.json: drop @tauri-apps/* deps and the tauri script.
- Docs/.gitignore: drop src-tauri references.
daw-backend/ (shared audio engine) and the top-level WASM lightningbeam-core/
are untouched. Editor builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve all compiler warnings across daw-backend, lightningbeam-core, and
lightningbeam-editor:
- Delete dead code: the superseded CPU raster tools in raster_tool.rs
(EffectBrush/Smudge/Gradient/Transform/Warp/Liquify/Selection — replaced by
the GPU path), plus orphaned helpers and never-read struct fields.
- Mechanical fixes: drop unused imports/variables/mut, underscore unused params,
`drop(&x)` -> `let _ = x`, deprecated egui::Rounding -> CornerRadius, snake_case
rename, elided-lifetime Cow<'_, [u8]>.
- Keep the WIP CSS theming system (theme.rs/theme_render.rs) under
#[allow(dead_code)] rather than deleting it.
Editor checks warning-free; 293 core tests pass.
`tween_after == Shape` was stored on keyframes but never read. Now the
render path morphs geometry across a shape-tween span:
- VectorGraph::interpolated(other, t): same-topology lerp of vertex
positions, edge curves, stroke widths and stroke/fill colours. Returns
None when topology differs (counts, deleted flags, edge endpoints, fill
boundaries), so the caller holds the source keyframe.
- VectorLayer::tweened_graph_at(time): returns an owned morphed graph for
a shape-tween span whose two keyframes share topology, else borrows the
held keyframe. Editing still uses graph_at_time (the held keyframe).
- Renderer (Vello + CPU paths) renders via tweened_graph_at.
- SetTweenAction + wired the previously-stubbed "Add Shape Tween" menu.
The typical workflow — keyframe, duplicate it (same topology), move
vertices, Add Shape Tween — now morphs between the two. Non-matching
topology falls back to a hold.
Collapse the two parallel selection systems into one. The RegionSelect
tool (rect + lasso) now cuts the geometry along the region outline and
selects the resulting sub-pieces into the standard `Selection` ID-sets,
exactly like every other tool. The vestigial floating `RegionSelection`
(drag never wired; commit/delete/copy were stubbed) and all its plumbing
are removed, so Group, Convert-to-Movie-Clip, Delete, and Properties all
operate uniformly from lasso, rect, marquee, and click selections.
Region cutting is reworked onto a robust planar arrangement:
- Replace fragile incremental "split a fill by one cut edge" logic with
planar face re-tracing (`retrace_fills_after_cut` + `trace_faces`),
which correctly handles arbitrary holed/concave fills.
- `extract_subgraph` no longer frees vertices still referenced by kept
boundary edges (fixed Group leaving freed-but-referenced vertices that
a later alloc reused and corrupted).
- `split_fill_by_*` direction fix (was producing disconnected boundaries
rendered as stray diagonals).
- `fill_interior_point` (area-centroid + inward-step fallback) for
reliable inside/outside classification of non-convex pieces.
- Coincident-edge dedupe + degenerate-fill removal (edge-adjacent shapes
no longer make zero-area sliver fills).
- Dangling-edge pruning, near-coincident endpoint welding, induced-
subgraph expansion, and tracking of `split_edge` sub-edges, so
self-intersecting freehand lassos cut correctly.
Region-select capture is available behind LIGHTNINGBEAM_DUMP_REGION=1 for
turning a misbehaving cut into a deterministic test. Extensive regression
tests added in vector_graph/tests/region_cut_select.rs.
Both actions were DCEL-stubs (no-ops). They now extract the selected geometry from a
vector layer's active keyframe into a new VectorClip (group vs movie clip) and place a
ClipInstance in its place (identity transform → renders where the geometry was), which
the existing transform-animation system can motion-tween.
- Shared `clip_from_geometry::extract_geometry_to_clip` (+ undo) does the work; the
actions are thin wrappers. Undo snapshots the source graph + removes the clip/instance.
- `extract_subgraph` now DERIVES shared-fill boundary edges internally (an inside edge
still used by a non-extracted fill must be duplicated, not moved) and unions them with
the caller's `explicit_boundary` — so a plain geometry selection needs no boundary
analysis (the selection already includes fill boundary edges via `select_fill`). The
region-select caller keeps passing its lasso boundary.
- Handlers: Group / Convert to Movie Clip on a geometry selection now build these actions
from `selected_fills`/`selected_edges`.
Next: shape tweens (same-topology lerp).
The export pump built a fresh `vello::Renderer` (full wgpu pipeline init, ~tens of ms)
AND an empty `ImageCache` on every egui repaint — i.e. per output frame. That was the
dominant per-frame cost (and the code comment already flagged it). Now the renderer +
image cache are built once on the first export pump, reused for every frame, and
dropped when the export finishes.
Also fixes a latent bug: the throwaway export ImageCache had no container path, so with
Phase 4 Tier 1 (lazy image bytes) images stored only in the container wouldn't render
in exports. The persistent cache now gets the container path.
Audit found the original "per-frame seek" theory was wrong — export decodes the source
sequentially; readback is already async/triple-buffered. Remaining wins: cache swscale
contexts (rebuilt per frame), GPU YUV conversion, decouple from the repaint cadence.
Project load no longer eager-reads all image bytes — `ImageAsset.data` stays empty
and the renderer's ImageCache pages compressed bytes from the `.beam` on a decode
miss (read_packed_media_readonly by asset id), decoding into the byte-bounded Tier-2
cache. Result: instant load, and compressed bytes don't accumulate on the heap.
- ImageCache: `container_path` + `resolve_bytes` (asset.data if resident — fresh
import or old base64 project — else page from the container); decoders take `&[u8]`
and use the decoded dimensions.
- Container path threaded App.current_file_path → SharedPaneState → VelloRenderContext,
set on the cache each prepare.
- load_beam_sqlite drops the 3.5b eager read.
(Refinement: a persistent read connection instead of open-per-miss.)
Replaces the DCEL "not yet supported" stubs so importing/dropping an image actually
puts it in the vector scene.
- AddShapeAction gains an `image_fill` + `AddShapeAction::image_rect(...)` constructor:
a borderless rectangle (invisible edges) whose enclosed region is paint-bucketed and
tagged with an image asset id. The renderer already prioritises `image_fill`.
- Direct import (auto_place_asset): an imported image is placed centered on the canvas
at native size on a vector layer.
- Drag from the asset library onto the stage: image-filled rect at the drop point
(centered), native size, using the asset's dimensions.
Next: SetImageFillAction + an Info-Panel image-fill picker for existing shapes; then
3.5b container persistence.
The standalone egui window didn't fit the UI. Replaced it with a collapsible
"Onion Skin" section at the bottom of the Info Panel (Enabled checkbox + frames
before/after + opacity), available regardless of selection. SharedPaneState carries
a mutable `onion_skin` ref for the controls (distinct from the gated `onion` copy
used by rendering).
A small 'Onion Skin' egui window (shown while onion skinning is enabled) with
DragValues for frames_before/after (0..=5) and an opacity slider, rendered next to
the F3 debug overlay. Lets the user tune the configurable settings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Skyler Lehmkuhl