Adds a cycle (loop) region: a range on the timeline ruler that the
transport wraps at during playback. This is the substrate for
GarageBand-style multi-take cycle recording (phases 2 and 3).
The region is authored in BEATS so it stays put musically across tempo
changes. It lives on the Document (saved in the .beam, serde-defaulted
so old files load) and is edited through SetCycleRegionAction, so it is
undoable and marks the document modified like any other edit.
Backend:
- Engine gains loop_region/loop_enabled plus a wrap at the single
playhead-advance point in process(). The wrap is phase-preserving
(modulo, so an overshoot larger than the loop can't strand the
playhead outside the region) and gated on playhead >= 0 so a count-in
pre-roll never wraps.
- Sounding voices are deliberately NOT reset at the wrap the way
Command::Seek does, since that would chop sustain and reverb tails at
every pass.
- MidiRecordingState::wrap_at_cycle writes note-offs for held notes at
the region end and re-opens them at the region start, so a key held
across the boundary can't end up with a negative duration or hang.
- loop_bounds_frozen freezes the region's sample bounds for the duration
of an *audio* recording. Phrased positively on audio so future
multi-track recording inherits it: MIDI is beat-segmented and
tempo-invariant, but audio is segmented geometrically and we have no
time-stretch, so cross-tempo audio takes wouldn't be compable anyway.
- Command::Play jumps to loop_start when starting from outside the
region; starting inside it plays from where you are.
Editor:
- Cycle lane along the bottom of the ruler (bottom, so it doesn't cover
the bar numbers), painted inside render_ruler under the ticks. Only
exists while looping is armed; with cycle off the ruler is entirely
the playhead scrubber, as before.
- Drag to create/move/resize with a three-zone hit test, previewed
locally and committed as ONE action on release. Driven off raw pointer
state rather than an egui Response: the lane sits inside the timeline's
content response, and a second widget on the same pixels just contests
hover every frame.
- snap_to_grid/quantize_grid_size take a min_grid_px "visual coarseness"
parameter instead of a hardcoded constant, with two named profiles:
SNAP_PX_FINE for the playhead and clip edges, SNAP_PX_CYCLE (coarser)
for the cycle region, so loops land on bars rather than odd
subdivisions.
- Cycle toggle button using the Lucide repeat glyph.
Cargo.lock picks up the 1.0.9-alpha version bump it missed.
The MIDI-clip-end-grows-too-fast bug was a units confusion: AudioClip.duration
is documented "seconds" but MIDI clips store their length in BEATS (they share
the AudioClip struct with sampled clips). The timeline's effective_clip_duration
read it as raw seconds, so at 120 BPM a MIDI clip rendered ~2x too long. The old
backend-snapshot display had hidden this by forcing timeline_duration in beats.
Root fix — make the domain explicit and unforgeable:
- `duration` is now a private field. Reading it in the wrong unit is impossible
because access goes through typed accessors: AudioClip::content_duration() ->
ClipDuration (a Seconds|Beats enum tagged by clip_type) and
set_content_duration(). serde still serializes the private field, so the .beam
format is unchanged (bare number).
- ClipDuration::to_seconds(tempo_map) for display/sizing; ::native() for code
that already works in the clip's native domain (trim math shares it).
- get_clip_duration + the timeline-endpoint calc go through to_seconds, so MIDI
is converted correctly; effective_clip_duration delegates to get_clip_duration.
- Recording mirrors (audio/MIDI progress + finalize) write via set_content_duration
(debug-asserts the value's domain matches the clip type).
Every former raw read/write of the field (core actions, timeline, piano roll,
infopanel, asset library, recording handlers) now goes through the accessors.
Whole workspace compiles; 299 core tests pass.
The timeline was the only place that reconstructed audio-layer clip instances
from a backend snapshot (build_audio_clip_cache) instead of reading the
document like every other layer type. But the document is the actual source of
truth: clip actions mutate *Layer::clip_instances, it's what persists to .beam,
and it drives the backend on load. The snapshot is downstream of it.
That dual source caused the second-recording preview bug: a recording clip
lives only in the document (its backend clip has a temporary pool index of 0,
so the snapshot can't represent it). layer_clips() only fell back to the doc
when the snapshot cache was *empty*, so the first recording previewed but the
second — once the layer already had a finalized clip — was dropped and showed
as zero-length until finalized.
Fix: make layer_clips() read <layer>.clip_instances for audio too, and delete
build_audio_clip_cache + the audio_cache plumbing threaded through ~7 methods
and ~25 call sites. Recording clips grow via the existing RecordingProgress /
MidiRecordingProgress mirror to the doc; finalized clips come from the actions;
both states now live in one place.
Root fix this exposed: get_clip_duration() wrapped every audio clip's duration
as Seconds, but MIDI clips store their duration in BEATS (they share AudioClip
with sampled clips). The snapshot had masked this by always forcing a MIDI
clip's timeline_duration; reading from the doc requires the value to be right,
so get_clip_duration now converts a MIDI clip's beats duration to seconds.
Net -120 lines. Whole workspace compiles; 299 core tests pass. The piano roll
still uses the snapshot for its own MIDI editor (separate pane, out of scope).
Push Beats/Seconds through the remaining controller methods that took a bare
f64 and let the audio thread wrap it in a newtype, so the caller's domain is
now compiler-checked (the seam that hid the recording bug):
- seek -> Seconds
- set_trim_start/set_trim_end -> Seconds / Option<Seconds> (metatrack, always seconds)
- add_midi_note, add_loaded_midi_clip, update_midi_clip_notes -> Beats
- add_automation_point, remove_automation_point, automation_add_keyframe,
automation_remove_keyframe -> Beats
Command enums stay raw f64 transport; only the public signatures + call sites
change. No behavior change — every caller already passed the right domain, this
just makes it enforced.
Two deliberate exceptions, documented in place:
- trim_clip stays f64: the TrimClip handler interprets it as Seconds for a
sampled-audio clip but Beats for a MIDI clip, so no single newtype fits;
callers pass the clip's own trim value, which matches its content domain.
- The piano-roll MIDI note model stays f64 internally (a beats-only subsystem
with no seconds anywhere); it's converted to Beats at the update_midi_clip_notes
boundary in UpdateMidiNotesAction, same as trim_start f64 -> Seconds at add_audio_clip.
The reported bug (a second recording lands early and overlaps the first)
survived the timeline type refactor: start_recording/create_midi_clip/
start_midi_recording took f64 and wrapped Beats(x) internally, so the type
boundary stopped at the method and the timeline handed them *shared.playback_time
(seconds). At 120 BPM a 5s playhead (=10 beats) was recorded at beat 5 = 2.5s.
Type all three backend methods to take Beats so the caller must convert; the
timeline now converts the seconds playhead once (start_beats) and passes it to
every recording command and the placeholder clip. TUI debug caller updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Thread Beats/Seconds through the editor now that lightningbeam-core is typed.
The timeline UI is seconds-domain (pixels_per_second, viewport_start_time,
playback_time, drag_offset), while ClipInstance.timeline_start/duration/
loop_before are beats — the compiler flagged every place the two were mixed.
Fixes the reported bugs and the whole class behind them:
- Recording placement (audio + MIDI + webcam): the new clip's timeline_start
was written from playback_time (seconds) into a beats field, so a second
recording landed at the wrong time. Now converted via the tempo map.
- Drag/move: introduce snapped_move_offset -> Beats (snap in the seconds/pixel
domain, express the anchor's movement in beats) and moved_start(); the group
clamp, live preview, and commit all use one uniform beats offset instead of
adding a seconds delta to a beats position.
- Trim/loop drag preview + commit: overlap limits come from the timeline
(beats) but trims are content seconds — converted at the boundary.
- Drop (asset drag, stage + timeline), paste, duplicate, split-at-playhead:
all convert the seconds drop/playhead position to beats before placing.
- Stage playback gates + clip-local time remap: compared seconds playback_time
against beats timeline_start; now both in seconds.
- Piano roll / infopanel / effect export: clip_dur is Seconds; effect and
waveform data converted at use.
Core API refinement (motivated by the above): find_max_trim_extend_left/right
now return Seconds (the content-seconds gap) instead of raw Beats, since every
trim caller wants seconds; loop-extend callers convert to beats. Added a
beats_to_x() timeline helper.
Whole workspace compiles; 299 core tests pass.
Fixes from a review of the mobile-UI branch.
- Guard the inspector size clamps so min<=max (f32::clamp panics otherwise) and set a
window min_inner_size; prevents a crash / degenerate layout at small sizes. Same
small-size guards for the piano-roll landscape toggle bar and the intent grid.
- Landscape 2->1 edge reveal: snap to fullscreen at frac>=0.5 (the preview's phase
boundary) instead of COLLAPSE_HI, so releasing mid-drag no longer jumps backward.
Both edges.
- Gate the double-tap-drag marquee to mobile so it can't hijack a desktop Brush/Draw
drag.
- Timeline mobile long-press: request a repaint while counting down so it fires even
when the finger is held perfectly still.
- Landscape folded top bar: lay the filename+search+overflow cluster out within a
reserved center span (gutters for the pane's own label/buttons) and elide the
filename, so it can't overlap or overflow the header.
- Node editor connect/disconnect: assert + document the top-level-graph assumption.
- Palette search field focuses only when unfocused (no per-frame focus stomp); reset
the inspector tap-anchor on orientation change; remove stale main.rs.backup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hook up menu-bar actions that lacked a mobile affordance, and implement z-order.
- Stage/object long-press menu (stage.rs): add Paste (on clip, geometry, and
empty stage) and Send to Back / Bring to Front on the selection.
- Timeline long-press menu (timeline.rs): the timeline had no mobile context
menu — add one via manual long-press detection. Clip actions (Split,
Duplicate, Cut, Copy, Paste, Delete) on a clip; animation actions (New/Blank
keyframe, Add keyframe at playhead, Duplicate keyframe, Delete frame, Add
motion/shape tween) on an empty lane. Gate the desktop right-click menu to
!is_mobile so mobile shows only the long-press menu.
- Implement Send to Back / Bring to Front (were // TODO no-ops): new undoable
ReorderClipInstancesAction in lightningbeam-core reorders the selected
instances within their layer's clip_instances Vec (stacking order; last =
on top; geometry stays flattened underneath). Wired via handle_menu_action.
The mobile Piano Roll becomes a unified, keyboard-primary instrument surface
(wireframe Plate 04/08): a playable keyboard as the base, revealing a
Synthesia-style falling-notes roll above it when the pane is tall enough.
- Shared keyboard geometry (panes/keyboard_layout.rs): width-driven, pan-aware
pitch->x, so the roll's note columns stay aligned with the Virtual Piano keys.
Shared keyboard_octave + keyboard_pan_x in SharedPaneState.
- Piano Roll vertical mode (is_mobile): notes as columns falling toward an amber
now-line by the keys, tempo-map converted (beats->seconds) so onsets cross the
line exactly when they sound. Vertical drag scrubs the timeline; horizontal
drag smoothly scrolls the keys (snaps to nearest key on release). Long-press
creates a note (drag to size) or resizes an existing one; pan suppresses it.
The keyboard is embedded (reuses Virtual Piano render + MIDI); the standalone
VirtualPiano stack slot is removed. show_roll is driven by the snapped pane
size-class so the keyboard<->roll reveal lands on a stack snap point.
- Virtual Piano: renders via the shared layout on mobile, colors playback_notes
like pressed keys, gates note-on/glissando to presses that start on the keys
(never gating release), and hides QWERTY hints on mobile.
- Transport formats by document.timeline_mode (Measures->bar.beat.tick,
Frames->MM:SS:FF, Seconds->MM:SS.mmm) — per project type, like desktop.
- In-pane instrument header (name + Presets + REC). Recording is driven from the
app each frame (not the Timeline pane's render) so REC works regardless of
visible panes, and stopping playback stops recording.
- Compose/Record intent opens Timeline + instrument pane; mobile central panel is
full-bleed (no inner margin) so panes sit flush.
- phone-ui-sketches.html: inst-bar moved to the top of the music surface.
Gestures (Stage + Timeline):
- Pinch-zoom via zoom_delta() (touch pinch + Ctrl+wheel), unified with the raw-wheel
path so Ctrl+wheel zooms exactly once; plain wheel zoom + trackpad pan preserved.
- Double-tap empty → zoom-to-fit (Stage: artboard via zoom_to_fit; Timeline: new
fit_to_project over the full duration).
- Double-tap-drag on empty → transient marquee regardless of the active tool.
Long-press context menu: a shared SharedPaneState.mobile_context_menu that panes
populate with (label, MenuAction) items on secondary_clicked(); the mobile shell
renders one persistent popup (styled like the timeline menu) and dispatches via
pending_menu_actions. Stage offers Cut/Copy/Duplicate/Delete for clip instances and
Cut/Copy/Convert-to-movie-clip/Delete for geometry.
Fixes surfaced along the way:
- Geometry delete now frees selected fills (free_fill) and GCs isolated vertices
(new VectorGraph::gc_isolated_vertices) — no more orphaned fills / phantom snap
points; applies to the Delete key too.
- clipboard_delete_selection clears focus so deleting dismisses the mobile inspector.
- Mobile inspector: appears on pointer release (not press) so drags aren't
interrupted; reflows only when the tapped point is actually behind the sheet; taps
outside the sheet dismiss it; rounded-corner border no longer detaches.
- Add `is_mobile` to SharedPaneState (set in build_frame). On mobile the timeline
layer-header column collapses to a minimal type-colored swatch (1:2 w:h, ~30px),
brighter/outlined when active/selected; tapping it selects the layer (handled by
the existing header-column input path, which now uses the narrower width).
Desktop headers unchanged.
- Inspector reflow: the sheet overlaps the stack by default, but when the selected
pane would be covered by the sheet, the stack reflows (renders above the sheet)
so it stays visible. Restore on dismiss is automatic since reflow only changes
the render rect; manual divider moves while the sheet is up persist. Adds
stack::pane_bottom_in and inspector::target_slot.
Introduce editable text layers: a resizable text box with editable text,
font size, color, font family, and alignment.
Core:
- New TextLayer/TextContent (text_layer.rs), wired into AnyLayer/LayerType
and all the exhaustive match sites; structured so content can be keyframed
later via content_at().
- fonts.rs: thread-local parley FontContext with three bundled fonts
(Liberation Sans/Serif/Mono, SIL OFL), system-font enumeration consolidated
to base families, document-embedded fonts, glyph/caret/selection geometry,
and a background preloader for the picker fonts.
- Rendering via parley layout + Scene::draw_glyphs (renderer.rs); text
composites through the vector path.
- Actions: CreateTextClipAction (vector-layer branch, undoable),
SetTextContentAction, ResizeTextBoxAction.
- .beam font embedding: MediaKind::Font rows (content-hash dedupe) written on
save and registered on load, with bundled-default fallback.
- VectorClip content bounds include text boxes so text-only clips are
selectable/draggable.
Editor:
- Text tool: click empty/raster/video to create a top-level text layer, or a
vector layer to create+enter a clip containing the text; click an existing
box to edit it.
- Hybrid in-place editing: a hidden egui TextEdit drives input/IME/caret while
the text and caret/selection render in Vello; empty just-created layers are
removed on commit.
- Selection outline + 8 resize handles (re-wrap text) with hover cursors;
factored the corner/edge resize-cursor mapping shared with the Transform tool.
- Info panel: edit text, size, color, alignment, box size, and a font-family
picker that previews each entry in its own font (fonts preloaded in the
background to avoid hitches).
Deps: add parley (git, pinned to match vello's peniko); bundle Liberation
fonts under lightningbeam-core/assets/fonts. Gitignore the local
.cargo/config.toml used to select a machine's ffmpeg.
A) Imported video had a different aspect ratio depending on how it was placed:
direct import used a uniform scale + center (preserve aspect), while the
asset-library timeline drag used an independent scale_x/scale_y (stretch to fill).
Add Transform::fit_centered (uniform scale, centered, aspect-preserving) and route
both paths through it, so a clip looks identical however it's added.
B) Exported video was stretched when the export resolution's aspect differed from
the document's (base_transform was always scale_non_uniform). Add
ExportFitMode {Stretch, Letterbox (default), Crop} on VideoExportSettings + a "Fit"
dropdown in advanced export settings, and a shared export_base_transform() helper:
Letterbox = uniform fit centered (black bars), Crop = uniform fill centered (trim),
Stretch = the old distort-to-fill. Threaded through the software, HDR, and
zero-copy export render paths (image export is doc-sized → identity).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve all compiler warnings across daw-backend, lightningbeam-core, and
lightningbeam-editor:
- Delete dead code: the superseded CPU raster tools in raster_tool.rs
(EffectBrush/Smudge/Gradient/Transform/Warp/Liquify/Selection — replaced by
the GPU path), plus orphaned helpers and never-read struct fields.
- Mechanical fixes: drop unused imports/variables/mut, underscore unused params,
`drop(&x)` -> `let _ = x`, deprecated egui::Rounding -> CornerRadius, snake_case
rename, elided-lifetime Cow<'_, [u8]>.
- Keep the WIP CSS theming system (theme.rs/theme_render.rs) under
#[allow(dead_code)] rather than deleting it.
Editor checks warning-free; 293 core tests pass.
- Pointing-hand cursor when hovering a clickable keyframe diamond.
- Prefetch (Phase 3e, playback only): each update during playback, page in the next
few upcoming keyframes (PREFETCH_AHEAD=4) per raster layer that aren't resident, via
the existing async worker. Their full pixels land before the playhead reaches them,
so playback shows full frames instead of the low-res proxy on every frame (the
proxy→full pop was the "flicker"). Reactive faults still cover scrubbing.
render_layers now records each drawn keyframe diamond's screen rect + exact time in
`keyframe_diamond_hits`; handle_input hit-tests a click against them and sets the
playhead (and seeks the audio controller) to the keyframe's exact time. Uses the
previous frame's rects — diamonds don't move between frames, so the click lands
right — which sidesteps the input-before-render ordering and the drag/scroll Y math.
Works for both raster and vector keyframes.
Make raster layers behave like vector on the timeline.
- Timeline: draw a diamond per `RasterKeyframe` (mirrors the vector keyframe block).
- New Keyframe (K / menu): on a raster layer, insert a BLANK cel at the playhead via
a new undoable `AddRasterKeyframeAction` (+ `RasterLayer::insert_blank_keyframe_at`
/ `remove_keyframe`). Vector path unchanged.
- Stop lazy creation: paint tools now edit the ACTIVE keyframe (at-or-before the
playhead) instead of creating one. The brush captures the active keyframe's exact
time; `RasterStroke`/`RasterFillAction` resolve via `keyframe_at_mut` (error if
none); the tool-site `ensure_keyframe_at` blocks (brush/fill/bucket/wand/quick-
select/floating-lift) are removed — each read already bails when no keyframe exists.
New layers still seed a keyframe at the playhead, so there's normally one to paint
into; painting before the first keyframe is now a no-op (as intended).
Next: onion skinning.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Thumbnail rendering fixes:
- Strip now tiles from each clip's true (unclamped) origin and draws only the
tiles intersecting the visible rect, so it scrolls correctly and shows the
right frames when a clip is scrolled partly off the left. Both render sites
(collapsed group + expanded track) share one draw_video_thumbnail_strip helper.
- On-clip strip no longer freezes on the first thumbnail: get_thumbnail_at now
returns the actual thumbnail timestamp and the GPU texture cache keys on it, so
tiles refresh as closer thumbnails finish generating.
- Hover preview derives content time from the clip's true origin too (matches the
strip when scrolled off-screen).
- insert_thumbnail keeps the cache sorted + deduped (fixes a latent unsorted
binary_search bug, and makes concurrent restore + resume race-safe).
Thumbnail persistence (mirrors waveform persistence):
- MediaKind::Thumbnail rows, keyed by thumbnail_media_id(clip_id) (clip id XOR a
sentinel). Each clip's thumbnails PNG-encoded into one opaque LBTN blob (editor
owns the format), snapshotted cheaply (Arc clones) and encoded off the UI thread.
- Save writes the packs (kept in place on re-save); load reads them into
LoadedProject.thumbnail_blobs; the editor decodes + inserts them on a background
thread, so reload shows thumbnails instantly with no re-decode (even if the
source video file is missing).
- Partial sets are persisted with a complete flag and RESUMED on load:
generate_keyframe_thumbnails takes a should_skip predicate so a save made
mid-generation continues from where it left off instead of redoing the work.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migrate the .beam container to SQLite and stream media from it instead of
decoding whole files into RAM on import/load.
Container & large files:
- SQLite .beam container (beam_archive) with in-place transactional saves and an
incremental BlobReader; supports both packed (chunked blobs) and referenced
(external path) media, with a user preference + first-import prompt for files
over the large-media threshold.
Audio streaming:
- Stream packed compressed audio on load via an inversion-of-control blob factory
(AudioBlobSourceFactory): daw-backend defines the trait, core implements it
over BlobReader, so the audio engine stays container-agnostic.
- Bulk-activate disk streaming for all loaded clips after SetProject.
- Sample-accurate compressed seek (SeekMode::Accurate; Coarse mislands on VBR).
Video:
- Video frames decoded/streamed on demand; thumbnails generated asynchronously
on a dedicated decoder so import/load never blocks the UI.
- The video's audio track is streamed on demand via an ffmpeg VideoAudioReader
as a separate editable AudioClip (no /tmp WAV extraction).
Waveform overview:
- Streaming min/max LOD pyramid (waveform_pyramid), bounded memory, configurable
floor B; serialized into the container and restored on load (or generated in
the background from the packed blob when absent), so no re-decode on reload.
- GPU min/max upload path; integer-LOD textureLoad fixes zoom-dependent wobble.
Skyler Lehmkuhl