Three bugs in a row came from the same root: a time value crossing an API
boundary as a bare f64, with the caller and the callee disagreeing about
whether it meant seconds or beats. Recording landed at the wrong time,
MIDI clips grew too fast, and a 1-second split played back as half a
second. Each was "obviously" one domain at the call site and read as the
other on the far side. This makes the mismatch a compile error.
Backend API — every time-carrying f64 is gone:
- Commands: Seek/SetOffset/SetTrimStart/SetTrimEnd -> Seconds; MoveClip/
ExtendClip/CreateMidiClip/AddMidiNote/AddLoadedMidiClip/
UpdateMidiClipNotes/AddMidiClipSync and all four automation commands ->
Beats; TrimClip -> TrimRange.
- Events/queries: PlaybackPosition, WaveformChunksReady's time range,
AudioFileReady::duration, PoolFileInfo, get_playhead_seconds -> Seconds.
- Serialized: MidiClipData::duration and AutomationKeyframeData::time ->
Beats. Both newtypes are #[serde(transparent)], so the .beam on-disk
format is unchanged.
- Several controller methods ALREADY took Beats and unwrapped it to shove
into the command — the newtype was being discarded at the very boundary
it existed to protect.
TrimRange, for the domain-polymorphic case: a clip's content time is
SECONDS for sampled audio but BEATS for MIDI, so a single newtype can't
express it (there was even a comment in engine.rs saying so, and that
rationalization is what let the bug through). A domain-tagged enum can.
The engine rejects a range whose domain doesn't match the track, and the
range is built from the clip (clip.trim_range()) so callers can't pick
the wrong variant.
ContentTime, for the trim fields: ClipInstance::trim_start/trim_end are
content times, and were the last untyped f64 — the actual root of the
split bug. ContentTime is deliberately a DEAD END: no .to_seconds(), no
.to_beats(), no arithmetic with Seconds or Beats. Content times combine
freely with each other (same clip, same domain — safe), so the ~100
passthrough sites cost nothing; the only exit is resolving against the
clip that knows the domain (AudioClip::resolve_content_time /
Document::resolve_content_time / ClipDuration::same_domain). Mixing
domains no longer compiles.
Two more live bugs the types surfaced:
- ClipInstance::effective_duration_beats took a SECONDS clip duration and
subtracted trim_start from it. For a TRIMMED MIDI clip that subtracted
a beats offset from a seconds duration, so the clip's timeline length
was wrong at any tempo but 60 BPM. Untrimmed clips happened to work,
which is why it hid. It now takes a ClipDuration and resolves in the
clip's own domain: beats content carries over directly (tempo-
invariant), wall-clock content converts at the clip's position.
Regression test asserts a clip trimmed to beats 2..6 is 4 beats long at
60/90/120 BPM.
- Trim validation clamped a content-domain trim against a wall-clock gap.
gap_to_content/content_to_secs now convert at the clip's position.
Also folds two more copies of the backend add-logic into
BackendContext::add_clip_instance (split and remove_clip_instances both
re-add clips), so the trim/duration conversions live in exactly one place
instead of four.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Recording into a cycle region now produces one take per pass. Pick a take
from a badge on the clip; split the clip and pick different takes on the
halves, and you've comped.
Data model (phase 2):
- AudioClipType::TakeFolder { takes, recorded_loop_beats } holds the take
list on the CLIP; ClipInstance::active_take holds the selection on the
INSTANCE. That split is what makes comping fall out of the existing
split action for free — split clones the instance, so the two halves
share one take list but choose independently. recorded_loop_beats lets
a future time-stretch/conform pass reconcile audio takes if the tempo
moves under them.
- AudioClip::resolve(active_take) -> ResolvedContent{Audio|Midi|Recording}
collapses a take folder to what an instance actually plays. A folder is
not a distinct *case* at call sites — it's an audio or MIDI clip whose
identity depends on which take is live — so every backend-sync site now
resolves through this instead of matching clip_type raw. Reverse lookups
go through owns_audio_pool_index/owns_midi_clip_id, since a folder owns
one pool file per take, not just the active one.
- BackendContext::add_clip_instance/remove_clip_instance: switching takes
is a remove + re-add (there's no in-place pool-swap command), and that's
the same work AddClipInstanceAction does. One implementation, on the
context that already owns the controller and both ID maps, so the
seconds-vs-beats conversions can't drift between copies.
Capture (phase 3):
- Takes are cut GEOMETRICALLY at stop, in exact loop-length multiples. The
playhead advances before the capture block in process(), so the wrap
instant isn't sample-exact against the buffer just captured — but the
geometry is. wrap_count only decides *whether* the recording is
multi-take, never where the cuts land.
- Partial passes are padded with silence: punch in mid-region and take 1
gets silence prepended back to the region start; stop mid-pass and the
last take gets silence appended. Every take is the same length, which is
the invariant comping depends on. A final take under 50ms of real audio
is dropped as a stop artifact (but a take that FILLED the region never
is, however short the region).
- MIDI merges, and it falls out for free: anchoring the recording at
loop_start rather than the punch-in point means the transport always
wraps back INTO the region, so every note's offset already lands inside
[0, loop_len) and passes overdub with no folding logic at all.
- The whole session commits as ONE undoable action via push_applied.
Fixes found on the way:
- Split was seconds/beats confused on MIDI. trim_start/trim_end are
domain-polymorphic exactly like AudioClip::duration was — SECONDS for
audio/video/vector, BEATS for MIDI — and split mapped the split point
into clip content in seconds unconditionally. Now it works in the clip's
own domain via Document::clip_trim_duration(). Regression test included.
- TrimClip took raw f64s whose meaning flipped by track type, and the
engine set an AUDIO clip's external_duration = Beats(end - start) where
those bounds were SECONDS — so a 1-second split played back as half a
second at 120 BPM. Replaced with a domain-tagged TrimRange, built from
the clip (clip.trim_range()) so the wrong unit isn't expressible, and
the span is now converted at the clip's position on the timeline.
- The live preview grew past the loop end while the playhead wrapped. It
now grows through the first pass then pins at the region length, and the
waveform inside restarts at the region start on each pass. The pass
offset is derived from the captured buffer, not the playhead — those
advance on different clocks, and differencing them made the waveform
jitter horizontally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The MIDI-clip-end-grows-too-fast bug was a units confusion: AudioClip.duration
is documented "seconds" but MIDI clips store their length in BEATS (they share
the AudioClip struct with sampled clips). The timeline's effective_clip_duration
read it as raw seconds, so at 120 BPM a MIDI clip rendered ~2x too long. The old
backend-snapshot display had hidden this by forcing timeline_duration in beats.
Root fix — make the domain explicit and unforgeable:
- `duration` is now a private field. Reading it in the wrong unit is impossible
because access goes through typed accessors: AudioClip::content_duration() ->
ClipDuration (a Seconds|Beats enum tagged by clip_type) and
set_content_duration(). serde still serializes the private field, so the .beam
format is unchanged (bare number).
- ClipDuration::to_seconds(tempo_map) for display/sizing; ::native() for code
that already works in the clip's native domain (trim math shares it).
- get_clip_duration + the timeline-endpoint calc go through to_seconds, so MIDI
is converted correctly; effective_clip_duration delegates to get_clip_duration.
- Recording mirrors (audio/MIDI progress + finalize) write via set_content_duration
(debug-asserts the value's domain matches the clip type).
Every former raw read/write of the field (core actions, timeline, piano roll,
infopanel, asset library, recording handlers) now goes through the accessors.
Whole workspace compiles; 299 core tests pass.
Type ClipInstance.timeline_start/timeline_duration/loop_before as Beats and
thread Beats/Seconds through core so the compiler catches the seconds-vs-beats
mismatches behind the audio-clip placement/drag/trim bugs.
Fixes latent mixups surfaced by the types:
- add/remove/split clip instance: the audio "effective duration" fallback was
the content-seconds span treated as beats (clips stopped early off 60 BPM);
now converted via the tempo map at the clip's start.
- trim validation: extend-left/right clamped a content-seconds delta against a
timeline-beats gap (and moved trim_start + timeline_start by the same raw
amount, assuming 1:1). Now the gap is converted to content seconds and the
timeline moves by the beats-equivalent.
- split content-split point mixed beats into a seconds trim value.
- set_keyframe: visibility-start fallback returned beats where seconds expected.
- hit_test: timeline_time (s) compared against beats without conversion.
Typed backend boundaries that were passing beats/seconds as bare f64:
add_audio_clip (start/dur Beats, offset Seconds), move_clip/extend_clip (Beats),
set_offset (Seconds). Command enum transport stays f64.
Serialization is unchanged (Beats/Seconds are #[serde(transparent)]).
lightningbeam-core builds; 299 core tests pass. The editor call sites
(recording/drop/paste/drag placement) are updated in stage 2.
Introduce editable text layers: a resizable text box with editable text,
font size, color, font family, and alignment.
Core:
- New TextLayer/TextContent (text_layer.rs), wired into AnyLayer/LayerType
and all the exhaustive match sites; structured so content can be keyframed
later via content_at().
- fonts.rs: thread-local parley FontContext with three bundled fonts
(Liberation Sans/Serif/Mono, SIL OFL), system-font enumeration consolidated
to base families, document-embedded fonts, glyph/caret/selection geometry,
and a background preloader for the picker fonts.
- Rendering via parley layout + Scene::draw_glyphs (renderer.rs); text
composites through the vector path.
- Actions: CreateTextClipAction (vector-layer branch, undoable),
SetTextContentAction, ResizeTextBoxAction.
- .beam font embedding: MediaKind::Font rows (content-hash dedupe) written on
save and registered on load, with bundled-default fallback.
- VectorClip content bounds include text boxes so text-only clips are
selectable/draggable.
Editor:
- Text tool: click empty/raster/video to create a top-level text layer, or a
vector layer to create+enter a clip containing the text; click an existing
box to edit it.
- Hybrid in-place editing: a hidden egui TextEdit drives input/IME/caret while
the text and caret/selection render in Vello; empty just-created layers are
removed on commit.
- Selection outline + 8 resize handles (re-wrap text) with hover cursors;
factored the corner/edge resize-cursor mapping shared with the Transform tool.
- Info panel: edit text, size, color, alignment, box size, and a font-family
picker that previews each entry in its own font (fonts preloaded in the
background to avoid hitches).
Deps: add parley (git, pinned to match vello's peniko); bundle Liberation
fonts under lightningbeam-core/assets/fonts. Gitignore the local
.cargo/config.toml used to select a machine's ffmpeg.
Video was the only media type always kept external (VideoClip.file_path),
so a project with video wasn't self-contained. Now video packs into the
SQLite container under the same large-media policy as audio (pack < 2 GB
unless the user chose Reference), and both the frames and the embedded audio
track decode by streaming directly from the blob — no temp files.
- New crate ffmpeg-blob-io: an AVIOContext-over-Read+Seek shim (BlobInput)
that lets ffmpeg demux from an arbitrary byte source. Isolates all the
unsafe FFI + ffmpeg ABI coupling (version-pinned =8.0.0/=8.0.1). Manual
Drop teardown order; AVSEEK_SIZE restores the read position (FFmpeg assumes
a size query doesn't move it — required for MP4 moov-at-end).
- Schema/save/load: VideoClip.media_id; save_beam packs/references video as
MediaKind::Video (keyed by clip id); load resolves packed vs referenced and
reports missing sources. A packed clip points its linked video-audio pool
entry's media_id at the video row so the audio streams from the same blob.
- Frames: video.rs VideoSource{Path,Packed} threaded through new/seek/scan/
probe/thumbnails (a fresh BlobReader per open); editor builds the source
from current_file_path (now set before register_loaded_videos).
- Audio: VideoAudioReader::open_source via BlobInput; the disk_reader
StreamSource block on packed video-audio is removed; the engine's existing
factory activation routes it unchanged.
Tests: ffmpeg-blob-io AVIO unit tests (WAV via Cursor, seek, open/drop loop);
core packed_video_stream (blob->AVIO->Input) and beam_archive video round-trip;
daw-backend open_source test (compiles; links/runs only off-container).
Runtime-verified: a packed video plays frames + audio after the source file
is removed.
Image asset bytes are now stored as MediaKind::ImageAsset rows in the SQLite
container (chunked, kept-in-place on re-save) instead of base64-embedded in the
project JSON — the pageable storage Phase 4 needs.
- ImageAsset.data is `#[serde(default, skip_serializing)]`: never written to JSON,
but still deserialized for old projects (base64) which then migrate to the
container on the next save.
- save_beam writes each asset's bytes (keyed by asset id; ext from the source path),
keeping an existing row when bytes aren't resident; live_media covers them so orphan
cleanup doesn't drop them.
- load_beam_sqlite eager-reads the bytes back into `data` (Phase 4 makes this lazy +
LRU). Old base64 projects keep their JSON-deserialized data (no container row).
The lib unit tests had gone stale (time values became newtypes) and no longer
compiled. Updated the test code to the current API and fixed the few real issues
the now-running tests surfaced.
Test-only:
- Wrap raw f64 time literals in Beats(...) where the API now takes Beats
(automation.rs); pass &TempoMap / Beats where signatures changed (clip.rs,
effect_layer.rs).
- shape.rs: assert the documented no-fill default (fill_color None) instead of Some.
- add_clip_instance / trim_clip_instances tests: register a vector clip with the
test's clip_id so the action's get_clip_duration lookup succeeds.
Production fix (delete_folder.rs):
- DeleteFolderAction(MoveToParent) reparented child subfolders to the deleted
folder's parent but never restored them on undo, orphaning them. Track the moved
subfolder ids and restore their parent on rollback.
Result: daw-backend lib 17 passed; lightningbeam-core lib 264 passed.
Skyler Lehmkuhl