Commit Graph

21 Commits

Author SHA1 Message Date
Skyler Lehmkuhl b301c63387 Address code-review findings across export, decode, and data model
Export correctness:
- Honor the user's color-range (Limited/Full) on the software encode path:
  thread full_range through gpu_yuv (new shader range uniform), the CPU
  swscale fallback (sws_setColorspaceDetails → BT.709 + range, fixing the
  BT.601 hue/level shift on odd-width exports), and the encoder color tags.
- Reject HDR + WebM up front with a clear message and log the forced HEVC
  override instead of producing an unplayable file.
- Delete dead render_frame_to_rgba_hdr (hardcoded Stretch; live HDR path
  already honors the fit mode).

Decode/playback (video.rs):
- Drain the decoder at EOF (send_eof + flush) so the final B-frame-delayed
  frames render instead of erroring; per-frame logic extracted to a helper.
- Missing-PTS frames continue monotonically rather than snapping to ts=0.
- Force exact thumbnail width so sub-128px sources aren't shown stretched.

Resource leaks (gpu-video-encoder):
- dmabuf import_raw: RAII guard frees the duped fd + partial VkImages/memory
  on every error path.
- vaapi alloc: free device/frames-ctx/AVFrames on the unexpected-DRM path.

Data model / robustness:
- collapse_boundary_spikes requires a full curve reversal (all control
  points) so it no longer deletes a real lens/sliver and drops the fill.
- Export audio spin-wait ignores a stale `finished` flag when a forward
  seek is pending (was rendering silence over real audio).
- RasterDiff apply_before/after take current dims and skip on a post-resize
  mismatch.
- beam_archive read_media_full caps the preallocation from untrusted total_len.

UI/visual:
- SVG export skips hidden layers/empty groups; import folds fill-opacity into
  gradients and surfaces failures as a notification.
- Active raster-layer border uses playback_time + overlay_transform.
- gpu_brush remove_layer_texture also evicts the stale low-res proxy.
- ensure_raster_resident_for_undo registers faulted frames in the LRU so
  resident RAM stays bounded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 17:47:32 -04:00
Skyler Lehmkuhl 70ac0cde00 Fix audio export hang when a video's audio is shorter than the video
Offline export blocks each streaming source until decoded frames are
available. For a video whose audio track ends before the video (or a
container like WebM/Opus with no exact stream duration), the tail chunks
wait for frames that never arrive — the 10s safety valve fires per chunk,
so the export appears to hang indefinitely.

Add a `finished` flag to ReadAheadBuffer, set when the disk reader hits
EOF (cleared on seek). The export-mode wait now breaks immediately once
the source is finished and the requested frames aren't present, rendering
silence for the missing tail instead of timing out chunk-by-chunk. Only
the export path reads the flag, so playback is unaffected.
2026-06-23 19:06:21 -04:00
Skyler Lehmkuhl ce151ffd61 Pack video into .beam and stream frames + audio from the blob
Video was the only media type always kept external (VideoClip.file_path),
so a project with video wasn't self-contained. Now video packs into the
SQLite container under the same large-media policy as audio (pack < 2 GB
unless the user chose Reference), and both the frames and the embedded audio
track decode by streaming directly from the blob — no temp files.

- New crate ffmpeg-blob-io: an AVIOContext-over-Read+Seek shim (BlobInput)
  that lets ffmpeg demux from an arbitrary byte source. Isolates all the
  unsafe FFI + ffmpeg ABI coupling (version-pinned =8.0.0/=8.0.1). Manual
  Drop teardown order; AVSEEK_SIZE restores the read position (FFmpeg assumes
  a size query doesn't move it — required for MP4 moov-at-end).
- Schema/save/load: VideoClip.media_id; save_beam packs/references video as
  MediaKind::Video (keyed by clip id); load resolves packed vs referenced and
  reports missing sources. A packed clip points its linked video-audio pool
  entry's media_id at the video row so the audio streams from the same blob.
- Frames: video.rs VideoSource{Path,Packed} threaded through new/seek/scan/
  probe/thumbnails (a fresh BlobReader per open); editor builds the source
  from current_file_path (now set before register_loaded_videos).
- Audio: VideoAudioReader::open_source via BlobInput; the disk_reader
  StreamSource block on packed video-audio is removed; the engine's existing
  factory activation routes it unchanged.

Tests: ffmpeg-blob-io AVIO unit tests (WAV via Cursor, seek, open/drop loop);
core packed_video_stream (blob->AVIO->Input) and beam_archive video round-trip;
daw-backend open_source test (compiles; links/runs only off-container).
Runtime-verified: a packed video plays frames + audio after the source file
is removed.
2026-06-22 09:17:10 -04:00
Skyler Lehmkuhl d001ca1083 Fix save crash on zero/sparse-audio projects (phantom pool placeholder)
`AudioPool::load_from_serialized` sizes the slot Vec by pool_index and fills gaps
with empty `AudioFile::new(PathBuf::new(), …)` placeholders. Two bugs let a
placeholder reach the next save and abort it with "Is a directory":

- Off-by-one: `entries.max().unwrap_or(0) + 1` made an *empty* pool length 1, so a
  project with no audio still got one placeholder. Size by `max(pool_index + 1)`
  → empty entries yield length 0.
- `serialize()` emitted placeholder slots: an empty path round-trips to
  `relative_path = Some("")`, which `save_beam` resolves to the project directory
  (`join("")`) and tries to read as media. Skip empty-path / no-packed-media slots.
- Defense in `save_beam`: gate referenced-media packing on `full.is_file()` (not
  `exists()`), so any blank/dir path falls through to embedded data instead of
  reading a directory.

Pre-existing; surfaced by a save → reload → save cycle on a raster-only project.
2026-06-20 18:46:19 -04:00
Skyler Lehmkuhl 4ad95e6755 Downmix surround to stereo + reload video audio via FFmpeg
Surround → stereo downmix:
- render_from_file folds multichannel sources (5.1/7.1/…) down to stereo with
  proper coefficients (full level for the matching front channel, 1/√2 for centre
  + each surround, LFE dropped), normalized per row to avoid clipping (matching
  ffmpeg's default). Applied uniformly to both the direct-copy and sinc-resample
  paths and to every storage type (PCM, compressed, video audio), only when
  dst==2 && src>2; unknown layouts fall back to front L/R. Previously it just took
  FL/FR, dropping centre dialog + surrounds.

Proper video-audio reload:
- A video's audio track is now stored as a path reference to the video (never
  packed/embedded as audio media) and re-probed via FFmpeg on load into a
  streaming VideoAudio entry, so multichannel audio survives reload (the old
  Symphonia reconstitution collapsed it, breaking the downmix). Driven by a new
  AudioPoolEntry.is_video_audio flag across serialize / save_beam / load. Also
  removes the decode-whole-video-to-RAM + temp-file path on load.

Fix video scaling:
- Any video with dimensions larger than the stage was being scaled down into the corner incorrectly; we now bake the frame-clip scale into the instance transform.
2026-06-17 18:30:42 -04:00
Skyler Lehmkuhl 3d7cff9ad0 Stream audio & video from .beam container; waveform LOD pyramid + persistence
Migrate the .beam container to SQLite and stream media from it instead of
decoding whole files into RAM on import/load.

Container & large files:
- SQLite .beam container (beam_archive) with in-place transactional saves and an
  incremental BlobReader; supports both packed (chunked blobs) and referenced
  (external path) media, with a user preference + first-import prompt for files
  over the large-media threshold.

Audio streaming:
- Stream packed compressed audio on load via an inversion-of-control blob factory
  (AudioBlobSourceFactory): daw-backend defines the trait, core implements it
  over BlobReader, so the audio engine stays container-agnostic.
- Bulk-activate disk streaming for all loaded clips after SetProject.
- Sample-accurate compressed seek (SeekMode::Accurate; Coarse mislands on VBR).

Video:
- Video frames decoded/streamed on demand; thumbnails generated asynchronously
  on a dedicated decoder so import/load never blocks the UI.
- The video's audio track is streamed on demand via an ffmpeg VideoAudioReader
  as a separate editable AudioClip (no /tmp WAV extraction).

Waveform overview:
- Streaming min/max LOD pyramid (waveform_pyramid), bounded memory, configurable
  floor B; serialized into the container and restored on load (or generated in
  the background from the packed blob when absent), so no re-decode on reload.
- GPU min/max upload path; integer-LOD textureLoad fixes zoom-dependent wobble.
2026-06-17 13:52:38 -04:00
Skyler Lehmkuhl f372a84313 Massive tempo refactor - make beats canonical time rep and allow them to be non constant 2026-04-02 10:26:01 -04:00
Skyler Lehmkuhl 7387299b52 node graph fixes 2026-02-15 09:11:39 -05:00
Skyler Lehmkuhl c6a8b944e5 Add copy and paste 2026-02-15 04:38:42 -05:00
Skyler Lehmkuhl 068715c0fa Use CQT transform for spectrograph instead of FFT 2026-02-14 21:18:30 -05:00
Skyler Lehmkuhl f924b4c0cd Stream audio instead of loading the whole thing into memory 2026-02-11 19:07:48 -05:00
Skyler Lehmkuhl 8e38c0c5a1 Fix audio overruns 2026-02-11 16:15:16 -05:00
Skyler Lehmkuhl 727d782190 initial audio export 2025-12-03 17:54:15 -05:00
Skyler Lehmkuhl cffb61e5a8 Load audio clips 2025-12-01 22:03:20 -05:00
Skyler Lehmkuhl ba9a4ee812 File save/load 2025-12-01 09:18:49 -05:00
Skyler Lehmkuhl c943f7bfe6 Add info panel 2025-11-30 10:01:10 -05:00
Skyler Lehmkuhl a1ad0b44b1 Add support for audio trimming and looping 2025-11-28 06:53:28 -05:00
Skyler Lehmkuhl 1ee86af94d File save/load for audio projects 2025-11-03 02:46:43 -05:00
Skyler Lehmkuhl d4fb8b721a better time stretching 2025-10-18 23:28:20 -04:00
Skyler Lehmkuhl f9e2d36f3a add metatracks 2025-10-18 22:56:38 -04:00
Skyler Lehmkuhl 9414bdcd74 Work on daw backend 2025-10-18 18:09:07 -04:00