skyler
/
Lightningbeam
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Address code-review findings across export, decode, and data model 

Export correctness:
- Honor the user's color-range (Limited/Full) on the software encode path:
  thread full_range through gpu_yuv (new shader range uniform), the CPU
  swscale fallback (sws_setColorspaceDetails → BT.709 + range, fixing the
  BT.601 hue/level shift on odd-width exports), and the encoder color tags.
- Reject HDR + WebM up front with a clear message and log the forced HEVC
  override instead of producing an unplayable file.
- Delete dead render_frame_to_rgba_hdr (hardcoded Stretch; live HDR path
  already honors the fit mode).

Decode/playback (video.rs):
- Drain the decoder at EOF (send_eof + flush) so the final B-frame-delayed
  frames render instead of erroring; per-frame logic extracted to a helper.
- Missing-PTS frames continue monotonically rather than snapping to ts=0.
- Force exact thumbnail width so sub-128px sources aren't shown stretched.

Resource leaks (gpu-video-encoder):
- dmabuf import_raw: RAII guard frees the duped fd + partial VkImages/memory
  on every error path.
- vaapi alloc: free device/frames-ctx/AVFrames on the unexpected-DRM path.

Data model / robustness:
- collapse_boundary_spikes requires a full curve reversal (all control
  points) so it no longer deletes a real lens/sliver and drops the fill.
- Export audio spin-wait ignores a stale `finished` flag when a forward
  seek is pending (was rendering silence over real audio).
- RasterDiff apply_before/after take current dims and skip on a post-resize
  mismatch.
- beam_archive read_media_full caps the preallocation from untrusted total_len.

UI/visual:
- SVG export skips hidden layers/empty groups; import folds fill-opacity into
  gradients and surfaces failures as a notification.
- Active raster-layer border uses playback_time + overlay_transform.
- gpu_brush remove_layer_texture also evicts the stale low-res proxy.
- ensure_raster_resident_for_undo registers faulted frames in the LRU so
  resident RAM stays bounded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Skyler Lehmkuhl 2026-06-26 17:47:32 -04:00
parent 1fa4d744be
commit b301c63387
20 changed files with 527 additions and 440 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
daw-backend/src/audio/pool.rs
View File

@ -582,8 +582,21 @@ impl AudioClipPool {
// or a container without exact stream duration). They will never // or a container without exact stream duration). They will never
// arrive — stop waiting and let the render fill silence, instead of // arrive — stop waiting and let the render fill silence, instead of
// burning the 10s safety valve on every remaining chunk. // burning the 10s safety valve on every remaining chunk.
//
// BUT `finished` may be stale: we just moved the target with
// `set_target_frame`, and on a forward jump past the buffer the reader
// will reset()+seek() (clearing `finished`) on its next poll. The reader
// re-seeks when `target < buf_start || target > buf_end + sample_rate`
// (see disk_reader.rs), so only trust EOF when the target is inside the
// current window — otherwise a stale flag from the previous region would
// make us render silence over audio that's about to be decoded.
if ra.is_finished() { if ra.is_finished() {
break; let (buf_start, buf_end) = ra.snapshot();
let sr = audio_file.sample_rate as u64;
let seek_pending = src_start < buf_start || src_start > buf_end + sr;
if !seek_pending {
break;
}
} }
std::thread::sleep(std::time::Duration::from_micros(100)); std::thread::sleep(std::time::Duration::from_micros(100));
wait_iters += 1; wait_iters += 1;

58
lightningbeam-ui/gpu-video-encoder/src/dmabuf.rs
View File

@ -36,6 +36,47 @@ impl Drop for MemoryGuard {
} }
} }
/// Frees the duplicated dma-buf fd and any partially-created Vulkan objects when an import
/// errors out before ownership has been handed to wgpu/`MemoryGuard`. `commit()` disarms it on
/// the success path; `fd_consumed()` is called once `vkAllocateMemory` has taken the fd.
struct ImportGuard {
device: ash::Device,
fd: libc::c_int,
img_y: vk::Image,
img_uv: vk::Image,
memory: vk::DeviceMemory,
armed: bool,
}
impl ImportGuard {
fn fd_consumed(&mut self) {
self.fd = -1; // vkAllocateMemory owns the fd now; don't close it ourselves
}
fn commit(&mut self) {
self.armed = false;
}
}
impl Drop for ImportGuard {
fn drop(&mut self) {
if !self.armed {
return;
}
unsafe {
if self.img_uv != vk::Image::null() {
self.device.destroy_image(self.img_uv, None);
}
if self.img_y != vk::Image::null() {
self.device.destroy_image(self.img_y, None);
}
if self.memory != vk::DeviceMemory::null() {
self.device.free_memory(self.memory, None);
}
if self.fd >= 0 {
libc::close(self.fd);
}
}
}
}
/// A VAAPI surface imported as two wgpu plane textures. The underlying Vulkan image/ /// A VAAPI surface imported as two wgpu plane textures. The underlying Vulkan image/
/// memory are destroyed by wgpu (via drop callbacks) when these textures drop. /// memory are destroyed by wgpu (via drop callbacks) when these textures drop.
pub struct ImportedNv12 { pub struct ImportedNv12 {
@ -103,6 +144,16 @@ pub fn import_raw(
if dup_fd < 0 { if dup_fd < 0 {
return Err("dup(dma-buf fd) failed".into()); return Err("dup(dma-buf fd) failed".into());
} }
// Owns the fd + any Vk objects created below until ownership transfers to wgpu; on any
// early `?`/return before that, its Drop frees them (was leaking on every failed import).
let mut guard = ImportGuard {
device: raw_device.clone(),
fd: dup_fd,
img_y: vk::Image::null(),
img_uv: vk::Image::null(),
memory: vk::DeviceMemory::null(),
armed: true,
};
// 16-bit-norm plane formats (P010) are NOT renderable, so the import is sample-only for // 16-bit-norm plane formats (P010) are NOT renderable, so the import is sample-only for
// those (decode path). 8-bit planes keep COLOR_ATTACHMENT for the encoder's RGBA→NV12 write. // those (decode path). 8-bit planes keep COLOR_ATTACHMENT for the encoder's RGBA→NV12 write.
@ -155,7 +206,9 @@ pub fn import_raw(
}; };
let img_y = make_image(vk_y, buf.width, buf.height, buf.y_pitch)?; let img_y = make_image(vk_y, buf.width, buf.height, buf.y_pitch)?;
guard.img_y = img_y;
let img_uv = make_image(vk_uv, buf.width / 2, buf.height / 2, buf.uv_pitch)?; let img_uv = make_image(vk_uv, buf.width / 2, buf.height / 2, buf.uv_pitch)?;
guard.img_uv = img_uv;
let fd_dev = ash::khr::external_memory_fd::Device::new(instance, &raw_device); let fd_dev = ash::khr::external_memory_fd::Device::new(instance, &raw_device);
let mut fd_props = vk::MemoryFdPropertiesKHR::default(); let mut fd_props = vk::MemoryFdPropertiesKHR::default();
@ -180,6 +233,8 @@ pub fn import_raw(
let memory = raw_device let memory = raw_device
.allocate_memory(&alloc, None) .allocate_memory(&alloc, None)
.map_err(|e| format!("vkAllocateMemory(import dma-buf) failed: {e:?}"))?; .map_err(|e| format!("vkAllocateMemory(import dma-buf) failed: {e:?}"))?;
guard.fd_consumed(); // the import transferred fd ownership to Vulkan
guard.memory = memory;
raw_device raw_device
.bind_image_memory(img_y, memory, buf.y_offset) .bind_image_memory(img_y, memory, buf.y_offset)
@ -242,6 +297,9 @@ pub fn import_raw(
}, },
) )
}; };
// Ownership of the images (→ texture drop callbacks) and memory (→ MemoryGuard) has now
// transferred to wgpu; disarm the cleanup guard so it doesn't double-free them.
guard.commit();
let y = wrap(img_y, wgpu_y, buf.width, buf.height); let y = wrap(img_y, wgpu_y, buf.width, buf.height);
let uv = wrap(img_uv, wgpu_uv, buf.width / 2, buf.height / 2); let uv = wrap(img_uv, wgpu_uv, buf.width / 2, buf.height / 2);
drop(hal_device); drop(hal_device);

12
lightningbeam-ui/gpu-video-encoder/src/vaapi.rs
View File

@ -157,10 +157,18 @@ impl MappedSurface {
let desc = (*drm).data[0] as *const ff::AVDRMFrameDescriptor; let desc = (*drm).data[0] as *const ff::AVDRMFrameDescriptor;
// Expect 1 object, 2 layers (Y=R8, UV=GR88). // Expect 1 object, 2 layers (Y=R8, UV=GR88).
if (*desc).nb_objects != 1 || (*desc).nb_layers != 2 { if (*desc).nb_objects != 1 || (*desc).nb_layers != 2 {
return Err(format!( let msg = format!(
"unexpected DRM layout: {} objects, {} layers", "unexpected DRM layout: {} objects, {} layers",
(*desc).nb_objects, (*desc).nb_layers (*desc).nb_objects, (*desc).nb_layers
)); );
// Free everything mapped/allocated above (this path was leaking the device,
// frames context, and both AVFrames on every odd-layout surface).
ff::av_frame_free(&mut (drm as *mut _));
ff::av_frame_free(&mut (surf as *mut _));
let mut fr = frames_ref;
ff::av_buffer_unref(&mut fr);
ff::av_buffer_unref(&mut hw_device);
return Err(msg);
} }
let obj = &(*desc).objects[0]; let obj = &(*desc).objects[0];
let y = &(*desc).layers[0].planes[0]; let y = &(*desc).layers[0].planes[0];

32
lightningbeam-ui/lightningbeam-core/src/actions/raster_diff.rs
View File

@ -98,11 +98,16 @@ impl RasterDiff {
self.before_region.len() + self.after_region.len() self.before_region.len() + self.after_region.len()
} }
/// Restore the pre-edit pixels into `raw` (undo / first-execute rollback). /// Restore the pre-edit pixels into `raw` (undo / first-execute rollback). `cur_w`/`cur_h`
pub fn apply_before(&self, raw: &mut Vec<u8>) { /// are the keyframe's current dimensions; if they differ from the diff's captured size the
/// diff predates a resize and is skipped rather than applied at mismatched dims.
pub fn apply_before(&self, raw: &mut Vec<u8>, cur_w: u32, cur_h: u32) {
if self.bbox.is_none() { if self.bbox.is_none() {
return; // no change return; // no change
} }
if cur_w != self.full_width || cur_h != self.full_height {
return; // diff predates a keyframe resize
}
if self.before_blank { if self.before_blank {
// The frame was blank before this edit (it was the first stroke); undoing // The frame was blank before this edit (it was the first stroke); undoing
// it returns to blank regardless of the current buffer. // it returns to blank regardless of the current buffer.
@ -112,11 +117,16 @@ impl RasterDiff {
self.stamp_resident(&self.before_region, raw); self.stamp_resident(&self.before_region, raw);
} }
/// Apply the post-edit pixels into `raw` (commit / redo). /// Apply the post-edit pixels into `raw` (commit / redo). `cur_w`/`cur_h` are the keyframe's
pub fn apply_after(&self, raw: &mut Vec<u8>) { /// current dimensions; a mismatch with the captured size means the diff predates a resize, so
/// it's skipped rather than rebuilding the buffer at stale dimensions.
pub fn apply_after(&self, raw: &mut Vec<u8>, cur_w: u32, cur_h: u32) {
if self.bbox.is_none() { if self.bbox.is_none() {
return; // no change return; // no change
} }
if cur_w != self.full_width || cur_h != self.full_height {
return; // diff predates a keyframe resize
}
if self.before_blank { if self.before_blank {
// Base was blank: build a full transparent buffer then stamp the bbox. The // Base was blank: build a full transparent buffer then stamp the bbox. The
// commit/redo path frequently starts from empty `raw_pixels` here. // commit/redo path frequently starts from empty `raw_pixels` here.
@ -175,9 +185,9 @@ mod tests {
assert_eq!(diff.bbox, Some((3, 2, 2, 2))); assert_eq!(diff.bbox, Some((3, 2, 2, 2)));
let mut buf = after.clone(); let mut buf = after.clone();
diff.apply_before(&mut buf); diff.apply_before(&mut buf, w, h);
assert_eq!(buf, before, "undo must reproduce the pre-edit buffer exactly"); assert_eq!(buf, before, "undo must reproduce the pre-edit buffer exactly");
diff.apply_after(&mut buf); diff.apply_after(&mut buf, w, h);
assert_eq!(buf, after, "redo must reproduce the post-edit buffer exactly"); assert_eq!(buf, after, "redo must reproduce the post-edit buffer exactly");
} }
@ -195,15 +205,15 @@ mod tests {
// First execute / redo from EMPTY raw_pixels (the real commit path): builds // First execute / redo from EMPTY raw_pixels (the real commit path): builds
// the full buffer from transparent + the stroke. // the full buffer from transparent + the stroke.
let mut buf: Vec<u8> = Vec::new(); let mut buf: Vec<u8> = Vec::new();
diff.apply_after(&mut buf); diff.apply_after(&mut buf, w, h);
assert_eq!(buf, after, "commit/redo must build the frame from a blank base"); assert_eq!(buf, after, "commit/redo must build the frame from a blank base");
// Undo the first stroke → back to blank (empty). // Undo the first stroke → back to blank (empty).
diff.apply_before(&mut buf); diff.apply_before(&mut buf, w, h);
assert!(buf.is_empty(), "undoing the first stroke restores the blank keyframe"); assert!(buf.is_empty(), "undoing the first stroke restores the blank keyframe");
// Redo again from the now-empty buffer. // Redo again from the now-empty buffer.
diff.apply_after(&mut buf); diff.apply_after(&mut buf, w, h);
assert_eq!(buf, after); assert_eq!(buf, after);
} }
@ -215,7 +225,7 @@ mod tests {
assert_eq!(diff.bbox, None); assert_eq!(diff.bbox, None);
assert_eq!(diff.byte_size(), 0); assert_eq!(diff.byte_size(), 0);
let mut b = buf.clone(); let mut b = buf.clone();
diff.apply_before(&mut b); diff.apply_before(&mut b, w, h);
assert_eq!(b, buf); assert_eq!(b, buf);
} }
@ -226,7 +236,7 @@ mod tests {
let after = solid(w, h, [1, 2, 3, 255]); let after = solid(w, h, [1, 2, 3, 255]);
let diff = RasterDiff::compute(&before, &after, w, h); let diff = RasterDiff::compute(&before, &after, w, h);
let mut empty: Vec<u8> = Vec::new(); let mut empty: Vec<u8> = Vec::new();
diff.apply_before(&mut empty); // base not resident diff.apply_before(&mut empty, w, h); // base not resident
assert!(empty.is_empty(), "must not resize/corrupt a non-resident base"); assert!(empty.is_empty(), "must not resize/corrupt a non-resident base");
} }
} }

4
lightningbeam-ui/lightningbeam-core/src/actions/raster_fill.rs
View File

@ -53,7 +53,7 @@ impl Action for RasterFillAction {
if let Some(full) = self.full_after.take() { if let Some(full) = self.full_after.take() {
kf.raw_pixels = full; kf.raw_pixels = full;
} else { } else {
self.diff.apply_after(&mut kf.raw_pixels); self.diff.apply_after(&mut kf.raw_pixels, kf.width, kf.height);
} }
kf.texture_dirty = true; kf.texture_dirty = true;
kf.dirty = true; kf.dirty = true;
@ -71,7 +71,7 @@ impl Action for RasterFillAction {
let kf = raster let kf = raster
.keyframe_at_mut(self.time) .keyframe_at_mut(self.time)
.ok_or_else(|| format!("No raster keyframe at/before t={}", self.time))?; .ok_or_else(|| format!("No raster keyframe at/before t={}", self.time))?;
self.diff.apply_before(&mut kf.raw_pixels); self.diff.apply_before(&mut kf.raw_pixels, kf.width, kf.height);
kf.texture_dirty = true; kf.texture_dirty = true;
kf.dirty = true; kf.dirty = true;
Ok(()) Ok(())

4
lightningbeam-ui/lightningbeam-core/src/actions/raster_stroke.rs
View File

@ -61,7 +61,7 @@ impl Action for RasterStrokeAction {
kf.raw_pixels = full; kf.raw_pixels = full;
} else { } else {
// Redo: replay via the diff onto the (resident) base. // Redo: replay via the diff onto the (resident) base.
self.diff.apply_after(&mut kf.raw_pixels); self.diff.apply_after(&mut kf.raw_pixels, kf.width, kf.height);
} }
kf.texture_dirty = true; kf.texture_dirty = true;
kf.dirty = true; kf.dirty = true;
@ -70,7 +70,7 @@ impl Action for RasterStrokeAction {
fn rollback(&mut self, document: &mut Document) -> Result<(), String> { fn rollback(&mut self, document: &mut Document) -> Result<(), String> {
let kf = get_keyframe_mut(document, &self.layer_id, self.time, self.width, self.height)?; let kf = get_keyframe_mut(document, &self.layer_id, self.time, self.width, self.height)?;
self.diff.apply_before(&mut kf.raw_pixels); self.diff.apply_before(&mut kf.raw_pixels, kf.width, kf.height);
kf.texture_dirty = true; kf.texture_dirty = true;
kf.dirty = true; kf.dirty = true;
Ok(()) Ok(())

6
lightningbeam-ui/lightningbeam-core/src/beam_archive.rs
View File

@ -374,7 +374,11 @@ impl BeamArchive {
let rows = stmt let rows = stmt
.query_map([&id_bytes], |r| r.get::<_, Vec<u8>>(0)) .query_map([&id_bytes], |r| r.get::<_, Vec<u8>>(0))
.map_err(map_sql)?; .map_err(map_sql)?;
let mut out = Vec::with_capacity(info.total_len as usize); // `total_len` is an untrusted DB field; cap the preallocation so a corrupt/oversized
// value can't trigger a multi-GB eager allocation (or, on 32-bit, a truncated `as usize`).
// The Vec still grows to fit the actual chunk bytes regardless.
const PREALLOC_CAP: u64 = 64 * 1024 * 1024;
let mut out = Vec::with_capacity(info.total_len.min(PREALLOC_CAP) as usize);
for row in rows { for row in rows {
out.extend_from_slice(&row.map_err(map_sql)?); out.extend_from_slice(&row.map_err(map_sql)?);
} }

22
lightningbeam-ui/lightningbeam-core/src/svg_export.rs
View File

@ -266,6 +266,9 @@ pub fn document_to_svg(document: &Document, time: f64) -> String {
fn layer_to_svg(layer: &AnyLayer, time: f64, parent_opacity: f64, body: &mut String, defs: &mut String, grad_n: &mut usize) { fn layer_to_svg(layer: &AnyLayer, time: f64, parent_opacity: f64, body: &mut String, defs: &mut String, grad_n: &mut usize) {
match layer { match layer {
AnyLayer::Vector(vl) => { AnyLayer::Vector(vl) => {
if !vl.layer.visible {
return; // hidden layers are not rendered, so don't export them
}
let opacity = parent_opacity * vl.layer.opacity; let opacity = parent_opacity * vl.layer.opacity;
if let Some(graph) = vl.tweened_graph_at(time) { if let Some(graph) = vl.tweened_graph_at(time) {
let wrap = opacity < 0.999; let wrap = opacity < 0.999;
@ -281,12 +284,21 @@ fn layer_to_svg(layer: &AnyLayer, time: f64, parent_opacity: f64, body: &mut Str
// exported — a refinement once loose-geometry export is verified. // exported — a refinement once loose-geometry export is verified.
} }
AnyLayer::Group(g) => { AnyLayer::Group(g) => {
let opacity = parent_opacity * g.layer.opacity; if !g.layer.visible {
body.push_str(&format!(r#"<g opacity="{opacity:.4}">"#)); return;
for child in &g.children { }
layer_to_svg(child, time, 1.0, body, defs, grad_n); // Render children first; only emit the <g> wrapper if it has exportable content
// (avoids empty groups when every child is a non-vector/hidden layer).
let mut inner = String::new();
for child in &g.children {
layer_to_svg(child, time, 1.0, &mut inner, defs, grad_n);
}
if !inner.is_empty() {
let opacity = parent_opacity * g.layer.opacity;
body.push_str(&format!(r#"<g opacity="{opacity:.4}">"#));
body.push_str(&inner);
body.push_str("</g>");
} }
body.push_str("</g>");
} }
// Raster/Video/Audio/Effect have no lossless vector representation — skipped this pass. // Raster/Video/Audio/Effect have no lossless vector representation — skipped this pass.
_ => {} _ => {}

30
lightningbeam-ui/lightningbeam-core/src/vector_graph/mod.rs
View File

@ -1302,21 +1302,26 @@ impl VectorGraph {
/// bouncing across near-coincident duplicate edges). These are zero-area and would make /// bouncing across near-coincident duplicate edges). These are zero-area and would make
/// `boundary_to_bezpath` render a stray hair; collapsing them yields a simple loop. /// `boundary_to_bezpath` render a stray hair; collapsing them yields a simple loop.
fn collapse_boundary_spikes(&self, face: &mut Vec<(EdgeId, Direction)>) { fn collapse_boundary_spikes(&self, face: &mut Vec<(EdgeId, Direction)>) {
let dstart = |entry: &(EdgeId, Direction)| -> Point { // The four control points of an entry's curve in its traversal order.
let traversed = |entry: &(EdgeId, Direction)| -> [Point; 4] {
let c = self.edges[entry.0.idx()].curve; let c = self.edges[entry.0.idx()].curve;
match entry.1 { match entry.1 {
Direction::Forward => c.p0, Direction::Forward => [c.p0, c.p1, c.p2, c.p3],
Direction::Backward => c.p3, Direction::Backward => [c.p3, c.p2, c.p1, c.p0],
}
};
let dend = |entry: &(EdgeId, Direction)| -> Point {
let c = self.edges[entry.0.idx()].curve;
match entry.1 {
Direction::Forward => c.p3,
Direction::Backward => c.p0,
} }
}; };
const EPS: f64 = 0.5; const EPS: f64 = 0.5;
// Entries i and j cancel only when j is the *exact reverse* of i — every control point of
// j matches the mirror of i. Testing endpoints alone would also collapse a genuine
// lens/sliver (two distinct edges that merely share near-coincident endpoints), silently
// deleting real boundary geometry and dropping the fill.
let reverses = |a: &(EdgeId, Direction), b: &(EdgeId, Direction)| -> bool {
let (ca, cb) = (traversed(a), traversed(b));
(0..4).all(|k| {
let (p, q) = (ca[k], cb[3 - k]);
(p.x - q.x).hypot(p.y - q.y) < EPS
})
};
loop { loop {
let n = face.len(); let n = face.len();
if n < 2 { if n < 2 {
@ -1325,10 +1330,7 @@ impl VectorGraph {
let mut collapsed = false; let mut collapsed = false;
for i in 0..n { for i in 0..n {
let j = (i + 1) % n; let j = (i + 1) % n;
// entries i and j cancel when j ends back at i's start. if reverses(&face[i], &face[j]) {
let si = dstart(&face[i]);
let ej = dend(&face[j]);
if (si.x - ej.x).hypot(si.y - ej.y) < EPS {
let (hi, lo) = if i > j { (i, j) } else { (j, i) }; let (hi, lo) = if i > j { (i, j) } else { (j, i) };
face.remove(hi); face.remove(hi);
face.remove(lo); face.remove(lo);

356
lightningbeam-ui/lightningbeam-core/src/video.rs
View File

@ -120,6 +120,153 @@ enum DecodedFrame {
Gpu(GpuVideoFrame), Gpu(GpuVideoFrame),
} }
/// Result of processing one decoded frame against the requested target timestamp.
enum FrameOutcome {
/// Frame consumed; keep decoding.
Continue,
/// `current_frame_ts >= frame_ts` — caller should return the best frame found.
ReachedTarget,
/// Hardware surface import failed; caller should fall back to software.
HwImportFailed,
}
/// Process one decoded `frame`: update the best-so-far (`best_*`) toward `frame_ts`, importing a
/// GPU surface (`gpu_out`) or swscaling to RGBA otherwise. Pulled out of the decode loop so the
/// same logic runs both for packet-fed frames and for frames flushed out of the codec at EOF.
#[allow(clippy::too_many_arguments)]
fn process_decoded_video_frame(
frame: &mut ffmpeg::util::frame::Video,
decoder: &ffmpeg::decoder::Video,
gpu_out: bool,
hw: bool,
out_w: u32,
out_h: u32,
frame_ts: i64,
importer: Option<&Arc<dyn HwVideoImporter>>,
scaler: &mut Option<(ffmpeg::format::Pixel, u32, u32, u32, u32, SendScaler)>,
best_frame_data: &mut Option<Vec<u8>>,
best_gpu: &mut Option<GpuVideoFrame>,
best_frame_ts: &mut Option<i64>,
last_decoded_ts: &mut i64,
scale_time_ms: &mut u128,
) -> Result<FrameOutcome, String> {
use std::time::Instant;
// A frame with no PTS continues monotonically from the last decoded position rather than
// snapping to 0 — treating "no timestamp" as ts=0 would look like a huge backward jump and
// corrupt the best-frame / seek logic on streams with missing PTS.
let current_frame_ts = frame.timestamp().unwrap_or(*last_decoded_ts + 1);
*last_decoded_ts = current_frame_ts;
let is_better = match *best_frame_ts {
None => true,
Some(best_ts) => (current_frame_ts - frame_ts).abs() < (best_ts - frame_ts).abs(),
};
if is_better {
if gpu_out {
// VAAPI hw frames often don't carry the stream's colour tags, so the importer would
// mis-detect transfer/gamut. Copy the authoritative values from the codec context
// (parsed from the bitstream) onto the frame when it left them unspecified.
unsafe {
use ffmpeg::ffi::*;
let fp = frame.as_mut_ptr();
let cp = decoder.as_ptr();
if (*fp).color_trc == AVColorTransferCharacteristic::AVCOL_TRC_UNSPECIFIED {
(*fp).color_trc = (*cp).color_trc;
}
if (*fp).color_primaries == AVColorPrimaries::AVCOL_PRI_UNSPECIFIED {
(*fp).color_primaries = (*cp).color_primaries;
}
if (*fp).colorspace == AVColorSpace::AVCOL_SPC_UNSPECIFIED {
(*fp).colorspace = (*cp).colorspace;
}
if (*fp).color_range == AVColorRange::AVCOL_RANGE_UNSPECIFIED {
(*fp).color_range = (*cp).color_range;
}
}
let importer = importer.unwrap();
match unsafe { importer.import(frame.as_mut_ptr() as *mut std::ffi::c_void) } {
Some(gpu) => {
*best_gpu = Some(gpu);
*best_frame_ts = Some(current_frame_ts);
}
None => return Ok(FrameOutcome::HwImportFailed),
}
} else {
let t_scale_start = Instant::now();
// A hardware decoder produces VAAPI surfaces; a CPU consumer (export) downloads to
// system memory first, then swscales like the software path.
let downloaded;
let src: &ffmpeg::util::frame::Video = if hw {
let mut dl = ffmpeg::util::frame::Video::empty();
let r = unsafe {
ffmpeg::ffi::av_hwframe_transfer_data(dl.as_mut_ptr(), frame.as_ptr(), 0)
};
if r < 0 {
return Err(format!("av_hwframe_transfer_data failed: {r}"));
}
downloaded = dl;
&downloaded
} else {
&*frame
};
// Reuse the RGBA scaler across frames; rebuild only if the input format/size or the
// requested output size changes.
let need_new = match scaler {
Some((fmt, w, h, ow, oh, _)) => {
*fmt != src.format() || *w != src.width() || *h != src.height()
|| *ow != out_w || *oh != out_h
}
None => true,
};
if need_new {
let ctx = ffmpeg::software::scaling::context::Context::get(
src.format(),
src.width(),
src.height(),
ffmpeg::format::Pixel::RGBA,
out_w,
out_h,
ffmpeg::software::scaling::flag::Flags::BILINEAR,
).map_err(|e| e.to_string())?;
*scaler = Some((src.format(), src.width(), src.height(), out_w, out_h, SendScaler(ctx)));
}
let scaler = &mut scaler.as_mut().unwrap().5.0;
let mut rgb_frame = ffmpeg::util::frame::Video::empty();
scaler.run(src, &mut rgb_frame)
.map_err(|e| e.to_string())?;
// Remove stride padding to create tightly packed RGBA data
let width = out_w as usize;
let height = out_h as usize;
let stride = rgb_frame.stride(0);
let row_size = width * 4; // RGBA = 4 bytes per pixel
let source_data = rgb_frame.data(0);
let mut packed_data = Vec::with_capacity(row_size * height);
for y in 0..height {
let row_start = y * stride;
let row_end = row_start + row_size;
packed_data.extend_from_slice(&source_data[row_start..row_end]);
}
*scale_time_ms += t_scale_start.elapsed().as_millis();
*best_frame_data = Some(packed_data);
*best_frame_ts = Some(current_frame_ts);
}
}
if current_frame_ts >= frame_ts {
Ok(FrameOutcome::ReachedTarget)
} else {
Ok(FrameOutcome::Continue)
}
}
/// `get_format` callback for hardware decode: select VAAPI surfaces. With `hw_device_ctx` set, /// `get_format` callback for hardware decode: select VAAPI surfaces. With `hw_device_ctx` set,
/// FFmpeg auto-allocates the frames context. /// FFmpeg auto-allocates the frames context.
unsafe extern "C" fn get_vaapi_format( unsafe extern "C" fn get_vaapi_format(
@ -459,159 +606,73 @@ impl VideoDecoder {
let mut scale_time_ms = 0u128; let mut scale_time_ms = 0u128;
let mut hw_import_failed = false; let mut hw_import_failed = false;
'decode: for (stream, packet) in input.packets() { 'decode: {
if stream.index() == self.stream_index { for (stream, packet) in input.packets() {
decoder.send_packet(&packet) if stream.index() == self.stream_index {
.map_err(|e| e.to_string())?; decoder.send_packet(&packet)
.map_err(|e| e.to_string())?;
let mut frame = ffmpeg::util::frame::Video::empty();
while decoder.receive_frame(&mut frame).is_ok() {
decode_count += 1;
match process_decoded_video_frame(
&mut frame, decoder, gpu_out, hw, out_w, out_h, frame_ts,
self.importer.as_ref(), &mut self.scaler,
&mut best_frame_data, &mut best_gpu, &mut best_frame_ts,
&mut self.last_decoded_ts, &mut scale_time_ms,
)? {
FrameOutcome::Continue => {}
FrameOutcome::ReachedTarget => break 'decode,
FrameOutcome::HwImportFailed => {
self.hw_failed = true;
hw_import_failed = true;
break 'decode;
}
}
}
}
}
// Flush: the codec may still hold buffered frames (B-frame reorder delay) past the
// last packet. Drain them so requesting the final frame(s) of a clip — scrubbing to
// the end or exporting the tail — doesn't fail with "Failed to decode frame".
if !hw_import_failed {
let _ = decoder.send_eof();
let mut frame = ffmpeg::util::frame::Video::empty(); let mut frame = ffmpeg::util::frame::Video::empty();
while decoder.receive_frame(&mut frame).is_ok() { while decoder.receive_frame(&mut frame).is_ok() {
decode_count += 1; decode_count += 1;
let current_frame_ts = frame.timestamp().unwrap_or(0); match process_decoded_video_frame(
self.last_decoded_ts = current_frame_ts; // Update last decoded position &mut frame, decoder, gpu_out, hw, out_w, out_h, frame_ts,
self.importer.as_ref(), &mut self.scaler,
// Check if this frame is closer to our target than the previous best &mut best_frame_data, &mut best_gpu, &mut best_frame_ts,
let is_better = match best_frame_ts { &mut self.last_decoded_ts, &mut scale_time_ms,
None => true, )? {
Some(best_ts) => { FrameOutcome::Continue => {}
(current_frame_ts - frame_ts).abs() < (best_ts - frame_ts).abs() FrameOutcome::ReachedTarget => break 'decode,
FrameOutcome::HwImportFailed => {
self.hw_failed = true;
hw_import_failed = true;
break 'decode;
} }
};
if is_better {
if gpu_out {
// Hardware + GPU consumer: import the VAAPI surface as wgpu NV12 textures
// (no CPU copy).
// VAAPI hw frames often don't carry the stream's colour tags, so the
// importer (which only sees the frame) would mis-detect transfer/gamut.
// Copy the authoritative values from the codec context (parsed from the
// bitstream) onto the frame when it left them unspecified.
unsafe {
use ffmpeg::ffi::*;
let fp = frame.as_mut_ptr();
let cp = decoder.as_ptr();
if (*fp).color_trc == AVColorTransferCharacteristic::AVCOL_TRC_UNSPECIFIED {
(*fp).color_trc = (*cp).color_trc;
}
if (*fp).color_primaries == AVColorPrimaries::AVCOL_PRI_UNSPECIFIED {
(*fp).color_primaries = (*cp).color_primaries;
}
if (*fp).colorspace == AVColorSpace::AVCOL_SPC_UNSPECIFIED {
(*fp).colorspace = (*cp).colorspace;
}
if (*fp).color_range == AVColorRange::AVCOL_RANGE_UNSPECIFIED {
(*fp).color_range = (*cp).color_range;
}
}
let importer = self.importer.as_ref().unwrap();
match unsafe { importer.import(frame.as_mut_ptr() as *mut std::ffi::c_void) } {
Some(gpu) => {
best_gpu = Some(gpu);
best_frame_ts = Some(current_frame_ts);
}
None => {
// Import failed → fall back to software for this clip.
self.hw_failed = true;
hw_import_failed = true;
break 'decode;
}
}
} else {
let t_scale_start = Instant::now();
// A hardware decoder produces VAAPI surfaces; a CPU consumer (export)
// downloads to system memory first, then swscales like the software path.
let downloaded;
let src: &ffmpeg::util::frame::Video = if hw {
let mut dl = ffmpeg::util::frame::Video::empty();
let r = unsafe {
ffmpeg::ffi::av_hwframe_transfer_data(dl.as_mut_ptr(), frame.as_ptr(), 0)
};
if r < 0 {
return Err(format!("av_hwframe_transfer_data failed: {r}"));
}
downloaded = dl;
&downloaded
} else {
&frame
};
// Reuse the RGBA scaler across frames; rebuild only if the input
// format/size or the requested output size changes.
let need_new = match &self.scaler {
Some((fmt, w, h, ow, oh, _)) => {
*fmt != src.format() || *w != src.width() || *h != src.height()
|| *ow != out_w || *oh != out_h
}
None => true,
};
if need_new {
let ctx = ffmpeg::software::scaling::context::Context::get(
src.format(),
src.width(),
src.height(),
ffmpeg::format::Pixel::RGBA,
out_w,
out_h,
ffmpeg::software::scaling::flag::Flags::BILINEAR,
).map_err(|e| e.to_string())?;
self.scaler = Some((src.format(), src.width(), src.height(), out_w, out_h, SendScaler(ctx)));
}
let scaler = &mut self.scaler.as_mut().unwrap().5.0;
let mut rgb_frame = ffmpeg::util::frame::Video::empty();
scaler.run(src, &mut rgb_frame)
.map_err(|e| e.to_string())?;
// Remove stride padding to create tightly packed RGBA data
let width = out_w as usize;
let height = out_h as usize;
let stride = rgb_frame.stride(0);
let row_size = width * 4; // RGBA = 4 bytes per pixel
let source_data = rgb_frame.data(0);
let mut packed_data = Vec::with_capacity(row_size * height);
for y in 0..height {
let row_start = y * stride;
let row_end = row_start + row_size;
packed_data.extend_from_slice(&source_data[row_start..row_end]);
}
scale_time_ms += t_scale_start.elapsed().as_millis();
best_frame_data = Some(packed_data);
best_frame_ts = Some(current_frame_ts);
}
}
// If we've reached or passed the target timestamp, we can stop
if current_frame_ts >= frame_ts {
if video_debug() {
let total_time = t_start.elapsed().as_millis();
let decode_time = t_decode_start.elapsed().as_millis();
eprintln!("[Video Timing] ts={:.3}s | Decoded {} frames in {}ms | Scale: {}ms | Total: {}ms | {}",
timestamp, decode_count, decode_time, scale_time_ms, total_time, if hw { "hw" } else { "sw" });
}
if gpu_out {
if let Some(gpu) = best_gpu.take() {
return Ok(DecodedFrame::Gpu(gpu));
}
} else if let Some(data) = best_frame_data {
self.frame_cache.put(cache_key, data.clone());
return Ok(DecodedFrame::Cpu { rgba: data, width: out_w, height: out_h });
}
break 'decode;
} }
} }
} }
} }
// Reached EOF without hitting the target, or HW import failed mid-stream. if video_debug() {
let total_time = t_start.elapsed().as_millis();
let decode_time = t_decode_start.elapsed().as_millis();
eprintln!("[Video Timing] ts={:.3}s | Decoded {} frames in {}ms | Scale: {}ms | Total: {}ms | {}",
timestamp, decode_count, decode_time, scale_time_ms, total_time, if hw { "hw" } else { "sw" });
}
// Reached the target, EOF, or HW import failed mid-stream.
if hw_import_failed { if hw_import_failed {
self.decoder = None; // force a software rebuild next call (decoder borrow ended here) self.decoder = None; // force a software rebuild next call (decoder borrow ended here)
self.input = None; self.input = None;
return Err("hardware frame import failed; retrying software".to_string()); return Err("hardware frame import failed; retrying software".to_string());
} }
// EOF: return the closest frame we found, if any. // Return the closest frame we found, if any.
if gpu_out { if gpu_out {
if let Some(gpu) = best_gpu.take() { if let Some(gpu) = best_gpu.take() {
return Ok(DecodedFrame::Gpu(gpu)); return Ok(DecodedFrame::Gpu(gpu));
@ -673,8 +734,23 @@ pub fn generate_keyframe_thumbnails(
} }
// Decode at the thumbnail width (large height so width is the constraint), capped to native. // Decode at the thumbnail width (large height so width is the constraint), capped to native.
// Thumbnail decoders are always software (no hardware importer). // Thumbnail decoders are always software (no hardware importer).
if let Ok(DecodedFrame::Cpu { rgba, .. }) = decoder.get_frame(ks, thumb_width, 100_000, false) { if let Ok(DecodedFrame::Cpu { rgba, width, height }) = decoder.get_frame(ks, thumb_width, 100_000, false) {
on_thumb(ks, Arc::new(rgba)); // `capped_output` never upscales, so a source narrower than `thumb_width` decodes
// smaller — but `get_thumbnail_at` reconstructs height assuming an exact `thumb_width`.
// Force the exact width here (rare path) so that assumption holds and the thumbnail
// isn't shown stretched.
let data = if width == thumb_width || width == 0 || height == 0 {
rgba
} else {
let new_h = ((thumb_width as u64 * height as u64) / width as u64).max(1) as u32;
match image::RgbaImage::from_raw(width, height, rgba) {
Some(img) => image::imageops::resize(
&img, thumb_width, new_h, image::imageops::FilterType::Triangle,
).into_raw(),
None => continue,
}
};
on_thumb(ks, Arc::new(data));
} }
} }
Ok(()) Ok(())

27
lightningbeam-ui/lightningbeam-editor/src/export/cpu_yuv_converter.rs
View File

@ -26,9 +26,9 @@ impl CpuYuvConverter {
/// # Arguments /// # Arguments
/// * `width` - Frame width in pixels /// * `width` - Frame width in pixels
/// * `height` - Frame height in pixels /// * `height` - Frame height in pixels
pub fn new(width: u32, height: u32) -> Result<Self, String> { pub fn new(width: u32, height: u32, full_range: bool) -> Result<Self, String> {
// BT.709 (HD) RGBA→YUV420p context, created once. // BT.709 (HD) RGBA→YUV420p context, created once.
let scaler = ffmpeg::software::scaling::Context::get( let mut scaler = ffmpeg::software::scaling::Context::get(
ffmpeg::format::Pixel::RGBA, ffmpeg::format::Pixel::RGBA,
width, width,
height, height,
@ -38,6 +38,23 @@ impl CpuYuvConverter {
ffmpeg::software::scaling::Flags::BILINEAR, ffmpeg::software::scaling::Flags::BILINEAR,
) )
.map_err(|e| format!("Failed to create swscale context: {}", e))?; .map_err(|e| format!("Failed to create swscale context: {}", e))?;
// swscale defaults to BT.601 + limited range; force BT.709 with the requested output
// range so this fallback matches the GPU path and the encoder's color tags
// (otherwise non-%8-width exports come out with shifted hue / wrong levels). There is
// no safe ffmpeg-next wrapper for sws_setColorspaceDetails, so this is the raw call.
unsafe {
let coeffs = ffmpeg::ffi::sws_getCoefficients(ffmpeg::ffi::SWS_CS_ITU709 as i32);
let dst_range = if full_range { 1 } else { 0 };
let one = 1 << 16; // 16.16 fixed-point 1.0
ffmpeg::ffi::sws_setColorspaceDetails(
scaler.as_mut_ptr(),
coeffs, 1, // source table (RGB input is full-range)
coeffs, dst_range, // dest table = BT.709, dest range = requested
0, one, one, // brightness, contrast, saturation (neutral)
);
}
let rgba_frame = ffmpeg::frame::Video::new(ffmpeg::format::Pixel::RGBA, width, height); let rgba_frame = ffmpeg::frame::Video::new(ffmpeg::format::Pixel::RGBA, width, height);
let yuv_frame = ffmpeg::frame::Video::new(ffmpeg::format::Pixel::YUV420P, width, height); let yuv_frame = ffmpeg::frame::Video::new(ffmpeg::format::Pixel::YUV420P, width, height);
Ok(Self { width, height, scaler, rgba_frame, yuv_frame }) Ok(Self { width, height, scaler, rgba_frame, yuv_frame })
@ -90,13 +107,13 @@ mod tests {
#[test] #[test]
fn test_converter_creation() { fn test_converter_creation() {
let converter = CpuYuvConverter::new(1920, 1080); let converter = CpuYuvConverter::new(1920, 1080, true);
assert!(converter.is_ok()); assert!(converter.is_ok());
} }
#[test] #[test]
fn test_conversion_output_sizes() { fn test_conversion_output_sizes() {
let mut converter = CpuYuvConverter::new(1920, 1080).unwrap(); let mut converter = CpuYuvConverter::new(1920, 1080, true).unwrap();
// Create dummy RGBA data (all black) // Create dummy RGBA data (all black)
let rgba_data = vec![0u8; 1920 * 1080 * 4]; let rgba_data = vec![0u8; 1920 * 1080 * 4];
@ -117,7 +134,7 @@ mod tests {
#[test] #[test]
#[should_panic(expected = "RGBA data size mismatch")] #[should_panic(expected = "RGBA data size mismatch")]
fn test_wrong_input_size_panics() { fn test_wrong_input_size_panics() {
let mut converter = CpuYuvConverter::new(1920, 1080).unwrap(); let mut converter = CpuYuvConverter::new(1920, 1080, true).unwrap();
// Wrong size input // Wrong size input
let rgba_data = vec![0u8; 1000]; let rgba_data = vec![0u8; 1000];

76
lightningbeam-ui/lightningbeam-editor/src/export/gpu_yuv.rs
View File

@ -6,8 +6,8 @@
//! 8.3 MB RGBA) and — more importantly — the per-frame CPU `rgba_to_yuv420p` (swscale) //! 8.3 MB RGBA) and — more importantly — the per-frame CPU `rgba_to_yuv420p` (swscale)
//! is eliminated. //! is eliminated.
//! //!
//! Color math is BT.709 **full-range** (JPEG range), matching the encoder color tags //! Color math is BT.709; the Y/chroma scale+offset (full vs limited range) is selected by
//! set in `setup_video_encoder` (`Space::BT709` + `Range::JPEG`). //! the `full_range` flag and must match the encoder color tags set in `setup_video_encoder`.
//! //!
//! Output buffer layout (tight, little-endian byte packing into `array<u32>`): //! Output buffer layout (tight, little-endian byte packing into `array<u32>`):
//! - `[0, W*H)` Y plane, row stride `W` //! - `[0, W*H)` Y plane, row stride `W`
@ -30,15 +30,36 @@ pub fn yuv420p_len(width: u32, height: u32) -> usize {
y + 2 * c y + 2 * c
} }
/// `(y_offset, y_scale, chroma_offset, chroma_scale)` as fractions of 255, selecting
/// limited (TV, 16–235 / 16–240) vs full (PC, 0–255) range. Mirrors `render_nv12`.
fn range_params(full_range: bool) -> [f32; 4] {
if full_range {
[0.0, 1.0, 128.0 / 255.0, 1.0]
} else {
[16.0 / 255.0, 219.0 / 255.0, 128.0 / 255.0, 224.0 / 255.0]
}
}
/// GPU compute pipeline: `Rgba8Unorm` texture → tight planar YUV420p storage buffer. /// GPU compute pipeline: `Rgba8Unorm` texture → tight planar YUV420p storage buffer.
pub struct GpuYuv { pub struct GpuYuv {
y_pipeline: wgpu::ComputePipeline, y_pipeline: wgpu::ComputePipeline,
uv_pipeline: wgpu::ComputePipeline, uv_pipeline: wgpu::ComputePipeline,
bind_group_layout: wgpu::BindGroupLayout, bind_group_layout: wgpu::BindGroupLayout,
range_buffer: wgpu::Buffer,
} }
impl GpuYuv { impl GpuYuv {
pub fn new(device: &wgpu::Device) -> Self { /// `full_range`: true → full/PC range (Y 0–255), false → limited/TV range (Y 16–235).
/// The encoder must tag the stream to match (`setup_video_encoder`'s `full_range`).
pub fn new(device: &wgpu::Device, full_range: bool) -> Self {
use wgpu::util::DeviceExt;
let params = range_params(full_range);
let range_buffer = device.create_buffer_init(&wgpu::util::BufferInitDescriptor {
label: Some("gpu_yuv_range"),
contents: bytemuck::cast_slice(&params),
usage: wgpu::BufferUsages::UNIFORM,
});
let bind_group_layout = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor { let bind_group_layout = device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("gpu_yuv_bgl"), label: Some("gpu_yuv_bgl"),
entries: &[ entries: &[
@ -64,6 +85,17 @@ impl GpuYuv {
}, },
count: None, count: None,
}, },
// 2: range params (y_offset, y_scale, chroma_offset, chroma_scale)
wgpu::BindGroupLayoutEntry {
binding: 2,
visibility: wgpu::ShaderStages::COMPUTE,
ty: wgpu::BindingType::Buffer {
ty: wgpu::BufferBindingType::Uniform,
has_dynamic_offset: false,
min_binding_size: None,
},
count: None,
},
], ],
}); });
@ -93,6 +125,7 @@ impl GpuYuv {
y_pipeline: mk("y_main"), y_pipeline: mk("y_main"),
uv_pipeline: mk("uv_main"), uv_pipeline: mk("uv_main"),
bind_group_layout, bind_group_layout,
range_buffer,
} }
} }
@ -118,6 +151,7 @@ impl GpuYuv {
entries: &[ entries: &[
wgpu::BindGroupEntry { binding: 0, resource: wgpu::BindingResource::TextureView(rgba_view) }, wgpu::BindGroupEntry { binding: 0, resource: wgpu::BindingResource::TextureView(rgba_view) },
wgpu::BindGroupEntry { binding: 1, resource: yuv_buffer.as_entire_binding() }, wgpu::BindGroupEntry { binding: 1, resource: yuv_buffer.as_entire_binding() },
wgpu::BindGroupEntry { binding: 2, resource: self.range_buffer.as_entire_binding() },
], ],
}); });
@ -142,12 +176,12 @@ impl GpuYuv {
/// CPU reference for the exact math/layout the shader produces — used by unit tests so /// CPU reference for the exact math/layout the shader produces — used by unit tests so
/// the packing and BT.709 coefficients stay verifiable without a GPU. /// the packing and BT.709 coefficients stay verifiable without a GPU.
#[cfg(test)] fn cpu_reference(rgba: &[u8], width: u32, height: u32, full_range: bool) -> Vec<u8> {
fn cpu_reference(rgba: &[u8], width: u32, height: u32) -> Vec<u8> {
let w = width as usize; let w = width as usize;
let h = height as usize; let h = height as usize;
let cw = w / 2; let cw = w / 2;
let ch = h / 2; let ch = h / 2;
let [yo, ys, co, cs] = range_params(full_range);
let mut out = vec![0u8; yuv420p_len(width, height)]; let mut out = vec![0u8; yuv420p_len(width, height)];
let to_byte = |v: f32| (v.clamp(0.0, 1.0) * 255.0 + 0.5) as u8; let to_byte = |v: f32| (v.clamp(0.0, 1.0) * 255.0 + 0.5) as u8;
let px = |x: usize, y: usize| { let px = |x: usize, y: usize| {
@ -158,7 +192,8 @@ fn cpu_reference(rgba: &[u8], width: u32, height: u32) -> Vec<u8> {
for y in 0..h { for y in 0..h {
for x in 0..w { for x in 0..w {
let p = px(x, y); let p = px(x, y);
out[y * w + x] = to_byte(0.2126 * p[0] + 0.7152 * p[1] + 0.0722 * p[2]); let yy = 0.2126 * p[0] + 0.7152 * p[1] + 0.0722 * p[2];
out[y * w + x] = to_byte(yo + ys * yy);
} }
} }
// U/V (2x2 average) // U/V (2x2 average)
@ -172,19 +207,21 @@ fn cpu_reference(rgba: &[u8], width: u32, height: u32) -> Vec<u8> {
acc[0] += p[0]; acc[1] += p[1]; acc[2] += p[2]; acc[0] += p[0]; acc[1] += p[1]; acc[2] += p[2];
} }
let a = [acc[0] / 4.0, acc[1] / 4.0, acc[2] / 4.0]; let a = [acc[0] / 4.0, acc[1] / 4.0, acc[2] / 4.0];
let u = -0.1146 * a[0] - 0.3854 * a[1] + 0.5000 * a[2] + 0.5; let uc = -0.1146 * a[0] - 0.3854 * a[1] + 0.5000 * a[2];
let v = 0.5000 * a[0] - 0.4542 * a[1] - 0.0458 * a[2] + 0.5; let vc = 0.5000 * a[0] - 0.4542 * a[1] - 0.0458 * a[2];
out[y_size + cy * cw + cx] = to_byte(u); out[y_size + cy * cw + cx] = to_byte(co + cs * uc);
out[y_size + uv_size + cy * cw + cx] = to_byte(v); out[y_size + uv_size + cy * cw + cx] = to_byte(co + cs * vc);
} }
} }
out out
} }
const SHADER: &str = r#" const SHADER: &str = r#"
// RGBA -> tight planar YUV420p (BT.709 full-range), packed 4 bytes/u32. // RGBA -> tight planar YUV420p (BT.709), packed 4 bytes/u32.
// rng = (y_offset, y_scale, chroma_offset, chroma_scale): selects limited vs full range.
@group(0) @binding(0) var input_rgba: texture_2d<f32>; @group(0) @binding(0) var input_rgba: texture_2d<f32>;
@group(0) @binding(1) var<storage, read_write> out_buf: array<u32>; @group(0) @binding(1) var<storage, read_write> out_buf: array<u32>;
@group(0) @binding(2) var<uniform> rng: vec4<f32>;
fn to_byte(v: f32) -> u32 { return u32(clamp(v, 0.0, 1.0) * 255.0 + 0.5); } fn to_byte(v: f32) -> u32 { return u32(clamp(v, 0.0, 1.0) * 255.0 + 0.5); }
@ -201,7 +238,7 @@ fn y_main(@builtin(global_invocation_id) gid: vec3<u32>) {
for (var i = 0u; i < 4u; i = i + 1u) { for (var i = 0u; i < 4u; i = i + 1u) {
let c = textureLoad(input_rgba, vec2<u32>(x4 + i, y), 0).rgb; let c = textureLoad(input_rgba, vec2<u32>(x4 + i, y), 0).rgb;
let yy = 0.2126 * c.r + 0.7152 * c.g + 0.0722 * c.b; let yy = 0.2126 * c.r + 0.7152 * c.g + 0.0722 * c.b;
packed = packed | (to_byte(yy) << (8u * i)); packed = packed | (to_byte(rng.x + rng.y * yy) << (8u * i));
} }
out_buf[(y * w + x4) / 4u] = packed; out_buf[(y * w + x4) / 4u] = packed;
} }
@ -230,10 +267,11 @@ fn uv_main(@builtin(global_invocation_id) gid: vec3<u32>) {
let p01 = textureLoad(input_rgba, vec2<u32>(sx, sy + 1u), 0).rgb; let p01 = textureLoad(input_rgba, vec2<u32>(sx, sy + 1u), 0).rgb;
let p11 = textureLoad(input_rgba, vec2<u32>(sx + 1u, sy + 1u), 0).rgb; let p11 = textureLoad(input_rgba, vec2<u32>(sx + 1u, sy + 1u), 0).rgb;
let a = (p00 + p10 + p01 + p11) * 0.25; let a = (p00 + p10 + p01 + p11) * 0.25;
let u = -0.1146 * a.r - 0.3854 * a.g + 0.5000 * a.b + 0.5; // Centered chroma in [-0.5, 0.5], then map to range via (offset + scale*coef).
let v = 0.5000 * a.r - 0.4542 * a.g - 0.0458 * a.b + 0.5; let uc = -0.1146 * a.r - 0.3854 * a.g + 0.5000 * a.b;
up = up | (to_byte(u) << (8u * i)); let vc = 0.5000 * a.r - 0.4542 * a.g - 0.0458 * a.b;
vp = vp | (to_byte(v) << (8u * i)); up = up | (to_byte(rng.z + rng.w * uc) << (8u * i));
vp = vp | (to_byte(rng.z + rng.w * vc) << (8u * i));
} }
out_buf[(y_size + cy * cw + cx4) / 4u] = up; out_buf[(y_size + cy * cw + cx4) / 4u] = up;
out_buf[(y_size + uv_size + cy * cw + cx4) / 4u] = vp; out_buf[(y_size + uv_size + cy * cw + cx4) / 4u] = vp;
@ -264,14 +302,14 @@ mod tests {
fn reference_known_colors() { fn reference_known_colors() {
// 8x2 solid white → Y≈255, U≈V≈128. Solid black → Y=0, U=V≈128. // 8x2 solid white → Y≈255, U≈V≈128. Solid black → Y=0, U=V≈128.
let white = vec![255u8; 8 * 2 * 4]; let white = vec![255u8; 8 * 2 * 4];
let out = cpu_reference(&white, 8, 2); let out = cpu_reference(&white, 8, 2, true);
let (cw, ch) = (4usize, 1usize); let (cw, ch) = (4usize, 1usize);
let y_size = 8 * 2; let y_size = 8 * 2;
for &y in &out[..y_size] { assert!(y >= 254, "white Y={y}"); } for &y in &out[..y_size] { assert!(y >= 254, "white Y={y}"); }
for &u in &out[y_size..y_size + cw * ch] { assert!((u as i32 - 128).abs() <= 1, "white U={u}"); } for &u in &out[y_size..y_size + cw * ch] { assert!((u as i32 - 128).abs() <= 1, "white U={u}"); }
let black = vec![0u8; 8 * 2 * 4]; let black = vec![0u8; 8 * 2 * 4];
let out = cpu_reference(&black, 8, 2); let out = cpu_reference(&black, 8, 2, true);
for &y in &out[..y_size] { assert_eq!(y, 0); } for &y in &out[..y_size] { assert_eq!(y, 0); }
for &v in &out[y_size + cw * ch..] { assert!((v as i32 - 128).abs() <= 1, "black V={v}"); } for &v in &out[y_size + cw * ch..] { assert!((v as i32 - 128).abs() <= 1, "black V={v}"); }
} }
@ -280,7 +318,7 @@ mod tests {
fn reference_red_bt709() { fn reference_red_bt709() {
// Solid red (255,0,0): Y=0.2126*255≈54; V high, U low (full range). // Solid red (255,0,0): Y=0.2126*255≈54; V high, U low (full range).
let red: Vec<u8> = (0..8 * 2).flat_map(|_| [255u8, 0, 0, 255]).collect(); let red: Vec<u8> = (0..8 * 2).flat_map(|_| [255u8, 0, 0, 255]).collect();
let out = cpu_reference(&red, 8, 2); let out = cpu_reference(&red, 8, 2, true);
assert!((out[0] as i32 - 54).abs() <= 1, "red Y={}", out[0]); assert!((out[0] as i32 - 54).abs() <= 1, "red Y={}", out[0]);
let y_size = 8 * 2; let y_size = 8 * 2;
let u = out[y_size]; let u = out[y_size];

25
lightningbeam-ui/lightningbeam-editor/src/export/mod.rs
View File

@ -57,6 +57,9 @@ pub struct VideoExportState {
hdr: lightningbeam_core::export::HdrExportMode, hdr: lightningbeam_core::export::HdrExportMode,
/// How the document is fit into the export frame (stretch/letterbox/crop). /// How the document is fit into the export frame (stretch/letterbox/crop).
fit: lightningbeam_core::export::ExportFitMode, fit: lightningbeam_core::export::ExportFitMode,
/// SDR color range: true = full (PC, 0–255), false = limited (TV, 16–235). The YUV
/// converters and the encoder color tag must agree on this.
full_range: bool,
/// Channel to send rendered frames to encoder thread /// Channel to send rendered frames to encoder thread
frame_tx: Option<Sender<VideoFrameMessage>>, frame_tx: Option<Sender<VideoFrameMessage>>,
/// HDR GPU resources for compositing pipeline (effects, color conversion) /// HDR GPU resources for compositing pipeline (effects, color conversion)
@ -868,6 +871,7 @@ impl ExportOrchestrator {
) -> (std::thread::JoinHandle<()>, VideoExportState) { ) -> (std::thread::JoinHandle<()>, VideoExportState) {
let hdr = settings.hdr; let hdr = settings.hdr;
let fit = settings.fit; let fit = settings.fit;
let full_range = settings.color_range.is_full();
let handle = std::thread::spawn(move || { let handle = std::thread::spawn(move || {
Self::run_video_encoder(settings, output_path, frame_rx, progress_tx, cancel_flag, total_frames); Self::run_video_encoder(settings, output_path, frame_rx, progress_tx, cancel_flag, total_frames);
}); });
@ -882,6 +886,7 @@ impl ExportOrchestrator {
height, height,
hdr, hdr,
fit, fit,
full_range,
frame_tx: Some(frame_tx), frame_tx: Some(frame_tx),
gpu_resources: None, gpu_resources: None,
readback_pipeline: None, readback_pipeline: None,
@ -1333,8 +1338,8 @@ impl ExportOrchestrator {
if !gpu_yuv_tight { if !gpu_yuv_tight {
println!("🎬 [VIDEO EXPORT] YUV planes are padded at {width}x{height}; using CPU YUV path"); println!("🎬 [VIDEO EXPORT] YUV planes are padded at {width}x{height}; using CPU YUV path");
} }
state.readback_pipeline = Some(readback_pipeline::ReadbackPipeline::new(device, queue, width, height, gpu_yuv_tight)); state.readback_pipeline = Some(readback_pipeline::ReadbackPipeline::new(device, queue, width, height, gpu_yuv_tight, state.full_range));
state.cpu_yuv_converter = Some(cpu_yuv_converter::CpuYuvConverter::new(width, height)?); state.cpu_yuv_converter = Some(cpu_yuv_converter::CpuYuvConverter::new(width, height, state.full_range)?);
println!("🚀 [ASYNC PIPELINE] Triple-buffered pipeline initialized"); println!("🚀 [ASYNC PIPELINE] Triple-buffered pipeline initialized");
println!("🚀 [CPU YUV] swscale converter initialized"); println!("🚀 [CPU YUV] swscale converter initialized");
} }
@ -1638,6 +1643,21 @@ impl ExportOrchestrator {
// Convert codec enum to FFmpeg codec ID. HDR requires 10-bit HEVC (Main10), so force HEVC // Convert codec enum to FFmpeg codec ID. HDR requires 10-bit HEVC (Main10), so force HEVC
// regardless of the chosen codec when an HDR mode is selected. // regardless of the chosen codec when an HDR mode is selected.
let codec_id = if settings.hdr.is_hdr() { let codec_id = if settings.hdr.is_hdr() {
// HEVC can only be muxed into MP4/MOV, not WebM — reject the incompatible combo up
// front with a clear message instead of letting the muxer fail cryptically.
if settings.codec.container_format() == "webm" {
return Err(format!(
"HDR export needs H.265/HEVC in an MP4 container, but {} uses WebM. \
Pick H.265 (or H.264) for HDR.",
settings.codec.name()
));
}
if !matches!(settings.codec, VideoCodec::H265) {
println!(
"⚠️ [ENCODER] HDR selected: overriding codec {} → H.265/HEVC (Main10)",
settings.codec.name()
);
}
ffmpeg_next::codec::Id::HEVC ffmpeg_next::codec::Id::HEVC
} else { } else {
match settings.codec { match settings.codec {
@ -1690,6 +1710,7 @@ impl ExportOrchestrator {
framerate, framerate,
bitrate_kbps, bitrate_kbps,
settings.hdr, settings.hdr,
settings.color_range.is_full(),
)?; )?;
// Pixel format the encoder frames are built in (matches setup_video_encoder). // Pixel format the encoder frames are built in (matches setup_video_encoder).

4
lightningbeam-ui/lightningbeam-editor/src/export/readback_pipeline.rs
View File

@ -91,12 +91,12 @@ impl ReadbackPipeline {
/// `enable_gpu_yuv` should be `true` only when the caller has verified the encoder's /// `enable_gpu_yuv` should be `true` only when the caller has verified the encoder's
/// `YUV420P` plane strides are tight (== width / width-2), so the packed GPU planes /// `YUV420P` plane strides are tight (== width / width-2), so the packed GPU planes
/// drop straight into the `AVFrame` without row misalignment. /// drop straight into the `AVFrame` without row misalignment.
pub fn new(device: &wgpu::Device, queue: &wgpu::Queue, width: u32, height: u32, enable_gpu_yuv: bool) -> Self { pub fn new(device: &wgpu::Device, queue: &wgpu::Queue, width: u32, height: u32, enable_gpu_yuv: bool, full_range: bool) -> Self {
let (readback_tx, readback_rx) = channel(); let (readback_tx, readback_rx) = channel();
// GPU YUV conversion when enabled AND the dimensions fit the packed shader; else RGBA + CPU. // GPU YUV conversion when enabled AND the dimensions fit the packed shader; else RGBA + CPU.
let gpu_yuv = if enable_gpu_yuv && super::gpu_yuv::supports(width, height) { let gpu_yuv = if enable_gpu_yuv && super::gpu_yuv::supports(width, height) {
Some(super::gpu_yuv::GpuYuv::new(device)) Some(super::gpu_yuv::GpuYuv::new(device, full_range))
} else { } else {
None None
}; };

233
lightningbeam-ui/lightningbeam-editor/src/export/video_exporter.rs
View File

@ -560,6 +560,7 @@ pub fn setup_video_encoder(
framerate: f64, framerate: f64,
bitrate_kbps: u32, bitrate_kbps: u32,
hdr: lightningbeam_core::export::HdrExportMode, hdr: lightningbeam_core::export::HdrExportMode,
full_range: bool,
) -> Result<(ffmpeg::encoder::Video, ffmpeg::Codec), String> { ) -> Result<(ffmpeg::encoder::Video, ffmpeg::Codec), String> {
// Try to find codec by ID first // Try to find codec by ID first
println!("🔍 Looking for codec: {:?}", codec_id); println!("🔍 Looking for codec: {:?}", codec_id);
@ -641,7 +642,12 @@ pub fn setup_video_encoder(
color_opts.set("profile", "main10"); color_opts.set("profile", "main10");
} else { } else {
encoder.set_colorspace(ffmpeg::color::Space::BT709); encoder.set_colorspace(ffmpeg::color::Space::BT709);
encoder.set_color_range(ffmpeg::color::Range::JPEG); // full range // Range must match what the YUV converters (gpu_yuv / cpu_yuv) actually produce.
encoder.set_color_range(if full_range {
ffmpeg::color::Range::JPEG // full (PC, 0–255)
} else {
ffmpeg::color::Range::MPEG // limited (TV, 16–235)
});
color_opts.set("color_primaries", "bt709"); color_opts.set("color_primaries", "bt709");
color_opts.set("color_trc", "bt709"); color_opts.set("color_trc", "bt709");
} }
@ -1133,232 +1139,9 @@ fn upload_transient_texture(
tex tex
} }
/// Render a document frame using the HDR compositing pipeline with effects
///
/// This function uses the same rendering pipeline as the stage preview,
/// ensuring effects are applied correctly during export.
///
/// # Arguments
/// * `document` - Document to render (current_time will be modified)
/// * `timestamp` - Time in seconds to render at
/// * `width` - Frame width in pixels
/// * `height` - Frame height in pixels
/// * `device` - wgpu device
/// * `queue` - wgpu queue
/// * `renderer` - Vello renderer
/// * `image_cache` - Image cache for rendering
/// * `video_manager` - Video manager for video clips
/// * `gpu_resources` - HDR GPU resources for compositing
///
/// # Returns
/// Ok((y_plane, u_plane, v_plane)) with YUV420p planes on success, Err with message on failure
pub fn render_frame_to_rgba_hdr(
document: &mut Document,
timestamp: f64,
width: u32,
height: u32,
device: &wgpu::Device,
queue: &wgpu::Queue,
renderer: &mut vello::Renderer,
image_cache: &mut ImageCache,
video_manager: &Arc<std::sync::Mutex<VideoManager>>,
gpu_resources: &mut ExportGpuResources,
) -> Result<(Vec<u8>, Vec<u8>, Vec<u8>), String> {
use vello::kurbo::Affine;
// Set document time to the frame timestamp
document.current_time = timestamp;
// Scale the document to the export resolution. The core renderer bakes this
// base transform into every layer (vector scenes, raster and video layer
// transforms), so the whole stage scales up/down to fill the output. When the
// export size matches the document this is the identity.
let base_transform = if document.width > 0.0 && document.height > 0.0 {
Affine::scale_non_uniform(
width as f64 / document.width,
height as f64 / document.height,
)
} else {
Affine::IDENTITY
};
// Export composites on the encoder's own device, not the shared one, so it cannot use
// hardware-decoded GPU frames (textures can't cross devices). Force software frames; a
// hardware decoder downloads its surface to CPU instead.
if let Ok(mut vm) = video_manager.lock() {
vm.set_render_hardware_ok(false);
}
// Render document for compositing (returns per-layer scenes)
let composite_result = render_document_for_compositing(
document,
base_transform,
image_cache,
video_manager,
None, // No webcam during export
None, // No floating selection during export
false, // No checkerboard in export
);
// Video export is never transparent.
composite_document_to_hdr(&composite_result, document, device, queue, renderer, gpu_resources, width, height, false)?;
// Use persistent output texture (already created in ExportGpuResources)
let output_view = &gpu_resources.output_texture_view;
// Convert HDR to sRGB for output
let bind_group = device.create_bind_group(&wgpu::BindGroupDescriptor {
label: Some("export_linear_to_srgb_bind_group"),
layout: &gpu_resources.linear_to_srgb_bind_group_layout,
entries: &[
wgpu::BindGroupEntry {
binding: 0,
resource: wgpu::BindingResource::TextureView(&gpu_resources.hdr_texture_view),
},
wgpu::BindGroupEntry {
binding: 1,
resource: wgpu::BindingResource::Sampler(&gpu_resources.linear_to_srgb_sampler),
},
],
});
let mut encoder = device.create_command_encoder(&wgpu::CommandEncoderDescriptor {
label: Some("export_linear_to_srgb_encoder"),
});
{
let mut render_pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some("export_linear_to_srgb_pass"),
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
view: &output_view,
resolve_target: None,
ops: wgpu::Operations {
load: wgpu::LoadOp::Clear(wgpu::Color::BLACK),
store: wgpu::StoreOp::Store,
},
depth_slice: None,
})],
depth_stencil_attachment: None,
occlusion_query_set: None,
timestamp_writes: None,
});
let final_pipeline = match document.hdr_output_mode {
lightningbeam_core::document::HdrOutputMode::HighlightRolloff => &gpu_resources.linear_to_srgb_pipeline_rolloff,
lightningbeam_core::document::HdrOutputMode::Clip => &gpu_resources.linear_to_srgb_pipeline,
};
render_pass.set_pipeline(final_pipeline);
render_pass.set_bind_group(0, &bind_group, &[]);
render_pass.draw(0..4, 0..1);
}
queue.submit(Some(encoder.finish()));
// GPU YUV conversion: Convert RGBA output to YUV420p
let mut yuv_encoder = device.create_command_encoder(&wgpu::CommandEncoderDescriptor {
label: Some("export_yuv_conversion_encoder"),
});
gpu_resources.yuv_converter.convert(
device,
&mut yuv_encoder,
output_view,
&gpu_resources.yuv_texture_view,
width,
height,
);
// Copy YUV texture to persistent staging buffer
let yuv_height = height + height / 2; // Y plane + U plane + V plane
yuv_encoder.copy_texture_to_buffer(
wgpu::TexelCopyTextureInfo {
texture: &gpu_resources.yuv_texture,
mip_level: 0,
origin: wgpu::Origin3d::ZERO,
aspect: wgpu::TextureAspect::All,
},
wgpu::TexelCopyBufferInfo {
buffer: &gpu_resources.staging_buffer,
layout: wgpu::TexelCopyBufferLayout {
offset: 0,
bytes_per_row: Some(width * 4), // Rgba8Unorm = 4 bytes per pixel
rows_per_image: Some(yuv_height),
},
},
wgpu::Extent3d {
width,
height: yuv_height,
depth_or_array_layers: 1,
},
);
queue.submit(Some(yuv_encoder.finish()));
// Map buffer and read YUV pixels (synchronous)
let buffer_slice = gpu_resources.staging_buffer.slice(..);
let (sender, receiver) = std::sync::mpsc::channel();
buffer_slice.map_async(wgpu::MapMode::Read, move |result| {
sender.send(result).ok();
});
let _ = device.poll(wgpu::PollType::wait_indefinitely());
receiver
.recv()
.map_err(|_| "Failed to receive buffer mapping result")?
.map_err(|e| format!("Failed to map buffer: {:?}", e))?;
// Extract Y, U, V planes from packed YUV buffer
let data = buffer_slice.get_mapped_range();
let width_usize = width as usize;
let height_usize = height as usize;
// Y plane: rows 0 to height-1 (extract R channel from Rgba8Unorm)
let y_plane_size = width_usize * height_usize;
let mut y_plane = vec![0u8; y_plane_size];
for y in 0..height_usize {
let src_row_offset = y * width_usize * 4; // 4 bytes per pixel (Rgba8Unorm)
let dst_row_offset = y * width_usize;
for x in 0..width_usize {
y_plane[dst_row_offset + x] = data[src_row_offset + x * 4]; // Extract R channel
}
}
// U and V planes: rows height to height + height/2 - 1 (half resolution, side-by-side layout)
// U plane is in left half (columns 0 to width/2-1), V plane is in right half (columns width/2 to width-1)
let chroma_width = width_usize / 2;
let chroma_height = height_usize / 2;
let chroma_row_start = height_usize * width_usize * 4; // Start of chroma rows in bytes
let mut u_plane = vec![0u8; chroma_width * chroma_height];
let mut v_plane = vec![0u8; chroma_width * chroma_height];
for y in 0..chroma_height {
let row_offset = chroma_row_start + y * width_usize * 4; // Full width rows in chroma region
// Extract U plane (left half: columns 0 to chroma_width-1)
let u_start = row_offset;
let dst_offset = y * chroma_width;
for x in 0..chroma_width {
u_plane[dst_offset + x] = data[u_start + x * 4]; // Extract R channel
}
// Extract V plane (right half: columns width/2 to width/2+chroma_width-1)
let v_start = row_offset + chroma_width * 4;
for x in 0..chroma_width {
v_plane[dst_offset + x] = data[v_start + x * 4]; // Extract R channel
}
}
drop(data);
gpu_resources.staging_buffer.unmap();
Ok((y_plane, u_plane, v_plane))
}
/// Render frame to GPU RGBA texture (non-blocking, for async pipeline) /// Render frame to GPU RGBA texture (non-blocking, for async pipeline)
/// ///
/// Similar to render_frame_to_rgba_hdr but renders to an external RGBA texture view /// Renders to an external RGBA texture view
/// (provided by ReadbackPipeline) and returns the command encoder WITHOUT blocking on readback. /// (provided by ReadbackPipeline) and returns the command encoder WITHOUT blocking on readback.
/// The caller (ReadbackPipeline) will submit the encoder and handle async readback. /// The caller (ReadbackPipeline) will submit the encoder and handle async readback.
/// ///

16
lightningbeam-ui/lightningbeam-editor/src/gpu_brush.rs
View File

@ -1627,12 +1627,24 @@ impl GpuBrushEngine {
self.proxy_layer_cache.get(kf_id) self.proxy_layer_cache.get(kf_id)
} }
/// Remove the cached texture for a raster layer keyframe (e.g. when deleted). /// Remove the cached texture for a raster layer keyframe (e.g. when deleted or edited).
pub fn remove_layer_texture(&mut self, kf_id: &Uuid) { pub fn remove_layer_texture(&mut self, kf_id: &Uuid) {
if self.raster_layer_cache.remove(kf_id).is_some() { let mut changed = self.raster_layer_cache.remove(kf_id).is_some();
if changed {
if let Some(pos) = self.raster_layer_lru.iter().position(|id| id == kf_id) { if let Some(pos) = self.raster_layer_lru.iter().position(|id| id == kf_id) {
self.raster_layer_lru.remove(pos); self.raster_layer_lru.remove(pos);
} }
}
// Also drop the low-res proxy: proxies are uploaded once and never refreshed, so a
// stale pre-edit proxy left here would be blitted (flashing old content) if the full-res
// texture is later evicted before the edited pixels page back in.
if self.proxy_layer_cache.remove(kf_id).is_some() {
if let Some(pos) = self.proxy_layer_lru.iter().position(|id| id == kf_id) {
self.proxy_layer_lru.remove(pos);
}
changed = true;
}
if changed {
self.report_raster_cache_vram(); self.report_raster_cache_vram();
} }
} }

11
lightningbeam-ui/lightningbeam-editor/src/main.rs
View File

@ -2392,6 +2392,12 @@ impl EditorApp {
kf.needs_fault_in = false; kf.needs_fault_in = false;
} }
} }
// Track these resident pixels in the LRU so they count toward
// RASTER_RESIDENT_MAX and can be evicted later; without this, a frame
// faulted in for undo/redo that ends up clean would stay resident forever,
// letting resident RAM grow past the cap.
self.raster_resident_lru.retain(|id| *id != kf_id);
self.raster_resident_lru.push_back(kf_id);
} }
} }
} }
@ -4523,7 +4529,9 @@ impl EditorApp {
let bytes = match std::fs::read(path) { let bytes = match std::fs::read(path) {
Ok(b) => b, Ok(b) => b,
Err(e) => { Err(e) => {
eprintln!("❌ Failed to read SVG {}: {}", path.display(), e); let msg = format!("Failed to read SVG: {}", e);
eprintln!("❌ {} ({})", msg, path.display());
notifications::notify_error("SVG Import Failed", &msg);
return; return;
} }
}; };
@ -4532,6 +4540,7 @@ impl EditorApp {
Ok(g) => g, Ok(g) => g,
Err(e) => { Err(e) => {
eprintln!("❌ {}", e); eprintln!("❌ {}", e);
notifications::notify_error("SVG Import Failed", &e);
return; return;
} }
}; };

7
lightningbeam-ui/lightningbeam-editor/src/notifications.rs
View File

@ -32,6 +32,11 @@ pub fn notify_export_complete(output_path: &Path) {
/// Show a desktop notification for an export error (fire-and-forget). /// Show a desktop notification for an export error (fire-and-forget).
pub fn notify_export_error(error_message: &str) { pub fn notify_export_error(error_message: &str) {
notify_error("Export Failed", error_message);
}
/// Show a desktop error notification with a custom title (fire-and-forget).
pub fn notify_error(title: &'static str, error_message: &str) {
// Truncate very long error messages (on a char boundary). // Truncate very long error messages (on a char boundary).
let truncated = if error_message.chars().count() > 100 { let truncated = if error_message.chars().count() > 100 {
let prefix: String = error_message.chars().take(97).collect(); let prefix: String = error_message.chars().take(97).collect();
@ -42,7 +47,7 @@ pub fn notify_export_error(error_message: &str) {
std::thread::spawn(move || { std::thread::spawn(move || {
if let Err(e) = Notification::new() if let Err(e) = Notification::new()
.summary("Export Failed") .summary(title)
.body(&truncated) .body(&truncated)
.icon("dialog-error") // Standard error icon .icon("dialog-error") // Standard error icon
.timeout(10000) // 10 seconds for errors (longer to read) .timeout(10000) // 10 seconds for errors (longer to read)

10
lightningbeam-ui/lightningbeam-editor/src/panes/stage.rs
View File

@ -2080,7 +2080,11 @@ impl egui_wgpu::CallbackTrait for VelloCallback {
// fills the black's gaps → interlocking black/yellow marching-ants. // fills the black's gaps → interlocking black/yellow marching-ants.
if let Some(active_id) = self.ctx.active_layer_id { if let Some(active_id) = self.ctx.active_layer_id {
if let Some(lightningbeam_core::layer::AnyLayer::Raster(rl)) = self.ctx.document.get_layer(&active_id) { if let Some(lightningbeam_core::layer::AnyLayer::Raster(rl)) = self.ctx.document.get_layer(&active_id) {
if let Some(kf) = rl.keyframe_at(self.ctx.document.current_time) { // Use playback_time (clip-local when editing a movie clip) like every other
// keyframe lookup in prepare(), and overlay_transform so the outline tracks the
// layer's pixels through any clip-instance affine (it equals camera_transform
// outside clip-edit mode).
if let Some(kf) = rl.keyframe_at(self.ctx.playback_time) {
let rect = vello::kurbo::Rect::new(0.0, 0.0, kf.width as f64, kf.height as f64); let rect = vello::kurbo::Rect::new(0.0, 0.0, kf.width as f64, kf.height as f64);
// Sizes are in document space; divide by zoom so they're ~constant on screen. // Sizes are in document space; divide by zoom so they're ~constant on screen.
let inv_zoom = 1.0 / (self.ctx.zoom as f64).max(1e-3); let inv_zoom = 1.0 / (self.ctx.zoom as f64).max(1e-3);
@ -2089,14 +2093,14 @@ impl egui_wgpu::CallbackTrait for VelloCallback {
let pattern = [dash, dash]; let pattern = [dash, dash];
scene.stroke( scene.stroke(
&vello::kurbo::Stroke::new(stroke_w).with_dashes(0.0, pattern), &vello::kurbo::Stroke::new(stroke_w).with_dashes(0.0, pattern),
camera_transform, overlay_transform,
vello::peniko::Color::new([0.0, 0.0, 0.0, 1.0]), vello::peniko::Color::new([0.0, 0.0, 0.0, 1.0]),
None, None,
&rect, &rect,
); );
scene.stroke( scene.stroke(
&vello::kurbo::Stroke::new(stroke_w).with_dashes(dash, pattern), &vello::kurbo::Stroke::new(stroke_w).with_dashes(dash, pattern),
camera_transform, overlay_transform,
vello::peniko::Color::new([1.0, 0.85, 0.0, 1.0]), vello::peniko::Color::new([1.0, 0.85, 0.0, 1.0]),
None, None,
&rect, &rect,

19
lightningbeam-ui/lightningbeam-editor/src/svg_import.rs
View File

@ -118,10 +118,14 @@ fn convert_path(path: &usvg::Path, graph: &mut VectorGraph) {
slot.color = Some(ShapeColor::rgba(c.red, c.green, c.blue, opacity_u8(fill.opacity()))); slot.color = Some(ShapeColor::rgba(c.red, c.green, c.blue, opacity_u8(fill.opacity())));
} }
usvg::Paint::LinearGradient(g) => { usvg::Paint::LinearGradient(g) => {
slot.gradient_fill = Some(linear_gradient(g, ts)); let mut grad = linear_gradient(g, ts);
apply_fill_opacity(&mut grad, fill.opacity());
slot.gradient_fill = Some(grad);
} }
usvg::Paint::RadialGradient(g) => { usvg::Paint::RadialGradient(g) => {
slot.gradient_fill = Some(radial_gradient(g, ts)); let mut grad = radial_gradient(g, ts);
apply_fill_opacity(&mut grad, fill.opacity());
slot.gradient_fill = Some(grad);
} }
usvg::Paint::Pattern(_) => { usvg::Paint::Pattern(_) => {
// Patterns aren't representable yet — neutral gray so the shape stays visible. // Patterns aren't representable yet — neutral gray so the shape stays visible.
@ -217,6 +221,17 @@ fn radial_gradient(g: &usvg::RadialGradient, abs: usvg::Transform) -> ShapeGradi
} }
} }
/// Fold the path's `fill-opacity` into a gradient's stop alphas (SVG multiplies them).
fn apply_fill_opacity(grad: &mut ShapeGradient, op: usvg::Opacity) {
let f = op.get();
if f >= 1.0 {
return;
}
for s in &mut grad.stops {
s.color.a = (s.color.a as f32 * f).round().clamp(0.0, 255.0) as u8;
}
}
fn gradient_stops(base: &usvg::BaseGradient) -> Vec<GradientStop> { fn gradient_stops(base: &usvg::BaseGradient) -> Vec<GradientStop> {
base.stops() base.stops()
.iter() .iter()