export: cache the static background + bilinear video, add render profiling

The export render bucket was dominated by re-rendering the static document
background through Vello every frame and by nearest-sampling the video on
upscale.

- Background cache: render the (static) background through Vello once, snapshot
  the composited HDR accumulator, and restore it with a single texture copy on
  every later frame instead of a Vello render + sRGB-convert + composite (+2
  submits). Invalidated on resize. background-render 3.6ms -> 0.56ms (1080p),
  7.5ms -> ~0.5ms (4K).
- blit_straight now uses the bilinear sampler — video frames are scaled to the
  output size, and nearest made that blocky. Fixes export and live preview.
- LB_RENDER_PROFILE: gated per-frame timing split (build/decode vs composite/
  upload vs srgb, and background vs layers) used to find all of the above. Kept
  as a debug aid for the remaining decode stages.

Net: export render ~12.8ms -> ~7.4ms/frame on a 1080p video clip.

lightningbeam-ui/GPU_VIDEO_DECODE_PLAN.md

@@ -0,0 +1,102 @@
+# GPU-resident video decode + dynamic decode resolution
+
+## Context
+
+Profiling the zero-copy H.264 export (single Group[Video, Audio] clip, `LB_RENDER_PROFILE=1`)
+broke the per-frame CPU "render" bucket down as:
+
+| Cost (ms/frame)         | 1080p | 4K    | What it is                                          |
+|-------------------------|-------|-------|-----------------------------------------------------|
+| decode                  | 3.1   | 19.0  | software ffmpeg decode (`video.rs::get_frame`)      |
+| background re-render     | 3.6   | 7.5   | static background pushed through Vello *every frame* |
+| video upload + blit      | 4.1   | 4.2   | per-frame transient texture alloc + `write_texture`  |
+| srgb                    | 0.4   | 0.4   | linear→sRGB pass                                     |
+
+The video correctly takes the GPU Video-instance path (not Vello-baked) — `LB_LAYER_DEBUG=1`
+shows `Video (1 instance)`. So the cost is **the video frame itself**: software decode, then an
+8 MB `write_texture` upload of the decoded RGBA every frame. At 4K, software decode (19 ms)
+dominates everything.
+
+### Two correctness problems found alongside the perf issue
+
+1. **Decode resolution is frozen to document size at import.** `load_video(clip, src, doc_w, doc_h)`
+   (`main.rs:4302`) sizes the decoder's swscale output to the document, capped to never upscale
+   (`video.rs:149`). Export *reuses that decoder*, so exporting **above** document resolution yields
+   video that was decoded to ≤document res and then GPU-**up**scaled — real source detail thrown away.
+2. **It can't follow the consumer or a document resize.** Preview wants small/fast frames; export
+   wants full res; changing the document size should re-target the decode. None of that works with a
+   size frozen at import.
+
+## Goal
+
+Decouple **decode resolution** from import/document size: the renderer requests a frame *at a target
+resolution*, and the decode path produces it. Hardware-decode H.264 (and later HEVC/AV1) into a GPU
+surface and keep it GPU-resident through composite into the encoder — no CPU frame copy in either
+direction. Software decode stays a **first-class** path (codecs/platforms without HW support), decoding
+at the requested target res. This fixes the 4K decode wall, the 8 MB upload, *and* the resolution bugs.
+
+## Design principles
+
+- **Decode native, scale to the consumer's target.**
+  - *Hardware path:* decode into a native VAAPI surface → import as a wgpu texture (reuse the
+    `gpu-video-encoder` `dmabuf.rs` / `vk_device.rs` plumbing, read direction) → the GPU blit that
+    already composites the Video instance scales native→target for free. Handles any target res and
+    document resizes inherently; the cached frame is a native GPU texture.
+  - *Software path:* decode native → `swscale` to the requested target (the reusable scaler is keyed
+    on input format/size **and** output size — rebuilt when the target changes). Preview requests
+    preview res (cheap); export requests export res (full quality).
+- **`VideoManager::get_frame` takes a target `(w, h)`** instead of relying on a frozen output size.
+  The frame cache is keyed to handle multiple live targets (preview + export) — either cache native
+  frames and scale on demand, or key by `(clip, ts, target)`; decide in Stage 2 by measuring cache
+  hit/scale tradeoff.
+- **Software is not optional.** Hardware decode is an acceleration of the same `get_frame` contract,
+  selected per source when the codec/driver supports it; everything falls back to software cleanly.
+
+## Approach (staged; each stage compiles + is independently useful)
+
+### Stage 0 — independent quick wins (not blocked on decode)
+- **Cache the static background** (`composite_document_to_hdr`): render once, reuse via a persistent
+  HDR texture (copy-in each frame) instead of a full Vello render + 2 passes/submits every frame.
+  Recovers ~3.6 ms (1080p) / ~7.5 ms (4K) per frame on *every* export. (In flight.)
+
+### Stage 1 — software: decode at the requested target res (testable; fixes the quality bug now)
+- Change `VideoManager::get_frame(clip, ts)` → `get_frame(clip, ts, target_w, target_h)`; thread the
+  target from the renderer (preview = current doc/preview res, export = export res). Cap at native.
+- Rework `VideoDecoder` so output size is per-request, not frozen at construction; cache the swscale
+  context per output size (already cached per stream — extend the key). Adjust the frame cache key.
+- Result: software exports are full-quality at any export res, and document resizes re-target decode.
+  No hardware needed; this is the correctness fix for the codecs HW can't handle anyway.
+
+### Stage 2 — hardware decode primitive (headless-testable here, like the 8 encode tests)
+- In `gpu-video-encoder` (rename → `gpu-video-codec`): `h264_vaapi`-style **decode** → VAAPI surface →
+  export DMA-BUF → import as a wgpu texture. Hardware test: decode a known file, verify dims/contents.
+
+### Stage 3 — wire hardware decode into `get_frame` (blind; user-verifies)
+- When the source codec/driver is HW-decodable, `get_frame` returns a **GPU texture** (native res)
+  instead of `Arc<Vec<u8>>`; the compositor uses it directly (no `write_texture`), GPU-scaling to the
+  target. For the zero-copy export the frame never leaves the GPU: **decode → composite → encode** on
+  one device. Software path is the fallback for everything else.
+
+## Critical files
+- `lightningbeam-core/src/video.rs` — `VideoDecoder` (per-request output size, scaler cache),
+  `VideoManager::get_frame` (target param, cache key).
+- `lightningbeam-core/src/renderer.rs` — pass the render target res into the video-instance build.
+- `lightningbeam-editor/src/export/video_exporter.rs` — background cache (Stage 0); consume a GPU
+  texture instead of uploading RGBA (Stage 3).
+- `gpu-video-encoder/` (→ `gpu-video-codec`) — `dmabuf.rs`/`vk_device.rs` reused for the decode import.
+
+## Risks
+- **Codec coverage** — only some codecs are HW-decodable per GPU/driver; software must stay correct
+  and well-tested. Selection must probe support per source, not assume.
+- **Cache memory** — native-res GPU textures (esp. 4K) are large; the frame cache budget needs revisiting.
+- **Colorspace/format** — VAAPI decode surfaces are NV12/tiled; the existing import handles NV12, but
+  10-bit/HDR sources (P010) need format handling.
+- **Preview vs export sharing** — two live targets (preview res + export res) from the same source; the
+  cache/scaler design must serve both without thrashing.
+
+## Verification
+- Stage 0/1: visual — export above document res is now full-quality (not upscaled); profile shows
+  background ≈ 0 and (Stage 1) software export correct at the chosen res.
+- Stage 2: headless hardware test in `gpu-video-codec` (decode → wgpu texture, ffprobe/byte checks).
+- Stage 3 (user): 1080p + 4K H.264 export — decode/upload buckets collapse; software fallback for a
+  non-HW codec (e.g. ProRes) still produces correct full-res output.

lightningbeam-ui/lightningbeam-editor/src/export/video_exporter.rs

@@ -83,6 +83,11 @@ pub struct ExportGpuResources {
     pub canvas_blit: crate::gpu_brush::CanvasBlitPipeline,
     /// Per-keyframe GPU texture cache for raster layers during export.
     pub raster_cache: std::collections::HashMap<uuid::Uuid, crate::gpu_brush::CanvasPair>,
+    /// Cached HDR accumulator state after the (static) background is composited in. The document
+    /// background doesn't change across an export, so it's rendered once and restored with a cheap
+    /// texture copy each frame instead of a full Vello render + 2 passes/submits. `None` until the
+    /// first frame; invalidated on resize.
+    cached_bg_hdr: Option<wgpu::Texture>,
 }
 
 impl ExportGpuResources {
@@ -108,7 +113,8 @@ impl ExportGpuResources {
             format: HDR_FORMAT,
             usage: wgpu::TextureUsages::RENDER_ATTACHMENT
                 | wgpu::TextureUsages::TEXTURE_BINDING
-                | wgpu::TextureUsages::COPY_SRC,
+                | wgpu::TextureUsages::COPY_SRC
+                | wgpu::TextureUsages::COPY_DST, // restore cached background each frame
             view_formats: &[],
         });
         let hdr_texture_view = hdr_texture.create_view(&wgpu::TextureViewDescriptor::default());
@@ -261,6 +267,7 @@ impl ExportGpuResources {
             linear_to_srgb_sampler,
             canvas_blit,
             raster_cache: std::collections::HashMap::new(),
+            cached_bg_hdr: None,
         }
     }
 
@@ -279,10 +286,12 @@ impl ExportGpuResources {
             format: HDR_FORMAT,
             usage: wgpu::TextureUsages::RENDER_ATTACHMENT
                 | wgpu::TextureUsages::TEXTURE_BINDING
-                | wgpu::TextureUsages::COPY_SRC,
+                | wgpu::TextureUsages::COPY_SRC
+                | wgpu::TextureUsages::COPY_DST,
             view_formats: &[],
         });
         self.hdr_texture_view = self.hdr_texture.create_view(&wgpu::TextureViewDescriptor::default());
+        self.cached_bg_hdr = None; // dimensions changed — rebuild the background cache
     }
 }
 
@@ -748,29 +757,74 @@ fn composite_document_to_hdr(
         antialiasing_method: vello::AaConfig::Area,
     };
 
-    // --- Background ---
+    let prof = render_profile_enabled();
-    let bg_srgb = gpu_resources.buffer_pool.acquire(device, layer_spec);
+    let t_c0 = std::time::Instant::now();
-    let bg_hdr  = gpu_resources.buffer_pool.acquire(device, hdr_spec);
+
-    if let (Some(bg_srgb_view), Some(bg_hdr_view)) = (
+    // --- Background (cached) ---
-        gpu_resources.buffer_pool.get_view(bg_srgb),
+    // The document background is static across an export, so render it through Vello exactly once
-        gpu_resources.buffer_pool.get_view(bg_hdr),
+    // (into the accumulator) and snapshot the result; every later frame restores it with a single
-    ) {
+    // GPU texture copy instead of a Vello render + sRGB-convert + composite (+2 submits).
-        renderer.render_to_texture(device, queue, &composite_result.background, bg_srgb_view, &layer_render_params)
+    let bg_cached = matches!(
-            .map_err(|e| format!("Failed to render background: {e}"))?;
+        &gpu_resources.cached_bg_hdr,
-        let mut enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("export_bg_srgb_to_linear") });
+        Some(t) if t.width() == width && t.height() == height
-        gpu_resources.srgb_to_linear.convert(device, &mut enc, bg_srgb_view, bg_hdr_view);
+    );
+    let copy_size = wgpu::Extent3d { width, height, depth_or_array_layers: 1 };
+    if bg_cached {
+        // Restore the cached background into the accumulator.
+        let cached = gpu_resources.cached_bg_hdr.as_ref().unwrap();
+        let mut enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("export_bg_restore") });
+        enc.copy_texture_to_texture(
+            wgpu::TexelCopyTextureInfo { texture: cached, mip_level: 0, origin: wgpu::Origin3d::ZERO, aspect: wgpu::TextureAspect::All },
+            wgpu::TexelCopyTextureInfo { texture: &gpu_resources.hdr_texture, mip_level: 0, origin: wgpu::Origin3d::ZERO, aspect: wgpu::TextureAspect::All },
+            copy_size,
+        );
         queue.submit(Some(enc.finish()));
-        let bg_layer = CompositorLayer::normal(bg_hdr, 1.0);
+    } else {
-        let mut enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("export_bg_composite") });
+        // First frame (or after a resize): full background render into the accumulator.
-        // When transparency is allowed, start from transparent black so the background's
+        let bg_srgb = gpu_resources.buffer_pool.acquire(device, layer_spec);
-        // native alpha is preserved. Otherwise force an opaque black underlay.
+        let bg_hdr  = gpu_resources.buffer_pool.acquire(device, hdr_spec);
-        let clear = if allow_transparency { [0.0, 0.0, 0.0, 0.0] } else { [0.0, 0.0, 0.0, 1.0] };
+        if let (Some(bg_srgb_view), Some(bg_hdr_view)) = (
-        gpu_resources.compositor.composite(device, queue, &mut enc, &[bg_layer],
+            gpu_resources.buffer_pool.get_view(bg_srgb),
-            &gpu_resources.buffer_pool, &gpu_resources.hdr_texture_view, Some(clear));
+            gpu_resources.buffer_pool.get_view(bg_hdr),
+        ) {
+            renderer.render_to_texture(device, queue, &composite_result.background, bg_srgb_view, &layer_render_params)
+                .map_err(|e| format!("Failed to render background: {e}"))?;
+            let mut enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("export_bg_srgb_to_linear") });
+            gpu_resources.srgb_to_linear.convert(device, &mut enc, bg_srgb_view, bg_hdr_view);
+            queue.submit(Some(enc.finish()));
+            let bg_layer = CompositorLayer::normal(bg_hdr, 1.0);
+            let mut enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("export_bg_composite") });
+            // When transparency is allowed, start from transparent black so the background's
+            // native alpha is preserved. Otherwise force an opaque black underlay.
+            let clear = if allow_transparency { [0.0, 0.0, 0.0, 0.0] } else { [0.0, 0.0, 0.0, 1.0] };
+            gpu_resources.compositor.composite(device, queue, &mut enc, &[bg_layer],
+                &gpu_resources.buffer_pool, &gpu_resources.hdr_texture_view, Some(clear));
+            queue.submit(Some(enc.finish()));
+        }
+        gpu_resources.buffer_pool.release(bg_srgb);
+        gpu_resources.buffer_pool.release(bg_hdr);
+
+        // Snapshot the composited background for reuse on subsequent frames.
+        let cached = device.create_texture(&wgpu::TextureDescriptor {
+            label: Some("export_cached_bg_hdr"),
+            size: copy_size,
+            mip_level_count: 1,
+            sample_count: 1,
+            dimension: wgpu::TextureDimension::D2,
+            format: HDR_FORMAT,
+            usage: wgpu::TextureUsages::COPY_SRC | wgpu::TextureUsages::COPY_DST,
+            view_formats: &[],
+        });
+        let mut enc = device.create_command_encoder(&wgpu::CommandEncoderDescriptor { label: Some("export_bg_snapshot") });
+        enc.copy_texture_to_texture(
+            wgpu::TexelCopyTextureInfo { texture: &gpu_resources.hdr_texture, mip_level: 0, origin: wgpu::Origin3d::ZERO, aspect: wgpu::TextureAspect::All },
+            wgpu::TexelCopyTextureInfo { texture: &cached, mip_level: 0, origin: wgpu::Origin3d::ZERO, aspect: wgpu::TextureAspect::All },
+            copy_size,
+        );
         queue.submit(Some(enc.finish()));
+        gpu_resources.cached_bg_hdr = Some(cached);
     }
-    gpu_resources.buffer_pool.release(bg_srgb);
+    let t_bg = std::time::Instant::now();
-    gpu_resources.buffer_pool.release(bg_hdr);
 
     // --- Layers ---
     for rendered_layer in &composite_result.layers {
@@ -906,10 +960,33 @@ fn composite_document_to_hdr(
         }
     }
 
+    if prof {
+        record_composite_profile(t_bg.duration_since(t_c0), t_bg.elapsed());
+    }
+
     gpu_resources.buffer_pool.next_frame();
     Ok(())
 }
 
+/// Split of `composite_document_to_hdr`: static-background re-render vs. the layer loop
+/// (video upload + blits). Prints a running average every 200 frames under LB_RENDER_PROFILE.
+fn record_composite_profile(background: std::time::Duration, layers: std::time::Duration) {
+    use std::sync::atomic::{AtomicU64, Ordering};
+    static BG_US: AtomicU64 = AtomicU64::new(0);
+    static LAYERS_US: AtomicU64 = AtomicU64::new(0);
+    static N: AtomicU64 = AtomicU64::new(0);
+    BG_US.fetch_add(background.as_micros() as u64, Ordering::Relaxed);
+    LAYERS_US.fetch_add(layers.as_micros() as u64, Ordering::Relaxed);
+    let n = N.fetch_add(1, Ordering::Relaxed) + 1;
+    if n % 200 == 0 {
+        println!(
+            "📊 [COMPOSITE PROFILE] {n} frames avg: background-render {:.2}ms | layers(video upload+blit) {:.2}ms",
+            BG_US.load(Ordering::Relaxed) as f64 / n as f64 / 1000.0,
+            LAYERS_US.load(Ordering::Relaxed) as f64 / n as f64 / 1000.0,
+        );
+    }
+}
+
 /// Upload `pixels` to a transient GPU texture (TEXTURE_BINDING | COPY_DST) in the
 /// given format. Use `Rgba8UnormSrgb` to upload raw sRGB bytes and let the GPU
 /// decode to linear on sample (no CPU conversion).
@@ -1224,6 +1301,12 @@ pub fn render_frame_to_gpu_rgba(
 ) -> Result<wgpu::CommandEncoder, String> {
     use vello::kurbo::Affine;
 
+    // One-shot profiling of the render-bucket split (LB_RENDER_PROFILE=1): how much of the
+    // per-frame CPU "render" is document build (incl. video decode) vs. composite-command
+    // recording (incl. the frame texture upload) vs. the sRGB pass. Prints a running average.
+    let prof = render_profile_enabled();
+    let t0 = std::time::Instant::now();
+
     // Set document time to the frame timestamp
     document.current_time = timestamp;
 
@@ -1256,8 +1339,10 @@ pub fn render_frame_to_gpu_rgba(
         floating_selection,
         false, // No checkerboard in export
     );
+    let t_build = std::time::Instant::now();
 
     composite_document_to_hdr(&composite_result, document, device, queue, renderer, gpu_resources, width, height, allow_transparency)?;
+    let t_composite = std::time::Instant::now();
 
     // Convert HDR to sRGB (linear → sRGB), render directly to external RGBA texture
     let output_view = rgba_texture_view;
@@ -1302,11 +1387,49 @@ pub fn render_frame_to_gpu_rgba(
         render_pass.draw(0..4, 0..1);
     }
 
+    if prof {
+        record_render_profile(
+            t_build.duration_since(t0),
+            t_composite.duration_since(t_build),
+            t_composite.elapsed(),
+        );
+    }
+
     // Return encoder for caller to submit (ReadbackPipeline will handle submission and async readback)
     // Frame is already rendered to external RGBA texture, no GPU YUV conversion needed
     Ok(encoder)
 }
 
+/// `LB_RENDER_PROFILE` gate, checked once.
+fn render_profile_enabled() -> bool {
+    static V: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
+    *V.get_or_init(|| std::env::var("LB_RENDER_PROFILE").is_ok())
+}
+
+/// Accumulate the per-frame render split and print a running average every 200 frames.
+/// `build` = document build incl. video decode; `composite` = composite-command recording
+/// incl. the frame texture upload; `srgb` = the linear→sRGB pass.
+fn record_render_profile(build: std::time::Duration, composite: std::time::Duration, srgb: std::time::Duration) {
+    use std::sync::atomic::{AtomicU64, Ordering};
+    static BUILD_US: AtomicU64 = AtomicU64::new(0);
+    static COMPOSITE_US: AtomicU64 = AtomicU64::new(0);
+    static SRGB_US: AtomicU64 = AtomicU64::new(0);
+    static N: AtomicU64 = AtomicU64::new(0);
+    BUILD_US.fetch_add(build.as_micros() as u64, Ordering::Relaxed);
+    COMPOSITE_US.fetch_add(composite.as_micros() as u64, Ordering::Relaxed);
+    SRGB_US.fetch_add(srgb.as_micros() as u64, Ordering::Relaxed);
+    let n = N.fetch_add(1, Ordering::Relaxed) + 1;
+    if n % 200 == 0 {
+        let (b, c, s) = (BUILD_US.load(Ordering::Relaxed), COMPOSITE_US.load(Ordering::Relaxed), SRGB_US.load(Ordering::Relaxed));
+        println!(
+            "📊 [RENDER PROFILE] {n} frames avg: build(+decode) {:.2}ms | composite(+upload) {:.2}ms | srgb {:.2}ms",
+            b as f64 / n as f64 / 1000.0,
+            c as f64 / n as f64 / 1000.0,
+            s as f64 / n as f64 / 1000.0,
+        );
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;

lightningbeam-ui/lightningbeam-editor/src/gpu_brush.rs

@@ -2238,6 +2238,8 @@ impl CanvasBlitPipeline {
     /// Blit a **straight-alpha** source (e.g. a video frame uploaded to an
     /// `Rgba8UnormSrgb` texture, hardware-decoded to linear on sample). Uses the
     /// `fs_main_straight` pipeline, which skips the unpremultiply that `blit` does.
+    /// Bilinear-sampled: video frames are scaled to the output size (document→export, or any
+    /// non-1:1 transform), and nearest sampling makes that look blocky.
     pub fn blit_straight(
         &self,
         device:      &wgpu::Device,
@@ -2247,7 +2249,7 @@ impl CanvasBlitPipeline {
         transform:   &BlitTransform,
         mask_view:   Option<&wgpu::TextureView>,
     ) {
-        self.blit_with(device, queue, canvas_view, target_view, transform, mask_view, &self.sampler, &self.pipeline_straight);
+        self.blit_with(device, queue, canvas_view, target_view, transform, mask_view, &self.linear_sampler, &self.pipeline_straight);
     }
 
     #[allow(clippy::too_many_arguments)]