Export correctness:
- Honor the user's color-range (Limited/Full) on the software encode path:
thread full_range through gpu_yuv (new shader range uniform), the CPU
swscale fallback (sws_setColorspaceDetails → BT.709 + range, fixing the
BT.601 hue/level shift on odd-width exports), and the encoder color tags.
- Reject HDR + WebM up front with a clear message and log the forced HEVC
override instead of producing an unplayable file.
- Delete dead render_frame_to_rgba_hdr (hardcoded Stretch; live HDR path
already honors the fit mode).
Decode/playback (video.rs):
- Drain the decoder at EOF (send_eof + flush) so the final B-frame-delayed
frames render instead of erroring; per-frame logic extracted to a helper.
- Missing-PTS frames continue monotonically rather than snapping to ts=0.
- Force exact thumbnail width so sub-128px sources aren't shown stretched.
Resource leaks (gpu-video-encoder):
- dmabuf import_raw: RAII guard frees the duped fd + partial VkImages/memory
on every error path.
- vaapi alloc: free device/frames-ctx/AVFrames on the unexpected-DRM path.
Data model / robustness:
- collapse_boundary_spikes requires a full curve reversal (all control
points) so it no longer deletes a real lens/sliver and drops the fill.
- Export audio spin-wait ignores a stale `finished` flag when a forward
seek is pending (was rendering silence over real audio).
- RasterDiff apply_before/after take current dims and skip on a post-resize
mismatch.
- beam_archive read_media_full caps the preallocation from untrusted total_len.
UI/visual:
- SVG export skips hidden layers/empty groups; import folds fill-opacity into
gradients and surfaces failures as a notification.
- Active raster-layer border uses playback_time + overlay_transform.
- gpu_brush remove_layer_texture also evicts the stale low-res proxy.
- ensure_raster_resident_for_undo registers faulted frames in the LRU so
resident RAM stays bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
R16Unorm/Rg16Unorm aren't renderable, but the import gave every plane texture
RENDER_ATTACHMENT/COLOR_ATTACHMENT usage (needed only for the encoder's RGBA→NV12
write path) — so create_view panicked on 10-bit decode ("R16Unorm is not
renderable"). Gate the render-target usage (vk COLOR_ATTACHMENT, hal COLOR_TARGET,
wgpu RENDER_ATTACHMENT) on 8-bit; 16-bit P010 planes are sample-only
(TEXTURE_BINDING/RESOURCE + COPY_SRC), which is all the decode path needs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
10/12-bit HDR decodes to P010-style VAAPI surfaces (16-bit planes), which the
8-bit NV12 import couldn't handle — real HDR clips fell back to software (losing
the HDR). Import them as R16/Rg16 plane textures:
- dmabuf::Nv12DmaBuf gains `ten_bit`; import_raw picks R16_UNORM/R16G16_UNORM (vk)
+ R16Unorm/Rg16Unorm (wgpu) when set, else the existing R8/Rg8.
- hw_video.rs detects 10-bit from the hw frames context sw_format (P010/P012/P016)
and sets it. The NV12→RGB shader is unchanged: it samples normalized floats, and
the limited/full de-quant lands at the same values regardless of bit depth.
- vk_device requests TEXTURE_FORMAT_16BIT_NORM when the adapter supports it (R16/
Rg16 need it); absent → 10-bit falls back to software, 8-bit unaffected.
Pairs with the PQ/HLG/BT.2020 colour math so HDR10/HLG clips now reach the GPU
path end-to-end. NV12 callers (encoder, decode primitive) pass ten_bit=false.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the dormant core HW-decode engine live for the preview path:
- hw_video.rs: editor's HwVideoImporter — maps a decoded VAAPI surface to a
DRM-PRIME DMA-BUF and imports it as wgpu NV12 plane textures on the *shared*
device (the only one with the import extensions). install() creates the VAAPI
device and injects it + the importer into the VideoManager.
- main.rs: track whether the shared device is actually in use; only then (Linux,
not LB_NO_SHARED_DEVICE) install hardware decode, using the CreationContext's
shared device + adapter.
- nv12_blit.rs + nv12_blit.wgsl: NV12 plane textures → BT.709 → sRGB-encoded →
linear, written straight into the Rgba16Float HDR layer (no CPU upload). Colour
math mirrors the software path so HW/SW video match; honours full_range.
- stage.rs: the preview Video arm branches on inst.gpu (NV12 blit) vs rgba_data
(existing upload+blit_straight); sets render_hardware_ok = !cpu_renderer so the
CPU fallback still gets software frames.
- video_exporter.rs: sets render_hardware_ok(false) before both compositing
passes — export composites on the encoder's separate device, so a hardware
decoder downloads to CPU instead (export stays software, correct).
- dmabuf.rs: imported plane textures now also carry SAMPLED/TEXTURE_BINDING so
they can be sampled by the NV12 blit (they were render-target-only); into_planes
hands the textures to the longer-lived GpuVideoFrame.
- video.rs: cache-key the GPU/CPU representation on want_gpu (HW-configured AND
render_hardware_ok) so software-only decode keeps a single cache entry.
Preview only this pass; export GPU-residency is the 3c-export follow-up. Untested
at runtime here (no GPU/display in container) — both crates compile.
import_raw now takes (&wgpu::Device, &wgpu::Adapter) and extracts the raw Vulkan
handles via as_hal, instead of taking the crate's own DrmDevice. This lets a
VAAPI surface be imported onto ANY DMA-BUF-import-capable wgpu device — the
encoder/decoder's own device today, and the editor's shared device (Stage 3a)
next, so hardware-decoded frames are usable by the preview compositor.
Encoder/decoder pass their DrmDevice's device+adapter; round-trip decode test
still passes.
ZeroCopyEncoder::new now takes an output path and writes a real container
(format inferred from the extension, e.g. .mp4): create an output format
context, add the h264 stream from the encoder, write header; encode_rgba
rescales each packet's ts and av_interleaved_write_frame's it; finish
flushes + writes the trailer + closes. Sets AV_CODEC_FLAG_GLOBAL_HEADER for
mp4/mov so SPS/PPS land in extradata. This lets the editor's existing
mux_video_and_audio consume the temp video file unchanged.
The zerocopy_encode test now writes a .mp4 and ffprobe-verifies the codec,
dimensions, and frame count. Also let wgpu own the imported plane-image
destruction via texture_from_raw drop callbacks (clears two warnings).
Build the full end-to-end zero-copy encoder, validated on Intel/VAAPI:
- render_nv12: fragment-shader RGBA->NV12 that renders luma/chroma into
the imported R8/RG8 plane render targets (compute storage can't write
the DMA-BUF-backed planes; render attachments can).
- dmabuf: import_raw imports an NV12 DMA-BUF by explicit layout; the two
plane images + shared memory are now destroyed by wgpu via texture_from_raw
drop callbacks (Arc MemoryGuard frees the memory once both images are
gone, in wgpu's wait-idle'd deferred pass) -- fixes the teardown segfault.
- encoder::ZeroCopyEncoder: renders an RGBA texture straight into a pooled
VAAPI surface (imports cached by VASurface id) and encodes with h264_vaapi.
encode_rgba + finish; the caller renders on device().
Tests: real-frame render into the surface matches the CPU NV12 reference,
and a 30-frame encode produces valid H.264 (ffprobe-verified) with clean
teardown. Not yet wired into the editor.
New workspace crate isolating the unsafe GPU<->encoder interop for
zero-copy hardware video encoding. Every link is validated by a test on
real Intel/Mesa/iHD hardware:
- nv12: GPU RGBA->NV12 compute (BT.709 full-range), byte-exact vs a CPU
reference.
- vaapi: VAAPI hwcontext + h264_vaapi encode (CPU-fed NV12 -> valid H.264),
and DRM-PRIME surface layout probing.
- vk_device: a custom wgpu Vulkan device that adds
VK_EXT_image_drm_format_modifier (+ external-memory fd/dma-buf) via the
wgpu-hal device-from-raw path, so a tiled VAAPI surface can be imported.
- dmabuf: import a VAAPI NV12 surface's tiled DMA-BUF as two aliasing wgpu
textures (Y=R8, UV=RG8) at the plane offsets.
- zerocopy test: render values via Vulkan straight into the VAAPI surface
and read them back 100% correct -- proving the GPU writes into the
encoder surface with no CPU copy.
Not yet wired into the editor; real-frame render + encode-from-surface +
fallback wiring follow. Linux-only (libva); other platforms fall back.
Skyler Lehmkuhl