Export correctness:
- Honor the user's color-range (Limited/Full) on the software encode path:
thread full_range through gpu_yuv (new shader range uniform), the CPU
swscale fallback (sws_setColorspaceDetails → BT.709 + range, fixing the
BT.601 hue/level shift on odd-width exports), and the encoder color tags.
- Reject HDR + WebM up front with a clear message and log the forced HEVC
override instead of producing an unplayable file.
- Delete dead render_frame_to_rgba_hdr (hardcoded Stretch; live HDR path
already honors the fit mode).
Decode/playback (video.rs):
- Drain the decoder at EOF (send_eof + flush) so the final B-frame-delayed
frames render instead of erroring; per-frame logic extracted to a helper.
- Missing-PTS frames continue monotonically rather than snapping to ts=0.
- Force exact thumbnail width so sub-128px sources aren't shown stretched.
Resource leaks (gpu-video-encoder):
- dmabuf import_raw: RAII guard frees the duped fd + partial VkImages/memory
on every error path.
- vaapi alloc: free device/frames-ctx/AVFrames on the unexpected-DRM path.
Data model / robustness:
- collapse_boundary_spikes requires a full curve reversal (all control
points) so it no longer deletes a real lens/sliver and drops the fill.
- Export audio spin-wait ignores a stale `finished` flag when a forward
seek is pending (was rendering silence over real audio).
- RasterDiff apply_before/after take current dims and skip on a post-resize
mismatch.
- beam_archive read_media_full caps the preallocation from untrusted total_len.
UI/visual:
- SVG export skips hidden layers/empty groups; import folds fill-opacity into
gradients and surfaces failures as a notification.
- Active raster-layer border uses playback_time + overlay_transform.
- gpu_brush remove_layer_texture also evicts the stale low-res proxy.
- ensure_raster_resident_for_undo registers faulted frames in the LRU so
resident RAM stays bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Export (lightningbeam-core/svg_export.rs): document_to_svg walks vector
layers/groups at a given frame and emits <path> per fill (solid or
gradient via <defs>) and per stroked edge. Wired into the export dialog
as an "SVG" tab; written synchronously. Raster/video/effect layers are
skipped (vector-only, lossless), structured for a later rasterize pass.
Import (lightningbeam-editor/svg_import.rs): import_svg parses via usvg,
bakes each path's absolute transform into geometry, converts segments to
cubic edges, and maps solid/linear/radial paint to ShapeColor/
ShapeGradient. .svg is detected in the Ctrl+I Import handler and added as
a new vector layer (keyframe at the playhead). file_types gains
FileType::Vector + VECTOR_EXTENSIONS.
Tests: svg_export::export_tests (core) and svg_import::tests (editor).
ImageExportSettings gains `fit: ExportFitMode` (default Letterbox) + a "Fit"
dropdown in advanced image settings; the image render path uses it instead of the
hardcoded Letterbox. Image export supports a resolution override, so the mode
matters when the override aspect differs from the document.
A) Imported video had a different aspect ratio depending on how it was placed:
direct import used a uniform scale + center (preserve aspect), while the
asset-library timeline drag used an independent scale_x/scale_y (stretch to fill).
Add Transform::fit_centered (uniform scale, centered, aspect-preserving) and route
both paths through it, so a clip looks identical however it's added.
B) Exported video was stretched when the export resolution's aspect differed from
the document's (base_transform was always scale_non_uniform). Add
ExportFitMode {Stretch, Letterbox (default), Crop} on VideoExportSettings + a "Fit"
dropdown in advanced export settings, and a shared export_base_transform() helper:
Letterbox = uniform fit centered (black bars), Crop = uniform fill centered (trim),
Stretch = the old distort-to-fill. Threaded through the software, HDR, and
zero-copy export render paths (image export is doc-sized → identity).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Makes the common Linux H.264 export fully GPU-resident: decode (HW NV12) →
composite → VAAPI encode all on one device, no CPU round-trips.
- gpu-video-encoder: ZeroCopyEncoder now holds wgpu (device,queue,adapter) handles
instead of owning a DrmDevice. New `new_on_device(device,queue,adapter,...)` runs
the RGBA→NV12 render + DMA-BUF import on a passed device; `new` keeps building its
own. The encoder only *imports* VAAPI surfaces (not export), so the shared
import-capable device works directly.
- editor: stash the shared device handles on EditorApp (set in the creation closure
from the eframe render_state when the shared device is active) and thread them
through start_video_export / start_video_with_audio_export → try_build_zero_copy,
which uses new_on_device when available. ZeroCopyVideo carries on_shared_device →
the export composite uses hardware_ok=true (consuming HW-decoded GPU frames from
pt 1) only then.
Falls back to the own-device encoder (decode downloads to CPU) when no shared
device. Tradeoff: the export now shares the GPU with the UI thread (was a separate
device); acceptable for the GPU-resident-decode win. Compiles; crate tests build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The software and HDR export paths already composite on the shared device (the
eframe device, via render_next_video_frame), so they can consume the same
hardware-decoded NV12 GPU frames the preview uses — no CPU download/upload:
- ExportGpuResources gains the Nv12BlitPipeline; the export compositor's Video arm
branches on inst.gpu (NV12 blit, mirroring stage.rs) vs the RGBA upload path.
- render_frame_to_gpu_rgba takes a hardware_ok flag: true for the software video +
image export (shared device), false for the zero-copy path (own device, must
download). render_frame_to_yuv10_hdr sets it true.
Drops the 4K software/HDR-export decode wall (HW decode + GPU composite, no
per-frame RGBA upload). The common Linux H.264 zero-copy path is unchanged
(separate device) — that's pt 2. Falls back to software when no shared device /
importer (flag is harmless then: get_frame returns CPU anyway). Compiles.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire HDR frame production end-to-end (encoder side was pt 1):
- linear_to_pq.wgsl: linear scene HDR (BT.709, white=1.0) → BT.2020 primaries →
PQ (203-nit) or HLG OETF → gamma-encoded R'G'B'. Inverse of the nv12 decode, so
a decode→encode round-trip is the identity.
- hdr_frame.rs (isolated module): runs that pass into an Rgba16Float target, does a
synchronous readback (f16-decoded on CPU), then BT.2020 R'G'B'→Y'CbCr limited
4:2:0 10-bit pack → YUV420P10LE planes. Rgba16Float avoids the 16BIT_NORM device
feature dependency.
- video_exporter::render_frame_to_yuv10_hdr: composite + HDR encode, returning
10-bit planes; lazily builds the pipeline.
- orchestrator: HDR uses a synchronous 1-frame-per-call path (the async RGBA
pipeline is 8-bit only); zero-copy is skipped for HDR.
- dialog: "Dynamic range" dropdown (SDR / HDR10 PQ / HLG).
Software HEVC Main10 path; favors correctness over throughput. Compiles; runtime
verification needs a GPU + HDR-capable player.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Settings + encoder side of HDR video export (frame data still SDR until pt 2):
- core: HdrExportMode {Sdr (default), Pq, Hlg} on VideoExportSettings (serde
default), with transfer-name helpers.
- setup_video_encoder takes the mode: HDR → YUV420P10LE, BT.2020 NCL matrix,
limited range, color_primaries=bt2020, color_trc=smpte2084/arib-std-b67,
profile=main10. SDR path unchanged (8-bit full-range BT.709).
- run_video_encoder forces HEVC when HDR is selected (the only 10-bit codec wired
up). encode_frame is parameterized by pixel format and now copies planes
row-by-row honoring stride (10-bit / non-aligned widths can have row padding).
Dormant: no UI exposes the mode yet and the frame data is still 8-bit SDR, so
selecting HDR is not yet wired. The render→PQ/HLG 10-bit frame path is pt 2.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a per-document HDR→SDR mapping applied at the final linear→sRGB encode, so
super-white (HDR) video highlights can be recovered instead of hard-clipped:
- core: HdrOutputMode {Clip (default), HighlightRolloff} on Document (serde
default), with a SetDocumentPropertiesAction variant for undoable edits.
- shaders: a fs_main_rolloff entry point (preview linear_to_srgb.wgsl + the export
inline shader) applying a C1 highlight knee — identity below 0.8, smooth rolloff
[0.8,∞)→[0.8,1). SDR below the knee is untouched; Clip stays the historical path.
- preview (stage.rs) and both export encodes (video_exporter.rs) pick the pipeline
variant from document.hdr_output_mode — one value per frame, so no per-pixel
uniform; mirrors the existing fs_main_straight pattern.
- UI: an "HDR output" dropdown in the Document section of the info panel.
Default (Clip) is bit-identical to previous behaviour. Completes Stage A:
HDR-correct input (pt 1) + SDR-safe output mapping. HDR export (10-bit P010/PQ)
and HDR display remain Stages B/C.
Make the dormant core HW-decode engine live for the preview path:
- hw_video.rs: editor's HwVideoImporter — maps a decoded VAAPI surface to a
DRM-PRIME DMA-BUF and imports it as wgpu NV12 plane textures on the *shared*
device (the only one with the import extensions). install() creates the VAAPI
device and injects it + the importer into the VideoManager.
- main.rs: track whether the shared device is actually in use; only then (Linux,
not LB_NO_SHARED_DEVICE) install hardware decode, using the CreationContext's
shared device + adapter.
- nv12_blit.rs + nv12_blit.wgsl: NV12 plane textures → BT.709 → sRGB-encoded →
linear, written straight into the Rgba16Float HDR layer (no CPU upload). Colour
math mirrors the software path so HW/SW video match; honours full_range.
- stage.rs: the preview Video arm branches on inst.gpu (NV12 blit) vs rgba_data
(existing upload+blit_straight); sets render_hardware_ok = !cpu_renderer so the
CPU fallback still gets software frames.
- video_exporter.rs: sets render_hardware_ok(false) before both compositing
passes — export composites on the encoder's separate device, so a hardware
decoder downloads to CPU instead (export stays software, correct).
- dmabuf.rs: imported plane textures now also carry SAMPLED/TEXTURE_BINDING so
they can be sampled by the NV12 blit (they were render-target-only); into_planes
hands the textures to the longer-lived GpuVideoFrame.
- video.rs: cache-key the GPU/CPU representation on want_gpu (HW-configured AND
render_hardware_ok) so software-only decode keeps a single cache entry.
Preview only this pass; export GPU-residency is the 3c-export follow-up. Untested
at runtime here (no GPU/display in container) — both crates compile.
The zero-copy VAAPI encoder emitted full-range BT.709 NV12 but wrote no color
tags, so players assumed limited range and stretched it — the H.264 output
looked dark and oversaturated (preview and VP9/software were fine).
- Rgba2Nv12 takes `full_range` and applies the matching Y/chroma scale+offset
(limited 16-235 / full 0-255) via a uniform; the encoder sets color_range +
bt709 colorspace/primaries/transfer tags to match. ffprobe-confirmed.
- New ColorRange { Limited, Full } on VideoExportSettings (default Limited, the
universally-correct choice; serde(default) so saved settings still load),
surfaced as a "Color range" dropdown in advanced export settings for H.264.
The swscale software fallback still emits Limited regardless of the toggle
(Full only affects the VAAPI zero-copy path).
The export render bucket was dominated by re-rendering the static document
background through Vello every frame and by nearest-sampling the video on
upscale.
- Background cache: render the (static) background through Vello once, snapshot
the composited HDR accumulator, and restore it with a single texture copy on
every later frame instead of a Vello render + sRGB-convert + composite (+2
submits). Invalidated on resize. background-render 3.6ms -> 0.56ms (1080p),
7.5ms -> ~0.5ms (4K).
- blit_straight now uses the bilinear sampler — video frames are scaled to the
output size, and nearest made that blocky. Fixes export and live preview.
- LB_RENDER_PROFILE: gated per-frame timing split (build/decode vs composite/
upload vs srgb, and background vs layers) used to find all of the above. Kept
as a debug aid for the remaining decode stages.
Net: export render ~12.8ms -> ~7.4ms/frame on a 1080p video clip.
gpu_video_encoder's encoder/vaapi/vk_device/dmabuf modules are #[cfg(linux)]
(VAAPI is Linux-only), but the editor referenced them unconditionally, so
macOS/Windows failed to compile (cannot find `encoder` in `gpu_video_encoder`).
Gate the zero-copy path (ZeroCopyVideo, try_build_zero_copy,
run_zerocopy_video_export, and the branches in start_video_export /
start_video_with_audio_export) behind cfg(target_os = "linux"). Factor the
software encoder-thread spawn into spawn_software_video so both the None arm
and the non-Linux path share it without duplication. Non-Linux always uses
the software path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Video-only (no-audio) H.264 export now uses the same background-thread zero-copy
VAAPI path as video+audio, instead of the software encoder. It writes the output
.mp4 directly (no mux step) and reports through the single-export progress
channel; completion clears that channel so the dialog closes cleanly.
Factored the encoder/renderer/resources setup shared by both entry points into
ExportOrchestrator::try_build_zero_copy. start_video_export gains the document/
video_manager/raster_store/container_path inputs to seed the off-thread render
from a Document snapshot. Non-H264 / non-VAAPI still falls back to software.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The zero-copy VAAPI export previously rendered one frame per egui repaint on
the UI thread, which pinned throughput to the 60Hz vsync of the present loop
(measured exactly 16ms/frame) -- so the near-free hardware encode bought
nothing. Because the export runs on its own VAAPI device (independent of
eframe), it can run entirely off the UI thread.
- run_zerocopy_video_export: a background thread owning the encoder + its own
vello renderer/device, a Document snapshot (Document is Clone+Send; the UI
keeps the live one), its own ImageCache, and a RasterStore clone. Renders +
hardware-encodes every frame and reports through the same video_progress
channel the software encoder thread uses.
- start_video_with_audio_export takes the document/video_manager/raster_store/
container_path to seed the thread; video_state is None for this path.
- Throttle export-time UI repaints (~6Hz) and the thread's progress sends so
the render thread keeps the cores; the breakdown print stays.
- cancel() tears down parallel_export (detaches threads, removes temp files)
so the progress dialog dismisses; the call site closes the dialog.
- Gate the progress poll loop on has_pending_progress() so it stops once the
export ends instead of polling/logging every repaint forever; the single
export path clears its channel on the terminal event.
Vsync overhead is gone (0.1ms/frame); export is now render-bound (~11ms/frame
Vello scene-build). ~1:50 -> ~56s (~2x) on the validation clip.
When exporting H.264 with audio, try the gpu-video-encoder ZeroCopyEncoder:
render each frame to RGBA and hardware-encode it into a VAAPI surface
inline, on the encoder's own VAAPI-capable wgpu device — no GPU->CPU
readback, no swscale, no software-encoder thread. Falls back to the
existing software path verbatim when VAAPI/the device is unavailable
(non-Linux, non-H264, or init failure), so it's additive.
- VideoExportState gains zero_copy: Option<ZeroCopyVideo> (encoder + its own
vello renderer + ExportGpuResources + a reused RGBA target, all on the
encoder's device).
- start_video_with_audio_export builds it for H.264 and skips spawning the
software encoder thread when present.
- render_next_video_frame routes to a zero-copy arm that reuses
render_frame_to_gpu_rgba on the encoder's device, then encode_rgba; on the
last frame finish() writes the temp .mp4 and sets video_progress=Complete
so the existing mux runs. video_thread=None makes the mux join a no-op.
Separate export device (vs modifying the eframe device) keeps this contained
to export. Video-only export stays on the software path for now. Runtime
verification (an actual H.264 export) is pending — cannot run the editor in
the dev container.
The suite had accumulated breakage from prior refactors:
- selection unification: rewrite the integration tests for the single
unified clip_instances collection (shapes+clips are one set now).
- tempo-map: thread a TempoMap (constant 60 BPM = identity) into the
clip remap_time tests so the second-based expectations hold.
- drop two dead rgba_to_yuv420p tests that asserted tight plane sizes
incompatible with the function's 16-macroblock alignment.
- ignore the WIP theme var() cascade test (theme system not wired up).
Also a real bug the tests caught: auto_key_ranges produced overlapping
sample key ranges (the midpoint key mapped to both adjacent samples).
Start each range one past the previous midpoint.
The export read back 8MB RGBA per frame and ran swscale RGBA->YUV420p on
the UI thread (~6ms/frame). Add a tight GPU compute converter (gpu_yuv,
BT.709 full-range matching the encoder tags) and wire it into the
triple-buffered ReadbackPipeline: render to RGBA, convert on the GPU, read
back ~3MB of planar YUV, and skip the CPU pass. Gated on a runtime check
that the encoder's YUV420P plane strides are tight (no linesize padding),
with the swscale path as fallback for other dimensions; LB_DISABLE_GPU_YUV
forces the CPU path. Includes a CPU reference + unit tests for the packing.
Also guard render_next_video_frame against re-initializing/re-emitting
"Complete" every frame after the render finishes while the encoder/mux
drains (the completion nulled gpu_resources but left video_state set).
Imported video is a Group[Video, Audio] that rendered as a Vello-baked
Vector layer, re-uploading the full frame to Vello's image atlas every
frame (~17ms/frame at 1080p, hitting playback and export alike). Extract
video frames out of the Group/clip scene recursion into
VideoRenderInstances so they composite via the GPU Video path; mixed
video+vector containers fall back to Vello (correct, unaccelerated).
Also route video through hardware sRGB decode: upload raw sRGB bytes to an
Rgba8UnormSrgb texture and blit with a non-unpremultiplying shader variant
(blit_straight), removing the per-frame per-pixel CPU sRGB->linear pass.
Add an F3 GPU-timestamp timer and a per-frame video texture cache.
Drops the live composite of a 1080p video from ~17ms to ~2-3ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve all compiler warnings across daw-backend, lightningbeam-core, and
lightningbeam-editor:
- Delete dead code: the superseded CPU raster tools in raster_tool.rs
(EffectBrush/Smudge/Gradient/Transform/Warp/Liquify/Selection — replaced by
the GPU path), plus orphaned helpers and never-read struct fields.
- Mechanical fixes: drop unused imports/variables/mut, underscore unused params,
`drop(&x)` -> `let _ = x`, deprecated egui::Rounding -> CornerRadius, snake_case
rename, elided-lifetime Cow<'_, [u8]>.
- Keep the WIP CSS theming system (theme.rs/theme_render.rs) under
#[allow(dead_code)] rather than deleting it.
Editor checks warning-free; 293 core tests pass.
CpuYuvConverter::convert rebuilt the swscale context + both ffmpeg frames on every
call (per output frame), despite the converter persisting for the whole export. Now
the scaler + reusable source/dest frames are built once in new() and reused each
convert(). (convert is &mut self; the caller already held it mutably.)
Raster keyframes are no longer eagerly decoded at load — `raw_pixels` stays empty
and is paged in on demand from the project container, so a big paint project opens
instantly and only touched frames hit RAM.
- core: `read_packed_media_readonly` (fresh read-only connection, can't conflict
with an in-place save) + `RasterStore` (holds the container path; `load_pixels`
reads+decodes a keyframe's PNG by id). `load_beam_sqlite` stops eager-decoding and
instead marks every raster keyframe `needs_fault_in` (recursively, incl. nested);
a freshly-created keyframe stays false (blank-resident, nothing to page). Added
`Document::all_layers_mut`.
- editor: the canvas records a fault-in request when it needs a paged-out keyframe
(empty pixels && needs_fault_in); the App drains the sink at the top of update(),
pages the pixels in via the store, clears the flag, and repaints. Store path is set
on load and after save. Export faults in synchronously per frame.
Cold-scrub still shows a 1-frame gap and the page-in is synchronous; the image proxy
(3a-2) and async load (3a-3) remove those next.
- VideoManager.frame_cache: unbounded HashMap (grew per distinct frame during
playback) -> LruCache evicted by a 256MB byte budget. Byte-budget rather than
frame count is robust across resolutions (a 4K frame is ~33MB vs ~2MB at
800x600). unload_video pops per-clip keys (LruCache has no retain).
- mux_video_and_audio: stream-merge the two inputs by PTS with one pending
packet per stream (O(1) memory) instead of collecting every packet into Vecs
first (O(duration)). Output is byte-identical.
- export AAC: sanitize the planar-f32 path (non-finite -> 0, finite clamped to
[-1,1]) like the integer paths, with a one-time warning. A stray NaN/Inf
render sample no longer fails the whole export.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Scale the document to the selected output resolution. It was rendered
at document size regardless of the export dimensions, so picking a
different resolution didn't scale the stage.
- Run the audio+video mux on a background thread instead of the UI
thread, keeping the app responsive (showing "Finalizing") during the
re-mux pass.
- Send desktop notifications fire-and-forget. notify_rust's show() is a
synchronous D-Bus call that blocked the UI for the full service
activation timeout (~25s) when no notification daemon is running.
Color correctness:
- Unpremultiply before the sRGB OETF on the display and export blits;
encoding premultiplied color corrupted antialiased/transparent edges.
- Tag exported video as full-range BT.709 (matrix/primaries/transfer).
- Run perception effects (invert, brightness/contrast, hue/saturation)
in gamma space to match standard editors.
- Interpolate gradients in gamma space across the raster and vector paths.
- Render effect thumbnails in the same linear space as the live pipeline.
Brush performance:
- Store the raster canvas as Rgba16Float (no shadow banding from 8-bit
linear), with an incremental per-tile ping-pong sync replacing the
per-frame full-canvas copy.
- Do the linear->sRGB readback conversion on the GPU and reuse a cached
scratch texture, dropping a ~110ms-per-stroke CPU decode.
Cleanup:
- Single COLOR_WGSL prelude and shared CPU sRGB scalars instead of ~8
duplicated copies of the transfer functions.
- Shared compute-pipeline builder; smudge folded onto the tile-sync path.
Skyler Lehmkuhl