import_raw now takes (&wgpu::Device, &wgpu::Adapter) and extracts the raw Vulkan
handles via as_hal, instead of taking the crate's own DrmDevice. This lets a
VAAPI surface be imported onto ANY DMA-BUF-import-capable wgpu device — the
encoder/decoder's own device today, and the editor's shared device (Stage 3a)
next, so hardware-decoded frames are usable by the preview compositor.
Encoder/decoder pass their DrmDevice's device+adapter; round-trip decode test
still passes.
Foundation for hardware video decode in BOTH preview and export: wgpu textures
can't cross devices, and a hardware-decoded frame is a DMA-BUF-imported texture
that needs the import extensions (only addable via wgpu-hal device_from_raw). So
eframe + the compositor + decode + encode must share ONE custom device.
- vk_device::create_windowed(): the existing import-capable DrmDevice plus
VK_KHR_swapchain (the WSI surface instance extensions are already enabled by
Instance::init), for use as the editor's main device.
- main.rs: on Linux, build the shared device and inject it into eframe via
WgpuSetup::Existing (the egui fork supports it); fall back to wgpu's normal
device + software decode on any failure, on other platforms, or via
LB_NO_SHARED_DEVICE. The DrmDevice's wgpu handles are cloned into eframe
(Arc-backed), so the VkDevice persists with them.
Runtime-verified: the editor launches and renders identically (canvas/vello,
video, panels) on the shared device, and the env-var fallback works.
Headless decode primitive, the mirror of the encoder: VaapiDecoder opens a file,
hardware-decodes H.264 into VAAPI NV12 surfaces (hw_device_ctx + a get_format
callback selecting AV_PIX_FMT_VAAPI), maps each surface to a DRM-PRIME DMA-BUF,
and imports it as two wgpu plane textures via the existing dmabuf::import_raw —
the same path the encoder uses, in the read direction. Frames stay GPU-resident
(no CPU copy).
Validated by a round-trip test: encode solid gray with ZeroCopyEncoder, decode
it back, read the Y plane (mean ≈ 128). All 9 crate tests pass on the container's
Intel GPU.
Next (Stage 3): wire this into VideoManager/get_frame so the compositor consumes
a GPU texture directly (no write_texture upload), with software decode fallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The zero-copy VAAPI encoder emitted full-range BT.709 NV12 but wrote no color
tags, so players assumed limited range and stretched it — the H.264 output
looked dark and oversaturated (preview and VP9/software were fine).
- Rgba2Nv12 takes `full_range` and applies the matching Y/chroma scale+offset
(limited 16-235 / full 0-255) via a uniform; the encoder sets color_range +
bt709 colorspace/primaries/transfer tags to match. ffprobe-confirmed.
- New ColorRange { Limited, Full } on VideoExportSettings (default Limited, the
universally-correct choice; serde(default) so saved settings still load),
surfaced as a "Color range" dropdown in advanced export settings for H.264.
The swscale software fallback still emits Limited regardless of the toggle
(Full only affects the VAAPI zero-copy path).
The decoder's output size was frozen to the document size at import, and export
reused that decoder — so exporting above document res upscaled the video (real
source detail discarded) and a document resize never re-targeted the decode.
Decode size is now chosen per get_frame call: VideoDecoder::get_frame and
VideoManager::get_frame take a target (w, h), capped to native (never upscale),
with the swscale context and frame caches keyed on the output size so preview
(preview res) and an in-progress export (export res) don't collide. The renderer
derives the target from the document->output base_transform, so export decodes
at export res (full detail) and the canvas at preview res. Thumbnails/asset
library pass small targets.
The export render bucket was dominated by re-rendering the static document
background through Vello every frame and by nearest-sampling the video on
upscale.
- Background cache: render the (static) background through Vello once, snapshot
the composited HDR accumulator, and restore it with a single texture copy on
every later frame instead of a Vello render + sRGB-convert + composite (+2
submits). Invalidated on resize. background-render 3.6ms -> 0.56ms (1080p),
7.5ms -> ~0.5ms (4K).
- blit_straight now uses the bilinear sampler — video frames are scaled to the
output size, and nearest made that blocky. Fixes export and live preview.
- LB_RENDER_PROFILE: gated per-frame timing split (build/decode vs composite/
upload vs srgb, and background vs layers) used to find all of the above. Kept
as a debug aid for the remaining decode stages.
Net: export render ~12.8ms -> ~7.4ms/frame on a 1080p video clip.
get_frame rebuilt the RGBA swscale context on every decoded frame and printed
[Video Timing] lines unconditionally. A stream's frames share one input
format/size, so build the scaler once (keyed on format+dims, rebuilt only if
they change) and reuse it; gate the per-frame traces behind LB_VIDEO_DEBUG.
Cuts export wall time ~10% on a 1080p video clip (the scaler rebuild was the
bulk of the per-frame "scale" cost; it's now ~0ms). SwsContext is !Send, so the
cached scaler is wrapped in a SendScaler — sound because a VideoDecoder is only
ever touched under the VideoManager mutex (same invariant as its decoder/input).
gpu_video_encoder's encoder/vaapi/vk_device/dmabuf modules are #[cfg(linux)]
(VAAPI is Linux-only), but the editor referenced them unconditionally, so
macOS/Windows failed to compile (cannot find `encoder` in `gpu_video_encoder`).
Gate the zero-copy path (ZeroCopyVideo, try_build_zero_copy,
run_zerocopy_video_export, and the branches in start_video_export /
start_video_with_audio_export) behind cfg(target_os = "linux"). Factor the
software encoder-thread spawn into spawn_software_video so both the None arm
and the non-Linux path share it without duplication. Non-Linux always uses
the software path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Video-only (no-audio) H.264 export now uses the same background-thread zero-copy
VAAPI path as video+audio, instead of the software encoder. It writes the output
.mp4 directly (no mux step) and reports through the single-export progress
channel; completion clears that channel so the dialog closes cleanly.
Factored the encoder/renderer/resources setup shared by both entry points into
ExportOrchestrator::try_build_zero_copy. start_video_export gains the document/
video_manager/raster_store/container_path inputs to seed the off-thread render
from a Document snapshot. Non-H264 / non-VAAPI still falls back to software.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Linux release ffmpeg is built statically from source, and ffmpeg only
autodetects --enable-vaapi when libva headers are present at configure time.
Neither build path installed them, so every release shipped a static ffmpeg
with no h264_vaapi encoder -- the zero-copy export silently fell back to
software 100% of the time.
- Containerfile + build.yml: install libva-dev + libdrm-dev so the from-source
ffmpeg gets VAAPI.
- build.yml: assert the release binary links libva, so a missing dep can't
silently regress to a software-only build again.
- deb/rpm: libva (libva2/libva-drm2/libdrm2, libva/libdrm) is a hard runtime
dep -- the vaapi-enabled ffmpeg DT_NEEDEDs it, so the app won't launch
without it. The VA driver is a soft recommends (absent it, export falls back
to software): va-driver-all (deb), intel-media-driver/mesa-va-drivers (rpm).
build.rs needs no change: releases link ffmpeg statically (its bundling path
is skipped) and the AppImage is thin, taking libva from the host like libvulkan.
The zero-copy VAAPI export previously rendered one frame per egui repaint on
the UI thread, which pinned throughput to the 60Hz vsync of the present loop
(measured exactly 16ms/frame) -- so the near-free hardware encode bought
nothing. Because the export runs on its own VAAPI device (independent of
eframe), it can run entirely off the UI thread.
- run_zerocopy_video_export: a background thread owning the encoder + its own
vello renderer/device, a Document snapshot (Document is Clone+Send; the UI
keeps the live one), its own ImageCache, and a RasterStore clone. Renders +
hardware-encodes every frame and reports through the same video_progress
channel the software encoder thread uses.
- start_video_with_audio_export takes the document/video_manager/raster_store/
container_path to seed the thread; video_state is None for this path.
- Throttle export-time UI repaints (~6Hz) and the thread's progress sends so
the render thread keeps the cores; the breakdown print stays.
- cancel() tears down parallel_export (detaches threads, removes temp files)
so the progress dialog dismisses; the call site closes the dialog.
- Gate the progress poll loop on has_pending_progress() so it stops once the
export ends instead of polling/logging every repaint forever; the single
export path clears its channel on the terminal event.
Vsync overhead is gone (0.1ms/frame); export is now render-bound (~11ms/frame
Vello scene-build). ~1:50 -> ~56s (~2x) on the validation clip.
Three fixes found while running the zero-copy export on real Intel hardware:
- vaapi::create_device() retries LIBVA_DRIVER_NAME in order iHD -> auto ->
i965 -> radeonsi. libva was auto-selecting the legacy i965 driver, which
fails on newer Intel GPUs; the modern iHD (intel-media-driver) is needed.
encoder.rs now builds its hwdevice through this helper.
- vk_device: request the adapter's full limits instead of downlevel_defaults.
Vello's compute pipelines need max_storage_buffers_per_shader_stage >= 5
(downlevel caps at 4), which panicked Vello's shader init on the export
device. This device only ever runs on a real VAAPI GPU.
- ZeroCopyEncoder: unsafe impl Send. It owns its FFmpeg/Vulkan handles
exclusively and is only moved (onto the export thread), never shared.
When exporting H.264 with audio, try the gpu-video-encoder ZeroCopyEncoder:
render each frame to RGBA and hardware-encode it into a VAAPI surface
inline, on the encoder's own VAAPI-capable wgpu device — no GPU->CPU
readback, no swscale, no software-encoder thread. Falls back to the
existing software path verbatim when VAAPI/the device is unavailable
(non-Linux, non-H264, or init failure), so it's additive.
- VideoExportState gains zero_copy: Option<ZeroCopyVideo> (encoder + its own
vello renderer + ExportGpuResources + a reused RGBA target, all on the
encoder's device).
- start_video_with_audio_export builds it for H.264 and skips spawning the
software encoder thread when present.
- render_next_video_frame routes to a zero-copy arm that reuses
render_frame_to_gpu_rgba on the encoder's device, then encode_rgba; on the
last frame finish() writes the temp .mp4 and sets video_progress=Complete
so the existing mux runs. video_thread=None makes the mux join a no-op.
Separate export device (vs modifying the eframe device) keeps this contained
to export. Video-only export stays on the software path for now. Runtime
verification (an actual H.264 export) is pending — cannot run the editor in
the dev container.
ZeroCopyEncoder::new now takes an output path and writes a real container
(format inferred from the extension, e.g. .mp4): create an output format
context, add the h264 stream from the encoder, write header; encode_rgba
rescales each packet's ts and av_interleaved_write_frame's it; finish
flushes + writes the trailer + closes. Sets AV_CODEC_FLAG_GLOBAL_HEADER for
mp4/mov so SPS/PPS land in extradata. This lets the editor's existing
mux_video_and_audio consume the temp video file unchanged.
The zerocopy_encode test now writes a .mp4 and ffprobe-verifies the codec,
dimensions, and frame count. Also let wgpu own the imported plane-image
destruction via texture_from_raw drop callbacks (clears two warnings).
Build the full end-to-end zero-copy encoder, validated on Intel/VAAPI:
- render_nv12: fragment-shader RGBA->NV12 that renders luma/chroma into
the imported R8/RG8 plane render targets (compute storage can't write
the DMA-BUF-backed planes; render attachments can).
- dmabuf: import_raw imports an NV12 DMA-BUF by explicit layout; the two
plane images + shared memory are now destroyed by wgpu via texture_from_raw
drop callbacks (Arc MemoryGuard frees the memory once both images are
gone, in wgpu's wait-idle'd deferred pass) -- fixes the teardown segfault.
- encoder::ZeroCopyEncoder: renders an RGBA texture straight into a pooled
VAAPI surface (imports cached by VASurface id) and encodes with h264_vaapi.
encode_rgba + finish; the caller renders on device().
Tests: real-frame render into the surface matches the CPU NV12 reference,
and a 30-frame encode produces valid H.264 (ffprobe-verified) with clean
teardown. Not yet wired into the editor.
New workspace crate isolating the unsafe GPU<->encoder interop for
zero-copy hardware video encoding. Every link is validated by a test on
real Intel/Mesa/iHD hardware:
- nv12: GPU RGBA->NV12 compute (BT.709 full-range), byte-exact vs a CPU
reference.
- vaapi: VAAPI hwcontext + h264_vaapi encode (CPU-fed NV12 -> valid H.264),
and DRM-PRIME surface layout probing.
- vk_device: a custom wgpu Vulkan device that adds
VK_EXT_image_drm_format_modifier (+ external-memory fd/dma-buf) via the
wgpu-hal device-from-raw path, so a tiled VAAPI surface can be imported.
- dmabuf: import a VAAPI NV12 surface's tiled DMA-BUF as two aliasing wgpu
textures (Y=R8, UV=RG8) at the plane offsets.
- zerocopy test: render values via Vulkan straight into the VAAPI surface
and read them back 100% correct -- proving the GPU writes into the
encoder surface with no CPU copy.
Not yet wired into the editor; real-frame render + encode-from-surface +
fallback wiring follow. Linux-only (libva); other platforms fall back.
The suite had accumulated breakage from prior refactors:
- selection unification: rewrite the integration tests for the single
unified clip_instances collection (shapes+clips are one set now).
- tempo-map: thread a TempoMap (constant 60 BPM = identity) into the
clip remap_time tests so the second-based expectations hold.
- drop two dead rgba_to_yuv420p tests that asserted tight plane sizes
incompatible with the function's 16-macroblock alignment.
- ignore the WIP theme var() cascade test (theme system not wired up).
Also a real bug the tests caught: auto_key_ranges produced overlapping
sample key ranges (the midpoint key mapped to both adjacent samples).
Start each range one past the previous midpoint.
The export read back 8MB RGBA per frame and ran swscale RGBA->YUV420p on
the UI thread (~6ms/frame). Add a tight GPU compute converter (gpu_yuv,
BT.709 full-range matching the encoder tags) and wire it into the
triple-buffered ReadbackPipeline: render to RGBA, convert on the GPU, read
back ~3MB of planar YUV, and skip the CPU pass. Gated on a runtime check
that the encoder's YUV420P plane strides are tight (no linesize padding),
with the swscale path as fallback for other dimensions; LB_DISABLE_GPU_YUV
forces the CPU path. Includes a CPU reference + unit tests for the packing.
Also guard render_next_video_frame against re-initializing/re-emitting
"Complete" every frame after the render finishes while the encoder/mux
drains (the completion nulled gpu_resources but left video_state set).
Imported video is a Group[Video, Audio] that rendered as a Vello-baked
Vector layer, re-uploading the full frame to Vello's image atlas every
frame (~17ms/frame at 1080p, hitting playback and export alike). Extract
video frames out of the Group/clip scene recursion into
VideoRenderInstances so they composite via the GPU Video path; mixed
video+vector containers fall back to Vello (correct, unaccelerated).
Also route video through hardware sRGB decode: upload raw sRGB bytes to an
Rgba8UnormSrgb texture and blit with a non-unpremultiplying shader variant
(blit_straight), removing the per-frame per-pixel CPU sRGB->linear pass.
Add an F3 GPU-timestamp timer and a per-frame video texture cache.
Drops the live composite of a 1080p video from ~17ms to ~2-3ms.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Video was the only media type always kept external (VideoClip.file_path),
so a project with video wasn't self-contained. Now video packs into the
SQLite container under the same large-media policy as audio (pack < 2 GB
unless the user chose Reference), and both the frames and the embedded audio
track decode by streaming directly from the blob — no temp files.
- New crate ffmpeg-blob-io: an AVIOContext-over-Read+Seek shim (BlobInput)
that lets ffmpeg demux from an arbitrary byte source. Isolates all the
unsafe FFI + ffmpeg ABI coupling (version-pinned =8.0.0/=8.0.1). Manual
Drop teardown order; AVSEEK_SIZE restores the read position (FFmpeg assumes
a size query doesn't move it — required for MP4 moov-at-end).
- Schema/save/load: VideoClip.media_id; save_beam packs/references video as
MediaKind::Video (keyed by clip id); load resolves packed vs referenced and
reports missing sources. A packed clip points its linked video-audio pool
entry's media_id at the video row so the audio streams from the same blob.
- Frames: video.rs VideoSource{Path,Packed} threaded through new/seek/scan/
probe/thumbnails (a fresh BlobReader per open); editor builds the source
from current_file_path (now set before register_loaded_videos).
- Audio: VideoAudioReader::open_source via BlobInput; the disk_reader
StreamSource block on packed video-audio is removed; the engine's existing
factory activation routes it unchanged.
Tests: ffmpeg-blob-io AVIO unit tests (WAV via Cursor, seek, open/drop loop);
core packed_video_stream (blob->AVIO->Input) and beam_archive video round-trip;
daw-backend open_source test (compiles; links/runs only off-container).
Runtime-verified: a packed video plays frames + audio after the source file
is removed.
Address the code smells flagged in the .beam format spec:
- Write the project's actual sample rate on save instead of a hardcoded 48000
(add AudioProject::sample_rate()).
- Remove the vestigial RasterKeyframe.media_path field (it was only used by the
legacy ZIP loader, which now derives "media/raster/<id>.png" from the keyframe
id) and the dead buffer_path_at_time accessor. Backward-compatible: older files
carrying media_path deserialize fine (the field is ignored).
- Drop the unused SaveSettings fields auto_embed_threshold_bytes / force_embed_all
/ force_link_all; only large_media_mode was ever consulted. Un-prefix the now-used
`settings` parameter.
Update BEAM_FILE_FORMAT.md to match. The remaining notes (reserved MediaKind::Video,
exact-match version check) are design choices, left as-is.
The shipping product is the pure-Rust app in lightningbeam-ui/; the old
JavaScript UI in src/ is browser-only (window.__TAURI__ is shimmed by
src/tauri_polyfill.js), so the src-tauri Tauri backend is dead. It also no
longer compiled (mid-refactor: missing video_server module, handler list
referencing deleted commands), and CI only ever built lightningbeam-ui.
- Delete src-tauri/ entirely (commands, native renderer/render-window,
websocket frame streamers, Tauri config, generated schemas, icons).
- Relocate icon.icns into lightningbeam-ui/lightningbeam-editor/assets/icons/
(the only src-tauri asset still needed) and repoint the editor's window-icon
include_bytes! at assets/icons/256x256.png (was src-tauri/icons/icon.png).
- CI/packaging: drop the redundant icon-copy steps (PNGs are committed in the
editor assets) and point macOS bundling at the relocated icns.
- package.json: drop @tauri-apps/* deps and the tauri script.
- Docs/.gitignore: drop src-tauri references.
daw-backend/ (shared audio engine) and the top-level WASM lightningbeam-core/
are untouched. Editor builds clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolve all compiler warnings across daw-backend, lightningbeam-core, and
lightningbeam-editor:
- Delete dead code: the superseded CPU raster tools in raster_tool.rs
(EffectBrush/Smudge/Gradient/Transform/Warp/Liquify/Selection — replaced by
the GPU path), plus orphaned helpers and never-read struct fields.
- Mechanical fixes: drop unused imports/variables/mut, underscore unused params,
`drop(&x)` -> `let _ = x`, deprecated egui::Rounding -> CornerRadius, snake_case
rename, elided-lifetime Cow<'_, [u8]>.
- Keep the WIP CSS theming system (theme.rs/theme_render.rs) under
#[allow(dead_code)] rather than deleting it.
Editor checks warning-free; 293 core tests pass.
Block geometry/clip edits on frames that fall strictly inside a tween span,
where an edit would silently mutate the bracketing keyframe instead:
- Shape tweens: gate vector editing (vertices, curves, DCEL hits) and all
geometry tools in stage.rs behind VectorLayer::is_tween_inbetween.
- Motion tweens: block selecting/dragging/transforming clip instances whose
transform is mid-tween, via AnimationData::is_object_tweened_at.
Also: inserting a keyframe mid-tween now captures the interpolated geometry
shown at that frame (not the left keyframe's) and inherits the shape tween,
so the new keyframe continues morphing toward the right keyframe.
`tween_after == Shape` was stored on keyframes but never read. Now the
render path morphs geometry across a shape-tween span:
- VectorGraph::interpolated(other, t): same-topology lerp of vertex
positions, edge curves, stroke widths and stroke/fill colours. Returns
None when topology differs (counts, deleted flags, edge endpoints, fill
boundaries), so the caller holds the source keyframe.
- VectorLayer::tweened_graph_at(time): returns an owned morphed graph for
a shape-tween span whose two keyframes share topology, else borrows the
held keyframe. Editing still uses graph_at_time (the held keyframe).
- Renderer (Vello + CPU paths) renders via tweened_graph_at.
- SetTweenAction + wired the previously-stubbed "Add Shape Tween" menu.
The typical workflow — keyframe, duplicate it (same topology), move
vertices, Add Shape Tween — now morphs between the two. Non-matching
topology falls back to a hold.
Within a vector layer, groups and movie clips (clip instances) were drawn
before the layer's own VectorGraph, so they appeared underneath the loose
shapes. Draw the loose shapes first and the clip instances on top, in both
the Vello and CPU render paths.
Creating the first transform keyframe for a clip instance at frame N left
the curve with a single keyframe, which Hold-extrapolates backward — so
moving it at frame N also moved it on every earlier frame (frame 1).
When SetKeyframeAction creates a brand-new curve for a clip instance and
the clip already existed before `time`, also anchor a keyframe at the
clip's start (its group visibility start, or timeline_start for movie
clips) with the original value. Earlier frames now hold the original
position and the move produces a proper tween from start to N.
Also capture the clip instance's actual on-stage value when keying (its
base transform), instead of a generic 0/identity default, so a new
keyframe doesn't snap the clip to the origin.
Adds VectorLayer::group_visibility_start and tests covering the anchor and
the no-double-anchor case (keying at the clip's own start).
A dense self-intersecting freehand lasso leaves clusters of near-coincident
duplicate sub-pixel edges (split products the coincident-edge dedupe can't
reach). The planar face trace bounces back and forth across them, producing
a degenerate "spike" boundary (an edge used twice). Add
`collapse_boundary_spikes`, run on each traced face before it becomes a
fill: it removes consecutive out-and-back entries (where the boundary
returns to where it started) until the loop is simple.
Embed the captured region-select dumps as committed fixtures under
tests/region_dumps/ (replayed by `dumped_region_selects_are_valid`) so the
regression survives /tmp being cleared. dump3 is the boundary-spike repro;
it fails this test without the collapse and passes with it. Loosen the
boundary-connectivity test tolerance to 1e-2 (above sub-micropixel float
drift, far below any real gap).
Collapse the two parallel selection systems into one. The RegionSelect
tool (rect + lasso) now cuts the geometry along the region outline and
selects the resulting sub-pieces into the standard `Selection` ID-sets,
exactly like every other tool. The vestigial floating `RegionSelection`
(drag never wired; commit/delete/copy were stubbed) and all its plumbing
are removed, so Group, Convert-to-Movie-Clip, Delete, and Properties all
operate uniformly from lasso, rect, marquee, and click selections.
Region cutting is reworked onto a robust planar arrangement:
- Replace fragile incremental "split a fill by one cut edge" logic with
planar face re-tracing (`retrace_fills_after_cut` + `trace_faces`),
which correctly handles arbitrary holed/concave fills.
- `extract_subgraph` no longer frees vertices still referenced by kept
boundary edges (fixed Group leaving freed-but-referenced vertices that
a later alloc reused and corrupted).
- `split_fill_by_*` direction fix (was producing disconnected boundaries
rendered as stray diagonals).
- `fill_interior_point` (area-centroid + inward-step fallback) for
reliable inside/outside classification of non-convex pieces.
- Coincident-edge dedupe + degenerate-fill removal (edge-adjacent shapes
no longer make zero-area sliver fills).
- Dangling-edge pruning, near-coincident endpoint welding, induced-
subgraph expansion, and tracking of `split_edge` sub-edges, so
self-intersecting freehand lassos cut correctly.
Region-select capture is available behind LIGHTNINGBEAM_DUMP_REGION=1 for
turning a misbehaving cut into a deterministic test. Extensive regression
tests added in vector_graph/tests/region_cut_select.rs.
Both actions were DCEL-stubs (no-ops). They now extract the selected geometry from a
vector layer's active keyframe into a new VectorClip (group vs movie clip) and place a
ClipInstance in its place (identity transform → renders where the geometry was), which
the existing transform-animation system can motion-tween.
- Shared `clip_from_geometry::extract_geometry_to_clip` (+ undo) does the work; the
actions are thin wrappers. Undo snapshots the source graph + removes the clip/instance.
- `extract_subgraph` now DERIVES shared-fill boundary edges internally (an inside edge
still used by a non-extracted fill must be duplicated, not moved) and unions them with
the caller's `explicit_boundary` — so a plain geometry selection needs no boundary
analysis (the selection already includes fill boundary edges via `select_fill`). The
region-select caller keeps passing its lasso boundary.
- Handlers: Group / Convert to Movie Clip on a geometry selection now build these actions
from `selected_fills`/`selected_edges`.
Next: shape tweens (same-topology lerp).
CpuYuvConverter::convert rebuilt the swscale context + both ffmpeg frames on every
call (per output frame), despite the converter persisting for the whole export. Now
the scaler + reusable source/dest frames are built once in new() and reused each
convert(). (convert is &mut self; the caller already held it mutably.)
The export pump built a fresh `vello::Renderer` (full wgpu pipeline init, ~tens of ms)
AND an empty `ImageCache` on every egui repaint — i.e. per output frame. That was the
dominant per-frame cost (and the code comment already flagged it). Now the renderer +
image cache are built once on the first export pump, reused for every frame, and
dropped when the export finishes.
Also fixes a latent bug: the throwaway export ImageCache had no container path, so with
Phase 4 Tier 1 (lazy image bytes) images stored only in the container wouldn't render
in exports. The persistent cache now gets the container path.
Audit found the original "per-frame seek" theory was wrong — export decodes the source
sequentially; readback is already async/triple-buffered. Remaining wins: cache swscale
contexts (rebuilt per frame), GPU YUV conversion, decouple from the repaint cadence.
`assets_needed_at(document, time)` (core) enumerates the image asset ids referenced by
the visible vector layers' active keyframes at a time (top-level + group children).
During playback the stage decodes the images needed ~0.5s ahead into the bounded
ImageCache, so a keyframe that swaps image fills doesn't hitch when the playhead
reaches it. Gated on is_playing; nested clip-instance recursion + background decode
are refinements.
Completes Phase 4 (image asset paging: Tier 2 decoded-cache LRU, Tier 1 lazy bytes,
playback prefetch).
Project load no longer eager-reads all image bytes — `ImageAsset.data` stays empty
and the renderer's ImageCache pages compressed bytes from the `.beam` on a decode
miss (read_packed_media_readonly by asset id), decoding into the byte-bounded Tier-2
cache. Result: instant load, and compressed bytes don't accumulate on the heap.
- ImageCache: `container_path` + `resolve_bytes` (asset.data if resident — fresh
import or old base64 project — else page from the container); decoders take `&[u8]`
and use the decoded dimensions.
- Container path threaded App.current_file_path → SharedPaneState → VelloRenderContext,
set on the cache each prepare.
- load_beam_sqlite drops the 3.5b eager read.
(Refinement: a persistent read connection instead of open-per-miss.)
The decoded-image cache (peniko ImageBrush + tiny-skia Pixmap, ~w·h·4 each) was
unbounded — the main asset-memory cost. Now capped at 256 MB with usage-LRU eviction:
every get_or_decode bumps the asset's recency, and inserts past the budget evict the
least-recently-used (a miss re-decodes from the resident asset.data). Images actually
rendered each frame stay resident; unused ones age out under pressure. invalidate/clear
keep the lru + byte accounting in sync.
Next (4b): lazy-load asset.data from the container instead of eager on project open.
Image asset bytes are now stored as MediaKind::ImageAsset rows in the SQLite
container (chunked, kept-in-place on re-save) instead of base64-embedded in the
project JSON — the pageable storage Phase 4 needs.
- ImageAsset.data is `#[serde(default, skip_serializing)]`: never written to JSON,
but still deserialized for old projects (base64) which then migrate to the
container on the next save.
- save_beam writes each asset's bytes (keyed by asset id; ext from the source path),
keeping an existing row when bytes aren't resident; live_media covers them so orphan
cleanup doesn't drop them.
- load_beam_sqlite eager-reads the bytes back into `data` (Phase 4 makes this lazy +
LRU). Old base64 projects keep their JSON-deserialized data (no container row).
Image fill is now a tab in the Fill type row rather than a separate dropdown. When
Image is active, an asset-picker combo selects which image; switching to None/Solid/
Gradient clears the image fill (it otherwise overrides them). The Image tab only
appears when there are imported image assets.
- SetImageFillAction (core): set/clear `image_fill` on the selected VectorGraph fills,
with per-fill undo (mirrors SetFillPaintAction). Image takes render priority; clearing
reveals the colour/gradient underneath.
- Info Panel Shape section: an "Image:" combo listing the document's image assets (+ None)
for the selected fill(s), showing the current assignment. Assign/clear pushes the action.
This lets an existing shape be given (or cleared of) an image fill, complementing the
import/drop placement. Next: 3.5b — persist image assets in the .beam container.
The renderer painted the image brush at its native origin (0,0) with no brush
transform, so an image-filled rect drawn anywhere but the world origin only showed
the overlapping corner of the image. Both render paths now map the image's native
pixel space onto the fill's bounding box (Vello brush_transform; tiny-skia Pattern
transform) — 1:1 for an image-sized rectangle, stretch-to-bbox for arbitrary shapes.
Replaces the DCEL "not yet supported" stubs so importing/dropping an image actually
puts it in the vector scene.
- AddShapeAction gains an `image_fill` + `AddShapeAction::image_rect(...)` constructor:
a borderless rectangle (invisible edges) whose enclosed region is paint-bucketed and
tagged with an image asset id. The renderer already prioritises `image_fill`.
- Direct import (auto_place_asset): an imported image is placed centered on the canvas
at native size on a vector layer.
- Drag from the asset library onto the stage: image-filled rect at the drop point
(centered), native size, using the asset's dimensions.
Next: SetImageFillAction + an Info-Panel image-fill picker for existing shapes; then
3.5b container persistence.
The standalone egui window didn't fit the UI. Replaced it with a collapsible
"Onion Skin" section at the bottom of the Info Panel (Enabled checkbox + frames
before/after + opacity), available regardless of selection. SharedPaneState carries
a mutable `onion_skin` ref for the controls (distinct from the gated `onion` copy
used by rendering).
- Core compositor gains an optional screen-blend tint per CompositorLayer
([0,0,0,0] default = no-op, so existing compositing is unaffected);
CompositorLayer::with_tint sets it — giving the Vello/vector path a tint hook.
- For the active VECTOR layer with onion on, build ghost scenes at each neighbouring
keyframe's time via render_layer_isolated (reusing the prepare's image cache), then
render each scene → sRGB → linear → composite with warm/cool tint + opacity falloff,
behind the current frame. Off during playback; active layer only.
Completes onion skinning for all layer types (raster + vector).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A small 'Onion Skin' egui window (shown while onion skinning is enabled) with
DragValues for frames_before/after (0..=5) and an opacity slider, rendered next to
the F3 debug overlay. Lets the user tune the configurable settings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Removed the early `continue` that skipped a layer with no resident content — it
also skipped the active raster layer's onion ghosts when the current cel was blank
(the exact case you trace a new cel from neighbours). Each render arm already
guards its own empty case, so the skip was redundant.
- Ghost texture resolution now falls through cache → resident raw_pixels (upload) →
in-memory proxy → request fault-in, so neighbours that were paged out (e.g. after a
seek, or saved-this-session with no decoded proxy) page in and ghost in, instead of
silently rendering nothing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Multiplicative tint left blacks black, so outlines (the main thing to ghost in line
art) never picked up the warm/cool color. Switched the canvas-blit tint to a screen
blend (out = base + tint - base*tint): black → tint color, white unchanged, and a
clean no-op at tint=(0,0,0) for all normal blits. Reverted the default .w slots to 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- OnionSkinSettings (editor view state): enabled, frames_before/after (default 2/2),
opacity (0.35), warm past / cool future tints, linear falloff. Threaded App →
SharedPaneState (gated off during playback) → VelloRenderContext.
- Toggle: View ▸ Onion Skinning + the `O` shortcut (AppAction::ToggleOnionSkin).
- Tint: packed an RGB multiply into the canvas-blit matrix uniform's unused .w slots
(1,1,1 = no tint for all normal blits); BlitTransform::with_tint. Shader multiplies.
- Render: for the ACTIVE raster layer only, blit the N neighbouring keyframes (full
texture if resident, else the low-res proxy — uploaded on demand) tinted + faint,
composited behind the current frame. Off during playback.
Next: a settings panel (frame count/opacity) and vector-layer ghosts.
- Pointing-hand cursor when hovering a clickable keyframe diamond.
- Prefetch (Phase 3e, playback only): each update during playback, page in the next
few upcoming keyframes (PREFETCH_AHEAD=4) per raster layer that aren't resident, via
the existing async worker. Their full pixels land before the playhead reaches them,
so playback shows full frames instead of the low-res proxy on every frame (the
proxy→full pop was the "flicker"). Reactive faults still cover scrubbing.
render_layers now records each drawn keyframe diamond's screen rect + exact time in
`keyframe_diamond_hits`; handle_input hit-tests a click against them and sets the
playhead (and seeks the audio controller) to the keyframe's exact time. Uses the
previous frame's rects — diamonds don't move between frames, so the click lands
right — which sidesteps the input-before-render ordering and the drag/scroll Y math.
Works for both raster and vector keyframes.
Skyler Lehmkuhl