12 KiB

Raw Blame History

Lightningbeam Rust UI - Implementation Plan

Project Overview

Goal: Complete migration from JavaScript/Tauri to Rust/egui

Scope: ~10,000+ lines of code migration
Source: ~/Dev/Lightningbeam-2/src/ (JS models, actions, main.js)
Target: Full-featured Rust animation editor with native performance
Motivation: IPC overhead between Rust↔JS too slow for real-time performance

UI Boundaries

In Scope (This UI):

Layout system (panes, splits, resize)
Stage rendering (layer compositing with Vello)
Timeline (frame scrubbing, keyframes)
Tools (pen, select, transform)
Property panels
User interaction & editing

Out of Scope (External Systems):

Video import/export: Handled by separate video processing module
Audio playback/processing: Handled by daw-backend
File I/O: Coordinated with backend systems
Plugin architecture: TBD in separate crate

Technology Stack

UI Framework

Primary: eframe 0.29 + egui (immediate-mode GUI)
Theming: egui-aesthetix for professional appearance
Windowing: winit 0.30
Native Menus: muda for OS-integrated menus (File, Edit, etc.)

Rendering

GPU: wgpu 22 for low-level GPU access
2D Graphics: Vello 0.3 for high-performance vector rendering
- 84% faster than iced in benchmarks
- Used for: Stage, Timeline, Node Editor, Virtual Piano
Architecture: Layer-based rendering
- Each layer (2D animation, video, etc.) renders to texture
- Textures composited together on Stage canvas

Serialization

Format: JSON (serde_json)
Compatibility: Match existing JS JSON schema for layouts

Implementation Phases

✅ Phase 1: Layout System (COMPLETE)

Status: Fully implemented and tested

Features:

Workspace structure (lightningbeam-core + lightningbeam-editor)
JSON layout loading (8 predefined layouts)
Recursive pane tree rendering
Layout switching via menu
Drag-to-resize dividers with visual feedback
Split operations with live preview
Join operations (remove splits)
Context menus on dividers
ESC/click-outside cancellation
Pane selection and type switching

Files:

lightningbeam-core/src/layout.rs - Core data structures
lightningbeam-editor/src/main.rs - Rendering and interaction
assets/layouts.json - Layout definitions

🔄 Phase 2: Pane Architecture (CURRENT)

Goal: Define proper pane abstraction with header + content sections

Requirements:

Pane Trait/Struct:

trait Pane {
    fn header(&mut self, ui: &mut egui::Ui) -> Option<Response>;
    fn content(&mut self, ui: &mut egui::Ui, rect: Rect);
    fn name(&self) -> &str;
}

Header Section:
- Optional controls (used by Timeline pane)
- Play/pause, zoom, frame counter
- Collapsible/expandable
Content Section:
- Main pane body
- Custom rendering per pane type
- Can use egui widgets or custom GPU rendering
Integration:
- Update render_pane() to use new trait
- Support dynamic pane instantiation
- Maintain layout tree structure

Deliverables:

Pane trait in lightningbeam-core/src/pane.rs
Example placeholder pane implementation
Update main.rs to use pane trait
Document pane interface

Goal: OS-integrated menu bar using muda

Features:

File menu (New, Open, Save, Export)
Edit menu (Undo, Redo, Cut, Copy, Paste)
View menu (Layouts, Zoom, Panels)
Help menu (Documentation, About)
Platform-specific integration (macOS menu bar, Windows menu bar)
Keyboard shortcuts

Dependencies:

muda = "*"  # Native menu system (what Tauri uses)

Phase 4: Stage Pane Implementation

Goal: Main canvas with Vello-based layer compositing

Architecture:

Stage Pane
├── wgpu Surface
├── Vello Scene
└── Layer Renderer
    ├── Layer 1 → Texture
    ├── Layer 2 → Texture
    ├── Layer 3 → Texture (video from external)
    └── Composite → Final render

Features:

wgpu surface integration with egui
Vello scene management
Layer texture rendering
Compositing pipeline
Camera/viewport controls (pan, zoom)
Selection visualization
Tool integration (later phase)

Performance Targets:

60 FPS at 1920x1080
Sub-16ms frame time
Smooth layer compositing

Phase 5: Timeline Pane Implementation

Goal: Frame-based animation timeline with Vello rendering

Features:

Frame scrubber with Vello
Layer tracks
Keyframe visualization
Playback controls in header
Zoom/pan timeline
Frame selection
Drag keyframes
Multi-selection

Header Controls:

Play/pause button
Current frame indicator
FPS selector
Zoom slider

Audio Integration:

Display audio waveforms (data from daw-backend)
Sync playback with audio system
No direct audio processing in UI

Phase 6: Additional Panes

Priority Order:

Toolbar (simple, egui widgets)
Info Panel (property editor, egui widgets)
Outliner (layer hierarchy, egui tree)
Node Editor (Vello-based graph)
Piano Roll (MIDI editor, Vello)
Preset Browser (file list, egui)

Phase 7: Core Class Migration

Source: ~/Dev/Lightningbeam-2/src/models/

Models to Migrate

From ~/Dev/Lightningbeam-2/src/models/:

root.js (34 lines)
- Document root structure
- → lightningbeam-core/src/document.rs
layer.js
- Layer types (2D, video, audio, etc.)
- Transform properties
- Visibility, opacity
- → lightningbeam-core/src/layer.rs
shapes.js (752 lines)
- Path, Rectangle, Ellipse, Star, Polygon
- Stroke/fill properties
- Bezier curves
- → lightningbeam-core/src/shape.rs
animation.js
- Keyframe data structures
- Interpolation curves
- → lightningbeam-core/src/animation.rs
graphics-object.js
- Base graphics properties
- Transform matrices
- → lightningbeam-core/src/graphics.rs

Actions to Migrate

From ~/Dev/Lightningbeam-2/src/actions/:

index.js (2,615 lines)
- Drawing tools
- Transform operations
- Layer operations
- → lightningbeam-core/src/actions/
selection-actions.js (166 lines)
- Selection state management
- → lightningbeam-core/src/selection.rs

Migration Strategy

Rust-first design: Leverage Rust's type system, don't just transliterate JS
Serde compatibility: Ensure classes can serialize/deserialize from existing JSON
Performance: Use Arc<RwLock<T>> for shared mutable state where needed
Memory safety: Eliminate runtime errors through compile-time checks

Phase 8: Tools & Interaction

After core classes are migrated:

Pen tool (Bezier curves)
Select tool (bounding box, direct selection)
Transform tool (rotate, scale, skew)
Shape tools (rectangle, ellipse, star, polygon)
Text tool
Eyedropper
Zoom/pan

Phase 9: Feature Parity

Remaining features from JS version:

Onion skinning
Frame export (PNG sequence)
Project save/load
Undo/redo system
Preferences/settings
Keyboard shortcuts
Help/documentation
Clipboard operations (copy/paste)

Architecture Decisions

Why egui over iced?

Vello compatibility: iced has issues with Vello integration
Immediate mode: Simpler state management for complex UI
Maturity: More stable and well-documented
Performance: Good enough for our needs

Layer Rendering Strategy

// Conceptual pipeline
for layer in document.layers {
    let texture = layer.render_to_texture(vello_renderer);
    composite_textures.push(texture);
}
let final_image = compositor.blend(composite_textures);
stage.display(final_image);

State Management

Document state: Arc<RwLock<Document>> - shared across panes
Selection state: Event-based updates
UI state: Local to each pane (egui handles this)

External System Integration

Video Layers:

UI requests frame texture from video processing module
Module returns GPU texture handle
UI composites texture with other layers

Audio Playback:

UI sends playback commands to daw-backend
Backend handles audio processing/mixing
UI displays waveforms (data from backend)

File Structure (Target)

lightningbeam-ui/
├── lightningbeam-core/          # Pure Rust library
│   ├── src/
│   │   ├── lib.rs
│   │   ├── layout.rs            # ✅ Layout system
│   │   ├── pane.rs              # ⏳ Pane trait
│   │   ├── document.rs          # ❌ Document root
│   │   ├── layer.rs             # ❌ Layer types
│   │   ├── shape.rs             # ❌ Shape primitives
│   │   ├── animation.rs         # ❌ Keyframes
│   │   ├── graphics.rs          # ❌ Graphics base
│   │   ├── selection.rs         # ❌ Selection state
│   │   └── actions/             # ❌ Action system
│   └── Cargo.toml
│
├── lightningbeam-editor/        # egui application
│   ├── src/
│   │   ├── main.rs              # ✅ App + layout rendering
│   │   ├── menu.rs              # ❌ Native menu integration (muda)
│   │   ├── panes/               # ⏳ Pane implementations
│   │   │   ├── stage.rs         # ❌ Stage pane
│   │   │   ├── timeline.rs      # ❌ Timeline pane
│   │   │   ├── toolbar.rs       # ❌ Toolbar
│   │   │   ├── infopanel.rs     # ❌ Info panel
│   │   │   └── outliner.rs      # ❌ Outliner
│   │   ├── rendering/           # ❌ Vello integration
│   │   └── tools/               # ❌ Drawing tools
│   ├── assets/
│   │   └── layouts.json         # ✅ Layout definitions
│   └── Cargo.toml
│
└── Cargo.toml                   # Workspace

Legend: ✅ Complete | ⏳ In Progress | ❌ Not Started

Dependencies Roadmap

Current (Phase 1)

eframe = "0.29"           # UI framework
wgpu = "22"               # GPU
vello = "0.3"             # 2D rendering
kurbo = "0.11"            # 2D geometry
peniko = "0.5"            # 2D primitives
serde = "1.0"             # Serialization
serde_json = "1.0"        # JSON

Phase 2-3 Additions

muda = "*"                # Native OS menus

Phase 4+ Additions

image = "*"               # Image loading
egui-aesthetix = "*"      # Theming
clipboard = "*"           # Copy/paste

Performance Goals

Rendering

Stage: 60 FPS @ 1080p, 100+ layers
Timeline: Smooth scrolling with 1000+ frames
Memory: < 500MB for typical project

Benchmarks to Track

Layer render time
Composite time
UI frame time
Memory usage per layer

Testing Strategy

Unit tests: Core classes (Layer, Shape, Animation)
Integration tests: Pane rendering
Visual tests: Screenshot comparison for rendering
Performance tests: Benchmark critical paths
Manual testing: UI interaction, edge cases

Migration Checklist

Before Deprecating JS Version

All panes functional
Core classes migrated
Project save/load working
Performance meets/exceeds JS version
No critical bugs
User testing complete
Native menus integrated
External system integration verified (video, audio)

Assets Management

Current Approach:

SVG icons referenced from ~/Dev/Lightningbeam-2/src/assets/
Allows pulling in new icons/changes from upstream during rewrite
Using egui_extras::RetainedImage for cached SVG rendering

TODO (Before Release):

Move assets to lightningbeam-editor/assets/icons/
Update asset paths in code
Set up asset bundling/embedding

Open Questions

Plugin system: Support for extensions?
Scripting: Embed Lua/JavaScript for automation?
Collaborative editing: Future consideration?

References

JS Source: ~/Dev/Lightningbeam-2/src/
egui docs: https://docs.rs/egui
Vello docs: https://docs.rs/vello
wgpu docs: https://docs.rs/wgpu
muda docs: https://docs.rs/muda

12 KiB Raw Blame History

Lightningbeam Rust UI - Implementation Plan

Project Overview

UI Boundaries

Technology Stack

UI Framework

Rendering

Serialization

Implementation Phases

✅ Phase 1: Layout System (COMPLETE)

🔄 Phase 2: Pane Architecture (CURRENT)

Phase 3: Native Menu Integration

Phase 4: Stage Pane Implementation

Phase 5: Timeline Pane Implementation

Phase 6: Additional Panes

Phase 7: Core Class Migration

Models to Migrate

Actions to Migrate

Migration Strategy

Phase 8: Tools & Interaction

Phase 9: Feature Parity

Architecture Decisions

Why egui over iced?

Layer Rendering Strategy

State Management

External System Integration

File Structure (Target)

Dependencies Roadmap

Current (Phase 1)

Phase 2-3 Additions

Phase 4+ Additions

Performance Goals

Rendering

Benchmarks to Track

Testing Strategy

Migration Checklist

Before Deprecating JS Version

Assets Management

Open Questions

References

12 KiB

Raw Blame History