Defora turns Stable Diffusion Forge + Deforumation into a playable instrument: live visuals, prompt morphing, camera motion, beat-synced controls, and a neon-styled web UI for performance.

Control your AI video generation in real-time with multiple interface options:

Web UI — Browser-based Performance Dashboard Neon-styled web interface with macro sliders, motion presets, and live parameter control

TUI — Terminal-based Control Center Full-featured ncurses interface for terminal enthusiasts with ASCII preview and parameter control

Morph between multiple prompts and control narrative flow:

Web UI: Visual prompt morphing with slots and blend control

TUI: Text-based prompt management with morph ranges

Gamepad-style camera controls and curve editing:

Web UI: Visual motion curves with preset system

TUI: Multi-lane curve editor in your terminal

Sync your visuals to audio with beat-driven macros:

Web UI: Waveform visualization with macro rack

TUI: Beat-synced parameter automation

Fine-tune your generation with ControlNet and render settings:

Web UI: ControlNet slots and engine configuration

TUI: Complete control from your terminal

Map any MIDI controller to live parameters:

• Web UI: Browser-based Web MIDI support
• TUI: CC mapping with learn mode
• Full parameter control via hardware knobs and faders

• Moderately fast GPU (4070ti / 5060ti tested) with at least 12G VRAM
• Models that have either Lightning support or accompanying LORa that need 1-2 steps for a frame. SDXL Lightning is proven to be working fine
• 32GB ram is minimum, 64gb ram recommended
• Stable Diffusion Forge + Deforum extension from https://github.com/Tok/sd-forge-deforum and https://github.com/lllyasviel/stable-diffusion-webui-forge
• There is docker stack including these in the package but it is recommended to run on external node

• forge_cli — Model-aware txt2img/Deforum CLI with preset support and sensible defaults
• defora_tui — Full multi-tab ncurses interface (LIVE, PROMPTS, MOTION, AUDIO, CONTROLNET, SETTINGS)
• deforumation_cli_panel — Lightweight control panel with rebindable hotkeys (strength/CFG/noise/pan/zoom/rot/FOV)
• deforumation_dashboard — Curses dashboard mirroring the Deforumation GUI tabs

• audio_reactive_modulator — Map audio bands to mediator parameters; output schedules or stream live
• Beat-synced macros — Drive parameters with tempo-aligned modulation

• deforumation_runs_cli — Browse run manifests, set overrides, rerun/continue generations
• deforumation_request_dispatcher — Merge manifests/presets/overrides and execute runs
• monitor_cli — Tail latest frames and show live mediator values

• stream_helper — Push rendered frames to RTMP/SRT/WHIP via ffmpeg
• Web streaming stack — Docker compose with ffmpeg encoder, Nginx/Node (HLS), RabbitMQ, and mediator bridge
• Web UI — Browser-based performance interface with WebSocket controls and HLS video

• docs/ — Architecture, workflows, server targeting, schemas, troubleshooting guides

git clone https://github.com/janiluuk/defora.git
cd defora
git submodule update --init --recursive
# or: ./scripts/clone_deforumation.sh
pip install -r requirements.txt

Start Forge with Deforum API enabled:

# Example: assuming Forge is installed in ~/stable-diffusion-webui-forge
cd ~/stable-diffusion-webui-forge
./webui.sh --deforum-api

Generate images or animations with smart defaults:

# Single image
./forge_cli "a synthwave city at night"

# Animation (240 frames)
./forge_cli deforum -f 240 "surreal biomechanical cathedral"

Full browser-based performance interface:

docker-compose up --build
# Open http://localhost:8080

Features: Live macro sliders, prompt morphing, motion curves, audio sync, MIDI control

Full-featured ncurses interface for terminal users:

./defora_tui

Navigation: F1-F6 to switch tabs, ←/→ to adjust parameters, Q to quit

Minimal control panel with hotkey bindings:

./deforumation_cli_panel --host 127.0.0.1 --port 8766

Browse & Rerun Saved Generations:

./deforumation_runs_cli

Audio-Reactive Modulation:

# Generate audio schedule
./audio_reactive_modulator --audio song.wav --fps 24 --output audio_mod.json

# Stream live to mediator
./audio_reactive_modulator --audio song.wav --fps 24 --live \
  --mediator-host 127.0.0.1 --mediator-port 8766

Monitor Live Generation:

./monitor_cli --frames runs/<id>/frames

Stream to RTMP/SRT:

./stream_helper start --source runs/<id>/frames \
  --target rtmp://example/live/key --fps 24

• Audio-visual instrument: Treat prompts, camera, and ControlNet as live parameters; drive them via CLI/TUI/Web or controllers.
• Presets & manifests: Deforum JSON presets and run manifests can be merged with CLI overrides. The runs TUI writes *_request.json that the dispatcher consumes.
• Mediator control: The panel, dashboard, and audio modulator talk to the mediator websocket so you can steer generation without the full UI.
• Model-aware defaults: forge_cli picks steps/CFG/sampler based on the active model (Flux/SDXL/SD1.5) and can auto-switch to Flux-schnell.

• The DeforumationQT mediator is vendored under deforumation/. Run it to bridge SD-Forge/Deforum to the Deforumation UI and our CLI tools (panel/dashboard/audio modulator).
• See docs/mediator_setup.md for mediator startup steps and for installing the sd-forge Deforum bridge from deforumation/Deforum_Version/sd-forge/.
• Docker users: docker-compose up --build mediator sd-forge will start the mediator (ports 8765/8766) and a Forge container with the Deforumation-patched Deforum extension pre-installed (UI on port 7860).

• FORGE_API_BASE — target Forge server (e.g., http://192.168.2.101:7860).
• FORGE_OUT_DIR — output directory for forge_cli.
• DEFORUMATION_FORGE_CLI — path override for Forge CLI when auto-dispatching.
• DEFORUMATION_AUTO_DISPATCH=1 — have deforumation_runs_cli immediately dispatch requests.
• DEFORUMATION_FRAMES_DIR — default frames path for monitor_cli.
• DEFORUMATION_ASCII_PREVIEW=1 — enable ASCII thumbnails in monitor/runs TUI (needs Pillow).
• DEFORUMATION_MEDIATOR_HOST/DEFORUMATION_MEDIATOR_PORT — defaults for mediator-backed UIs (panel, dashboard, monitor).
• CONTROL_TOKEN — WebSocket control token for the web UI (set when running docker-compose).
• MEDIATOR_HOST (compose bridge) — set this if host.docker.internal is not available on your host (common on Linux) so the control bridge can reach the mediator.
• Web MIDI: enable in your browser and map controls in the web UI (MIDI tab) to live parameters.

• CLI/package code: defora_cli/
• Executable wrappers: ./forge_cli, ./deforumation_request_dispatcher, ./deforumation_runs_cli, ./deforumation_cli_panel, ./deforumation_dashboard, ./defora_tui, ./monitor_cli, ./stream_helper, ./audio_reactive_modulator
• Web UI & streaming stack: docker-compose.yml, docker/web/ (Nginx+Node+Vue front-end, HLS, controls), docker/bridge/ (mediator bridge)
• Docs: docs/ (workflows, server targeting, schema, streaming)
• Tests: tests/ (Python) and docker/web/test (web UI smoke tests)
• Logo: assets/defora_logo.svg (dark-mode friendly, neon gradient)

Run the suite (requires pytest installed):

./scripts/run_tests.sh  # or: python -m pytest