Film tools (ComfyUI + Coqui TTS)#

Home-manager modules for the solo animated-short pipeline on Apple Silicon (modules/home-manager/film-tools/). Enabled on work-laptop-m4 (M4 Pro), where they share the 24 GB unified-memory budget with the Ollama model set — avoid running ComfyUI generation concurrently with the qwen2.5:14b batch jobs.

Both tools are deliberately venv-based rather than nix-packaged: PyTorch Metal wheels are not in nixpkgs, and forcing them through derivations breaks on every torch release. Nix provides the interpreters and wrapper scripts; pip owns the ML dependency tree inside a per-tool venv.

Enabling#

home-manager.users.<user>.imports = [
  ../../../modules/home-manager/film-tools
];
home-manager.users.<user>.tsunaminoai.film-tools = {
  comfyui.enable = true;
  tts.enable = true;
};

ComfyUI (image generation, Metal/MPS)#

Command	Purpose
`comfyui`	Start the server on `http://127.0.0.1:8188` (first run clones ComfyUI and builds the venv)
`comfyui-update`	`git pull` ComfyUI + reinstall requirements

Options: tsunaminoai.film-tools.comfyui.{dataDir,port,fp16,python,extraArgs}. Everything lives under dataDir (default ~/comfyui) — point it at external storage when checkpoints outgrow the internal disk. Drop model checkpoints (.safetensors) into <dataDir>/models/checkpoints/, LoRAs into <dataDir>/models/loras/.

Coqui TTS (scratch voices)#

Command	Purpose
`film-tts-setup`	One-time venv bootstrap (PyTorch + coqui-tts[codec] + nix ffmpeg for torchcodec) into `~/.film-tts/.venv`
`film-tts "text" [out.wav]`	Synthesize a single line
`film-tts-batch script.txt [outdir/]`	One wav per non-empty line (`line_001.wav`, …); `speakerID<TAB>text` lines cast a voice per line
`film-tts-update`	Upgrade the TTS package in the venv

The venv installs coqui-tts (the maintained idiap fork — the original PyPI TTS package is abandoned and does not resolve on Python ≥ 3.12; the fork keeps the same tts CLI and model names).

Override the model and voice per session:

FILM_TTS_MODEL="tts_models/en/vctk/vits" FILM_TTS_SPEAKER=p225 film-tts "Hello." out.wav

Useful models: tts_models/en/ljspeech/tacotron2-DDC (default; fast, single speaker), tts_models/en/vctk/vits (multi-speaker, ~109 voices), tts_models/multilingual/multi-dataset/xtts_v2 (best quality, heavy). List a multi-speaker model’s voices with tts --model_name tts_models/en/vctk/vits --list_speaker_idxs.

Multi-voice workflow (scratch dialogue / radio segments)#

For scenes with a cast, use a multi-speaker model and give film-tts-batch a TSV script — plain lines use the default/FILM_TTS_SPEAKER voice, and speakerID<TAB>text lines pick a voice per line:

p270<TAB>Three a.m. The schoolyard where she died is dark and empty.
p294<TAB>So, Naru-chan. How does it feel to be the new candidate?
p225<TAB>How could you? All of you. I thought you were my friends!

(<TAB> = a literal tab character.)

FILM_TTS_MODEL="tts_models/en/vctk/vits" film-tts-batch scene01.tsv audio/scene01/

That yields one numbered wav per line (line_001.wav, …) for assembly in an editor or ffmpeg. The full production pattern — per-character ffmpeg FX chains (ghost reverb, radio EQ), layered crowd lines, an all-generated drone bed, loudnorm mastering — is demonstrated end-to-end by the film project’s radio-segment/build.sh (in the film working tree at ~/code/film/usagi-is-dead/, not this repo), which produced the first finished audio segment from chapter 1 of the source fic.