Documentation

Installing and running clipfarmer

clipfarmer runs on your own machine as a local web app, with a command-line interface behind it. Everything it produces stays in a working directory on your disk.

Requirements

Python 3.11 or newer.
ffmpeg on your PATH — it does the decoding, cutting and rendering.
Disk space. A single 8-hour VOD and its intermediates run to tens of gigabytes. Finished jobs can be archived to reclaim it.
A local language model for the judge, captions and header lines. Optional — the signal ranking works without one.

Setup

python3 -m venv .venv
source .venv/bin/activate
pip install .

Re-run pip install . after changing anything under src/clipfarmer/. For live reload while working on clipfarmer itself, run commands with PYTHONPATH=src instead of installing.

The app

clipfarmer serve --port 8420

Then open http://127.0.0.1:8420. The app binds to localhost — it is not exposed to your network, and there is no login because there is nothing remote to log in to. Everything below can be done from the app; the CLI is there for scripting and for the steps you want to run overnight.

The short version

# 1. Download a VOD and extract audio
clipfarmer ingest "https://www.twitch.tv/videos/123456789"

# 2. Transcribe with word-level timestamps
clipfarmer transcribe <job_id>

# 3. Score and rank candidate moments
clipfarmer rank <job_id>

# 4. Render quick previews of the top candidates so you can review them
clipfarmer preview <job_id>

# 5. Render the ones worth keeping as finished vertical clips
clipfarmer render <job_id> --select 1,4,7

Command reference

Command	What it does
`ingest <url>`	Download a Twitch VOD or YouTube video and extract audio. `--skip-chat` to skip chat replay.
`chat <job>`	Fetch chat replay separately, if ingest skipped it.
`transcribe <job>`	Transcribe with word-level timestamps.
`frames <job>`	Cache keyframes across the VOD for layout and header decisions.
`slots <job>`	Sample the whole VOD and cluster where the avatar sits. Run once, after `frames`.
`rank <job>`	Score and rank candidate moments.
`judge <job>`	Re-rank with a local model and report recall for signal, judge and blend.
`preview <job>`	Render quick, subtitle-free trims of the top candidates.
`render <job> --select`	Render finished vertical clips for the given candidate ids.
`qa <job>`	Review rendered clips against the criteria. Defaults to all rendered.
`import-clips <job>`	Fetch viewer-made clips for this VOD.
`groundtruth <job>`	Cluster viewer clips into labelled moments for evaluation.
`eval <job>`	Score the current ranking against those labels — recall@K per tier, and what it missed.
`watch`	Monitor a channel and ingest new VODs as they appear.
`jobs`	List jobs and their state.
`prune`	Archive finished jobs and reclaim disk.
`serve`	Run the local web app.

Where things land

Each ingest creates a working directory named for the job:

jobs/<job_id>/
  source video, extracted audio
  transcript with word timings
  chat.json           chat replay
  frames/             cached keyframes
  candidates          the ranking
  preview/            watchable trims
  final/              finished vertical clips

These are large binaries and are not tracked in version control. Nothing in here is uploaded anywhere except the finished clip you choose to post.

Configuration

A single config.yaml holds the settings, so tuning does not mean editing code:

Scoring — weights for each signal, candidate count, clip length bounds, hook phrase lists.
Ingest — ingest.video_height. Set this to 1080 if you want punch-ins; a 720p source does not survive the upscale.
Render — subtitle styling, layout behaviour, punch-in strength.
Headers — the motion ceiling used alongside the silent scroll test.
Transcription — whisper model size.
Prompts — the text sent to the local model for judging, captions and header lines, editable in the app's Settings.

Connecting TikTok

Register an app in the TikTok developer portal, put its client key and secret in your local .env, then use the connect action in clipfarmer to run TikTok's authorisation flow. clipfarmer reports the three preconditions separately — an app exists, a creator authorised it, and the app has been approved for the product you want — because they fail at different times and a single "connected" light would hide which one is wrong.

The TikTok page documents the scopes, the data involved, and the difference between the draft route and a direct post.

Common questions Get help