diff --git a/ARR-STACK.md b/ARR-STACK.md new file mode 100644 index 0000000..7d71194 --- /dev/null +++ b/ARR-STACK.md @@ -0,0 +1,476 @@ +# Media automation stack (*arr) for `barrios` + +Status: **proposed / not yet deployed.** This document is the design and setup guide. No +services in `docker-compose.yml` have been changed yet. + +## Why + +Media acquisition is currently manual: someone asks for a film or series, and it has to be +found, downloaded, and copied into the right folder under `/mertz/data/` for Jellyfin to +pick up. This stack automates that end to end: a request made in a web UI results in the +title being found, downloaded, subtitled (English **and** French — required for every +item), renamed, filed into the existing Jellyfin libraries, and surfaced in Jellyfin, with +no manual step. + +## Decisions + +| Topic | Decision | Consequence | +|---|---|---| +| Download source | **Usenet only** (SABnzbd + paid provider + indexer). | No BitTorrent, so **no VPN / gluetun** needed. A VPN-gated qBittorrent can be added later as a fallback for what Usenet can't find. | +| Subtitles | **Bazarr**, requiring **English + French** on every movie and episode. | Bazarr fetches `.en.srt` / `.fr.srt` sidecar files next to each video. | +| Request UI | **Jellyseerr**, **internet-exposed** at `jellyseerr.dylanbaird.com`, its own login (reuses Jellyfin accounts). | Requests can be made from anywhere. It is the only new service facing the internet. | +| Admin UIs (Prowlarr / Sonarr / Radarr / Bazarr / SABnzbd) | **Caddy subdomains, LAN-only.** | Real HTTPS hostnames, but each site drops connections from non-private source IPs. | + +Out of scope here: Jellyfin hardware transcoding (`/dev/dri`), de-rooting the Jellyfin +container, BitTorrent + VPN, forward-auth / SSO. + +--- + +## The components + +| App | Image | Port | Job | +|---|---|---|---| +| **Prowlarr** | `lscr.io/linuxserver/prowlarr` | 9696 | Indexer manager. Configure Usenet indexers once here; it syncs them into Sonarr + Radarr. | +| **Radarr** | `lscr.io/linuxserver/radarr` | 7878 | Movies: find, download, rename, file, monitor. | +| **Sonarr** | `lscr.io/linuxserver/sonarr` | 8989 | TV series: same, season/episode-aware, keeps watching ongoing shows. | +| **Bazarr** | `lscr.io/linuxserver/bazarr` | 6767 | Subtitles for everything Radarr/Sonarr manage, in the required languages. | +| **SABnzbd** | `lscr.io/linuxserver/sabnzbd` | 8080 | Usenet downloader. Talks to the provider over SSL; no VPN. | +| **Jellyseerr** | `fallenbagel/jellyseerr` | 5055 | The request front-end. Approved requests flow into Radarr/Sonarr. | + +## How the finished flow works + +``` +requester -> Jellyseerr (jellyseerr.dylanbaird.com, login) + | approved request + v + Radarr / Sonarr --search--> Prowlarr (indexers) --> best NZB + | | + | send NZB v + +------------------------------------------> SABnzbd (SSL download to + /mertz/data/downloads/usenet/) + +-------------------------------------------------+ + v + Radarr / Sonarr import -> hardlink / instant-move into + /mertz/data/movies/... and /mertz/data/television/... + | + +-> Bazarr -> Movie.en.srt / Movie.fr.srt beside the video + +-> "Connect" webhook -> Jellyfin library scan -> appears on the TV +``` + +All new containers join the existing `barrios-network` and address each other by +`container_name` (e.g. `http://radarr:7878`). Every media container mounts the **same** +`/mertz/data` tree at the **same** path `/data`, so no *arr "remote path mapping" is +needed and imports are instant moves / hardlinks, not copies. **Jellyfin is not modified** — +files land in its existing `/mertz/data/movies` and `/mertz/data/television` libraries. + +--- + +## Usenet: what it is and exactly what to sign up for + +Usenet is a decades-old global network of servers now used, in practice, as a paid +high-speed file-distribution backbone. For this stack it replaces "find a torrent" with +"SABnzbd pulls the file directly from a commercial server over an encrypted connection." +No peers see your IP, nothing uploads from the house, there is no seeding ratio, and +downloads run at full line speed — which is why **no VPN is required**. + +It is **not** free. You pay for two (optionally three) separate things from separate +accounts. New users drown in comparison threads, so here are single concrete picks. You +can revisit later; these are known-good starting points. + +### 1. A Usenet provider — stores the data + +Providers differ by which physical network ("backbone") they sit on, retention (how many +days of history they keep), connection count, and price. As a beginner the only thing that +matters: **one unlimited plan on the Omicron backbone with long retention.** + +> **Pick: Newshosting**, annual plan (often ~$5–8/mo billed yearly, ~$50–80/yr). +> Unlimited data, ~5,900-day retention, up to 100 connections, SSL, built-in search. +> The most commonly recommended "just works" first provider. Same-backbone cheaper +> alternatives: **Frugal Usenet** or **Eweka**. + +Do **not** buy multiple unlimited providers. One is enough. + +**Optional, later:** a one-time **block account** on a *different* backbone — e.g. +**Newsgroup Ninja** or **UsenetExpress**, pay once for a data allotment (e.g. 500 GB–1 TB) +that never expires. SABnzbd uses it only to fetch parts your main provider is missing. Add +it only if you start seeing "article not found" / repair failures. + +### 2. Usenet indexer(s) — the search engines that produce `.nzb` files + +An indexer is a searchable catalogue of what's on Usenet; it hands Prowlarr an `.nzb` +(a pointer file) that SABnzbd downloads from your provider. Coverage varies, so people run +two or three — but for automation, two is plenty. + +> **Pick two:** +> - **NZBGeek** — paid, ~$12–20/yr (or a one-time "lifetime" fee during its periodic +> sales). Reliable, always open for signup, first-class Prowlarr support, beginner- +> friendly. The standard first indexer. +> - **one free indexer** as a secondary source — **NZBPlanet** or **abNZB** free tier +> (daily API-call caps, fine for a household). +> +> If you only want to pay for one thing, NZBGeek alone gets you running. Add a second +> paid indexer later only if titles that clearly exist come back "not found." + +Skip invite-only indexers (DrunkenSlug, NZBFinder when closed) for now. + +### 3. SABnzbd — the downloader (deployed by this stack) + +Holds your **provider** credentials. Your **indexer** API keys go in Prowlarr, not here. + +### What you enter, and where + +| Value | Where you get it | Where it goes | +|---|---|---| +| Provider hostname, port `563` (SSL), username, password, max connections | Provider dashboard after signup | **SABnzbd → Config → Servers** | +| Indexer API key + site URL | Each indexer's profile / API page | **Prowlarr → Indexers → Add** (one per indexer) | +| Sonarr / Radarr API keys | Those apps, Settings → General | **Prowlarr → Settings → Apps** | + +### Year-one cost, minimum viable + +- Provider (Newshosting annual): **~$50–80/yr** +- Indexer (NZBGeek): **~$15/yr** +- Optional OpenSubtitles.com VIP: ~$20/yr (free tier is fine to start) +- **Total ~$65–95/yr.** No VPN, no per-download charges. + +### What to expect + +- New mainstream film/series: usually grabbable within a few hours of release, then + downloads in minutes at full line speed. +- Back-catalogue titles: effectively instant; retention covers ~15+ years. +- Very obscure, foreign-only, or niche content: occasionally "not found" — the gap a + future VPN + torrent client would fill. + +### Failure modes you'll eventually see + +- **"Article not found" / par2 repair fails** on some downloads → provider is missing + parts. Fix: add a block account on another backbone. +- **"No results" in Sonarr/Radarr** for something that clearly exists → indexer coverage + gap. Fix: add another indexer in Prowlarr. +- **Download finishes but import fails** → path/permissions (see host prep) or an + unparseable release name — rare. + +--- + +## Implementation + +### Step 1 — Host preparation on `barrios` (run once, before `docker compose up`) + +```sh +# config dirs (one per service, matching the repo's /mertz/ convention) +mkdir -p /mertz/{prowlarr,sonarr,radarr,bazarr,sabnzbd,jellyseerr}/config + +# download tree — same mergerfs pool as the libraries, so imports are instant moves +mkdir -p /mertz/data/downloads/usenet/{movies,tv,incomplete} + +# ownership: the *arr containers run as 1000:1000; Jellyfin has run as root, so existing +# library files/dirs may be root-owned and unwritable by the *arr apps. Check first: +ls -la /mertz/data +chown -R 1000:1000 /mertz/{prowlarr,sonarr,radarr,bazarr,sabnzbd,jellyseerr} +chown -R 1000:1000 /mertz/data/downloads +chown -R 1000:1000 /mertz/data/movies /mertz/data/television +``` + +Confirm the intended owner is uid/gid `1000` with `id` on the host; adjust the values in +`.env` (below) if not. `/mertz/data/music` and `/mertz/data/disney-films` are left alone — +not managed by this stack. Radarr can later add `/data/disney-films` as a second root +folder if wanted. + +**mergerfs caveat:** hardlinks and instant moves only work when source and destination +land on the **same** underlying disk. Check how `/mertz` is mounted +(`mount | grep mergerfs`, and the fstab entry) and prefer a path-preserving create policy +(e.g. `category.create=epmfs` or `eplfs`) plus `moveonenospc=true`, so a movie/show folder +stays on one branch. If not, Sonarr/Radarr fall back to a copy within the pool — it still +works, but uses temporary extra space and isn't instant. This is a host fstab change, +outside this repo. + +**DNS prerequisites:** + +- Public DNS for `dylanbaird.com`: ensure `sonarr`, `radarr`, `prowlarr`, `bazarr`, + `sabnzbd`, `jellyseerr` subdomains resolve to the server's public IP (a wildcard + `*.dylanbaird.com` covers all; otherwise add six A/AAAA records) so Caddy can complete + the ACME challenge. Only `jellyseerr` needs to be publicly *usable*; the other five are + gated by `lan_only` but still need the name to resolve for cert issuance. +- Pi-hole local DNS: add A records for the same six names → the server's **LAN** IP, so + LAN clients reach Caddy directly and the `lan_only` matcher sees real private source IPs. + +### Step 2 — `.env` (append) + +``` +# ARR STACK +PUID=1000 +PGID=1000 +TZ=America/Toronto +``` + +No API keys go in `.env` — the apps generate their own at first run and are wired together +through their web UIs. Provider credentials are entered in the SABnzbd UI. + +### Step 3 — `docker-compose.yml` (add services) + +Place after `jellyfin`, before `registry`. No `ports:` are published — these are reached +only through Caddy, which enforces the LAN restriction; publishing host ports would bypass +that on an internet-facing box. `restart: unless-stopped` matches the existing +`jellyfin` / `pihole` / `nextcloud` services. + +```yaml + prowlarr: + image: lscr.io/linuxserver/prowlarr:latest + container_name: prowlarr + environment: + - PUID=${PUID} + - PGID=${PGID} + - TZ=${TZ} + volumes: + - /mertz/prowlarr/config:/config + restart: unless-stopped + networks: + - barrios-network + + sonarr: + image: lscr.io/linuxserver/sonarr:latest + container_name: sonarr + environment: + - PUID=${PUID} + - PGID=${PGID} + - TZ=${TZ} + - UMASK=002 + volumes: + - /mertz/sonarr/config:/config + - /mertz/data:/data + depends_on: + - prowlarr + - sabnzbd + restart: unless-stopped + networks: + - barrios-network + + radarr: + image: lscr.io/linuxserver/radarr:latest + container_name: radarr + environment: + - PUID=${PUID} + - PGID=${PGID} + - TZ=${TZ} + - UMASK=002 + volumes: + - /mertz/radarr/config:/config + - /mertz/data:/data + depends_on: + - prowlarr + - sabnzbd + restart: unless-stopped + networks: + - barrios-network + + bazarr: + image: lscr.io/linuxserver/bazarr:latest + container_name: bazarr + environment: + - PUID=${PUID} + - PGID=${PGID} + - TZ=${TZ} + - UMASK=002 + volumes: + - /mertz/bazarr/config:/config + - /mertz/data:/data + depends_on: + - sonarr + - radarr + restart: unless-stopped + networks: + - barrios-network + + sabnzbd: + image: lscr.io/linuxserver/sabnzbd:latest + container_name: sabnzbd + environment: + - PUID=${PUID} + - PGID=${PGID} + - TZ=${TZ} + - UMASK=002 + volumes: + - /mertz/sabnzbd/config:/config + - /mertz/data:/data + restart: unless-stopped + networks: + - barrios-network + + jellyseerr: + image: fallenbagel/jellyseerr:latest + container_name: jellyseerr + environment: + - TZ=${TZ} + - LOG_LEVEL=info + volumes: + - /mertz/jellyseerr/config:/app/config + depends_on: + - radarr + - sonarr + restart: unless-stopped + networks: + - barrios-network +``` + +Notes: + +- SABnzbd's web port 8080 collides with `nextcloud`'s **published** host port — irrelevant + here because sabnzbd publishes nothing; Caddy reaches it at `sabnzbd:8080` on the + network. +- Tags left at `latest` to match the repo convention. Consider pinning to major tags + (`:4`) and backing up `/mertz/*/config` before `docker compose pull`, since Sonarr/ + Radarr occasionally run DB migrations on major bumps. +- Optional later: a `recyclarr` container (`ghcr.io/recyclarr/recyclarr`) on a cron to + push TRaSH-guide quality profiles / custom formats (incl. "prefer releases with + subtitles") into Sonarr & Radarr. Not required for the core flow. + +### Step 4 — `caddy/Caddyfile` (add hostnames) + +Add a reusable snippet at the **top** of the file, then one block per service. The `handle` +blocks are mutually exclusive and evaluated in order, so the abort reliably pre-empts the +proxy for non-private clients. `private_ranges` is a built-in Caddy matcher token +(Caddy ≥ 2.7). + +```caddyfile +(lan_only) { + @external not remote_ip private_ranges + handle @external { + abort + } +} + +sonarr.dylanbaird.com { + import lan_only + handle { + reverse_proxy sonarr:8989 + } +} + +radarr.dylanbaird.com { + import lan_only + handle { + reverse_proxy radarr:7878 + } +} + +prowlarr.dylanbaird.com { + import lan_only + handle { + reverse_proxy prowlarr:9696 + } +} + +bazarr.dylanbaird.com { + import lan_only + handle { + reverse_proxy bazarr:6767 + } +} + +sabnzbd.dylanbaird.com { + import lan_only + handle { + reverse_proxy sabnzbd:8080 + } +} + +jellyseerr.dylanbaird.com { + reverse_proxy jellyseerr:5055 +} +``` + +**Deploy note:** the Caddyfile is baked into the image `boulangermontrealais/barrios:caddy`, +not bind-mounted. After editing, run `caddy/build.sh` (rebuilds + `docker push`; it also +recompiles the static sites for `dylanbaird.com` / `judo` / `alloro` — harmless), then on +the server `docker compose pull caddy && docker compose up -d caddy`. Alternative: uncomment +the `build:` block already present in the `caddy` service and use +`docker compose up -d --build caddy`. + +**Stricter variant:** to avoid the five admin hostnames existing in public DNS / getting +public certs at all, add `tls internal` inside each of their blocks and skip their public +DNS records (browsers then need Caddy's local root trusted). + +### Step 5 — Bring it up + +```sh +docker compose config # parses cleanly +docker compose up -d # starts the six new containers +docker compose ps # all healthy, no port-bind errors +``` + +--- + +## Per-app configuration (through each web UI) + +Order matters. Grab API keys from each app's *Settings → General*. + +1. **SABnzbd** (`sabnzbd.dylanbaird.com`): run the wizard. Add the Usenet provider + (host, port `563`, SSL on, username/password, connections per the provider's limit). + *Config → Folders*: incomplete `/data/downloads/usenet/incomplete`, completed + `/data/downloads/usenet`. *Config → Categories*: `movies` → `/data/downloads/usenet/movies`, + `tv` → `/data/downloads/usenet/tv`. Set a fixed API key. +2. **Prowlarr** (`prowlarr.dylanbaird.com`): add your Usenet indexer(s) with their API + keys and Test. *Settings → Apps*: add Sonarr (`http://sonarr:8989`) and Radarr + (`http://radarr:7878`) with their API keys; set the Prowlarr server URL to + `http://prowlarr:9696`. Sync — indexers propagate to both apps. +3. **Radarr** (`radarr.dylanbaird.com`): *Settings → Download Clients* → add SABnzbd + (`http://sabnzbd:8080`, API key, category `movies`). *Media Management* → add root + folder `/data/movies`; enable *Use Hardlinks instead of Copy*. Pick a 1080p quality + profile. *Settings → Connect* → add **Jellyfin** (`http://jellyfin:8096` + a Jellyfin + API key) so imports trigger a library scan. +4. **Sonarr** (`sonarr.dylanbaird.com`): same as Radarr but download-client category `tv`, + root folder `/data/television`, and the Jellyfin *Connect* entry. +5. **Bazarr** (`bazarr.dylanbaird.com`): connect Sonarr (`http://sonarr:8989`) and Radarr + (`http://radarr:7878`) with API keys and path `/data`. *Languages*: add **English** and + **French**, create a language profile requiring both, set it as the default for Series + and Movies. Add subtitle providers (OpenSubtitles.com account, Podnapisi, etc.). + Optionally set a minimum score and enable "upgrade subtitles". +6. **Jellyseerr** (`jellyseerr.dylanbaird.com`): sign in with the Jellyfin admin account; + point it at Jellyfin, then add Radarr and Sonarr (URLs, API keys, default root folders / + quality profiles). Add other users' Jellyfin accounts; either enable auto-approve for + trusted users (removes the manual approval step) or leave approval on with a + notification agent (email / Discord / ntfy). +7. **Jellyfin**: no config change required. Confirm the Radarr/Sonarr *Connect* triggers + scans; otherwise the existing scheduled scan still catches new files. Subtitle files + land next to the video and are selectable at playback. + +--- + +## Verification (end to end) + +1. `docker compose ps` — all six containers healthy, no port conflicts. +2. **LAN restriction:** from a LAN browser, `https://sonarr.dylanbaird.com` loads with a + valid cert. From off-LAN (phone on cellular) the same URL fails to connect. + `https://jellyseerr.dylanbaird.com` loads from **both**. + - If off-LAN also loads the admin UIs, the `remote_ip` match is seeing Docker's bridge + gateway instead of the real client — set `"userland-proxy": false` in + `/etc/docker/daemon.json` and restart Docker. +3. Prowlarr indexer **Test** is green; "Sync App Indexers" populates Sonarr & Radarr. +4. SABnzbd **Test Server** for the provider is green. +5. In Radarr, add one movie → *Search* → SABnzbd shows the download → on completion Radarr + imports it into `/mertz/data/movies/ (Year)/…` with correct naming. Confirm the + import was a hardlink/instant move: `ls -li` the source in `downloads/usenet/movies` and + the imported file share an inode, or Radarr's log says "Hardlinking" / "moved". +6. Repeat with one episode in Sonarr → lands in `/mertz/data/television/…`. +7. Bazarr shows the new item and downloads `….en.srt` and `….fr.srt` beside the video + (run "Search" manually the first time). +8. The title appears in Jellyfin within a minute (Connect scan) or after the next + scheduled scan; both subtitle tracks are offered during playback. +9. In Jellyseerr, sign in as a non-admin account, submit a request → it appears in + Radarr/Sonarr, downloads, and shows up in Jellyfin, with no manual step. + +--- + +## Risks / watch-items + +- **mergerfs hardlinks** — without a path-preserving policy, imports become in-pool copies: + functional, slower, temporary ~2× space per item. +- **Docker source-IP preservation** for the `lan_only` matcher — verify step 2 above + immediately; a wrong result either exposes the admin UIs publicly or 403s LAN users. +- **`latest` image tags** — `docker compose pull` could pull a breaking major. Consider + pinning and backing up `/mertz/*/config` before pulls. +- **Jellyseerr is the one new internet-facing surface** — strong admin password, keep the + image updated, consider fail2ban / Caddy rate-limiting later. +- **`caddy/build.sh` side-effects** — it also rebuilds the static sites; make sure that + source is in a good state before running. +- **Existing library ownership** — root-owned files from the manual era must be chowned to + uid 1000 or the *arr apps can't rename / upgrade / delete existing items.