Remove YouTube integration from backend and deployment config

Drops the YouTube-backed second catalog source entirely: the
Local/External split on Track/Album collapses into single concrete
entities, Artist/TrackArtist/ExternalTrack/ExternalAlbum and the
YouTube service/download pipeline (yt-dlp, MassTransit consumer,
routing slip) are deleted, and mixes now source discovery only from
local listening history. Includes the RemoveYouTubeIntegration EF
Core migration and matching Worker/deploy/compose cleanup (no more
yt-dlp/python/deno in the Worker image, no thumbnail-cache/YtDlp
config sections).
This commit is contained in:
2026-09-15 21:17:03 +02:00
parent 5696683859
commit 31032bd8e8
137 changed files with 1933 additions and 3969 deletions
-5
View File
@@ -63,8 +63,3 @@ AUTH_ALLOW_INSECURE_HTTP=false
AUTH_REQUIRE_HTTPS=true
# Largest request body nginx accepts (audio uploads go through the web player).
MAX_UPLOAD_SIZE=1024m
# Disk nginx gives to the cached YouTube artwork it serves for the web player.
THUMBNAIL_CACHE_SIZE=2g
# Extra yt-dlp flags for the Worker. The cookie jar avoids YouTube's bot checks;
# drop it here if you do not have one: deploy/secrets/ is mounted at /secrets.
YTDLP_ADDITIONAL_ARGUMENTS=--cookies /secrets/youtube_cookies.txt --extractor-retries 1 --socket-timeout 15 -N 4
+5 -13
View File
@@ -7,7 +7,7 @@ same set of compose files — no wrapper scripts, just `docker compose`.
|---|---|
| `compose.yml` | Every service, including the one-shot bootstrap jobs. No published ports, no domains, no environment-specific values. |
| `compose.dev.yml` | Development overlay: publishes the ports to the host, adds pgAdmin. |
| `compose.prod.yml` | Production overlay: adds the nginx edge and the mounted secrets. |
| `compose.prod.yml` | Production overlay: adds the nginx edge. |
| `.env` / `.env.prod` | The values compose interpolates. Not versioned; templates in `.env.example` / `.env.prod.example`. |
The four applications (`api`, `worker`, `gateway`, `web-player`) sit behind the
@@ -76,9 +76,9 @@ Add `--profile tools` to also start pgAdmin.
| pgAdmin (`--profile tools`) | `5050` | Postgres UI |
| API, gateway, web player (`--profile apps`) | `5111` / `8081` / `3000` | |
Host requirements: .NET 10 SDK, Node 22, Docker Desktop. (ffmpeg and yt-dlp
are only needed on the host if you run the Worker outside Docker — inside
Docker its image already has them.)
Host requirements: .NET 10 SDK, Node 22, Docker Desktop. (ffmpeg is only
needed on the host if you run the Worker outside Docker — inside Docker its
image already has it.)
### Why `host.docker.internal`
@@ -136,13 +136,6 @@ infra containers and only rebuilds and restarts the apps).
Only nginx publishes ports (80/443); everything else is reachable only on the
internal compose network.
### Optional secrets
`deploy/secrets/` is mounted read-only at `/secrets` in the Worker. Drop a
`youtube_cookies.txt` there if you want yt-dlp to use a cookie jar — it is
referenced from `YTDLP_ADDITIONAL_ARGUMENTS` in `.env.prod`, which you can
empty if you do not have one.
---
## Layout
@@ -151,10 +144,9 @@ empty if you do not have one.
deploy/
├─ compose.yml # all services, incl. keys-init / migrate / zitadel-init
├─ compose.dev.yml # dev overlay: host ports, pgAdmin
├─ compose.prod.yml # prod overlay: nginx edge, secrets
├─ compose.prod.yml # prod overlay: nginx edge
├─ .env.example / .env.prod.example
├─ keys/ # RS256 stream-ticket key pair (generated)
├─ secrets/ # mounted into the Worker (prod, optional)
├─ nginx/
│ ├─ templates/default.conf.template # vhosts, rendered with the domain by envsubst
│ ├─ snippets/proxy.conf # proxy headers shared by every vhost
-6
View File
@@ -13,11 +13,6 @@ name: musify-prod
# the first command just wrote into deploy/.env.prod. See deploy/README.md.
services:
worker:
# yt-dlp reads the YouTube cookie jar from here (see YTDLP_ADDITIONAL_ARGUMENTS).
volumes:
- ./secrets:/secrets:ro
# zitadel-init (compose.yml) edits whichever env file is mounted here.
zitadel-init:
volumes:
@@ -40,7 +35,6 @@ services:
# The filter keeps it from eating nginx's own $host / $remote_addr.
MUSIFY_ROOT_DOMAIN: ${ROOT_DOMAIN}
MUSIFY_MAX_UPLOAD_SIZE: ${MAX_UPLOAD_SIZE:-1024m}
MUSIFY_THUMBNAIL_CACHE_SIZE: ${THUMBNAIL_CACHE_SIZE:-2g}
NGINX_ENVSUBST_FILTER: ^MUSIFY_
ports:
- "80:80"
-1
View File
@@ -447,7 +447,6 @@ services:
DOTNET_ENVIRONMENT: ${DOTNET_ENVIRONMENT}
AudioTranscoder__Routes__WorkingDirectory: /tmp/musify
Workers__Routes__TemporaryFilesDirectory: /tmp/musify
YtDlp__AdditionalArguments: ${YTDLP_ADDITIONAL_ARGUMENTS:-}
restart: unless-stopped
# YARP in front of the SeaweedFS filer: validates the RS256 stream ticket and
@@ -9,12 +9,6 @@
resolver 127.0.0.11 valid=10s;
# YouTube artwork the web player proxies from Google. Caching it on disk keeps
# repeated searches off both the node process and Google, which answers 429
# once a client pulls a whole grid of covers at a time.
proxy_cache_path /var/cache/nginx/thumbnails levels=1:2 keys_zone=thumbnails:10m
max_size=${MUSIFY_THUMBNAIL_CACHE_SIZE} inactive=30d use_temp_path=off;
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
@@ -40,21 +34,6 @@ server {
ssl_certificate /etc/nginx/certs/origin.pem;
ssl_certificate_key /etc/nginx/certs/origin.key;
location = /api/youtube/thumbnail {
set $upstream http://web-player:3000;
proxy_pass $upstream;
include /etc/nginx/snippets/proxy.conf;
proxy_cache thumbnails;
proxy_cache_key $uri$is_args$args;
proxy_cache_valid 200 30d;
proxy_cache_valid 400 404 5m;
proxy_cache_lock on;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_cache_background_update on;
add_header X-Cache-Status $upstream_cache_status;
}
location / {
set $upstream http://web-player:3000;
proxy_pass $upstream;