NotS1mp/Musify
0
0
Fork 0
Code Issues 23 Pull Requests Actions Packages Projects Releases Wiki Activity

Replace deploy scripts with self-contained compose bootstrap jobs

Browse Source 
Removes every wrapper script under deploy/ (up.ps1, up.sh, scripts/*,
zitadel/provision.*) and folds their logic into three one-shot services
in compose.yml, following the pattern already used by seaweedfs-init:

  keys-init     generates the RS256 stream-ticket key pair
  migrate       applies the EF Core migrations (backend/Musify.Migrator.Dockerfile)
  zitadel-init  creates the Musify project + OIDC apps, writes the client
                ids back into the active env file

Deploying is now two plain `docker compose` commands, no scripts to run
or chmod:

  docker compose -f deploy/compose.yml -f deploy/compose.dev.yml up -d
  docker compose -f deploy/compose.yml -f deploy/compose.dev.yml --profile apps up -d --build

(swap compose.dev.yml for compose.prod.yml + --env-file deploy/.env.prod
for production). The first command brings up infra and lets the bootstrap
jobs run to completion; the second builds and starts the apps, which by
then read real OIDC values instead of empty ones — compose interpolates
${VAR} once per invocation, so this can't collapse into a single command
without the apps reading their own config file at container start instead
of via compose env (out of scope here).

Also:
- DatabaseDesignTimeFactory now also reads Database__ConnectionString
  from the environment, and DesignSettings.json is optional, so the new
  migrate container works without user-secrets or a config file excluded
  from the Docker build context by .dockerignore.
- nginx's depends_on no longer lists the apps-profile services: it does
  not need them at its own startup (every vhost resolves its upstream at
  request time via the embedded Docker resolver), and requiring them made
  `compose.prod.yml up -d` (without --profile apps) an invalid project.
- env_get/env_set in zitadel-init strip \r: the mounted env file can have
  CRLF line endings on Windows, which otherwise makes an empty-looking
  value non-empty to `[ -z ... ]`. .env.example/.env.prod.example are
  pinned to LF via .gitattributes for the same reason. Also fixed:
  rewriting the bind-mounted env file in place instead of renaming a temp
  file over it (rename onto a bind-mounted inode fails with "Resource
  busy").
- Removed compose.share.yml and zitadel/.output-share/ — a "temporary"
  personal overlay that had leaked a real Zitadel service-account PAT
  into git history (committed in 70968b3). That token should be rotated.
- Removed deploy/.tmp/ from .gitignore: nothing mounts or creates it
  anymore (it never actually did, either — no compose service referenced
  it, only the now-removed up.ps1 did).

Verified end-to-end with Docker: a clean `down -v` + fresh `up -d` +
`--profile apps up -d --build` on the dev stack brings up all four one-shot
jobs successfully and all four apps healthy (api/gateway return 200,
worker starts its consumers with no errors, web-player responds). Also
verified compose.prod.yml starts nginx cleanly with dummy certs and no
apps running, confirming the depends_on fix.
This commit is contained in:
Admin
2026-09-09 15:08:34 +02:00
parent 89569736b7
commit cf8ac706b6
26 changed files with 408 additions and 862 deletions
Download Patch File Download Diff File Expand all files Collapse all files
deploy/.env.example
+8 -3
View File
@@ -1,5 +1,7 @@
# Development configuration for deploy/compose.yml + compose.dev.yml.
# Copy to deploy/.env (up.ps1 does it for you) and adjust. Not versioned.
# Copy to deploy/.env and adjust. Not versioned.
#
# cp deploy/.env.example deploy/.env
DOTNET_ENVIRONMENT=Development
@@ -22,7 +24,7 @@ POSTGRES_HOST_PORT=59000
# --- Zitadel (OIDC) ----------------------------------------------------------
# The masterkey MUST be exactly 32 characters. The external domain/port form
# the token issuer, so they have to match PUBLIC_AUTH_URL. Changing them needs
# a re-init: ./up.ps1 -Destroy; ./up.ps1
# a re-init: docker compose -f deploy/compose.yml -f deploy/compose.dev.yml down -v
ZITADEL_MASTERKEY=MasterkeyNeedsToHave32Characters
ZITADEL_EXTERNAL_DOMAIN=host.docker.internal
ZITADEL_EXTERNAL_PORT=8080
@@ -31,8 +33,11 @@ ZITADEL_DATABASE_PASSWORD=zitadel
# Console login: <username>@zitadel.<external domain>
ZITADEL_ADMIN_USERNAME=admin
ZITADEL_ADMIN_PASSWORD=Password1!
# Allows the OIDC apps to use plain-http redirect URIs (dev only).
ZITADEL_DEV_MODE=true
# OIDC clients, filled in by deploy/zitadel/provision.ps1.
# OIDC clients: the zitadel-init one-shot service (see compose.yml) fills
# these in automatically on `docker compose up -d`.
AUTH_CLIENT_ID=
WEB_CLIENT_ID=
WEB_CLIENT_SECRET=
deploy/.env.prod.example
+5 -2
View File
@@ -1,5 +1,5 @@
# Production configuration for deploy/compose.yml + compose.prod.yml.
# Copy to deploy/.env.prod (up.sh does it for you), then:
# Copy to deploy/.env.prod, then:
#
# 1. replace every `example.com` with your domain,
# 2. replace every CHANGE_ME with a strong random value
@@ -35,8 +35,11 @@ ZITADEL_DATABASE_PASSWORD=CHANGE_ME
# Console login: <username>@zitadel.<external domain>
ZITADEL_ADMIN_USERNAME=admin
ZITADEL_ADMIN_PASSWORD=CHANGE_ME
# Keep OIDC apps strict about https redirect URIs in production.
ZITADEL_DEV_MODE=false
# OIDC clients, filled in by deploy/zitadel/provision.sh.
# OIDC clients: the zitadel-init one-shot service (see compose.yml) fills
# these in automatically on `docker compose up -d`.
AUTH_CLIENT_ID=
WEB_CLIENT_ID=
WEB_CLIENT_SECRET=
deploy/README.md
+88 -60
View File
@@ -1,44 +1,68 @@
# Deploy
Everything needed to run Musify, in development and in production, from the
same set of compose files.
same set of compose files — no wrapper scripts, just `docker compose`.
| File | Role |
|---|---|
| `compose.yml` | Every service. No published ports, no domains, no environment-specific values. |
| `compose.yml` | Every service, including the one-shot bootstrap jobs. No published ports, no domains, no environment-specific values. |
| `compose.dev.yml` | Development overlay: publishes the ports to the host, adds pgAdmin. |
| `compose.prod.yml` | Production overlay: adds the nginx edge and the mounted secrets. |
| `.env` / `.env.prod` | The values compose interpolates. Not versioned; templates in `.env.example` / `.env.prod.example`. |
| `up.ps1` | Development orchestrator (Windows). |
| `up.sh` | Production orchestrator (the server). |
The four applications (`api`, `worker`, `gateway`, `web-player`) sit behind the
`apps` compose profile, so a plain `up` starts infrastructure only — the usual
development setup, with the apps running from the IDE.
### The one-shot bootstrap jobs
Three services in `compose.yml` run once and exit, so a plain `docker compose
up -d` leaves the stack fully ready with no manual steps:
| Service | Does |
|---|---|
| `keys-init` | Generates the RS256 stream-ticket key pair in `./keys`, if it is not there already. |
| `migrate` | Applies the EF Core migrations (built from `backend/Musify.Migrator.Dockerfile`). |
| `zitadel-init` | Creates the Musify project and its two OIDC apps in Zitadel (or reuses them), then writes `AUTH_CLIENT_ID` / `WEB_CLIENT_ID` / `WEB_CLIENT_SECRET` into the env file. |
`api` and `worker` wait for `migrate`; `api` and `gateway` wait for
`keys-init` (`depends_on: condition: service_completed_successfully`). They do
**not** wait for `zitadel-init` — compose interpolates `${AUTH_CLIENT_ID}` once,
when a command starts, so a container that starts in the *same* `up` as
`zitadel-init` would still get the empty value it had before provisioning ran.
That's why bringing up the apps is a second command: by the time you run it,
`zitadel-init` has already finished and the env file has real values.
---
## Development
```powershell
./deploy/up.ps1 # infrastructure only, apps from the IDE
./deploy/up.ps1 -Apps # build and run the apps in Docker too
```sh
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml up -d
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml --profile apps up -d --build
```
`up.ps1` does the whole bootstrap, with no manual steps afterwards:
The first command starts the infrastructure and runs the bootstrap jobs
(keys, migrations, Zitadel project/apps). The second builds and runs `api`,
`worker`, `gateway` and `web-player` in Docker too — skip it to run the apps
from the IDE instead (the usual dev loop):
1. Creates `deploy/.env` from `.env.example` if it is missing.
2. Generates the RS256 stream-ticket keys in `deploy/keys/` and the Worker
scratch directory `deploy/.tmp/`.
3. Starts the stack and waits for Postgres.
4. Applies the EF Core migrations.
5. Provisions Zitadel: project, API OIDC app and web player OIDC app, then
writes the client ids into `deploy/.env`, `Musify.Api` user-secrets and
`web-player/.env`.
6. Prints a summary with every URL and credential.
```sh
dotnet run --project backend/Hosts/Musify.Api # :5111
dotnet run --project backend/Hosts/Musify.StreamingGateway # :8081
dotnet run --project backend/Hosts/Musify.Worker # background
npm --prefix web-player run dev # :5173
```
Other switches: `-Tools` (pgAdmin), `-SkipMigrations`, `-Down` (stop, keep
data), `-Destroy` (stop and wipe the volumes).
When running the apps from the IDE, `Musify.Api` reads `Authentication:*`
from user-secrets and `web-player` reads its own `web-player/.env` — neither
looks at `deploy/.env`. Copy the three values `zitadel-init` wrote
(`AUTH_CLIENT_ID`, `WEB_CLIENT_ID`, `WEB_CLIENT_SECRET`) into
`dotnet user-secrets set Authentication:ClientId <value> --project
backend/Hosts/Musify.Api` and into `web-player/.env` once; they stay valid
until you wipe the Zitadel volume.
Add `--profile tools` to also start pgAdmin.
### Ports
@@ -49,20 +73,12 @@ data), `-Destroy` (stop and wipe the volumes).
| RabbitMQ | `5672` / `15672` | MassTransit / management UI |
| SeaweedFS | `8333` / `8888` / `9333` | S3 / filer (→ gateway) / master |
| Jaeger | `16686` / `4317` / `4318` | traces UI / OTLP gRPC / OTLP HTTP |
| pgAdmin (`-Tools`) | `5050` | Postgres UI |
| API, gateway, web player (`-Apps`) | `5111` / `8081` / `3000` | |
| pgAdmin (`--profile tools`) | `5050` | Postgres UI |
| API, gateway, web player (`--profile apps`) | `5111` / `8081` / `3000` | |
### Running the apps from the IDE
```powershell
dotnet run --project backend/Hosts/Musify.Api # :5111
dotnet run --project backend/Hosts/Musify.StreamingGateway # :8081
dotnet run --project backend/Hosts/Musify.Worker # background
npm --prefix web-player run dev # :5173
```
Host requirements: .NET 10 SDK, Node 22, ffmpeg and yt-dlp on PATH (the Worker
transcodes and downloads audio), Docker Desktop.
Host requirements: .NET 10 SDK, Node 22, Docker Desktop. (ffmpeg and yt-dlp
are only needed on the host if you run the Worker outside Docker — inside
Docker its image already has them.)
### Why `host.docker.internal`
@@ -71,36 +87,51 @@ resolves to the same address from the browser, from apps running on the host
and from inside the containers. That keeps the OIDC token issuer and the
presigned S3 URLs valid whichever way the apps are running. Zitadel routes by
the `Host` header, so the domain has to be identical everywhere; changing it
requires a re-init (`./up.ps1 -Destroy; ./up.ps1`).
requires wiping the Postgres/Zitadel volumes and starting over (`docker
compose -f compose.yml -f compose.dev.yml down -v`, then `up -d` again).
### Resetting
```sh
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml down # stop, keep data
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml down -v # stop, wipe volumes
```
---
## Production
One command on the server, from the repository root:
On a server with a domain, from the repository root:
```sh
./deploy/up.sh
cp deploy/.env.prod.example deploy/.env.prod
```
It refuses to start until the two things it cannot generate are in place:
Fill it in:
1. **`deploy/.env.prod`** — copied from `.env.prod.example` on the first run.
Replace every `example.com` with your domain and every `CHANGE_ME` with a
random value (`openssl rand -hex 24`; the Zitadel masterkey needs exactly 32
characters). `chmod 600` it.
2. **`deploy/nginx/certs/origin.pem` and `origin.key`** — the origin
1. Replace every `example.com` with your domain and every `CHANGE_ME` with a
random value (`openssl rand -hex 24`; the Zitadel masterkey needs exactly
32 characters). `chmod 600 deploy/.env.prod`.
2. Drop **`deploy/nginx/certs/origin.pem` and `origin.key`** — the origin
certificate nginx serves. The stack assumes Cloudflare in *Full (strict)*
mode in front of it, so a Cloudflare Origin Certificate is enough; any
certificate valid for the five hostnames works.
3. Point `example.com`, `www`, `api`, `auth`, `stream` and `s3` at the server,
proxied/TLS-terminated the same way.
DNS: point `example.com`, `www`, `api`, `auth`, `stream` and `s3` at the server.
Then, same two commands as development, with the prod env file and overlay:
Then `up.sh` generates the stream keys, starts the infrastructure and the edge,
provisions Zitadel (it needs `curl` and `jq` on the server), applies the
migrations and finally builds and starts the applications. It is idempotent —
run it again to deploy a new version, or `./deploy/up.sh --no-build` to restart
without rebuilding.
```sh
docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml up -d
docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml --profile apps up -d --build
```
The first command starts the infrastructure, the nginx edge and the bootstrap
jobs; the second builds and starts the applications, already wired to the
OIDC client ids the first one produced. Both are idempotent — run them again
to deploy a new version (`docker compose ... --profile apps up -d --build`
alone is enough for an update once the stack already exists; it reuses the
infra containers and only rebuilds and restarts the apps).
Only nginx publishes ports (80/443); everything else is reachable only on the
internal compose network.
@@ -118,30 +149,23 @@ empty if you do not have one.
```
deploy/
├─ compose.yml # all services, environment-independent
├─ compose.yml # all services, incl. keys-init / migrate / zitadel-init
├─ compose.dev.yml # dev overlay: host ports, pgAdmin
├─ compose.prod.yml # prod overlay: nginx edge, secrets
├─ .env.example / .env.prod.example
├─ up.ps1 # dev orchestrator
├─ up.sh # prod orchestrator
├─ keys/ # RS256 stream-ticket key pair (generated)
├─ secrets/ # mounted into the Worker (prod, optional)
├─ nginx/
│ ├─ templates/default.conf.template # vhosts, rendered with the domain by envsubst
│ ├─ snippets/proxy.conf # proxy headers shared by every vhost
│ └─ certs/ # origin.pem + origin.key (not versioned)
├─ scripts/
│ ├─ common.ps1 / lib.sh # shared helpers
│ ├─ stream-keys.ps1 / .sh # RS256 key pair generation
│ └─ migrate.ps1 / .sh # EF Core migrations
└─ zitadel/
├─ provision.ps1 / .sh # project + OIDC apps, wires the client ids
└─ .output/ # PAT written by Zitadel on init (not versioned)
└─ .output/ # PAT written by Zitadel on init (not versioned)
```
The application images are built from `backend/Musify.*.Dockerfile` (build
context `backend/`, so restore can copy the central props and every csproj) and
`web-player/Dockerfile`.
context `backend/`, so restore can copy the central props and every csproj)
and `web-player/Dockerfile`.
## Notes
@@ -151,8 +175,12 @@ context `backend/`, so restore can copy the central props and every csproj) and
- **Zitadel** runs `start-from-init`, which is idempotent: the same command
works on an empty volume and on an existing one.
- **`latest` Zitadel images** require the standalone Login UI v2 container,
which this stack does not run. The provisioning scripts turn that requirement
off so the built-in `/ui/login` is used.
which this stack does not run. `zitadel-init` turns that requirement off so
the built-in `/ui/login` is used.
- **`AppSettings*.json` are PascalCase** in `Musify.Api` and `Musify.Worker`;
on case-sensitive Linux the host looks for `appsettings*.json`, so their
Dockerfiles create lowercase copies at publish time.
- **Re-running `zitadel-init` or `migrate`** is safe: the former reuses the
existing project/apps by name (regenerating the web app's secret only if it
is missing from the env file), the latter is a normal `dotnet ef database
update`, a no-op once the schema is current.
deploy/compose.dev.yml
+5
View File
@@ -40,6 +40,11 @@ services:
- "4317:4317" # OTLP gRPC
- "4318:4318" # OTLP HTTP
# zitadel-init (compose.yml) edits whichever env file is mounted here.
zitadel-init:
volumes:
- ./.env:/workspace/.env
# Optional Postgres UI: docker compose ... --profile tools up -d
pgadmin:
profiles: [tools]
deploy/compose.prod.yml
+19 -8
View File
@@ -4,11 +4,13 @@ name: musify-prod
# Nothing but nginx publishes a port; every other service is reachable only on
# the internal `musify-prod_default` network.
#
# docker compose --env-file deploy/.env.prod \
# -f deploy/compose.yml -f deploy/compose.prod.yml --profile apps up -d --build
# docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml up -d
# docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml --profile apps up -d --build
#
# Use ./up.sh, which also generates the stream keys, provisions Zitadel and
# applies the database migrations.
# The first command starts the infrastructure, the nginx edge and the
# one-shot bootstrap jobs (stream keys, migrations, Zitadel project/apps); the
# second builds and starts the applications, picking up the OIDC client ids
# the first command just wrote into deploy/.env.prod. See deploy/README.md.
services:
worker:
@@ -16,14 +18,23 @@ services:
volumes:
- ./secrets:/secrets:ro
# zitadel-init (compose.yml) edits whichever env file is mounted here.
zitadel-init:
volumes:
- ./.env.prod:/workspace/.env
# Every vhost proxies through `set $upstream ...; proxy_pass $upstream;`
# with the embedded Docker DNS resolver (see nginx/templates), so nginx
# does not need api/gateway/web-player to exist at its own startup — only
# at request time. That is what lets it start in the FIRST command
# (profile-gated services aren't part of that invocation yet) and still be
# reachable at the auth domain for zitadel-init to provision through.
nginx:
image: nginx:1.27-alpine
container_name: musify-nginx
depends_on:
- api
- gateway
- zitadel
- web-player
zitadel:
condition: service_started
environment:
# The entrypoint renders /etc/nginx/templates/*.template with envsubst.
# The filter keeps it from eating nginx's own $host / $remote_addr.
deploy/compose.share.yml
-15
View File
@@ -1,15 +0,0 @@
name: musify-share
# Temporary overlay for exposing the stack outside the LAN, on top of
# compose.dev.yml. Only difference from a normal dev run: Zitadel writes its
# init PAT to a separate host directory, so it never overwrites
# zitadel/.output (used by the musify-dev project's own Zitadel instance).
#
# docker compose --env-file deploy/.env.share \
# -f deploy/compose.yml -f deploy/compose.dev.yml -f deploy/compose.share.yml \
# --profile apps up -d --build api gateway web-player
services:
zitadel:
volumes:
- ./zitadel/.output-share:/output
deploy/compose.yml
+209 -7
View File
@@ -8,9 +8,12 @@
# prod docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml up -d
#
# The four application services sit behind the `apps` profile, so a plain `up`
# starts only the infrastructure (the usual dev setup: apps from the IDE).
# Add `--profile apps` to run them in Docker. Use ./up.ps1 (dev) or ./up.sh
# (prod) instead of calling compose by hand.
# starts the infrastructure plus the one-shot bootstrap jobs (stream-ticket
# keys, database migrations, Zitadel project/apps) and stops there — the usual
# dev setup, with the apps running from the IDE. Add `--profile apps up -d
# --build` as a SECOND command to also build and run them in Docker: by then
# zitadel-init has already written the OIDC client ids into the env file, so
# the apps start with real values. See deploy/README.md.
x-app-depends: &app-depends
postgres:
@@ -168,6 +171,196 @@ services:
aws --endpoint-url http://seaweedfs:8333 s3 ls
restart: "no"
# One-shot: generates the RS256 key pair used to sign (API) and verify
# (Gateway) stream tickets, if it is not already in ./keys. No host tools
# required — openssl runs inside the container.
keys-init:
image: alpine:3.20
container_name: musify-keys-init
entrypoint: ["/bin/sh", "-c"]
command:
- |
set -e
if [ -f /keys/stream_private.pem ] && [ -f /keys/stream_public.pem ]; then
echo "stream-ticket keys already present"
else
apk add --no-cache openssl >/dev/null
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out /keys/stream_private.pem
openssl rsa -in /keys/stream_private.pem -pubout -out /keys/stream_public.pem
chmod 644 /keys/stream_public.pem
chmod 600 /keys/stream_private.pem
echo "generated stream-ticket keys"
fi
volumes:
- ./keys:/keys
restart: "no"
# One-shot: applies the EF Core migrations, then exits. See
# backend/Musify.Migrator.Dockerfile.
migrate:
build:
context: ../backend
dockerfile: Musify.Migrator.Dockerfile
container_name: musify-migrate
depends_on:
postgres:
condition: service_healthy
environment:
Database__ConnectionString: "Host=postgres;Port=5432;Database=${POSTGRES_DB};Username=${POSTGRES_USER};Password=${POSTGRES_PASSWORD}"
restart: "no"
# One-shot: creates the Musify project and its two OIDC apps in Zitadel (or
# reuses them if they already exist), then writes AUTH_CLIENT_ID,
# WEB_CLIENT_ID and WEB_CLIENT_SECRET back into the env file mounted at
# /workspace/.env (see compose.dev.yml / compose.prod.yml) — so a later
# `--profile apps up` picks up real values instead of empty ones.
zitadel-init:
image: alpine:3.20
container_name: musify-zitadel-init
depends_on:
zitadel:
condition: service_started
environment:
PUBLIC_AUTH_URL: ${PUBLIC_AUTH_URL}
PUBLIC_API_URL: ${PUBLIC_API_URL}
PUBLIC_WEB_URL: ${PUBLIC_WEB_URL}
ZITADEL_DEV_MODE: ${ZITADEL_DEV_MODE:-false}
volumes:
- ./zitadel/.output:/output:ro
entrypoint: ["/bin/sh", "-c"]
command:
- |
set -e
apk add --no-cache curl jq >/dev/null
ENV_FILE=/workspace/.env
if [ ! -f "$$ENV_FILE" ]; then
echo "no env file mounted at $$ENV_FILE, skipping"
exit 0
fi
# The env file may have CRLF line endings (edited on Windows), so
# strip \r on read and drop it on write — otherwise a "blank" value
# is actually a lone \r, and `[ -z ... ]` lies.
env_get() { grep -E "^$$2=" "$$1" 2>/dev/null | tail -n1 | cut -d= -f2- | tr -d '\r'; }
env_set() {
# $$1 is a bind-mounted file: rewrite its content in place rather
# than renaming a temp file over it (rename onto a bind-mounted
# inode fails with "Resource busy").
if grep -qE "^$$2=" "$$1"; then
awk -v k="$$2" -v v="$$3" '{ sub(/\r$$/, "") } index($$0,k "=")==1{print k "=" v; next} {print}' "$$1" >"$$1.tmp" && cat "$$1.tmp" >"$$1" && rm -f "$$1.tmp"
else
printf '%s=%s\n' "$$2" "$$3" >>"$$1"
fi
}
echo "==> waiting for the Zitadel service-account PAT"
deadline=$$(( $$(date +%s) + 180 ))
while [ ! -f /output/admin-sa.pat ]; do
[ "$$(date +%s)" -lt "$$deadline" ] || { echo "PAT not found, check 'docker logs musify-zitadel'" >&2; exit 1; }
sleep 3
done
PAT=$$(tr -d '\r\n' </output/admin-sa.pat)
MGMT="$$PUBLIC_AUTH_URL/management/v1"
echo "==> waiting for Zitadel to answer at $$PUBLIC_AUTH_URL"
deadline=$$(( $$(date +%s) + 120 ))
until curl -fsS "$$PUBLIC_AUTH_URL/.well-known/openid-configuration" >/dev/null 2>&1; do
[ "$$(date +%s)" -lt "$$deadline" ] || { echo "Zitadel did not respond at $$PUBLIC_AUTH_URL" >&2; exit 1; }
sleep 3
done
api_call() {
if [ $$# -ge 3 ]; then
curl -fsS -X "$$1" "$$MGMT$$2" -H "Authorization: Bearer $$PAT" -H "Content-Type: application/json" -d "$$3"
else
curl -fsS -X "$$1" "$$MGMT$$2" -H "Authorization: Bearer $$PAT"
fi
}
name_query() { printf '{"queries":[{"nameQuery":{"name":"%s","method":"TEXT_QUERY_METHOD_EQUALS"}}]}' "$$1"; }
# `latest` images require the standalone Login UI v2 container, which
# this stack does not run; opting out keeps the built-in /ui/login.
curl -fsS -X PUT "$$PUBLIC_AUTH_URL/v2/features/instance" \
-H "Authorization: Bearer $$PAT" -H "Content-Type: application/json" \
-d '{"loginV2":{"required":false}}' >/dev/null 2>&1 || echo "warning: could not disable Login UI v2"
echo "==> project"
PROJECT_NAME=Musify
PROJECT_ID=$$(api_call POST /projects/_search "$$(name_query "$$PROJECT_NAME")" | jq -r --arg n "$$PROJECT_NAME" '.result // [] | map(select(.name==$$n)) | first | .id // empty')
if [ -z "$$PROJECT_ID" ]; then
PROJECT_ID=$$(api_call POST /projects "$$(printf '{"name":"%s"}' "$$PROJECT_NAME")" | jq -r '.id')
echo " created (id $$PROJECT_ID)"
else
echo " reused (id $$PROJECT_ID)"
fi
find_app() { api_call POST "/projects/$$PROJECT_ID/apps/_search" "$$(name_query "$$1")" | jq -r --arg n "$$1" '.result // [] | map(select(.name==$$n)) | first | .id // empty'; }
echo "==> API app"
API_APP_NAME="Musify API (Scalar)"
API_APP_ID=$$(find_app "$$API_APP_NAME")
if [ -n "$$API_APP_ID" ]; then
API_CLIENT_ID=$$(api_call GET "/projects/$$PROJECT_ID/apps/$$API_APP_ID" | jq -r '.app.oidcConfig.clientId')
echo " reused (clientId $$API_CLIENT_ID)"
else
API_CLIENT_ID=$$(api_call POST "/projects/$$PROJECT_ID/apps/oidc" "$$(jq -n --arg name "$$API_APP_NAME" --arg api "$$PUBLIC_API_URL" '{
name: $$name,
redirectUris: [($$api + "/scalar/"), ($$api + "/scalar/oauth2-redirect.html")],
postLogoutRedirectUris: [($$api + "/scalar/")],
responseTypes: ["OIDC_RESPONSE_TYPE_CODE"],
grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE"],
appType: "OIDC_APP_TYPE_USER_AGENT",
authMethodType: "OIDC_AUTH_METHOD_TYPE_NONE",
version: "OIDC_VERSION_1_0",
devMode: ($$ENV.ZITADEL_DEV_MODE == "true"),
accessTokenType: "OIDC_TOKEN_TYPE_JWT",
accessTokenRoleAssertion: true,
idTokenRoleAssertion: true,
idTokenUserinfoAssertion: true
}')" | jq -r '.clientId')
echo " created (clientId $$API_CLIENT_ID)"
fi
echo "==> web app"
WEB_APP_NAME="Musify Web"
WEB_APP_ID=$$(find_app "$$WEB_APP_NAME")
WEB_CLIENT_SECRET=$$(env_get "$$ENV_FILE" WEB_CLIENT_SECRET)
if [ -n "$$WEB_APP_ID" ]; then
WEB_CLIENT_ID=$$(api_call GET "/projects/$$PROJECT_ID/apps/$$WEB_APP_ID" | jq -r '.app.oidcConfig.clientId')
if [ -z "$$WEB_CLIENT_SECRET" ]; then
WEB_CLIENT_SECRET=$$(api_call POST "/projects/$$PROJECT_ID/apps/$$WEB_APP_ID/oidc_config/_generate_client_secret" '{}' | jq -r '.clientSecret')
echo " reused (clientId $$WEB_CLIENT_ID, secret regenerated)"
else
echo " reused (clientId $$WEB_CLIENT_ID)"
fi
else
CREATED=$$(api_call POST "/projects/$$PROJECT_ID/apps/oidc" "$$(jq -n --arg name "$$WEB_APP_NAME" --arg web "$$PUBLIC_WEB_URL" '{
name: $$name,
redirectUris: [($$web + "/auth/callback")],
postLogoutRedirectUris: [($$web + "/")],
responseTypes: ["OIDC_RESPONSE_TYPE_CODE"],
grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE", "OIDC_GRANT_TYPE_REFRESH_TOKEN"],
appType: "OIDC_APP_TYPE_WEB",
authMethodType: "OIDC_AUTH_METHOD_TYPE_BASIC",
version: "OIDC_VERSION_1_0",
devMode: ($$ENV.ZITADEL_DEV_MODE == "true"),
accessTokenType: "OIDC_TOKEN_TYPE_JWT",
accessTokenRoleAssertion: true,
idTokenRoleAssertion: true,
idTokenUserinfoAssertion: true
}')")
WEB_CLIENT_ID=$$(echo "$$CREATED" | jq -r '.clientId')
WEB_CLIENT_SECRET=$$(echo "$$CREATED" | jq -r '.clientSecret')
echo " created (clientId $$WEB_CLIENT_ID)"
fi
env_set "$$ENV_FILE" AUTH_CLIENT_ID "$$API_CLIENT_ID"
env_set "$$ENV_FILE" WEB_CLIENT_ID "$$WEB_CLIENT_ID"
env_set "$$ENV_FILE" WEB_CLIENT_SECRET "$$WEB_CLIENT_SECRET"
echo "==> wired into $$ENV_FILE"
restart: "no"
# OTLP collector + trace UI (MassTransit / EF Core / HTTP spans).
jaeger:
image: jaegertracing/all-in-one:latest
@@ -184,7 +377,12 @@ services:
context: ../backend
dockerfile: Musify.Api.Dockerfile
container_name: musify-api
depends_on: *app-depends
depends_on:
<<: *app-depends
migrate:
condition: service_completed_successfully
keys-init:
condition: service_completed_successfully
environment:
<<: *backend-env
ASPNETCORE_ENVIRONMENT: ${DOTNET_ENVIRONMENT}
@@ -192,8 +390,7 @@ services:
StreamGateway__PublicBaseUrl: ${PUBLIC_STREAM_URL}
StreamTicket__PrivateKeyPath: /keys/stream_private.pem
Workers__Routes__TemporaryFilesDirectory: /tmp/musify
# Client id and secret are written to the .env file by the Zitadel
# provisioning script (deploy/zitadel/provision.*).
# AUTH_CLIENT_ID is written to the env file by the zitadel-init service.
Authentication__MetadataAddress: "${PUBLIC_AUTH_URL}/.well-known/openid-configuration"
Authentication__IssuerAddress: ${PUBLIC_AUTH_URL}
Authentication__ClientId: ${AUTH_CLIENT_ID}
@@ -215,7 +412,10 @@ services:
context: ../backend
dockerfile: Musify.Worker.Dockerfile
container_name: musify-worker
depends_on: *app-depends
depends_on:
<<: *app-depends
migrate:
condition: service_completed_successfully
environment:
<<: *backend-env
DOTNET_ENVIRONMENT: ${DOTNET_ENVIRONMENT}
@@ -235,6 +435,8 @@ services:
depends_on:
seaweedfs:
condition: service_healthy
keys-init:
condition: service_completed_successfully
environment:
ASPNETCORE_ENVIRONMENT: ${DOTNET_ENVIRONMENT}
ASPNETCORE_URLS: http://+:8081
deploy/scripts/common.ps1
-26
View File
@@ -1,26 +0,0 @@
#requires -Version 5.1
# Shared helpers for the development deploy scripts.
function Write-Step([string]$Message) { Write-Host "`n==> $Message" -ForegroundColor Cyan }
# Parses a .env file into an ordered hashtable (ignores comments and blank lines).
function Read-DotEnv([string]$Path) {
$values = [ordered]@{}
if (Test-Path $Path) {
Get-Content $Path | ForEach-Object {
if ($_ -match '^\s*([^#=\s][^=]*)=(.*)$') { $values[$matches[1].Trim()] = $matches[2].Trim() }
}
}
return $values
}
# Sets (or appends) a key in a .env file, preserving the rest of the file.
function Set-DotEnvValue([string]$Path, [string]$Key, [string]$Value) {
$lines = @(Get-Content $Path)
if ($lines -match "^\s*$Key=") {
$lines = $lines -replace "^\s*$Key=.*", "$Key=$Value"
} else {
$lines += "$Key=$Value"
}
Set-Content -Path $Path -Value $lines -Encoding utf8
}
deploy/scripts/lib.sh
-31
View File
@@ -1,31 +0,0 @@
#!/bin/sh
# Shared helpers for the production deploy scripts.
step() { printf '\n==> %s\n' "$1"; }
fail() { printf 'error: %s\n' "$1" >&2; exit 1; }
require_command() {
command -v "$1" >/dev/null 2>&1 || fail "$1 is required but not installed."
}
# env_get <file> <key> — prints the value, empty if the key is absent.
env_get() {
grep -E "^$2=" "$1" 2>/dev/null | tail -n 1 | cut -d= -f2-
}
# env_set <file> <key> <value> — replaces the key in place, appends if missing.
env_set() {
_file=$1; _key=$2; _value=$3
_tmp=$(mktemp)
if grep -qE "^$_key=" "$_file"; then
awk -v k="$_key" -v v="$_value" '
index($0, k "=") == 1 { print k "=" v; next }
{ print }
' "$_file" >"$_tmp"
else
cp "$_file" "$_tmp"
printf '%s=%s\n' "$_key" "$_value" >>"$_tmp"
fi
cat "$_tmp" >"$_file"
rm -f "$_tmp"
}
deploy/scripts/migrate.ps1
-15
View File
@@ -1,15 +0,0 @@
#requires -Version 5.1
# Applies EF Core migrations to the Musify database.
[CmdletBinding()]
param([Parameter(Mandatory)][string]$ConnectionString)
$ErrorActionPreference = "Stop"
$infraProject = Resolve-Path (Join-Path $PSScriptRoot "..\..\backend\Core\Musify.Infrastructure")
# The design-time factory reads the connection string from user-secrets.
dotnet user-secrets set "Database:ConnectionString" $ConnectionString --project $infraProject | Out-Null
# Infrastructure is also the startup project: it owns the IDesignTimeDbContextFactory
# and the EFCore.Design package (Musify.Api does not reference Design).
dotnet ef database update --project $infraProject --startup-project $infraProject --context Database
if ($LASTEXITCODE -ne 0) { throw "EF Core migration failed" }
deploy/scripts/migrate.sh
-28
View File
@@ -1,28 +0,0 @@
#!/bin/sh
# Applies the EF Core migrations to the production database from a throwaway SDK
# container attached to the compose network (Postgres is not published to the host).
set -eu
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
DEPLOY_DIR=$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)
BACKEND_DIR=$(CDPATH= cd -- "$DEPLOY_DIR/../backend" && pwd)
ENV_FILE="$DEPLOY_DIR/.env.prod"
NETWORK=${MUSIFY_NETWORK:-musify-prod_default}
. "$SCRIPT_DIR/lib.sh"
CONN="Host=postgres;Port=5432;Database=$(env_get "$ENV_FILE" POSTGRES_DB);Username=$(env_get "$ENV_FILE" POSTGRES_USER);Password=$(env_get "$ENV_FILE" POSTGRES_PASSWORD)"
# Musify.Infrastructure is both the migrations and the startup project: it owns
# the IDesignTimeDbContextFactory and the EFCore.Design package.
docker run --rm \
--network "$NETWORK" \
-v "$BACKEND_DIR":/src \
-w /src/Core/Musify.Infrastructure \
-e CONN="$CONN" \
mcr.microsoft.com/dotnet/sdk:10.0 \
sh -c 'dotnet tool install --global dotnet-ef >/dev/null 2>&1 || true
export PATH=$PATH:/root/.dotnet/tools
dotnet restore Musify.Infrastructure.csproj
dotnet user-secrets set "Database:ConnectionString" "$CONN" >/dev/null
dotnet ef database update --context Database'
deploy/scripts/stream-keys.ps1
-28
View File
@@ -1,28 +0,0 @@
#requires -Version 5.1
# Generates the RS256 key pair used to sign (API) and verify (Gateway) stream tickets.
# No-op if both keys already exist.
[CmdletBinding()]
param([Parameter(Mandatory)][string]$KeysDir)
$ErrorActionPreference = "Stop"
$privateKey = Join-Path $KeysDir "stream_private.pem"
$publicKey = Join-Path $KeysDir "stream_public.pem"
if ((Test-Path $privateKey) -and (Test-Path $publicKey)) {
Write-Host "Stream-ticket keys already present in $KeysDir"
return
}
if (-not (Test-Path $KeysDir)) { New-Item -ItemType Directory -Path $KeysDir | Out-Null }
if (Get-Command openssl -ErrorAction SilentlyContinue) {
& openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out $privateKey
& openssl rsa -in $privateKey -pubout -out $publicKey
} else {
$rsa = [System.Security.Cryptography.RSA]::Create(2048)
[System.IO.File]::WriteAllText($privateKey, $rsa.ExportPkcs8PrivateKeyPem())
[System.IO.File]::WriteAllText($publicKey, $rsa.ExportSubjectPublicKeyInfoPem())
$rsa.Dispose()
}
Write-Host "Generated stream-ticket keys in $KeysDir"
deploy/scripts/stream-keys.sh
-21
View File
@@ -1,21 +0,0 @@
#!/bin/sh
# Generates the RS256 key pair used to sign (API) and verify (Gateway) stream
# tickets. No-op if both keys already exist.
set -eu
KEYS_DIR=${1:?usage: stream-keys.sh <keys-dir>}
PRIVATE_KEY="$KEYS_DIR/stream_private.pem"
PUBLIC_KEY="$KEYS_DIR/stream_public.pem"
if [ -f "$PRIVATE_KEY" ] && [ -f "$PUBLIC_KEY" ]; then
echo "Stream-ticket keys already present in $KEYS_DIR"
exit 0
fi
mkdir -p "$KEYS_DIR"
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out "$PRIVATE_KEY" 2>/dev/null
openssl rsa -in "$PRIVATE_KEY" -pubout -out "$PUBLIC_KEY" 2>/dev/null
chmod 644 "$PUBLIC_KEY"
chmod 600 "$PRIVATE_KEY"
echo "Generated stream-ticket keys in $KEYS_DIR"
deploy/up.ps1
-145
View File
@@ -1,145 +0,0 @@
#requires -Version 5.1
<#
.SYNOPSIS
Brings up the Musify development environment.
.DESCRIPTION
Starts the infrastructure stack, generates the stream-ticket keys, applies
the EF Core migrations and provisions Zitadel (project + OIDC apps). The
.NET apps and the web player then run from the IDE; add -Apps to run them
in Docker instead.
.EXAMPLE
./up.ps1 # Infrastructure only (apps from the IDE)
./up.ps1 -Apps # Also build and run api, worker, gateway, web player
./up.ps1 -Tools # Also start pgAdmin
./up.ps1 -SkipMigrations # Skip the database migration step
./up.ps1 -Down # Stop and remove containers (keep volumes)
./up.ps1 -Destroy # Stop and remove containers + volumes (wipe data)
#>
[CmdletBinding()]
param(
[switch]$Apps,
[switch]$Tools,
[switch]$SkipMigrations,
[switch]$Down,
[switch]$Destroy
)
$ErrorActionPreference = "Stop"
$deployDir = $PSScriptRoot
$compose = @(
"-f", (Join-Path $deployDir "compose.yml"),
"-f", (Join-Path $deployDir "compose.dev.yml")
)
$profiles = @("--profile", "apps", "--profile", "tools")
. (Join-Path $deployDir "scripts\common.ps1")
# Teardown -------------------------------------------------------------------
if ($Down -or $Destroy) {
if ($Destroy) {
Write-Step "Stopping the stack and REMOVING volumes (data is wiped)"
docker compose @compose @profiles down -v --remove-orphans
# Drop the Zitadel init secrets so the next start re-initializes cleanly.
Get-ChildItem (Join-Path $deployDir "zitadel\.output") -File -ErrorAction SilentlyContinue | Remove-Item -Force
} else {
Write-Step "Stopping the stack (volumes are kept)"
docker compose @compose @profiles down --remove-orphans
}
return
}
# Config ---------------------------------------------------------------------
$envFile = Join-Path $deployDir ".env"
if (-not (Test-Path $envFile)) {
Write-Step "Creating .env from .env.example"
Copy-Item (Join-Path $deployDir ".env.example") $envFile
}
$cfg = Read-DotEnv $envFile
# Host directories the apps expect: RS256 keys (mounted into the api/gateway
# containers, read directly by the IDE runs) and the Worker scratch space.
& (Join-Path $deployDir "scripts\stream-keys.ps1") -KeysDir (Join-Path $deployDir "keys")
$tempDir = Join-Path $deployDir ".tmp"
if (-not (Test-Path $tempDir)) { New-Item -ItemType Directory -Path $tempDir | Out-Null }
# Infrastructure -------------------------------------------------------------
Write-Step "Starting the infrastructure stack"
$upArgs = @("up", "-d")
if ($Tools) { $upArgs = @("--profile", "tools") + $upArgs }
docker compose @compose @upArgs
if ($LASTEXITCODE -ne 0) { throw "docker compose up failed" }
Write-Step "Waiting for Postgres to become healthy"
$deadline = (Get-Date).AddMinutes(2)
do {
Start-Sleep -Seconds 2
$status = docker inspect --format '{{.State.Health.Status}}' musify-postgres 2>$null
} until ($status -eq "healthy" -or (Get-Date) -gt $deadline)
if ($status -ne "healthy") { throw "Postgres did not become healthy in time" }
# Database migrations --------------------------------------------------------
if ($SkipMigrations) {
Write-Host "`nSkipping migrations (-SkipMigrations)."
} else {
Write-Step "Applying EF Core migrations"
$conn = "Host=localhost;Port=$($cfg['POSTGRES_HOST_PORT']);Database=$($cfg['POSTGRES_DB']);Username=$($cfg['POSTGRES_USER']);Password=$($cfg['POSTGRES_PASSWORD'])"
& (Join-Path $deployDir "scripts\migrate.ps1") -ConnectionString $conn
}
# Zitadel provisioning -------------------------------------------------------
# Writes the client ids into .env, Musify.Api user-secrets and web-player/.env,
# so it has to run before the app containers start.
& (Join-Path $deployDir "zitadel\provision.ps1")
# Applications ---------------------------------------------------------------
if ($Apps) {
Write-Step "Building and starting the applications"
docker compose @compose --profile apps up -d --build
if ($LASTEXITCODE -ne 0) { throw "docker compose up (apps) failed" }
}
# Summary --------------------------------------------------------------------
$cfg = Read-DotEnv $envFile
$authUrl = $cfg['PUBLIC_AUTH_URL']
$appLines = if ($Apps) {
@"
Musify API $($cfg['PUBLIC_API_URL'])/scalar/v1
Streaming gateway $($cfg['PUBLIC_STREAM_URL'])/health
Web player $($cfg['PUBLIC_WEB_URL'])
"@
} else {
@"
Run the apps from the IDE (or ./up.ps1 -Apps to run them in Docker):
dotnet run --project backend/Hosts/Musify.Api # :5111
dotnet run --project backend/Hosts/Musify.StreamingGateway # :8081
dotnet run --project backend/Hosts/Musify.Worker # background
npm --prefix web-player run dev # :5173
"@
}
Write-Host @"
============================================================
Musify development environment READY
============================================================
Service URL / port
------------------ --------------------------------------
PostgreSQL localhost:$($cfg['POSTGRES_HOST_PORT']) (db: $($cfg['POSTGRES_DB']))
Zitadel console $authUrl/ui/console
RabbitMQ mgmt http://localhost:15672 ($($cfg['RABBITMQ_DEFAULT_USER'])/$($cfg['RABBITMQ_DEFAULT_PASS']))
SeaweedFS S3 http://localhost:8333 (bucket: $($cfg['S3_BUCKET']))
SeaweedFS filer http://localhost:8888
SeaweedFS master http://localhost:9333
Jaeger traces http://localhost:16686 (OTLP: localhost:4317)
pgAdmin (-Tools) http://localhost:$($cfg['PGADMIN_HOST_PORT']) ($($cfg['PGADMIN_DEFAULT_EMAIL'])/$($cfg['PGADMIN_DEFAULT_PASSWORD']))
Zitadel admin $($cfg['ZITADEL_ADMIN_USERNAME'])@zitadel.$($cfg['ZITADEL_EXTERNAL_DOMAIN']) / $($cfg['ZITADEL_ADMIN_PASSWORD'])
API client id $($cfg['AUTH_CLIENT_ID'])
Web client id $($cfg['WEB_CLIENT_ID'])
$appLines
============================================================
"@ -ForegroundColor Green
deploy/up.sh
-103
View File
@@ -1,103 +0,0 @@
#!/bin/sh
# Brings up (or updates) the Musify production stack on the server.
#
# ./up.sh build + start everything, provision Zitadel, migrate
# ./up.sh --no-build same, reusing the images that are already built
# ./up.sh --down stop and remove the containers (volumes are kept)
#
# Everything it needs lives in deploy/.env.prod (see .env.prod.example) and in
# deploy/nginx/certs/ (origin.pem + origin.key).
set -eu
DEPLOY_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
ENV_FILE="$DEPLOY_DIR/.env.prod"
INFRA="postgres zitadel rabbitmq seaweedfs seaweedfs-init jaeger"
. "$DEPLOY_DIR/scripts/lib.sh"
require_command docker
require_command openssl
compose() {
docker compose --env-file "$ENV_FILE" \
-f "$DEPLOY_DIR/compose.yml" -f "$DEPLOY_DIR/compose.prod.yml" \
--profile apps "$@"
}
BUILD=--build
for arg in "$@"; do
case "$arg" in
--no-build) BUILD= ;;
--down) step "Stopping the stack (volumes are kept)"; compose down --remove-orphans; exit 0 ;;
*) fail "unknown option: $arg" ;;
esac
done
# Preflight ------------------------------------------------------------------
if [ ! -f "$ENV_FILE" ]; then
cp "$DEPLOY_DIR/.env.prod.example" "$ENV_FILE"
chmod 600 "$ENV_FILE"
fail "created $ENV_FILE from the template — fill it in (domain + secrets) and run again."
fi
if grep -q 'CHANGE_ME\|example\.com' "$ENV_FILE"; then
fail "$ENV_FILE still contains CHANGE_ME or example.com placeholders."
fi
for cert in origin.pem origin.key; do
[ -f "$DEPLOY_DIR/nginx/certs/$cert" ] \
|| fail "missing nginx/certs/$cert (Cloudflare Origin Certificate)."
done
mkdir -p "$DEPLOY_DIR/secrets" "$DEPLOY_DIR/keys" "$DEPLOY_DIR/zitadel/.output"
case "$(env_get "$ENV_FILE" YTDLP_ADDITIONAL_ARGUMENTS)" in
*--cookies*)
[ -f "$DEPLOY_DIR/secrets/youtube_cookies.txt" ] \
|| echo "warning: YTDLP_ADDITIONAL_ARGUMENTS expects secrets/youtube_cookies.txt, which is missing." ;;
esac
step "Ensuring stream-ticket keys"
sh "$DEPLOY_DIR/scripts/stream-keys.sh" "$DEPLOY_DIR/keys"
# Infrastructure -------------------------------------------------------------
step "Starting the infrastructure"
# shellcheck disable=SC2086
compose up -d $INFRA
step "Waiting for Postgres"
deadline=$(( $(date +%s) + 120 ))
until [ "$(docker inspect --format '{{.State.Health.Status}}' musify-postgres 2>/dev/null)" = healthy ]; do
[ "$(date +%s)" -lt "$deadline" ] || fail "Postgres did not become healthy in time."
sleep 2
done
# nginx has to be up before provisioning: it terminates TLS for the Zitadel
# domain. --no-deps keeps the apps from starting before they have a client id.
step "Starting the edge"
compose up -d --no-deps nginx
# Zitadel + database ---------------------------------------------------------
sh "$DEPLOY_DIR/zitadel/provision.sh"
step "Applying EF Core migrations"
sh "$DEPLOY_DIR/scripts/migrate.sh"
# Applications ---------------------------------------------------------------
step "Starting the applications"
# shellcheck disable=SC2086
compose up -d $BUILD
step "Done"
compose ps
cat <<EOF
Web player $(env_get "$ENV_FILE" PUBLIC_WEB_URL)
API $(env_get "$ENV_FILE" PUBLIC_API_URL)/scalar/v1
Streaming $(env_get "$ENV_FILE" PUBLIC_STREAM_URL)/health
Zitadel $(env_get "$ENV_FILE" PUBLIC_AUTH_URL)/ui/console
$(env_get "$ENV_FILE" ZITADEL_ADMIN_USERNAME)@zitadel.$(env_get "$ENV_FILE" ZITADEL_EXTERNAL_DOMAIN)
Jaeger collects traces but publishes no port. To look at them, forward its
container address over SSH from your machine:
ssh -L 16686:\$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' musify-jaeger):16686 <server>
EOF
deploy/zitadel/.output-share/admin-sa.pat
-1
View File
@@ -1 +0,0 @@
eyJhbGciOiJBMjU2R0NNS1ciLCJlbmMiOiJBMjU2R0NNIiwiaXYiOiJ5Yk1hUFdVUmNqcU44WDNGIiwia2lkIjoib2lkY0tleSIsInRhZyI6Ikw2NjYwN3FRVzZNYVJLaVl3Q0w1N3cifQ.BE_7qqA1Auer5yEoGnyN0v35JlZlK3XgCo0s_CQtG7s.cOf9oudMHRnVTOGj.0DRh9kj7oahl4imvMjxqRLUOZo6NhYlQyslf9_x4XF_TKi1dnQ.Uu36qEL5J1VKSIYcvWlbAg
deploy/zitadel/provision.ps1
-193
View File
@@ -1,193 +0,0 @@
#requires -Version 5.1
<#
.SYNOPSIS
Provisions Zitadel for Musify in development (idempotent).
.DESCRIPTION
Creates the Musify project, the API OIDC app (public, PKCE, JWT access
token) and the web player OIDC app (confidential), then wires the resulting
client ids into deploy/.env, Musify.Api user-secrets and web-player/.env.
Authenticates with the service-account PAT that Zitadel writes on init
(zitadel/.output/admin-sa.pat). Re-running reuses the existing project and
apps and just refreshes the wired values.
#>
[CmdletBinding()]
param(
[string]$ProjectName = "Musify",
[string]$ApiAppName = "Musify API (Scalar)",
[string]$WebAppName = "Musify Web",
# Alternate .env to read/write (e.g. deploy/.env.share). Defaults to deploy/.env.
[string]$EnvFile,
# Skip writing Musify.Api user-secrets and web-player/.env: use when the apps
# only ever run in Docker for this environment and get their config from compose.
[switch]$SkipHostWiring,
# Matches the zitadel volume mount for this environment (see compose.share.yml).
[string]$OutputDir = ".output"
)
$ErrorActionPreference = "Stop"
$deployDir = Split-Path $PSScriptRoot -Parent
$repoDir = Split-Path $deployDir -Parent
$apiProject = Join-Path $repoDir "backend\Hosts\Musify.Api"
$webEnvFile = Join-Path $repoDir "web-player\.env"
$envFile = if ($EnvFile) { $EnvFile } else { Join-Path $deployDir ".env" }
$patFile = Join-Path $PSScriptRoot "$OutputDir\admin-sa.pat"
. (Join-Path $deployDir "scripts\common.ps1")
$cfg = Read-DotEnv $envFile
$issuer = $cfg['PUBLIC_AUTH_URL']
$apiUrl = $cfg['PUBLIC_API_URL']
$webUrl = $cfg['PUBLIC_WEB_URL']
$viteUrl = "http://localhost:5173" # `npm run dev` in web-player/
Write-Step "Provisioning Zitadel at $issuer"
# Zitadel writes the PAT during init; on a cold start it may take a moment.
$deadline = (Get-Date).AddMinutes(3)
while (-not (Test-Path $patFile) -and (Get-Date) -lt $deadline) { Start-Sleep -Seconds 3 }
if (-not (Test-Path $patFile)) { throw "PAT not found ($patFile). Check 'docker logs musify-zitadel'." }
$headers = @{ Authorization = "Bearer $((Get-Content $patFile -Raw).Trim())" }
$mgmt = "$issuer/management/v1"
function Invoke-Zitadel($method, $path, $body) {
$params = @{ Method = $method; Uri = "$mgmt$path"; Headers = $headers; ContentType = "application/json"; TimeoutSec = 20 }
if ($body) { $params.Body = ($body | ConvertTo-Json -Depth 10) }
Invoke-RestMethod @params
}
$deadline = (Get-Date).AddMinutes(2)
do {
try { Invoke-RestMethod "$issuer/.well-known/openid-configuration" -TimeoutSec 5 | Out-Null; $ready = $true }
catch { $ready = $false; Start-Sleep 3 }
} until ($ready -or (Get-Date) -gt $deadline)
if (-not $ready) { throw "Zitadel did not respond in time." }
# The images tagged `latest` require the standalone Login UI v2 container, which
# this stack does not run, so OIDC would redirect to /ui/v2/login and 404.
# Opting out keeps the built-in v1 login.
try {
Invoke-RestMethod -Method PUT -Uri "$issuer/v2/features/instance" -Headers $headers `
-ContentType "application/json" -Body (@{ loginV2 = @{ required = $false } } | ConvertTo-Json) -TimeoutSec 20 | Out-Null
} catch {
Write-Host " warning: could not disable the required Login UI v2: $($_.Exception.Message)"
}
# Project --------------------------------------------------------------------
$project = (Invoke-Zitadel POST "/projects/_search" @{
queries = @(@{ nameQuery = @{ name = $ProjectName; method = "TEXT_QUERY_METHOD_EQUALS" } })
}).result | Where-Object { $_.name -eq $ProjectName } | Select-Object -First 1
if ($project) {
$projectId = $project.id
Write-Host " project '$ProjectName' reused (id $projectId)"
} else {
$projectId = (Invoke-Zitadel POST "/projects" @{ name = $ProjectName }).id
Write-Host " project '$ProjectName' created (id $projectId)"
}
function Get-ZitadelApp($name) {
(Invoke-Zitadel POST "/projects/$projectId/apps/_search" @{
queries = @(@{ nameQuery = @{ name = $name; method = "TEXT_QUERY_METHOD_EQUALS" } })
}).result | Where-Object { $_.name -eq $name } | Select-Object -First 1
}
# API app: public client used by the Scalar UI to obtain a token -------------
$apiApp = Get-ZitadelApp $ApiAppName
if ($apiApp) {
$apiClientId = (Invoke-Zitadel GET "/projects/$projectId/apps/$($apiApp.id)" $null).app.oidcConfig.clientId
Write-Host " app '$ApiAppName' reused (clientId $apiClientId)"
} else {
$apiClientId = (Invoke-Zitadel POST "/projects/$projectId/apps/oidc" @{
name = $ApiAppName
redirectUris = @("$apiUrl/scalar/", "$apiUrl/scalar/oauth2-redirect.html")
postLogoutRedirectUris = @("$apiUrl/scalar/")
responseTypes = @("OIDC_RESPONSE_TYPE_CODE")
grantTypes = @("OIDC_GRANT_TYPE_AUTHORIZATION_CODE")
appType = "OIDC_APP_TYPE_USER_AGENT"
authMethodType = "OIDC_AUTH_METHOD_TYPE_NONE"
version = "OIDC_VERSION_1_0"
devMode = $true # allow http redirect URIs
accessTokenType = "OIDC_TOKEN_TYPE_JWT" # required so JwtBearer can validate it
accessTokenRoleAssertion = $true
idTokenRoleAssertion = $true
idTokenUserinfoAssertion = $true
}).clientId
Write-Host " app '$ApiAppName' created (clientId $apiClientId)"
}
# Web app: confidential client used by the SvelteKit server ------------------
$webApp = Get-ZitadelApp $WebAppName
$webSecret = $cfg['WEB_CLIENT_SECRET']
if ($webApp) {
$webClientId = (Invoke-Zitadel GET "/projects/$projectId/apps/$($webApp.id)" $null).app.oidcConfig.clientId
# The secret is only returned once, so regenerate it if we no longer have it.
if (-not $webSecret) {
$webSecret = (Invoke-Zitadel POST "/projects/$projectId/apps/$($webApp.id)/oidc_config/_generate_client_secret" @{}).clientSecret
Write-Host " app '$WebAppName' reused (clientId $webClientId, secret regenerated)"
} else {
Write-Host " app '$WebAppName' reused (clientId $webClientId)"
}
} else {
$created = Invoke-Zitadel POST "/projects/$projectId/apps/oidc" @{
name = $WebAppName
redirectUris = @("$webUrl/auth/callback", "$viteUrl/auth/callback")
postLogoutRedirectUris = @("$webUrl/", "$viteUrl/")
responseTypes = @("OIDC_RESPONSE_TYPE_CODE")
grantTypes = @("OIDC_GRANT_TYPE_AUTHORIZATION_CODE", "OIDC_GRANT_TYPE_REFRESH_TOKEN")
appType = "OIDC_APP_TYPE_WEB"
authMethodType = "OIDC_AUTH_METHOD_TYPE_BASIC"
version = "OIDC_VERSION_1_0"
devMode = $true
accessTokenType = "OIDC_TOKEN_TYPE_JWT"
accessTokenRoleAssertion = $true
idTokenRoleAssertion = $true
idTokenUserinfoAssertion = $true
}
$webClientId = $created.clientId
$webSecret = $created.clientSecret
Write-Host " app '$WebAppName' created (clientId $webClientId)"
}
# Wire the values ------------------------------------------------------------
Set-DotEnvValue $envFile "AUTH_CLIENT_ID" $apiClientId
Set-DotEnvValue $envFile "WEB_CLIENT_ID" $webClientId
Set-DotEnvValue $envFile "WEB_CLIENT_SECRET" $webSecret
if ($SkipHostWiring) {
Write-Host " wired into $envFile only (-SkipHostWiring: apps run in Docker)"
} else {
# Musify.Api reads these from user-secrets when it runs outside Docker.
@{
"Authentication:ClientId" = $apiClientId
"Authentication:AudienceAddress" = $apiClientId
"Authentication:IssuerAddress" = $issuer
"Authentication:MetadataAddress" = "$issuer/.well-known/openid-configuration"
"Authentication:AuthorizationEndpoint" = "$issuer/oauth/v2/authorize?prompt=login"
"Authentication:TokenEndpoint" = "$issuer/oauth/v2/token"
"Authentication:ScalarRedirectUri" = "$apiUrl/scalar/"
"Authentication:RequireHttpsMetadata" = "false"
}.GetEnumerator() | ForEach-Object {
dotnet user-secrets set $_.Key $_.Value --project $apiProject | Out-Null
}
# web-player/.env is what `npm run dev` reads (the container gets its config
# from compose instead).
if (-not (Test-Path $webEnvFile)) {
Copy-Item (Join-Path $repoDir "web-player\.env.example") $webEnvFile
}
@{
ZITADEL_ISSUER = $issuer
ZITADEL_CLIENT_ID = $webClientId
ZITADEL_CLIENT_SECRET = $webSecret
AUTH_REDIRECT_URI = "$viteUrl/auth/callback"
AUTH_POST_LOGOUT_URI = "$viteUrl/"
SESSION_SECRET = $cfg['SESSION_SECRET']
API_BASE_URL = $apiUrl
}.GetEnumerator() | ForEach-Object {
Set-DotEnvValue $webEnvFile $_.Key $_.Value
}
Write-Host " wired into deploy/.env, Musify.Api user-secrets and web-player/.env"
}
deploy/zitadel/provision.sh
-145
View File
@@ -1,145 +0,0 @@
#!/bin/sh
# Provisions Zitadel for Musify in production (idempotent).
#
# Creates the Musify project, the API OIDC app (public, PKCE, JWT access token)
# and the web player OIDC app (confidential), then writes AUTH_CLIENT_ID,
# WEB_CLIENT_ID and WEB_CLIENT_SECRET into deploy/.env.prod. Authenticates with
# the service-account PAT that Zitadel writes on init. Re-running reuses the
# existing project and apps.
set -eu
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
DEPLOY_DIR=$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)
ENV_FILE="$DEPLOY_DIR/.env.prod"
PAT_FILE="$SCRIPT_DIR/.output/admin-sa.pat"
. "$DEPLOY_DIR/scripts/lib.sh"
require_command curl
require_command jq
PROJECT_NAME=${PROJECT_NAME:-Musify}
API_APP_NAME=${API_APP_NAME:-Musify API}
WEB_APP_NAME=${WEB_APP_NAME:-Musify Web}
ISSUER=$(env_get "$ENV_FILE" PUBLIC_AUTH_URL)
API_URL=$(env_get "$ENV_FILE" PUBLIC_API_URL)
WEB_URL=$(env_get "$ENV_FILE" PUBLIC_WEB_URL)
MGMT="$ISSUER/management/v1"
step "Provisioning Zitadel at $ISSUER"
# Zitadel writes the PAT during init; on a cold start it may take a moment.
deadline=$(( $(date +%s) + 180 ))
while [ ! -f "$PAT_FILE" ] && [ "$(date +%s)" -lt "$deadline" ]; do sleep 3; done
[ -f "$PAT_FILE" ] || fail "PAT not found ($PAT_FILE). Check 'docker logs musify-zitadel'."
PAT=$(tr -d '\r\n' <"$PAT_FILE")
api_call() {
# api_call <method> <path> [json body]
if [ $# -ge 3 ]; then
curl -fsS -X "$1" "$MGMT$2" \
-H "Authorization: Bearer $PAT" -H "Content-Type: application/json" -d "$3"
else
curl -fsS -X "$1" "$MGMT$2" -H "Authorization: Bearer $PAT"
fi
}
deadline=$(( $(date +%s) + 120 ))
until curl -fsS "$ISSUER/.well-known/openid-configuration" >/dev/null 2>&1; do
[ "$(date +%s)" -lt "$deadline" ] || fail "Zitadel did not respond at $ISSUER."
sleep 3
done
# The images tagged `latest` require the standalone Login UI v2 container, which
# this stack does not run, so OIDC would redirect to /ui/v2/login and 404.
curl -fsS -X PUT "$ISSUER/v2/features/instance" \
-H "Authorization: Bearer $PAT" -H "Content-Type: application/json" \
-d '{"loginV2":{"required":false}}' >/dev/null 2>&1 \
|| echo " warning: could not disable the required Login UI v2"
name_query() {
printf '{"queries":[{"nameQuery":{"name":"%s","method":"TEXT_QUERY_METHOD_EQUALS"}}]}' "$1"
}
# Project --------------------------------------------------------------------
PROJECT_ID=$(api_call POST "/projects/_search" "$(name_query "$PROJECT_NAME")" \
| jq -r --arg n "$PROJECT_NAME" '.result // [] | map(select(.name == $n)) | first | .id // empty')
if [ -n "$PROJECT_ID" ]; then
echo " project '$PROJECT_NAME' reused (id $PROJECT_ID)"
else
PROJECT_ID=$(api_call POST "/projects" "$(printf '{"name":"%s"}' "$PROJECT_NAME")" | jq -r '.id')
echo " project '$PROJECT_NAME' created (id $PROJECT_ID)"
fi
find_app() {
api_call POST "/projects/$PROJECT_ID/apps/_search" "$(name_query "$1")" \
| jq -r --arg n "$1" '.result // [] | map(select(.name == $n)) | first | .id // empty'
}
# API app: public client used by the Scalar UI to obtain a token -------------
API_APP_ID=$(find_app "$API_APP_NAME")
if [ -n "$API_APP_ID" ]; then
API_CLIENT_ID=$(api_call GET "/projects/$PROJECT_ID/apps/$API_APP_ID" | jq -r '.app.oidcConfig.clientId')
echo " app '$API_APP_NAME' reused (clientId $API_CLIENT_ID)"
else
API_CLIENT_ID=$(api_call POST "/projects/$PROJECT_ID/apps/oidc" "$(jq -n \
--arg name "$API_APP_NAME" --arg api "$API_URL" '{
name: $name,
redirectUris: [($api + "/scalar/"), ($api + "/scalar/oauth2-redirect.html")],
postLogoutRedirectUris: [($api + "/scalar/")],
responseTypes: ["OIDC_RESPONSE_TYPE_CODE"],
grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE"],
appType: "OIDC_APP_TYPE_USER_AGENT",
authMethodType: "OIDC_AUTH_METHOD_TYPE_NONE",
version: "OIDC_VERSION_1_0",
devMode: false,
accessTokenType: "OIDC_TOKEN_TYPE_JWT",
accessTokenRoleAssertion: true,
idTokenRoleAssertion: true,
idTokenUserinfoAssertion: true
}')" | jq -r '.clientId')
echo " app '$API_APP_NAME' created (clientId $API_CLIENT_ID)"
fi
# Web app: confidential client used by the SvelteKit server ------------------
WEB_APP_ID=$(find_app "$WEB_APP_NAME")
WEB_CLIENT_SECRET=$(env_get "$ENV_FILE" WEB_CLIENT_SECRET)
if [ -n "$WEB_APP_ID" ]; then
WEB_CLIENT_ID=$(api_call GET "/projects/$PROJECT_ID/apps/$WEB_APP_ID" | jq -r '.app.oidcConfig.clientId')
if [ -z "$WEB_CLIENT_SECRET" ]; then
# The secret is only returned once, so regenerate it if we no longer have it.
WEB_CLIENT_SECRET=$(api_call POST "/projects/$PROJECT_ID/apps/$WEB_APP_ID/oidc_config/_generate_client_secret" '{}' \
| jq -r '.clientSecret')
echo " app '$WEB_APP_NAME' reused (clientId $WEB_CLIENT_ID, secret regenerated)"
else
echo " app '$WEB_APP_NAME' reused (clientId $WEB_CLIENT_ID)"
fi
else
CREATED=$(api_call POST "/projects/$PROJECT_ID/apps/oidc" "$(jq -n \
--arg name "$WEB_APP_NAME" --arg web "$WEB_URL" '{
name: $name,
redirectUris: [($web + "/auth/callback")],
postLogoutRedirectUris: [($web + "/")],
responseTypes: ["OIDC_RESPONSE_TYPE_CODE"],
grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE", "OIDC_GRANT_TYPE_REFRESH_TOKEN"],
appType: "OIDC_APP_TYPE_WEB",
authMethodType: "OIDC_AUTH_METHOD_TYPE_BASIC",
version: "OIDC_VERSION_1_0",
devMode: false,
accessTokenType: "OIDC_TOKEN_TYPE_JWT",
accessTokenRoleAssertion: true,
idTokenRoleAssertion: true,
idTokenUserinfoAssertion: true
}')")
WEB_CLIENT_ID=$(echo "$CREATED" | jq -r '.clientId')
WEB_CLIENT_SECRET=$(echo "$CREATED" | jq -r '.clientSecret')
echo " app '$WEB_APP_NAME' created (clientId $WEB_CLIENT_ID)"
fi
env_set "$ENV_FILE" AUTH_CLIENT_ID "$API_CLIENT_ID"
env_set "$ENV_FILE" WEB_CLIENT_ID "$WEB_CLIENT_ID"
env_set "$ENV_FILE" WEB_CLIENT_SECRET "$WEB_CLIENT_SECRET"
echo " wired into $(basename "$ENV_FILE")"