NotS1mp/Musify
0
0
Fork 0
Code Issues 23 Pull Requests Actions Packages Projects Releases Wiki Activity

Replace deploy scripts with self-contained compose bootstrap jobs

Browse Source 
Removes every wrapper script under deploy/ (up.ps1, up.sh, scripts/*,
zitadel/provision.*) and folds their logic into three one-shot services
in compose.yml, following the pattern already used by seaweedfs-init:

  keys-init     generates the RS256 stream-ticket key pair
  migrate       applies the EF Core migrations (backend/Musify.Migrator.Dockerfile)
  zitadel-init  creates the Musify project + OIDC apps, writes the client
                ids back into the active env file

Deploying is now two plain `docker compose` commands, no scripts to run
or chmod:

  docker compose -f deploy/compose.yml -f deploy/compose.dev.yml up -d
  docker compose -f deploy/compose.yml -f deploy/compose.dev.yml --profile apps up -d --build

(swap compose.dev.yml for compose.prod.yml + --env-file deploy/.env.prod
for production). The first command brings up infra and lets the bootstrap
jobs run to completion; the second builds and starts the apps, which by
then read real OIDC values instead of empty ones — compose interpolates
${VAR} once per invocation, so this can't collapse into a single command
without the apps reading their own config file at container start instead
of via compose env (out of scope here).

Also:
- DatabaseDesignTimeFactory now also reads Database__ConnectionString
  from the environment, and DesignSettings.json is optional, so the new
  migrate container works without user-secrets or a config file excluded
  from the Docker build context by .dockerignore.
- nginx's depends_on no longer lists the apps-profile services: it does
  not need them at its own startup (every vhost resolves its upstream at
  request time via the embedded Docker resolver), and requiring them made
  `compose.prod.yml up -d` (without --profile apps) an invalid project.
- env_get/env_set in zitadel-init strip \r: the mounted env file can have
  CRLF line endings on Windows, which otherwise makes an empty-looking
  value non-empty to `[ -z ... ]`. .env.example/.env.prod.example are
  pinned to LF via .gitattributes for the same reason. Also fixed:
  rewriting the bind-mounted env file in place instead of renaming a temp
  file over it (rename onto a bind-mounted inode fails with "Resource
  busy").
- Removed compose.share.yml and zitadel/.output-share/ — a "temporary"
  personal overlay that had leaked a real Zitadel service-account PAT
  into git history (committed in 70968b3). That token should be rotated.
- Removed deploy/.tmp/ from .gitignore: nothing mounts or creates it
  anymore (it never actually did, either — no compose service referenced
  it, only the now-removed up.ps1 did).

Verified end-to-end with Docker: a clean `down -v` + fresh `up -d` +
`--profile apps up -d --build` on the dev stack brings up all four one-shot
jobs successfully and all four apps healthy (api/gateway return 200,
worker starts its consumers with no errors, web-player responds). Also
verified compose.prod.yml starts nginx cleanly with dummy certs and no
apps running, confirming the depends_on fix.
This commit is contained in:
Admin
2026-09-09 15:08:34 +02:00
parent 89569736b7
commit cf8ac706b6
26 changed files with 408 additions and 862 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitattributes
+7
View File
@@ -4,6 +4,13 @@
# Force LF for shell scripts # Force LF for shell scripts
*.sh text eol=lf *.sh text eol=lf
# Env templates get `cp`-ed straight into deploy/.env / deploy/.env.prod,
# which containers (e.g. zitadel-init in deploy/compose.yml) parse with
# plain shell text tools — keep them LF so a Windows checkout can't
# reintroduce CRLF into a value that then looks non-empty but isn't.
.env.example text eol=lf
.env.prod.example text eol=lf
# Treat common binaries as binary # Treat common binaries as binary
*.png binary *.png binary
*.jpg binary *.jpg binary
.gitignore
-1
View File
@@ -27,4 +27,3 @@ deploy/keys/
deploy/secrets/ deploy/secrets/
deploy/nginx/certs/ deploy/nginx/certs/
deploy/zitadel/.output/ deploy/zitadel/.output/
deploy/.tmp/
README.md
+13 -7
View File
@@ -7,26 +7,32 @@ and stream them without the bytes ever going through the API.
``` ```
backend/ .NET 10 — Domain / Application / Infrastructure / Api / Worker / StreamingGateway backend/ .NET 10 — Domain / Application / Infrastructure / Api / Worker / StreamingGateway
web-player/ SvelteKit 2 + Svelte 5 + Tailwind 4 web client web-player/ SvelteKit 2 + Svelte 5 + Tailwind 4 web client
deploy/ Docker Compose stacks and the scripts that bootstrap them deploy/ Docker Compose stacks — self-contained, no wrapper scripts
docs/ What each piece does and why (in Spanish) docs/ What each piece does and why (in Spanish)
``` ```
## Run it ## Run it
Development (Windows, Docker Desktop): Development (Docker Desktop):
```powershell ```sh
./deploy/up.ps1 # infrastructure; the apps run from the IDE docker compose -f deploy/compose.yml -f deploy/compose.dev.yml up -d
./deploy/up.ps1 -Apps # everything in Docker
``` ```
Infrastructure plus stream-ticket keys, database migrations and Zitadel OIDC
provisioning — all as one-shot jobs baked into the compose file. Apps run from
the IDE by default; add `--profile apps up -d --build` as a second command to
run them in Docker instead.
Production, on a server with a domain: Production, on a server with a domain:
```sh ```sh
./deploy/up.sh # fill in deploy/.env.prod and the TLS certs when it asks cp deploy/.env.prod.example deploy/.env.prod # fill in the domain and secrets
docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml up -d
docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml --profile apps up -d --build
``` ```
Both are idempotent and print what they did. See Idempotent — run the same commands again to deploy an update. See
[deploy/README.md](deploy/README.md) for the details, the port map and the [deploy/README.md](deploy/README.md) for the details, the port map and the
production prerequisites. production prerequisites.
backend/Core/Musify.Infrastructure/Persistence/DatabaseDesignTimeFactory.cs
+2 -1
View File
@@ -11,8 +11,9 @@ namespace Musify.Infrastructure.Persistence
{ {
var configuration = new ConfigurationBuilder() var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory()) .SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("DesignSettings.json") .AddJsonFile("DesignSettings.json", optional: true)
.AddUserSecrets<DatabaseDesignTimeFactory>() .AddUserSecrets<DatabaseDesignTimeFactory>()
.AddEnvironmentVariables()
.Build(); .Build();
var databaseConfiguration = configuration var databaseConfiguration = configuration
backend/Musify.Migrator.Dockerfile
+25
View File
@@ -0,0 +1,25 @@
# Build context: backend/ (docker compose -f deploy/compose.yml up migrate)
#
# Applies EF Core migrations, then exits. Musify.Infrastructure is both the
# migrations and the design-time startup project: it owns the
# IDesignTimeDbContextFactory and the EFCore.Design package. The connection
# string comes from the Database__ConnectionString env var (compose sets it),
# read via DatabaseDesignTimeFactory's AddEnvironmentVariables().
FROM mcr.microsoft.com/dotnet/sdk:10.0
WORKDIR /src
COPY Directory.Build.props Directory.Packages.props nuget.config ./
COPY Core/Musify.Domain/Musify.Domain.csproj Core/Musify.Domain/
COPY Core/Musify.Application/Musify.Application.csproj Core/Musify.Application/
COPY Core/Musify.Infrastructure/Musify.Infrastructure.csproj Core/Musify.Infrastructure/
RUN dotnet restore Core/Musify.Infrastructure/Musify.Infrastructure.csproj
COPY Core/Musify.Domain Core/Musify.Domain/
COPY Core/Musify.Application Core/Musify.Application/
COPY Core/Musify.Infrastructure Core/Musify.Infrastructure/
RUN dotnet tool install --global dotnet-ef --version 10.0.8
ENV PATH="$PATH:/root/.dotnet/tools"
WORKDIR /src/Core/Musify.Infrastructure
ENTRYPOINT ["dotnet", "ef", "database", "update", "--context", "Database"]
deploy/.env.example
+8 -3
View File
@@ -1,5 +1,7 @@
# Development configuration for deploy/compose.yml + compose.dev.yml. # Development configuration for deploy/compose.yml + compose.dev.yml.
# Copy to deploy/.env (up.ps1 does it for you) and adjust. Not versioned. # Copy to deploy/.env and adjust. Not versioned.
#
# cp deploy/.env.example deploy/.env
DOTNET_ENVIRONMENT=Development DOTNET_ENVIRONMENT=Development
@@ -22,7 +24,7 @@ POSTGRES_HOST_PORT=59000
# --- Zitadel (OIDC) ---------------------------------------------------------- # --- Zitadel (OIDC) ----------------------------------------------------------
# The masterkey MUST be exactly 32 characters. The external domain/port form # The masterkey MUST be exactly 32 characters. The external domain/port form
# the token issuer, so they have to match PUBLIC_AUTH_URL. Changing them needs # the token issuer, so they have to match PUBLIC_AUTH_URL. Changing them needs
# a re-init: ./up.ps1 -Destroy; ./up.ps1 # a re-init: docker compose -f deploy/compose.yml -f deploy/compose.dev.yml down -v
ZITADEL_MASTERKEY=MasterkeyNeedsToHave32Characters ZITADEL_MASTERKEY=MasterkeyNeedsToHave32Characters
ZITADEL_EXTERNAL_DOMAIN=host.docker.internal ZITADEL_EXTERNAL_DOMAIN=host.docker.internal
ZITADEL_EXTERNAL_PORT=8080 ZITADEL_EXTERNAL_PORT=8080
@@ -31,8 +33,11 @@ ZITADEL_DATABASE_PASSWORD=zitadel
# Console login: <username>@zitadel.<external domain> # Console login: <username>@zitadel.<external domain>
ZITADEL_ADMIN_USERNAME=admin ZITADEL_ADMIN_USERNAME=admin
ZITADEL_ADMIN_PASSWORD=Password1! ZITADEL_ADMIN_PASSWORD=Password1!
# Allows the OIDC apps to use plain-http redirect URIs (dev only).
ZITADEL_DEV_MODE=true
# OIDC clients, filled in by deploy/zitadel/provision.ps1. # OIDC clients: the zitadel-init one-shot service (see compose.yml) fills
# these in automatically on `docker compose up -d`.
AUTH_CLIENT_ID= AUTH_CLIENT_ID=
WEB_CLIENT_ID= WEB_CLIENT_ID=
WEB_CLIENT_SECRET= WEB_CLIENT_SECRET=
deploy/.env.prod.example
+5 -2
View File
@@ -1,5 +1,5 @@
# Production configuration for deploy/compose.yml + compose.prod.yml. # Production configuration for deploy/compose.yml + compose.prod.yml.
# Copy to deploy/.env.prod (up.sh does it for you), then: # Copy to deploy/.env.prod, then:
# #
# 1. replace every `example.com` with your domain, # 1. replace every `example.com` with your domain,
# 2. replace every CHANGE_ME with a strong random value # 2. replace every CHANGE_ME with a strong random value
@@ -35,8 +35,11 @@ ZITADEL_DATABASE_PASSWORD=CHANGE_ME
# Console login: <username>@zitadel.<external domain> # Console login: <username>@zitadel.<external domain>
ZITADEL_ADMIN_USERNAME=admin ZITADEL_ADMIN_USERNAME=admin
ZITADEL_ADMIN_PASSWORD=CHANGE_ME ZITADEL_ADMIN_PASSWORD=CHANGE_ME
# Keep OIDC apps strict about https redirect URIs in production.
ZITADEL_DEV_MODE=false
# OIDC clients, filled in by deploy/zitadel/provision.sh. # OIDC clients: the zitadel-init one-shot service (see compose.yml) fills
# these in automatically on `docker compose up -d`.
AUTH_CLIENT_ID= AUTH_CLIENT_ID=
WEB_CLIENT_ID= WEB_CLIENT_ID=
WEB_CLIENT_SECRET= WEB_CLIENT_SECRET=
deploy/README.md
+87 -59
View File
@@ -1,44 +1,68 @@
# Deploy # Deploy
Everything needed to run Musify, in development and in production, from the Everything needed to run Musify, in development and in production, from the
same set of compose files. same set of compose files — no wrapper scripts, just `docker compose`.
| File | Role | | File | Role |
|---|---| |---|---|
| `compose.yml` | Every service. No published ports, no domains, no environment-specific values. | | `compose.yml` | Every service, including the one-shot bootstrap jobs. No published ports, no domains, no environment-specific values. |
| `compose.dev.yml` | Development overlay: publishes the ports to the host, adds pgAdmin. | | `compose.dev.yml` | Development overlay: publishes the ports to the host, adds pgAdmin. |
| `compose.prod.yml` | Production overlay: adds the nginx edge and the mounted secrets. | | `compose.prod.yml` | Production overlay: adds the nginx edge and the mounted secrets. |
| `.env` / `.env.prod` | The values compose interpolates. Not versioned; templates in `.env.example` / `.env.prod.example`. | | `.env` / `.env.prod` | The values compose interpolates. Not versioned; templates in `.env.example` / `.env.prod.example`. |
| `up.ps1` | Development orchestrator (Windows). |
| `up.sh` | Production orchestrator (the server). |
The four applications (`api`, `worker`, `gateway`, `web-player`) sit behind the The four applications (`api`, `worker`, `gateway`, `web-player`) sit behind the
`apps` compose profile, so a plain `up` starts infrastructure only — the usual `apps` compose profile, so a plain `up` starts infrastructure only — the usual
development setup, with the apps running from the IDE. development setup, with the apps running from the IDE.
### The one-shot bootstrap jobs
Three services in `compose.yml` run once and exit, so a plain `docker compose
up -d` leaves the stack fully ready with no manual steps:
| Service | Does |
|---|---|
| `keys-init` | Generates the RS256 stream-ticket key pair in `./keys`, if it is not there already. |
| `migrate` | Applies the EF Core migrations (built from `backend/Musify.Migrator.Dockerfile`). |
| `zitadel-init` | Creates the Musify project and its two OIDC apps in Zitadel (or reuses them), then writes `AUTH_CLIENT_ID` / `WEB_CLIENT_ID` / `WEB_CLIENT_SECRET` into the env file. |
`api` and `worker` wait for `migrate`; `api` and `gateway` wait for
`keys-init` (`depends_on: condition: service_completed_successfully`). They do
**not** wait for `zitadel-init` — compose interpolates `${AUTH_CLIENT_ID}` once,
when a command starts, so a container that starts in the *same* `up` as
`zitadel-init` would still get the empty value it had before provisioning ran.
That's why bringing up the apps is a second command: by the time you run it,
`zitadel-init` has already finished and the env file has real values.
--- ---
## Development ## Development
```powershell ```sh
./deploy/up.ps1 # infrastructure only, apps from the IDE docker compose -f deploy/compose.yml -f deploy/compose.dev.yml up -d
./deploy/up.ps1 -Apps # build and run the apps in Docker too docker compose -f deploy/compose.yml -f deploy/compose.dev.yml --profile apps up -d --build
``` ```
`up.ps1` does the whole bootstrap, with no manual steps afterwards: The first command starts the infrastructure and runs the bootstrap jobs
(keys, migrations, Zitadel project/apps). The second builds and runs `api`,
`worker`, `gateway` and `web-player` in Docker too — skip it to run the apps
from the IDE instead (the usual dev loop):
1. Creates `deploy/.env` from `.env.example` if it is missing. ```sh
2. Generates the RS256 stream-ticket keys in `deploy/keys/` and the Worker dotnet run --project backend/Hosts/Musify.Api # :5111
scratch directory `deploy/.tmp/`. dotnet run --project backend/Hosts/Musify.StreamingGateway # :8081
3. Starts the stack and waits for Postgres. dotnet run --project backend/Hosts/Musify.Worker # background
4. Applies the EF Core migrations. npm --prefix web-player run dev # :5173
5. Provisions Zitadel: project, API OIDC app and web player OIDC app, then ```
writes the client ids into `deploy/.env`, `Musify.Api` user-secrets and
`web-player/.env`.
6. Prints a summary with every URL and credential.
Other switches: `-Tools` (pgAdmin), `-SkipMigrations`, `-Down` (stop, keep When running the apps from the IDE, `Musify.Api` reads `Authentication:*`
data), `-Destroy` (stop and wipe the volumes). from user-secrets and `web-player` reads its own `web-player/.env` — neither
looks at `deploy/.env`. Copy the three values `zitadel-init` wrote
(`AUTH_CLIENT_ID`, `WEB_CLIENT_ID`, `WEB_CLIENT_SECRET`) into
`dotnet user-secrets set Authentication:ClientId <value> --project
backend/Hosts/Musify.Api` and into `web-player/.env` once; they stay valid
until you wipe the Zitadel volume.
Add `--profile tools` to also start pgAdmin.
### Ports ### Ports
@@ -49,20 +73,12 @@ data), `-Destroy` (stop and wipe the volumes).
| RabbitMQ | `5672` / `15672` | MassTransit / management UI | | RabbitMQ | `5672` / `15672` | MassTransit / management UI |
| SeaweedFS | `8333` / `8888` / `9333` | S3 / filer (→ gateway) / master | | SeaweedFS | `8333` / `8888` / `9333` | S3 / filer (→ gateway) / master |
| Jaeger | `16686` / `4317` / `4318` | traces UI / OTLP gRPC / OTLP HTTP | | Jaeger | `16686` / `4317` / `4318` | traces UI / OTLP gRPC / OTLP HTTP |
| pgAdmin (`-Tools`) | `5050` | Postgres UI | | pgAdmin (`--profile tools`) | `5050` | Postgres UI |
| API, gateway, web player (`-Apps`) | `5111` / `8081` / `3000` | | | API, gateway, web player (`--profile apps`) | `5111` / `8081` / `3000` | |
### Running the apps from the IDE Host requirements: .NET 10 SDK, Node 22, Docker Desktop. (ffmpeg and yt-dlp
are only needed on the host if you run the Worker outside Docker — inside
```powershell Docker its image already has them.)
dotnet run --project backend/Hosts/Musify.Api # :5111
dotnet run --project backend/Hosts/Musify.StreamingGateway # :8081
dotnet run --project backend/Hosts/Musify.Worker # background
npm --prefix web-player run dev # :5173
```
Host requirements: .NET 10 SDK, Node 22, ffmpeg and yt-dlp on PATH (the Worker
transcodes and downloads audio), Docker Desktop.
### Why `host.docker.internal` ### Why `host.docker.internal`
@@ -71,36 +87,51 @@ resolves to the same address from the browser, from apps running on the host
and from inside the containers. That keeps the OIDC token issuer and the and from inside the containers. That keeps the OIDC token issuer and the
presigned S3 URLs valid whichever way the apps are running. Zitadel routes by presigned S3 URLs valid whichever way the apps are running. Zitadel routes by
the `Host` header, so the domain has to be identical everywhere; changing it the `Host` header, so the domain has to be identical everywhere; changing it
requires a re-init (`./up.ps1 -Destroy; ./up.ps1`). requires wiping the Postgres/Zitadel volumes and starting over (`docker
compose -f compose.yml -f compose.dev.yml down -v`, then `up -d` again).
### Resetting
```sh
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml down # stop, keep data
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml down -v # stop, wipe volumes
```
--- ---
## Production ## Production
One command on the server, from the repository root: On a server with a domain, from the repository root:
```sh ```sh
./deploy/up.sh cp deploy/.env.prod.example deploy/.env.prod
``` ```
It refuses to start until the two things it cannot generate are in place: Fill it in:
1. **`deploy/.env.prod`** — copied from `.env.prod.example` on the first run. 1. Replace every `example.com` with your domain and every `CHANGE_ME` with a
Replace every `example.com` with your domain and every `CHANGE_ME` with a random value (`openssl rand -hex 24`; the Zitadel masterkey needs exactly
random value (`openssl rand -hex 24`; the Zitadel masterkey needs exactly 32 32 characters). `chmod 600 deploy/.env.prod`.
characters). `chmod 600` it. 2. Drop **`deploy/nginx/certs/origin.pem` and `origin.key`** — the origin
2. **`deploy/nginx/certs/origin.pem` and `origin.key`** — the origin
certificate nginx serves. The stack assumes Cloudflare in *Full (strict)* certificate nginx serves. The stack assumes Cloudflare in *Full (strict)*
mode in front of it, so a Cloudflare Origin Certificate is enough; any mode in front of it, so a Cloudflare Origin Certificate is enough; any
certificate valid for the five hostnames works. certificate valid for the five hostnames works.
3. Point `example.com`, `www`, `api`, `auth`, `stream` and `s3` at the server,
proxied/TLS-terminated the same way.
DNS: point `example.com`, `www`, `api`, `auth`, `stream` and `s3` at the server. Then, same two commands as development, with the prod env file and overlay:
Then `up.sh` generates the stream keys, starts the infrastructure and the edge, ```sh
provisions Zitadel (it needs `curl` and `jq` on the server), applies the docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml up -d
migrations and finally builds and starts the applications. It is idempotent — docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml --profile apps up -d --build
run it again to deploy a new version, or `./deploy/up.sh --no-build` to restart ```
without rebuilding.
The first command starts the infrastructure, the nginx edge and the bootstrap
jobs; the second builds and starts the applications, already wired to the
OIDC client ids the first one produced. Both are idempotent — run them again
to deploy a new version (`docker compose ... --profile apps up -d --build`
alone is enough for an update once the stack already exists; it reuses the
infra containers and only rebuilds and restarts the apps).
Only nginx publishes ports (80/443); everything else is reachable only on the Only nginx publishes ports (80/443); everything else is reachable only on the
internal compose network. internal compose network.
@@ -118,30 +149,23 @@ empty if you do not have one.
``` ```
deploy/ deploy/
├─ compose.yml # all services, environment-independent ├─ compose.yml # all services, incl. keys-init / migrate / zitadel-init
├─ compose.dev.yml # dev overlay: host ports, pgAdmin ├─ compose.dev.yml # dev overlay: host ports, pgAdmin
├─ compose.prod.yml # prod overlay: nginx edge, secrets ├─ compose.prod.yml # prod overlay: nginx edge, secrets
├─ .env.example / .env.prod.example ├─ .env.example / .env.prod.example
├─ up.ps1 # dev orchestrator
├─ up.sh # prod orchestrator
├─ keys/ # RS256 stream-ticket key pair (generated) ├─ keys/ # RS256 stream-ticket key pair (generated)
├─ secrets/ # mounted into the Worker (prod, optional) ├─ secrets/ # mounted into the Worker (prod, optional)
├─ nginx/ ├─ nginx/
│ ├─ templates/default.conf.template # vhosts, rendered with the domain by envsubst │ ├─ templates/default.conf.template # vhosts, rendered with the domain by envsubst
│ ├─ snippets/proxy.conf # proxy headers shared by every vhost │ ├─ snippets/proxy.conf # proxy headers shared by every vhost
│ └─ certs/ # origin.pem + origin.key (not versioned) │ └─ certs/ # origin.pem + origin.key (not versioned)
├─ scripts/
│ ├─ common.ps1 / lib.sh # shared helpers
│ ├─ stream-keys.ps1 / .sh # RS256 key pair generation
│ └─ migrate.ps1 / .sh # EF Core migrations
└─ zitadel/ └─ zitadel/
├─ provision.ps1 / .sh # project + OIDC apps, wires the client ids
└─ .output/ # PAT written by Zitadel on init (not versioned) └─ .output/ # PAT written by Zitadel on init (not versioned)
``` ```
The application images are built from `backend/Musify.*.Dockerfile` (build The application images are built from `backend/Musify.*.Dockerfile` (build
context `backend/`, so restore can copy the central props and every csproj) and context `backend/`, so restore can copy the central props and every csproj)
`web-player/Dockerfile`. and `web-player/Dockerfile`.
## Notes ## Notes
@@ -151,8 +175,12 @@ context `backend/`, so restore can copy the central props and every csproj) and
- **Zitadel** runs `start-from-init`, which is idempotent: the same command - **Zitadel** runs `start-from-init`, which is idempotent: the same command
works on an empty volume and on an existing one. works on an empty volume and on an existing one.
- **`latest` Zitadel images** require the standalone Login UI v2 container, - **`latest` Zitadel images** require the standalone Login UI v2 container,
which this stack does not run. The provisioning scripts turn that requirement which this stack does not run. `zitadel-init` turns that requirement off so
off so the built-in `/ui/login` is used. the built-in `/ui/login` is used.
- **`AppSettings*.json` are PascalCase** in `Musify.Api` and `Musify.Worker`; - **`AppSettings*.json` are PascalCase** in `Musify.Api` and `Musify.Worker`;
on case-sensitive Linux the host looks for `appsettings*.json`, so their on case-sensitive Linux the host looks for `appsettings*.json`, so their
Dockerfiles create lowercase copies at publish time. Dockerfiles create lowercase copies at publish time.
- **Re-running `zitadel-init` or `migrate`** is safe: the former reuses the
existing project/apps by name (regenerating the web app's secret only if it
is missing from the env file), the latter is a normal `dotnet ef database
update`, a no-op once the schema is current.
deploy/compose.dev.yml
+5
View File
@@ -40,6 +40,11 @@ services:
- "4317:4317" # OTLP gRPC - "4317:4317" # OTLP gRPC
- "4318:4318" # OTLP HTTP - "4318:4318" # OTLP HTTP
# zitadel-init (compose.yml) edits whichever env file is mounted here.
zitadel-init:
volumes:
- ./.env:/workspace/.env
# Optional Postgres UI: docker compose ... --profile tools up -d # Optional Postgres UI: docker compose ... --profile tools up -d
pgadmin: pgadmin:
profiles: [tools] profiles: [tools]
deploy/compose.prod.yml
+19 -8
View File
@@ -4,11 +4,13 @@ name: musify-prod
# Nothing but nginx publishes a port; every other service is reachable only on # Nothing but nginx publishes a port; every other service is reachable only on
# the internal `musify-prod_default` network. # the internal `musify-prod_default` network.
# #
# docker compose --env-file deploy/.env.prod \ # docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml up -d
# -f deploy/compose.yml -f deploy/compose.prod.yml --profile apps up -d --build # docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml --profile apps up -d --build
# #
# Use ./up.sh, which also generates the stream keys, provisions Zitadel and # The first command starts the infrastructure, the nginx edge and the
# applies the database migrations. # one-shot bootstrap jobs (stream keys, migrations, Zitadel project/apps); the
# second builds and starts the applications, picking up the OIDC client ids
# the first command just wrote into deploy/.env.prod. See deploy/README.md.
services: services:
worker: worker:
@@ -16,14 +18,23 @@ services:
volumes: volumes:
- ./secrets:/secrets:ro - ./secrets:/secrets:ro
# zitadel-init (compose.yml) edits whichever env file is mounted here.
zitadel-init:
volumes:
- ./.env.prod:/workspace/.env
# Every vhost proxies through `set $upstream ...; proxy_pass $upstream;`
# with the embedded Docker DNS resolver (see nginx/templates), so nginx
# does not need api/gateway/web-player to exist at its own startup — only
# at request time. That is what lets it start in the FIRST command
# (profile-gated services aren't part of that invocation yet) and still be
# reachable at the auth domain for zitadel-init to provision through.
nginx: nginx:
image: nginx:1.27-alpine image: nginx:1.27-alpine
container_name: musify-nginx container_name: musify-nginx
depends_on: depends_on:
- api zitadel:
- gateway condition: service_started
- zitadel
- web-player
environment: environment:
# The entrypoint renders /etc/nginx/templates/*.template with envsubst. # The entrypoint renders /etc/nginx/templates/*.template with envsubst.
# The filter keeps it from eating nginx's own $host / $remote_addr. # The filter keeps it from eating nginx's own $host / $remote_addr.
deploy/compose.share.yml
-15
View File
@@ -1,15 +0,0 @@
name: musify-share
# Temporary overlay for exposing the stack outside the LAN, on top of
# compose.dev.yml. Only difference from a normal dev run: Zitadel writes its
# init PAT to a separate host directory, so it never overwrites
# zitadel/.output (used by the musify-dev project's own Zitadel instance).
#
# docker compose --env-file deploy/.env.share \
# -f deploy/compose.yml -f deploy/compose.dev.yml -f deploy/compose.share.yml \
# --profile apps up -d --build api gateway web-player
services:
zitadel:
volumes:
- ./zitadel/.output-share:/output
deploy/compose.yml
+209 -7
View File
@@ -8,9 +8,12 @@
# prod docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml up -d # prod docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml up -d
# #
# The four application services sit behind the `apps` profile, so a plain `up` # The four application services sit behind the `apps` profile, so a plain `up`
# starts only the infrastructure (the usual dev setup: apps from the IDE). # starts the infrastructure plus the one-shot bootstrap jobs (stream-ticket
# Add `--profile apps` to run them in Docker. Use ./up.ps1 (dev) or ./up.sh # keys, database migrations, Zitadel project/apps) and stops there — the usual
# (prod) instead of calling compose by hand. # dev setup, with the apps running from the IDE. Add `--profile apps up -d
# --build` as a SECOND command to also build and run them in Docker: by then
# zitadel-init has already written the OIDC client ids into the env file, so
# the apps start with real values. See deploy/README.md.
x-app-depends: &app-depends x-app-depends: &app-depends
postgres: postgres:
@@ -168,6 +171,196 @@ services:
aws --endpoint-url http://seaweedfs:8333 s3 ls aws --endpoint-url http://seaweedfs:8333 s3 ls
restart: "no" restart: "no"
# One-shot: generates the RS256 key pair used to sign (API) and verify
# (Gateway) stream tickets, if it is not already in ./keys. No host tools
# required — openssl runs inside the container.
keys-init:
image: alpine:3.20
container_name: musify-keys-init
entrypoint: ["/bin/sh", "-c"]
command:
- |
set -e
if [ -f /keys/stream_private.pem ] && [ -f /keys/stream_public.pem ]; then
echo "stream-ticket keys already present"
else
apk add --no-cache openssl >/dev/null
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out /keys/stream_private.pem
openssl rsa -in /keys/stream_private.pem -pubout -out /keys/stream_public.pem
chmod 644 /keys/stream_public.pem
chmod 600 /keys/stream_private.pem
echo "generated stream-ticket keys"
fi
volumes:
- ./keys:/keys
restart: "no"
# One-shot: applies the EF Core migrations, then exits. See
# backend/Musify.Migrator.Dockerfile.
migrate:
build:
context: ../backend
dockerfile: Musify.Migrator.Dockerfile
container_name: musify-migrate
depends_on:
postgres:
condition: service_healthy
environment:
Database__ConnectionString: "Host=postgres;Port=5432;Database=${POSTGRES_DB};Username=${POSTGRES_USER};Password=${POSTGRES_PASSWORD}"
restart: "no"
# One-shot: creates the Musify project and its two OIDC apps in Zitadel (or
# reuses them if they already exist), then writes AUTH_CLIENT_ID,
# WEB_CLIENT_ID and WEB_CLIENT_SECRET back into the env file mounted at
# /workspace/.env (see compose.dev.yml / compose.prod.yml) — so a later
# `--profile apps up` picks up real values instead of empty ones.
zitadel-init:
image: alpine:3.20
container_name: musify-zitadel-init
depends_on:
zitadel:
condition: service_started
environment:
PUBLIC_AUTH_URL: ${PUBLIC_AUTH_URL}
PUBLIC_API_URL: ${PUBLIC_API_URL}
PUBLIC_WEB_URL: ${PUBLIC_WEB_URL}
ZITADEL_DEV_MODE: ${ZITADEL_DEV_MODE:-false}
volumes:
- ./zitadel/.output:/output:ro
entrypoint: ["/bin/sh", "-c"]
command:
- |
set -e
apk add --no-cache curl jq >/dev/null
ENV_FILE=/workspace/.env
if [ ! -f "$$ENV_FILE" ]; then
echo "no env file mounted at $$ENV_FILE, skipping"
exit 0
fi
# The env file may have CRLF line endings (edited on Windows), so
# strip \r on read and drop it on write — otherwise a "blank" value
# is actually a lone \r, and `[ -z ... ]` lies.
env_get() { grep -E "^$$2=" "$$1" 2>/dev/null | tail -n1 | cut -d= -f2- | tr -d '\r'; }
env_set() {
# $$1 is a bind-mounted file: rewrite its content in place rather
# than renaming a temp file over it (rename onto a bind-mounted
# inode fails with "Resource busy").
if grep -qE "^$$2=" "$$1"; then
awk -v k="$$2" -v v="$$3" '{ sub(/\r$$/, "") } index($$0,k "=")==1{print k "=" v; next} {print}' "$$1" >"$$1.tmp" && cat "$$1.tmp" >"$$1" && rm -f "$$1.tmp"
else
printf '%s=%s\n' "$$2" "$$3" >>"$$1"
fi
}
echo "==> waiting for the Zitadel service-account PAT"
deadline=$$(( $$(date +%s) + 180 ))
while [ ! -f /output/admin-sa.pat ]; do
[ "$$(date +%s)" -lt "$$deadline" ] || { echo "PAT not found, check 'docker logs musify-zitadel'" >&2; exit 1; }
sleep 3
done
PAT=$$(tr -d '\r\n' </output/admin-sa.pat)
MGMT="$$PUBLIC_AUTH_URL/management/v1"
echo "==> waiting for Zitadel to answer at $$PUBLIC_AUTH_URL"
deadline=$$(( $$(date +%s) + 120 ))
until curl -fsS "$$PUBLIC_AUTH_URL/.well-known/openid-configuration" >/dev/null 2>&1; do
[ "$$(date +%s)" -lt "$$deadline" ] || { echo "Zitadel did not respond at $$PUBLIC_AUTH_URL" >&2; exit 1; }
sleep 3
done
api_call() {
if [ $$# -ge 3 ]; then
curl -fsS -X "$$1" "$$MGMT$$2" -H "Authorization: Bearer $$PAT" -H "Content-Type: application/json" -d "$$3"
else
curl -fsS -X "$$1" "$$MGMT$$2" -H "Authorization: Bearer $$PAT"
fi
}
name_query() { printf '{"queries":[{"nameQuery":{"name":"%s","method":"TEXT_QUERY_METHOD_EQUALS"}}]}' "$$1"; }
# `latest` images require the standalone Login UI v2 container, which
# this stack does not run; opting out keeps the built-in /ui/login.
curl -fsS -X PUT "$$PUBLIC_AUTH_URL/v2/features/instance" \
-H "Authorization: Bearer $$PAT" -H "Content-Type: application/json" \
-d '{"loginV2":{"required":false}}' >/dev/null 2>&1 || echo "warning: could not disable Login UI v2"
echo "==> project"
PROJECT_NAME=Musify
PROJECT_ID=$$(api_call POST /projects/_search "$$(name_query "$$PROJECT_NAME")" | jq -r --arg n "$$PROJECT_NAME" '.result // [] | map(select(.name==$$n)) | first | .id // empty')
if [ -z "$$PROJECT_ID" ]; then
PROJECT_ID=$$(api_call POST /projects "$$(printf '{"name":"%s"}' "$$PROJECT_NAME")" | jq -r '.id')
echo " created (id $$PROJECT_ID)"
else
echo " reused (id $$PROJECT_ID)"
fi
find_app() { api_call POST "/projects/$$PROJECT_ID/apps/_search" "$$(name_query "$$1")" | jq -r --arg n "$$1" '.result // [] | map(select(.name==$$n)) | first | .id // empty'; }
echo "==> API app"
API_APP_NAME="Musify API (Scalar)"
API_APP_ID=$$(find_app "$$API_APP_NAME")
if [ -n "$$API_APP_ID" ]; then
API_CLIENT_ID=$$(api_call GET "/projects/$$PROJECT_ID/apps/$$API_APP_ID" | jq -r '.app.oidcConfig.clientId')
echo " reused (clientId $$API_CLIENT_ID)"
else
API_CLIENT_ID=$$(api_call POST "/projects/$$PROJECT_ID/apps/oidc" "$$(jq -n --arg name "$$API_APP_NAME" --arg api "$$PUBLIC_API_URL" '{
name: $$name,
redirectUris: [($$api + "/scalar/"), ($$api + "/scalar/oauth2-redirect.html")],
postLogoutRedirectUris: [($$api + "/scalar/")],
responseTypes: ["OIDC_RESPONSE_TYPE_CODE"],
grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE"],
appType: "OIDC_APP_TYPE_USER_AGENT",
authMethodType: "OIDC_AUTH_METHOD_TYPE_NONE",
version: "OIDC_VERSION_1_0",
devMode: ($$ENV.ZITADEL_DEV_MODE == "true"),
accessTokenType: "OIDC_TOKEN_TYPE_JWT",
accessTokenRoleAssertion: true,
idTokenRoleAssertion: true,
idTokenUserinfoAssertion: true
}')" | jq -r '.clientId')
echo " created (clientId $$API_CLIENT_ID)"
fi
echo "==> web app"
WEB_APP_NAME="Musify Web"
WEB_APP_ID=$$(find_app "$$WEB_APP_NAME")
WEB_CLIENT_SECRET=$$(env_get "$$ENV_FILE" WEB_CLIENT_SECRET)
if [ -n "$$WEB_APP_ID" ]; then
WEB_CLIENT_ID=$$(api_call GET "/projects/$$PROJECT_ID/apps/$$WEB_APP_ID" | jq -r '.app.oidcConfig.clientId')
if [ -z "$$WEB_CLIENT_SECRET" ]; then
WEB_CLIENT_SECRET=$$(api_call POST "/projects/$$PROJECT_ID/apps/$$WEB_APP_ID/oidc_config/_generate_client_secret" '{}' | jq -r '.clientSecret')
echo " reused (clientId $$WEB_CLIENT_ID, secret regenerated)"
else
echo " reused (clientId $$WEB_CLIENT_ID)"
fi
else
CREATED=$$(api_call POST "/projects/$$PROJECT_ID/apps/oidc" "$$(jq -n --arg name "$$WEB_APP_NAME" --arg web "$$PUBLIC_WEB_URL" '{
name: $$name,
redirectUris: [($$web + "/auth/callback")],
postLogoutRedirectUris: [($$web + "/")],
responseTypes: ["OIDC_RESPONSE_TYPE_CODE"],
grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE", "OIDC_GRANT_TYPE_REFRESH_TOKEN"],
appType: "OIDC_APP_TYPE_WEB",
authMethodType: "OIDC_AUTH_METHOD_TYPE_BASIC",
version: "OIDC_VERSION_1_0",
devMode: ($$ENV.ZITADEL_DEV_MODE == "true"),
accessTokenType: "OIDC_TOKEN_TYPE_JWT",
accessTokenRoleAssertion: true,
idTokenRoleAssertion: true,
idTokenUserinfoAssertion: true
}')")
WEB_CLIENT_ID=$$(echo "$$CREATED" | jq -r '.clientId')
WEB_CLIENT_SECRET=$$(echo "$$CREATED" | jq -r '.clientSecret')
echo " created (clientId $$WEB_CLIENT_ID)"
fi
env_set "$$ENV_FILE" AUTH_CLIENT_ID "$$API_CLIENT_ID"
env_set "$$ENV_FILE" WEB_CLIENT_ID "$$WEB_CLIENT_ID"
env_set "$$ENV_FILE" WEB_CLIENT_SECRET "$$WEB_CLIENT_SECRET"
echo "==> wired into $$ENV_FILE"
restart: "no"
# OTLP collector + trace UI (MassTransit / EF Core / HTTP spans). # OTLP collector + trace UI (MassTransit / EF Core / HTTP spans).
jaeger: jaeger:
image: jaegertracing/all-in-one:latest image: jaegertracing/all-in-one:latest
@@ -184,7 +377,12 @@ services:
context: ../backend context: ../backend
dockerfile: Musify.Api.Dockerfile dockerfile: Musify.Api.Dockerfile
container_name: musify-api container_name: musify-api
depends_on: *app-depends depends_on:
<<: *app-depends
migrate:
condition: service_completed_successfully
keys-init:
condition: service_completed_successfully
environment: environment:
<<: *backend-env <<: *backend-env
ASPNETCORE_ENVIRONMENT: ${DOTNET_ENVIRONMENT} ASPNETCORE_ENVIRONMENT: ${DOTNET_ENVIRONMENT}
@@ -192,8 +390,7 @@ services:
StreamGateway__PublicBaseUrl: ${PUBLIC_STREAM_URL} StreamGateway__PublicBaseUrl: ${PUBLIC_STREAM_URL}
StreamTicket__PrivateKeyPath: /keys/stream_private.pem StreamTicket__PrivateKeyPath: /keys/stream_private.pem
Workers__Routes__TemporaryFilesDirectory: /tmp/musify Workers__Routes__TemporaryFilesDirectory: /tmp/musify
# Client id and secret are written to the .env file by the Zitadel # AUTH_CLIENT_ID is written to the env file by the zitadel-init service.
# provisioning script (deploy/zitadel/provision.*).
Authentication__MetadataAddress: "${PUBLIC_AUTH_URL}/.well-known/openid-configuration" Authentication__MetadataAddress: "${PUBLIC_AUTH_URL}/.well-known/openid-configuration"
Authentication__IssuerAddress: ${PUBLIC_AUTH_URL} Authentication__IssuerAddress: ${PUBLIC_AUTH_URL}
Authentication__ClientId: ${AUTH_CLIENT_ID} Authentication__ClientId: ${AUTH_CLIENT_ID}
@@ -215,7 +412,10 @@ services:
context: ../backend context: ../backend
dockerfile: Musify.Worker.Dockerfile dockerfile: Musify.Worker.Dockerfile
container_name: musify-worker container_name: musify-worker
depends_on: *app-depends depends_on:
<<: *app-depends
migrate:
condition: service_completed_successfully
environment: environment:
<<: *backend-env <<: *backend-env
DOTNET_ENVIRONMENT: ${DOTNET_ENVIRONMENT} DOTNET_ENVIRONMENT: ${DOTNET_ENVIRONMENT}
@@ -235,6 +435,8 @@ services:
depends_on: depends_on:
seaweedfs: seaweedfs:
condition: service_healthy condition: service_healthy
keys-init:
condition: service_completed_successfully
environment: environment:
ASPNETCORE_ENVIRONMENT: ${DOTNET_ENVIRONMENT} ASPNETCORE_ENVIRONMENT: ${DOTNET_ENVIRONMENT}
ASPNETCORE_URLS: http://+:8081 ASPNETCORE_URLS: http://+:8081
deploy/scripts/common.ps1
-26
View File
@@ -1,26 +0,0 @@
#requires -Version 5.1
# Shared helpers for the development deploy scripts.
function Write-Step([string]$Message) { Write-Host "`n==> $Message" -ForegroundColor Cyan }
# Parses a .env file into an ordered hashtable (ignores comments and blank lines).
function Read-DotEnv([string]$Path) {
$values = [ordered]@{}
if (Test-Path $Path) {
Get-Content $Path | ForEach-Object {
if ($_ -match '^\s*([^#=\s][^=]*)=(.*)$') { $values[$matches[1].Trim()] = $matches[2].Trim() }
}
}
return $values
}
# Sets (or appends) a key in a .env file, preserving the rest of the file.
function Set-DotEnvValue([string]$Path, [string]$Key, [string]$Value) {
$lines = @(Get-Content $Path)
if ($lines -match "^\s*$Key=") {
$lines = $lines -replace "^\s*$Key=.*", "$Key=$Value"
} else {
$lines += "$Key=$Value"
}
Set-Content -Path $Path -Value $lines -Encoding utf8
}
deploy/scripts/lib.sh
-31
View File
@@ -1,31 +0,0 @@
#!/bin/sh
# Shared helpers for the production deploy scripts.
step() { printf '\n==> %s\n' "$1"; }
fail() { printf 'error: %s\n' "$1" >&2; exit 1; }
require_command() {
command -v "$1" >/dev/null 2>&1 || fail "$1 is required but not installed."
}
# env_get <file> <key> — prints the value, empty if the key is absent.
env_get() {
grep -E "^$2=" "$1" 2>/dev/null | tail -n 1 | cut -d= -f2-
}
# env_set <file> <key> <value> — replaces the key in place, appends if missing.
env_set() {
_file=$1; _key=$2; _value=$3
_tmp=$(mktemp)
if grep -qE "^$_key=" "$_file"; then
awk -v k="$_key" -v v="$_value" '
index($0, k "=") == 1 { print k "=" v; next }
{ print }
' "$_file" >"$_tmp"
else
cp "$_file" "$_tmp"
printf '%s=%s\n' "$_key" "$_value" >>"$_tmp"
fi
cat "$_tmp" >"$_file"
rm -f "$_tmp"
}
deploy/scripts/migrate.ps1
-15
View File
@@ -1,15 +0,0 @@
#requires -Version 5.1
# Applies EF Core migrations to the Musify database.
[CmdletBinding()]
param([Parameter(Mandatory)][string]$ConnectionString)
$ErrorActionPreference = "Stop"
$infraProject = Resolve-Path (Join-Path $PSScriptRoot "..\..\backend\Core\Musify.Infrastructure")
# The design-time factory reads the connection string from user-secrets.
dotnet user-secrets set "Database:ConnectionString" $ConnectionString --project $infraProject | Out-Null
# Infrastructure is also the startup project: it owns the IDesignTimeDbContextFactory
# and the EFCore.Design package (Musify.Api does not reference Design).
dotnet ef database update --project $infraProject --startup-project $infraProject --context Database
if ($LASTEXITCODE -ne 0) { throw "EF Core migration failed" }
deploy/scripts/migrate.sh
-28
View File
@@ -1,28 +0,0 @@
#!/bin/sh
# Applies the EF Core migrations to the production database from a throwaway SDK
# container attached to the compose network (Postgres is not published to the host).
set -eu
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
DEPLOY_DIR=$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)
BACKEND_DIR=$(CDPATH= cd -- "$DEPLOY_DIR/../backend" && pwd)
ENV_FILE="$DEPLOY_DIR/.env.prod"
NETWORK=${MUSIFY_NETWORK:-musify-prod_default}
. "$SCRIPT_DIR/lib.sh"
CONN="Host=postgres;Port=5432;Database=$(env_get "$ENV_FILE" POSTGRES_DB);Username=$(env_get "$ENV_FILE" POSTGRES_USER);Password=$(env_get "$ENV_FILE" POSTGRES_PASSWORD)"
# Musify.Infrastructure is both the migrations and the startup project: it owns
# the IDesignTimeDbContextFactory and the EFCore.Design package.
docker run --rm \
--network "$NETWORK" \
-v "$BACKEND_DIR":/src \
-w /src/Core/Musify.Infrastructure \
-e CONN="$CONN" \
mcr.microsoft.com/dotnet/sdk:10.0 \
sh -c 'dotnet tool install --global dotnet-ef >/dev/null 2>&1 || true
export PATH=$PATH:/root/.dotnet/tools
dotnet restore Musify.Infrastructure.csproj
dotnet user-secrets set "Database:ConnectionString" "$CONN" >/dev/null
dotnet ef database update --context Database'
deploy/scripts/stream-keys.ps1
-28
View File
@@ -1,28 +0,0 @@
#requires -Version 5.1
# Generates the RS256 key pair used to sign (API) and verify (Gateway) stream tickets.
# No-op if both keys already exist.
[CmdletBinding()]
param([Parameter(Mandatory)][string]$KeysDir)
$ErrorActionPreference = "Stop"
$privateKey = Join-Path $KeysDir "stream_private.pem"
$publicKey = Join-Path $KeysDir "stream_public.pem"
if ((Test-Path $privateKey) -and (Test-Path $publicKey)) {
Write-Host "Stream-ticket keys already present in $KeysDir"
return
}
if (-not (Test-Path $KeysDir)) { New-Item -ItemType Directory -Path $KeysDir | Out-Null }
if (Get-Command openssl -ErrorAction SilentlyContinue) {
& openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out $privateKey
& openssl rsa -in $privateKey -pubout -out $publicKey
} else {
$rsa = [System.Security.Cryptography.RSA]::Create(2048)
[System.IO.File]::WriteAllText($privateKey, $rsa.ExportPkcs8PrivateKeyPem())
[System.IO.File]::WriteAllText($publicKey, $rsa.ExportSubjectPublicKeyInfoPem())
$rsa.Dispose()
}
Write-Host "Generated stream-ticket keys in $KeysDir"
deploy/scripts/stream-keys.sh
-21
View File
@@ -1,21 +0,0 @@
#!/bin/sh
# Generates the RS256 key pair used to sign (API) and verify (Gateway) stream
# tickets. No-op if both keys already exist.
set -eu
KEYS_DIR=${1:?usage: stream-keys.sh <keys-dir>}
PRIVATE_KEY="$KEYS_DIR/stream_private.pem"
PUBLIC_KEY="$KEYS_DIR/stream_public.pem"
if [ -f "$PRIVATE_KEY" ] && [ -f "$PUBLIC_KEY" ]; then
echo "Stream-ticket keys already present in $KEYS_DIR"
exit 0
fi
mkdir -p "$KEYS_DIR"
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out "$PRIVATE_KEY" 2>/dev/null
openssl rsa -in "$PRIVATE_KEY" -pubout -out "$PUBLIC_KEY" 2>/dev/null
chmod 644 "$PUBLIC_KEY"
chmod 600 "$PRIVATE_KEY"
echo "Generated stream-ticket keys in $KEYS_DIR"
deploy/up.ps1
-145
View File
@@ -1,145 +0,0 @@
#requires -Version 5.1
<#
.SYNOPSIS
Brings up the Musify development environment.
.DESCRIPTION
Starts the infrastructure stack, generates the stream-ticket keys, applies
the EF Core migrations and provisions Zitadel (project + OIDC apps). The
.NET apps and the web player then run from the IDE; add -Apps to run them
in Docker instead.
.EXAMPLE
./up.ps1 # Infrastructure only (apps from the IDE)
./up.ps1 -Apps # Also build and run api, worker, gateway, web player
./up.ps1 -Tools # Also start pgAdmin
./up.ps1 -SkipMigrations # Skip the database migration step
./up.ps1 -Down # Stop and remove containers (keep volumes)
./up.ps1 -Destroy # Stop and remove containers + volumes (wipe data)
#>
[CmdletBinding()]
param(
[switch]$Apps,
[switch]$Tools,
[switch]$SkipMigrations,
[switch]$Down,
[switch]$Destroy
)
$ErrorActionPreference = "Stop"
$deployDir = $PSScriptRoot
$compose = @(
"-f", (Join-Path $deployDir "compose.yml"),
"-f", (Join-Path $deployDir "compose.dev.yml")
)
$profiles = @("--profile", "apps", "--profile", "tools")
. (Join-Path $deployDir "scripts\common.ps1")
# Teardown -------------------------------------------------------------------
if ($Down -or $Destroy) {
if ($Destroy) {
Write-Step "Stopping the stack and REMOVING volumes (data is wiped)"
docker compose @compose @profiles down -v --remove-orphans
# Drop the Zitadel init secrets so the next start re-initializes cleanly.
Get-ChildItem (Join-Path $deployDir "zitadel\.output") -File -ErrorAction SilentlyContinue | Remove-Item -Force
} else {
Write-Step "Stopping the stack (volumes are kept)"
docker compose @compose @profiles down --remove-orphans
}
return
}
# Config ---------------------------------------------------------------------
$envFile = Join-Path $deployDir ".env"
if (-not (Test-Path $envFile)) {
Write-Step "Creating .env from .env.example"
Copy-Item (Join-Path $deployDir ".env.example") $envFile
}
$cfg = Read-DotEnv $envFile
# Host directories the apps expect: RS256 keys (mounted into the api/gateway
# containers, read directly by the IDE runs) and the Worker scratch space.
& (Join-Path $deployDir "scripts\stream-keys.ps1") -KeysDir (Join-Path $deployDir "keys")
$tempDir = Join-Path $deployDir ".tmp"
if (-not (Test-Path $tempDir)) { New-Item -ItemType Directory -Path $tempDir | Out-Null }
# Infrastructure -------------------------------------------------------------
Write-Step "Starting the infrastructure stack"
$upArgs = @("up", "-d")
if ($Tools) { $upArgs = @("--profile", "tools") + $upArgs }
docker compose @compose @upArgs
if ($LASTEXITCODE -ne 0) { throw "docker compose up failed" }
Write-Step "Waiting for Postgres to become healthy"
$deadline = (Get-Date).AddMinutes(2)
do {
Start-Sleep -Seconds 2
$status = docker inspect --format '{{.State.Health.Status}}' musify-postgres 2>$null
} until ($status -eq "healthy" -or (Get-Date) -gt $deadline)
if ($status -ne "healthy") { throw "Postgres did not become healthy in time" }
# Database migrations --------------------------------------------------------
if ($SkipMigrations) {
Write-Host "`nSkipping migrations (-SkipMigrations)."
} else {
Write-Step "Applying EF Core migrations"
$conn = "Host=localhost;Port=$($cfg['POSTGRES_HOST_PORT']);Database=$($cfg['POSTGRES_DB']);Username=$($cfg['POSTGRES_USER']);Password=$($cfg['POSTGRES_PASSWORD'])"
& (Join-Path $deployDir "scripts\migrate.ps1") -ConnectionString $conn
}
# Zitadel provisioning -------------------------------------------------------
# Writes the client ids into .env, Musify.Api user-secrets and web-player/.env,
# so it has to run before the app containers start.
& (Join-Path $deployDir "zitadel\provision.ps1")
# Applications ---------------------------------------------------------------
if ($Apps) {
Write-Step "Building and starting the applications"
docker compose @compose --profile apps up -d --build
if ($LASTEXITCODE -ne 0) { throw "docker compose up (apps) failed" }
}
# Summary --------------------------------------------------------------------
$cfg = Read-DotEnv $envFile
$authUrl = $cfg['PUBLIC_AUTH_URL']
$appLines = if ($Apps) {
@"
Musify API $($cfg['PUBLIC_API_URL'])/scalar/v1
Streaming gateway $($cfg['PUBLIC_STREAM_URL'])/health
Web player $($cfg['PUBLIC_WEB_URL'])
"@
} else {
@"
Run the apps from the IDE (or ./up.ps1 -Apps to run them in Docker):
dotnet run --project backend/Hosts/Musify.Api # :5111
dotnet run --project backend/Hosts/Musify.StreamingGateway # :8081
dotnet run --project backend/Hosts/Musify.Worker # background
npm --prefix web-player run dev # :5173
"@
}
Write-Host @"
============================================================
Musify development environment READY
============================================================
Service URL / port
------------------ --------------------------------------
PostgreSQL localhost:$($cfg['POSTGRES_HOST_PORT']) (db: $($cfg['POSTGRES_DB']))
Zitadel console $authUrl/ui/console
RabbitMQ mgmt http://localhost:15672 ($($cfg['RABBITMQ_DEFAULT_USER'])/$($cfg['RABBITMQ_DEFAULT_PASS']))
SeaweedFS S3 http://localhost:8333 (bucket: $($cfg['S3_BUCKET']))
SeaweedFS filer http://localhost:8888
SeaweedFS master http://localhost:9333
Jaeger traces http://localhost:16686 (OTLP: localhost:4317)
pgAdmin (-Tools) http://localhost:$($cfg['PGADMIN_HOST_PORT']) ($($cfg['PGADMIN_DEFAULT_EMAIL'])/$($cfg['PGADMIN_DEFAULT_PASSWORD']))
Zitadel admin $($cfg['ZITADEL_ADMIN_USERNAME'])@zitadel.$($cfg['ZITADEL_EXTERNAL_DOMAIN']) / $($cfg['ZITADEL_ADMIN_PASSWORD'])
API client id $($cfg['AUTH_CLIENT_ID'])
Web client id $($cfg['WEB_CLIENT_ID'])
$appLines
============================================================
"@ -ForegroundColor Green
deploy/up.sh
-103
View File
@@ -1,103 +0,0 @@
#!/bin/sh
# Brings up (or updates) the Musify production stack on the server.
#
# ./up.sh build + start everything, provision Zitadel, migrate
# ./up.sh --no-build same, reusing the images that are already built
# ./up.sh --down stop and remove the containers (volumes are kept)
#
# Everything it needs lives in deploy/.env.prod (see .env.prod.example) and in
# deploy/nginx/certs/ (origin.pem + origin.key).
set -eu
DEPLOY_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
ENV_FILE="$DEPLOY_DIR/.env.prod"
INFRA="postgres zitadel rabbitmq seaweedfs seaweedfs-init jaeger"
. "$DEPLOY_DIR/scripts/lib.sh"
require_command docker
require_command openssl
compose() {
docker compose --env-file "$ENV_FILE" \
-f "$DEPLOY_DIR/compose.yml" -f "$DEPLOY_DIR/compose.prod.yml" \
--profile apps "$@"
}
BUILD=--build
for arg in "$@"; do
case "$arg" in
--no-build) BUILD= ;;
--down) step "Stopping the stack (volumes are kept)"; compose down --remove-orphans; exit 0 ;;
*) fail "unknown option: $arg" ;;
esac
done
# Preflight ------------------------------------------------------------------
if [ ! -f "$ENV_FILE" ]; then
cp "$DEPLOY_DIR/.env.prod.example" "$ENV_FILE"
chmod 600 "$ENV_FILE"
fail "created $ENV_FILE from the template — fill it in (domain + secrets) and run again."
fi
if grep -q 'CHANGE_ME\|example\.com' "$ENV_FILE"; then
fail "$ENV_FILE still contains CHANGE_ME or example.com placeholders."
fi
for cert in origin.pem origin.key; do
[ -f "$DEPLOY_DIR/nginx/certs/$cert" ] \
|| fail "missing nginx/certs/$cert (Cloudflare Origin Certificate)."
done
mkdir -p "$DEPLOY_DIR/secrets" "$DEPLOY_DIR/keys" "$DEPLOY_DIR/zitadel/.output"
case "$(env_get "$ENV_FILE" YTDLP_ADDITIONAL_ARGUMENTS)" in
*--cookies*)
[ -f "$DEPLOY_DIR/secrets/youtube_cookies.txt" ] \
|| echo "warning: YTDLP_ADDITIONAL_ARGUMENTS expects secrets/youtube_cookies.txt, which is missing." ;;
esac
step "Ensuring stream-ticket keys"
sh "$DEPLOY_DIR/scripts/stream-keys.sh" "$DEPLOY_DIR/keys"
# Infrastructure -------------------------------------------------------------
step "Starting the infrastructure"
# shellcheck disable=SC2086
compose up -d $INFRA
step "Waiting for Postgres"
deadline=$(( $(date +%s) + 120 ))
until [ "$(docker inspect --format '{{.State.Health.Status}}' musify-postgres 2>/dev/null)" = healthy ]; do
[ "$(date +%s)" -lt "$deadline" ] || fail "Postgres did not become healthy in time."
sleep 2
done
# nginx has to be up before provisioning: it terminates TLS for the Zitadel
# domain. --no-deps keeps the apps from starting before they have a client id.
step "Starting the edge"
compose up -d --no-deps nginx
# Zitadel + database ---------------------------------------------------------
sh "$DEPLOY_DIR/zitadel/provision.sh"
step "Applying EF Core migrations"
sh "$DEPLOY_DIR/scripts/migrate.sh"
# Applications ---------------------------------------------------------------
step "Starting the applications"
# shellcheck disable=SC2086
compose up -d $BUILD
step "Done"
compose ps
cat <<EOF
Web player $(env_get "$ENV_FILE" PUBLIC_WEB_URL)
API $(env_get "$ENV_FILE" PUBLIC_API_URL)/scalar/v1
Streaming $(env_get "$ENV_FILE" PUBLIC_STREAM_URL)/health
Zitadel $(env_get "$ENV_FILE" PUBLIC_AUTH_URL)/ui/console
$(env_get "$ENV_FILE" ZITADEL_ADMIN_USERNAME)@zitadel.$(env_get "$ENV_FILE" ZITADEL_EXTERNAL_DOMAIN)
Jaeger collects traces but publishes no port. To look at them, forward its
container address over SSH from your machine:
ssh -L 16686:\$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' musify-jaeger):16686 <server>
EOF
deploy/zitadel/.output-share/admin-sa.pat
-1
View File
@@ -1 +0,0 @@
eyJhbGciOiJBMjU2R0NNS1ciLCJlbmMiOiJBMjU2R0NNIiwiaXYiOiJ5Yk1hUFdVUmNqcU44WDNGIiwia2lkIjoib2lkY0tleSIsInRhZyI6Ikw2NjYwN3FRVzZNYVJLaVl3Q0w1N3cifQ.BE_7qqA1Auer5yEoGnyN0v35JlZlK3XgCo0s_CQtG7s.cOf9oudMHRnVTOGj.0DRh9kj7oahl4imvMjxqRLUOZo6NhYlQyslf9_x4XF_TKi1dnQ.Uu36qEL5J1VKSIYcvWlbAg
deploy/zitadel/provision.ps1
-193
View File
@@ -1,193 +0,0 @@
#requires -Version 5.1
<#
.SYNOPSIS
Provisions Zitadel for Musify in development (idempotent).
.DESCRIPTION
Creates the Musify project, the API OIDC app (public, PKCE, JWT access
token) and the web player OIDC app (confidential), then wires the resulting
client ids into deploy/.env, Musify.Api user-secrets and web-player/.env.
Authenticates with the service-account PAT that Zitadel writes on init
(zitadel/.output/admin-sa.pat). Re-running reuses the existing project and
apps and just refreshes the wired values.
#>
[CmdletBinding()]
param(
[string]$ProjectName = "Musify",
[string]$ApiAppName = "Musify API (Scalar)",
[string]$WebAppName = "Musify Web",
# Alternate .env to read/write (e.g. deploy/.env.share). Defaults to deploy/.env.
[string]$EnvFile,
# Skip writing Musify.Api user-secrets and web-player/.env: use when the apps
# only ever run in Docker for this environment and get their config from compose.
[switch]$SkipHostWiring,
# Matches the zitadel volume mount for this environment (see compose.share.yml).
[string]$OutputDir = ".output"
)
$ErrorActionPreference = "Stop"
$deployDir = Split-Path $PSScriptRoot -Parent
$repoDir = Split-Path $deployDir -Parent
$apiProject = Join-Path $repoDir "backend\Hosts\Musify.Api"
$webEnvFile = Join-Path $repoDir "web-player\.env"
$envFile = if ($EnvFile) { $EnvFile } else { Join-Path $deployDir ".env" }
$patFile = Join-Path $PSScriptRoot "$OutputDir\admin-sa.pat"
. (Join-Path $deployDir "scripts\common.ps1")
$cfg = Read-DotEnv $envFile
$issuer = $cfg['PUBLIC_AUTH_URL']
$apiUrl = $cfg['PUBLIC_API_URL']
$webUrl = $cfg['PUBLIC_WEB_URL']
$viteUrl = "http://localhost:5173" # `npm run dev` in web-player/
Write-Step "Provisioning Zitadel at $issuer"
# Zitadel writes the PAT during init; on a cold start it may take a moment.
$deadline = (Get-Date).AddMinutes(3)
while (-not (Test-Path $patFile) -and (Get-Date) -lt $deadline) { Start-Sleep -Seconds 3 }
if (-not (Test-Path $patFile)) { throw "PAT not found ($patFile). Check 'docker logs musify-zitadel'." }
$headers = @{ Authorization = "Bearer $((Get-Content $patFile -Raw).Trim())" }
$mgmt = "$issuer/management/v1"
function Invoke-Zitadel($method, $path, $body) {
$params = @{ Method = $method; Uri = "$mgmt$path"; Headers = $headers; ContentType = "application/json"; TimeoutSec = 20 }
if ($body) { $params.Body = ($body | ConvertTo-Json -Depth 10) }
Invoke-RestMethod @params
}
$deadline = (Get-Date).AddMinutes(2)
do {
try { Invoke-RestMethod "$issuer/.well-known/openid-configuration" -TimeoutSec 5 | Out-Null; $ready = $true }
catch { $ready = $false; Start-Sleep 3 }
} until ($ready -or (Get-Date) -gt $deadline)
if (-not $ready) { throw "Zitadel did not respond in time." }
# The images tagged `latest` require the standalone Login UI v2 container, which
# this stack does not run, so OIDC would redirect to /ui/v2/login and 404.
# Opting out keeps the built-in v1 login.
try {
Invoke-RestMethod -Method PUT -Uri "$issuer/v2/features/instance" -Headers $headers `
-ContentType "application/json" -Body (@{ loginV2 = @{ required = $false } } | ConvertTo-Json) -TimeoutSec 20 | Out-Null
} catch {
Write-Host " warning: could not disable the required Login UI v2: $($_.Exception.Message)"
}
# Project --------------------------------------------------------------------
$project = (Invoke-Zitadel POST "/projects/_search" @{
queries = @(@{ nameQuery = @{ name = $ProjectName; method = "TEXT_QUERY_METHOD_EQUALS" } })
}).result | Where-Object { $_.name -eq $ProjectName } | Select-Object -First 1
if ($project) {
$projectId = $project.id
Write-Host " project '$ProjectName' reused (id $projectId)"
} else {
$projectId = (Invoke-Zitadel POST "/projects" @{ name = $ProjectName }).id
Write-Host " project '$ProjectName' created (id $projectId)"
}
function Get-ZitadelApp($name) {
(Invoke-Zitadel POST "/projects/$projectId/apps/_search" @{
queries = @(@{ nameQuery = @{ name = $name; method = "TEXT_QUERY_METHOD_EQUALS" } })
}).result | Where-Object { $_.name -eq $name } | Select-Object -First 1
}
# API app: public client used by the Scalar UI to obtain a token -------------
$apiApp = Get-ZitadelApp $ApiAppName
if ($apiApp) {
$apiClientId = (Invoke-Zitadel GET "/projects/$projectId/apps/$($apiApp.id)" $null).app.oidcConfig.clientId
Write-Host " app '$ApiAppName' reused (clientId $apiClientId)"
} else {
$apiClientId = (Invoke-Zitadel POST "/projects/$projectId/apps/oidc" @{
name = $ApiAppName
redirectUris = @("$apiUrl/scalar/", "$apiUrl/scalar/oauth2-redirect.html")
postLogoutRedirectUris = @("$apiUrl/scalar/")
responseTypes = @("OIDC_RESPONSE_TYPE_CODE")
grantTypes = @("OIDC_GRANT_TYPE_AUTHORIZATION_CODE")
appType = "OIDC_APP_TYPE_USER_AGENT"
authMethodType = "OIDC_AUTH_METHOD_TYPE_NONE"
version = "OIDC_VERSION_1_0"
devMode = $true # allow http redirect URIs
accessTokenType = "OIDC_TOKEN_TYPE_JWT" # required so JwtBearer can validate it
accessTokenRoleAssertion = $true
idTokenRoleAssertion = $true
idTokenUserinfoAssertion = $true
}).clientId
Write-Host " app '$ApiAppName' created (clientId $apiClientId)"
}
# Web app: confidential client used by the SvelteKit server ------------------
$webApp = Get-ZitadelApp $WebAppName
$webSecret = $cfg['WEB_CLIENT_SECRET']
if ($webApp) {
$webClientId = (Invoke-Zitadel GET "/projects/$projectId/apps/$($webApp.id)" $null).app.oidcConfig.clientId
# The secret is only returned once, so regenerate it if we no longer have it.
if (-not $webSecret) {
$webSecret = (Invoke-Zitadel POST "/projects/$projectId/apps/$($webApp.id)/oidc_config/_generate_client_secret" @{}).clientSecret
Write-Host " app '$WebAppName' reused (clientId $webClientId, secret regenerated)"
} else {
Write-Host " app '$WebAppName' reused (clientId $webClientId)"
}
} else {
$created = Invoke-Zitadel POST "/projects/$projectId/apps/oidc" @{
name = $WebAppName
redirectUris = @("$webUrl/auth/callback", "$viteUrl/auth/callback")
postLogoutRedirectUris = @("$webUrl/", "$viteUrl/")
responseTypes = @("OIDC_RESPONSE_TYPE_CODE")
grantTypes = @("OIDC_GRANT_TYPE_AUTHORIZATION_CODE", "OIDC_GRANT_TYPE_REFRESH_TOKEN")
appType = "OIDC_APP_TYPE_WEB"
authMethodType = "OIDC_AUTH_METHOD_TYPE_BASIC"
version = "OIDC_VERSION_1_0"
devMode = $true
accessTokenType = "OIDC_TOKEN_TYPE_JWT"
accessTokenRoleAssertion = $true
idTokenRoleAssertion = $true
idTokenUserinfoAssertion = $true
}
$webClientId = $created.clientId
$webSecret = $created.clientSecret
Write-Host " app '$WebAppName' created (clientId $webClientId)"
}
# Wire the values ------------------------------------------------------------
Set-DotEnvValue $envFile "AUTH_CLIENT_ID" $apiClientId
Set-DotEnvValue $envFile "WEB_CLIENT_ID" $webClientId
Set-DotEnvValue $envFile "WEB_CLIENT_SECRET" $webSecret
if ($SkipHostWiring) {
Write-Host " wired into $envFile only (-SkipHostWiring: apps run in Docker)"
} else {
# Musify.Api reads these from user-secrets when it runs outside Docker.
@{
"Authentication:ClientId" = $apiClientId
"Authentication:AudienceAddress" = $apiClientId
"Authentication:IssuerAddress" = $issuer
"Authentication:MetadataAddress" = "$issuer/.well-known/openid-configuration"
"Authentication:AuthorizationEndpoint" = "$issuer/oauth/v2/authorize?prompt=login"
"Authentication:TokenEndpoint" = "$issuer/oauth/v2/token"
"Authentication:ScalarRedirectUri" = "$apiUrl/scalar/"
"Authentication:RequireHttpsMetadata" = "false"
}.GetEnumerator() | ForEach-Object {
dotnet user-secrets set $_.Key $_.Value --project $apiProject | Out-Null
}
# web-player/.env is what `npm run dev` reads (the container gets its config
# from compose instead).
if (-not (Test-Path $webEnvFile)) {
Copy-Item (Join-Path $repoDir "web-player\.env.example") $webEnvFile
}
@{
ZITADEL_ISSUER = $issuer
ZITADEL_CLIENT_ID = $webClientId
ZITADEL_CLIENT_SECRET = $webSecret
AUTH_REDIRECT_URI = "$viteUrl/auth/callback"
AUTH_POST_LOGOUT_URI = "$viteUrl/"
SESSION_SECRET = $cfg['SESSION_SECRET']
API_BASE_URL = $apiUrl
}.GetEnumerator() | ForEach-Object {
Set-DotEnvValue $webEnvFile $_.Key $_.Value
}
Write-Host " wired into deploy/.env, Musify.Api user-secrets and web-player/.env"
}
deploy/zitadel/provision.sh
-145
View File
@@ -1,145 +0,0 @@
#!/bin/sh
# Provisions Zitadel for Musify in production (idempotent).
#
# Creates the Musify project, the API OIDC app (public, PKCE, JWT access token)
# and the web player OIDC app (confidential), then writes AUTH_CLIENT_ID,
# WEB_CLIENT_ID and WEB_CLIENT_SECRET into deploy/.env.prod. Authenticates with
# the service-account PAT that Zitadel writes on init. Re-running reuses the
# existing project and apps.
set -eu
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
DEPLOY_DIR=$(CDPATH= cd -- "$SCRIPT_DIR/.." && pwd)
ENV_FILE="$DEPLOY_DIR/.env.prod"
PAT_FILE="$SCRIPT_DIR/.output/admin-sa.pat"
. "$DEPLOY_DIR/scripts/lib.sh"
require_command curl
require_command jq
PROJECT_NAME=${PROJECT_NAME:-Musify}
API_APP_NAME=${API_APP_NAME:-Musify API}
WEB_APP_NAME=${WEB_APP_NAME:-Musify Web}
ISSUER=$(env_get "$ENV_FILE" PUBLIC_AUTH_URL)
API_URL=$(env_get "$ENV_FILE" PUBLIC_API_URL)
WEB_URL=$(env_get "$ENV_FILE" PUBLIC_WEB_URL)
MGMT="$ISSUER/management/v1"
step "Provisioning Zitadel at $ISSUER"
# Zitadel writes the PAT during init; on a cold start it may take a moment.
deadline=$(( $(date +%s) + 180 ))
while [ ! -f "$PAT_FILE" ] && [ "$(date +%s)" -lt "$deadline" ]; do sleep 3; done
[ -f "$PAT_FILE" ] || fail "PAT not found ($PAT_FILE). Check 'docker logs musify-zitadel'."
PAT=$(tr -d '\r\n' <"$PAT_FILE")
api_call() {
# api_call <method> <path> [json body]
if [ $# -ge 3 ]; then
curl -fsS -X "$1" "$MGMT$2" \
-H "Authorization: Bearer $PAT" -H "Content-Type: application/json" -d "$3"
else
curl -fsS -X "$1" "$MGMT$2" -H "Authorization: Bearer $PAT"
fi
}
deadline=$(( $(date +%s) + 120 ))
until curl -fsS "$ISSUER/.well-known/openid-configuration" >/dev/null 2>&1; do
[ "$(date +%s)" -lt "$deadline" ] || fail "Zitadel did not respond at $ISSUER."
sleep 3
done
# The images tagged `latest` require the standalone Login UI v2 container, which
# this stack does not run, so OIDC would redirect to /ui/v2/login and 404.
curl -fsS -X PUT "$ISSUER/v2/features/instance" \
-H "Authorization: Bearer $PAT" -H "Content-Type: application/json" \
-d '{"loginV2":{"required":false}}' >/dev/null 2>&1 \
|| echo " warning: could not disable the required Login UI v2"
name_query() {
printf '{"queries":[{"nameQuery":{"name":"%s","method":"TEXT_QUERY_METHOD_EQUALS"}}]}' "$1"
}
# Project --------------------------------------------------------------------
PROJECT_ID=$(api_call POST "/projects/_search" "$(name_query "$PROJECT_NAME")" \
| jq -r --arg n "$PROJECT_NAME" '.result // [] | map(select(.name == $n)) | first | .id // empty')
if [ -n "$PROJECT_ID" ]; then
echo " project '$PROJECT_NAME' reused (id $PROJECT_ID)"
else
PROJECT_ID=$(api_call POST "/projects" "$(printf '{"name":"%s"}' "$PROJECT_NAME")" | jq -r '.id')
echo " project '$PROJECT_NAME' created (id $PROJECT_ID)"
fi
find_app() {
api_call POST "/projects/$PROJECT_ID/apps/_search" "$(name_query "$1")" \
| jq -r --arg n "$1" '.result // [] | map(select(.name == $n)) | first | .id // empty'
}
# API app: public client used by the Scalar UI to obtain a token -------------
API_APP_ID=$(find_app "$API_APP_NAME")
if [ -n "$API_APP_ID" ]; then
API_CLIENT_ID=$(api_call GET "/projects/$PROJECT_ID/apps/$API_APP_ID" | jq -r '.app.oidcConfig.clientId')
echo " app '$API_APP_NAME' reused (clientId $API_CLIENT_ID)"
else
API_CLIENT_ID=$(api_call POST "/projects/$PROJECT_ID/apps/oidc" "$(jq -n \
--arg name "$API_APP_NAME" --arg api "$API_URL" '{
name: $name,
redirectUris: [($api + "/scalar/"), ($api + "/scalar/oauth2-redirect.html")],
postLogoutRedirectUris: [($api + "/scalar/")],
responseTypes: ["OIDC_RESPONSE_TYPE_CODE"],
grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE"],
appType: "OIDC_APP_TYPE_USER_AGENT",
authMethodType: "OIDC_AUTH_METHOD_TYPE_NONE",
version: "OIDC_VERSION_1_0",
devMode: false,
accessTokenType: "OIDC_TOKEN_TYPE_JWT",
accessTokenRoleAssertion: true,
idTokenRoleAssertion: true,
idTokenUserinfoAssertion: true
}')" | jq -r '.clientId')
echo " app '$API_APP_NAME' created (clientId $API_CLIENT_ID)"
fi
# Web app: confidential client used by the SvelteKit server ------------------
WEB_APP_ID=$(find_app "$WEB_APP_NAME")
WEB_CLIENT_SECRET=$(env_get "$ENV_FILE" WEB_CLIENT_SECRET)
if [ -n "$WEB_APP_ID" ]; then
WEB_CLIENT_ID=$(api_call GET "/projects/$PROJECT_ID/apps/$WEB_APP_ID" | jq -r '.app.oidcConfig.clientId')
if [ -z "$WEB_CLIENT_SECRET" ]; then
# The secret is only returned once, so regenerate it if we no longer have it.
WEB_CLIENT_SECRET=$(api_call POST "/projects/$PROJECT_ID/apps/$WEB_APP_ID/oidc_config/_generate_client_secret" '{}' \
| jq -r '.clientSecret')
echo " app '$WEB_APP_NAME' reused (clientId $WEB_CLIENT_ID, secret regenerated)"
else
echo " app '$WEB_APP_NAME' reused (clientId $WEB_CLIENT_ID)"
fi
else
CREATED=$(api_call POST "/projects/$PROJECT_ID/apps/oidc" "$(jq -n \
--arg name "$WEB_APP_NAME" --arg web "$WEB_URL" '{
name: $name,
redirectUris: [($web + "/auth/callback")],
postLogoutRedirectUris: [($web + "/")],
responseTypes: ["OIDC_RESPONSE_TYPE_CODE"],
grantTypes: ["OIDC_GRANT_TYPE_AUTHORIZATION_CODE", "OIDC_GRANT_TYPE_REFRESH_TOKEN"],
appType: "OIDC_APP_TYPE_WEB",
authMethodType: "OIDC_AUTH_METHOD_TYPE_BASIC",
version: "OIDC_VERSION_1_0",
devMode: false,
accessTokenType: "OIDC_TOKEN_TYPE_JWT",
accessTokenRoleAssertion: true,
idTokenRoleAssertion: true,
idTokenUserinfoAssertion: true
}')")
WEB_CLIENT_ID=$(echo "$CREATED" | jq -r '.clientId')
WEB_CLIENT_SECRET=$(echo "$CREATED" | jq -r '.clientSecret')
echo " app '$WEB_APP_NAME' created (clientId $WEB_CLIENT_ID)"
fi
env_set "$ENV_FILE" AUTH_CLIENT_ID "$API_CLIENT_ID"
env_set "$ENV_FILE" WEB_CLIENT_ID "$WEB_CLIENT_ID"
env_set "$ENV_FILE" WEB_CLIENT_SECRET "$WEB_CLIENT_SECRET"
echo " wired into $(basename "$ENV_FILE")"
docs/development.md
+20 -16
View File
@@ -5,17 +5,15 @@ producción) está en [deploy/README.md](../deploy/README.md).
## Arranque ## Arranque
```powershell ```sh
./deploy/up.ps1 # infraestructura; las apps se ejecutan desde el IDE docker compose -f deploy/compose.yml -f deploy/compose.dev.yml up -d
./deploy/up.ps1 -Apps # todo en Docker (api, worker, gateway, web player)
``` ```
`up.ps1` hace el bootstrap completo: crea `deploy/.env`, genera las claves RS256 Ese único comando levanta la infraestructura y corre los jobs *one-shot* de
de stream-ticket en `deploy/keys/`, levanta el stack, aplica las migraciones y bootstrap: genera las claves RS256 de stream-ticket en `deploy/keys/`, aplica
provisiona Zitadel (proyecto + apps OIDC), dejando los client ids en las migraciones EF Core y provisiona Zitadel (proyecto + apps OIDC), dejando
`deploy/.env`, en los user-secrets de `Musify.Api` y en `web-player/.env`. los client ids en `deploy/.env`. Detalle completo, incluido cómo arrancar las
apps en Docker con `--profile apps`, en [deploy/README.md](../deploy/README.md).
Otros modificadores: `-Tools` (pgAdmin), `-SkipMigrations`, `-Down`, `-Destroy`.
## Aplicaciones ## Aplicaciones
@@ -46,21 +44,25 @@ DB: `musify_db`, usuario `postgres`/`postgres`. Bucket S3: `webapi-storage`
## Migraciones ## Migraciones
No hay migración automática al arrancar; `up.ps1` las aplica en el bootstrap. El servicio `migrate` de `deploy/compose.yml` las aplica en cada `docker
A mano, tras cambios de esquema: compose up -d` (no-op si el esquema ya está al día). A mano, tras cambios de
esquema, contra la base local:
```powershell ```powershell
dotnet ef database update --project backend/Core/Musify.Infrastructure --startup-project backend/Core/Musify.Infrastructure --context Database dotnet ef database update --project backend/Core/Musify.Infrastructure --startup-project backend/Core/Musify.Infrastructure --context Database
``` ```
`Musify.Infrastructure` es a la vez el proyecto de migraciones y el de arranque: `Musify.Infrastructure` es a la vez el proyecto de migraciones y el de arranque:
tiene el `IDesignTimeDbContextFactory` y el paquete `EFCore.Design`. La cadena de tiene el `IDesignTimeDbContextFactory` y el paquete `EFCore.Design`. La cadena
conexión se lee de sus user-secrets (`up.ps1` la deja puesta). de conexión se lee de sus user-secrets, o de la variable de entorno
`Database__ConnectionString` si no hay user-secrets (así es como la usa el
contenedor `migrate`, ver `backend/Musify.Migrator.Dockerfile`).
## Stream-tickets (claves RS256) ## Stream-tickets (claves RS256)
La API firma los stream-tickets con la clave privada y el gateway los valida con La API firma los stream-tickets con la clave privada y el gateway los valida con
la pública. `up.ps1` las genera en `deploy/keys/`; a mano: la pública. El servicio `keys-init` de `deploy/compose.yml` las genera en
`deploy/keys/` si no existen; a mano:
```powershell ```powershell
openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out deploy/keys/stream_private.pem openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048 -out deploy/keys/stream_private.pem
@@ -75,8 +77,10 @@ openssl rsa -in deploy/keys/stream_private.pem -pubout -out deploy/keys/stream_p
Convención: `AppSettings.json` es la plantilla con valores vacíos o neutros; Convención: `AppSettings.json` es la plantilla con valores vacíos o neutros;
`AppSettings.Development.json` tiene los valores reales de dev; en Docker todo `AppSettings.Development.json` tiene los valores reales de dev; en Docker todo
se sobreescribe con variables de entorno (`Seccion__Clave`) desde se sobreescribe con variables de entorno (`Seccion__Clave`) desde
`deploy/compose.yml`. Los valores de `Authentication` los deja el provisioning `deploy/compose.yml`. Para correr `Musify.Api` desde el IDE, los valores de
en user-secrets, no en el repo. `Authentication` van en user-secrets (no en el repo) — cópialos de
`deploy/.env` una vez que `zitadel-init` los haya generado (ver
[deploy/README.md](../deploy/README.md)).
Secciones clave: `Authentication` (Zitadel), `Database`, `MassTransit`, Secciones clave: `Authentication` (Zitadel), `Database`, `MassTransit`,
`InfrastructureStorage` (S3), `ApplicationStorage` (bucket), `Track`, `PlayList`, `InfrastructureStorage` (S3), `ApplicationStorage` (bucket), `Track`, `PlayList`,
web-player/.env.example
+3 -2
View File
@@ -1,5 +1,6 @@
# Local configuration for `npm run dev`. deploy/zitadel/provision.ps1 fills in # Local configuration for `npm run dev`. Copy the WEB_CLIENT_ID /
# the Zitadel values; in Docker the same variables come from deploy/compose.yml. # WEB_CLIENT_SECRET that deploy/compose.yml's zitadel-init service writes to
# deploy/.env; in Docker the same variables come from deploy/compose.yml.
# Zitadel OIDC (server-side auth) # Zitadel OIDC (server-side auth)
ZITADEL_ISSUER=http://host.docker.internal:8080 ZITADEL_ISSUER=http://host.docker.internal:8080
web-player/README.md
+4 -4
View File
@@ -12,10 +12,10 @@ npm install
npm run dev # :5173 npm run dev # :5173
``` ```
Configuration lives in `.env` (template in `.env.example`). Configuration lives in `.env` (template in `.env.example`). Bring the
`deploy/zitadel/provision.ps1` fills in the Zitadel values automatically, so the infrastructure stack up first (see [../deploy/README.md](../deploy/README.md))
usual flow is to bring the stack up first (`./deploy/up.ps1`) and then run the and copy the `WEB_CLIENT_ID` / `WEB_CLIENT_SECRET` it writes to `deploy/.env`
dev server. into this `.env` once, then run the dev server.
```sh ```sh
npm run check # svelte-check — must be clean before closing a change npm run check # svelte-check — must be clean before closing a change