Use the shared edge nginx instead of Musify's own
CI/CD / BackEnd (push) Successful in 1m27s
CI/CD / WebPlayer (push) Successful in 39s
CI/CD / Deploy (push) Skipped
CI/CD / BackEnd (pull_request) Successful in 1m16s
CI/CD / WebPlayer (pull_request) Successful in 39s
CI/CD / Deploy (pull_request) Skipped
CI/CD / BackEnd (push) Successful in 1m27s
CI/CD / WebPlayer (push) Successful in 39s
CI/CD / Deploy (push) Skipped
CI/CD / BackEnd (pull_request) Successful in 1m16s
CI/CD / WebPlayer (pull_request) Successful in 39s
CI/CD / Deploy (pull_request) Skipped
Remove deploy/nginx and the nginx service from compose.prod.yml: the shared edge in Infrastructure owns ports 80/443 and serves every domain. Make the deploy job fail when zitadel-init or seaweedfs-init fail, and document the runner and edge setup.
This commit is contained in:
+43
-75
@@ -13,7 +13,7 @@ wherever you actually cloned it.
|
||||
|---|---|
|
||||
| `compose.yml` | Musify's own services, including the one-shot bootstrap jobs. No published ports, no domains, no environment-specific values. |
|
||||
| `compose.dev.yml` | Development overlay: publishes Musify's ports to the host. |
|
||||
| `compose.prod.yml` | Production overlay: adds the nginx edge and joins Infrastructure's `infra-net` network. |
|
||||
| `compose.prod.yml` | Production overlay: joins Infrastructure's `infra-net` network. |
|
||||
| `.env` / `.env.prod` | The values compose interpolates. Not versioned; templates in `.env.example` / `.env.prod.example`. |
|
||||
|
||||
The four applications (`api`, `worker`, `gateway`, `web-player`) sit behind
|
||||
@@ -113,31 +113,26 @@ SeaweedFS is done from the `Infrastructure` repo instead.
|
||||
|
||||
## Production
|
||||
|
||||
On a server with a domain:
|
||||
Musify doesn't run its own nginx: the shared edge in Infrastructure
|
||||
(`edge/`) owns host ports 80/443, terminates TLS for every project's domains
|
||||
and proxies to Musify's containers over `infra-net`. On a server with
|
||||
domains:
|
||||
|
||||
1. Bring up the `Infrastructure` repo first, with its own `.env.prod` filled
|
||||
in. It creates the `infra-net` network this stack joins.
|
||||
2. From this repository's root:
|
||||
2. Bring up its `edge/` (see Infrastructure's README) and add Musify's
|
||||
vhosts and certificate there (`edge/conf.d/musify.conf`). The five
|
||||
public hostnames (web, api, stream, auth, s3) are independent: they
|
||||
don't need to share a root domain.
|
||||
3. From this repository's root:
|
||||
|
||||
```sh
|
||||
cp deploy/.env.prod.example deploy/.env.prod
|
||||
```
|
||||
|
||||
Fill it in:
|
||||
|
||||
1. Replace every `*.example.com` with your actual domains and every
|
||||
`CHANGE_ME` with a random value (`openssl rand -hex 24`). Each of
|
||||
`WEB_DOMAIN`/`API_DOMAIN`/`STREAM_DOMAIN`/`AUTH_DOMAIN`/`S3_DOMAIN` is
|
||||
independent — they don't need to share a root domain (e.g.
|
||||
`musify.example.com` + `musifyapi.example.com`, or sibling subdomains of
|
||||
a domain whose DNS provider doesn't allow multi-level subdomains).
|
||||
`chmod 600 deploy/.env.prod`.
|
||||
2. Drop **`deploy/nginx/certs/origin.pem` and `origin.key`**, the origin
|
||||
certificate nginx serves. The stack assumes Cloudflare in *Full (strict)*
|
||||
mode in front of it, so a Cloudflare Origin Certificate is enough; any
|
||||
certificate valid for the five hostnames works.
|
||||
3. Point the five `*_DOMAIN` values at the server, proxied and
|
||||
TLS-terminated the same way.
|
||||
Replace every `*.example.com` with your domains (they must match the
|
||||
edge's vhosts) and every `CHANGE_ME` with a random value
|
||||
(`openssl rand -hex 24`). `chmod 600 deploy/.env.prod`.
|
||||
|
||||
Then:
|
||||
|
||||
@@ -146,68 +141,45 @@ docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compo
|
||||
docker compose --env-file deploy/.env.prod -f deploy/compose.yml -f deploy/compose.prod.yml --profile apps up -d --build
|
||||
```
|
||||
|
||||
The first command starts Musify's bootstrap jobs and the nginx edge; the
|
||||
second builds and starts the applications, already wired to the OIDC client
|
||||
ids the first one produced. Both are idempotent, so running them again
|
||||
deploys a new version (`... --profile apps up -d --build` alone is enough for
|
||||
an update once the stack already exists).
|
||||
The first command starts Musify's bootstrap jobs (`zitadel-init` and
|
||||
`seaweedfs-init` reach Zitadel/SeaweedFS through the public `auth.`/`s3.`
|
||||
URLs, so the edge has to be serving them by then); the second builds and
|
||||
starts the applications, already wired to the OIDC client ids the first one
|
||||
produced. Both are idempotent, so running them again deploys a new version
|
||||
(`... --profile apps up -d --build` alone is enough for an update once the
|
||||
stack already exists).
|
||||
|
||||
Only nginx publishes ports (80/443). Every other Musify service, and all of
|
||||
Infrastructure's, is reachable only on the internal `infra-net` network.
|
||||
Nothing in this stack publishes a port; everything is reachable only on the
|
||||
internal `infra-net` network, through the edge.
|
||||
|
||||
### Continuous deployment
|
||||
|
||||
`.gitea/workflows/ci-cd.yml` runs the backend and web-player checks on every
|
||||
push/PR, and on a successful push to `master` deploys automatically. The
|
||||
deploy job runs on `gitea-runner`, the existing self-hosted Gitea Actions
|
||||
runner on the production host itself (`gitea.ikerdemo.cat`, in
|
||||
`~/gitea/docker-compose.yml`), which also carries a `production:host`
|
||||
label registered directly in `~/gitea/runner-data/.runner` (act_runner
|
||||
reads `labels` from that file on every start, not from
|
||||
`GITEA_RUNNER_LABELS`, once a runner is already registered). Running in
|
||||
native mode means the label's steps execute inside the runner container
|
||||
itself rather than spinning up a fresh one — that container has
|
||||
`/var/run/docker.sock` mounted, so `docker`/`docker compose` there still
|
||||
control the host's real Docker daemon (sibling containers, "Docker outside
|
||||
of Docker"). Its compose service also bind-mounts `/home/ubuntu/deploys`
|
||||
into itself at the *same* path, which matters because compose's own
|
||||
relative bind mounts (`./keys`, `./nginx/...`) are resolved by the host
|
||||
daemon against that literal path — mounting it anywhere else would silently
|
||||
bind the wrong (empty) directory.
|
||||
deploy job runs on `gitea-runner`, the self-hosted Gitea Actions runner on
|
||||
the production host (`~/gitea/docker-compose.yml`), configured by
|
||||
`~/gitea/runner-data/config.yaml` (`CONFIG_FILE`): job containers are pinned
|
||||
to the `gitea_gitea` network so `actions/checkout` can resolve `gitea`,
|
||||
`ubuntu-latest` is `node:22` (JS actions need Node inside the job
|
||||
container), and a `production:host` label runs steps natively.
|
||||
|
||||
Native mode means the label's steps execute inside the runner container
|
||||
itself rather than a fresh one. That container has `/var/run/docker.sock`
|
||||
mounted, so `docker`/`docker compose` there still control the host's real
|
||||
Docker daemon (sibling containers, "Docker outside of Docker"). Its compose
|
||||
service also bind-mounts `/home/ubuntu/deploys` into itself at the *same*
|
||||
path, which matters because compose's own relative bind mounts (`./keys`)
|
||||
are resolved by the host daemon against that literal path; mounting it
|
||||
anywhere else would silently bind the wrong (empty) directory.
|
||||
|
||||
`DEPLOY_PATH` (hardcoded in the workflow as `/home/ubuntu/deploys/musify`)
|
||||
is a persistent clone on that host, reused instead of a fresh checkout each
|
||||
run since `deploy/.env.prod` is untracked and only lives there.
|
||||
`/home/ubuntu/deploys/SharedServices` is the sibling clone of the
|
||||
`Infrastructure`/`SharedServices` repo, already running (`infra-net`
|
||||
created); its `.env.prod` shares the Postgres/RabbitMQ/S3 credentials with
|
||||
Musify's own `deploy/.env.prod` by design (see that repo's README).
|
||||
|
||||
**Pending DNS on the registered domains:** `deploy/.env.prod`'s five
|
||||
`*_DOMAIN` values are set (`musify.ikerdemo.cat`, `musifyapi.ikerdemo.cat`,
|
||||
`musifystream.ikerdemo.cat`, `auth.ikerdemo.cat`, `s3.ikerdemo.cat` —
|
||||
siblings, not `api.musify.ikerdemo.cat`, because the DNS provider here
|
||||
doesn't allow multi-level subdomains), but no DNS record points any of them
|
||||
at the server yet, so:
|
||||
|
||||
- `zitadel-init` and `seaweedfs-init` (they verify themselves through the
|
||||
public `auth.`/`s3.` URLs) fail every deploy until those two resolve and
|
||||
are actually reachable — the workflow tolerates this on purpose, see its
|
||||
comments. Until then, `api`/`worker` build and start but can't do
|
||||
anything OIDC- or storage-dependent (`AUTH_CLIENT_ID`/`WEB_CLIENT_ID`/
|
||||
`WEB_CLIENT_SECRET` in `.env.prod` stay blank); `gateway`/`web-player`
|
||||
are unaffected.
|
||||
- The `nginx` service from `compose.prod.yml` is intentionally **not**
|
||||
brought up by the deploy job: it would try to publish host ports 80/443,
|
||||
which `gitea-nginx` already owns for `gitea.ikerdemo.cat` (certbot). Once
|
||||
the five DNS records above exist, merge Musify's vhosts
|
||||
(`deploy/nginx/templates`) into `~/gitea/nginx/conf.d` instead — same
|
||||
nginx, additional `server_name` blocks proxying to Musify's containers
|
||||
over `infra-net` — rather than running a second nginx that would conflict
|
||||
on those ports. That also needs a certificate valid for the 5 hostnames
|
||||
(the existing setup uses certbot/Let's Encrypt, not the Cloudflare Origin
|
||||
cert `compose.prod.yml`'s nginx assumes — reconcile whichever way when
|
||||
that's set up).
|
||||
`Infrastructure` repo (shared services + `edge/`), already running; its
|
||||
`.env.prod` shares the Postgres/RabbitMQ/S3 credentials with Musify's own
|
||||
`deploy/.env.prod` by design (see that repo's README). The edge is deployed
|
||||
from that repo, not from this pipeline.
|
||||
|
||||
---
|
||||
|
||||
@@ -218,13 +190,9 @@ deploy/
|
||||
├─ compose.yml # Musify's own services: wait-for-postgres, keys-init,
|
||||
│ # migrate, seaweedfs-init, zitadel-init, api/worker/gateway/web-player
|
||||
├─ compose.dev.yml # dev overlay: host ports
|
||||
├─ compose.prod.yml # prod overlay: nginx edge, joins infra-net
|
||||
├─ compose.prod.yml # prod overlay: joins infra-net
|
||||
├─ .env.example / .env.prod.example
|
||||
├─ keys/ # RS256 stream-ticket key pair (generated)
|
||||
└─ nginx/
|
||||
├─ templates/default.conf.template # vhosts, rendered with the domain by envsubst
|
||||
├─ snippets/proxy.conf # proxy headers shared by every vhost
|
||||
└─ certs/ # origin.pem + origin.key (not versioned)
|
||||
└─ keys/ # RS256 stream-ticket key pair (generated)
|
||||
```
|
||||
|
||||
The application images are built from `backend/Musify.*.Dockerfile` (build
|
||||
|
||||
Reference in New Issue
Block a user