Keep RabbitMQ data on recreate and proxy to unique container names
RabbitMQ stores its data under rabbit@<hostname>, and the hostname changed with every recreated container, so an image update or an .env change came up with an empty broker. Pin it to `rabbitmq` and document how to carry the definitions over on an existing deployment. Every project shares infra-net, where a generic service name like `api` resolves to every container that has it, round-robin, across projects. The edge now proxies to container names (musify-api, infra-zitadel, ...).
This commit is contained in:
@@ -52,7 +52,30 @@ docker compose --env-file .env.prod -f compose.yml up -d
|
||||
|
||||
A consuming project's own production compose joins `infra-net` as an
|
||||
external network to reach `postgres`, `zitadel`, `seaweedfs`, `rabbitmq` and
|
||||
`jaeger` by container name (see Musify's `deploy/compose.prod.yml`).
|
||||
`jaeger` by name (see Musify's `deploy/compose.prod.yml`).
|
||||
|
||||
Every project on `infra-net` shares one DNS namespace, and Docker answers a
|
||||
service name with *every* container that has it, round-robin. Two projects
|
||||
each with a service called `api` would get each other's traffic. So anything
|
||||
that crosses projects (the edge, or one project calling into this stack)
|
||||
should use the unique `container_name` (`infra-postgres`, `musify-api`, ...),
|
||||
never a generic service name.
|
||||
|
||||
### RabbitMQ and its hostname
|
||||
|
||||
RabbitMQ keeps its data under its node name, `rabbit@<hostname>`, so the
|
||||
container has a fixed `hostname: rabbitmq`. Without it every recreated
|
||||
container (an image update, an `.env` change) came up with a new hostname
|
||||
and an empty broker. A deployment that ran before this was fixed has its
|
||||
data under the old random name; carry the definitions (users, vhosts,
|
||||
permissions, queues, exchanges) over when applying the change, ideally with
|
||||
the queues drained, since queued messages aren't part of the export:
|
||||
|
||||
```bash
|
||||
docker exec infra-rabbitmq rabbitmqctl export_definitions /var/lib/rabbitmq/defs.json
|
||||
docker compose --env-file .env.prod -f compose.yml up -d rabbitmq
|
||||
docker exec infra-rabbitmq rabbitmqctl import_definitions /var/lib/rabbitmq/defs.json
|
||||
```
|
||||
|
||||
### Ports (development)
|
||||
|
||||
@@ -143,7 +166,8 @@ docker compose -f edge/compose.yml up -d
|
||||
- **Vhosts** live in `edge/conf.d/` (one file per project; `00-common.conf`
|
||||
has the shared resolver). Every upstream is a `set $upstream ...`, resolved
|
||||
per request, so a project's stack being down never stops the edge from
|
||||
starting. Shared bits are in `edge/snippets/`. Check a change with
|
||||
starting. Upstreams use the target's `container_name` (see "Running it"
|
||||
above for why). Shared bits are in `edge/snippets/`. Check a change with
|
||||
`docker exec edge-nginx nginx -t`, then `docker exec edge-nginx nginx -s reload`.
|
||||
- **Certificates** are state, not versioned (`edge/certbot/`). To issue one
|
||||
for new hostnames (their DNS has to already point at the host):
|
||||
|
||||
@@ -192,6 +192,10 @@ services:
|
||||
rabbitmq:
|
||||
image: rabbitmq:4-management-alpine
|
||||
container_name: infra-rabbitmq
|
||||
# The node is named rabbit@<hostname> and its data lives under that name.
|
||||
# Without a fixed hostname every recreated container gets a new one and
|
||||
# starts on an empty data directory (users, vhosts, queues all gone).
|
||||
hostname: rabbitmq
|
||||
environment:
|
||||
RABBITMQ_DEFAULT_USER: ${RABBITMQ_DEFAULT_USER}
|
||||
RABBITMQ_DEFAULT_PASS: ${RABBITMQ_DEFAULT_PASS}
|
||||
|
||||
@@ -27,7 +27,7 @@ server {
|
||||
include /etc/nginx/snippets/tls.conf;
|
||||
|
||||
location / {
|
||||
set $upstream http://web-player:3000;
|
||||
set $upstream http://musify-web-player:3000;
|
||||
proxy_pass $upstream;
|
||||
include /etc/nginx/snippets/proxy.conf;
|
||||
}
|
||||
@@ -44,7 +44,7 @@ server {
|
||||
include /etc/nginx/snippets/tls.conf;
|
||||
|
||||
location / {
|
||||
set $upstream http://api:5111;
|
||||
set $upstream http://musify-api:5111;
|
||||
proxy_pass $upstream;
|
||||
include /etc/nginx/snippets/proxy.conf;
|
||||
}
|
||||
@@ -62,7 +62,7 @@ server {
|
||||
include /etc/nginx/snippets/tls.conf;
|
||||
|
||||
location / {
|
||||
set $upstream http://gateway:8081;
|
||||
set $upstream http://musify-gateway:8081;
|
||||
proxy_pass $upstream;
|
||||
include /etc/nginx/snippets/proxy.conf;
|
||||
proxy_buffering off;
|
||||
@@ -88,7 +88,7 @@ server {
|
||||
}
|
||||
|
||||
location / {
|
||||
set $upstream http://zitadel:8080;
|
||||
set $upstream http://infra-zitadel:8080;
|
||||
proxy_pass $upstream;
|
||||
include /etc/nginx/snippets/proxy.conf;
|
||||
}
|
||||
@@ -107,7 +107,7 @@ server {
|
||||
include /etc/nginx/snippets/tls.conf;
|
||||
|
||||
location / {
|
||||
set $upstream http://seaweedfs:8333;
|
||||
set $upstream http://infra-seaweedfs:8333;
|
||||
proxy_pass $upstream;
|
||||
include /etc/nginx/snippets/proxy.conf;
|
||||
proxy_request_buffering off;
|
||||
|
||||
Reference in New Issue
Block a user