Postgres, Zitadel, SeaweedFS, RabbitMQ and Jaeger no longer run as part
of this stack. They're shared services (used by Musify, Ping and
docker-manager) that now live in a separate Infrastructure repository
and must already be running.
deploy/compose.yml keeps only what's specific to Musify: its own
database connection, its own OIDC app and storage bucket (created
against the shared Zitadel/SeaweedFS via zitadel-init/seaweedfs-init),
and the application containers. A new wait-for-postgres one-shot
replaces the in-file Postgres health check, which no longer works now
that Postgres lives in a different compose project.
compose.prod.yml joins Infrastructure's infra-net network instead of
running its own copies. Docs and READMEs updated to match.
Translate all of docs/ from Spanish to English and rewrite the root
README as a short project overview instead of a technical run guide.
Remove web-player/design_handoff_musify_player, an old design-tool
export that was no longer needed.
Drops the YouTube-backed second catalog source entirely: the
Local/External split on Track/Album collapses into single concrete
entities, Artist/TrackArtist/ExternalTrack/ExternalAlbum and the
YouTube service/download pipeline (yt-dlp, MassTransit consumer,
routing slip) are deleted, and mixes now source discovery only from
local listening history. Includes the RemoveYouTubeIntegration EF
Core migration and matching Worker/deploy/compose cleanup (no more
yt-dlp/python/deno in the Worker image, no thumbnail-cache/YtDlp
config sections).
Second pass on backend/Tests/, covering what Domain+Application (the first
pass) deliberately left out — the parts that talk to real infrastructure:
Musify.Infrastructure.Tests Database/migrations, StorageService,
StreamTicketService, AuditableEntityInterceptor,
PictureService, SingleFlightCache,
PlayListPresetSeeder.
Musify.Api.Tests Real HTTP endpoints via WebApplicationFactory
(routing, auth, ValidationFilter, ErrorOr->HTTP
mapping), plus isolated unit tests for
ErrorOrHttpExtensions, CurrentUser and the
FluentValidation validators.
Musify.StreamingGateway.Tests TicketValidator (RS256) and
TicketValidationMiddleware (path traversal,
prefix boundary, ticket extraction).
Musify.Worker gets no test project: its Program.cs is pure DI/host wiring
(Hangfire, MassTransit, jobs); the logic it runs lives in Musify.Infrastructure,
already covered there.
Per the user's direction, the parts that touch real infrastructure use
Testcontainers instead of fakes: a real Postgres (Testcontainers.PostgreSql)
for Infrastructure.Tests and Api.Tests, and — matching deploy/compose.yml
exactly (chrislusf/seaweedfs:latest, same startup command), not MinIO — a
real SeaweedFS container for StorageService. Api.Tests boots the actual
Musify.Api host against that Postgres (real routing/auth/validation) but
substitutes the services that talk over the network (storage, YouTube, the
message bus, stream-ticket signing) with NSubstitute fakes, and swaps the
JwtBearer scheme for a test-only header-based one (FakeAuthenticationHandler)
so tests don't need a reachable Zitadel. Application.Tests' approach — a real
DbContext (SQLite in-memory) rather than mocking IDatabase, since it exposes
DbSet<T> directly and handlers compose real LINQ queries against it — carries
over into Infrastructure.Tests/Api.Tests via real Postgres instead.
Testing against the real backend (not a fake) found two genuine production
bugs, both fixed here:
- StorageService.RemoveFolderAsync threw NullReferenceException against
SeaweedFS: its DeleteObjects response omits the <Error> list entirely when
nothing failed, which the AWS SDK deserializes as null rather than an empty
list (real AWS S3 always returns one). Now treats null as no errors.
- AuditableEntityInterceptor never bumped UpdatedAt on a normal update (load,
mutate a property, SaveChanges) — only on insert. SavingChanges(Async) fires
before EF's own automatic change detection for that save, so an entity
modified through a plain property setter still reported EntityState.Unchanged
inside the interceptor. Now calls ChangeTracker.DetectChanges() explicitly
first, which is the documented fix for exactly this interceptor pattern.
Also required to make the Api host testable: Program.cs now ends with `public
partial class Program;` (WebApplicationFactory<Program> needs the top-level-
statements Program class to be public — no behavior change), and
Directory.Packages.props gains Microsoft.AspNetCore.Mvc.Testing, Testcontainers
and Testcontainers.PostgreSql.
249 tests total across all five projects, all green: `dotnet test
backend/Musify.slnx` (needs Docker running for Infrastructure.Tests and
Api.Tests).
Two xUnit projects, wired into Musify.slnx under a /Tests/ folder that
mirrors Core/ and Hosts/:
Musify.Domain.Tests Value-object logic (TrackAudio/TrackPictures'
IsProcessed/IsFailed/IsInProgress). Entities are
otherwise anemic EF models with no behavior to
test.
Musify.Application.Tests All ~45 Application handlers and services
(Albums, Tracks, PlayLists, Users, Mixes,
YouTube, plus TrackStreamIssuer,
UploadIntentValidator, YouTubeTrackProvisioner).
IDatabase exposes DbSet<T> directly, so handlers compose real LINQ queries
against it — that can't be expressed with a mock (no IQueryable provider
behind one). TestSupport/TestDatabase.cs is instead a real DbContext on a
private SQLite in-memory connection per test (fresh per test since xUnit
instantiates the test class per test method): fast, no external dependency,
and — unlike EF Core's InMemory provider — SQLite enforces real foreign keys
and supports transactions, which two handlers need. External dependencies
(IStorageService, IEventBus, IYouTubeMusicService, ...) are substituted with
NSubstitute.
TestSupport/TestEntities.cs and TestConfigurations.cs build fully-populated
domain entities and configuration POCOs with sensible defaults for every
required member, so no test method needs to fill in properties it doesn't
care about, and no builder ever hands back null.
Also, needed to make the Application layer itself testable without a real
Postgres or user-secrets:
- DatabaseDesignTimeFactory now falls back to the Database__ConnectionString
env var, and DesignSettings.json is optional — unblocks the `migrate`
one-shot container from deploy/compose.yml, which hit the same gap.
- Directory.Packages.props: xunit, xunit.runner.visualstudio, NSubstitute,
Microsoft.EntityFrameworkCore.Sqlite (pinned to the same 10.0.8 as the rest
of EF Core), Microsoft.NET.Test.Sdk, coverlet.collector. Also pins
SQLitePCLRaw.bundle_e_sqlite3 to 2.1.13 — EFCore.Sqlite 10.0.8 pulls 2.1.11
transitively, which has a known high-severity advisory (GHSA-2m69-gcr7-jv3q).
- backend/.dockerignore excludes Tests/: no application Dockerfile needs it
in its build context.
162 tests, all green: `dotnet test backend/Musify.slnx`.
Removes every wrapper script under deploy/ (up.ps1, up.sh, scripts/*,
zitadel/provision.*) and folds their logic into three one-shot services
in compose.yml, following the pattern already used by seaweedfs-init:
keys-init generates the RS256 stream-ticket key pair
migrate applies the EF Core migrations (backend/Musify.Migrator.Dockerfile)
zitadel-init creates the Musify project + OIDC apps, writes the client
ids back into the active env file
Deploying is now two plain `docker compose` commands, no scripts to run
or chmod:
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml up -d
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml --profile apps up -d --build
(swap compose.dev.yml for compose.prod.yml + --env-file deploy/.env.prod
for production). The first command brings up infra and lets the bootstrap
jobs run to completion; the second builds and starts the apps, which by
then read real OIDC values instead of empty ones — compose interpolates
${VAR} once per invocation, so this can't collapse into a single command
without the apps reading their own config file at container start instead
of via compose env (out of scope here).
Also:
- DatabaseDesignTimeFactory now also reads Database__ConnectionString
from the environment, and DesignSettings.json is optional, so the new
migrate container works without user-secrets or a config file excluded
from the Docker build context by .dockerignore.
- nginx's depends_on no longer lists the apps-profile services: it does
not need them at its own startup (every vhost resolves its upstream at
request time via the embedded Docker resolver), and requiring them made
`compose.prod.yml up -d` (without --profile apps) an invalid project.
- env_get/env_set in zitadel-init strip \r: the mounted env file can have
CRLF line endings on Windows, which otherwise makes an empty-looking
value non-empty to `[ -z ... ]`. .env.example/.env.prod.example are
pinned to LF via .gitattributes for the same reason. Also fixed:
rewriting the bind-mounted env file in place instead of renaming a temp
file over it (rename onto a bind-mounted inode fails with "Resource
busy").
- Removed compose.share.yml and zitadel/.output-share/ — a "temporary"
personal overlay that had leaked a real Zitadel service-account PAT
into git history (committed in 70968b3). That token should be rotated.
- Removed deploy/.tmp/ from .gitignore: nothing mounts or creates it
anymore (it never actually did, either — no compose service referenced
it, only the now-removed up.ps1 did).
Verified end-to-end with Docker: a clean `down -v` + fresh `up -d` +
`--profile apps up -d --build` on the dev stack brings up all four one-shot
jobs successfully and all four apps healthy (api/gateway return 200,
worker starts its consumers with no errors, web-player responds). Also
verified compose.prod.yml starts nginx cleanly with dummy certs and no
apps running, confirming the depends_on fix.
Moves the six backend projects on disk into Core/ and Hosts/
subfolders, matching the solution folders already defined in
Musify.slnx, so the file explorer reflects the same layout as the
solution:
backend/Core/Musify.Domain
backend/Core/Musify.Application
backend/Core/Musify.Infrastructure
backend/Hosts/Musify.Api
backend/Hosts/Musify.Worker
backend/Hosts/Musify.StreamingGateway
Updates every reference to the old paths: ProjectReference entries in
Musify.Api.csproj and Musify.Worker.csproj, the three Dockerfiles,
deploy/scripts/migrate.ps1|sh, deploy/zitadel/provision.ps1,
deploy/up.ps1, deploy/README.md and docs/development.md. Also fixes a
stale Dockerfile comment pointing at a compose file that no longer
exists.
Verified with `dotnet build Musify.slnx` against the new layout.
The root README was a design handoff for a prototype that is not in this repo,
web-player/README.md was the sv template, and deploy/README.md only covered
development. Replace all three and document the production deploy: its
prerequisites, what up.sh does and why nginx sits in front.
The docs under docs/ still used the pre-monorepo project paths, Ardalis.Result
instead of ErrorOr and the player.html test console that no longer exists.
Single docker-compose stack for local development (Postgres, Zitadel,
RabbitMQ, SeaweedFS + bucket init, Jaeger) plus an optional overlay to run
the .NET apps in Docker. up.ps1 orchestrates the full bootstrap: stack up,
stream-ticket keys, EF Core migrations and Zitadel provisioning.
- Zitadel is provisioned automatically (service-account PAT -> project +
OIDC app with JWT access token); the generated ClientId is wired into
Musify.Api user-secrets and .env. No manual console steps.
- ZITADEL_EXTERNAL_DOMAIN=host.docker.internal so token validation works
both with local apps and full-docker.
- App Dockerfiles copy AppSettings*.json to lowercase so config loads on
case-sensitive Linux; Worker image ships ffmpeg.
- Scripts split by responsibility (common/stream-keys/migrate/provision);
apps overlay de-duped via YAML anchors.
- Remove the superseded standalone seaweedfs compose; point docs/development
at the unified stack.
Admin