Removes the last YouTube-specific routes (/youtube/*) and merges them
into the general track/album/playlist endpoints, keyed by a `source`
string validated against TrackSource. This mirrors the earlier track
search merge for albums and playlist-track additions:
- GetAlbums now merges local albums with live YouTube Music search
results, paginated via a youtubeContinuationToken, deduped against
already-materialized ExternalAlbum rows.
- New GET /albums/external/{source}/{externalId} and
GET /tracks/external/{source}/{externalId}/stream replace the
YouTube-only equivalents.
- POST /playlists/{id}/tracks replaces the separate
.../youtube-tracks endpoint, accepting an optional Source/ExternalId
pair alongside TrackId.
- AlbumSearchItemResponse/AlbumsSearchResponse follow the same
discriminated-union + trailing-HasNextPage-field pattern already
used for track search, so HasNextPage stays a required field in the
generated OpenAPI schema.
- Frontend routes renamed accordingly (/albums/youtube/[albumId] ->
/albums/external/[source]/[externalId], etc.) with a single
/api/thumbnail proxy replacing /api/youtube/thumbnail.
Musify.Api.Endpoints.YouTubeEndpoints and the standalone
SearchYouTubeAlbums query are removed as a result.
Each .csproj is renamed to just the project's own name within its folder
(Core/Musify.Application/Musify.Application.csproj -> Application.csproj,
and so on for every Core/Hosts/Tests project) — the Musify.* folder name
already carries that information, so the file name was redundant. Explicit
<AssemblyName>/<RootNamespace> keep the compiled assembly and C# namespaces
unchanged, so this is a build-file-only rename, not a namespace migration.
Dockerfiles, Musify.slnx, and every ProjectReference are updated to match.
New Playback:AllowAnonymousListening / Playback:AnonymousFragmentSeconds
settings control whether a request without a session can stream at all,
and if so, whether it's limited to a fragment instead of the full track:
- GET /tracks/{id}/stream and GET /youtube/tracks/{videoId}/stream no
longer require authorization outright; the handlers gate on the config
instead, so both keep working exactly as before when it's off (the
default).
- The fragment cap is enforced by the Streaming Gateway, not the client:
the stream ticket now carries an optional maxBytes claim (computed from
AnonymousFragmentSeconds and the transcoder's known bitrate), and
TicketValidationMiddleware clamps the Range it forwards to SeaweedFS to
stay under it — synthesizing one if the request didn't send one, and
answering 416 outright if the requested range starts past the cap.
- A YouTube track that hasn't been provisioned server-side yet streams
straight from YouTube's CDN, bypassing the Gateway entirely — there's no
way to cap that, so an anonymous request for one is refused (409) instead
of silently handing out an uncapped stream.
- ListeningHistory has a non-nullable UserId, so anonymous plays simply
don't get recorded rather than trying to force a row through the FK.
Also fixes a real, pre-existing bug this surfaced: the AppSettings.Development.json
of the Api, Worker, and StreamingGateway all pointed StreamTicket/Workers
paths at "../../deploy/..." — one directory short of deploy/'s actual
location at the repo root relative to each project's own folder. Docker
never hit this (it mounts an absolute /keys path), so it had apparently
never been exercised via `dotnet run` before.
GET /tracks now merges the local catalog page with a live YouTube Music
search (deduplicated against tracks already provisioned locally) whenever a
name filter is given, appending the YouTube hits after the local matches and
carrying a continuation token for further pages. Each item is tagged with its
source via the new TrackSearchItemResponse/TracksSearchResponse shapes,
wrapping the existing TrackApplicationResponse or a raw YouTubeSongResult so
neither type had to change.
GET /youtube/search is removed along with its now-unused handler; the
web-player's explore page, its /api/tracks proxy, and the YouTube filler
helper are updated to call the combined endpoint instead of hitting the two
separately.
- Remove all // and /// comments from the backend test suite.
- Replace hardcoded normalized-form string literals (e.g. NormalizedTitle/
NormalizedName duplicating a title/name in upper case) with a variable
plus .ToUpperInvariant(), so the two values can't drift apart.
Converts all 73 test source files from file-scoped (namespace X;) to
block-scoped (namespace X { ... }) declarations, re-indenting their
contents accordingly. Matches the existing convention already used in
several production files (e.g. Musify.Infrastructure.Configuration,
Musify.Application.Contracts).
Mechanical change only, done with a script, verified with a full
rebuild (0 warnings, 0 errors) and test run (257/257 passing).
- YouTubeConfiguration.Enabled (default true, Infrastructure): single
switch for the whole source
- DisabledYouTubeMusicService: null-object IYouTubeMusicService that
fails every call with Forbidden, registered instead of the real one
when disabled (AddYouTubeMusicService) so no client is ever built
- AppSettings.json (Api + Worker): document YouTube.Enabled explicitly;
overridable via YouTube__Enabled env var like every other section
- CurrentUser.cs: small unrelated cleanup, collapse the IsAuthenticated
computation to a single null-coalescing expression
Search, albums, stream resolution and adding a new YouTube track all
surface as 403 Forbidden through the existing ErrorOr -> HTTP mapping.
Daily mix generation already treated a YouTube search failure as "no
YouTube candidates" and keeps working with local-only candidates —
added a test locking that in. Already-downloaded YouTube tracks are
unaffected since playback never goes through IYouTubeMusicService.
257/257 tests passing (7 new), 0 build warnings.
- Directory.Packages.props: xunit 2.9.3 -> xunit.v3 4.0.0, bump
xunit.runner.visualstudio to the matching 4.0.0
- All 5 test csproj: xunit -> xunit.v3 package reference, add
<OutputType>Exe</OutputType> (v3 test projects are standalone
executables)
- backend/global.json: opt dotnet test into the new MTP-native mode
(.NET 10 SDK removed the old VSTest bridge for MTP-based test
projects like xunit v3)
- IAsyncLifetime implementations (HandlerTestBase, ApiTestFixture,
InfrastructureTestFixture, StorageServiceTests): v3's IAsyncLifetime
now IS IAsyncDisposable, so InitializeAsync/DisposeAsync return
ValueTask instead of Task. ApiTestFixture specifically now overrides
WebApplicationFactory.DisposeAsync instead of implementing
IAsyncLifetime.DisposeAsync explicitly, since v3 collapses both to
the same single DisposeAsync slot
- Replaced CancellationToken.None with TestContext.Current.CancellationToken
across the suite (a v3-only API) and added it to every call that
was silently omitting a CancellationToken argument, per the new
xUnit1051 analyzer warnings that came with the v3 package
Verified: dotnet build (0 warnings, 0 errors) and dotnet test, 250/250
passing under the new MTP runner.
- StorageServiceTests: replace out-param + manual dispose with IAsyncLifetime
- PictureServiceTests: Assert.True bound checks -> Assert.InRange
- CreateAlbumCommandHandlerTests: assert the specific NotFound error type
instead of just IsError, matching what the test name promises
- CreateAlbumRequestValidatorTests: replace a boundary Theory that computed
its expected result inline with explicit Facts against the real
AlbumReleaseYear constants
- Remove unused using directives across a handful of production and test
files found during the review
Full suite verified green: 250/250 tests passing, 0 build warnings.
Second pass on backend/Tests/, covering what Domain+Application (the first
pass) deliberately left out — the parts that talk to real infrastructure:
Musify.Infrastructure.Tests Database/migrations, StorageService,
StreamTicketService, AuditableEntityInterceptor,
PictureService, SingleFlightCache,
PlayListPresetSeeder.
Musify.Api.Tests Real HTTP endpoints via WebApplicationFactory
(routing, auth, ValidationFilter, ErrorOr->HTTP
mapping), plus isolated unit tests for
ErrorOrHttpExtensions, CurrentUser and the
FluentValidation validators.
Musify.StreamingGateway.Tests TicketValidator (RS256) and
TicketValidationMiddleware (path traversal,
prefix boundary, ticket extraction).
Musify.Worker gets no test project: its Program.cs is pure DI/host wiring
(Hangfire, MassTransit, jobs); the logic it runs lives in Musify.Infrastructure,
already covered there.
Per the user's direction, the parts that touch real infrastructure use
Testcontainers instead of fakes: a real Postgres (Testcontainers.PostgreSql)
for Infrastructure.Tests and Api.Tests, and — matching deploy/compose.yml
exactly (chrislusf/seaweedfs:latest, same startup command), not MinIO — a
real SeaweedFS container for StorageService. Api.Tests boots the actual
Musify.Api host against that Postgres (real routing/auth/validation) but
substitutes the services that talk over the network (storage, YouTube, the
message bus, stream-ticket signing) with NSubstitute fakes, and swaps the
JwtBearer scheme for a test-only header-based one (FakeAuthenticationHandler)
so tests don't need a reachable Zitadel. Application.Tests' approach — a real
DbContext (SQLite in-memory) rather than mocking IDatabase, since it exposes
DbSet<T> directly and handlers compose real LINQ queries against it — carries
over into Infrastructure.Tests/Api.Tests via real Postgres instead.
Testing against the real backend (not a fake) found two genuine production
bugs, both fixed here:
- StorageService.RemoveFolderAsync threw NullReferenceException against
SeaweedFS: its DeleteObjects response omits the <Error> list entirely when
nothing failed, which the AWS SDK deserializes as null rather than an empty
list (real AWS S3 always returns one). Now treats null as no errors.
- AuditableEntityInterceptor never bumped UpdatedAt on a normal update (load,
mutate a property, SaveChanges) — only on insert. SavingChanges(Async) fires
before EF's own automatic change detection for that save, so an entity
modified through a plain property setter still reported EntityState.Unchanged
inside the interceptor. Now calls ChangeTracker.DetectChanges() explicitly
first, which is the documented fix for exactly this interceptor pattern.
Also required to make the Api host testable: Program.cs now ends with `public
partial class Program;` (WebApplicationFactory<Program> needs the top-level-
statements Program class to be public — no behavior change), and
Directory.Packages.props gains Microsoft.AspNetCore.Mvc.Testing, Testcontainers
and Testcontainers.PostgreSql.
249 tests total across all five projects, all green: `dotnet test
backend/Musify.slnx` (needs Docker running for Infrastructure.Tests and
Api.Tests).
Fixes: ResolvePackageAssets fails with "Unable to find fallback package
folder 'D:\Microsoft Visual Studio\Shared\NuGetPackages'".
That folder comes from a machine-wide Visual Studio installer config
(C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.FallbackLocation.config),
not from anything in this repo — it points at a fallback package cache on
whatever drive VS was originally installed on, and NuGet fails outright if
that path doesn't exist on the current machine. nuget.config already had
<clear/> under packageSources for the same reason (pin restore to nuget.org
only); it just didn't do the equivalent for fallbackPackageFolders. Restore
and build now depend only on this file, not on machine-specific VS install
history.
Two xUnit projects, wired into Musify.slnx under a /Tests/ folder that
mirrors Core/ and Hosts/:
Musify.Domain.Tests Value-object logic (TrackAudio/TrackPictures'
IsProcessed/IsFailed/IsInProgress). Entities are
otherwise anemic EF models with no behavior to
test.
Musify.Application.Tests All ~45 Application handlers and services
(Albums, Tracks, PlayLists, Users, Mixes,
YouTube, plus TrackStreamIssuer,
UploadIntentValidator, YouTubeTrackProvisioner).
IDatabase exposes DbSet<T> directly, so handlers compose real LINQ queries
against it — that can't be expressed with a mock (no IQueryable provider
behind one). TestSupport/TestDatabase.cs is instead a real DbContext on a
private SQLite in-memory connection per test (fresh per test since xUnit
instantiates the test class per test method): fast, no external dependency,
and — unlike EF Core's InMemory provider — SQLite enforces real foreign keys
and supports transactions, which two handlers need. External dependencies
(IStorageService, IEventBus, IYouTubeMusicService, ...) are substituted with
NSubstitute.
TestSupport/TestEntities.cs and TestConfigurations.cs build fully-populated
domain entities and configuration POCOs with sensible defaults for every
required member, so no test method needs to fill in properties it doesn't
care about, and no builder ever hands back null.
Also, needed to make the Application layer itself testable without a real
Postgres or user-secrets:
- DatabaseDesignTimeFactory now falls back to the Database__ConnectionString
env var, and DesignSettings.json is optional — unblocks the `migrate`
one-shot container from deploy/compose.yml, which hit the same gap.
- Directory.Packages.props: xunit, xunit.runner.visualstudio, NSubstitute,
Microsoft.EntityFrameworkCore.Sqlite (pinned to the same 10.0.8 as the rest
of EF Core), Microsoft.NET.Test.Sdk, coverlet.collector. Also pins
SQLitePCLRaw.bundle_e_sqlite3 to 2.1.13 — EFCore.Sqlite 10.0.8 pulls 2.1.11
transitively, which has a known high-severity advisory (GHSA-2m69-gcr7-jv3q).
- backend/.dockerignore excludes Tests/: no application Dockerfile needs it
in its build context.
162 tests, all green: `dotnet test backend/Musify.slnx`.
Removes every wrapper script under deploy/ (up.ps1, up.sh, scripts/*,
zitadel/provision.*) and folds their logic into three one-shot services
in compose.yml, following the pattern already used by seaweedfs-init:
keys-init generates the RS256 stream-ticket key pair
migrate applies the EF Core migrations (backend/Musify.Migrator.Dockerfile)
zitadel-init creates the Musify project + OIDC apps, writes the client
ids back into the active env file
Deploying is now two plain `docker compose` commands, no scripts to run
or chmod:
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml up -d
docker compose -f deploy/compose.yml -f deploy/compose.dev.yml --profile apps up -d --build
(swap compose.dev.yml for compose.prod.yml + --env-file deploy/.env.prod
for production). The first command brings up infra and lets the bootstrap
jobs run to completion; the second builds and starts the apps, which by
then read real OIDC values instead of empty ones — compose interpolates
${VAR} once per invocation, so this can't collapse into a single command
without the apps reading their own config file at container start instead
of via compose env (out of scope here).
Also:
- DatabaseDesignTimeFactory now also reads Database__ConnectionString
from the environment, and DesignSettings.json is optional, so the new
migrate container works without user-secrets or a config file excluded
from the Docker build context by .dockerignore.
- nginx's depends_on no longer lists the apps-profile services: it does
not need them at its own startup (every vhost resolves its upstream at
request time via the embedded Docker resolver), and requiring them made
`compose.prod.yml up -d` (without --profile apps) an invalid project.
- env_get/env_set in zitadel-init strip \r: the mounted env file can have
CRLF line endings on Windows, which otherwise makes an empty-looking
value non-empty to `[ -z ... ]`. .env.example/.env.prod.example are
pinned to LF via .gitattributes for the same reason. Also fixed:
rewriting the bind-mounted env file in place instead of renaming a temp
file over it (rename onto a bind-mounted inode fails with "Resource
busy").
- Removed compose.share.yml and zitadel/.output-share/ — a "temporary"
personal overlay that had leaked a real Zitadel service-account PAT
into git history (committed in 70968b3). That token should be rotated.
- Removed deploy/.tmp/ from .gitignore: nothing mounts or creates it
anymore (it never actually did, either — no compose service referenced
it, only the now-removed up.ps1 did).
Verified end-to-end with Docker: a clean `down -v` + fresh `up -d` +
`--profile apps up -d --build` on the dev stack brings up all four one-shot
jobs successfully and all four apps healthy (api/gateway return 200,
worker starts its consumers with no errors, web-player responds). Also
verified compose.prod.yml starts nginx cleanly with dummy certs and no
apps running, confirming the depends_on fix.
Moves the six backend projects on disk into Core/ and Hosts/
subfolders, matching the solution folders already defined in
Musify.slnx, so the file explorer reflects the same layout as the
solution:
backend/Core/Musify.Domain
backend/Core/Musify.Application
backend/Core/Musify.Infrastructure
backend/Hosts/Musify.Api
backend/Hosts/Musify.Worker
backend/Hosts/Musify.StreamingGateway
Updates every reference to the old paths: ProjectReference entries in
Musify.Api.csproj and Musify.Worker.csproj, the three Dockerfiles,
deploy/scripts/migrate.ps1|sh, deploy/zitadel/provision.ps1,
deploy/up.ps1, deploy/README.md and docs/development.md. Also fixes a
stale Dockerfile comment pointing at a compose file that no longer
exists.
Verified with `dotnet build Musify.slnx` against the new layout.
The playlist cards were showing "No description was pr…" because the API
stored that English sentence whenever a playlist was created without a
description. Descriptions are optional now and the old rows are cleared.
Section headings drop to 18/20px so they stop competing with the content,
and the space moved from inside the sections to between them.
The three media cards repeated the same anchor, art class and label
block, so they share a MediaCard that also carries the hover lift the
track tiles already had.
Recently played keeps its shape. Mixes and playlists now share one row,
half and half on large screens with smaller cards, capped at six each
with a link to the full list. Albums are the ones the user listened to
most recently, which is a new query joining the listening history with
the album tracks. The endless track wall is now "Descubrir".
A mix should read as one list, so the detail page drops the per-source
counters and the YouTube tag on each row, and the generated subtitles no
longer name Musify or YouTube Music.
ChipGroup is orphaned since the explore rewrite dropped the filter chips.
Mix.GeneratedOn was written on every generation and never read, and it
duplicated CreatedAt, which IAuditable already sets at the same moment.
Moves the remaining uncommitted work: the home gets quick mixes and a
"Tus álbumes" shelf, saving a YouTube album now goes through
addYoutubeAlbumToPlaylistAction instead of the SaveYouTubeAlbum command,
and UserAlbum drops its unused ExternalId.
The mixes section keeps its current shape (no refresh action, no badge,
hidden when empty) rather than the older version the worktree carried.
Mixes are their own concept, not playlists: they live in their own
Mixes/MixItems tables, cannot be edited and never show up among the
user's playlists. Each item is either a Musify track or a YouTube Music
reference, so YouTube songs stream directly instead of being downloaded.
A daily Hangfire job seeds the mixes from the listening history: one
discovery mix, one daily mix and one per top artist.
The per-user YouTube filler query was already in the working tree and
rides along here because it shares files with this change.
POST /albums/youtube/{albumId} creates the album in your library and links
every song in it, provisioning each one as an external track without queueing
any download. The external id is kept on the album so saving it twice returns
a conflict instead of a duplicate, and albums can now hold external tracks,
not just the ones you uploaded.
Two read-only endpoints: the album search and the album detail with its
tracklist. Both go through the same single-flight cache as the song search,
and the detail one especially benefits since it costs two upstream calls, the
browse id lookup and the album itself, for metadata that never changes.
Paginator handling is now shared between the song and album searches.
Eight routes under /albums covering the listing, the detail, the tracklist and
the owner-only writes, wired to the same validation filter and ErrorOr mapping
the playlists use.
Create, update, delete and list albums, plus adding and removing their tracks,
mirroring the playlist handlers. Only tracks the user uploaded can go into an
album, and removing one renumbers the rest so the tracklist stays 1..n.
Album is abstract with UserAlbum as its first table-per-type child, the same
split Track already uses, so the YouTube albums can join later without moving
anything. Tracks are linked through AlbumHasTrack rather than a column on the
track itself: an external track is shared between users and the same song
shows up in more than one album.
Titles were capped at 50 characters and the YouTube provisioner silently cut
them there, which mangles anything with a feature or a remix suffix. Albums
are about to make that far more visible, so the column now holds 200.
The ytimg variants are 16:9, so picking one by width handed a square tile far
fewer pixels than it needed: a 96px cover got the 120x90 file and kept 90px
after the crop. They are chosen by their short side now, the album art is
preferred over the video frame when the search returns both, and the display
sizes account for the tiles being up to 180px on a 2x screen.
Search results now carry a small thumbnail, so the artwork downloaded when a
YouTube track is saved would have shrunk with them. The URL is upsized to the
configured artwork resolution right before the download event goes out.
Repeated searches and song lookups now reuse a single-flight memory cache, so
concurrent or debounced queries hit YouTube once instead of on every keystroke.
Thumbnails are resolved at a configurable size instead of the largest variant,
which cuts the image payload the browser pulls from Google.
Drop the dead Keycloak and ConnectionStrings sections from the Worker and move
its hardcoded dev values to AppSettings.Development.json, so the base file is
the empty template the convention describes. Remove the stale Authentication
block from the API dev settings (provisioning writes those to user-secrets) and
the ManifestFileName left over from DASH. Point the stream-ticket key and the
Worker scratch directory at deploy/keys and deploy/.tmp instead of D:\weed and
D:\tempsFilesDev, and make the API listen on localhost instead of a fixed LAN
address.
The YARP routes, the bucket path transform and the StreamTicket section only
existed in appsettings.Development.json, so a Production run had no routes at
all and every /media request fell through to a 404. Move them to
appsettings.json and leave only the environment-specific overrides (key path)
in the Development file; the container gets the filer address, the bucket and
the allowed CORS origins from environment variables.
YouTubeMusicService swallowed the exception into an ErrorOr description
with no trace on the server side. Adds ILogger to the three operations.
The download activity no longer names yt-dlp in its messages, since the
downloader is an implementation detail behind IYouTubeDownloaderService.
yt-dlp goes through pip in a venv rather than the standalone PyInstaller
binary, which unpacks on every invocation and roughly doubles latency.
deno solves the nsig challenges.
Code review of the Artist/ownership refactor surfaced three gaps:
- SyncUser updated User.Name on every login but never the linked
Artist row, so renames never reached track credits - defeating the
point of centralizing names in Artist.
- YouTubeTrackProvisioner looked up artists by ExternalId only when a
browseId was known, so it could never reuse the ExternalId-less
rows the migration backfilled from legacy Artist strings, creating
duplicate Artist rows for the same person. It now adopts the
backfilled row (assigning it the browseId) when one matches by
normalized name.
- AddArtistsAndTrackOwnership deduped legacy YouTube artist strings
case-sensitively before hashing them into a case-insensitive unique
index, and never guarded against local tracks with no UserHasTrack
row - either would abort the migration on real data. Backfill now
dedupes via DISTINCT ON over the normalized name and drops orphaned
local tracks before the ownership check constraint is added.
Track mixed catalog fields (Title, Artist credit, Duration) with
picture-processing and audio-transcode pipeline state as ten flat
columns plus two computed [NotMapped] flags, making the entity read
like a media-pipeline worksheet.
Introduce TrackPictures and TrackAudio as EF owned types mapped onto
the same existing columns (explicit column names, no schema change -
verified via `dotnet ef migrations has-pending-model-changes`).
Track now reads as Title/Artist/Duration/Pictures/Audio instead of
fourteen loose properties.
Drop the now-redundant Track.Artist string column now that every
track (local and YouTube) links its credited artists through
TrackArtist. TrackApplicationResponse formats the display string by
joining TrackArtists ordered by position, and every read query that
projects a track now includes TrackArtists/Artist so the join isn't
empty at response time.
Track get-or-create for YouTube videos was duplicated between the
old GetYouTubeTrackStream query handler (which also wrote to the
database) and AddYouTubeTrackToPlayList, with frontend-supplied
metadata (title/artist/duration/thumbnail) trusted at add-to-playlist
time instead of resolving it server-side.
Extract YouTubeTrackProvisioner as the single get-or-create path,
resolving structured artist refs (id + name) from YouTube Music so
multi-artist songs map to distinct Artist rows. Rename
GetYouTubeTrackStream to ResolveYouTubeTrackStream (it writes, so it
is a command, not a query) and delegate to the provisioner.
AddYouTubeTrackToPlayList slims down to (UserId, PlayListId, VideoId)
since all metadata now comes from the provisioner/YouTube API.
Track.Artist was a plain string snapshot with no identity, and
UserHasTrack doubled as both "in my library" and "I own this",
conflating collection membership with administrative ownership.
Introduce Artist/TrackArtist to give artist credits their own
identity (needed so YouTube tracks with multiple credited artists,
and future renames, can be modeled), and Track.OwnerUserId to
represent ownership explicitly (local tracks only, enforced by a
check constraint). CreateTrack now provisions the uploader's Artist
row and links it via TrackArtist; DeleteTrack authorizes against
OwnerUserId instead of any UserHasTrack row.
Artist, ExternalId, OriginalPictureName and OriginalAudioName were
non-nullable with '' sentinels, which forced the unique (Source,
ExternalId) index to rely on a filtered "<> ''" clause. Make them
truly nullable and drop the empty-string sentinels, backfilling
existing rows to NULL. Also fixes a formatting regression
(`track != null&&`) introduced alongside the nullability change.
Admin