One Song, Many Systems: The Backend Engineering Behind Music Data

To a listener, a song is one thing. To a backend, it's a distributed-data problem: dozens of records, owned by organizations that share no schema, no primary keys, and sometimes no definition of the entity they're describing.

A streaming service logs plays of a recording. A distributor holds delivery metadata and revenue reports. A performing rights organization (PRO) like ASCAP or BMI registers the underlying composition. The MLC holds its own copy of that composition data. A label keeps contracts, ownership records, and royalty statements. Each system sees one slice of the song's commercial life, and none of them is authoritative for all of it.

Collecting those records is the easy part. The hard part is deciding which records describe the same entity, what each one actually asserts, and how those assertions change over time. That's an entity-resolution, provenance, and temporal-modeling problem, and it's why music data is unusually hard to engineer.

The domain model: one song, three entity types

Before any pipeline work, you need the right model. A "song" is really at least three kinds of entity, each with its own lifecycle and its own systems of record:

• Recording (the master): a specific captured performance. Delivered by distributors, played on streaming services, identified by an ISRC.
• Composition (the work): the melody and lyrics. Registered with PROs and the MLC, identified by an ISWC.
• Parties: writers, performers, publishers, labels, and administrators, each with their own IDs and naming conventions.

The relationships are many-to-many. One composition can be embodied by many recordings (the original, a radio edit, a remaster, a live cut). One recording can embody more than one composition (a medley, a sample). Parties attach to both sides with different roles and shares.

Now follow a single release, "Stay," through those systems:

A person who knows the catalog sees one song. The backend sees unrelated strings, mismatched IDs, and what look like different people.

Identifiers are join keys, not ground truth

Engineers prefer stable IDs over names because names drift. Music has good ones: ISRC for recordings, ISWC for compositions, IPI and similar codes for parties. But in practice they behave like a sparse, dirty foreign-key column:

• Missing: a revenue report carries an ISRC but nothing about the composition. A spreadsheet carries neither.
• Unlinked: a composition registration lists writers but not which ISRCs embody the work, so there's no edge to traverse.
• Wrong: legacy catalogs contain mistyped, reused, or duplicated codes.
• Intentionally different: a new live recording gets a different ISRC while embodying the same composition.

Even a perfect ID only answers a narrow question. An ISRC tells you which recording a row refers to. It doesn't tell you who owns that recording today, which contract governs it, or how revenue should be split.

Treat identifiers as high-confidence matching signals, not as the identity itself. Keep your own internal surrogate keys for canonical entities, and store external IDs as attributes with their source attached. That way a bad ISRC from one feed doesn't silently merge two recordings across your whole graph.

No single source of truth: store claims, not values

In most systems you can name one database as the authority. If two services disagree about a customer's address, the customer service wins. Music rights don't work that way, because authority is per-fact, not per-system:

• The distributor is authoritative for what metadata was delivered to a platform.
• The contract is authoritative for the economic deal between two parties.
• A PRO or MLC record establishes what share was registered or claimed in that system, not necessarily the legally correct ownership share.
• The royalty statement is authoritative for what was reported, not for what should have been.

So last-write-wins is the wrong conflict strategy. If one source says a writer's share is 25% and another says 50%, overwriting either one destroys information. The gap might be a disputed split, a corrected registration, different territories, or two values that were each true in different periods.

Model each incoming value as a claim: a record of who asserted what about which entity, when, and of what kind (delivered metadata, contractual term, registration, reported usage). The canonical view is then a derived projection over claims, with explicit precedence rules per field. When claims conflict and no rule resolves them, the system flags the conflict rather than picking a winner. Its job isn't to make disagreement disappear; it's to surface it with enough context for someone to resolve it.

Time is part of the data: model it bitemporally

Music data is constantly revised. Ownership transfers, administrators change, catalogs move between distributors, and platforms send corrections months after the original usage report. If you store only the latest state, you can answer today's question and lose the ability to explain yesterday's answer.

Consider a recording whose ownership changes on July 1. Assume the transfer agreement allocates revenue by usage period. Under that assumption, June revenue belongs to the previous owner and August revenue to the new one. Then a September report arrives with adjustments for both June and August. File arrival time tells you almost nothing about who gets paid.

You need at least two time axes on every claim:

• Valid time: when the fact was true in the world (the usage period, a contract's effective dates, an ownership window).
• Transaction time: when your system learned it (file received, record ingested).

With both, you can answer "who owned this in June, as we understood it in August?" and replay a past royalty run exactly. Pair that with append-only storage and explicit links between a correction and the record it supersedes. History isn't clutter here; it's part of the business logic.

Entity resolution is a confidence problem

When identifiers are present and consistent, matching is a straightforward join. When they aren't, you fall back on noisy secondary signals: title, artist, writers, duration, release date, territory, and catalog relationships. Two different recordings can share a title. One recording can appear under several titles. Artist names change, featured performers are formatted inconsistently, and a spelling fix can look like a brand-new contributor.

So the matcher shouldn't return a bare yes or no. It should return a match with a confidence tier and the evidence behind it:

Persist the tier, score, and contributing signals on the link itself, and make every link reversible. When a reviewer splits a bad merge, everything derived from it, such as rollups and royalty allocations, should be recomputable.

Automation matters because catalogs run to millions of rows. Human review matters because some ambiguities are contractual or contextual, not computational. The design goal is to let clear cases flow through and route uncertain ones to people who have the context.

Putting it together: the pipeline

It's tempting to frame all this as ingestion: connect the sources, normalize the files, load a database. That's only the first layer. A reliable system preserves raw inputs, turns them into claims, resolves entities with explicit confidence, and keeps a bitemporal history that every output can be traced through.

Data flows from distributors, platforms, PROs and the MLC, and labels through four layers:

1. Land raw files. Store every file immutably, exactly as received, tagged with its source and arrival time.
2. Normalize into claims. Parse each source format into a common schema. Every value becomes a claim: who asserted it, what kind of claim it is, when it was true, and when we learned it.
3. Resolve entities. Link claims to canonical recordings, works, and parties. Each link stores a confidence tier and its evidence. Possible matches go to a human review queue, and reviewers' decisions can be reversed.
4. Bitemporal claim store and canonical view. Append-only history. The canonical view is projected from claims with per-field precedence rules; conflicts no rule can settle are flagged for review, never overwritten.

The outputs sit on top: royalty runs that can be replayed for any past date, statements and reports with per-field provenance, and an audit trail that traces any figure back to raw files. The key property is lineage: any royalty figure or report row can be walked back through the canonical view and the claims behind it to the raw files that asserted them. Uncertain matches and unresolvable conflicts leave the automated path and go to people, and their decisions flow back in as new, auditable records.

Why this is a good backend problem

Scale matters, but volume isn't the hardest part. The harder part is preserving meaning as data crosses organizations that each describe a different slice of the same commercial reality. A listener presses play on one song. Behind that, the industry produces a lot of records. Turning them into a reliable account of ownership, usage, and revenue takes more than putting them in one database. It takes knowing what each record means, where it came from, how certain the match is, and when it was true.