Never trust the client's completion notification alone. The client can lie about what it uploaded, or upload nothing and claim success, or succeed and then vanish before telling you. So the authoritative signal is a storage event delivered to a queue (Module P-19), and the worker verifies the object with a HeadObject before marking anything ready. The client's callback is a latency optimisation, not evidence.

Large uploads need multipart and resumability. Above roughly 100 MB, use multipart upload: parts uploaded in parallel (throughput scales), each retryable independently, and the upload ID persisted so a client can resume after a network drop rather than restarting a 4 GB transfer.

And the FinOps detail that catches everyone: incomplete multipart uploads are billed and invisible. Parts from abandoned uploads sit in the bucket, do not appear in a normal object listing, and accrue storage charges indefinitely. A lifecycle rule to abort incomplete multipart uploads after a few days is one line of configuration and has saved many teams a recurring, unexplained storage bill (Module O-8).

Orphans go both ways, and need a sweeper. This is Module P-16's atomicity gap: a row with no object (upload never completed) and an object with no row (row insert failed, or a rollback after upload). Both accumulate. A periodic reconciliation — expire pending rows older than a day, and list-versus-database compare to find unreferenced objects — is the same required backstop as Module P-24's payment reconciliation, for the same reason.

---

Uploaded Files Are Attacker-Controlled Input

An upload endpoint is a channel for putting arbitrary bytes into your infrastructure, and the attacks are specific.

• Never trust the declared content type. Content-Type: image/jpeg is a claim. Sniff the actual bytes (magic numbers) and validate against an allowlist. A file named .jpg containing PHP or a polyglot is the classic upload exploit.
• Decompression bombs. A 10 MB PNG can decode to 50 GB of pixels; a small ZIP can expand to terabytes. Any decoder must run with explicit limits on output dimensions, pixel count and memory — and preferably in a separate process with a memory cap, so a bomb kills a worker rather than your service (Module A-7's bulkhead).
• SVG is executable. An SVG can contain <script>, so serving a user-uploaded SVG from your own origin is stored XSS. Which leads to the most important structural defence here:

Serve user-uploaded content from a separate domain, not a path on your main one. A different origin means uploaded content cannot access your cookies, localStorage, or same-origin APIs even if it manages to execute. Combine with Content-Disposition: attachment where appropriate, X-Content-Type-Options: nosniff, and a restrictive CSP.

• Strip EXIF. Photographs commonly carry GPS coordinates, device identifiers and timestamps. Publishing them verbatim discloses a user's home address, which is a privacy incident rather than a bug (Module O-9). Strip metadata on ingest, keeping only what you need.
• Scan for malware if users share files with each other, and quarantine until scanned rather than serving immediately.
• Charge uploads against a quota (Module P-18), because storage is the one resource where an abusive user's cost to you is unbounded and permanent.

---

The Media Pipeline Is a Job DAG Over an Immutable Original

Once the bytes are stored, derivatives are produced. This is Module P-19's territory with a specific shape:

Design points that matter:

Keep the original forever and treat every derivative as a cache. When a codec improves, a thumbnail size changes, or a transcode bug is found, you re-derive. A pipeline that discards the original after transcoding has converted a re-runnable job into permanent data loss — and this is the single most consequential decision in the whole pipeline.

Transcoding is long, CPU/GPU-heavy work that does not fit a request or a short job. A two-hour video is tens of minutes of CPU. Two techniques make it tractable:

• Segment-parallel transcoding. Split the source into chunks, transcode chunks in parallel across many workers, concatenate. A 30-minute job becomes 3 minutes with ten workers, and each chunk is independently retryable — the checkpointing pattern from Module P-19 applied by construction.
• Spot/preemptible capacity. Transcoding is the ideal spot workload: interruptible, retryable, and the dominant cost. With per-chunk checkpointing, a reclaimed instance costs one chunk (Module O-8).

Adaptive bitrate is why there are multiple renditions. HLS and DASH publish a manifest listing several quality levels, each split into short segments; the player measures its own bandwidth and switches level between segments. That is why a video is not one file but a manifest plus hundreds of small objects — which has consequences for signing, below.

Completion detection needs care. "All renditions ready" is a fan-in over independent jobs, so track required outputs per asset and mark published when the set is complete — rather than having the last job to finish assume it was last, which is a race.

Images: on-the-fly beats pre-generation, with one condition. Generating every rendition up front means guessing which sizes you will need and paying to store all of them. On-the-fly resizing behind a CDN generates a variant on first request and caches it, giving unlimited variants at the cost of one cold request each. The condition is that the parameters must be constrained — an allowlist of sizes, or a signed parameter set — because an open resize endpoint is a denial-of-service primitive: an attacker requests ten thousand distinct dimensions and each one is a cache miss that costs you CPU.

---

Delivery: CDN, and the Problem HLS Creates for Signed URLs

Always put a CDN in front (Module F-15). Three reasons, and the second is usually the largest: latency for distant users, egress from a CDN is materially cheaper than egress from a bucket, and origin request volume collapses.

Cache headers follow from key design. Content-addressed keys get max-age=31536000, immutable. Mutable keys need a short TTL plus revalidation, and a purge path — which is the invalidation problem you avoided by not using mutable keys.

Private content has two mechanisms, and choosing wrong is a common mistake:

• Signed URLs — a per-object URL with an expiry and a signature. Simple, granular, and correct for "download this invoice."
• Signed cookies / signed prefixes — one credential authorising a path prefix for a period. Necessary the moment a single logical asset is many objects.

That second case is exactly HLS: a manifest plus hundreds of segments, requested dynamically by the player as it switches quality. Signing each segment individually would require rewriting the manifest per user per session with hundreds of signed URLs that then expire mid-playback. Signed cookies over the whole asset prefix is the right tool, and discovering this after building per-object signing is a recognisable rite of passage.

Range requests matter for media. Seeking in a video is an HTTP Range request, so the CDN must support and cache ranges — otherwise every seek is an origin fetch of the whole file.

Hotlinking and abuse: referrer checks are trivially bypassed. Short-lived signed URLs bound to a session, plus per-user rate limiting on the URL-issuing endpoint (Module P-18), is the real defence. And remember that a signed URL, once issued, is a bearer token — anyone with the link has access until it expires, so keep expiries short for sensitive content and never put them in a page that gets cached or indexed.

---

Storage Tiering, Where the Obvious Saving Is Often a Loss

Lifecycle rules move objects from hot storage to infrequent-access to archive tiers, and the per-GB savings are large — 60–95%. The traps are equally large and less advertised:

The practical rule: tier large, genuinely cold, long-lived objects — originals, backups, old media — and leave small objects where they are. Then remember that for popular content, egress dominates storage entirely, so the meaningful optimisation is CDN hit ratio rather than the storage class (Module O-8).

Why this matters in production: the failures here are quiet and cumulative. Incomplete multipart uploads billed for years. Orphaned objects with no rows, and rows with no objects, both growing. A lifecycle rule that moved millions of thumbnails to archive and doubled the bill. A user's home address published in the EXIF of every photo they uploaded. An SVG avatar executing script on your main domain. None of these page anyone; all of them are found by an audit, a bill, or a security report. The mechanisms that prevent them — a lifecycle abort rule, a reconciliation sweeper, EXIF stripping, and a separate content domain — are each a day's work at most, and all of them are much harder to add later.

---

Where This Shows Up in Your Stack

• PostgreSQL: store the key, size, MIME type, checksum and a status column — never the bytes. The status field is what makes the two-phase upload safe, and the pending-row expiry plus a list-versus-database compare is the orphan reconciliation. A partial index on WHERE status = 'pending' keeps the sweeper cheap (Module F-13).
• Node.js: stream, never buffer. await request.arrayBuffer() on a 2 GB upload is an out-of-memory kill; pipeline() from the request stream to the uploader is bounded, and respects backpressure (Module A-8). If you must proxy an upload, stream it — but prefer pre-signed uploads so you are not in the path at all. Run image and video decoding in a separate process with memory limits, never in the process serving requests (Module P-19).
• Object storage (S3 and equivalents): enable versioning, set a lifecycle rule to abort incomplete multipart uploads, block public access and serve exclusively through the CDN, and use conditional writes (If-None-Match) where you need optimistic concurrency or fencing (Module A-2). Drive workflows from bucket events into a queue, never from LIST.
• CDN: the delivery layer and the cost control. Signed URLs for single objects, signed cookies or prefixes for multi-object assets like HLS, Range support for media, and long immutable TTLs made safe by content-addressed keys (Module F-15).
• Queues and workers: the pipeline spine — bucket event to queue to worker, with per-rendition fan-out, idempotent handlers keyed on (asset, rendition), and chunked checkpointing so a reclaimed spot instance costs one segment (Modules P-16, P-19, O-8).
• Serverless: a poor fit for transcoding — execution ceilings, small ephemeral /tmp, and no work after the response (Module F-10). Excellent for the orchestration around it: issuing pre-signed URLs, handling bucket events, and enqueueing jobs.

---

Summary

• The database stores metadata and a pointer; the object store stores the bytes. Large binaries in a column enter the WAL and therefore every replica and CDC consumer, inflate backups and restore time, pass through driver memory, hold a connection for the transfer, and pay OLTP storage prices for cold data. Small binaries under a few hundred kilobytes are a legitimate exception.
• Object storage is not a filesystem: a flat keyspace where prefixes only look like directories, no rename (copy plus delete, O(size), not atomic), whole-object writes with no in-place update, and no transactional multi-object operations.
• Read-after-write by key is strongly consistent on S3 now; LIST and cross-region replication are not. Drive workflows from a database row or a bucket event, never from a listing. And conditional writes (If-None-Match) now make object storage fenceable, which invalidates the old assumption from Module A-2.
• Durability is not backup. Eleven nines means the provider will not lose your bytes; it does not stop you, a bug, or ransomware deleting them. Versioning plus object lock is the backup.
• Make keys content-addressed and immutable, with the logical name in your database. This eliminates cache invalidation outright — an immutable object is cacheable forever because a change produces a different key — and gives free deduplication, verifiable integrity, and rollback as a metadata update. The costs are indirection and a mark-and-sweep garbage collector. Never build keys from user-controlled strings.
• Pre-signed uploads are the most important pattern here: the client uploads directly to the store while your tier only authorises, generates the key, and records a pending row. The constraints must be inside the signature — content-length-range, content-type prefix, key prefix, short expiry — or you have built an unbounded open file-drop.
• Never trust the client's completion notification. The authoritative signal is a storage event delivered to a queue, and the worker verifies the object with HeadObject before marking it ready.
• Use multipart above ~100 MB for parallel throughput, per-part retries and resumability — and set a lifecycle rule to abort incomplete multipart uploads, because abandoned parts are billed, invisible in normal listings, and accrue indefinitely.
• Orphans occur in both directions (rows without objects, objects without rows) because of the atomicity gap, so a reconciliation sweeper is required, not optional.
• Uploaded files are attacker-controlled input: sniff the real content type rather than trusting the header, run decoders with explicit pixel/memory limits in a separate process because a 10 MB PNG can decode to 50 GB, and serve user content from a separate domain since an SVG can execute script and a different origin cannot reach your cookies. Strip EXIF, because publishing a photograph's GPS coordinates discloses a user's home address. Charge uploads against a quota, since storage is the one resource whose abuse cost is unbounded and permanent.
• Keep the original forever and treat every derivative as a cache. Discarding it converts a re-runnable job into permanent data loss the first time a codec, a size, or a transcode bug changes.
• Transcoding is long CPU/GPU work: split it into segments, transcode in parallel, concatenate — which shortens wall-clock time and makes each chunk independently retryable — and run it on spot capacity with per-chunk checkpointing, since it is interruptible and the dominant cost.
• Adaptive bitrate is why one video is a manifest plus hundreds of segments, and completion is a fan-in over independent rendition jobs rather than whichever job finishes last.
• For images, on-the-fly resizing behind a CDN beats pre-generation — unlimited variants, one cold request each — provided the parameters are allowlisted or signed, because an open resize endpoint is a denial-of-service primitive.
• Always front the store with a CDN: latency, collapsed origin volume, and egress that is materially cheaper than from the bucket. Use signed URLs for single objects and signed cookies or prefixes for multi-object assets — HLS makes per-object signing impractical, since a player fetches hundreds of segments dynamically. Support Range or every video seek is a full origin fetch. And treat an issued signed URL as a bearer token.
• Tiering traps: retrieval charges and restore latency, minimum storage durations that make short-lived cold objects cost more, per-object transition fees that can exceed the savings for millions of small files, and minimum billable object sizes. Tier large, cold, long-lived objects only — and for popular content, egress dominates storage, so CDN hit ratio is the real lever.

The next module, RAG and LLM-Serving Architecture, applies this phase's thinking to a dependency with properties nothing else in the course has: it is non-deterministic, its latency is measured in seconds, its cost is per token rather than per request, and it streams. It builds on Module P-21's retrieval work and asks the architectural questions — how to cache something non-deterministic, how to budget capacity in tokens, and what a sensible fallback looks like when the dependency may return a confidently wrong answer rather than an error.

---