Full Stack
Intermediate 13 min read Full Stack

File Upload & Storage

Uploads move bytes, not JSON, and need their own pipeline: multipart parsing, size and type validation, virus scanning, and storage in object storage rather than the app server. Presigned URLs keep the API server out of the byte-transfer hot path.

multipart/form-datamulterS3presigned URLobject storagefile validationstreaming upload
Prerequisites: Forms & Validation
Introduction

What is this?

File upload is the pipeline that gets bytes from a browser to durable object storage. The client sends multipart/form-data, the server parses it with a tool like multer, validates size, type and filename, and writes the file to object storage such as S3, Cloudflare R2 or Vercel Blob. For large files, presigned URLs let the browser upload directly to storage, keeping the API server from proxying megabytes of bytes through itself.

Motivation

Why do you need it?

  • The app server's disk is ephemeral and tiny; object storage is durable, versioned and elastic.
  • Storing files separately from your code means redeploys and autoscaling never destroy user uploads.
  • Presigned URLs remove the server from the transfer path, cutting bandwidth costs and latency for big files.
  • Server-side validation (size, type, filename) is the only real gate — the client can claim anything.
  • A CDN in front of object storage serves avatars and images from the edge instead of your origin.
Think about it

Mental model

Imagine

Shipping a couch

Think of

A large file upload

You don't mail a couch through your home mailbox. You validate it at the loading dock (server), hand the trucker a special gate pass (presigned URL), and the truck drives straight to the warehouse (object storage). Your office is never in the way of the load.

Mechanics

How it works

  • The client builds a FormData with the file and sends a multipart POST to /api/upload.
  • multer parses the multipart body into a temporary file or buffer, enforcing limits like fileSize and files count.
  • The server validates the magic bytes / MIME type, filename, and runs antivirus if the policy requires.
  • The file is streamed into object storage under a generated key (random name, so no path traversal), and the URL/metadata is stored in the database.
  • For large files, the server issues a presigned PUT URL and the client streams bytes directly to storage, then notifies the server on completion.

Direct-to-storage upload with a presigned URL.

Client picks file
POST /api/upload
Server validates
Presigned URL
Client streams bytes
Storage notifies
Reference

Key commands

npm install multer

Add the multipart parser for Express.

added 8 packages ...
npm install @aws-sdk/client-s3 @aws-sdk/s3-request-presigner

AWS SDK for S3 plus presigned URL generation.

npm install @uploadthing/react

A batteries-included upload component with built-in validation.

curl -F "avatar=@./photo.jpg" http://localhost:3001/api/upload

Send a real file as multipart/form-data from the terminal.

{"key":"avatars/u1/1699999999-photo.jpg","size":1048576}
aws s3 ls s3://your-bucket/avatars/

Verify uploaded objects landed in the bucket.

Hands-on

Examples

Multipart upload with multer and Express

Multipart upload with multer and Express
const multer = require("multer")
const path = require("path")
 
const upload = multer({
  limits: { fileSize: 5 * 1024 * 1024 }, // 5 MB
  fileFilter(req, file, cb) {
    const ok = ["image/jpeg", "image/png", "image/webp"].includes(file.mimetype)
    cb(ok ? null : new Error("Unsupported file type"), ok)
  },
})
 
app.post("/api/upload", upload.single("avatar"), (req, res) => {
  if (!req.file) return res.status(400).json({ error: "No file" })
  const key = `avatars/${req.userId}/${Date.now()}-${req.file.originalname}`
  res.status(201).json({ key, size: req.file.size })
})

multer enforces a 5 MB limit and an image-only allowlist before your handler runs. The key is namespaced by user and timestamped so files never collide and users can't write outside their folder. In production the bytes move to object storage instead of the local disk.

Real world

In production

  • Profile avatars are presigned-uploaded to S3/R2 and served through a CDN with a cache-friendly filename.
  • CSV/PDF exports stream to storage and the user receives a short-lived download URL instead of a mega-response.
  • Videos upload chunked via presigned multipart uploads, with a worker transcoding after completion.
  • Virus scanning (e.g., ClamAV) runs on the object asynchronously; flagged files are quarantined and never served.
Guidelines

Best practices

  • Validate on the server: enforce fileSize via multer limits and allowlist MIME types — never trust the client.
  • Generate storage keys server-side with random names; never reuse user filenames directly (path traversal, collisions).
  • Keep files out of the app server's disk: stream to object storage, or use presigned URLs for direct upload.
  • Scan uploads for malware and restrict content-type headers served back to users (X-Content-Type-Options: nosniff).
  • Set short TTLs on presigned URLs and sign them per-user so a leaked URL expires quickly.
Watch out

Common mistakes

  • Saving files onto the server's local disk, which is wiped on redeploy and cannot scale horizontally.
  • Storing original filenames in object keys, enabling path traversal and filename collisions.
  • Only validating file type from the extension or Content-Type header, both trivially spoofable.
  • Sending huge files through the API server JSON pipeline, exhausting memory and bandwidth.
Root cause analysis

Common issues

Error
Error: Unsupported file type (LIMIT_UNEXPECTED_FILE or custom filter error)
Root cause

The MIME allowlist rejected the upload, or the field name doesn't match.

Fix

Check the form field name matches upload.single('avatar'), and confirm the MIME type is allowed.

Error
MulterError: File too large
Root cause

The file exceeds the fileSize limit.

Fix

Raise the limit, or switch to chunked/presigned uploads for big files, and tell the user the size cap.

Error
AccessDenied on the presigned PUT
Root cause

The bucket policy or the IAM role doesn't allow the s3:PutObject action for the generated key.

Fix

Grant s3:PutObject (and s3:GetObject) on the exact key prefix to the upload role, then regenerate the URL.

When things break

Debugging tips

  • Test the endpoint with curl -F before touching the UI — it isolates server config from client issues.
  • Check multer's errors: LIMIT_FILE_SIZE is thrown by limits, not by your validation code.
  • Verify the file actually reached storage: aws s3 ls and a direct GET on the object URL.
  • Watch req.headers['content-length'] vs req.file.size to spot truncated uploads or encoding issues.
Interviews

Interview checkpoint

Practice

Hands-on lab

Build a Validated Image Upload Endpoint
40 min · intermediate
In progress
Objectives
  • Accept an image via multipart/form-data
  • Enforce size and type limits with multer
  • Store the file under a generated key
  • Verify the file round-trips back from disk

Build a Validated Image Upload Endpoint

40 min · intermediate

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/3
1

Which content type is used to upload a file with additional form fields?

2

A presigned URL lets the browser upload a file directly to object storage without the API server relaying the bytes.

3

Users can upload any file, and you store it under the original filename. What security bug does this create?

Quick reference

Cheat sheet

Upload pipeline
enctype="multipart/form-data"Form that includes files
upload.single('field')Parse one file from the multipart body
limits: { fileSize }Reject oversized uploads server-side
fileFilterAllowlist MIME types
s3.getSignedUrl('putObject', ...)Issue a presigned PUT URL
aws s3 lsVerify objects in the bucket
Go deeper

Official documentation

Multer documentation
docs
AWS SDK — Presigned URLs
docs
MDN — Sending form data (multipart)
docs
Vercel Blob documentation
docs