Fill the Server: 80% off your first 4 monthly invoices, monthly billing. Ends Nov 30, 2026. 25 spots in this batch.FILL25
Fill the Server, annual: about 29.63% off your first annual invoice. Ends Nov 30, 2026.FILL25Y
Fill the Server, 2 year term: about 15.69% off your first biennial invoice. Ends Nov 30, 2026.FILL25B
Fill the Server, 3 year term: about 11.11% off your first triennial invoice. Ends Nov 30, 2026.FILL25T
80% off your first 3 months, monthly billing. Ends Dec 25, 2027.3MONTHS80
50% off your first annual Shared term. Ends Dec 31, 2027.50SHARED
Shared Starter, 3 year term: C$105.84 total.MOOSE294
20% off your first 3 monthly VPS invoices. Ends Dec 31, 2027.20VPS3
25% off your first annual base VPS term.25VPS

← All guides

Object storage

S3 multipart upload explained: part size, retries, checksums and cleanup

Published August 2026 · 9 min read · Updated 2026-09-07

An S3 multipart upload splits one object into numbered parts that can be transferred independently and then completed as one object. That makes failed-part retries smaller and can improve throughput, but the upload is not finished until the completion request succeeds. Amazon S3 also keeps incomplete uploaded parts until the upload is completed or aborted, so cleanup is part of the design rather than optional housekeeping.

An S3 multipart upload has a lifecycle

The client creates a multipart upload and receives an upload ID, uploads numbered parts, records the part numbers and returned ETags, and finally sends the ordered part list in a completion request. Amazon S3 assembles the completed object in ascending part-number order.

Until completion, those uploaded parts are not the final object the application expects to read. If the client crashes, the upload can remain incomplete and consume storage until it is completed or aborted.

AWS: multipart upload overview | AWS API: CompleteMultipartUpload

Know the part limits before choosing a chunk size

Amazon S3 currently documents part numbers from 1 through 10,000 and part sizes from 5 MiB through 5 GiB, with no minimum size for the last part. Its current multipart limits page lists a 48.8 TiB maximum object size. Those are Amazon S3 limits, not a promise that every S3-compatible service exposes the same ceiling.

Choose a part size that keeps the expected object comfortably below the backend's part-count limit. Larger parts reduce the number of requests and bookkeeping entries; smaller parts reduce the amount retransmitted after one failed part. Verify the exact limits of the S3-compatible service and SDK you will use.

AWS: multipart upload limits

Parallel parts can improve transfer speed

Sending several parts concurrently can use available bandwidth more effectively, especially over higher-latency links. Too much parallelism can exhaust memory, file descriptors or network capacity on the client.

Tune concurrency with representative object sizes and networks. A setting that helps a datacentre server can overwhelm a small office connection, and a provider may enforce request or connection limits before the client reaches its own ceiling.

Part retries are smaller than restarting the whole object

When one part fails, a well-behaved client retransmits only that part, so a 100 GB object does not restart from byte zero. This is one reason backup and transfer tools often use multipart operations for large archives.

Retry logic should still have limits and visible errors. Infinite retries can hide a broken credential, endpoint, network path or quota condition for hours. Record the upload ID and part state in workflows that need to resume safely after a process restart.

Do not treat the final multipart ETag as a whole-file MD5

On Amazon S3, the ETag of an object created by multipart upload is not the MD5 digest of the entire object. If end-to-end integrity matters, use the checksum mechanism supported by the exact SDK and storage backend instead of comparing the final ETag to a local whole-file MD5.

Amazon S3 currently supports full-object and composite checksum modes for multipart uploads, with algorithm-specific rules. For example, part-level composite checksums require consecutive part numbers beginning with 1. S3-compatible services can differ, so exercise the checksum mode you intend to depend on instead of assuming AWS-identical behavior.

AWS: checking multipart object integrity

Capacity planning must include in-progress parts

Amazon S3 bills storage for uploaded multipart parts until the upload is completed or aborted. Other providers may account for incomplete parts differently, but a client near its hard capacity limit can still fail while assembling a large object even if the final object would barely fit.

Leave operational headroom and monitor usage reported by the storage platform. Treat quota failures as a capacity event and resolve the capacity problem before retrying writes.

Abort abandoned uploads, then verify cleanup

A small SDK upload can use one PUT and never exercise multipart APIs. Force a multipart test object, verify the completed object, start a second upload, abort it and confirm its unfinished parts are removed.

Amazon S3 warns that part uploads already in flight can still succeed or fail after an abort request, so complete cleanup can require another abort and a ListParts check until the parts list is empty. Amazon S3 also supports a lifecycle action for automatically aborting incomplete multipart uploads after a configured number of days; verify that an S3-compatible backend supports equivalent lifecycle behavior before relying on it.

AWS API: AbortMultipartUpload | AWS: lifecycle cleanup for incomplete multipart uploads

Next step

Learn how S3 multipart upload works: numbered parts, AWS part-size limits, parallel retries, completion, checksums and cleanup of abandoned uploads.

Explore DotMoose Object Storage

Keep reading

Related guides.

boto3 S3-compatible storage: custom endpoint, put_object and connection tests

Connect boto3 to S3-compatible storage with endpoint_url. Start with a read-only list, then optionally verify put_object with one isolated write/read/delete round trip.

Read the guide

S3 presigned URLs explained: temporary access without sharing keys

Learn how S3 presigned URLs authorize a specific operation for a limited time, where they help, what they expose and how to use them safely.

Read the guide

Veeam S3-compatible repository: setup and compatibility checklist

Use this checklist before adding an S3-compatible repository to Veeam Backup & Replication: protocol requirements, bucket ownership, immutability, credentials, capacity and restore testing.

Read the guide

More object storage guides

We will handle the move. You check it first.

Your current site stays online until your new one is ready. You approve the copy first, and if anything is not right we switch you back right away. The price you pick is the price you renew at.

Start my free moveSee plans and prices
See pricesFree move