-
Notifications
You must be signed in to change notification settings - Fork 999
Description
Describe the bug
2025.7.0 causes AWS S3 (minio) chunked uploads to fail with <Code>IncompleteBody</Code><Message>You did not provide the number of bytes specified by the Content-Length HTTP header.</Message>
due to incorrectly set content-length: 0
by cloudflared.
To Reproduce
Upload a file with aws like aws --endpoint-url=https://cloudflare-proxied-s3.example.com s3 cp file.png s3://bucket/file.png
Expected behavior
Cloudflared should not set content-length to 0 for chunked transfers.
Environment and versions
- OS: Ubuntu Linux 24.04.2 LTS
- Architecture: AMD64
- Version: 2025.7.0
Logs and errors
Below is an example request.
PUT /xxxx-docs/file.txt HTTP/1.1
Host: s3.xxxx.net
User-Agent: aws-cli/1.30.6 md/Botocore#1.37.28 ua/2.1 os/linux#6.8.0-64-generic md/arch#x86_64 lang/python#3.10.12 md/pyimpl#CPython m/N cfg/retry-mode#legacy botocore/1.37.28
Content-Length: 0
Accept-Encoding: gzip, br
Amz-Sdk-Invocation-Id: xxxx
Amz-Sdk-Request: attempt=1
Authorization: AWS4-HMAC-SHA256 Credential=xxxx/20250801/us-east-1/s3/aws4_request, SignedHeaders=content-encoding;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-sdk-checksum-algorithm;x-amz-trailer, Signature=xxxx
Cdn-Loop: cloudflare; loops=1
Cf-Connecting-Ip: xxxx
Cf-Ipcountry: XX
Cf-Ray: xxxx-CPH
Cf-Visitor: {"scheme":"https"}
Cf-Warp-Tag-Id: xxxx
Connection: keep-alive
Content-Encoding: aws-chunked
Content-Type: text/plain
X-Amz-Content-Sha256: STREAMING-UNSIGNED-PAYLOAD-TRAILER
X-Amz-Date: 20250801T234228Z
X-Amz-Decoded-Content-Length: 13
X-Amz-Sdk-Checksum-Algorithm: CRC32
X-Amz-Trailer: x-amz-checksum-crc32
X-Forwarded-For: xxxx
X-Forwarded-Proto: https
Additional context
I believe this is caused due to go net module upgrade. Uploads work fine on 2025.5.0.
New go net module sets content-length:0 if NoBody is set: golang/go@450f3f6#diff-a50801014d4fc5373ebbddfedea688dbf90f9c68542b6d62b3ee4b2fa641e986R72
This is set by cloudflared in connection/quic_connection.go:370 , (!isWebsocket && !isTransferEncodingChunked(req) && req.ContentLength == 0)
. When content-length is unset, it defaults to 0, and AWS uses content-encoding: aws-chunked
instead of transfer-encoding
, causing cloudflare to set NoBody, causing go net to explicitly set content-length: 0
. Also checking for content-encoding
seems to fix the issue. Will be creating a pull request. sh4dowb@7705a80