S3 ListObjectsV2Pages Collation

[kudla]

Hi guys. Please for the ListObjectsV2Pages (hopefully the same as for ListObjectsV2) what is the actuall ordering symbols collation. It turned out it even differs from the one AWS S3 Console lists the objects. It is for sure not the same for underscore _ symbol. It goes before and after alphanumerics for Console and SDK respectively.
E.g.

# AWS S3 Console
_abc
abc

# AWS S3 go sdk
abc
_abc

Still what would be the symbols that goes just before alphanumerics still begin visible (it really fits for whitespace yet the nonblank readable one is needed)

And looking forward would actually github.com/aws/aws-sdk-go-v2 behave the same way in this regard.

Thank you

[Madrigal]

From the S3 documentation

Sorting order of returned objects

• General purpose bucket - For general purpose buckets, ListObjectsV2 returns objects in lexicographical order based on their key names.
• Directory bucket - For directory buckets, ListObjectsV2 does not return objects in lexicographical order.

This should be the way SDKs behave listing objects. We don't want to add any sorting behavior on top of what S3 offers.

I can confirm that this behaves listing underscore _ before alphanumeric, consistent with documentation

package main

import (
    "context"
    "log"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {
    ctx := context.Background()
    cfg, err := config.LoadDefaultConfig(ctx,
        config.WithRegion("us-west-2"),
    )
    if err != nil {
        log.Fatal(err)
    }

    client := s3.NewFromConfig(cfg)
    input := &s3.ListObjectsV2Input{
        Bucket: aws.String("some-bucket"),
    }

    result, err := client.ListObjectsV2(ctx, input)
    if err != nil {
        log.Fatal(err)
    }

    for _, object := range result.Contents {
        log.Printf("Key: %s, Size: %d\n", *object.Key, object.Size)
    }
}

even on a sample single Go v1 SDK this is listing _ before alphanumeric

package main

import (
    "fmt"
    "log"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

func main() {
    sess, err := session.NewSession(&aws.Config{
        Region: aws.String("us-west-2"),
    })
    if err != nil {
        log.Fatal(err)
    }

    svc := s3.New(sess)

    input := &s3.ListObjectsV2Input{
        Bucket: aws.String("lmadrig-test-bucket"),
    }

    result, err := svc.ListObjectsV2(input)
    if err != nil {
        log.Fatal(err)
    }
    
    for _, object := range result.Contents {
        fmt.Printf("Key: %s, Size: %d\n", *object.Key, *object.Size)
    }
}

[kudla]

@Madrigal thank you. I'll double check. Maybe I just failed organizing my debug flow.

[kudla]

I've double checked.

Actually here's simplified code example

import(
	"github.com/aws/aws-sdk-go/service/s3"
        ...
)

...
err := s3Client.ListObjectsV2PagesWithContext(
	ctx,
	&s3.ListObjectsV2Input{
		Bucket:  utils.Ptr(bucket),
		Prefix:  utils.Ptr(fmt.Sprintf("%s/", filepath.Join(keyPrefix))),
		MaxKeys: utils.Ptr(int64(maxKeys)),
	},
	func(page *s3.ListObjectsV2Output, exhausted bool) bool {
		for _, object := range page.Contents {
			fmt.Println(*object.Key)
		}
		return !exhausted
	},
)

The bucket is as well at us-west-2
And the output really differs for the and _ characters. Here're the real outputs

wal/2025-05.19/000000020000000000000037.br
wal/2025-05.19/000000020000000000000038.br
wal/2025-05.19/000000020000000000000039.br
wal/2025-05.19/00000002000000000000003A.00000028.backup.br
wal/2025-05.19/00000002000000000000003A.br
wal/2025-05.19/_2546347877831__s3-index.json

wal/2025-05.19/ 2546347276823__s3-index.json
wal/2025-05.19/000000020000000000000037.br
wal/2025-05.19/000000020000000000000038.br
wal/2025-05.19/000000020000000000000039.br
wal/2025-05.19/00000002000000000000003A.00000028.backup.br
wal/2025-05.19/00000002000000000000003A.br

The main prerequisite differences are sdk versions github.com/aws/aws-sdk-go vs github.com/aws/aws-sdk-go-v2 and the method being called ListObjectsV2 vs paged one ListObjectsV2PagesWithContext. I'm not sure should these really matter anything.