Skip to content

Bug: S3FileSigner does not respect core-site.xml #721

Open
@Benno49

Description

@Benno49

Configuring aws endpoint or credentials in core-site.xml does not effect S3FileSigner.
Line 57 CloudFileSighner.scala:

  private val s3Client = ReflectionUtils.newInstance(classOf[DefaultS3ClientFactory], conf)
    .createS3Client(name, new S3ClientCreationParameters())

Parameters get initialized (new S3ClientCreationParameters()) but never configured. Compare to bindAWSClient in S3AFileSystem.java:

parameters = new S3ClientFactory.S3ClientCreationParameters()
        .withCredentialSet(credentials)
        .withEndpoint(endpoint)
        .withMetrics(statisticsContext.newStatisticsFromAwsSdk())
        .withPathStyleAccess(conf.getBoolean(PATH_STYLE_ACCESS, false))
        .withUserAgentSuffix(uaSuffix)
        .withRequestHandlers(auditManager.createRequestHandlers());

    s3 = ReflectionUtils.newInstance(s3ClientFactoryClass, conf)
        .createS3Client(getUri(),
            parameters);

I am not sure why the s3 client of the FileSystem is not used here anyway:

    fs match {
      case _: S3AFileSystem =>
        new S3FileSigner(dataPath, conf, preSignedUrlTimeoutSeconds)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions