Skip to content

AWS: KeyManagementClient implementation that works with AWS KMS #13136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

szlta
Copy link
Contributor

@szlta szlta commented May 23, 2025

To be used in table encryption for:

  • wrapping/unwrapping encryption keys
  • generating data keys (available specs: AES_256, AES_128)

Added integration test for verification.

@szlta
Copy link
Contributor Author

szlta commented May 23, 2025

cc: @amogh-jahagirdar

@jbonofre
Copy link
Member

@nastra @Fokko ^^

@jbonofre
Copy link
Member

I will take a look as well. Thanks !

@amogh-jahagirdar amogh-jahagirdar self-requested a review June 8, 2025 02:19
@amogh-jahagirdar
Copy link
Contributor

Sorry for the delay, I'm taking a look

Copy link
Contributor

@amogh-jahagirdar amogh-jahagirdar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still need to take a pass over tests but left some comments!

* encrypting/decrypting keys with a KMS-managed master key, (by referencing its key ID), and for
* the generation of new encryption keys.
*/
public class AwsKeyManagementClient implements KeyManagementClient {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will need to be serializable so that it can be used in a distributed setting (since an EncryptionManager instance can also be serialized, and generally we'd expect an EncryptionManager to also contain an instance of KeyManagementClient

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah KeyManagementClient does implement Serializable as is (though looking at the implementation, wonder how necessary that is - it's transient in the StandardEncryptionmanager both now and after #7770, cc @ggershinsky)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @smaheshwar-pltr points it out the class implements Serializable by interface but I also have doubts whether that's actually needed. KMS interactions for table encryption only concerns the master key which I'd expect to take place on driver / coordinator nodes, encrypting/decrypting the metadata json or manifest lists. Everything else down to the data files will not require a KMS.

@@ -24,7 +24,7 @@
import java.util.Map;

/** A minimum client interface to connect to a key management service (KMS). */
interface KeyManagementClient extends Serializable, Closeable {
public interface KeyManagementClient extends Serializable, Closeable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will necessarily need to be public so that implementations can reside in other modules, so I do agree with this change.

It'd be good if @rdblue can confirm that or if some other implementation pattern was intended here.

Copy link
Contributor

@smaheshwar-pltr smaheshwar-pltr Jun 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I believe that it being public lets catalogs / table operations in other packages use the KMS client - e.g. #13066 / #13225.

szlta added 2 commits June 10, 2025 10:56
To be used in table encryption for:
- wrapping/unwrapping encryption keys
- generating data keys (available specs: AES_256, AES_128)

Added integration test for verification.
}
}

@Test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we parameterize this with @ParameterizedTest?


assertThat(keyManagementClient.unwrapKey(result.wrappedKey(), keyId)).isEqualTo(result.key());
assertThat(result.key().limit())
.isEqualTo(DataKeySpec.AES_128.equals(dataKeySpec) ? 128 / 8 : 256 / 8);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this to a helper function expectedBytes(DataKeySpec spec)? I think it'd be easier to read that way

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants