feat!: Added import table feature and AWS provider v5 upgrade (#77)

Co-authored-by: magreenbaum <magreenbaum>

Author: magreenbaum

README.md

@@ -53,20 +53,21 @@ Users of Terragrunt can achieve similar results by using modules provided in the
 - [Basic example](https://github.com/terraform-aws-modules/terraform-aws-dynamodb-table/tree/master/examples/basic)
 - [Autoscaling example](https://github.com/terraform-aws-modules/terraform-aws-dynamodb-table/tree/master/examples/autoscaling)
 - [Global tables example](https://github.com/terraform-aws-modules/terraform-aws-dynamodb-table/tree/master/examples/global-tables)
+- [S3 import examples](https://github.com/terraform-aws-modules/terraform-aws-dynamodb-table/tree/master/examples/s3-import)
 
 <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
 ## Requirements
 
 | Name | Version |
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
-| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.59 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.21 |
 
 ## Providers
 
 | Name | Version |
 |------|---------|
-| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.59 |
+| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.21 |
 
 ## Modules
 
@@ -104,6 +105,7 @@ No modules.
 | <a name="input_global_secondary_indexes"></a> [global\_secondary\_indexes](#input\_global\_secondary\_indexes) | Describe a GSI for the table; subject to the normal limits on the number of GSIs, projected attributes, etc. | `any` | `[]` | no |
 | <a name="input_hash_key"></a> [hash\_key](#input\_hash\_key) | The attribute to use as the hash (partition) key. Must also be defined as an attribute | `string` | `null` | no |
 | <a name="input_ignore_changes_global_secondary_index"></a> [ignore\_changes\_global\_secondary\_index](#input\_ignore\_changes\_global\_secondary\_index) | Whether to ignore changes lifecycle to global secondary indices, useful for provisioned tables with scaling | `bool` | `false` | no |
+| <a name="input_import_table"></a> [import\_table](#input\_import\_table) | Configurations for importing s3 data into a new table. | `any` | `{}` | no |
 | <a name="input_local_secondary_indexes"></a> [local\_secondary\_indexes](#input\_local\_secondary\_indexes) | Describe an LSI on the table; these can only be allocated at creation so you cannot change this definition after you have created the resource. | `any` | `[]` | no |
 | <a name="input_name"></a> [name](#input\_name) | Name of the DynamoDB table | `string` | `null` | no |
 | <a name="input_point_in_time_recovery_enabled"></a> [point\_in\_time\_recovery\_enabled](#input\_point\_in\_time\_recovery\_enabled) | Whether to enable point-in-time recovery | `bool` | `false` | no |

examples/autoscaling/README.md

@@ -20,7 +20,7 @@ Note that this example may create resources which can cost money (AWS Elastic IP
 | Name | Version |
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
-| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.69 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.21 |
 | <a name="requirement_random"></a> [random](#requirement\_random) | >= 2.0 |
 
 ## Providers

examples/autoscaling/versions.tf

@@ -4,7 +4,7 @@ terraform {
   required_providers {
     aws = {
       source  = "hashicorp/aws"
-      version = ">= 3.69"
+      version = ">= 5.21"
     }
     random = {
       source  = "hashicorp/random"

examples/basic/README.md

@@ -20,7 +20,7 @@ Note that this example may create resources which can cost money (AWS Elastic IP
 | Name | Version |
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
-| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 3.69 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.21 |
 | <a name="requirement_random"></a> [random](#requirement\_random) | >= 2.0 |
 
 ## Providers

examples/basic/versions.tf

@@ -4,7 +4,7 @@ terraform {
   required_providers {
     aws = {
       source  = "hashicorp/aws"
-      version = ">= 3.69"
+      version = ">= 5.21"
     }
     random = {
       source  = "hashicorp/random"

examples/global-tables/README.md

@@ -20,15 +20,15 @@ Note that this example may create resources which can cost money (AWS Elastic IP
 | Name | Version |
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
-| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.23 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.21 |
 | <a name="requirement_random"></a> [random](#requirement\_random) | >= 2.0 |
 
 ## Providers
 
 | Name | Version |
 |------|---------|
-| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.23 |
-| <a name="provider_aws.euwest2"></a> [aws.euwest2](#provider\_aws.euwest2) | >= 4.23 |
+| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 5.21 |
+| <a name="provider_aws.euwest2"></a> [aws.euwest2](#provider\_aws.euwest2) | >= 5.21 |
 | <a name="provider_random"></a> [random](#provider\_random) | >= 2.0 |
 
 ## Modules

examples/global-tables/versions.tf

@@ -4,7 +4,7 @@ terraform {
   required_providers {
     aws = {
       source  = "hashicorp/aws"
-      version = ">= 4.23"
+      version = ">= 5.21"
     }
     random = {
       source  = "hashicorp/random"

examples/s3-import/README.md

@@ -0,0 +1,64 @@
+# DynamoDB Table s3 import example
+
+Configuration in this directory creates an AWS DynamoDB table created from s3 imports (both json and csv examples).
+
+## Usage
+
+To run this example you need to execute:
+
+```bash
+$ terraform init
+$ terraform plan
+$ terraform apply
+```
+
+Note that this example may create resources which can cost money (AWS Elastic IP, for example). Run `terraform destroy` when you don't need these resources.
+
+<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+## Requirements
+
+| Name | Version |
+|------|---------|
+| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
+| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.21 |
+| <a name="requirement_random"></a> [random](#requirement\_random) | >= 2.0 |
+
+## Providers
+
+| Name | Version |
+|------|---------|
+| <a name="provider_random"></a> [random](#provider\_random) | >= 2.0 |
+
+## Modules
+
+| Name | Source | Version |
+|------|--------|---------|
+| <a name="module_import_csv_table"></a> [import\_csv\_table](#module\_import\_csv\_table) | ../../ | n/a |
+| <a name="module_import_json_table"></a> [import\_json\_table](#module\_import\_json\_table) | ../../ | n/a |
+| <a name="module_s3_bucket"></a> [s3\_bucket](#module\_s3\_bucket) | terraform-aws-modules/s3-bucket/aws | ~> 3.15 |
+| <a name="module_s3_import_object_csv"></a> [s3\_import\_object\_csv](#module\_s3\_import\_object\_csv) | terraform-aws-modules/s3-bucket/aws//modules/object | ~> 3.15 |
+| <a name="module_s3_import_object_json"></a> [s3\_import\_object\_json](#module\_s3\_import\_object\_json) | terraform-aws-modules/s3-bucket/aws//modules/object | ~> 3.15 |
+
+## Resources
+
+| Name | Type |
+|------|------|
+| [random_pet.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
+
+## Inputs
+
+No inputs.
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| <a name="output_import_csv_table_arn"></a> [import\_csv\_table\_arn](#output\_import\_csv\_table\_arn) | ARN of the DynamoDB table |
+| <a name="output_import_csv_table_id"></a> [import\_csv\_table\_id](#output\_import\_csv\_table\_id) | ID of the DynamoDB table |
+| <a name="output_import_csv_table_stream_arn"></a> [import\_csv\_table\_stream\_arn](#output\_import\_csv\_table\_stream\_arn) | The ARN of the Table Stream. Only available when var.stream\_enabled is true |
+| <a name="output_import_csv_table_stream_label"></a> [import\_csv\_table\_stream\_label](#output\_import\_csv\_table\_stream\_label) | A timestamp, in ISO 8601 format of the Table Stream. Only available when var.stream\_enabled is true |
+| <a name="output_import_json_table_arn"></a> [import\_json\_table\_arn](#output\_import\_json\_table\_arn) | ARN of the DynamoDB table |
+| <a name="output_import_json_table_id"></a> [import\_json\_table\_id](#output\_import\_json\_table\_id) | ID of the DynamoDB table |
+| <a name="output_import_json_table_stream_arn"></a> [import\_json\_table\_stream\_arn](#output\_import\_json\_table\_stream\_arn) | The ARN of the Table Stream. Only available when var.stream\_enabled is true |
+| <a name="output_import_json_table_stream_label"></a> [import\_json\_table\_stream\_label](#output\_import\_json\_table\_stream\_label) | A timestamp, in ISO 8601 format of the Table Stream. Only available when var.stream\_enabled is true |
+<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

examples/s3-import/files/sample.csv

@@ -0,0 +1,3 @@
+id;title
+02;csv
+03;csvtest

examples/s3-import/files/sample.json

@@ -0,0 +1,11 @@
+{"Item":
+{
+  "id":{"N":"01"},
+  "title":{"S":"example1"}
+}}
+
+{"Item":
+{
+  "id":{"N":"02"},
+  "title":{"S":"example2"}
+}}

examples/s3-import/main.tf

@@ -0,0 +1,107 @@
+provider "aws" {
+  region = "eu-west-1"
+}
+
+resource "random_pet" "this" {
+  length = 2
+}
+
+module "import_json_table" {
+  source = "../../"
+
+  name                        = "import-json-${random_pet.this.id}"
+  hash_key                    = "id"
+  range_key                   = "title"
+  table_class                 = "STANDARD"
+  deletion_protection_enabled = false
+
+  attributes = [
+    {
+      name = "id"
+      type = "N"
+    },
+    {
+      name = "title"
+      type = "S"
+    },
+  ]
+
+  import_table = {
+    input_format           = "DYNAMODB_JSON"
+    input_compression_type = "NONE"
+    bucket                 = module.s3_bucket.s3_bucket_id
+    key_prefix             = "import-json-${random_pet.this.id}"
+  }
+
+  tags = {
+    Terraform   = "true"
+    Environment = "staging"
+  }
+}
+
+module "import_csv_table" {
+  source = "../../"
+
+  name                        = "import-csv-${random_pet.this.id}"
+  hash_key                    = "id"
+  range_key                   = "title"
+  table_class                 = "STANDARD"
+  deletion_protection_enabled = false
+
+  attributes = [
+    {
+      name = "id"
+      type = "N"
+    },
+    {
+      name = "title"
+      type = "S"
+    },
+  ]
+
+  import_table = {
+    input_format           = "CSV"
+    input_compression_type = "NONE"
+    bucket                 = module.s3_bucket.s3_bucket_id
+    key_prefix             = "import-csv-${random_pet.this.id}"
+    input_format_options = {
+      csv = {
+        delimiter = ";"
+      }
+    }
+  }
+
+  tags = {
+    Terraform   = "true"
+    Environment = "staging"
+  }
+}
+
+module "s3_bucket" {
+  source  = "terraform-aws-modules/s3-bucket/aws"
+  version = "~> 3.15"
+
+  bucket = "import-example-${random_pet.this.id}"
+
+  force_destroy = true
+}
+
+module "s3_import_object_json" {
+  source  = "terraform-aws-modules/s3-bucket/aws//modules/object"
+  version = "~> 3.15"
+
+  bucket = module.s3_bucket.s3_bucket_id
+  key    = "import-json-${random_pet.this.id}/sample.json"
+
+  content_base64 = filebase64("./files/sample.json")
+}
+
+module "s3_import_object_csv" {
+  source  = "terraform-aws-modules/s3-bucket/aws//modules/object"
+  version = "~> 3.15"
+
+  bucket = module.s3_bucket.s3_bucket_id
+  key    = "import-csv-${random_pet.this.id}/sample.csv"
+
+  content_base64 = filebase64("./files/sample.csv")
+}

examples/s3-import/outputs.tf

@@ -0,0 +1,39 @@
+output "import_json_table_arn" {
+  description = "ARN of the DynamoDB table"
+  value       = module.import_json_table.dynamodb_table_arn
+}
+
+output "import_json_table_id" {
+  description = "ID of the DynamoDB table"
+  value       = module.import_json_table.dynamodb_table_id
+}
+
+output "import_json_table_stream_arn" {
+  description = "The ARN of the Table Stream. Only available when var.stream_enabled is true"
+  value       = module.import_json_table.dynamodb_table_stream_arn
+}
+
+output "import_json_table_stream_label" {
+  description = "A timestamp, in ISO 8601 format of the Table Stream. Only available when var.stream_enabled is true"
+  value       = module.import_json_table.dynamodb_table_stream_label
+}
+
+output "import_csv_table_arn" {
+  description = "ARN of the DynamoDB table"
+  value       = module.import_csv_table.dynamodb_table_arn
+}
+
+output "import_csv_table_id" {
+  description = "ID of the DynamoDB table"
+  value       = module.import_csv_table.dynamodb_table_id
+}
+
+output "import_csv_table_stream_arn" {
+  description = "The ARN of the Table Stream. Only available when var.stream_enabled is true"
+  value       = module.import_csv_table.dynamodb_table_stream_arn
+}
+
+output "import_csv_table_stream_label" {
+  description = "A timestamp, in ISO 8601 format of the Table Stream. Only available when var.stream_enabled is true"
+  value       = module.import_csv_table.dynamodb_table_stream_label
+}

examples/s3-import/variables.tf

@@ -0,0 +1,14 @@
+terraform {
+  required_version = ">= 1.0"
+
+  required_providers {
+    aws = {
+      source  = "hashicorp/aws"
+      version = ">= 5.21"
+    }
+    random = {
+      source  = "hashicorp/random"
+      version = ">= 2.0"
+    }
+  }
+}