Misc dynamodb integration test improvements #1046
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… tests * DynamoDBLogStore: initial version * - Refactoring - Refactor and reformat - Add test cases - Update after review - Add DynamoDB tests - Fix test name - Adopt new tests - Add docker support for tests Ignore DynamoDB test because of CircleCI doesn't support testcontainers in docker * bugfixes * move to contribs * add tests * store temp files in .temp directory * scala style, method rename * revert .circleci changes, update dynamodb_logstore.py * make AWS credentials configurable * move example to contrib subdir * refactoring * add python integration test * add --jars option * change log levels * change the test skipping logic and adding logging * after-review changes * store tablePath in db instead of _delta_log path * add example invocation of integration test * move FailingDynamoDBLogStore to test package * refactor error injection * changes after review * store temp files inside of _delta_log directory * linting * fix docs * remove unnecessary logic from getTablePath, fix tests * minor fix * move to storage-dynamodb * storage-dynamodb project: add TODO, linting Co-authored-by: Mateusz Sabat <[email protected]> Co-authored-by: Lukasz_P <[email protected]> Co-authored-by: L-Po <[email protected]>
* fixed * update example command; add TODOs to build.sbt * fix order of repos argument in spark submit cmd
* initial refactoring * save work * clean up * use environmental variable for logstore selection * rename args
…_scala_feature_branch
* java refactor * auto table creation * auto-create dynamodb table * minor changes: - create dynamodb client in constructor - do not apply extra filtering in listFrom - inline comments - add docs
…_java_feature_branch
scottsand-db
reviewed
Apr 1, 2022
mrk-its
force-pushed
the
integration_test2
branch
from
8ada7fb to
c78f1e5
Compare
mrk-its
force-pushed
the
integration_test2
branch
from
c78f1e5 to
6365248
Compare
|
@mrk-its if this PR is ready for review can you remove the Draft status? |
done |
* minor cleanup * add print statement to dynamodb_logstore.py * add comment to build.sbt on how to run FailingDynamoDBLogStore * rename api conf keys * Update dynamodb_logstore.py
|
will review once conflicts are resolved |
- add run_id column - cleanup
mrk-its
force-pushed
the
integration_test2
branch
from
6365248 to
8407106
Compare
scottsand-db
changed the base branch from
dynamodb_logstore_java_feature_branch
to
master
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.