Skip to content

Move some testing from older dqi into newer dqi #966

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 48 commits into
base: main
Choose a base branch
from

Conversation

ivy-tuva
Copy link
Contributor

@ivy-tuva ivy-tuva commented Jul 9, 2025

Describe your changes

-Problem: dbt_expectations does not support Fabric. For every macro in dbt_expectations_tests, sql logic for Fabric was added if it was missing. This logic was written to mimic the dbt_expectations tests.

-Moved dbt_expectations_tests under tests/generic because it needs to be here or under macros in order to reference the tests in other files.

How has this been tested?

This was tested locally for Snowflake and BigQuery. Waiting on Redshift access and troubleshooting Fabric connection issues in dbt core.

Reviewer focus

Checklist before requesting a review

  • I have added at least one Github label to this PR (bug, enhancement, breaking change,...)
  • My code follows style guidelines
  • (New models) YAML files are categorized by sub folder and models listed in alphabetical order
  • (New models) I have added a config to each new model to enable it for claims and/or clinical data
  • (New models) I have added the variable tuva_last_run to the final output
  • (Optional) I have recorded a Loom to explain this PR

(Optional) Gif of how this PR makes you feel

Loom link

@ivy-tuva ivy-tuva added the enhancement New feature or request label Jul 9, 2025
@github-project-automation github-project-automation bot moved this to 👀 Ready for Review in The Tuva Project Jul 9, 2025
Copy link

SQLFluff Linting Failed

Issues were found that require manual correction or are unfixable by sqlfluff fix.
Please review the output below (or the full logs), fix the issues locally, and commit the changes.

SQLFluff Run Log - Mode: ci, Fix: false, Time: Fri Jul 11 06:41:33 UTC 2025

===== SQLFLUFF LINT (Initial Check) OUTPUT (2025-07-11_06:46:01) (Filtered) =====

=== [dbt templater] Sorting Nodes...
=== [dbt templater] Compiling dbt project...
=== [dbt templater] Project Compiled.
== [models/data_quality/data_quality__exploratory_charts.sql] FAIL
L: 789 | P:  12 | CP03 | Function names must be consistently lower case.
                       | [capitalisation.functions]
L: 789 | P:  17 | CP03 | Function names must be consistently lower case.
                       | [capitalisation.functions]
L: 789 | P:  23 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  32 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  37 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  56 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  59 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  63 | CP04 | Boolean/null literals must be lower case.
                       | [capitalisation.literals]
L: 789 | P:  68 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  87 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 790 | P:  10 | CP03 | Function names must be consistently lower case.
                       | [capitalisation.functions]
L: 790 | P:  17 | CP03 | Function names must be consistently lower case.
                       | [capitalisation.functions]
L: 790 | P:  23 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 800 | P:   5 | AM05 | Join clauses should be fully qualified. [ambiguous.join]
L: 800 | P:  53 | AL01 | Implicit/explicit aliasing of table. [aliasing.table]
WARNING    Skipped file /home/runner/work/tuva/tuva/tests/generic/dbt_expectations_tests.sql because it is a macro 
WARNING    Skipped file /home/runner/work/tuva/tuva/tests/generic/test_warn_if_null_percentage_above_zero.sql because it is a macro 
All Finished!

=========================================================================


===== SQLFLUFF FIX (Check, Informational) OUTPUT (2025-07-11_06:50:34) (Filtered) =====

==== finding fixable violations ====
=== [dbt templater] Sorting Nodes...
=== [dbt templater] Compiling dbt project...
=== [dbt templater] Project Compiled.
== [models/data_quality/data_quality__exploratory_charts.sql] FAIL
L: 789 | P:  12 | CP03 | Function names must be consistently lower case.
                       | [capitalisation.functions]
L: 789 | P:  17 | CP03 | Function names must be consistently lower case.
                       | [capitalisation.functions]
L: 789 | P:  23 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  32 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  37 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  56 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  59 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  63 | CP04 | Boolean/null literals must be lower case.
                       | [capitalisation.literals]
L: 789 | P:  68 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 789 | P:  87 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 790 | P:  10 | CP03 | Function names must be consistently lower case.
                       | [capitalisation.functions]
L: 790 | P:  17 | CP03 | Function names must be consistently lower case.
                       | [capitalisation.functions]
L: 790 | P:  23 | CP01 | Keywords must be lower case. [capitalisation.keywords]
L: 800 | P:   5 | AM05 | Join clauses should be fully qualified. [ambiguous.join]
L: 800 | P:  53 | AL01 | Implicit/explicit aliasing of table. [aliasing.table]
L: 800 | P:  53 | CP01 | Keywords must be lower case. [capitalisation.keywords]
== [models/data_quality/data_quality__exploratory_charts.sql] FIXED
WARNING    Skipped file /home/runner/work/tuva/tuva/tests/generic/dbt_expectations_tests.sql because it is a macro 
WARNING    Skipped file /home/runner/work/tuva/tuva/tests/generic/test_warn_if_null_percentage_above_zero.sql because it is a macro 
16 fixable linting violations found

=========================================================================

@ivy-tuva ivy-tuva requested a review from chase-jones July 14, 2025 18:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: 👀 Ready for Review
Development

Successfully merging this pull request may close these issues.

2 participants