Skip to content

[TT-15111] probe AI PR reviewers #7209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 9, 2025
Merged

Conversation

lghiur
Copy link
Collaborator

@lghiur lghiur commented Jul 8, 2025

User description

TT-15216
Summary JIRA lint GH actions upgrades
Type Story Story
Status In Dev
Points N/A
Labels -

Addresses part of https://tyktech.atlassian.net/browse/TT-15216

Adds the following AI personas for reviewes:

  • connectivity reviewer
  • dependency reviewer
  • performance reviewer
  • security reviewer

Related Issue

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactoring or add test (improvements in base code or adds test coverage to functionality)

Checklist

  • I ensured that the documentation is up to date
  • I explained why this PR updates go.mod in detail with reasoning why it's required
  • I would like a code coverage CI quality gate exception and have explained why

PR Type

Enhancement, Documentation


Description

  • Adds automated dependency impact reviewer for PRs to master branch

  • Introduces manual and automatic connectivity and dependency review jobs

  • Provides detailed prompt for dependency impact review process

  • Enhances workflow to ensure cross-project compatibility checks


Changes diagram

flowchart LR
  A["PR Opened on master"] -- "Triggers" --> B["auto_connectivity_reviewer"]
  A -- "Triggers" --> C["auto_dependency_impact_reviewer"]
  D["Manual issue_comment"] -- "/connectivity-reviewer" --> E["connectivity_reviewer"]
  D -- "/dependency-impact-reviewer" --> F["dependency_impact_reviewer"]
  F -- "Uses" --> G["dependency-impact-reviewer.md prompt"]
  C -- "Uses" --> G
Loading

Changes walkthrough 📝

Relevant files
Enhancement
probe-connectivity-reviewer.yml
Add PR and comment-based connectivity and dependency review jobs

.github/workflows/probe-connectivity-reviewer.yml

  • Adds PR-based triggers for connectivity and dependency impact
    reviewers
  • Introduces separate jobs for manual and automatic review flows
  • Configures secrets and prompts for new jobs
  • Expands workflow to cover both connectivity and dependency impact
  • +48/-3   
    Documentation
    dependency-impact-reviewer.md
    Add detailed dependency impact reviewer prompt and guidelines

    .probe/dependency-impact-reviewer.md

  • Adds comprehensive prompt for dependency impact reviewer
  • Details cross-project impact validation categories and checks
  • Specifies response format and actionable review guidelines
  • Ensures reviewers assess compatibility and required downstream updates
  • +123/-0 

    Need help?
  • Type /help how to ... in the comments thread for any questions about PR-Agent usage.
  • Check out the documentation for more information.
  • @buger
    Copy link
    Member

    buger commented Jul 8, 2025

    Knock Knock! 🔍

    Just thought I'd let you know that your PR title and story title look quite different. PR titles that closely resemble the story title make it easier for reviewers to understand the context of the PR.

    An easy-to-understand PR title a day makes the reviewer review away! 😛⚡️
    Story Title JIRA lint GH actions upgrades
    PR Title added dependency impact reviewer

    Check out this guide to learn more about PR best-practices.

    Copy link
    Contributor

    github-actions bot commented Jul 8, 2025

    API Changes

    --- prev.txt	2025-07-09 13:30:23.637678973 +0000
    +++ current.txt	2025-07-09 13:30:14.300525759 +0000
    @@ -13194,7 +13194,6 @@
     	SyncStats func(Stat)
     	// Duration in which connection stats will be flushed. Defaults to one second.
     	StatsSyncInterval time.Duration
    -
     	// Has unexported fields.
     }
     
    @@ -13204,12 +13203,6 @@
     
     func (p *Proxy) Serve(l net.Listener) error
     
    -func (p *Proxy) SetShutdownContext(ctx context.Context)
    -    SetShutdownContext sets the shutdown context from the caller
    -
    -func (p *Proxy) Shutdown(ctx context.Context) error
    -    Shutdown initiates graceful shutdown and waits for all connections to finish
    -
     func (p *Proxy) Swap(new *Proxy)
     
     type Stat struct {

    Copy link
    Contributor

    github-actions bot commented Jul 8, 2025

    PR Code Suggestions ✨

    Explore these optional code suggestions:

    CategorySuggestion                                                                                                                                    Impact
    Possible issue
    Fix typo in secret environment variable

    The secret reference ${{ secrets.PROBE_ANTHROPIC_API_URL }} is inconsistent with the
    other secret names, which use PROBE_ANTHROPIC_URL. This typo will cause the workflow
    to fail due to a missing secret.

    .github/workflows/probe-connectivity-reviewer.yml [52-56]

    -+    if: github.event_name == 'pull_request'
    -+    uses: buger/probe/.github/workflows/probe.yml@main
    -+    with:
    -+      command_prefix: "auto-connectivity-review"
    -+      allow_edit: true
    -+      prompt: ./.probe/connectivity-reviewer.md
    -+    secrets:
     +      ANTHROPIC_API_KEY: ${{ secrets.PROBE_ANTHROPIC_API_KEY }}
    -+      ANTHROPIC_API_URL: ${{ secrets.PROBE_ANTHROPIC_API_URL }}
    ++      ANTHROPIC_API_URL: ${{ secrets.PROBE_ANTHROPIC_URL }}
     +      APP_ID: ${{ secrets.PROBE_APP_ID }}
     +      APP_PRIVATE_KEY: ${{ secrets.PROBE_APP_PRIVATE_KEY }}
    Suggestion importance[1-10]: 9

    __

    Why: This suggestion identifies a critical typo in the secret name that would cause the workflow to fail at runtime due to a missing secret. Correcting this is essential for the workflow's functionality.

    High
    Correct secret name for API URL

    The secret reference ${{ secrets.PROBE_ANTHROPIC_API_URL }} is inconsistent with the
    other secret names, which use PROBE_ANTHROPIC_URL. This typo will cause the workflow
    to fail due to a missing secret.

    .github/workflows/probe-connectivity-reviewer.yml [62-65]

    -+    if: github.event_name == 'pull_request'
    -+    uses: buger/probe/.github/workflows/probe.yml@main
    -+    with:
    -+      command_prefix: "auto-dependency-impact-review"
    -+      allow_edit: true
    -+      prompt: ./.probe/dependency-impact-reviewer.md
    -+    secrets:
     +      ANTHROPIC_API_KEY: ${{ secrets.PROBE_ANTHROPIC_API_KEY }}
    -+      ANTHROPIC_API_URL: ${{ secrets.PROBE_ANTHROPIC_API_URL }}
    ++      ANTHROPIC_API_URL: ${{ secrets.PROBE_ANTHROPIC_URL }}
     +      APP_ID: ${{ secrets.PROBE_APP_ID }}
    Suggestion importance[1-10]: 9

    __

    Why: The suggestion addresses a critical typo in the secret name for the API URL, which would result in workflow failures. Fixing this is vital for the workflow to function as intended.

    High
    Ensure workflow triggers on all main branches

    The branches filter under the pull_request trigger will only run the workflow for
    PRs targeting the master branch. If your repository uses main or other branch names,
    or if you want to support multiple branches, this could cause the workflow to not
    trigger as intended. Ensure this is the desired behavior.

    .github/workflows/probe-connectivity-reviewer.yml [8]

    -+    branches: [master]
    ++    branches: [master, main]
    Suggestion importance[1-10]: 7

    __

    Why: The suggestion highlights a potential issue where the workflow only triggers on the 'master' branch, which may not be the default branch for all repositories. Expanding to include 'main' (or other branches) improves reliability, but the suggestion only asks the user to verify/ensure the intended behavior, so the score is capped at 7.

    Medium

    Response Format
    ---------------

    Plain textANTLR4BashCC#CSSCoffeeScriptCMakeDartDjangoDockerEJSErlangGitGoGraphQLGroovyHTMLJavaJavaScriptJSONJSXKotlinLaTeXLessLuaMakefileMarkdownMATLABMarkupObjective-CPerlPHPPowerShell.propertiesProtocol BuffersPythonRRubySass (Sass)Sass (Scss)SchemeSQLShellSwiftSVGTSXTypeScriptWebAssemblyYAMLXML` ## Impact Assessment [Detailed analysis of how the changes in tyk might impact each downstream repository] ## Required Updates [Specific files or components in downstream repositories that need to be updated] ## Compatibility Concerns [Any potential backward compatibility issues that need to be addressed] ## Summary & Recommendations [Overall assessment and specific recommendations for ensuring cross-project compatibility] `
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Some mistake here?

    Copy link
    Collaborator Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    yes, it was a format error, fixed

    @lghiur lghiur changed the title added dependency impact reviewer [TT-15216] added dependency impact reviewer Jul 8, 2025
    lghiur added 2 commits July 9, 2025 11:23
    - Add GitHub workflows for connectivity, dependency, performance, and security reviews
    - Add probe configuration files for performance and security reviewers
    - Update dependency impact reviewer with standardized response format
    - Integrate with buger/probe for automated PR analysis and feedback
    @lghiur lghiur changed the title [TT-15216] added dependency impact reviewer [TT-15216] probe AI PR reviewers Jul 9, 2025

    jobs:
    connectivity_review:
    if: |
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Why you added this additional statement here?

    Copy link
    Collaborator Author

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    I wanted to prevent the job to run on every comment (I am under the impression that it might run even if the coment doesn't contain /probe or so?), I can get rid of github.event_name == 'pull_request' though

    @lghiur lghiur force-pushed the TT-15111-focused-code-reviewers branch from 186a793 to ff93818 Compare July 9, 2025 13:29
    @buger buger merged commit e548d4e into master Jul 9, 2025
    15 of 16 checks passed
    @buger buger deleted the TT-15111-focused-code-reviewers branch July 9, 2025 13:32
    Copy link

    sonarqubecloud bot commented Jul 9, 2025

    @lghiur lghiur changed the title [TT-15216] probe AI PR reviewers [TT-15111] probe AI PR reviewers Jul 14, 2025
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants