Boost: Re-enable Module E2E tests #44141

LiamSarsfield · 2025-06-30T17:26:45Z

Addresses HOG-193: Enable Module E2E tests

Proposed changes:

This PR re-enables projects/plugins/boost/tests/e2e/specs/modules tests e2e test suite that was temporarily disabled in commit 0eb34d93ca7522549dcec7ea35c09d4f39ef5406 due to flakiness:

Re-enabled test suites:

Jetpack Boost - Critical CSS (specs/critical-css) - Boost: Re-enable Critical CSS and Concatenate E2E tests #44105
Jetpack Boost - Concatenate JS and CSS (specs/concatenate) - Boost: Re-enable Critical CSS and Concatenate E2E tests #44105
Jetpack Boost - Base (specs/base) - Boost: Re-enable Base/Common E2E tests #44140
Jetpack Boost - Modules (specs/modules) - This PR

Test suites still disabled (to be addressed in separate PRs):

Jetpack Boost - Page Cache (specs/page-cache)
Jetpack Boost - Image CDN (specs/image-cdn)
Jetpack Boost - Image Guide (specs/image-guide)

Jetpack product discussion

N/A

Does this pull request change what data or activity we track or use?

No

Other information:

Have you written new tests for your changes, if applicable?
Have you checked the E2E test CI results, and verified that your changes do not break them?
Have you tested your changes on WordPress.com, if applicable (if so, you'll see a generated comment below with a script to run)?

Testing instructions:

From the projects/plugins/boost/tests/e2e directory:

Via ngrok (Recommended, but requires a subscription)

Replace $tunnel_url with your actual ngrok URL

Install E2E test dependencies:
```
pnpm install
```

Start the Docker environment and tunnel:

pnpm run env:up && TUNNEL_URL=$tunnel_url pnpm run tunnel:up

Start your ngrok tunnel

ngrok http --url=$tunnel_url http://localhost:8889/

Run the specific re-enabled test suites:

 TUNNEL_URL=$tunnel_url pnpm run test:run specs/modules

Via localtunnel (Not recommended as it may be slow)

Install E2E test dependencies:
```
pnpm install
```
Start the Docker environment and tunnel:
```
pnpm run env:up && pnpm run tunnel:up
```
Run the specific re-enabled test suites:
```
pnpm run test:run specs/modules
```

Expected behavior:

All Common/Base tests should pass without skipping or timeout errors
Tests should pass consistently when run multiple times

Verification steps:

Check that tests are running in CI:
- The re-enabled test suites should appear in the E2E test matrix
- CI should execute "Jetpack Boost - Modules"
Test stability:
- Run each test suite 3-5 times to ensure consistency
- No intermittent failures or timing-related issues should occur

github-actions · 2025-06-30T17:27:53Z

Thank you for your PR!

When contributing to Jetpack, we have a few suggestions that can help us test and review your patch:

✅ Include a description of your PR changes.
✅ Add a "[Status]" label (In Progress, Needs Review, ...).
✅ Add a "[Type]" label (Bug, Enhancement, Janitorial, Task).
✅ Add testing instructions.
✅ Specify whether this PR includes any changes to data or privacy.
✅ Add changelog entries to affected projects

This comment will be updated as you work on your PR and make changes. If you think that some of those checks are not needed for your PR, please explain why you think so. Thanks for cooperation 🤖

Follow this PR Review Process:

Ensure all required checks appearing at the bottom of this PR are passing.
Make sure to test your changes on all platforms that it applies to. You're responsible for the quality of the code you ship.
You can use GitHub's Reviewers functionality to request a review.
When it's reviewed and merged, you will be pinged in Slack to deploy the changes to WordPress.com simple once the build is done.

If you have questions about anything, reach out in #jetpack-developers for guidance!

Boost plugin:

No scheduled milestone found for this plugin.

If you have any questions about the release process, please ask in the #jetpack-releases channel on Slack.

jp-launch-control · 2025-06-30T17:30:57Z

Code Coverage Summary

Coverage changed in 1 file.

File	Coverage	Δ%	Δ Uncovered
projects/plugins/boost/app/assets/src/js/features/speed-score/speed-score.tsx	0/43 (0.00%)	0.00%	4 💔

Full summary · PHP report · JS report

_{Coverage check overridden by

I don't care about code coverage for this PR
Use this label to ignore the check for insufficient code coveage.
.}

LiamSarsfield · 2025-06-30T17:32:28Z

projects/plugins/boost/tests/e2e/specs/modules/modules-common.test.js

 import { test, expect } from '_jetpack-e2e-commons/fixtures/base-test.js';
 import playwrightConfig from '_jetpack-e2e-commons/playwright.config.mjs';
 import { boostPrerequisitesBuilder } from '../../lib/env/prerequisites.js';
 import { JetpackBoostPage } from '../../lib/pages/index.js';

 const modules = [
 	// ['MODULE_NAME', 'DEFAULT STATE'],
-	[ 'critical_css', 'disabled' ],
+	[ 'critical_css', 'enabled' ],


This should be enabled. We now enable local Critical CSS by default when the user selects a free plan.

LiamSarsfield · 2025-07-03T17:17:33Z

projects/plugins/boost/tests/e2e/lib/pages/wp-admin/JetpackBoostPage.js

@@ -3,7 +3,6 @@ import WpPage from '_jetpack-e2e-commons/pages/wp-page.js';

 const apiEndpointsRegex = {
 	'modules-state': /jetpack-boost-ds\/modules-state\/set/,
-	connection: /jetpack-boost\/v1\/connection/,


This was never used.

LiamSarsfield · 2025-07-07T13:25:40Z

projects/plugins/boost/tests/e2e/specs/modules/speed-score.test.js

 	} );

-	test.beforeEach( async function ( { page } ) {
+	test.beforeEach( async () => {


This beforeEach simply just visits the Boost page, the fact that this times out tells me there is a timeout issue with localtunnel, a retry resolved this issue.
specs/modules/speed-score.test.js:26:2 › Speed Score feature › The Speed Score section should display a mobile and desktop speed score greater than zero Test timeout of 300000ms exceeded while running "beforeEach" hook. 20 | } ); 21 | > 22 | test.beforeEach( async () => { | ^ 23 | jetpackBoostPage = await JetpackBoostPage.visit( page ); 24 | } ); 25 | at /home/runner/work/jetpack/jetpack/projects/plugins/boost/tests/e2e/specs/modules/speed-score.test.js:22:7

…e-tests-hog-193

github-actions bot added [Plugin] Boost A feature to speed up the site and improve performance. [Status] In Progress [Tests] Includes Tests Actions GitHub actions used to automate some of the work around releases and repository management E2E Tests labels Jun 30, 2025

github-actions bot added the [Status] Needs Author Reply We need more details from you. This label will be auto-added until the PR meets all requirements. label Jun 30, 2025

LiamSarsfield added the [Type] Janitorial label Jun 30, 2025

LiamSarsfield requested review from kraftbj, dilirity and a team June 30, 2025 17:30

LiamSarsfield removed the [Status] Needs Author Reply We need more details from you. This label will be auto-added until the PR meets all requirements. label Jun 30, 2025

LiamSarsfield commented Jun 30, 2025

View reviewed changes

LiamSarsfield marked this pull request as ready for review July 1, 2025 19:57

LiamSarsfield marked this pull request as draft July 1, 2025 20:30

Base automatically changed from fix/boost/common-e2e-tests-hog-192 to trunk July 2, 2025 13:17

LiamSarsfield added the I don't care about code coverage for this PR Use this label to ignore the check for insufficient code coveage. label Jul 2, 2025

LiamSarsfield commented Jul 3, 2025

View reviewed changes

github-actions bot added the Docker label Jul 7, 2025

LiamSarsfield changed the base branch from trunk to update/boost/e2e/mock-unrelated-testing-hog-200 July 7, 2025 11:22

LiamSarsfield mentioned this pull request Jul 7, 2025

Boost: Re-enable Page Cache, Image Guide, Image CDN E2E tests #44210

Merged

3 tasks

LiamSarsfield commented Jul 7, 2025

View reviewed changes

LiamSarsfield marked this pull request as ready for review July 7, 2025 13:26

Base automatically changed from update/boost/e2e/mock-unrelated-testing-hog-200 to trunk July 7, 2025 20:40

Re-introduce Boost Module E2E tests

3da2a49

LiamSarsfield force-pushed the fix/boost/modules-e2e-tests-hog-193 branch from 6890e54 to 3da2a49 Compare July 8, 2025 10:57

LiamSarsfield and others added 2 commits July 8, 2025 11:58

Merge remote-tracking branch 'origin/trunk' into fix/boost/modules-e2…

209517b

…e-tests-hog-193

Merge branch 'trunk' into fix/boost/modules-e2e-tests-hog-193

7054ab5

kraftbj approved these changes Jul 8, 2025

View reviewed changes

LiamSarsfield merged commit f5f4521 into trunk Jul 9, 2025
68 checks passed

LiamSarsfield deleted the fix/boost/modules-e2e-tests-hog-193 branch July 9, 2025 08:52

github-actions bot removed the [Status] In Progress label Jul 9, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Boost: Re-enable Module E2E tests #44141

Boost: Re-enable Module E2E tests #44141

Uh oh!

LiamSarsfield commented Jun 30, 2025 •

edited by kraftbj

Loading

Uh oh!

github-actions bot commented Jun 30, 2025 •

edited

Loading

Uh oh!

jp-launch-control bot commented Jun 30, 2025 •

edited

Loading

Uh oh!

LiamSarsfield Jun 30, 2025

Uh oh!

LiamSarsfield Jul 3, 2025

Uh oh!

LiamSarsfield Jul 7, 2025

Uh oh!

Uh oh!

Uh oh!

Boost: Re-enable Module E2E tests #44141

Boost: Re-enable Module E2E tests #44141

Uh oh!

Conversation

LiamSarsfield commented Jun 30, 2025 • edited by kraftbj Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes:

Re-enabled test suites:

Test suites still disabled (to be addressed in separate PRs):

Jetpack product discussion

Does this pull request change what data or activity we track or use?

Other information:

Testing instructions:

Via ngrok (Recommended, but requires a subscription)

Via localtunnel (Not recommended as it may be slow)

Expected behavior:

Verification steps:

Uh oh!

github-actions bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jp-launch-control bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Summary

Uh oh!

LiamSarsfield Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

LiamSarsfield Jul 3, 2025

Choose a reason for hiding this comment

Uh oh!

LiamSarsfield Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

LiamSarsfield commented Jun 30, 2025 •

edited by kraftbj

Loading

github-actions bot commented Jun 30, 2025 •

edited

Loading

jp-launch-control bot commented Jun 30, 2025 •

edited

Loading