23
23
24
24
Continuous Integration is important component of making Apache Beam robust and stable.
25
25
26
- Our execution environment for CI is mainly the Jenkins which is available at
27
- [ https://ci-beam.apache.org/ ] ( https://ci-beam.apache.org/ ) . See
28
- [ .test-infra/jenkins/README] ( .test-infra/jenkins/README.md )
29
- for trigger phrase, status and link of all Jenkins jobs. See Apache Beam Developer Guide for
30
- [ Jenkins Tips] ( https://cwiki.apache.org/confluence/display/BEAM/Jenkins+Tips ) .
31
-
32
- An additional execution environment for CI is [ GitHub Actions] ( https://github.com/features/actions ) . GitHub Actions
33
- (GA) are very well integrated with GitHub code and Workflow and it has evolved fast in 2019/2020 to become
34
- a fully-fledged CI environment, easy to use and develop for, so we decided to use it for building python source
35
- distribution and wheels.
26
+ Our execution environment for CI is the [ GitHub Actions] ( https://github.com/features/actions ) .
27
+ See [ .github/workflow/README] ( .github/workflow/README.md ) for trigger phrase,
28
+ status and link of all GHA jobs.
29
+
30
+ GitHub Actions (GHA) are very well integrated with GitHub code and Workflow and
31
+ it has evolved fast in 2019/2020 to become a fully-fledged CI environment, easy
32
+ to use and develop for, so we decided to use it firstly for a few workflows then
33
+ migrated all workflows previously run on Jenkins.
34
+
35
+ For this reason there are mainly two types of GHA workflows running
36
+ - Self-hosted runner GHAs. These were mifrated from Jenkins with workflow name
37
+ prefix (beam_ * .yml) as well as new workflows added following the same naming
38
+ convention, including PreCommit, PostCommit, LoadTest, PerformanceTest, and
39
+ several infrastructure jobs. See [ .github/workflow/README] ( .github/workflow/README.md )
40
+ for the complete list.
41
+ - GitHub-hosted runner GHAs. Most of these jobs exercises the workflow in different
42
+ OS (linux, macOS, Windows). They were added prior to Jenkins migration.
43
+ Some Linux jobs later migrated to use self-hosted runner.
36
44
37
45
## GitHub Actions
38
46
47
+ This section applies to GitHub-hosted runner GHAs. New workflows unless intended
48
+ to run on a matrix of OS should refer to Self-hosted runner GHAs [ .github/workflow/README] ( .github/workflow/README.md ) .
49
+
39
50
### GitHub actions run types
40
51
41
52
The following GA CI Job runs are currently run for Apache Beam, and each of the runs have different
42
53
purpose and context.
43
54
44
55
#### Pull request run
45
56
46
- Those runs are results of PR from the forks made by contributors. Most builds for Apache Beam fall
57
+ Those runs are results of PR from the forks made by contributors. Most builds for Apache Beam fall
47
58
into this category. They are executed in the context of the "Fork", not main
48
59
Beam Code Repository which means that they have only "read" permission to all the GitHub resources
49
60
(container registry, code repository). This is necessary as the code in those PRs (including CI job
@@ -57,15 +68,15 @@ the PR is ready to review and merge.
57
68
Those runs are results of direct pushes done by the committers or as result of merge of a Pull Request
58
69
by the committers. Those runs execute in the context of the Apache Beam Code Repository and have also
59
70
write permission for GitHub resources (container registry, code repository).
60
- The main purpose for the run is to check if the code after merge still holds all the assertions - like
71
+ The main purpose for the run is to check if the code after merge still holds all the assertions - like
61
72
whether it still builds, all tests are green.
62
73
63
74
This is needed because some of the conflicting changes from multiple PRs might cause build and test failures
64
75
after merge even if they do not fail in isolation.
65
76
66
77
#### Scheduled runs
67
78
68
- Those runs are results of (nightly) triggered job - only for ` master ` branch. The
79
+ Those runs are results of (nightly) triggered job - only for ` master ` branch. The
69
80
main purpose of the job is to check if there was no impact of external dependency changes on the Apache
70
81
Beam code (for example transitive dependencies released that fail the build). Another reason for the nightly
71
82
build is that the builds tags most recent master with ` nightly-master ` .
0 commit comments