Skip to content

Commit 6c564e5

Browse files
vordimousjfallowsbmaidicsankitk-me
authored
Examples using compose (#152)
* example-startup-compose - update set flag - cache docker images - make scripts runnable - feat: add test runner for examples with changes - feat: rename and remove examples as cookbooks - update karapace to schema registry - update compose files - remove authority from routes - update shell to sh - feat: add zilla version from env var - chore: update actions versions * update readmes * update action naming * clean up some docker syntax * remove uneeded compose * use kafkacat name * run formatter * remove the pull_policy and add versioning instructions * add params to the test runner * remove container name * add test. * feat: add tests and mark failures * fix: compare strings * fix: test syntax * feat: add sse tests * fix test * add grpc echo tests * add more tests * implement tests * Configure ghz to use plaintext * Increase bench total message count to 1000 * Change http.kafka.oneway example use sasl ssl (#154) * fix: asyncapi.http.kafka.proxy * `asyncapi.mqtt.kafka.proxy` test script fix * `grpc.echo` fix * `grpc.echo` test fix * `grpc.kafka.echo` & `grpc.echo` fix * `grpc.kafka.proxy` fix * fix `tls.echo` * fix `sse.kafka.fanout` * add delay to `sse.kafka.fanout` * update `sse.kafka.fanout` * create kafka topic `sse.kafka.fanout` * `sse.kafka.fanout` update * fix: `mqtt.jwt` * fix `teardown.sh` for `mqtt.jwt` * fix `http.kafka.avro.json` * update `karapace-registry` service in `http.kafka.avro.json` * fix `http.json.schema` * fix: `grpc.proxy` * add `test.sh` : `grpc.kafka.fanout` * fix `grpc.kafka.fanout` * add `test.sh` : `http.proxy` * add `test.sh` : `ws.echo` & `ws.reflect` * fix `ws.*` * fix `ws` * fix `ws` * install `wscat` as part of github action * add `test.sh` : `http.jwt` * update `http.jwt` * update `http.jwt` * update `http` * add `test.sh` : `mqtt.jwt` * add `test.sh` : `sse.jwt` * fix `sse.jwt` * fix `sse` * message update `sse.jwt` * fix `sse.jwt` * fix `ws.echo` * update `ws.echo` * use `websocat` * fix `ws.reflect` * `ws` update * update `ws` * update `ws` * update `ws.refect` * update * update `ws` * add `test` : http.kafka.async` * add `test` : `http.kafka.proto.oneway` * fix `http.kafka.avro.json` * add `test`: `tcp.reflect` * add `test` : `tls.reflect` * update * `http.kafka.proto.json` readme update * update: `optimise test script setup` * addressing review comments * `ws.reflect` update * update `ws.reflect` * Use insecure instead of skipTLS for plaintext * fix `grpc.proxy` health check * update `tls` examples with `-no_ign_eof` * add `mqtt.kafka.proxy` example * update `exit` * fix `mqtt` test * addressing review comments * rename `http.jwt` to `http.proxy.jwt` * addressing review comments * addressing review comments * update compose with `watch` * update `hostname` * addressing review comments * move `.proto` files under `./etc/protos` * update `grpc.echo` * fix `grpc.echo` * addressing review comments * update `test.sh` * update `compose` to remove `watch` * update `README` * remove `setup.sh` & `teardown.sh` * add issue link for failing tests --------- Co-authored-by: John Fallows <[email protected]> Co-authored-by: bmaidics <[email protected]> Co-authored-by: Ankit Kumar <[email protected]>
1 parent fffc7e2 commit 6c564e5

File tree

486 files changed

+5764
-12430
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

486 files changed

+5764
-12430
lines changed

‎examples/.github/workflows/example_artifacts.yaml

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name: Release Example Artifacts
33
on:
44
push:
55
tags:
6-
- 'v[0-9]+'
6+
- "v[0-9]+"
77

88
permissions:
99
contents: write
@@ -12,9 +12,7 @@ jobs:
1212
build:
1313
runs-on: ubuntu-latest
1414
steps:
15-
16-
- name: Checkout
17-
uses: actions/checkout@v4
15+
- uses: actions/checkout@v4
1816

1917
- name: Tar all examples
2018
run: for i in */; do tar -zcvf "${i%/}.tar.gz" "$i"; done

‎examples/.github/workflows/links-push.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ jobs:
1111
linkChecker:
1212
runs-on: ubuntu-latest
1313
steps:
14-
- uses: actions/checkout@v3
14+
- uses: actions/checkout@v4
1515

1616
- name: Link Checker
1717
uses: lycheeverse/[email protected]

‎examples/.github/workflows/reusable_test_runner.yaml

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
name: Reusable Workflow for running tests
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
example-dir:
7+
required: true
8+
description: Directory name for the example to test
9+
type: string
10+
zilla-image-tag:
11+
default: latest
12+
description: Zilla Image tag
13+
type: string
14+
zilla-image-artifact-name:
15+
default: zilla-image-artifact
16+
description: Artifact name for a Zilla Image
17+
type: string
18+
19+
workflow_call:
20+
inputs:
21+
example-dir:
22+
required: true
23+
type: string
24+
zilla-image-tag:
25+
type: string
26+
zilla-image-artifact-name:
27+
type: string
28+
29+
jobs:
30+
runner:
31+
runs-on: ubuntu-latest
32+
env:
33+
ZILLA_VERSION: ${{ inputs.zilla-image-tag }}
34+
steps:
35+
- name: Checkout
36+
uses: actions/checkout@v4
37+
with:
38+
sparse-checkout: ${{ inputs.example-dir }}
39+
40+
- name: Cache Docker images.
41+
if: ${{ hashFiles(format('{0}/compose.yaml', inputs.example-dir)) != '' }}
42+
uses: ScribeMD/[email protected]
43+
with:
44+
key: docker-${{ runner.os }}-${{ inputs.example-dir }}-${{ hashFiles(format('{0}/compose.yaml', inputs.example-dir)) }}
45+
46+
- name: Download artifact
47+
if: ${{ inputs.zilla-image-artifact-name != '' }}
48+
uses: actions/download-artifact@v4
49+
with:
50+
name: ${{ inputs.zilla-image-artifact-name }}
51+
path: /tmp
52+
53+
- name: Load image
54+
if: ${{ inputs.zilla-image-artifact-name != '' }}
55+
run: |
56+
docker load --input /tmp/${{ inputs.zilla-image-artifact-name }}.tar
57+
docker image ls -a
58+
59+
- name: Start Zilla and wait for it to be healthy
60+
working-directory: ${{ inputs.example-dir }}
61+
run: docker compose up -d --wait
62+
63+
- name: Execute Test
64+
if: ${{ hashFiles(format('{0}/.github/test.sh', inputs.example-dir)) != '' }}
65+
working-directory: ${{ inputs.example-dir }}
66+
run: |
67+
set -o pipefail
68+
./.github/test.sh | tee $GITHUB_STEP_SUMMARY
69+
70+
- name: Collect docker logs on failure
71+
if: failure()
72+
uses: jwalton/gh-docker-logs@v2
73+
with:
74+
dest: "./logs"
75+
- name: Tar logs
76+
if: failure()
77+
run: tar cvzf ./logs.tgz ./logs
78+
- name: Upload logs to GitHub
79+
if: failure()
80+
uses: actions/upload-artifact@v4
81+
with:
82+
name: ${{ inputs.example-dir }}_logs.tgz
83+
path: ./logs.tgz
84+
85+
- name: Teardown
86+
if: always() && ${{ hashFiles(format('{0}/teardown.sh', inputs.example-dir)) != '' }}
87+
working-directory: ${{ inputs.example-dir }}
88+
run: docker compose down --remove-orphans

‎examples/.github/workflows/test.yaml

Lines changed: 33 additions & 85 deletions
Original file line numberDiff line numberDiff line change
@@ -5,93 +5,41 @@ on:
55
workflow_dispatch:
66

77
jobs:
8-
test-tcp-echo:
8+
get-examples-with-changes:
99
runs-on: ubuntu-latest
10+
outputs:
11+
changed_directories: ${{ steps.set-output.outputs.changed_directories }}
1012
steps:
11-
- name: Checkout
12-
uses: actions/checkout@v3
13-
- name: Create k8s Kind Cluster
14-
uses: helm/kind-action@v1
15-
- name: Setup
16-
working-directory: tcp.echo
17-
run: ./setup.sh
18-
- name: Execute Test
19-
working-directory: tcp.echo
20-
run: |
21-
set -o pipefail
22-
./test.sh | tee $GITHUB_STEP_SUMMARY
23-
- name: Teardown
24-
if: always()
25-
working-directory: tcp.echo
26-
run: ./teardown.sh
27-
28-
test-tls-echo:
29-
runs-on: ubuntu-latest
30-
steps:
31-
- name: Checkout
32-
uses: actions/checkout@v3
33-
- name: Create k8s Kind Cluster
34-
uses: helm/kind-action@v1
35-
- name: Setup
36-
working-directory: tls.echo
37-
run: ./setup.sh
38-
- name: Execute Test
39-
working-directory: tls.echo
40-
run: |
41-
set -o pipefail
42-
./test.sh | tee $GITHUB_STEP_SUMMARY
43-
- name: Teardown
44-
if: always()
45-
working-directory: tls.echo
46-
run: ./teardown.sh
13+
- uses: actions/checkout@v4
14+
with:
15+
fetch-depth: 0
4716

48-
test-http-kafka-sync:
49-
runs-on: ubuntu-latest
50-
steps:
51-
- name: Install apt packages
52-
uses: awalsh128/cache-apt-pkgs-action@latest
17+
- name: Get changed files
18+
id: changed-files
19+
uses: tj-actions/[email protected]
5320
with:
54-
packages: librdkafka-dev libyajl-dev
55-
version: 1.0
56-
- name: Install kcat
57-
working-directory: /tmp
58-
run: |
59-
set -x
60-
curl -L -o kcat https://github.com/attilakreiner/kcat/releases/download/1.7.1/kcat-linux-$(arch)
61-
chmod +x kcat
62-
sudo mv kcat /usr/local/bin
63-
kcat 2>&1 | grep "Version 1.7.1"
64-
- name: Checkout
65-
uses: actions/checkout@v3
66-
- name: Create k8s Kind Cluster
67-
uses: helm/kind-action@v1
68-
- name: Setup
69-
working-directory: http.kafka.sync
70-
run: ./setup.sh
71-
- name: Execute Test
72-
working-directory: http.kafka.sync
73-
run: |
74-
set -o pipefail
75-
./test.sh | tee $GITHUB_STEP_SUMMARY
76-
- name: Teardown
77-
if: always()
78-
working-directory: http.kafka.sync
79-
run: ./teardown.sh
21+
# only get the top level example dirs that have changes
22+
dir_names: true
23+
dir_names_max_depth: 1
24+
dir_names_exclude_current_dir: true
25+
exclude_submodules: true
26+
files_ignore: |
27+
*.md
28+
.github/**/*
29+
json: true
30+
quotepath: false
8031

81-
test-openapi-proxy:
82-
runs-on: ubuntu-latest
83-
steps:
84-
- name: Checkout
85-
uses: actions/checkout@v3
86-
- name: Setup
87-
working-directory: openapi.proxy/docker/compose
88-
run: ./setup.sh
89-
- name: Execute Test
90-
working-directory: openapi.proxy
91-
run: |
92-
set -o pipefail
93-
./test.sh | tee $GITHUB_STEP_SUMMARY
94-
- name: Teardown
95-
if: always()
96-
working-directory: openapi.proxy/docker/compose
97-
run: ./teardown.sh
32+
- name: "Set output in the matrix format"
33+
id: set-output
34+
run: echo "changed_directories={\"dir\":${{ steps.changed-files.outputs.all_changed_files }}}" >> "$GITHUB_OUTPUT"
35+
36+
testing:
37+
if: ${{ needs.get-examples-with-changes.outputs.changed_directories != '' }}
38+
strategy:
39+
matrix: ${{fromJson(needs.get-examples-with-changes.outputs.changed_directories)}}
40+
fail-fast: false
41+
needs:
42+
- get-examples-with-changes
43+
uses: ./.github/workflows/reusable_test_runner.yaml
44+
with:
45+
example-dir: ${{ matrix.dir }}

‎examples/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,3 +31,4 @@ release.properties
3131
target/
3232
zilla
3333
zilla.yaml.generated
34+
*.out

0 commit comments

Comments
 (0)