Skip to content

Commit 3550029

Browse files
authored
Adding basic manual testing instructions for go-sdk (#51912)
* Adding basic manual testing instructions for go-sdk * update stopping celery queues
1 parent b38e896 commit 3550029

File tree

1 file changed

+53
-0
lines changed

1 file changed

+53
-0
lines changed

go-sdk/README.md

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,59 @@ The Task API however does not provide a means to get the `ExecuteTaskWorkload` t
3232

3333
Since Go is a compiled language (putting aside projects such as [YAEGI](https://github.com/traefik/yaegi) that allow go to be interpreted) all tasks must be a) compiled in to the binary, and b) "registered" inside the worker process in order to be executed.
3434

35+
## Quick Testing Setup
36+
37+
The Go SDK currently works with Airflow's Celery Executor setup. Here's how to get started:
38+
39+
### Prerequisites
40+
41+
- Go 1.21 or later
42+
- Docker and Docker Compose (for Breeze)
43+
- Redis (for Celery broker)
44+
45+
### Step 1: Start Airflow with Celery Executor
46+
47+
Start Breeze with Celery executor:
48+
49+
```bash
50+
breeze start-airflow --backend postgres --executor CeleryExecutor --load-example-dags
51+
```
52+
53+
This will start:
54+
55+
- Airflow API Server on `http://localhost:28080`
56+
- Celery workers (we will not utilise this)
57+
- Redis broker on `localhost:26379`
58+
- Loads the example DAGs
59+
60+
### Step 2: Stop the Celery Worker
61+
62+
We want to run the go workers instead of running the Celery ones. So in `breeze`, press CTRL+C to
63+
stop the Celery workers.
64+
65+
### Step 3: Run the Go SDK Worker
66+
67+
From the `go-sdk` directory, run the example worker:
68+
69+
```bash
70+
go run ./example/main.go run \
71+
--broker-address=localhost:26379 \
72+
--queues default \
73+
--execution-api-url http://localhost:28080/execution
74+
```
75+
76+
**Parameters explained:**
77+
78+
- `--broker-address=localhost:26379`: Redis broker address (default Celery broker)
79+
- `--queues default`: Queue name where Celery enqueues tasks
80+
- `--execution-api-url http://localhost:28080/execution`: Airflow's Task Execution API endpoint
81+
82+
### Step 4: Submit a Test Task
83+
84+
You can submit tasks through the Airflow UI for dag_id: `tutorial_dag`. The Go worker will pick up tasks from the Celery queue and execute them using the Task Execution Interface.
85+
86+
Observe the logs in the terminal where you run the test task.
87+
3588
## Current state
3689

3790
This SDK currently will:

0 commit comments

Comments
 (0)