-
Notifications
You must be signed in to change notification settings - Fork 80
fix(KONFLUX-8965): Add brief description for issues and CVEs fixed #1164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
+295
−43
Closed
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# enrich-advisory-metadata (Pipeline) | ||
|
||
This internal Tekton pipeline runs the `enrich-advisory-metadata` task to enrich advisory metadata with issue and CVE titles. It is intended to be run in an internal context with access to Red Hat internal APIs and secrets. | ||
|
||
## Parameters | ||
|
||
| Name | Description | Required | Default | | ||
|----------|--------------------------------------------------|----------|------------------------------| | ||
| dataPath | Path to the JSON string of the merged data to use| Yes | - | | ||
| dataDir | The location where data will be stored | No | $(workspaces.data.path) | | ||
|
||
## Workspaces | ||
|
||
- `data`: The workspace where the data JSON file resides | ||
|
||
## Description | ||
|
||
- Calls the `enrich-advisory-metadata` task to fetch and inject titles for issues and CVEs in the advisory data JSON. | ||
- Should be called from a managed/external pipeline via an internal request, before further processing or advisory creation. |
32 changes: 32 additions & 0 deletions
32
pipelines/internal/enrich-cve-metadata/enrich-cve-metadata.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
--- | ||
apiVersion: tekton.dev/v1 | ||
kind: Pipeline | ||
metadata: | ||
name: enrich-cve-metadata | ||
annotations: | ||
tekton.dev/pipelines.minVersion: "0.12.1" | ||
tekton.dev/tags: internal, enrichment | ||
spec: | ||
description: >- | ||
Pipeline to enrich CVE metadata with titles using the enrich-cve-metadata task. | ||
params: | ||
- name: cvesJson | ||
type: string | ||
description: JSON array of CVEs to enrich | ||
results: | ||
- name: enrichedCvesJson | ||
value: $(tasks.enrich-cve-metadata.results.enrichedCvesJson) | ||
tasks: | ||
- name: enrich-cve-metadata | ||
taskRef: | ||
resolver: "git" | ||
params: | ||
- name: url | ||
value: https://github.com/konflux-ci/release-service-catalog.git | ||
- name: revision | ||
value: main | ||
- name: pathInRepo | ||
value: tasks/internal/enrich-advisory-metadata/enrich-cve-metadata.yaml | ||
params: | ||
- name: cvesJson | ||
value: $(params.cvesJson) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,28 @@ | ||
# enrich-cve-metadata | ||
|
||
This internal Tekton task enriches CVE metadata by fetching and injecting CVE titles into the provided CVE JSON array. It is intended to be run in an internal context with access to Red Hat internal APIs and secrets. | ||
|
||
## Parameters | ||
|
||
| Name | Description | Required | Default | | ||
|----------|--------------------------------------------------|----------|---------| | ||
| cvesJson | JSON array of CVEs to enrich | Yes | - | | ||
|
||
## Results | ||
|
||
| Name | Description | | ||
|------------------|------------------------------------| | ||
| enrichedCvesJson | JSON array of enriched CVEs | | ||
|
||
## Secrets | ||
|
||
- `osidb-api-token`: Used to authenticate to OSIDB (must contain `token` key) | ||
|
||
## Description | ||
|
||
- For each CVE in the input JSON array, fetches the title/summary from OSIDB (or a public CVE API) and injects it as the `title` field. | ||
- Outputs the enriched CVEs as a JSON array in the `enrichedCvesJson` result. | ||
|
||
## Example Usage | ||
|
||
This task should be called from an internal pipeline (e.g., `enrich-cve-metadata` pipeline) via an internal request from a managed pipeline. The managed pipeline should parse the result and inject the enriched CVEs back into its data JSON. |
55 changes: 55 additions & 0 deletions
55
tasks/internal/enrich-cve-metadata/enrich-cve-metadata.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
--- | ||
apiVersion: tekton.dev/v1 | ||
kind: Task | ||
metadata: | ||
name: enrich-cve-metadata | ||
annotations: | ||
tekton.dev/pipelines.minVersion: "0.12.1" | ||
tekton.dev/tags: internal, enrichment | ||
spec: | ||
description: >- | ||
Enriches CVE metadata by fetching and injecting CVE titles into the provided CVE JSON array. | ||
params: | ||
- name: cvesJson | ||
description: JSON array of CVEs to enrich | ||
type: string | ||
results: | ||
- name: enrichedCvesJson | ||
description: JSON array of enriched CVEs | ||
steps: | ||
- name: enrich-cves-with-titles | ||
image: quay.io/konflux-ci/release-service-utils:latest | ||
computeResources: | ||
limits: | ||
memory: 256Mi | ||
requests: | ||
memory: 256Mi | ||
cpu: '1' # 1 is the max allowed by at least the staging cluster | ||
env: | ||
- name: OSIDB_TOKEN | ||
valueFrom: | ||
secretKeyRef: | ||
name: osidb-api-token | ||
key: token | ||
script: | | ||
#!/usr/bin/env bash | ||
set -ex | ||
CVES_JSON="$(params.cvesJson)" | ||
ENRICHED_CVES=() | ||
NUM_CVES=$(echo "$CVES_JSON" | jq 'length') | ||
for ((i = 0; i < NUM_CVES; i++)); do | ||
cve=$(echo "$CVES_JSON" | jq -c --argjson i "$i" '.[$i]') | ||
key=$(echo "$cve" | jq -r '.key') | ||
title="" | ||
# Fetch from OSIDB or CVE API | ||
title=$(curl -s -H "Authorization: Bearer $OSIDB_TOKEN" \ | ||
"https://osidb/api/v1/flaws?cve_id=$key" | \ | ||
jq -r '.results[0].title // .results[0].cve_description // \ | ||
.results[0].description // .results[0].summary // empty') | ||
if [ -n "$title" ] && [ "$title" != "null" ]; then | ||
cve=$(echo "$cve" | jq --arg title "$title" '.title = $title') | ||
fi | ||
ENRICHED_CVES+=("$cve") | ||
done | ||
# Output enriched CVEs as a JSON array | ||
echo -n "${ENRICHED_CVES[@]}" | jq -s '.' > "$(results.enrichedCvesJson.path)" |
44 changes: 44 additions & 0 deletions
44
tasks/internal/enrich-cve-metadata/tests/test-enrich-cve-metadata.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
--- | ||
apiVersion: tekton.dev/v1 | ||
kind: Pipeline | ||
metadata: | ||
name: test-enrich-cve-metadata | ||
spec: | ||
params: | ||
- name: cvesJson | ||
type: string | ||
default: '[{"key": "CVE-2024-0001", "component": "comp1"}, {"key": "CVE-2024-0002", "component": "comp2"}]' | ||
tasks: | ||
- name: run-task | ||
taskRef: | ||
name: enrich-cve-metadata | ||
params: | ||
- name: cvesJson | ||
value: $(params.cvesJson) | ||
- name: check-result | ||
runAfter: | ||
- run-task | ||
params: | ||
- name: enrichedCvesJson | ||
value: $(tasks.run-task.results.enrichedCvesJson) | ||
taskSpec: | ||
params: | ||
- name: enrichedCvesJson | ||
type: string | ||
steps: | ||
- name: check-result | ||
image: quay.io/konflux-ci/release-service-utils:latest | ||
script: | | ||
#!/usr/bin/env bash | ||
set -eux | ||
enriched_cves="$(params.enrichedCvesJson)" | ||
title1=$(echo "$enriched_cves" | jq -r '.[0].title') | ||
title2=$(echo "$enriched_cves" | jq -r '.[1].title') | ||
if [ -z "$title1" ] || [ "$title1" == "null" ]; then | ||
echo "Error: title field is missing for the first CVE" | ||
exit 1 | ||
fi | ||
if [ -z "$title2" ] || [ "$title2" == "null" ]; then | ||
echo "Error: title field is missing for the second CVE" | ||
exit 1 | ||
fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -69,48 +69,6 @@ spec: | |
- name: "DEBUG" | ||
value: "$(params.trustedArtifactsDebug)" | ||
steps: | ||
- name: skip-trusted-artifact-operations | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why is this removed There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It was unintentional |
||
computeResources: | ||
limits: | ||
memory: 32Mi | ||
requests: | ||
memory: 32Mi | ||
cpu: 20m | ||
ref: | ||
resolver: "git" | ||
params: | ||
- name: url | ||
value: $(params.taskGitUrl) | ||
- name: revision | ||
value: $(params.taskGitRevision) | ||
- name: pathInRepo | ||
value: stepactions/skip-trusted-artifact-operations/skip-trusted-artifact-operations.yaml | ||
params: | ||
- name: ociStorage | ||
value: $(params.ociStorage) | ||
- name: workDir | ||
value: $(params.dataDir) | ||
- name: use-trusted-artifact | ||
computeResources: | ||
limits: | ||
memory: 64Mi | ||
requests: | ||
memory: 64Mi | ||
cpu: 30m | ||
ref: | ||
resolver: "git" | ||
params: | ||
- name: url | ||
value: $(params.taskGitUrl) | ||
- name: revision | ||
value: $(params.taskGitRevision) | ||
- name: pathInRepo | ||
value: stepactions/use-trusted-artifact/use-trusted-artifact.yaml | ||
params: | ||
- name: workDir | ||
value: $(params.dataDir) | ||
- name: sourceDataArtifact | ||
value: $(params.sourceDataArtifact) | ||
- name: populate-release-notes-images | ||
image: quay.io/konflux-ci/release-service-utils:e85ceb962ee6f4d0672b4aa4e9946621ab302f20 | ||
computeResources: | ||
|
@@ -299,7 +257,96 @@ spec: | |
/tmp/data.tmp && mv /tmp/data.tmp "${DATA_FILE}" | ||
done | ||
done | ||
|
||
- name: enrich-metadata | ||
image: quay.io/konflux-ci/release-service-utils:latest | ||
computeResources: | ||
limits: | ||
memory: 32Mi | ||
requests: | ||
memory: 32Mi | ||
cpu: 10m | ||
env: | ||
- name: JIRA_TOKEN | ||
valueFrom: | ||
secretKeyRef: | ||
name: konflux-advisory-jira-secret | ||
key: token | ||
script: | | ||
#!/usr/bin/env bash | ||
set -ex | ||
DATA_FILE="$(params.dataDir)/$(params.dataPath)" | ||
# Enrich issue titles | ||
NUM_ISSUES=$(jq '.releaseNotes.issues.fixed | length' "$DATA_FILE") | ||
for ((i = 0; i < NUM_ISSUES; i++)); do | ||
issue=$(jq -c --argjson i "$i" '.releaseNotes.issues.fixed[$i]' "$DATA_FILE") | ||
id=$(jq -r '.id' <<< "$issue") | ||
source=$(jq -r '.source' <<< "$issue") | ||
title="" | ||
if [[ "$source" == "issues.redhat.com" ]]; then | ||
title=$(curl -s -H "Authorization: Bearer $JIRA_TOKEN" \ | ||
"https://$source/rest/api/2/issue/$id" | jq -r '.fields.summary') | ||
elif [[ "$source" == "bugzilla.redhat.com" ]]; then | ||
title=$(curl -s "https://$source/rest/bug/$id" | jq -r '.summary') | ||
fi | ||
if [ -n "$title" ] && [ "$title" != "null" ]; then | ||
jq --argjson i "$i" --arg title "$title" \ | ||
'.releaseNotes.issues.fixed[$i].title = $title' "$DATA_FILE" > /tmp/data.tmp && \ | ||
mv /tmp/data.tmp "$DATA_FILE" | ||
fi | ||
done | ||
# Enrich CVE titles via internal pipeline | ||
CVES_JSON=$(jq -c '.releaseNotes.cves' "$DATA_FILE") | ||
IR_FILE=$(mktemp) | ||
internal-request --pipeline "enrich-cve-metadata" \ | ||
-p cvesJson="$CVES_JSON" \ | ||
-s true > "$IR_FILE" | ||
ENRICHED_CVES=$(jq -r '.status.results[] | select(.name=="enrichedCvesJson") | .value' "$IR_FILE") | ||
if [ -n "$ENRICHED_CVES" ] && [ "$ENRICHED_CVES" != "null" ]; then | ||
jq --argjson cves "$ENRICHED_CVES" '.releaseNotes.cves = $cves' "$DATA_FILE" > /tmp/data.tmp && \ | ||
mv /tmp/data.tmp "$DATA_FILE" | ||
fi | ||
- name: skip-trusted-artifact-operations | ||
computeResources: | ||
limits: | ||
memory: 32Mi | ||
requests: | ||
memory: 32Mi | ||
cpu: 20m | ||
ref: | ||
resolver: "git" | ||
params: | ||
- name: url | ||
value: $(params.taskGitUrl) | ||
- name: revision | ||
value: $(params.taskGitRevision) | ||
- name: pathInRepo | ||
value: stepactions/skip-trusted-artifact-operations/skip-trusted-artifact-operations.yaml | ||
params: | ||
- name: ociStorage | ||
value: $(params.ociStorage) | ||
- name: workDir | ||
value: $(params.dataDir) | ||
- name: use-trusted-artifact | ||
computeResources: | ||
limits: | ||
memory: 64Mi | ||
requests: | ||
memory: 64Mi | ||
cpu: 30m | ||
ref: | ||
resolver: "git" | ||
params: | ||
- name: url | ||
value: $(params.taskGitUrl) | ||
- name: revision | ||
value: $(params.taskGitRevision) | ||
- name: pathInRepo | ||
value: stepactions/use-trusted-artifact/use-trusted-artifact.yaml | ||
params: | ||
- name: workDir | ||
value: $(params.dataDir) | ||
- name: sourceDataArtifact | ||
value: $(params.sourceDataArtifact) | ||
- name: populate-release-notes-type-and-references | ||
image: quay.io/konflux-ci/release-service-utils:e85ceb962ee6f4d0672b4aa4e9946621ab302f20 | ||
computeResources: | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We talked about https://issues.redhat.com/browse/KONFLUX-8965 within the team during yesterday's backlog refinement session. See David's comment in the Jira. I am really not sure we should bake descriptions into the advisory yaml. Those are not meant for humans to read. Users should consume these via the customer portal and if we think the advisory page should have more details for the jiras and cves, we should ask the owners of that to add these when displaying the links.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with @mmalina here.
In the Konflux advisory we should verify attached to advisory Jira issues like checking if it's not related to the embargoed content and show state if the Jira issue is publicly available or not. The Jira short description should be Konflux advisory consumer problem, in the same way how consumer parse the information if the linked Jira is publicly available or not.
How list of Jira issues or list of CVEs is displayed is up to the service which work on this metadata.