Skip to content

Update ASF "stash" action to allow list of keys #45268

@potiuk

Description

@potiuk
Member

In the new "caching" mechanism we are using "stash" GitHub action from @assignUser which only allows one key per action.

The action is in https://github.com/apache/infrastructure-actions/tree/main/stash

This is a a limiting factor when we want to download artifacts with a dynamic list of keys coming from a variable (say list of keys derived python versions - "3.9 3.10 3.11 3.12" -> "key-3.9 key-3.10 key-3.11 key-3.12".

Currently in #45266 (https://gtithub.com/apache/airflow/pull/45266/files#diff-a2dbc854d7d9f769ffe6efe98247baa68750cba87d785fb70d4616331010c7d0) we are hard-coding the possible actions to run with parameters and we are skipping those that are not on the list, but this is more of a hack than target solution.

Ideally we should be able to pass keys parameter to the action instead of key and be able to produce "hit-keys" list as output - then we could generate and pass the list of keys as input.

Activity

converted this from a draft issue on Dec 29, 2024
added
area:dev-envCI, pre-commit, pylint and other changes that do not change the behavior of the final code
on Dec 29, 2024
assignUser

assignUser commented on Dec 30, 2024

@assignUser
Member

So you would want the action to be able to download multiple 'stash' artifacts and unpack them into a single dir? I can see how this would be useful to simplify this: https://github.com/apache/airflow/pull/45266/files#diff-a2dbc854d7d9f769ffe6efe98247baa68750cba87d785fb70d4616331010c7d0R43

The main motivation for the action was caching compiler caches for which this doesn't really make sense but when used as a more general actions/cache replacement this makes sense.

potiuk

potiuk commented on Dec 30, 2024

@potiuk
MemberAuthor

Yeah. That's the main motivation - to avoid hard-coding of the conditional cache retrieval :) .. We might have either me or someone from our team to contribute that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:dev-envCI, pre-commit, pylint and other changes that do not change the behavior of the final codearea:dev-tools

    Type

    No type

    Projects

    Status

    Ready

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @potiuk@assignUser

        Issue actions

          Update ASF "stash" action to allow list of keys · Issue #45268 · apache/airflow