Skip to content

Change cache scope dynamically #6879

Open
@Mugenor

Description

@Mugenor

Feature Request

Describe the problem you need a feature to resolve.

We are creating multi-tenant K8S cluster and using HNC to structure tenants namespaces. We have a single root namespace, and a subnamespace for each tenant. Tenants can create their own subnamespaces if they need (for app deployment or testing).

We are working on an operator that manages CRs in tenants namespaces, meaning that it has to work with all namespaces in the hierarchy that is changes constantly throughout a day.

The current options we have:

  1. Configure operator cache to monitor the whole cluster. But we are constrained in terms of RBAC, so we can give it access only to the tenant NS hierarchy.
  2. It's possible to configure operator to monitor subset of NSs, the problem is that this subset is static and have to be configured at the startup, so it requires us to restart the operator every time namespace is created or deleted. While, this is technically working, it's very noisy and makes the operator restart pretty often.
  3. It's possible to create a cache wrapper that will restart just the cache instance in runtime when namespace is created or deleted and use it with NewCache option. Again, it's pretty noisy and while it helps to avoid operator restarts, it triggers reconciliation of everything, logs pollutes with errors from the controller runtime because of stopped cache.
  4. Write our own implementation of cache that will satisfy our requirements. This is error prone and it looks like it's not possible to reuse some code bits related to cache from the library, because almost everything is private, meaning that it will require huge duplication of code that already exists in the framework.

So, none of the options is really perfect.

Describe the solution you'd like.

It would be nice to have some kind of API to tweak scope of the cache.

I think it's reasonable to make the users code to decide when and how the cache scope should change. For example, I as a operator developer can decide how I watch for events in cluster and depending on custom conditions change update the cache scope.

I assume namespace granularity would suffice most of use cases.

A lot of operator implementations allows to specify the static list of watched namespaces, but not dynamic because of this issue. It will enable them for more flexible use.

P.S.: Looks like it's already implemented in java version of operator sdk, so it would be nice to have for go version as well.

Activity

openshift-bot

openshift-bot commented on Mar 17, 2025

@openshift-bot

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

added
lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
on Mar 17, 2025
Mugenor

Mugenor commented on Apr 13, 2025

@Mugenor
Author

/remove-lifecycle stale

removed
lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
on Apr 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/featureCategorizes issue or PR as related to a new feature.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @openshift-bot@Mugenor@acornett21

        Issue actions

          Change cache scope dynamically · Issue #6879 · operator-framework/operator-sdk