Skip to content

[FLINK-35746][Kubernetes-Operator] Add getJobConfiguration and getJobCheckpointConfiguration #1000

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

nishita-09
Copy link
Contributor

@nishita-09 nishita-09 commented Jul 27, 2025

What is the purpose of the change

This pull request adds a new capability to the operator, this would ensure the configuration that operator gets is consistent with actual resultant configuration that is set both programmatically and via the deployment configs.

Brief change log

(for example:)

  • Add getJobConfiguration and getJobCheckpointConfiguration which uses the apache flink REST API methods to get the accurate configs actually set by the user programmatically.
  • FlinkConfig Manager ensures the REST calls are made only once and then cached using the information {namespace, deployment name, jobId, updateTime}. This is to ensure the call is made whenever there is job update or any change in the mentioned metadata.
  • If the cache key is found , then it uses the config that was created else it creates the new resultant config using the earlier method of getting config from deployment specs, followed by overriding from the user specified configs which we get using the REST API call that we make to the :jobId/config and :jobId/checkpoints/config.
  • *This resultant config is then used as the source of truth for flinkDeployment in case of application mode and flinkSessionJob in case of sessionMode. *

Verifying this change

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).
[WIP]
(or)

This change added tests and can be verified as follows:

(example:)

  • Deployed the job with application mode with deployment (checkpoint disabled) but enabled via the user in job logic.
  • Here, are the logs from ClusterHealthEvaluator which confirm that the operator now recognised the user defined configs.
  • Before the change:
Screenshot 2025-07-24 at 5 09 16 PM
  • After the change:
Screenshot 2025-07-24 at 5 13 06 PM

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changes to the CustomResourceDescriptors: no
  • Core observer or reconciler logic that is regularly executed: yes

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant