Skip to content

Add docs for Table Maintenance in Flink #13307

Open
@manuzhang

Description

@manuzhang
Collaborator

Feature Request / Improvement

Flink Table Maintenance was added in #11144, but there's no doc on its usage yet. TestMaintenanceE2E is a good reference.

Query engine

Flink

Willingness to contribute

  • I can contribute this improvement/feature independently
    I would be willing to contribute this improvement/feature with guidance from the Iceberg community
    I cannot contribute this improvement/feature at this time

Activity

JeonDaehong

JeonDaehong commented on Jun 13, 2025

@JeonDaehong
Contributor

Hi, I’d like to take this issue.
I’ll open a PR shortly to add the documentation.
Could you please assign this issue to me?

Thanks!

manuzhang

manuzhang commented on Jun 14, 2025

@manuzhang
CollaboratorAuthor

@JeonDaehong Thanks, we generally avoid assigning issues. Look forward to your PR.

JeonDaehong

JeonDaehong commented on Jun 14, 2025

@JeonDaehong
Contributor

@JeonDaehong Thanks, we generally avoid assigning issues. Look forward to your PR.

Thanks for the clarification!
I understand.
I’ll proceed without the assignment.
I’ll make sure to open a PR very soon. Looking forward to contributing!

JeonDaehong

JeonDaehong commented on Jun 15, 2025

@JeonDaehong
Contributor

@manuzhang

Hello !
I’m planning to create a new file named flink-maintenance.md and include the content from this document:
https://docs.google.com/document/d/16g3vR18mVBy8jbFaLjf2JwAANuYOmIwr15yDDxovdnA/edit?tab=t.0

I’ll also add usage instructions in the markdown file.

Could you please confirm if this approach is okay before I proceed?

Thanks! :D

pvary

pvary commented on Jun 15, 2025

@pvary
Contributor

CC: @Guosmilesmile, as she is working on these features.

I would recommend a more user focused documentation. The linked doc contains many development related details which are not something the users are concerned about.

Also, I would recommend to document the configuration possibilities introduced by @Guosmilesmile, which is the main concern for the users

JeonDaehong

JeonDaehong commented on Jun 15, 2025

@JeonDaehong
Contributor

CC: @Guosmilesmile, as she is working on these features.

I would recommend a more user focused documentation. The linked doc contains many development related details which are not something the users are concerned about.

Also, I would recommend to document the configuration possibilities introduced by @Guosmilesmile, which is the main concern for the users

Thank you for the feedback!

I'll check the core features and configuration options, and work on a user-focused documentation accordingly.
@Guosmilesmile, since you were tagged and seem to be working on these features — is there anything I should refer to or be aware of?

If you can share anything helpful, I’ll make sure to include it in the updated docs.

Thanks again!

Guosmilesmile

Guosmilesmile commented on Jun 16, 2025

@Guosmilesmile
Contributor

I also suggest that the documentation should be more user-focused, as the architecture documentation is a bit too detailed on development aspects.

  1. How to properly configure Table Maintenance through code, including explanations of configuration options, lock settings, and so on.
  2. This PR contains an example of some configuration options (but only regarding enabling compression in the sink), which I hope can serve as a reference for you: Flink: Support compact in iceberg sink v2 #12979 (comment)
JeonDaehong

JeonDaehong commented on Jun 16, 2025

@JeonDaehong
Contributor

I also suggest that the documentation should be more user-focused, as the architecture documentation is a bit too detailed on development aspects.

  1. How to properly configure Table Maintenance through code, including explanations of configuration options, lock settings, and so on.
  2. This PR contains an example of some configuration options (but only regarding enabling compression in the sink), which I hope can serve as a reference for you: Flink: Support compact in iceberg sink v2 #12979 (comment)

Thank you so much for your thoughtful feedback!

Based on your suggestions, I’ll revise the documentation with a more user-centric approach. I plan to thoroughly test everything myself and include detailed explanations from the user’s perspective.

It might take a bit of time, but I’m committed to making steady progress and completing it properly.
So if it’s okay, I’d like to take full responsibility for this PR and continue working on it myself.

Thanks again, and I hope you have a great day! :D

pvary

pvary commented on Jun 16, 2025

@pvary
Contributor

So if it’s okay, I’d like to take full responsibility for this PR and continue working on it myself.

@JeonDaehong: Do so, and ping us if you need any help.

linked a pull request that will close this issue on Jun 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @manuzhang@Guosmilesmile@pvary@JeonDaehong

      Issue actions

        Add docs for Table Maintenance in Flink · Issue #13307 · apache/iceberg