Skip to content

network: Support IPv6 only interfaces in CNI #25997

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

gabivlj
Copy link
Contributor

@gabivlj gabivlj commented Jun 5, 2025

Description

In previous versions, Nomad supported IPv6 only interfaces by simply setting the V6 value in Address. This worked until the move to AddressV6 in the networking CNI code, making interfaces that only use IPv6 unable to be used.

This commit tries to come back to that behaviour, where Address could also be a V6.

The reason we are not renaming to AddressIPv4 is to be retrocompatible with stored allocation networks on places like BoltDB.

Testing & Reproduction steps

Try to setup a CNI allocation on an IPv6 only iface. Easy to repro on the unit test.

Contributor Checklist

  • Changelog Entry If this PR changes user-facing behavior, please generate and add a
    changelog entry using the make cl command.
  • Testing Please add tests to cover any new functionality or to demonstrate bug fixes and
    ensure regressions will be caught.
  • Documentation If the change impacts user-facing functionality such as the CLI, API, UI,
    and job configuration, please update the Nomad website documentation to reflect this. Refer to
    the website README for docs guidelines. Please also consider whether the
    change requires notes within the upgrade guide.

Reviewer Checklist

  • Backport Labels Please add the correct backport labels as described by the internal
    backporting document.
  • Commit Type Ensure the correct merge method is selected which should be "squash and merge"
    in the majority of situations. The main exceptions are long-lived feature branches or merges where
    history should be preserved.
  • Enterprise PRs If this is an enterprise only PR, please add any required changelog entry
    within the public repository.

@42wim
Copy link
Contributor

42wim commented Jun 5, 2025

Related #25627 and #25632 :)

@gabivlj
Copy link
Contributor Author

gabivlj commented Jun 9, 2025

I can confirm that we have tested this in some test clusters we have and the jobs that were broken are now working as expected with 1.9.7 + this commit.

@tgross tgross self-requested a review June 9, 2025 20:10
@tgross tgross moved this from Needs Triage to In Progress in Nomad - Community Issues Triage Jun 9, 2025
@tgross tgross self-assigned this Jun 9, 2025
@tgross tgross moved this from In Progress to Triaging in Nomad - Community Issues Triage Jun 9, 2025
Copy link
Member

@tgross tgross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR @gabivlj! I did a bunch of testing with both IPv6-only and dual-stack CNI configurations and everything looks as I'd expect.

I've left one remark where we can eliminate some extra work. Also, can I get you to add a changelog entry for the bugfix with make cl?

Comment on lines +492 to +494
if netStatus.AddressIPv6 != "" {
netStatus.InterfaceName = name
}
Copy link
Member

@tgross tgross Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we move the fallback behavior here we get the same result but only need to call setStatus once:

Suggested change
if netStatus.AddressIPv6 != "" {
netStatus.InterfaceName = name
}
if netStatus.AddressIPv6 != "" {
netStatus.Address = netStatus.AddressIPv6
netStatus.InterfaceName = name
return
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying to understand here why would it be the same behaviour. I think the suggestion could make an IPv6 only interface be chosen over one that has IPv4/IPv6. If that's okay, I can apply it, but I think it might not be the exact same thing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The existing behavior always picks up the first address that matches the criteria (Nomad's model of addressing only really supports a single IP address per network block, which is a whole other problem but out of scope for this PR). Much of the time the CNI result will have multiple interfaces but only one of those interfaces will have addresses.

In the IPv4 only case, we return and break the loop because we've found our address. We'll want the same behavior with IPv6, but the patch you have here we'll always have the last IPv6 address in the IPv6-only case instead of the first address. It'll also call setStatus twice and pick up extra addresses in the non-sandbox pass.

We could probably use some more test cases here for both the IPv6 only case and the dual-stack case.

Comment on lines +511 to +515
// If no IP address could be found, fallback to IPv6-only
if netStatus.Address == "" {
netStatus.Address = netStatus.AddressIPv6
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer needed if we move into setStatus

Suggested change
// If no IP address could be found, fallback to IPv6-only
if netStatus.Address == "" {
netStatus.Address = netStatus.AddressIPv6
}

In previous versions, Nomad supported IPv6 only interfaces by simply
setting the V6 value in `Address`. This worked until the move to
`AddressV6` in the networking CNI code, making interfaces that only use
IPv6 unable to be used.

This commit tries to come back to that behaviour, where Address could
also be a V6.

The reason we are not renaming to AddressIPv4 is to be retrocompatible
with stored allocation networks on places like BoltDB.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

Successfully merging this pull request may close these issues.

3 participants