Releases: kmesh-net/kmesh
Release v1.1.0
We are delighted to announce the release of Kmesh v1.1.0, a milestone achieved through the collective efforts of our global community over the past three months. Special recognition goes to the contributors from the LXF Project, whose dedication has been pivotal in driving this release forward.
Building on the foundation of v1.0.0, this release introduces significant enhancements to Kmesh’s architecture, observability, and ecosystem integration. The official Kmesh website has undergone a comprehensive redesign, offering an intuitive interface and streamlined documentation to empower both users and developers. Under the hood, we’ve refactored the DNS module and added metrics for long connections, providing deeper insights into more traffic patterns.
In Kernel-Native mode, we’ve reduced invasive kernel modifications. Also, we use global variables to replace the BPF config map to simplify the underlying complexity. Compatibility with Istio 1.25 has been rigorously validated, ensuring seamless interoperability with the latest Istio version. Notably, the persistent TestKmeshRestart E2E test case flaky—a long-standing issue—has been resolved through long-term investigation and reconstruction of the underlying BPF program, marking a leap forward in runtime reliability.
Main Features
Website overhaul
The Kmesh official website has undergone a complete redesign, offering an intuitive user experience with improved documentation, reorganized content hierarchy and streamlined navigation. In addressing feedback from the previous iteration, we focused on key areas where user experience could be enhanced. The original interface presented some usability challenges that occasionally led to navigation difficulties. Our blog module in particular required attention, as its content organization and visual hierarchy impacted content discoverability and readability. From an engineering perspective, we recognized opportunities to improve the code structure through better component organization and more systematic styling approaches, as the existing implementation had grown complex to maintain over time.
To address these problems, we shifted to React with Docusaurus, a modern documentation framework that's much more developer-friendly. This allowed us to create modular components, eliminating redundant code through reusability. Docusaurus provides built-in navigation systems specifically designed for documentation and blogs, plus version-controlled documentation features. We've implemented multilingual support with both English and Chinese documentation, added advanced search functionality, and completely reorganized the content structure. The result is a dramatically improved experience that makes the Kmesh site more accessible and valuable for all users.
Long connection metrics
Before this release, Kmesh provides access logs during termination and establishment of a TCP connection with more detailed information about the connection, such as bytes sent, received, packet lost, rtt and retransmits. Kmesh also provides workload and service specific metrics such as bytes sent and received, lost packets, minimum rtt, total connection opened and closed by a pod. These metrics are only updated after a connection is closed.
In this release, we implement access logs and metrics for TCP long connections, developing a continuous monitoring and reporting mechanism that captures detailed, real-time data throughout the lifetime of long-lived TCP connections. Access logs are reported periodically with information such as reporting time, connection establishment time, bytes sent, received, packet loss, rtt, retransmits and state. Metrics such as bytes sent and received, packet loss, retransmits are also reported periodically for long connections.
DNS refactor
The current DNS process includes the CDS refresh process. As a result, DNS is deeply coupled with kernel-native mode and cannot be used in dual-engine mode.
In release 1.1 we refactored the DNS module of Kmesh. Instead of a structure containing cds, the data looped through the refresh queue in the Dns is now a domain, so that the Dns module no longer cares about the Kmesh mode, only providing the hostname to be resolved.
BPF config map optimization
Kmesh has eliminated the dedicated kmesh_config_map BPF map, which previously stored global runtime configurations such as BPF logging level and monitoring toggle. These settings are now managed through global variables. Leveraging global variables simplifies BPF configuration management, enhancing runtime efficiency and maintainability.
Optimise Kernel Native mode to reduce intrusive modifications to the kernel
The kernel-native mode requires a large number of intrusive kernel reconstructions to implement HTTP-based traffic control. Some of these modifications may have a significant impact on the kernel, which makes the kernel-native mode difficult to deploy and use in a real production environment.
To resolve this problem, we have modified the kernel in kernel-native mode and the involved ko and eBPF synchronously. Through the optimization of this release. In kernel 5.10, the kernel modification is limited to four, and in kernel 6.6, the kernel modification is reduced to only one. This last one will be eliminated as much as possible, with the goal of eventually running kernel-native mode on native version 6.6 and above.
Adopt istio 1.25
Kmesh has verified compatibility with istio 1.25 and has added the corresponding E2E test to CI. The Kmesh community maintains verification of the three istio versions in CI, so the E2E test of istio 1.22 has been removed from CI.
Critical Bug Fix
-
kmeshctl install waypoint error (#1287)
-
TestKmeshRestart flaky (#1192)
-
TestServiceEntrySelectsWorkloadEntry flaky (#1352)
What's Changed
- improve xdp bpf log by @weli-l in #1158
- Ability to automatically push helm packages at publicize release by @LiZhenCheng9527 in #1174
- Can specify the out name for kmeshctl by @LiZhenCheng9527 in #1176
- fix DATA RACE in TestCertRoute by @lec-bit in #1168
- add scripts to change kmesh version automatically by @LiZhenCheng9527 in #1183
- adapt MAP_SIZE_OF_LISTENER into 8192 by @lec-bit in #1187
- fix Update mode failed by @lec-bit in #1188
- Bump google.golang.org/grpc from 1.69.0 to 1.69.4 by @dependabot in #1179
- Bump golang.org/x/net to address CVE-2024-45338 by @hzxuzhonghu in #1193
- Bump google.golang.org/protobuf from 1.36.1 to 1.36.3 by @dependabot in #1191
- fix kernel_enhanced lack of pkg general by @lec-bit in #1199
- bump version to 1.1-dev by @hzxuzhonghu in #1197
- Bump github.com/cilium/ebpf from 0.16.0 to 0.17.1 by @hzxuzhonghu in #1205
- Update meeting in README.md by @hzxuzhonghu in #1196
- add workload metrics by @LiZhenCheng9527 in #1105
- Bump the k8s-io group with 5 updates by @dependabot in #1201
- Fix typos by @hzxuzhonghu in #1219
- Improve Authz UX: Immediate Feedback & Status Subcommand by @ravjot07 in #1217
- chore: solved contruct_tuple typo #1221 by @yp969803 in #1223
- Modify the kmeshctl documentation according to make gen by @LiZhenCheng9527 in #1241
- add LiZhenCheng9527 to OWNERS by @LiZhenCheng9527 in #1244
- Using global variable to control bpf log level by @hzxuzhonghu in #1206
- adapt BPF_LOG in route_config.h by @lec-bit in #1220
- fix bookinfo issue 553 by @weli-l in #1245
- add sample yamls for useguide by @weli-l in #1248
- Bump github.com/cilium/ebpf from 0.17.1 to 0.17.3 by @dependabot in #1237
- Bump the k8s-io group with 5 updates by @dependabot in #1250
- new kernel adapt by @lec-bit in #1198
- Bump github.com/safchain/ethtool from 0.5.9 to 0.5.10 by @dependabot in #1252
- Bump github.com/go-jose/go-jose/v3 from 3.0.3 to 3.0.4 in the go_modules group by @dependabot in #1255
- Bump github.com/prometheus/client_golang from 1.20.5 to 1.21.0 by @dependabot in #1254
- feat: dump authorizationPolicy by @yp969803 in #1222
- Bump google.golang.org/grpc from 1.69.4 to 1.70.0 by @dependabot in #1258
- Bump istio.io/api from 1.24.2 to 1.24.3 by @dependabot in #1259
- optimizie xdp auth by @weli-l in #1256
- adapt doc by @lec-bit in #1268
- enable auth offload by default by @weli-l in #1274
- adapt bpf2go files in new kernel by @lec-bit in #1273
- pretty ...
v1.1.0-alpha
This is the alpha version of Kmesh 1.1.0
Release v1.0.0
At the start of the New Year 2025, we are pleased to announce the release of Kmesh v1.0.0. We would like to express our sincere gratitude to the Kmesh community contributors for their dedication and hard work over the past three months.
In this v1.0.0 release, we have made significant improvements to Kmesh's east-west traffic governance capabilities, particularly in the area of security. We have added support for Encrypted Communication and Authorization, enhancing the overall security posture of the Kmesh.
Furthermore, we have introduced support for headless services and locality load balancing, providing users with more flexibility and control over their service. Additionally, we have added service topology diagrams and optimized observability features, empowering users with better visibility into their services in Kmesh.
In the Kernel-Native mode, we have support for advanced capabilities such as circuit breaker, rate limiting, fully restart support and more, ensuring a comprehensive and robust set of features for our users. In this release cycle, Kmesh has been updated to support Istio version 1.24, and we have addressed several critical bugs while also reconstructing many key components. To ensure the stability and reliability of the platform, we have significantly increased the coverage of end-to-end tests and unit tests. We are excited to share these improvements with the Kmesh community and look forward to continued collaboration and growth in the future.
Main Features
Encrypted Communication:
Data transmitted in an unencrypted state poses a significant security risk. To address this concern, Kmesh has introduced a node encryption mode in its v1.0.0 release. This feature utilizes IPsec to encrypt the communication traffic between nodes, effectively eliminating security risks during data transmission.
Kmesh leverages the encryption capabilities provided by IPsec to secure the traffic flowing between nodes. The data encryption behavior of IPsec is finely controlled through the use of Kubernetes Custom Resource Definitions (CRDs), empowering users with granular control over the encryption policies.
Furthermore, Kmesh has built its node-to-node information synchronization mechanism on top of the Kubernetes api-service. This integration ensures that the encryption configuration and key management are seamlessly integrated with the Kubernetes ecosystem, simplifying the deployment and management of the secure communication infrastructure.
Offload Authorization Execution into XDP Prog:
In Kmesh v0.5.0, the authorization functionality has been offloaded to XDP eBPF. This integration has been further enhanced in the Kmesh v1.0.0 release.
Kmesh now supports IP-based authorization, where the authorization rules are determined within the eBPF program. When the authorization results in a rejection, the XDP program simply drops the request, ensuring that the client can perceive the connection failure.
This efficient authorization mechanism provides a secure and high-performance solution for service-to-service communication. Kmesh will support more authorisation features in XDP in future releases. Welcome to provide the relevant requests of users.
Locality Load Balance:
Locality Load Balancing is a critical optimization for performance and reliability in distributed systems. By directing traffic to the nearest service instances, this approach reduces latency, enhances availability, and lowers the costs associated with cross-region data transfers.
Kmesh now supports Locality Load Balancing across five different scopes: region, zone, subZone, nodeName, and clusterId. This fine-grained control allows users to configure load balancing policies based on the most appropriate locality level for their specific deployment topology and requirements
Observability enhancement:
Kmesh v1.0.0 introduces enhanced observability features, including the addition of service topology maps. By integrating Kmesh with Kiali, users can now visualize their service topology in a clear and intuitive manner.
Furthermore, Kmesh has optimized the metrics labels, ensuring that the presented metrics are more reasonable and understandable. This improvement enhances the overall clarity and usability of Kmesh's observability data.
Leveraging the easy access to network-native data provided by eBPF, Kmesh has also introduced new metrics related to TTL (Time To Live) and retries. This provides users with deeper insights into the state of the network, enabling them to better understand and monitor the performance and reliability of their distributed applications.
Kernel-Native Mode Support Restart:
Kmesh has further enhanced its capabilities by introducing the ability to gracefully reload eBPF maps and programs after a system reboot. This feature ensures that there is no need to re-register namespaces or specific pods into Kmesh following a reboot.
As a result, the service traffic remains uninterrupted during the reboot process, providing a significant benefit to Kmesh customers. This seamless handling of reboot scenarios helps maintain the availability and stability of the service mesh, minimizing the impact on end-users and applications.
Support Rate Limit and Circuit Breaker:
Kmesh v1.0.0 introduces enhanced traffic governance capabilities for its Kernel-Native Mode. This latest release now supports local rate limiting and circuit breaker functionality, addressing the core traffic management requirements of service mesh deployments.
Users can leverage these traffic governance features in the Kernel-Native Mode in the same way they would utilize rate limiting and circuit breaker capabilities in the Dual-Engine Mode. This consistency across the different operating modes ensures a seamless experience for Kmesh operators, regardless of their chosen deployment architecture.
Support Headless Service and ServiceEntry:
In the Kmesh Dual-Engine Mode, support for headless services and service entries has now been introduced.
Adopt istio 1.24:
Kmesh has been supported for istio 1.24 and there are e2e tests to ensure the stability of Kmesh in istio 1.24. Currently Kmesh is compatible with istio versions 1.22, 1.23 and 1.24.
Critical Bug Fix:
We have also made some big bug fix:
1.Support hostname type waypoint in workload
2.Check whether pods should be managed when ns is managed
3.Fix bpf ins over 100w
What's Changed
- [release-0.5] update VERSION by @hzxuzhonghu in #896
- Fix build warning and remove docker pull explicitly by @hzxuzhonghu in #893
- Revert "update VERSION" by @hzxuzhonghu in #902
- support large scale for ads mode by @nlgwcy in #610
- Update kmesh_deploy_and_develop_in_kind.md by @derekwin in #904
- Enhanced kernel: Build 2 series of eBPF objects(kernel ver. <5.13 & >=5.13) and load eBPF dynamically when Kmesh starts up by @tacslon in #914
kmeshctl waypoint
framework by @YaoZengzeng in #901- fix slow restart in workload mode by @nlgwcy in #924
- infra(ci): update copyright check ci by @yuluo-yx in #932
- Bump google.golang.org/protobuf from 1.34.2 to 1.35.1 by @dependabot in #933
- Fix argument list too long in manage controller by @tacslon in #711
- upgrade golang to 1.23 by @tjons in #929
- Add maintainers file by @hzxuzhonghu in #934
- Update contributing guide by @hzxuzhonghu in #936
- implement
kmeshctl version
by @YaoZengzeng in #919 - Refactor bpf loader by @hzxuzhonghu in #864
- enable/disable accesslog with kmeshctl by @LiZhenCheng9527 in #935
- change
ads
->kernel-native
andworkload
->duel-engine
in code by @LiZhenCheng9527 in #940 - Add a github action to publish release assets automatically by @LiZhenCheng9527 in #938
- fix format error of
kmeshctl waypoint status --help
by @YaoZengzeng in #942 - rm depends/include/bpf_helper_defs_ext.h by @hzxuzhonghu in #916
- Fix github action arm image built by @hzxuzhonghu in #948
- Fix when bpf loader failed, it should have a chance to dettach bpf pr… by @hzxuzhonghu in #952
- Ignore key not exist error when delete bpf map by @hzxuzhonghu in #957
- if sourceworkload is nil, not print metrics by @LiZhenCheng9527 in #960
- check whether pods should be managed when ns is managed by @YaoZengzeng in #958
- adjust log level for userspace authz by @hzxuzhonghu in #959
- Bump istio dependencies by @hzxuzhonghu in #950
- install istioctl in tmp dir when running E2E tests by @YaoZengzeng in #974
- Fix MAINTAINERS file misspelling by @wbc6080 in #976
- E2E Enhancement Draft by @noobwei in https://github.com/kmesh-net/kme...
v1.0.0-rc.0
This is the rc version of Kmesh 1.0.0
v1.0.0-alpha
This is the alpha version of Kmesh 1.0.0
Release v0.5.0
We are so excited to announce Kmesh v0.5.0. First thanks to our contributors for their hard work over the last two months. In release v0.5.0 we have made a lot of great enhancements, including command line tool kmeshctl
, more complete E2E test coverage, better visualization of underlying eBPF information, observability enhancement, fully restart support, improve CNI installer, RBAC in XDP prog. In addition, in this release cycle, many critical bugs have been fixed, some key code has been refactored and more tests have been covered, making Kmesh more stable and robust. The highlights are as follows:
Zero-Down time during kmesh restart
It is amazing now Kmesh can gracefully reload eBPF map and prog after restart , and also no need to re-enroll namespaces or specific pods into kmesh after restart. As a result, the traffic flow is not interrupted during the restart, which is a big benefit to users. After kmesh-daemon restarted, the bpf map configurations will be automatically updated to date.
As early as release v0.4.0, after kmesh restarted it would require all the pods managed by kmesh to be restarted to be re-managed, because this kmesh manage is triggered by cni plug-in. Now it can be done in kmesh-daemon so that pods do not need to be restarted to be re-managed.
Observability enhancement
Now Kmesh supports L4 access log, allowing users to clearly visualize the traffic managed by Kmesh. Note that accesslog is not enabled by default. You can start the accesslog function by modifying the --enable-accesslog
parameter of spec.containers.args
in Kmesh. We will support using kmeshctl
to enable accesslog dynamiclly . At the same time, Grafana addon adapted for Kmesh has been added to better visualize monitoring metrics in various dimensions. Also some key issues were fixed in observability, effectively improving its accuracy and stability.
Offload authorization execution into XDP prog:
As early as release v0.3.0, Kmesh already supported L4 RBAC, but the previous solution was to do rbac in the user space, which had some issues in terms of performance and functionality. Now we have offloaded it into xdp eBPF, and this feature will be truly available.
Currently, authorization rules are moved down to the eBPF map, which provides the capability of performing authorization completely in the eBPF program. When the authz result is to reject, the XDP prog directly drops the request packet so that the client can detect the connection failure.
Better debugability
Added a command line tool kmeshctl
:
Kmesh has its own command line tool! Now you no longer need to exec
into the corresponding Kmesh daemon pod to adjust the log level of the Kmesh daemon or dump the configuration. You can directly use kmeshctl:
# Adjust kmesh-daemon log level (e.g., debug | error | info)
kmeshctl log kmesh-6ct4h --set default:debug
# Dump config
kmeshctl dump kmesh-6ct4h workload
More features will be added to kmeshctl in the future, allowing users to better manage and debug Kmesh.
Better visualization of underlying bpf map:
Previously we have interfaces /debug/config_dump/ads
and /debug/config_dump/workload
to output the config content cached in Kmesh daemon. Due to various reasons, the config in Kmesh daemon cache and the actual eBPF may not be completely consistent. If we can get human-readable eBPF info, it will be more helpful for us to troubleshoot. Now we can get it through interfaces /debug/bpf/*
.
It will also be integrated into kmeshctl later, making it easier to view. And can even be further expanded to determine whether the underlying eBPF is synchronized with the configuration in the Kmesh daemon.
Improve CNI installer:
As cni installer is Kmesh daemon, if the kmesh-daemon crash unexpectedly or the machine suddenly loses power, the CNI doesn’t have chance to uninstall the CNI config. If the kubeconfig’s token installed is expired, no pod can startup successfully after kmesh-daemon exit abnormally. So we make use of the following two methods to resolve:
- Do clean up the cni config at the end of
start_kmesh.sh
- Add a separate go routine in cni installer, update the kubeconfig file once the token file is modified. This can make sure the kubeconfig file does not expire easily.
Support hostnetwork workloads
Now for Kmesh Dual-Engine mode, we support accessing a service with hostnetwork pods.
Performance improvement
In dual-engine mode, we largely optimized the bpf map update during Workload
and Service
response handling by using local cache instead of looping over the bpf map.
Critical Bug Fix
We have also made some big bug fixes:
-
Prevent losing control of traffic during workload resource updates by not deleting the frontend map.
-
Traffic from mamespaced waypoint will be redirected to waypoint again, it falls into a dead loop. Now we skipped managing traffic sent from waypoint.
-
Fixed previously when waypoint processes non-HTTP tcp traffic, it would unexpectedly return HTTP/1.1 400 Bad Request. #681
What's Changed
Full Changelog
* kmesh route samples by @lec-bit in https://github.com//pull/531 * Kmesh Observability by @LiZhenCheng9527 in https://github.com//pull/527 * fix unexpected log by @Okabe-Rintarou-0 in https://github.com//pull/535 * Fix TestPodSidecarLabelChangeTriggersAddIptablesAction flake by @hzxuzhonghu in https://github.com//pull/540 * Modifybpf map update to prevent potential bugs by @weli-l in https://github.com//pull/541 * add codecov config by @LiZhenCheng9527 in https://github.com//pull/537 * use latest waypoint image to run e2e by @YaoZengzeng in https://github.com//pull/554 * add document for deploying and developing in kind by @Okabe-Rintarou-0 in https://github.com//pull/559 * add Copyright check by @LiZhenCheng9527 in https://github.com//pull/561 * add security.md for kmesh by @LiZhenCheng9527 in https://github.com//pull/564 * Add bpf log level getter (#560) by @Okabe-Rintarou-0 in https://github.com//pull/562 * add document about using enhanced kernel by @Okabe-Rintarou-0 in https://github.com//pull/565 * update gitignore for enhanced kernel by @Okabe-Rintarou-0 in https://github.com//pull/572 * Add code spell check github workflow by @Okabe-Rintarou-0 in https://github.com//pull/573 * add badge in readme by @LiZhenCheng9527 in https://github.com//pull/576 * Provide a way to allow setting all logger level to debug by @hzxuzhonghu in https://github.com//pull/557 * Fix `make gen` problem by @Okabe-Rintarou-0 in https://github.com//pull/582 * fix make clean by @Okabe-Rintarou-0 in https://github.com//pull/587 * add some waypoint related E2E test cases by @YaoZengzeng in https://github.com//pull/580 * optimize workload update by @nlgwcy in https://github.com//pull/590 * remove arch info in build process by @Okabe-Rintarou-0 in https://github.com//pull/585 * kmesh security: pod manage by @lec-bit in https://github.com//pull/489 * fix bpf map look up failed by @LiZhenCheng9527 in https://github.com//pull/594 * add configuration to collect kmesh metrics using Prometheus by @LiZhenCheng9527 in https://github.com//pull/589 * Bump the k8s-io group with 3 updates by @dependabot in https://github.com//pull/609 * waypoint should not managed by Kmesh by @LiZhenCheng9527 in https://github.com//pull/611 * remove resync period by @hzxuzhonghu in https://github.com//pull/601 * Fix DNS cluster's endpoint ip addr check by @LiZhenCheng9527 in https://github.com//pull/604 * E2E test cases for service and pod ip access by @YaoZengzeng in https://github.com//pull/596 * remove build arch in documents by @Okabe-Rintarou-0 in https://github.com//pull/622 * Bypass only for sidecar by @hzxuzhonghu in https://github.com//pull/607 * Bump github.com/containernetworking/cni from 1.2.2 to 1.2.3 by @dependabot in https://github.com//pull/624 * update metric_key with direction & dst_port by @nlgwcy in https://github.com//pull/627 * E2E test cases for waypoint management by @YaoZengzeng in https://github.com//pull/625 * Support ipv6 in e2e test by @noobwei in https://github.com//pull/621 * Make kmesh cni and manage controller consitent during pod enrollment by @hzxuzhonghu in https://github.com//pull/623 * kmesh support restart by reload old bpf map and prog by @lec-bit in https://github.com//pull/475 * enable select some e2e cases to run or skip some cases by @YaoZengzeng in https://github.com//pull/638 * copy bytes optimize by @hzxuzhonghu in https://github.com//pull/633 * preclude pod with host network to be managed by kmesh by @hzxuzhonghu in https://github.com//pull/634 * remove bypass from bpf prog by @hzxuzhonghu in https://github.com//pull/635 * Enable cleanup in e2e by @noobwei in https://github.com//pull/649 * Fix kmesh daemon graceful exit by @hzxuzhonghu in https://github.com//pull/651 * Fix TestPodSidecarLabelChangeTriggersAddIptablesAction flake by @hzxuzhonghu in https://github.com//pull/636 * Fixed bug in bpf where IPv4 destination address was stored as IPv6 by @LiZhenCheng9527 in https://github.com//pull/648 * add some secure compilation options by @kwb0523 in h...Kmesh v0.5.0-rc.0
This is a pre-release for v0.5.0
Kmesh v0.4.1
Bug Fixes:
- Fixed waypoint failed to process ipv4-mapped ipv6 connections #785
- Fixed service access when deploying namespace and service scoped waypoint in mixed manner #754
- Fixed load balancing when svc endpoint count is zero #685
- Fixed incorrectly handling ipv4 mapped ipv6 address #660
- Fixed DNS cluster's endpoint ip addr check #616
- Skip manage waypoint to prevent bpf prog fall into deadloop and interrupting traffic #611
- Fix TestPodSidecarLabelChangeTriggersAddIptablesAction flake #543
- Fixed workload update can cause traffic unmanaged by kmesh #592
Feature Enhancement:
- Add service metric and use bpf ringbuf to report metrics #710
v0.4.0
2024 Mid-Year Review: We are pleased to announce release v0.4.0 after more than two months's hard work from all our contributors. In release v0.4.0 we have done many enhancements such as refined pod management, management of DNS typed services, IPv6 protocol support, and data plane communication metrics monitoring. Additionally, improvements have been made in performance, maintainability, and testability, making Kmesh more robust. The key features included in this release are listed but not limited to the following:
Fine-Grained Namespace and Pod Manage:
In addition to namespace-level management, Kmesh now supports fine-grained pod-level management. You can manage pods using the following commands:
# Particular pod manage
kubectl label pod <podName> istio.io/dataplane-mode=kmesh -n {namespace}
# Namespace scoped pods manage
kubectl label ns <namespace> istio.io/dataplane-mode=kmesh
Supports IPv6 Communication in Workload Mode:
Kmesh now supports IPv6 communication, catering to both public and private cloud environments. Even for IPv4 services, Java-based services default to using the IPv6 protocol family. This enhancement ensures broader service management scenarios.
The ads mode will also support IPv6 in the near future, and also we will make Kmesh dual-stack compatible.
Performance Optimization for Rule Refresh in ads Mode:
Previously in the ads mode, Kmesh suffered from slow rule refresh because of map-in-map model used. This release, we significantly improves rule refresh performance by orders of magnitude, laying the groundwork for large-scale cluster management.
Fine-Grained Waypoint Traffic Capture:
To stay in sync with Istio 1.22, Kmesh now supports the latest workload API model. You can now use the “istio.io/use-waypoint” label to enable waypoint capture at the namespace, service, or pod level. This flexibility allows precise and on-demand use of waypoints.
DNS-Type Services Support:
In Kmesh’s ads mode, HTTP protocol is managed using eBPF and kernel modules, along with Listener, Route, and Cluster APIs. However, DNS-typed services posed a challenge because their clusters use domain names as endpoints, making DNS resolution during load balancing impossible within eBPF.
Kmesh introduces a DNS resolve module within the Kmesh daemon. It resolves domain names in user space and rewrites clusters's inlined endpoints to prevent resolving DNS early. As a result, Kmesh now supports Kubernetes ExternalName Services and Istio DNS Resolution ServiceEntry.
Observability Support:
Kmesh prioritizes observability as a critical feature in traffic management. It achieves low-cost connection monitoring using eBPF in the kernel and supports Prometheus-based metrics collection and aggregation. Future updates will enhance other observability capabilities, accesslog and traces.
To query monitoring information, use the following command:
kubectl exec -ti -n kmesh-system kmesh-6ct4h -- curl http://127.0.0.1:15020/status/metrics
Dynamic Log Level Adjustment:
Kmesh now allows dynamic adjustment of log levels for both the kmesh-daemon and eBPF prog. Now kmesh also redirect eBPF data plane logs to user space printing to avoid trace-pipe cost (requires kernel version 5.13 or higher).
# Adjust kmesh-daemon log level (e.g., debug | error | info)
kubectl exec -ti -n kmesh-system kmesh-6ct4h -- kmesh-daemon log --set default:debug
# Adjust kmesh eBPF data plane log level
kubectl exec -ti -n kmesh-system kmesh-6ct4h -- kmesh-daemon log --set bpf:debug
Additionally, there are many other improvements are there:
- Kmesh has introduced an E2E testing framework, ensuring that each PR submission undergoes E2E tests to prevent regressions.
- Reliability reconnections with Istiod have been addressed, and community documentation has been enriched.
What's Changed
- Update Readme by @hzxuzhonghu in #240
- Delete unnecessary code and configuration by @lec-bit in #262
- Fix dns typed service panic by @hzxuzhonghu in #263
- Enable xdp auth when kmesh works in workload mode by @hzxuzhonghu in #260
- Enable dependa bot by @hzxuzhonghu in #231
- Added benchmark test for cluster.Flush() by @LiZhenCheng9527 in #259
- Added benchmark test for listener.Flush() by @LiZhenCheng9527 in #253
- fix authz on server listening on both ipv4 and ipv6 port by @hzxuzhonghu in #272
- opt cluster ut test by @LiZhenCheng9527 in #267
- add yaml of sample applications by @LiZhenCheng9527 in #276
- rename service_id to upstream_id in frontend_value struct by @kwb0523 in #278
- add ut of client.go by @LiZhenCheng9527 in #213
- add clang-format by @hzxuzhonghu in #266
- bump istio to fix proto conflict by @hzxuzhonghu in #284
- opt readme by @LiZhenCheng9527 in #290
- remove duplicate NODE_NAME env by @hzxuzhonghu in #280
- Ignore expected grpc error by @hzxuzhonghu in #283
- Enable merge queue by @hzxuzhonghu in #293
- FIX bpf_map_delete_elem unexpected error log by @hzxuzhonghu in #287
- fix clang format tool not install by @LiZhenCheng9527 in #299
- Kmesh: repair kmesh manager not effect in sockops on oe 23.03 by @bitcoffeeiux in #301
- Refactor: remove global package vars by @hzxuzhonghu in #296
- Fix the issue of incorrect domain matching by @lec-bit in #286
- remove unnecessary port fields in frontend_key by @kwb0523 in #300
- Add proposal template by @hzxuzhonghu in #302
- Fix the Enhanced Compilation Failure Issue by @lec-bit in #309
- apply reviewers and approvers by @supercharge-xsy in #307
- Currently supporting TCP protocol, add the corresponding checks by @supercharge-xsy in #311
- Fix the Enhanced Compilation Failure Issue by @bfforever in #312
- add benchmark of workload by @LiZhenCheng9527 in #316
- optimization: Accelerates the performance of updating the xDS configuration by @nlgwcy in #326
- add optional of start bypass by @LiZhenCheng9527 in #306
- update listener benchmark by @LiZhenCheng9527 in #328
- Update Readme by @hzxuzhonghu in #327
- only update by_key policy_store for workloadp scope authz by @supercharge-xsy in #333
- chore: add kmesh-system namespace api object in kmesh.yaml by @Okabe-Rintarou-0 in #334
- fix bpf-map test failed by @nlgwcy in #341
- Split the ebpf code directory of ads and workload. by @bfforever in #343
- deserialization module optimization by @nlgwcy in #331
- fix: should get authz policy from dst-worklod when rbac by @supercharge-xsy in #336
- Fix some typo errors of some documents by @Okabe-Rintarou-0 in #345
- update workload.proto by @kwb0523 in #346
- use ztunnel-role with workload mode, fix authz policy not work when update by @supercharge-xsy in #344
- docs: translate xds_handle.md to Chinese by @Okabe-Rintarou-0 in #354
- fix: fix issue template by @Okabe-Rintarou-0 in #357
- feat: support kmesh daemon version command(#198) by @Okabe-Rintarou-0 in #356
- fix: make clean does not clean up completely(#264) by @Okabe-Rintarou-0 in #338
- Added dns resolver proposal by @hzxuzhonghu in #305
- Certificate subscription management module to kmesh by @lec-bit in #310
- Monitoring code for UT coverage in github CI using codecov by @LiZhenCheng9527 in #360
- add ut of workload_stream.go by @LiZhenCheng9527 in #204
- add ut of ads_loader.go by @LiZhenCheng9527 in #183
- Improve bpf log and bugs fix by @bfforever in #364
- Fix status server panic by @hzxuzhonghu in #350
- Fix cleanup kmesh-cni failed bug by @bfforever in #367
- add bypass logic for kmesh by @weli-l in #297
- fix: make clean error(#376) by @Okabe-Rintarou-0 in #378
- some workload mode bugfix and waypoint optimization by @kwb0523 in #361...