- logs: really final output when creating/deleting nodes (so far, we were not outputting a final success message and the process was still doing stuff) (#640)
- fix an issue where the cluster creation would stall waiting for the `starting worker processes` log message from the loadbalancer/serverlb
- this was likely caused by a rounding issue when asking docker to get the container logs starting at a specific timestamp
- we now drop subsecond precision for this to avoid the rounding issue, which was confirmed to work
- see issues #592ɭ
### Misc
- to debug the issue mentioned above, we introduced a new environment variable `K3D_LOG_NODE_WAIT_LOGS`, which can be set to a list of node roles (e.g. `K3D_LOG_NODE_WAIT_LOGS=loadbalancer,agent`) to output the container logs that k3d inspects
- cgroupv2 support: to properly work on cgroupv2 systems, k3s has to move all the processes from the root cgroup to a new /init cgroup and enable subtree_control
- this is going to be included in the k3s agent code directly (<https://github.com/k3s-io/k3s/pull/3242>)
- for now we're overriding the container entrypoint with a script that does this (#579, compare <https://github.com/k3s-io/k3s/pull/3237>)
- thanks a lot for all the input and support @AkihiroSuda
- **Usage**: set the environment variable `K3D_FIX_CGROUPV2` to a `true` value before/when creating a cluster with k3d
- k3d-proxy: rename udp upstreams to avoid collisions/duplicates (#564)
### Features
- add *hidden* command `k3d runtime-info` used for debugging (#553)
- this comes with some additions on package/runtime level
- add *experimental*`--subnet` flag to get some k3d IPAM to ensure that server nodes keep static IPs across restarts (#560)
### Misc
- docs: fix typo (#556, @gcalmettes)
- docs: fix typo (#561, @alechartung)
- ci/drone: pre-release on `-dev.X` tags
- ci/drone: always build no matter the branch name (just not release)
- docs: add automatic command tree generation via cobra (#562)
- makefile: use `go env gopath` as install target for tools (as per #445)
- JSONSchema: add some examples and defaults (now also available via <https://raw.githubusercontent.com/rancher/k3d/main/pkg/config/v1alpha2/schema.json> in your IDE)
- use the new `//go:embed` feature to directly embed the jsonschema in the binary (#529)
- Add a status column to `k3d registry list` output (#496, @ebr)
- Allow non-prefixed (i.e. without `k3d-` prefix) user input when fetching resources (e.g. `k3d node get mycluster-server-0` would return successfully)
### Fixes
- Allow absolute paths for volumes on Windows (#510, @markrexwinkel)
- fix nil-pointer exception in case of non-existent IPAM network config
- Properly handle combinations of host/hostIP in kubeAPI settings reflected in the kubeconfig (#500, @fabricev)
### Misc
- docs: fix typo in stop command help text (#513, @searsaw)
- ci/ghaction: AUR (pre-)release now on Ubuntu 20.04 and latest archlinux image
- REMOVE incomplete and unused `containerd` runtime from codebase, as it was causing issues to build for windows and hasn't made any progress in quite some time now
- Using Viper brings us lots of nice features, but also one problem:
- We had to switch StringArray flags to StringSlice flags, which
- allow to use multiple flag values comma-separated in a single flag, but also
- split flag values that contain a comma into separate parts (and we cannot handle issues that arise due to this)
- so if you rely on commas in your flag values (e.g. for `--env X=a,b,c`), please consider filing an issue or supporting <https://github.com/spf13/viper/issues/246> and <https://github.com/spf13/viper/pull/398>
-`--env X=a,b,c` would be treated the same as `--env X=a`, `--env b`, `--env c`
- Communicate managed registry using the LocalRegistryHostingV1 spec from [KEP-1755](https://github.com/kubernetes/enhancements/blob/0d69f7cea6fbe73a7d70fab569c6898f5ccb7be0/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry/README.md)
- interesting especially for tools that reload images, like Tilt or Skaffold
- **Note**: The config file format (& feature) might still be a little rough around the edges and it's prone to change quickly until we hit a stable release of the config