<h2id="issues-with-btrfs">Issues with BTRFS<aclass="headerlink"href="#issues-with-btrfs"title="Permanent link">¶</a></h2>
<ul>
<li>As <ahref="https://github.com/jaredallard">@jaredallard</a><ahref="https://github.com/rancher/k3d/pull/48">pointed out</a>, people running <code>k3d</code> on a system with <strong>btrfs</strong>, may need to mount <code>/dev/mapper</code> into the nodes for the setup to work.</li>
<li>This will do: <code>k3d create cluster CLUSTER_NAME -v /dev/mapper:/dev/mapper</code></li>
<li>This will do: <code>k3d cluster create CLUSTER_NAME -v /dev/mapper:/dev/mapper</code></li>
</ul>
<h2id="issues-with-zfs">Issues with ZFS<aclass="headerlink"href="#issues-with-zfs"title="Permanent link">¶</a></h2>
<ul>
<li>
<p>k3s currently has <ahref="ttps://github.com/rancher/k3s/issues/66">no support for ZFS</a> and thus, creating multi-master setups (e.g. <code>k3d create cluster multimaster --masters 3</code>) fails, because the initializing master node (server flag <code>--cluster-init</code>) errors out with the following log:
<p>k3s currently has <ahref="ttps://github.com/rancher/k3s/issues/66">no support for ZFS</a> and thus, creating multi-master setups (e.g. <code>k3d cluster create multimaster --masters 3</code>) fails, because the initializing master node (server flag <code>--cluster-init</code>) errors out with the following log:
<divclass="highlight"><pre><span></span><code>starting kubernetes: preparing server: start cluster and https: raft_init<spanclass="o">()</span>: io: create I/O capabilities probe file: posix_allocate: operation not supported on socket
</code></pre></div></p>
</li>
@ -651,7 +651,7 @@
<li>Possible <ahref="https://github.com/rancher/k3d/issues/133#issuecomment-549065666">fix/workaround by @zer0def</a>:<ul>
<li>use a docker storage driver which cleans up properly (e.g. overlay2)</li>
<li>clean up or expand docker root filesystem</li>
<li>change the kubelet’s eviction thresholds upon cluster creation: <code>k3d create cluster --k3s-agent-arg '--kubelet-arg=eviction-hard=imagefs.available<1%,nodefs.available<1%' --k3s-agent-arg '--kubelet-arg=eviction-minimum-reclaim=imagefs.available=1%,nodefs.available=1%'</code></li>
<li>change the kubelet’s eviction thresholds upon cluster creation: <code>k3d cluster create --k3s-agent-arg '--kubelet-arg=eviction-hard=imagefs.available<1%,nodefs.available<1%' --k3s-agent-arg '--kubelet-arg=eviction-minimum-reclaim=imagefs.available=1%,nodefs.available=1%'</code></li>
</ul>
</li>
</ul>
@ -671,7 +671,7 @@
<divclass="md-source-date">
<small>
Last update: <spanclass="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">June 4, 2020</span>
Last update: <spanclass="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 14, 2020</span>
<p>Get the new cluster’s connection details merged into your default kubeconfig (usually specified using the <code>KUBECONFIG</code> environment variable or the default path <codeclass="highlight"><spanclass="nv">$HOME</span>/.kube/config</code>) and directly switch to the new context:</p>
<divclass="highlight"><pre><span></span><code>k3d get kubeconfig mycluster --switch
cluster <spanclass="o">[</span>CLUSTERNAME<spanclass="o">]</span><spanclass="c1"># default cluster name is 'k3s-default'</span>
--version <spanclass="c1"># show k3d and k3s version</span>
-h, --help <spanclass="c1"># show help text</span>
version <spanclass="c1"># show k3d and k3s version</span>
<spanclass="nb">help</span><spanclass="o">[</span>COMMAND<spanclass="o">]</span><spanclass="c1"># show help text for any command</span>
completion <spanclass="o">[</span>bash <spanclass="p">|</span> zsh <spanclass="p">|</span><spanclass="o">(</span>psh <spanclass="p">|</span> powershell<spanclass="o">)]</span><spanclass="c1"># generate completion scripts for common shells</span>
cluster <spanclass="o">[</span>CLUSTERNAME<spanclass="o">]</span><spanclass="c1"># default cluster name is 'k3s-default'</span>
create
-a, --api-port <spanclass="c1"># specify the port on which the cluster will be accessible (e.g. via kubectl)</span>
-i, --image <spanclass="c1"># specify which k3s image should be used for the nodes</span>
--k3s-agent-arg <spanclass="c1"># add additional arguments to the k3s agent (see https://rancher.com/docs/k3s/latest/en/installation/install-options/agent-config/#k3s-agent-cli-help)</span>
get <spanclass="o">(</span>CLUSTERNAME <spanclass="o">[</span>CLUSTERNAME ...<spanclass="o">]</span><spanclass="p">|</span> --all<spanclass="o">)</span><spanclass="c1"># get kubeconfig from cluster(s) and save it into a file in $HOME/.k3d</span>
-a, --all <spanclass="c1"># get kubeconfigs from all clusters</span>
merge <spanclass="o">(</span>CLUSTERNAME <spanclass="o">[</span>CLUSTERNAME ...<spanclass="o">]</span><spanclass="p">|</span> --all<spanclass="o">)</span><spanclass="c1"># get kubeconfig from cluster(s) and merge it/them into an existing kubeconfig</span>
-a, --all <spanclass="c1"># get kubeconfigs from all clusters</span>
--output <spanclass="c1"># specify the output file where the kubeconfig should be written to</span>
-d, --merge-default-kubeconfig <spanclass="c1"># update the default kubeconfig (usually $KUBECONFIG or $HOME/.kube/config)</span>
load
image <spanclass="o">[</span>IMAGE <spanclass="p">|</span> ARCHIVE <spanclass="o">[</span>IMAGE <spanclass="p">|</span> ARCHIVE ...<spanclass="o">]]</span><spanclass="c1"># Load one or more images from the local runtime environment or tar-archives into k3d clusters</span>
image
import<spanclass="o">[</span>IMAGE <spanclass="p">|</span> ARCHIVE <spanclass="o">[</span>IMAGE <spanclass="p">|</span> ARCHIVE ...<spanclass="o">]]</span><spanclass="c1"># Load one or more images from the local runtime environment or tar-archives into k3d clusters</span>
-c, --cluster <spanclass="c1"># clusters to load the image into</span>
-k, --keep-tarball <spanclass="c1"># do not delete the image tarball from the shared volume after completion</span>
@ -624,7 +624,7 @@ Therefore, we have to create the cluster in a way, that the internal port 80 (wh
</li>
<li>
<p>Get the kubeconfig file</p>
<p><codeclass="highlight"><spanclass="nb">export</span><spanclass="nv">KUBECONFIG</span><spanclass="o">=</span><spanclass="s2">"</span><spanclass="k">$(</span>k3d get kubeconfig k3s-default<spanclass="k">)</span><spanclass="s2">"</span></code></p>
<p><codeclass="highlight"><spanclass="nb">export</span><spanclass="nv">KUBECONFIG</span><spanclass="o">=</span><spanclass="s2">"</span><spanclass="k">$(</span>k3d kubeconfig get k3s-default<spanclass="k">)</span><spanclass="s2">"</span></code></p>
</li>
<li>
<p>Create a nginx deployment</p>
@ -663,7 +663,7 @@ Therefore, we have to create the cluster in a way, that the internal port 80 (wh
<ol>
<li>
<p>Create a cluster, mapping the port 30080 from worker-0 to localhost:8082</p>
<li>Note: Kubernetes’ default NodePort range is <ahref="https://kubernetes.io/docs/concepts/services-networking/service/#nodeport"><code>30000-32767</code></a></li>
</ul>
@ -705,7 +705,7 @@ Therefore, we have to create the cluster in a way, that the internal port 80 (wh
<divclass="md-source-date">
<small>
Last update: <spanclass="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">June 18, 2020</span>
Last update: <spanclass="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 14, 2020</span>
<p>This file is a regular <ahref="https://rancher.com/docs/k3s/latest/en/installation/private-registry/">k3s registries configuration file</a>, and looks like this:</p>
<p>and then mount it at <code>/var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl</code> (where <code>containerd</code> in your k3d nodes will load it) when creating the k3d cluster:</p>
@ -623,27 +623,27 @@ To get a kubeconfig set up for you to connect to a k3d cluster, you can go diffe
<h2id="getting-the-kubeconfig-for-a-newly-created-cluster">Getting the kubeconfig for a newly created cluster<aclass="headerlink"href="#getting-the-kubeconfig-for-a-newly-created-cluster"title="Permanent link">¶</a></h2>
<ol>
<li>Create a new kubeconfig file <strong>after</strong> cluster creation<ul>
<li><codeclass="highlight">k3d get kubeconfig mycluster</code></li>
<li><codeclass="highlight">k3d kubeconfig get mycluster</code></li>
<li><em>Note:</em> this will create (or update) the file <code>$HOME/.k3d/kubeconfig-mycluster.yaml</code></li>
<li><em>Tip:</em> Use it: <codeclass="highlight"><spanclass="nb">export</span><spanclass="nv">KUBECONFIG</span><spanclass="o">=</span><spanclass="k">$(</span>k3d get kubeconfig mycluster<spanclass="k">)</span></code></li>
<li><em>Tip:</em> Use it: <codeclass="highlight"><spanclass="nb">export</span><spanclass="nv">KUBECONFIG</span><spanclass="o">=</span><spanclass="k">$(</span>k3d kubeconfig get mycluster<spanclass="k">)</span></code></li>
</ul>
</li>
<li>Update your default kubeconfig <strong>upon</strong> cluster creation<ul>
<li><em>Note:</em> this won’t switch the current-context</li>
</ul>
</li>
@ -655,13 +655,13 @@ To get a kubeconfig set up for you to connect to a k3d cluster, you can go diffe
<pclass="admonition-title">Switching the current context</p>
<p>None of the above options switch the current-context by default.
This is intended to be least intrusive, since the current-context has a global effect.
You can switch the current-context directly with the <code>get kubeconfig</code> command by adding the <code>--switch-context</code> flag.</p>
You can switch the current-context directly with the <code>kubeconfig merge</code> command by adding the <code>--switch-context</code> flag.</p>
</div>
<h2id="removing-cluster-details-from-the-kubeconfig">Removing cluster details from the kubeconfig<aclass="headerlink"href="#removing-cluster-details-from-the-kubeconfig"title="Permanent link">¶</a></h2>
<p><codeclass="highlight">k3d delete cluster mycluster</code> will always remove the details for <code>mycluster</code> from the default kubeconfig.
<p><codeclass="highlight">k3d cluster delete mycluster</code> will always remove the details for <code>mycluster</code> from the default kubeconfig.
It will also delete the respective kubeconfig file in <code>$HOME/.k3d/</code> if it exists.</p>
<p><code>k3d get kubeconfig</code> let’s you specify one or more clusters via arguments <em>or</em> all via <code>--all</code>.
<p><code>k3d kubeconfig merge</code> let’s you specify one or more clusters via arguments <em>or</em> all via <code>--all</code>.
All kubeconfigs will then be merged into a single file if <code>--merge-default-kubeconfig</code> or <code>--output</code> is specified.
If none of those two flags was specified, a new file will be created per cluster and the merged path (e.g. <code>$HOME/.k3d/kubeconfig-cluster1.yaml:$HOME/.k3d/cluster2.yaml</code>) will be returned.
Note, that with multiple cluster specified, the <code>--switch-context</code> flag will change the current context to the cluster which was last in the list.</p>
@ -673,7 +673,7 @@ Note, that with multiple cluster specified, the <code>--switch-context</code> fl
<divclass="md-source-date">
<small>
Last update: <spanclass="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">June 22, 2020</span>
Last update: <spanclass="git-revision-date-localized-plugin git-revision-date-localized-plugin-date">July 14, 2020</span>
<p>Create a cluster with 3 master nodes using k3s’ embedded dqlite database.
The first master to be created will use the <code>--cluster-init</code> flag and k3d will wait for it to be up and running before creating (and connecting) the other master nodes.</p>
<h2id="adding-master-nodes-to-a-running-cluster">Adding master nodes to a running cluster<aclass="headerlink"href="#adding-master-nodes-to-a-running-cluster"title="Permanent link">¶</a></h2>
<p>In theory (and also in practice in most cases), this is as easy as executing the following command:</p>
iwilltry42
4 years ago