mirror of https://github.com/k3d-io/k3d
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
196 lines
8.7 KiB
196 lines
8.7 KiB
3 years ago
|
# Using Image Registries
|
||
4 years ago
|
|
||
|
## Registries configuration file
|
||
|
|
||
4 years ago
|
You can add registries by specifying them in a `registries.yaml` and referencing it at creation time:
|
||
|
`#!bash k3d cluster create mycluster --registry-config "/home/YOU/my-registries.yaml"`.
|
||
|
|
||
4 years ago
|
This file is a regular [k3s registries configuration file](https://rancher.com/docs/k3s/latest/en/installation/private-registry/), and looks like this:
|
||
|
|
||
|
```yaml
|
||
|
mirrors:
|
||
|
"my.company.registry:5000":
|
||
|
endpoint:
|
||
|
- http://my.company.registry:5000
|
||
|
```
|
||
|
|
||
3 years ago
|
In this example, an image with a name like `my.company.registry:5000/nginx:latest` would be _pulled_ from the registry running at `http://my.company.registry:5000`.
|
||
4 years ago
|
|
||
|
This file can also be used for providing additional information necessary for accessing some registries, like [authentication](#authenticated-registries) and [certificates](#secure-registries).
|
||
|
|
||
4 years ago
|
### Registries Configuration File embedded in k3d's SimpleConfig
|
||
|
|
||
|
If you're using a `SimpleConfig` file to configure your k3d cluster, you may as well embed the registries.yaml in there directly:
|
||
|
|
||
|
```yaml
|
||
3 years ago
|
apiVersion: k3d.io/v1alpha3
|
||
4 years ago
|
kind: Simple
|
||
|
name: test
|
||
|
servers: 1
|
||
|
agents: 2
|
||
|
registries:
|
||
3 years ago
|
create:
|
||
|
name: myregistry
|
||
4 years ago
|
config: |
|
||
|
mirrors:
|
||
|
"my.company.registry":
|
||
|
endpoint:
|
||
|
- http://my.company.registry:5000
|
||
|
```
|
||
|
|
||
3 years ago
|
Here, the config for the k3d-managed registry, created by the `create: {...}` option will be merged with the config specified under `config: |`.
|
||
4 years ago
|
|
||
4 years ago
|
### Authenticated registries
|
||
|
|
||
|
When using authenticated registries, we can add the _username_ and _password_ in a
|
||
|
`configs` section in the `registries.yaml`, like this:
|
||
|
|
||
|
```yaml
|
||
|
mirrors:
|
||
|
my.company.registry:
|
||
|
endpoint:
|
||
|
- http://my.company.registry
|
||
|
|
||
|
configs:
|
||
|
my.company.registry:
|
||
|
auth:
|
||
|
username: aladin
|
||
|
password: abracadabra
|
||
|
```
|
||
|
|
||
|
### Secure registries
|
||
|
|
||
|
When using secure registries, the [`registries.yaml` file](#registries-file) must include information about the certificates. For example, if you want to use images from the secure registry running at `https://my.company.registry`, you must first download a CA file valid for that server and store it in some well-known directory like `${HOME}/.k3d/my-company-root.pem`.
|
||
|
|
||
3 years ago
|
Then you have to mount the CA file in some directory in the nodes in the cluster and include that mounted file in a `configs` section in the [`registries.yaml` file](#registries-file).
|
||
4 years ago
|
For example, if we mount the CA file in `/etc/ssl/certs/my-company-root.pem`, the `registries.yaml` will look like:
|
||
|
|
||
|
```yaml
|
||
|
mirrors:
|
||
|
my.company.registry:
|
||
|
endpoint:
|
||
|
- https://my.company.registry
|
||
|
|
||
|
configs:
|
||
|
my.company.registry:
|
||
|
tls:
|
||
|
# we will mount "my-company-root.pem" in the /etc/ssl/certs/ directory.
|
||
|
ca_file: "/etc/ssl/certs/my-company-root.pem"
|
||
|
```
|
||
|
|
||
|
Finally, we can create the cluster, mounting the CA file in the path we specified in `ca_file`:
|
||
|
|
||
3 years ago
|
```bash
|
||
|
k3d cluster create \
|
||
|
--volume "${HOME}/.k3d/my-registries.yaml:/etc/rancher/k3s/registries.yaml" \
|
||
|
--volume "${HOME}/.k3d/my-company-root.pem:/etc/ssl/certs/my-company-root.pem"
|
||
|
```
|
||
4 years ago
|
|
||
|
## Using a local registry
|
||
|
|
||
4 years ago
|
### Using k3d-managed registries
|
||
4 years ago
|
|
||
4 years ago
|
#### Create a dedicated registry together with your cluster
|
||
|
|
||
3 years ago
|
1. `#!bash k3d cluster create mycluster --registry-create mycluster-registry`: This creates your cluster `mycluster` together with a registry container called `mycluster-registry`
|
||
3 years ago
|
|
||
|
- k3d sets everything up in the cluster for containerd to be able to pull images from that registry (using the `registries.yaml` file)
|
||
|
- the port, which the registry is listening on will be mapped to a random port on your host system
|
||
|
|
||
3 years ago
|
2. Check the k3d command output or `#!bash docker ps -f name=mycluster-registry` to find the exposed port (let's use `12345` here)
|
||
|
3. Pull some image (optional) `#!bash docker pull alpine:latest`, re-tag it to reference your newly created registry `#!bash docker tag alpine:latest mycluster-registry:12345/testimage:local` and push it `#!bash docker push mycluster-registry:12345/testimage:local`
|
||
|
4. Use kubectl to create a new pod in your cluster using that image to see, if the cluster can pull from the new registry: `#!bash kubectl run --image mycluster-registry:12345/testimage:local testimage --command -- tail -f /dev/null` (creates a container that will not do anything but keep on running)
|
||
4 years ago
|
|
||
|
#### Create a customized k3d-managed registry
|
||
|
|
||
4 years ago
|
1. `#!bash k3d registry create myregistry.localhost --port 12345` creates a new registry called `k3d-myregistry.localhost` (could be used with automatic resolution of `*.localhost`, see next section - also, **note the `k3d-` prefix** that k3d adds to all resources it creates)
|
||
|
2. `#!bash k3d cluster create newcluster --registry-use k3d-myregistry.localhost:12345` (make sure you use the **`k3d-` prefix** here) creates a new cluster set up to us that registry
|
||
4 years ago
|
3. continue with step 3 and 4 from the last section for testing
|
||
|
|
||
4 years ago
|
<!-- Admonition to describe usage of a non-k3d-managed registry -->
|
||
4 years ago
|
### Using your own (not k3d-managed) local registry
|
||
4 years ago
|
|
||
4 years ago
|
*We recommend using a k3d-managed registry, as it plays nicely together with k3d clusters, but here's also a guide to create your own (not k3d-managed) registry, if you need features or customizations, that k3d does not provide:*
|
||
4 years ago
|
|
||
4 years ago
|
??? nonk3dregistry "Using your own (not k3d-managed) local registry"
|
||
|
|
||
|
You can start your own local registry it with some `docker` commands, like:
|
||
4 years ago
|
|
||
4 years ago
|
```bash
|
||
|
docker volume create local_registry
|
||
|
docker container run -d --name registry.localhost -v local_registry:/var/lib/registry --restart always -p 5000:5000 registry:2
|
||
|
```
|
||
|
|
||
3 years ago
|
These commands will start your registry container with name and port `registry.localhost:5000`. In order to push to this registry, you will need to make it accessible as described in the next section.
|
||
4 years ago
|
Once your registry is up and running, we will need to add it to your `registries.yaml` configuration file.
|
||
|
Finally, you have to connect the registry network to the k3d cluster network: `#!bash docker network connect k3d-k3s-default registry.localhost`. And then you can [test your local registry](#testing-your-registry).
|
||
4 years ago
|
|
||
|
### Pushing to your local registry address
|
||
|
|
||
3 years ago
|
As per the guide above, the registry will be available as `registry.localhost:5000`.
|
||
4 years ago
|
|
||
|
All the nodes in your k3d cluster can resolve this hostname (thanks to the DNS server provided by the Docker daemon) but, in order to be able to push to this registry, this hostname also has to be resolved by your host.
|
||
4 years ago
|
|
||
4 years ago
|
!!! info "nss-myhostname to resolve `*.localhost`"
|
||
|
Luckily (for Linux users), [NSS-myhostname](http://man7.org/linux/man-pages/man8/nss-myhostname.8.html) ships with many Linux distributions
|
||
|
and should resolve `*.localhost` automatically to `127.0.0.1`.
|
||
|
Otherwise, it's installable using `sudo apt install libnss-myhostname`.
|
||
4 years ago
|
|
||
4 years ago
|
If your system does not provide/support tools that can auto-resolve specific names to `127.0.0.1`, you can manually add an entry in your `/etc/hosts` (`c:\windows\system32\drivers\etc\hosts` on Windows) file like this:
|
||
4 years ago
|
|
||
|
```bash
|
||
4 years ago
|
127.0.0.1 k3d-registry.localhost
|
||
4 years ago
|
```
|
||
|
|
||
3 years ago
|
!!! info "Just use localhost"
|
||
|
Alternatively, if you don't care about pretty names, just push directly to `localhost:5000` (or whatever port you used) and it will work.
|
||
|
If you later pull the image from the registry, only the repository path (e.g. `myrepo/myimage:mytag` in `registry.localhost:5000/myrepo/myimage:mytag`) matters to find your image in the targeted registry.
|
||
4 years ago
|
|
||
4 years ago
|
## Testing your registry
|
||
4 years ago
|
|
||
|
You should test that you can
|
||
|
|
||
4 years ago
|
- push to your registry from your local development machine.
|
||
|
- use images from that registry in `Deployments` in your k3d cluster.
|
||
4 years ago
|
|
||
3 years ago
|
We will verify these two things for a local registry (located at `k3d-registry.localhost:12345`) running in your development machine.
|
||
|
Things would be basically the same for checking an external registry, but some additional configuration could be necessary in your local machine when using an authenticated or secure registry (please refer to Docker's documentation for this).
|
||
4 years ago
|
|
||
|
First, we can download some image (like `nginx`) and push it to our local registry with:
|
||
|
|
||
4 years ago
|
```bash
|
||
4 years ago
|
docker pull nginx:latest
|
||
4 years ago
|
docker tag nginx:latest k3d-registry.localhost:5000/nginx:latest
|
||
|
docker push k3d-registry.localhost:5000/nginx:latest
|
||
4 years ago
|
```
|
||
|
|
||
|
Then we can deploy a pod referencing this image to your cluster:
|
||
|
|
||
4 years ago
|
```bash
|
||
4 years ago
|
cat <<EOF | kubectl apply -f -
|
||
|
apiVersion: apps/v1
|
||
|
kind: Deployment
|
||
|
metadata:
|
||
|
name: nginx-test-registry
|
||
|
labels:
|
||
|
app: nginx-test-registry
|
||
|
spec:
|
||
|
replicas: 1
|
||
|
selector:
|
||
|
matchLabels:
|
||
|
app: nginx-test-registry
|
||
|
template:
|
||
|
metadata:
|
||
|
labels:
|
||
|
app: nginx-test-registry
|
||
|
spec:
|
||
|
containers:
|
||
|
- name: nginx-test-registry
|
||
4 years ago
|
image: k3d-registry.localhost:12345/nginx:latest
|
||
4 years ago
|
ports:
|
||
|
- containerPort: 80
|
||
|
EOF
|
||
|
```
|
||
|
|
||
|
Then you should check that the pod is running with `kubectl get pods -l "app=nginx-test-registry"`.
|