Move out DCO

Signed-off-by: Alex Ellis (OpenFaaS Ltd) <alexellis2@gmail.com>

.DEREK.yml

@@ -1,10 +1,11 @@
 maintainers:
  - alexellis
- - rgee0
+ - Waterdrips
+ - utsavanand2
 
 features:
  - dco_check
  - comments
  - pr_description_required
 
-contributing_url: https://github.com/alexellis/k8s-on-raspbian/blob/master/README.md#contributions
+contributing_url: https://github.com/alexellis/k8s-on-raspbian/blob/master/CONTRIBUTING.md

.github/ISSUE_TEMPLATE/add-a-reader-s-cluster.md

@@ -0,0 +1,26 @@
+---
+name: Add a Reader's Cluster
+about: Describe this issue template's purpose here.
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+## Reader's cluster request
+
+My cluster is:
+
+- [ ] A blog post
+- [ ] A photo Tweet
+
+I have given attribution and linked / referenced any blog posts or images that I've used as part of the submission
+
+- [ ] Yes
+- [ ] No
+
+Please give the URL below:
+
+Title for blog post or Tweet:
+
+Your name:

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,45 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+<!--- Provide a general summary of the issue in the Title above -->
+
+## Expected Behaviour
+<!--- If you're describing a bug, tell us what should happen -->
+<!--- If you're suggesting a change/improvement, tell us how it should work -->
+
+## Current Behaviour
+<!--- If describing a bug, tell us what happens instead of the expected behavior -->
+<!--- If suggesting a change/improvement, explain the difference from current behavior -->
+
+## Possible Solution
+<!--- Not obligatory, but suggest a fix/reason for the bug, -->
+<!--- or ideas how to implement the addition or change -->
+
+## Steps to Reproduce (for bugs)
+<!--- Provide a link to a live example, or an unambiguous set of steps to -->
+<!--- reproduce this bug. Include code to reproduce, if relevant -->
+1.
+2.
+3.
+4.
+
+## Context
+<!--- How has this issue affected you? What are you trying to accomplish? -->
+<!--- Providing context helps us come up with a solution that is most useful in the real world -->
+
+## Your Environment
+
+<!--- Include as many relevant details about the environment you experienced the bug in -->
+* Docker version `docker version` (e.g. Docker 17.0.05 ):
+
+* What version of Kubernetes are you using? `kubectl version`:
+
+* Operating System and version (e.g. Linux, Windows, MacOS):
+
+* What ARM or Raspberry Pi board are you using?

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,33 @@
+<!--- Provide a general summary of your changes in the Title above -->
+
+## Description
+<!--- Describe your changes in detail -->
+
+## Motivation and Context
+<!--- Why is this change required? What problem does it solve? -->
+<!--- If it fixes an open issue, please link to the issue here. -->
+- [ ] I have raised an issue to propose this change ([required](https://github.com/alexellis/k8s-on-raspbian#contributions))
+
+
+## How Has This Been Tested?
+<!--- Please describe in detail how you tested your changes. -->
+<!--- Include details of your testing environment, and the tests you ran to -->
+<!--- see how your change affects other areas of the code, etc. -->
+
+## Types of changes
+<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
+- [ ] Bug fix (non-breaking change which fixes an issue)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to change)
+
+## Checklist:
+<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
+<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
+- [ ] My code follows the code style of this project.
+- [ ] I have tested this change and prove it works
+- [ ] I've read the [CONTRIBUTION](https://github.com/alexellis/k8s-on-raspbian#contributions) guide
+
+## Required for merge:
+
+- [ ] I have signed-off my commits, fix with: `git commit --amend --signoff`
+- [ ] My PR only contains one commit, fix with: `git rebase -i HEAD~N`, where `N` = number of commits.

CONTRIBUTING.md

@@ -0,0 +1,48 @@
+
+### Contributions
+
+Contributions are welcome, but must be tested and approved before being worked on. In practice, this means opening an issue before sending a PR for a change.
+
+Please make sure each commit is signed off with `git commit -s` (this means don't edit in the GitHub UI). 
+
+See below for more information.
+
+```
+Developer Certificate of Origin
+Version 1.1
+
+Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
+1 Letterman Drive
+Suite D4700
+San Francisco, CA, 94129
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+
+Developer's Certificate of Origin 1.1
+
+By making a contribution to this project, I certify that:
+
+(a) The contribution was created in whole or in part by me and I
+    have the right to submit it under the open source license
+    indicated in the file; or
+
+(b) The contribution is based upon previous work that, to the best
+    of my knowledge, is covered under an appropriate open source
+    license and I have the right under that license to submit that
+    work with modifications, whether created in whole or in part
+    by me, under the same open source license (unless I am
+    permitted to submit under a different license), as indicated
+    in the file; or
+
+(c) The contribution was provided directly to me by some other
+    person who certified (a), (b) or (c) and I have not modified
+    it.
+
+(d) I understand and agree that this project and the contribution
+    are public and that a record of the contribution (including all
+    personal information I submit with it, including my sign-off) is
+    maintained indefinitely and may be redistributed consistent with
+    this project or the open source license(s) involved.
+```

GUIDE.md

@@ -7,9 +7,15 @@ This is part of a blog post [Serverless Kubernetes home-lab with your Raspberry
 
 > Copyright disclaimer: Please provide a link to the post and give attribution to the author if you plan to use this content in your own materials.
 
+## Update - k3s and `docker`
+
+My current thinking is that [k3s](https://github.com/teamserverless/k8s-on-raspbian#pick-k3s) from Rancher Labs is a better option than `kubeadm` to bootstrap a cluster. Whilst both create a compliant Kubernetes cluster, k3s uses fewer resources, is faster and doesn't run into some of the timing issues we've seen in the community with `kubeadm`.
+
+You should also see my note on [installing Docker on Raspbian Buster](https://github.com/teamserverless/k8s-on-raspbian#fix-docker-for-raspbian-buster-optional)
+
 ## Pre-reqs:
 
-* You must use an RPi 2 or 3 for use with Kubernetes
+* To install and operate Kubernetes, you use only Raspberry Pi 3B, 3B+, or 4B
 * I'm assuming you're using wired ethernet (Wi-Fi also works, but it's not recommended)
 
 ## Master node setup
@@ -67,13 +73,24 @@ Change 100 for 101, 102, 103 etc.
 
 You may also need to make a reservation on your router's DHCP table so these addresses don't get given out to other devices on your network.
 
+* Enable `bridge-nf-call-iptables`
+
+```sh
+sudo sysctl net.bridge.bridge-nf-call-iptables=1
+```
+
 * Install Docker
 
 This installs 17.12 or newer.
 
 ```
-$ curl -sSL get.docker.com | sh && \
+curl -sSL get.docker.com | sh
+
+# Add current user to docker group:
+
 sudo usermod pi -aG docker
+# Refresh groups
+
 newgrp docker
 ```
 
@@ -89,6 +106,12 @@ $ sudo dphys-swapfile swapoff && \
   sudo update-rc.d dphys-swapfile remove
 ```
 
+For Debian, also run:
+
+```sh
+sudo systemctl disable dphys-swapfile
+```
+
 This should now show no entries:
 
 ```
@@ -122,6 +145,14 @@ $ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key a
  * kubeadm - used to create new clusters or join an existing one
  * kubectl - the CLI administration tool for Kubernetes
  
+* Pre-pull images
+
+`kubeadm` now has a command to pre-pull the requisites Docker images needed to run a Kubernetes master, type in:
+
+```
+$ sudo kubeadm config images pull -v3
+```
+
 If using Weave Net
 
 * Initialize your master node:
@@ -147,7 +178,8 @@ Note: This step can take a long time, even up to 15 minutes.
 Sometimes this stage can fail, if it does then you should patch the API Server to allow for a higher failure threshold during initialization around the time you see `[controlplane] wrote Static Pod manifest for component kube-apiserver to "/etc/kubernetes/manifests/kube-apiserver.yaml"`
 
 ```
-sudo sed -i 's/failureThreshold: 8/failureThreshold: 20/g' /etc/kubernetes/manifests/kube-apiserver.yaml
+sudo sed -i 's/failureThreshold: 8/failureThreshold: 20/g' /etc/kubernetes/manifests/kube-apiserver.yaml && \
+sudo sed -i 's/initialDelaySeconds: [0-9]\+/initialDelaySeconds: 360/' /etc/kubernetes/manifests/kube-apiserver.yaml
 ```
 
 After the `init` is complete run the snippet given to you on the command-line:
@@ -204,7 +236,7 @@ If you run into any issues with Weaveworks' networking then [flannel](https://gi
 Apply the Flannel driver on the master:
 
 ```
-$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/c5d10c8/Documentation/kube-flannel.yml
+$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
 ```
 
 On each node that joins including the master:
@@ -261,12 +293,17 @@ spec:
   selector:
     app: markdownrender
 ---
-apiVersion: apps/v1beta1 # for versions before 1.6.0 use extensions/v1beta1
+apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: markdownrender
+  labels:
+   app: markdownrender
 spec:
   replicas: 1
+  selector:
+    matchLabels:
+      app: markdownrender
   template:
     metadata:
       labels:
@@ -304,23 +341,23 @@ The dashboard can be useful for visualising the state and health of your system,
 echo -n 'apiVersion: rbac.authorization.k8s.io/v1beta1
 kind: ClusterRoleBinding
 metadata:
-  name: kubernetes-dashboard
+  name: kubernetes-dashboard-head
   labels:
-    k8s-app: kubernetes-dashboard
+    k8s-app: kubernetes-dashboard-head
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
   name: cluster-admin
 subjects:
 - kind: ServiceAccount
-  name: kubernetes-dashboard
+  name: kubernetes-dashboard-head
   namespace: kube-system' | kubectl apply -f -
 ```
 
 This is the development/alternative dashboard which has TLS disabled and is easier to use.
 
 ```
-$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/alternative/kubernetes-dashboard-arm.yaml
+$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/alternative/kubernetes-dashboard-arm-head.yaml
 ```
 
 You can then find the IP and port via `kubectl get svc -n kube-system`. To access this from your laptop you will need to use `kubectl proxy` and navigate to `http://localhost:8001/` on the master, or tunnel to this address with `ssh`.

README.md

@@ -1,57 +1,91 @@
 ## Kubernetes on Raspbian
 
-This repository holds the original tutorial for Kubernetes on Raspberry Pi by Alex Ellis
+This repository holds the "original tutorial" for "Kubernetes on Raspbian" by Alex Ellis using kubeadm. It also has a list of reader's clusters for your inspiration and Alex's newest work with k3s.
 
-[Kubernetes on (vanilla) Raspbian Lite](./GUIDE.md)
+![My modest stack of RPis](https://miro.medium.com/max/1400/1*wDwPc6YYVbu1I8Ef9m5lBg.jpeg)
+
+If you'd like to read more about Kubernetes clusters, see Alex's write-up over the past 5 years of building with Docker and these tiny devices:
+
+[Five years of Raspberry Pi Clusters](https://medium.com/@alexellisuk/five-years-of-raspberry-pi-clusters-77e56e547875)
+
+### Start the guide
+
+Pick between `k3s` or `kubeadm`.
+
+#### 1) Pick `k3s` (recommended)
+
+My current recommendation is to use [k3s](https://k3s.io) from Rancher Labs. It is normal Kubernetes and passes the conformance tests written by the CNCF. I'm yet to be convinced of why someone wouldn't use this for a hobbyist build and I've been pleasantly surprised by it. Rancher Labs offers commercial support and k3s is GA, even more reason to use this option.
+
+k3s is:
+
+* faster, and uses fewer resources - 300MB for a server, 50MB for an "agent"
+* well-maintained and ARMHF / ARM64 just works
+* HA is available as of k3s 1.0 along with Kubernetes 1.16
+* still normal, upstream, compliant Kubernetes
+* doesn't appear to run into some of the complicated issues we've seen with `kubeadm`
+
+![k3sup diagram](https://github.com/alexellis/k3sup/raw/master/docs/k3sup-cloud.png)
+
+You may also enjoy k3sup ("ketchup") - a tool I built with the community to bootstrap k3s clusters over SSH. It has a simple `k3sup install / join` syntax and is available on GitHub. See more at: [https://k3sup.dev](https://k3sup.dev)
+
+Start with a tutorial now (newest listed first):
+
+* [Walk-through — install Kubernetes to your Raspberry Pi in 15 minutes](https://medium.com/@alexellisuk/walk-through-install-kubernetes-to-your-raspberry-pi-in-15-minutes-84a8492dc95a)
+* [Kubernetes Homelab with Raspberry Pi and k3sup](https://blog.alexellis.io/raspberry-pi-homelab-with-k3sup/)
+* [Will it cluster? k3s on your Raspberry Pi](https://blog.alexellis.io/test-drive-k3s-on-raspberry-pi/)
+
+#### 2) Or pick `kubeadm` (advanced users)
+
+This guide is part of a larger blog post: [Build your own bare-metal ARM cluster](https://blog.alexellis.io/build-your-own-bare-metal-arm-cluster/).
+
+My current thinking is that only advanced users should attempt to install Kubernetes with `kubeadm`. Historically, it's had some unfortunate issues around timeouts and being slow, k3s makes installation and operation seamless.
+
+* Start the guide: [Kubernetes on (vanilla) Raspbian Lite](./GUIDE.md)
+
+Once you're up and running please share your clusters on Twitter with [@alexellisuk](https://twitter.com/alexellisuk).
+
+You can also join the OpenFaaS Slack community's dedicated channel for ARM and Raspberry Pi #arm-and-pi. Just email alex at openfaas dot com for your invitation.
 
 ### Attribution
 
-You're welcome to make use of this guide and to refer to it, but please do not copy it or pass it off as your own without giving attribution to the author(s).
+You're welcome to make use of this guide and to refer to it, but please do not copy it or pass it off as your own without giving attribution to the author(s). If you have suggestions or have found that some of the instructions have fallen out of date, then please see the Contributions section below on how to contribute.
+
+#### Reader's clusters
+
+* [Dino Fizzotti - Raspberry Pi Cluster Part 2: ToDo API running on Kubernetes with k3s](https://www.dinofizzotti.com/blog/2020-05-09-raspberry-pi-cluster-part-2-todo-api-running-on-kubernetes-with-k3s/)
+* [Jim Angel's Kubernetes cluster for testing unreleased alpha / beta k8s versions](https://twitter.com/JimmAngel/status/1265087793170178048)
+* [roncrivera's 4-node home-lab running OpenFaaS in a gun-case](https://twitter.com/roncrivera/status/1078552483029381121)
+* [Scott Hanselman's 6-node cluster running Kubernetes, OpenFaaS with the Pimoroni blinkt!](https://twitter.com/shanselman/status/953716434458247168) [Scott's cluster-selfie](https://twitter.com/alexellisuk/status/955568790061936640)
+* [Ken Fukuyama's Kubernetes cluster running OpenFaaS](https://twitter.com/kenfdev/status/954748775678976000) from Japan
+* [Karol Stępniewski's Asus Tinkerboard cluster running OpenFaaS and K8s](https://twitter.com/kars7e/status/948122096969818113)
+* [Bart Plasmeijer's K8s cluster](https://twitter.com/bartplasmeijer/status/933778520500731904)
+* [Burton Rheutan's cluster stashed away in a closet](https://twitter.com/_burtonr/status/1033745565379641344)
+* [Kevin Turcios' OpenFaaS cluster](https://twitter.com/kjturcios/status/1071253482441715713)
+* [Estelle Auberix' OpenFaaS and K8s cluster for ServerlessConf Paris](https://twitter.com/chussenot/status/960849791776419840)
+* [Davy's Kubernetes and OpenFaaS cluster](https://twitter.com/realDavyHua/status/1028862482259931137)
+* [Jaigouk Kim's OpenFaaS and K8s cluster with the Asus Tinkerboard](https://twitter.com/jaigouk/status/964529214564298756)
+* [Ram's 7-node homelab with OpenFaaS and Kubernetes](https://twitter.com/rprakashg/status/947347563912470528)
+* [David Muckle's OpenFaaS cluster](https://twitter.com/dvdmuckle/status/977297461210484737)
+* [Brian Moelk's battery-powered OpenFaaS cluster](https://twitter.com/brianmoelk/status/954459005149175809)
+* [Marcus Smallman' DIY Raspberry Pi Kubernetes Cluster](https://marcussmallman.io/2018/02/18/diy-rasberry-pi-kubernetes-cluster/)
+* [Mathias Deremer-Accettone's Serverless sur Raspberry PI avec Docker Swarm et OpenFaas](https://blog.ineat-conseil.fr/2019/01/serverless-sur-raspberry-pi-avec-docker-swarm-et-openfaas-partie-1-installation-dopenfaas/)
+* [Daniel Llewellyn's three node Raspberry Pi Swarm](https://twitter.com/diddledan/status/1088759711745351682)
+* [Gareth Bradley's 6 node Raspberry Pi Kubernetes Cluster](https://garfbradaz.github.io/blog/2019/02/12/RaspberryPi-Cluster-Kubernetes.html)
+* [Andreas Muttscheller's Raspberry Pi OpenFaaS k3s cluster provisioned with Ansible](https://blog.codecentric.de/en/2019/08/serverless-functions-k3s-openfaas-raspberry-pi/)
+* [Ruan Bekker's K3s Setup with Golang and Traefik on a Raspberry Pi 4](https://sysadmins.co.za/running-k3s-on-the-raspberrypi-4/?pk_campaign=github-teamserverless)
+
+Submit your cluster and description by creating a GitHub issue.
+
+#### Adaptations / derived works
+
+* ["Kubernetes on Raspbian" with Ansible](https://rak8s.io) by Chris Short
+* ["Kubernetes on Raspbian" for .NET Core / Windows developers](https://www.hanselman.com/blog/HowToBuildAKubernetesClusterWithARMRaspberryPiThenRunNETCoreOnOpenFaas.aspx) by Scott Hanselman
+* ["Kubernetes on Raspbian" broken-up into bash scripts with a custom laser-cut design](https://kubecloud.io/setup-a-kubernetes-1-9-0-raspberry-pi-cluster-on-raspbian-using-kubeadm-f8b3b85bc2d1) by Kasper Nissen
+* ["Kubernetes on Raspbian" with Packer](https://blog.codybunch.com/2018/01/05/OpenFaaS-on-Kubernetes-on-Raspberry-Pi/) by Cody Bunch
+* ["Kubernetes on Raspbian" adapted for HypriotOS](https://gist.github.com/elafargue/a822458ab1fe7849eff0a47bb512546f) by  Edouard Lafargue
+* [Make your very own Kubernetes cluster from Raspberry PI](https://medium.com/nycdev/k8s-on-pi-9cc14843d43) - unattributed, copy/paste from this guide/repository
+
 
 ### Contributions
 
-Contributions are welcome, but must be tested and justified.
-
-Please make sure each commit is signed off with `git commit -s` (this means don't edit in the GitHub UI). 
-
-See below for more information.
-
-```
-Developer Certificate of Origin
-Version 1.1
-
-Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
-1 Letterman Drive
-Suite D4700
-San Francisco, CA, 94129
-
-Everyone is permitted to copy and distribute verbatim copies of this
-license document, but changing it is not allowed.
-
-
-Developer's Certificate of Origin 1.1
-
-By making a contribution to this project, I certify that:
-
-(a) The contribution was created in whole or in part by me and I
-    have the right to submit it under the open source license
-    indicated in the file; or
-
-(b) The contribution is based upon previous work that, to the best
-    of my knowledge, is covered under an appropriate open source
-    license and I have the right under that license to submit that
-    work with modifications, whether created in whole or in part
-    by me, under the same open source license (unless I am
-    permitted to submit under a different license), as indicated
-    in the file; or
-
-(c) The contribution was provided directly to me by some other
-    person who certified (a), (b) or (c) and I have not modified
-    it.
-
-(d) I understand and agree that this project and the contribution
-    are public and that a record of the contribution (including all
-    personal information I submit with it, including my sign-off) is
-    maintained indefinitely and may be redistributed consistent with
-    this project or the open source license(s) involved.
-```
+See: [CONTRIBUTING](CONTRIBUTING.md)

script/prep.sh

@@ -0,0 +1,23 @@
+#!/bin/sh
+
+# This installs the base instructions up to the point of joining / creating a cluster
+
+curl -sSL get.docker.com | sh && \
+  sudo usermod pi -aG docker
+
+sudo dphys-swapfile swapoff && \
+  sudo dphys-swapfile uninstall && \
+  sudo update-rc.d dphys-swapfile remove
+
+curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - && \
+  echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list && \
+  sudo apt-get update -q && \
+  sudo apt-get install -qy kubeadm
+  
+echo Adding " cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory" to /boot/cmdline.txt
+
+sudo cp /boot/cmdline.txt /boot/cmdline_backup.txt
+orig="$(head -n1 /boot/cmdline.txt) cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory"
+echo $orig | sudo tee /boot/cmdline.txt
+
+echo Please reboot