Installing Prerequisites for Kubernetes Cluster Installation

Getting Started

If you want to install a Kubernetes cluster, you need to meet the following prerequisites before you start your actual work. Below requirements are taken from the official website:

• One or more machines running one of:

  • Ubuntu 16.04+

  • Debian 9+

  • CentOS 7

  • Red Hat Enterprise Linux (RHEL) 7

  • Fedora 25+

  • HypriotOS v1.0.1+

  • Flatcar Container Linux (tested with 2512.3.0)

• 2 GB or more of RAM per machine (any less will leave little room for your apps).

• 2 CPUs or more.

• Full network connectivity between all machines in the cluster (public or private network is fine).

• Unique hostname, MAC address, and product_uuid for every node. See here for more details.

• Certain ports are open on your machines. See here for more details.

• Swap disabled. You MUST disable swap in order for the kubelet to work properly.

I will try to cover CentOS 7 and Ubuntu 18.04 / 20.04 under that post. So let's write an Ansible role to cover the prerequisites.

Kubernetes cluster installation and configuration require certain prerequisites to be done in advance. I divided those prerequisites into 3 categories in my role:

1. Swap

2. SELinux

3. Firewall

I prefer to keep the above specific tasks into their own yaml files and import them in the role's main file. You can find the Ansible role here.

Disabling Swap

If you fail to disable swap space before your Kubernetes cluster setup, it will throw an ugly error and will ask you to disable it. Disabling swap space looks straightforward but if you want to automate it, it can be a tricky task because you can have either a swap volume or a file. There is also another risk that if you remove the swap logical volume and if your GRUB configuration expects for it, your server will not boot up after the first reboot. That's why I decided to only disable swap and uncomment it in the /etc/fstab file and let the administrators handle it properly if they want to reclaim the swap space.

I am disabling swap space in a separate playbook as below:

tasks/disable_swap.yaml

---
- name: Disable swap
  command: swapoff -a
  when: ansible_swaptotal_mb > 0
  tags:
    - skip_ansible_lint

# Removing swap LV can cause issues if your grub dependant in it
# I will only comment it out
- name: Remove swap from fstab
  replace:
    path: /etc/fstab
    regexp: '^([^#].*?\sswap\s.*)$'
    replace: '# \1'

- name: Remove swap file if exists
  file:
    path: "{{ swapfile_path }}"
    state: absent

Please pay attention to the swapfile_path variable in the vars/main.yml file and change it accordingly.

Please note that swap needs to be disabled on both control and worker nodes

Disable SELinux

I call the title of the section "disable" but I prefer to put SELinux into permissive mode. Since SELinux status is a variable (either permissive or disabled ), I kept it in vars/main.yml file as below:

vars/main.yml

selinux_policy: "targeted"
selinux_state: "permissive"

Ubuntu servers do not come with SELinux by default, so I decided to touch SELinux configuration only if its configuration file exists. The playbook to handle SELinux tasks will be like below:

tasks/selinux.yaml

---
- name: Check if SElinux config exists
  stat:
    path: /etc/selinux/config
  register: selinux_config

- name: Make SELinux permissive if not disabled
  selinux:
    policy: "{{ selinux_policy }}"
    state: "{{ selinux_state }}"
  when: selinux_config.stat.exists

Please note that. SELinux needs to be disabled on both control and worker nodes.

Enable Firewall Rules for Control and Worker Nodes

There are some ports to be enabled on both the control plane and worker nodes. The list of ports can be also found here.

Control-plane node(s)

Protocol	Direction	Port Range	Purpose	Used By
TCP	Inbound	6443*	Kubernetes API server	All
TCP	Inbound	2379-2380	etcd server client API	kube-apiserver, etcd
TCP	Inbound	10250	kubelet API	Self, Control plane
TCP	Inbound	10251	kube-scheduler	Self
TCP	Inbound	10252	kube-controller-manager	Self

Worker node(s)

Protocol	Direction	Port Range	Purpose	Used By
TCP	Inbound	10250	kubelet API	Self, Control plane
TCP	Inbound	30000-32767	NodePort Services†	All

CentOS servers by default use firewalld and Ubuntu servers use ufw to manage firewall rules. I will not make any exception here and assume you are using the default services. Then the playbook will look like below:

tasks/firewall.yaml

---
- name: Allow firewall for Control-plane node(s)
  block:
    - name: CentOS Control-plane node(s) firewall rules
      firewalld:
        port: "{{ item }}"
        permanent: yes
        state: enabled
      loop:
        - 6443/tcp
        - 2379-2380/tcp
        - 10250/tcp
        - 10251/tcp
        - 10252/tcp
      when: ansible_distribution | lower == "centos"

    - name: Ubuntu Control-plane node(s) firewall rules
      community.general.ufw:
        rule: allow
        port: "{{ item }}"
        proto: tcp
      loop:
        - 6443
        - 2379:2380
        - 10250
        - 10251
        - 10252
      when: ansible_distribution | lower == "ubuntu"
  when: inventory_hostname in groups['master']

- name: Allow firewall for Worker node(s)
  block:
    - name: CentOS Worker node(s) firewall rules
      firewalld:
        port: "{{ item }}"
        permanent: yes
        state: enabled
      loop:
        - 10250/tcp
        - 30000-32767/tcp
      when: ansible_distribution | lower == "centos"

    - name: Ubuntu Worker node(s) firewall rules
      community.general.ufw:
        rule: allow
        port: "{{ item }}"
        proto: tcp
      loop:
        - 10250
        - 30000:32767
      when: ansible_distribution | lower == "ubuntu"
  when: inventory_hostname in groups['node']

Putting things together

Now we are ready to put things together.

Please note that the control plane nodes should be under master group in the inventory file and worker nodes should be in node group.

What we need to do is relatively simple now. We just need to include our tasks in the main playbook like below:

tasks/main.yml

---
# tasks file for ansible-role-k8s_prereq
- name: Setting OS version fact
  set_fact:
    osversion: "{{ ansible_distribution | lower }}{{ ansible_distribution_major_version }}"

- name: Check OS version
  fail:
    msg: "OS Version( {{ ansible_distribution }}{{ ansible_distribution_major_version }} ) is not certified for the role"
  when:
    - osversion != "ubuntu20"
    - osversion != "ubuntu18"
    - osversion != "centos7"

- name: Disable swap
  include_tasks: disable_swap.yaml

- name: Disable SELinux
  include_tasks: selinux.yaml

- name: Apply firewall rules
  include_tasks: firewall.yaml

Now we are ready to install the container runtime interface.