Merge branch 'immediate-fixes' into 'development'
fix: Immediate fixes See merge request nofusscomputing/projects/ansible/kubernetes!2
This commit is contained in:
@ -117,6 +117,7 @@ k3s:
|
||||
# name: k3s-prod # Mandatory, String. Ansible inventory_host that will
|
||||
# # act as the prime master node.
|
||||
# networking:
|
||||
# encrypt: true # Optional, Boolean. default `false`. Install wireguard for inter-node encryption
|
||||
# podSubnet: 172.16.70.0/24 # Mandatory, String. CIDR
|
||||
# ServiceSubnet: 172.16.72.0/24 # Mandatory, String. CIDR
|
||||
# # Mandatory, String. Token to join nodes to the cluster
|
||||
|
||||
0
docs/projects/ansible/index.md
Normal file
0
docs/projects/ansible/index.md
Normal file
0
docs/projects/ansible/playbooks/index.md
Normal file
0
docs/projects/ansible/playbooks/index.md
Normal file
0
docs/projects/ansible/roles/index.md
Normal file
0
docs/projects/ansible/roles/index.md
Normal file
146
docs/projects/ansible/roles/kubernetes/ansible.md
Normal file
146
docs/projects/ansible/roles/kubernetes/ansible.md
Normal file
@ -0,0 +1,146 @@
|
||||
---
|
||||
title: Ansible
|
||||
description: No Fuss Computings Ansible role nfc_kubernetes Ansible docs
|
||||
date: 2023-10-24
|
||||
template: project.html
|
||||
about: https://gitlab.com/nofusscomputing/projects/ansible/roles/kubernetes
|
||||
---
|
||||
|
||||
This page intends to describe/explain the setup of ansible for this role.
|
||||
|
||||
## Inventory Setup
|
||||
|
||||
There are many ways to layout your inventory within Ansible. To take full advantage of this role the following could be used:
|
||||
|
||||
- A aroup containing all master nodes
|
||||
|
||||
- A group containing all worker nodes
|
||||
|
||||
- a group containing all nodes for a cluster
|
||||
|
||||
- All groups above made a subordinate of a master group
|
||||
|
||||
- variable `ansible_host`. _can be either DNS name, IPv4/IPv6 Address_
|
||||
|
||||
!!! info Info
|
||||
The nfc_kubernetes role uses this field for any configuration that requires a hostname. You are strongly encouraged to use DNS name and the DNS name be resolveable for each host accessing to the host in question. Using DNS host name is of paramount importance for a host with dynamic DHCP being used.
|
||||
|
||||
- variable `Kubernetes_Master` _boolean_ set for all host that are master nodes.
|
||||
|
||||
- hosts that require Kubernetes API access added to variable `kubernetes_config.cluster.access`
|
||||
|
||||
An example inventory file that would suffice.
|
||||
|
||||
``` yaml
|
||||
all:
|
||||
hosts:
|
||||
localhost:
|
||||
vars:
|
||||
ansible_connection: local
|
||||
children:
|
||||
|
||||
kubernetes:
|
||||
children:
|
||||
|
||||
k3s:
|
||||
hosts:
|
||||
|
||||
|
||||
k8s:
|
||||
hosts:
|
||||
|
||||
|
||||
kubernetes_cluster:
|
||||
children:
|
||||
|
||||
kubernetes_cluster_{cluster_name_here}:
|
||||
hosts:
|
||||
|
||||
|
||||
kubernetes_master:
|
||||
hosts:
|
||||
|
||||
|
||||
kubernetes_worker:
|
||||
hosts:
|
||||
|
||||
```
|
||||
|
||||
The reasoning for the layout above is:
|
||||
|
||||
- group `kubernetes` used as a selector within playbook or limitor when running a playbook to cover all kubernetes hosts.
|
||||
|
||||
- groups `kubernetes`, `k3s`, `k8s` and `kubernetes_cluster_{cluster_name_here}` used for variable files (`inventory/group_vars/{group_name}.yaml`). with the latter containing all settings for the cluster in question.
|
||||
|
||||
- Hosts are added to ALL groups relevent to them.
|
||||
|
||||
|
||||
The following group variable files will also need to be created:
|
||||
|
||||
- `inventory/group_vars/all.yaml` Variables applicable to all hosts
|
||||
|
||||
- `inventory/group_vars/kubernetes.yaml` software versions for kubernetes
|
||||
|
||||
- `inventory/group_vars/kubernetes_cluster_{cluster_name_here}.yaml` cluster configuration
|
||||
|
||||
|
||||
## Playbooks Setup
|
||||
|
||||
Whilst there are many ways to skin a cat, using the inventory layout as defined above, with the creation of playbooks as detailed below is a possible solution covering most basis' of using this role.
|
||||
|
||||
playbooks/kubernetes.yaml
|
||||
|
||||
``` yaml
|
||||
---
|
||||
- name: Kubernetes Group and sub-groups
|
||||
hosts: "{{ groups.kubernetes }}"
|
||||
gather_facts: true
|
||||
|
||||
roles: []
|
||||
|
||||
- name: Kubernetes Master
|
||||
import_playbook: kubernetes/master.yaml
|
||||
|
||||
- name: Kubernetes Worker
|
||||
import_playbook: kubernetes/worker.yaml
|
||||
```
|
||||
|
||||
playbooks/kubernetes/master.yaml
|
||||
``` yaml
|
||||
---
|
||||
- name: Kubernetes Master Nodes
|
||||
hosts: "{{ kubernetes_master }}"
|
||||
gather_facts: true
|
||||
|
||||
roles:
|
||||
- name: Kubernetes Setup
|
||||
role: nfc_kubernetes
|
||||
|
||||
```
|
||||
|
||||
playbooks/kubernetes/worker.yaml
|
||||
``` yaml
|
||||
---
|
||||
- name: Kubernetes worker Nodes
|
||||
hosts: "{{ kubernetes_worker }}"
|
||||
gather_facts: true
|
||||
|
||||
roles:
|
||||
- name: Kubernetes Setup
|
||||
role: nfc_kubernetes
|
||||
|
||||
```
|
||||
|
||||
Running the above playbooks with the inventory setup allows the following and more:
|
||||
|
||||
- Setup Kubernetes on all applicable kubernetes hosts
|
||||
|
||||
> `ansible-playbook -i inventory/production playbooks/kubernetes.yaml`
|
||||
|
||||
- Setup kubernetes cluster `{cluster_name}`
|
||||
|
||||
> `ansible-playbook --limit kubernetes_cluster_{cluster_name_here} -i inventory/production playbooks/kubernetes.yaml`
|
||||
|
||||
- Setup all Kubernetes master nodes, regardless of cluster
|
||||
|
||||
> `ansible-playbook --limit kubernetes_master -i inventory/production playbooks/kubernetes.yaml`
|
||||
@ -1,5 +1,5 @@
|
||||
---
|
||||
title: Firewall - nfc_kubernetes
|
||||
title: Firewall
|
||||
description: No Fuss Computings Ansible role nfc_kubernetes
|
||||
date: 2023-10-24
|
||||
template: project.html
|
||||
|
||||
@ -1,19 +1,59 @@
|
||||
---
|
||||
title: Kubernetes Ansible Role
|
||||
title: Kubernetes
|
||||
description: No Fuss Computings Ansible role nfc_kubernetes
|
||||
date: 2023-10-24
|
||||
template: project.html
|
||||
about: https://gitlab.com/nofusscomputing/projects/ansible/roles/kubernetes
|
||||
---
|
||||
|
||||
This Ansible roles purpose is to install and configure Kubernetes with configuration from code. You can also use [our playbooks](../../playbooks/index.md) to deploy using this role. this is especially useful if you are also using [our Ansible Execution Environment](../../execution_environment/index.md)
|
||||
|
||||
|
||||
Expected inventory setup:
|
||||
## Features
|
||||
|
||||
- each host has a host_vars file with `ansible_host` defined. _can be either DNS name, IPv4/IPv6 Address_
|
||||
- `k3s` host group with all hosts part of this group
|
||||
- `kubernetes_master` host group with all master nodes part of
|
||||
This role deploys a K3s cluster. In addition it has the following features:
|
||||
|
||||
- variable `Kubernetes_Master` _boolean_ set for all host that are master nodes.
|
||||
- CNI Setup
|
||||
|
||||
- hosts that require Kubernetes API access added to variable `kubernetes_config.cluster.access`
|
||||
- Configurable Container Registries
|
||||
|
||||
- _[ToDo-#3](https://gitlab.com/nofusscomputing/projects/ansible/kubernetes/-/issues/3)_ Encryption between nodes (Wireguard)
|
||||
|
||||
- [Firewall configured for kubernetes host](firewall.md)
|
||||
|
||||
- _[ToDo-#2](https://gitlab.com/nofusscomputing/projects/ansible/kubernetes/-/issues/2)_ Multi-node Deployment
|
||||
|
||||
- OpenID Connect SSO Authentication
|
||||
|
||||
- [Basic RBAC `ClusterRoles` and Bindings](rbac.md)
|
||||
|
||||
- _[ToDo-#5](https://gitlab.com/nofusscomputing/projects/ansible/kubernetes/-/issues/5)_ Restore backup on fresh install of a cluster
|
||||
|
||||
|
||||
## Role Workflow
|
||||
|
||||
The roles workflow is as follows
|
||||
|
||||
1. Download both install script and k3s binary to ansible controller
|
||||
|
||||
1. copy install script and k3s binary to host
|
||||
|
||||
1. Create required config files needed for installation
|
||||
|
||||
1. _(kubernetes prime only)_ Add install required config files
|
||||
|
||||
1. Install kubernetes
|
||||
|
||||
1. Configure Kubernetes
|
||||
|
||||
If the playbook is setup as per [our recommendation](ansible.md) step 2 onwards is first done on master nodes then worker nodes.
|
||||
|
||||
|
||||
## Default Variables
|
||||
|
||||
|
||||
``` yaml title="defaults/main.yaml" linenums="1"
|
||||
|
||||
--8<-- "defaults/main.yaml"
|
||||
|
||||
```
|
||||
37
docs/projects/ansible/roles/kubernetes/rbac.md
Normal file
37
docs/projects/ansible/roles/kubernetes/rbac.md
Normal file
@ -0,0 +1,37 @@
|
||||
---
|
||||
title: RBAC
|
||||
description: No Fuss Computings Ansible role nfc_kubernetes RBAC documentation.
|
||||
date: 2023-10-29
|
||||
template: project.html
|
||||
about: https://gitlab.com/nofusscomputing/projects/ansible/roles/kubernetes
|
||||
---
|
||||
|
||||
As part of this roles workflow, A set of Clester Roles and Cluster Bindings are deployed and ready to use. The intent of these roles is to create a default set of roles that only require the authorization system to provide the users groups. As they have been defined as Cluster Roles you can bind to both cluster and/or namespace.
|
||||
A minimum access required princible has been adopted in the creation of these roles. With the roles designed to be for whom would access/use the cluster (An End user).
|
||||
|
||||
!!! tip
|
||||
All Deployed `ClusterRole` include a labels `authorization/description` and `authorization/target` explaining their intended purpose and where they a recommended for binding.
|
||||
|
||||
|
||||
Currently the following roles are deployed as part of this Anible role:
|
||||
|
||||
- authorization:namespace:read
|
||||
|
||||
> Full read access to all objects except secrets
|
||||
|
||||
- authorization:full
|
||||
|
||||
> Full read/write access to all objects including secrets
|
||||
|
||||
- authorization:namespace:owner
|
||||
|
||||
> Full read/write access to all objects including secrets
|
||||
|
||||
- authorization:cluster:view-metrics
|
||||
|
||||
> View node and pod metrics
|
||||
|
||||
- **[ToDo-#6](https://gitlab.com/nofusscomputing/projects/ansible/kubernetes/-/issues/6)** authorization:cluster:admin
|
||||
|
||||
> Configure the cluster with this not including anything that can be deployed.
|
||||
|
||||
16
mkdocs.yml
16
mkdocs.yml
@ -19,14 +19,30 @@ nav:
|
||||
|
||||
- Ansible:
|
||||
|
||||
- projects/ansible/index.md
|
||||
|
||||
- Execution Environment:
|
||||
|
||||
- projects/ansible/execution_environment/index.md
|
||||
|
||||
- Playbooks:
|
||||
|
||||
- projects/ansible/playbooks/index.md
|
||||
|
||||
- Roles:
|
||||
|
||||
- projects/ansible/roles/index.md
|
||||
|
||||
- Kubernetes:
|
||||
|
||||
- projects/ansible/roles/kubernetes/index.md
|
||||
|
||||
- projects/ansible/roles/kubernetes/ansible.md
|
||||
|
||||
- projects/ansible/roles/kubernetes/firewall.md
|
||||
|
||||
- projects/ansible/roles/kubernetes/rbac.md
|
||||
|
||||
|
||||
- Operations:
|
||||
|
||||
|
||||
@ -185,13 +185,6 @@
|
||||
- containerRegistry
|
||||
|
||||
|
||||
- name: Restart ContainerD if required
|
||||
meta: flush_handlers
|
||||
tags:
|
||||
- install
|
||||
- containerRegistry
|
||||
|
||||
|
||||
- name: Install required python modules
|
||||
ansible.builtin.pip:
|
||||
name: kubernetes
|
||||
@ -295,14 +288,6 @@
|
||||
- firewall
|
||||
|
||||
|
||||
- name: Apply new firewall rules, if required
|
||||
meta: flush_handlers
|
||||
tags:
|
||||
- install
|
||||
- iptables
|
||||
- firewall
|
||||
|
||||
|
||||
- name: Create local workdir
|
||||
file:
|
||||
path: "{{ item }}"
|
||||
|
||||
263
tasks/k3s.yaml
263
tasks/k3s.yaml
@ -1,241 +1,32 @@
|
||||
---
|
||||
- name: Install Software
|
||||
ansible.builtin.include_role:
|
||||
name: nfc_common
|
||||
vars:
|
||||
common_gather_facts: false
|
||||
aptInstall:
|
||||
- name: curl
|
||||
- name: iptables
|
||||
# kubernetes_installed
|
||||
|
||||
- name: K3s Install
|
||||
ansible.builtin.include_tasks:
|
||||
file: k3s/install.yaml
|
||||
when: >
|
||||
install_kubernetes | default(true) | bool
|
||||
and
|
||||
not kubernetes_installed | default(false) | bool
|
||||
|
||||
|
||||
- name: Create Required directories
|
||||
ansible.builtin.file:
|
||||
name: "{{ item.name }}"
|
||||
state: "{{ item.state }}"
|
||||
mode: "{{ item.mode }}"
|
||||
loop: "{{ dirs }}"
|
||||
vars:
|
||||
dirs:
|
||||
- name: /etc/rancher/k3s
|
||||
state: directory
|
||||
mode: 700
|
||||
- name: /var/lib/rancher/k3s/server/logs
|
||||
state: directory
|
||||
mode: 700
|
||||
- name: /var/lib/rancher/k3s/server/manifests
|
||||
state: directory
|
||||
mode: 700
|
||||
- name: K3s Configure
|
||||
ansible.builtin.include_tasks:
|
||||
file: k3s/configure.yaml
|
||||
when: >
|
||||
install_kubernetes | default(true) | bool
|
||||
and
|
||||
kubernetes_installed | default(false) | bool
|
||||
|
||||
|
||||
# - name: Local Container Registry
|
||||
# ansible.builtin.copy:
|
||||
# content: |
|
||||
# #
|
||||
# # Private Container Registries for Kubernetes
|
||||
# #
|
||||
# # Managed By ansible/role/nfc_kubernetes
|
||||
# #
|
||||
# # Dont edit this file directly as it will be overwritten.
|
||||
# #
|
||||
|
||||
# {% set registries = kubernetes_private_container_registry | default([]) -%}
|
||||
|
||||
# {% if registries | length > 0 %}mirrors:
|
||||
# {% for entry in registries %}
|
||||
|
||||
# {{ entry.name }}:
|
||||
# endpoint:
|
||||
# - "{{ entry.url }}"
|
||||
|
||||
# {%- endfor %}
|
||||
# {% endif %}
|
||||
# dest: /etc/rancher/k3s/registries.yaml
|
||||
# owner: root
|
||||
# mode: '700'
|
||||
# # notify: "restart ContainerD"
|
||||
# # with_items: "{{ containerd.repositories }}"
|
||||
# # when:
|
||||
# # ansible_os_family == 'Debian'
|
||||
# # and
|
||||
# # Kubernetes_private_container_registry | default([]) | length > 0
|
||||
|
||||
|
||||
- name: Add sysctl net.ipv4.ip_forward
|
||||
sysctl:
|
||||
name: net.ipv4.ip_forward
|
||||
value: '1'
|
||||
sysctl_set: true
|
||||
state: present
|
||||
reload: true
|
||||
notify: reboot_host
|
||||
when:
|
||||
- ansible_os_family == 'Debian'
|
||||
# On change reboot
|
||||
|
||||
|
||||
- name: Check if K3s Installed
|
||||
ansible.builtin.shell:
|
||||
cmd: |
|
||||
if [[ $(service k3s status) ]]; then exit 0; else exit 1; fi
|
||||
executable: /bin/bash
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
register: k3s_installed
|
||||
|
||||
|
||||
# - name: Download K3s Binary
|
||||
# ansible.builtin.uri:
|
||||
# url: "{{ item.url }}"
|
||||
# method: GET
|
||||
# return_content: true
|
||||
# # body: ""
|
||||
# status_code:
|
||||
# - 200
|
||||
# - 304
|
||||
# # headers:
|
||||
# # App-Token: "{{ glpi.app_token }}"
|
||||
# # Authorization: "user_token {{ glpi.user_token }}"
|
||||
# #body_format: json
|
||||
# # validate_certs: false
|
||||
# dest: "{{ item.dest }}"
|
||||
# mode: "744"
|
||||
# register: k3s_download_files
|
||||
# delegate_to: localhost
|
||||
# # no_log: true
|
||||
# when: ansible_os_family == 'Debian'
|
||||
# loop: "{{ download_files }}"
|
||||
# vars:
|
||||
# ansible_connection: local
|
||||
# download_files:
|
||||
# - dest: /tmp/install.sh
|
||||
# url: https://get.k3s.io
|
||||
# - dest: "/tmp/k3s"
|
||||
# url: "https://github.com/k3s-io/k3s/releases/download/v{{ KubernetesVersion + KubernetesVersion_k3s_prefix | urlencode }}/k3s"
|
||||
|
||||
|
||||
# - name: "[TRACE] Downloaded File SHA256"
|
||||
# ansible.builtin.set_fact:
|
||||
# hash_sha256_k3s_downloaded_binary: "{{ lookup('ansible.builtin.file', '/tmp/k3s') | hash('sha256') | string }}"
|
||||
# delegate_to: localhost
|
||||
|
||||
|
||||
# - name: Existing k3s File hash
|
||||
# ansible.builtin.stat:
|
||||
# checksum_algorithm: sha256
|
||||
# name: /usr/local/bin/k3s
|
||||
# register: hash_sha256_k3s_existing_binary
|
||||
|
||||
|
||||
# - name: Copy K3s binary to Host
|
||||
# ansible.builtin.copy:
|
||||
# src: "/tmp/k3s"
|
||||
# dest: "/usr/local/bin/k3s"
|
||||
# mode: '740'
|
||||
# owner: root
|
||||
# group: root
|
||||
# when: hash_sha256_k3s_existing_binary.stat.checksum | default('0') != hash_sha256_k3s_downloaded_binary
|
||||
|
||||
# - name: Copy install script to Host
|
||||
# ansible.builtin.copy:
|
||||
# src: "/tmp/install.sh"
|
||||
# dest: "/tmp/install.sh"
|
||||
# mode: '755'
|
||||
# owner: root
|
||||
# group: root
|
||||
# # when: hash_sha256_k3s_existing_binary.stat.checksum | default('0') != hash_sha256_k3s_downloaded_binary
|
||||
|
||||
- name: Additional config files
|
||||
ansible.builtin.copy:
|
||||
content: |
|
||||
{{ item.content }}
|
||||
dest: "{{ item.path }}/{{ item.name }}"
|
||||
mode: '740'
|
||||
owner: root
|
||||
group: root
|
||||
loop: "{{ k3s.files }}"
|
||||
|
||||
|
||||
- name: Copy Templates
|
||||
ansible.builtin.template:
|
||||
src: "{{ item.src }}"
|
||||
dest: "{{ item.dest }}"
|
||||
owner: root
|
||||
mode: '700'
|
||||
force: true
|
||||
notify: "{{ item.notify | default(omit) }}"
|
||||
loop: "{{ templates_to_apply }}"
|
||||
vars:
|
||||
templates_to_apply:
|
||||
- src: "calico.yaml.j2"
|
||||
dest: /var/lib/rancher/k3s/server/manifests/calico.yaml
|
||||
- src: kubernetes-manifest-rbac.yaml.j2
|
||||
dest: /var/lib/rancher/k3s/server/manifests/rbac-authorization-common.yaml
|
||||
# - src: firewall-kubernetes.j2
|
||||
# dest: "/etc/network/if-up.d/firewall-kubernetes"
|
||||
|
||||
- src: iptables-kubernetes.rules.j2
|
||||
dest: "/etc/iptables.rules.d/iptables-kubernetes.rules"
|
||||
notify: firewall_reloader
|
||||
- src: k3s-registries.yaml.j2
|
||||
dest: /etc/rancher/k3s/registries.yaml
|
||||
notify: kubernetes_restart
|
||||
- src: k3s-config.yaml.j2
|
||||
dest: /etc/rancher/k3s/config.yaml
|
||||
notify: kubernetes_restart
|
||||
|
||||
|
||||
# - name: Templates IPv6
|
||||
# ansible.builtin.template:
|
||||
# src: iptables-kubernetes.rules.j2
|
||||
# dest: "/etc/ip6tables.rules.d/ip6tables-kubernetes.rules"
|
||||
# owner: root
|
||||
# mode: '700'
|
||||
# force: true
|
||||
# vars:
|
||||
# ipv6: true
|
||||
|
||||
|
||||
- name: Set IPTables to legacy mode
|
||||
ansible.builtin.command:
|
||||
cmd: update-alternatives --set iptables /usr/sbin/iptables-legacy
|
||||
changed_when: false
|
||||
|
||||
|
||||
# - name: Server install K3s
|
||||
# ansible.builtin.shell:
|
||||
# cmd: |
|
||||
# # INSTALL_K3S_SKIP_DOWNLOAD=true \
|
||||
# # INSTALL_K3S_VERSION="v{{ KubernetesVersion }}{{ KubernetesVersion_k3s_prefix }}" \
|
||||
# # /tmp/install.sh
|
||||
# curl -sfL https://get.k3s.io | \
|
||||
# INSTALL_K3S_VERSION="v1.26.9+k3s1" \
|
||||
# sh -
|
||||
# failed_when: false
|
||||
# # when: >
|
||||
# # k3s_installed.rc | int == 1
|
||||
# # and
|
||||
# # Kubernetes_Master | default(false)
|
||||
# when: Kubernetes_Master | default(false)
|
||||
|
||||
# - name: Agent install K3s
|
||||
# ansible.builtin.shell:
|
||||
# cmd: |
|
||||
# INSTALL_K3S_SKIP_DOWNLOAD=true \
|
||||
# INSTALL_K3S_VERSION="v{{ KubernetesVersion }}{{ KubernetesVersion_k3s_prefix }}" \
|
||||
# K3S_URL=https://{{ hostvars[kubernetes_config.cluster.prime.name].ansible_host }}:6443 \
|
||||
# K3S_TOKEN={{ node_token }} \
|
||||
# /tmp/install.sh
|
||||
# when: >
|
||||
# k3s_installed.rc | int == 1
|
||||
# and
|
||||
# not Kubernetes_Master | default(false)
|
||||
|
||||
|
||||
# # - name: Look up AAAA (IPv4) records for example.org
|
||||
# # ansible.builtin.debug:
|
||||
# # msg: "{{ query('community.dns.lookup', 'nww-au1.networkedweb.com.', type='A') }}"
|
||||
|
||||
# # - name: Look up AAAA (IPv6) records for example.org
|
||||
# # ansible.builtin.debug:
|
||||
# # msg: "{{ query('community.dns.lookup', 'nww-au1.networkedweb.com.', type='AAAA') }}"
|
||||
- name: Wireguard Cluster Encryption
|
||||
ansible.builtin.include_tasks:
|
||||
file: k3s/wireguard.yaml
|
||||
when: >
|
||||
install_kubernetes | default(true) | bool
|
||||
and
|
||||
kubernetes_installed | default(false) | bool
|
||||
and
|
||||
not kubernetes_installed_encryption | default(false) | bool
|
||||
and
|
||||
kubernetes_config.cluster.networking.encrypt | default(false) | bool
|
||||
|
||||
74
tasks/k3s/configure.yaml
Normal file
74
tasks/k3s/configure.yaml
Normal file
@ -0,0 +1,74 @@
|
||||
---
|
||||
|
||||
- name: Local Container Registry
|
||||
ansible.builtin.copy:
|
||||
content: |
|
||||
#
|
||||
# Private Container Registries for Kubernetes
|
||||
#
|
||||
# Managed By ansible/role/nfc_kubernetes
|
||||
#
|
||||
# Dont edit this file directly as it will be overwritten.
|
||||
#
|
||||
|
||||
{% set registries = kubernetes_private_container_registry | default([]) -%}
|
||||
|
||||
{% if registries | length > 0 %}mirrors:
|
||||
{% for entry in registries %}
|
||||
|
||||
{{ entry.name }}:
|
||||
endpoint:
|
||||
- "{{ entry.url }}"
|
||||
|
||||
{%- endfor %}
|
||||
{% endif %}
|
||||
dest: /etc/rancher/k3s/registries.yaml
|
||||
owner: root
|
||||
mode: '700'
|
||||
# notify: "restart ContainerD"
|
||||
# with_items: "{{ containerd.repositories }}"
|
||||
# when:
|
||||
# ansible_os_family == 'Debian'
|
||||
# and
|
||||
# Kubernetes_private_container_registry | default([]) | length > 0
|
||||
|
||||
|
||||
- name: Additional config files
|
||||
ansible.builtin.copy:
|
||||
content: |
|
||||
{{ item.content }}
|
||||
dest: "{{ item.path }}/{{ item.name }}"
|
||||
mode: '740'
|
||||
owner: root
|
||||
group: root
|
||||
loop: "{{ k3s.files }}"
|
||||
|
||||
|
||||
- name: Copy Templates
|
||||
ansible.builtin.template:
|
||||
src: "{{ item.src }}"
|
||||
dest: "{{ item.dest }}"
|
||||
owner: root
|
||||
mode: '700'
|
||||
force: true
|
||||
notify: "{{ item.notify | default(omit) }}"
|
||||
loop: "{{ templates_to_apply }}"
|
||||
vars:
|
||||
templates_to_apply:
|
||||
- src: "calico.yaml.j2"
|
||||
dest: /var/lib/rancher/k3s/server/manifests/calico.yaml
|
||||
|
||||
- src: kubernetes-manifest-rbac.yaml.j2
|
||||
dest: /var/lib/rancher/k3s/server/manifests/rbac-authorization-common.yaml
|
||||
|
||||
- src: iptables-kubernetes.rules.j2
|
||||
dest: "/etc/iptables.rules.d/iptables-kubernetes.rules"
|
||||
notify: firewall_reloader
|
||||
|
||||
- src: k3s-registries.yaml.j2
|
||||
dest: /etc/rancher/k3s/registries.yaml
|
||||
notify: kubernetes_restart
|
||||
|
||||
- src: k3s-config.yaml.j2
|
||||
dest: /etc/rancher/k3s/config.yaml
|
||||
notify: kubernetes_restart
|
||||
188
tasks/k3s/install.yaml
Normal file
188
tasks/k3s/install.yaml
Normal file
@ -0,0 +1,188 @@
|
||||
---
|
||||
- name: Install Software
|
||||
ansible.builtin.include_role:
|
||||
name: nfc_common
|
||||
vars:
|
||||
common_gather_facts: false
|
||||
aptInstall:
|
||||
- name: curl
|
||||
- name: iptables
|
||||
|
||||
|
||||
- name: Create Required directories
|
||||
ansible.builtin.file:
|
||||
name: "{{ item.name }}"
|
||||
state: "{{ item.state }}"
|
||||
mode: "{{ item.mode }}"
|
||||
loop: "{{ dirs }}"
|
||||
vars:
|
||||
dirs:
|
||||
- name: /etc/rancher/k3s
|
||||
state: directory
|
||||
mode: 700
|
||||
- name: /var/lib/rancher/k3s/server/logs
|
||||
state: directory
|
||||
mode: 700
|
||||
- name: /var/lib/rancher/k3s/server/manifests
|
||||
state: directory
|
||||
mode: 700
|
||||
|
||||
|
||||
- name: Add sysctl net.ipv4.ip_forward
|
||||
ansible.posix.sysctl:
|
||||
name: net.ipv4.ip_forward
|
||||
value: '1'
|
||||
sysctl_set: true
|
||||
state: present
|
||||
reload: true
|
||||
notify: reboot_host
|
||||
when:
|
||||
- ansible_os_family == 'Debian'
|
||||
# On change reboot
|
||||
|
||||
|
||||
- name: Check if K3s Installed
|
||||
ansible.builtin.shell:
|
||||
cmd: |
|
||||
if [[ $(service k3s status) ]]; then exit 0; else exit 1; fi
|
||||
executable: /bin/bash
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
register: k3s_installed
|
||||
|
||||
|
||||
- name: Download K3s Binary
|
||||
ansible.builtin.uri:
|
||||
url: "{{ item.url }}"
|
||||
method: GET
|
||||
return_content: true
|
||||
status_code:
|
||||
- 200
|
||||
- 304
|
||||
dest: "{{ item.dest }}"
|
||||
mode: "744"
|
||||
register: k3s_download_files
|
||||
delegate_to: localhost
|
||||
# no_log: true
|
||||
when: ansible_os_family == 'Debian'
|
||||
loop: "{{ download_files }}"
|
||||
vars:
|
||||
ansible_connection: local
|
||||
download_files:
|
||||
- dest: /tmp/install.sh
|
||||
url: https://get.k3s.io
|
||||
- dest: "/tmp/k3s"
|
||||
url: "https://github.com/k3s-io/k3s/releases/download/v{{ KubernetesVersion + KubernetesVersion_k3s_prefix | urlencode }}/k3s"
|
||||
|
||||
|
||||
- name: "[TRACE] Downloaded File SHA256"
|
||||
ansible.builtin.set_fact:
|
||||
hash_sha256_k3s_downloaded_binary: "{{ lookup('ansible.builtin.file', '/tmp/k3s') | hash('sha256') | string }}"
|
||||
delegate_to: localhost
|
||||
|
||||
|
||||
- name: Existing k3s File hash
|
||||
ansible.builtin.stat:
|
||||
checksum_algorithm: sha256
|
||||
name: /usr/local/bin/k3s
|
||||
register: hash_sha256_k3s_existing_binary
|
||||
|
||||
|
||||
- name: Copy K3s binary to Host
|
||||
ansible.builtin.copy:
|
||||
src: "/tmp/k3s"
|
||||
dest: "/usr/local/bin/k3s"
|
||||
mode: '740'
|
||||
owner: root
|
||||
group: root
|
||||
when: hash_sha256_k3s_existing_binary.stat.checksum | default('0') != hash_sha256_k3s_downloaded_binary
|
||||
|
||||
- name: Copy install script to Host
|
||||
ansible.builtin.copy:
|
||||
src: "/tmp/install.sh"
|
||||
dest: "/tmp/install.sh"
|
||||
mode: '755'
|
||||
owner: root
|
||||
group: root
|
||||
# when: hash_sha256_k3s_existing_binary.stat.checksum | default('0') != hash_sha256_k3s_downloaded_binary
|
||||
|
||||
- name: Required Initial config files
|
||||
ansible.builtin.copy:
|
||||
content: |
|
||||
{{ item.content }}
|
||||
dest: "{{ item.path }}/{{ item.name }}"
|
||||
mode: '740'
|
||||
owner: root
|
||||
group: root
|
||||
loop: "{{ k3s.files }}"
|
||||
when: >
|
||||
kubernetes_config.cluster.prime.name == inventory_hostname
|
||||
|
||||
|
||||
- name: Copy Intial required templates
|
||||
ansible.builtin.template:
|
||||
src: "{{ item.src }}"
|
||||
dest: "{{ item.dest }}"
|
||||
owner: root
|
||||
mode: '700'
|
||||
force: true
|
||||
notify: "{{ item.notify | default(omit) }}"
|
||||
loop: "{{ templates_to_apply }}"
|
||||
vars:
|
||||
templates_to_apply:
|
||||
- src: k3s-config.yaml.j2
|
||||
dest: /etc/rancher/k3s/config.yaml
|
||||
notify: kubernetes_restart
|
||||
when: >
|
||||
kubernetes_config.cluster.prime.name == inventory_hostname
|
||||
|
||||
# - name: Templates IPv6
|
||||
# ansible.builtin.template:
|
||||
# src: iptables-kubernetes.rules.j2
|
||||
# dest: "/etc/ip6tables.rules.d/ip6tables-kubernetes.rules"
|
||||
# owner: root
|
||||
# mode: '700'
|
||||
# force: true
|
||||
# vars:
|
||||
# ipv6: true
|
||||
|
||||
|
||||
- name: Set IPTables to legacy mode
|
||||
ansible.builtin.command:
|
||||
cmd: update-alternatives --set iptables /usr/sbin/iptables-legacy
|
||||
changed_when: false
|
||||
|
||||
|
||||
- name: Server install K3s
|
||||
ansible.builtin.shell:
|
||||
cmd: |
|
||||
# INSTALL_K3S_SKIP_DOWNLOAD=true \
|
||||
# INSTALL_K3S_VERSION="v{{ KubernetesVersion }}{{ KubernetesVersion_k3s_prefix }}" \
|
||||
# /tmp/install.sh
|
||||
curl -sfL https://get.k3s.io | \
|
||||
INSTALL_K3S_VERSION="v1.26.9+k3s1" \
|
||||
sh -
|
||||
failed_when: false
|
||||
# when: >
|
||||
# k3s_installed.rc | int == 1
|
||||
# and
|
||||
# Kubernetes_Master | default(false)
|
||||
when: Kubernetes_Master | default(false) | bool
|
||||
|
||||
|
||||
- name: Agent install K3s
|
||||
ansible.builtin.shell:
|
||||
cmd: |
|
||||
INSTALL_K3S_SKIP_DOWNLOAD=true \
|
||||
INSTALL_K3S_VERSION="v{{ KubernetesVersion }}{{ KubernetesVersion_k3s_prefix }}" \
|
||||
K3S_URL=https://{{ hostvars[kubernetes_config.cluster.prime.name].ansible_host }}:6443 \
|
||||
K3S_TOKEN={{ node_token }} \
|
||||
/tmp/install.sh
|
||||
when: >
|
||||
k3s_installed.rc | int == 1
|
||||
and
|
||||
not Kubernetes_Master | default(false) | bool
|
||||
|
||||
- name: Set Kubernetes Final Install Fact
|
||||
ansible.builtin.set_fact:
|
||||
kubernetes_installed: true
|
||||
22
tasks/k3s/wireguard.yaml
Normal file
22
tasks/k3s/wireguard.yaml
Normal file
@ -0,0 +1,22 @@
|
||||
---
|
||||
- name: Install Wireguard
|
||||
ansible.builtin.apt:
|
||||
name:
|
||||
- wireguard
|
||||
update_cache: false
|
||||
when: >
|
||||
ansible_os_family == 'Debian'
|
||||
# and
|
||||
# kubernetes.networking.encrypt | default(false) | bool
|
||||
|
||||
|
||||
- name: Enable Cluster Encryption
|
||||
ansible.builtin.command:
|
||||
cmd: kubectl patch felixconfiguration default --type='merge' -p '{"spec":{"wireguardEnabled":true,"wireguardEnabledV6":true}}'
|
||||
changed_when: false
|
||||
when: >
|
||||
kubernetes_config.cluster.prime.name == inventory_hostname
|
||||
|
||||
- name: Set Kubernetes Encryption Final Install Fact
|
||||
ansible.builtin.set_fact:
|
||||
kubernetes_installed_encryption: true
|
||||
@ -13,7 +13,7 @@ metadata:
|
||||
app.kubernetes.io/part-of: nfc_kubernetes
|
||||
app.kubernetes.io/managed-by: ansible
|
||||
app.kubernetes.io/version: ''
|
||||
name: authorization:common:full
|
||||
name: authorization:full
|
||||
rules:
|
||||
- apiGroups:
|
||||
- "*"
|
||||
@ -37,7 +37,7 @@ metadata:
|
||||
app.kubernetes.io/part-of: nfc_kubernetes
|
||||
app.kubernetes.io/managed-by: ansible
|
||||
app.kubernetes.io/version: ''
|
||||
name: authorization:common:namespace:read
|
||||
name: authorization:namespace:read
|
||||
rules:
|
||||
- apiGroups: # Get Metrics
|
||||
- metrics.k8s.io
|
||||
@ -88,7 +88,7 @@ metadata:
|
||||
app.kubernetes.io/part-of: nfc_kubernetes
|
||||
app.kubernetes.io/managed-by: ansible
|
||||
app.kubernetes.io/version: ''
|
||||
name: authorization:common:namespace:owner
|
||||
name: authorization:namespace:owner
|
||||
rules:
|
||||
- apiGroups: # Read-only access to resrouces
|
||||
- "*"
|
||||
@ -122,7 +122,7 @@ rules:
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: authorization:common:cluster:view-metrics
|
||||
name: authorization:cluster:view-metrics
|
||||
rules:
|
||||
- apiGroups:
|
||||
- metrics.k8s.io
|
||||
@ -144,7 +144,7 @@ metadata:
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: authorization:common:namespace:read
|
||||
name: authorization:namespace:read
|
||||
subjects:
|
||||
- kind: Group
|
||||
name: administrators
|
||||
@ -163,7 +163,7 @@ metadata:
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: authorization:common:cluster:view-metrics
|
||||
name: authorization:cluster:view-metrics
|
||||
subjects:
|
||||
- apiGroup: rbac.authorization.k8s.io
|
||||
kind: Group
|
||||
|
||||
Reference in New Issue
Block a user