As of this writing, Assisted Installer is the easiest way to install an OpenShift cluster on a custom/bespoke infrastructure. You do not need to manually deal with ignition files or manually configure the OpenShift installer. You do not even need to manually set up a bootstrap node.
In this post, I will walk you through how to create an OpenShift cluster using Assisted Installer on a bespoke infrastructure. To host VMs, I am using Proxmox VE, a lightweight open-source virtualization product. However, similar steps should also apply to other hypervisor products. If you want to know how to install Proxmox, see my previous post at https://blog.rossbrigoli.com/2020/10/running-openshift-at-home-part-34.html
First, we need to set up the infrastructure. We need to create VMs and then do the following steps to configure our infrastructure.
- Create and configure a DNS server
- Configure the DHCP server of our router
- Create and configure an HAProxy load balancer
We will run the supporting services in a dedicated VM called openshift-services.
The first step is to create a VM that will host all of the services required by the cluster.
Creating the Virtual Machines
1. Download the CentOS Stream image from https://www.centos.org/centos-stream/ and upload it to Proxmox. You may also let Proxmox download the image directly from a URL.
Download the correct architecture that matches your VM host. In my case, my VMs have x86_64 CPUs.
Upload the image to the Proxmox's local storage as shown below.
2. Create a VM in Proxmox and name it "openshift-services".
3. Use the CentOS Stream ISO file as the DVD image to boot from.
4. Choose the disk size. 100GB should be enough but you can make it bigger as well. You don't need a lot of resources for this VM as well as it is only doing DNS name resolution and load balancing. So you can use 2 CPUs (or less) and 4GB of RAM.
5. Review the configuration and create the VM.
6. Start the VM and follow the CentOS Stream installer instructions. Then wait for the installation to complete.
7. While waiting for the installation to finish, create the cluster from Red Hat Hybrid Console. Navigate to https://console.redhat.com > OpenShift, then click "Create Cluster".
If you don't have a Red Hat account yet, feel free to register for a Red Hat account, it's free.
8. Because we want to install OpenShift in our bespoke infrastructure, we need to choose the "Run it yourself" option and then the Platform Agnostic.
10. Fill up the details according to the domain and cluster name you choose.
My plan is to let the VMs get their IP Addresses from my router through DHCP, so I am selecting DHCP option for host's network configuration as shown below.
11. On the next page, there are optional operators that you can install. I don't need them for this installation so I will not select any.
12. On the third page, this is where you define the hosts/node. Click the Add Host button. You may also paste your SSH public key here so that you can SSH to the nodes for debugging later. Once all set, then click the Generate Discovery ISO. This will generate an ISO image.
13. Wait for the ISO to be generated, then download it. This will be the image that you will use to boot the VMs.
14. While waiting for the ISO to be downloaded, revisit the CentOS Stream installation on the openshift-services VM. By this time, the installation should have been completed and you just need to reboot the system.
15. Once the OpenShift Discovery ISO is downloaded, upload it to Proxmox local disk so that we can use it as the bootable installer for our VMs.
16. Once uploaded, let's start creating the VMs for our OpenShift cluster. We will be creating a total of 5 VMs, 3 for master nodes and 2 for worker nodes. You can choose to create more worker nodes as you wish.
Prepare the VMs according to the spec below.
Both Master and Worker Nodes:
CPU: 8
Memory: 16 GB
Disk: 250 GB
Use the ISO image you downloaded as the image for the DVD drive. In the above screenshot, it is the value of the ide2 field.
17. After creating the VMs, do not start the VMs yet as we still need to install the supporting services in the openshift-services VM. Your list of VM should look something like the screenshot below.
18. Using your router's built-in DHCP server, assign IP addresses to your VMs through the address reservation feature. This requires mapping MAC Addresses to IP Addresses. Do this before starting the VMs. Here is an example mapping I used in my router setup. I used x.x.x.210 as the IP address of the openshift-services. It is important to take note of this address because this will be the load balancer IP and DNS server IP.
DNS Server
19. Now let's install and configure a DNS Server in openshift-services VM. You need to SSH to the openshift-services VM.
OpenShift cluster requires a DNS server. We will use the openshift-services as the DNS server by installing Bind (a.k.a. named) to it.
Bind can be installed by running the following command inside openshift-service VM.
dnf install -y bind bind-utilsAfter installing, run the following command to enable, start and validate the status of named service.
systemctl enable named systemctl start named systemctl status named
Once we validated the installation, we need to configure the named service according to the IP addresses and DNS name mapping that we want to use for our worker nodes.
20. Configure the DNS Server so that the names will resolve to the IP addresses that we configured in the router's DHCP address reservation (step 9). I have prepared a named configuration for this setup which is available in GitHub at https://github.com/rossbrigoli/okd4_files. So let's get these files and configure our DNS server. Note that you can choose to use your own DNS server configuration.
git clone https://github.com/rossbrigoli/okd4_files.git
21. I have prepared a script that will edit the files according to the chosen cluster name and base domain name. These should be the same names as the ones you have chosen when downloading the OpenShift image from the Red Hat Hybrid console. In the same directory, run the setdomain.sh script.
cd okd4_files ./setdomain.sh mycluster mydomain.com
sudo cp named.conf /etc/named.conf
sudo cp named.conf.local /etc/named/
sudo mkdir /etc/named/zones
sudo cp db* /etc/named/zones
sudo systemctl restart named
sudo firewall-cmd --permanent --add-port=53/udp
sudo firewall-cmd --reload
sudo systemctl status named
nslookup okd4-compute-1.mycluster.mydomain.com
Load Balancer
dnf install -y haproxy
cd
sudo cp okd4_files/haproxy.cfg /etc/haproxy/haproxy.cfg
sudo setsebool -P haproxy_connect_any 1
sudo systemctl enable haproxy
sudo systemctl start haproxy
sudo systemctl status haproxysudo firewall-cmd --permanent --add-port=6443/tcp
sudo firewall-cmd --permanent --add-port=22623/tcp
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload
Post Installation
OpenShift CLI
oc) that matches your client computer's OS type and architecture.tar -xvzf ~/Downloads/openshift-client-mac.tar.gz
cp ~/Downloads/openshift-client-mac/oc /usr/local/bin
cp ~/Downloads/openshift-client-mac/kubectl /usr/local/bin
This will download a kubeconfig file. Now to access your cluster you need to set an environment variable KUBECONFIG with a value which is a path pointing to this kubeconfig file. The below example command works on Mac OS.
cp ~/Downloads/kubeconfig-noingress ~/
export KUBECONFIG=~/kubeconfig-noingressoc get nodesoc get co
Note: if you see errors in one or more Cluster Operator status, you can try to delete the pods of those operator, this will trigger self healing and help to un-stuck the installation process.
