OpenShift 4.15 Cluster Installation Guide Using Assisted Installer

Introduction

This guide provides a detailed walkthrough for installing an OpenShift 4.15 cluster on a VMware vSphere host using the Assisted Installer. This installation will set up a cluster with 5 nodes and includes steps for post-installation configurations such as LDAP integration, NFS dynamic storage provisioning, and configuring the Image Registry.

Prerequisites

Before beginning the installation, ensure you have the following prerequisites:

1. vSphere Infrastructure:

   • vSphere 6.7 or higher.
   • Administrative access to the vSphere environment.

2. Hardware Requirements:

   • Master Nodes: At least 3 nodes with a minimum of 8 vCPUs and 16 GB RAM each.
   • Worker Nodes: At least 2 nodes with a minimum of 2 vCPUs and 8 GB RAM each.
   • Storage: Sufficient disk space for the nodes, typically at least 120 GB for masters and 100 GB for workers.

3. Network Requirements:

   • Internet Access: Nodes must have access to the internet for pulling necessary images and packages.

     • DNS: Proper DNS resolution for the cluster domain.

       • Static name entries for the wildcard routes and the API:

       *.apps.<clustername>.<yourdomain>.        IN	A	192.168.1.11
       ;
       api.<clustername>.<yourdomain>.       IN	A	192.168.1.10
       api-int.<clustername>.<yourdomain>.       IN	A	192.168.1.10
       

       • Name resolution for master and worker nodes e.g.:

       control0.<clustername>.<yourdomain>.     IN	A	192.168.1.100
       control1.<clustername>.<yourdomain>.     IN	A	192.168.1.101
       control2.<clustername>.<yourdomain>.     IN	A	192.168.1.102
       ;
       worker0.<clustername>.<yourdomain>.      IN	A	192.168.1.110
       worker1.<clustername>.<yourdomain>.      IN	A	192.168.1.111
       

4. Software Requirements:

   • OC CLI: Install the OpenShift CLI (oc) tool on your local machine.

5. Accounts and Keys:

   • Access to the Red Hat OpenShift Cluster Manager.
   • Sufficient privileges to create resources in your infrastructure (bare metal, AWS, VMware, etc.).
   • Ensure you have a valid Red Hat subscription to access the necessary resources.
   • SSH key to connect to the Red Hat CoreOS system (RHCOS).

Assisted Installer

The Assisted Installer simplifies the deployment of OpenShift clusters. It provides a web-based interface to guide you through the installation process and automates many of the tasks involved.

Download the Assisted Installer ISO

1. Accessing the Assisted Installer:

   • Visit the Red Hat OpenShift Cluster Manager and log in with your Red Hat account.
   • Click on "Assisted Installer clusters"

2. Creating a New Cluster:

   • Click on "Create New Cluster."
   • Provide the cluster name, domain, and select the OpenShift version (4.15).
     • Cluster name: , for example "ocp-test"
     • Base domain: Your domain, for example "example.com"
     • Hosts' network configuration: "Static IP, bridges, and bonds"
   • Click "Next".

3. Static network configurations:

   • Configure the network settings (subnet, DNS, and gateway).
     • DNS: The IP address of your DNS server, e.g. "192.168.1.1".
     • Machine Network: The network in which the machine is located (CIDR format), e.g. "192.168.1.0/24".
     • Default Gateway: Your gateway, e.g. "192.168.1.1".
   • Click on "Next"
   • Define the 3 master (Host 1-3) and 2 worker nodes (Host 4-5).
     • MAC Address: Define the MAC Address for the Hosts, e.g. "00:50:56:b7:be:0e" (later used for the VM's).
     • IP address (IPv4): Define the IP Address for the Hosts, e.g. "192.168.1.10" (later used for the VM's).
   • Now click "Next" to go to the next step, which you can skip by clicking "Next" again.

4. Download discovery ISO:

   • Click on "Add hosts":
     • Provisioning type: Minimal image file.
     • SSH public key: Use the prepared ssh-key (id_rsa.pub).
   • Click "Generate Discovery ISO".
   • Click "Download Discovery ISO".
   • This ISO is used to boot the nodes and automatically register them with the "assisted installer".

VMware Preparation

Before beginning the installation of your OpenShift 4.15 cluster, you need to prepare the VMware vSphere environment and set up the virtual machines (VMs) for the cluster. Quick steps:

1. Log in to vSphere.
2. Upload the Discovery ISO.
3. Create the new virtual machine's with the specs mentioned above:
   • Use the "Discovery ISO" as CD-/DVD media.
   • Use the MAC and IP Addresses you already have assigned to the hosts.
   • The following options must be set:
     • Extended parameters: "disk.EnableUUID = TRUE"
     • CPU: "Provide hardware-supported virtualization for the guest operating system"

Installation

After preparing the VMs in your VMware vSphere environment and attaching the discovery ISO, you can proceed with the host discovery step in the Assisted Installer.

1. Booting the Nodes:

   • Power on each VM in your vSphere environment. Ensure they boot from the attached discovery ISO. This will automatically configure the nodes to register with the Assisted Installer.

2. Assigning Roles & Change Names:

   • In the Assisted Installer interface, you will need to assign roles to each of the registered nodes.
     • Assign the role of "Control plane node" (master) to the three master nodes (control0, control1, and control2).
     • Assign the role of "Worker" to the two worker nodes (worker1 and worker2).
   • If the host names are not correct, change them to the correct ones using the three bullet points.
   • Click "Next".

3. Storage:

   • Ensure that the storage options are set correctly.
   • Click "Next".

4. Networking:

   • API IP: Use the prepared api. IP address, e.g. "192.168.1.10".
   • Ingress IP: Use the prepared *.apps. IP address, e.g. "192.168.1.11".
   • Review and confirm the settings before proceeding.
   • Click "Next".

5. Review and create:

   • Check the settings in the cluster summary again.
   • If all is correct start the installation.
   • Click "Install cluster".
   • The Assisted Installer will handle the installation of the OpenShift 4.15 cluster, configuring the control plane, and setting up the worker nodes automatically.

6. Monitoring & Credentials:

   • Keep an eye on the installation's progress through the Assisted Installer's user interface. The installation may take several minutes.
   • Once the installation is complete, you will be notified that the cluster is ready.
   • Download the "kubeconfig" file. This file contains the necessary credentials and settings to securely connect to your OpenShift cluster.
   • Also write down the username and password that will be displayed later in the installation process.
   • Click "Launch OpenShift Console".

Congratulations! You have successfully installed your OpenShift 4.15 cluster using the Assisted Installer. Your cluster is now ready to access and perform some post-installation steps.

Post Installation

Once the installation of your OpenShift 4.15 cluster is complete, the focus will be on some post-installation tasks. These include accessing the cluster, establishing the LDAP connection, setting up NFS Dynamic Storage Provisioning and activating the image registry.

LDAP

1. Log in:

   • Open your web browser and navigate to the OpenShift web console URL, typically https://console-openshift-console.apps.<clustername>.<yourdomain>.
   • Log in with the "kubeadmin" account credentials you have written down earlier.

2. Access OAuth:

   • Click on the Administration menu on the left sidebar.
   • Select Cluster Settings from the dropdown.
   • Click on the Configuration tab.
   • Locate and click on OAuth in the list of configurations.

3. Add LDAP Identity Provider:

   • In the Identity Providers section, click Add.
   • Select LDAP from the list of identity provider types.

4. Configure LDAP Details:

   • Fill in the following details in the LDAP configuration form:
     • Name: A name for your LDAP provider (e.g., ldap).
     • URL: The URL of your LDAP server in the format ldaps://<ldap_server_address>:636/ou=Users,dc=example,dc=com?uid.
     • Bind DN: The distinguished name to bind as (e.g., cn=admin,dc=example,dc=com).
     • Bind Password: The password for the bind DN.
     • Attributes: Map the LDAP attributes to OpenShift user fields:
       • ID: e.g. dn.
       • Preferred Username: e.g. uid.
       • Name: e.g. cn.
       • Email: e.g. mail.
     • CA file: Optionally, if your LDAP server uses its own certification authority, upload the CA certificate file.
   • Click "Add".

5. Test LDAP Authentication:

   • Log out of the OpenShift web console.
   • Try logging in with an LDAP user account to ensure that the configuration is working correctly. (It may take a while for the LDAP icon to appear)

Disable kubeadmin account

The kubeadmin user is created during the OpenShift installation and provides temporary cluster administrator access. For security reasons, it is recommended to disable this account once permanent administrators are configured.

Prerequisites

• OC tools: The OpenShift CLI (oc) installed and configured to communicate with your OpenShift cluster.
• LDAP server: An LDAP identity provider already configured and operational.
• New Admin user: A user that you want to use as an administrator account, e.g. exampleAdmin

Disable kubeadmin

1. Open a terminal.

2. Log in to your OpenShift cluster using the kubeadmin credentials:

   oc login -u kubeadmin -p <password> https://api.<clustername>.<yourdomain>:6443
   

3. Give th cluster-admin role to the dedicated user account:

   oc adm policy add-cluster-role-to-user cluster-admin exampleAdmin
   

4. Verify the new account privileges:

   • Logout.

   • Login with the exampleAdmin account.

   • Ensure the privileges by running several administrative commands:

     oc get projects
     oc get nodes
     

5. Disable the kubeadmin Account by deleting the secret:

   oc delete secret kubeadmin -n kube-system
   

NFS Dynamic Storage Provisioning

This section details the steps required to set up NFS dynamic storage provisioning using the "nfs-subdir-external-provisioner" project from GitHub. This assumes that an NFS server is already available and configured.

Prerequisites

1. OpenShift 4.15 Cluster: Ensure your OpenShift cluster is up and running.
2. NFS Server: An operational NFS server with an exported directory that will be used for provisioning storage.

Installation

1. Copy rbac.yaml and deployment.yaml from GitHub:

   curl https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/rbac.yaml > nfs-subdir-external-provisioner-rbac.yaml
   curl https://raw.githubusercontent.com/kubernetes-sigs/nfs-subdir-external-provisioner/master/deploy/deployment.yaml > nfs-subdir-external-provisioner-deployment.yaml
   

2. Create the service account and roles:

   oc create -f nfs-subdir-external-provisioner-rbac.yaml
   oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:default:nfs-client-provisioner
   

3. Edit and apply nfs-subdir-external-provisioner-deployment.yaml:

   • Edit nfs-subdir-external-provisioner-deployment.yaml to suit your NFS server configuration. Update the environment variables with your NFS server's details:
     • YOUR_NFS_SERVER: e.g. "192.168.1.200".
     • YOUR_NFS_EXPORT_PATH: e.g. "/var/nfs".

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nfs-client-provisioner
     labels:
       app: nfs-client-provisioner
     # replace with namespace where provisioner is deployed
     namespace: default
   spec:
     replicas: 1
     strategy:
       type: Recreate
     selector:
       matchLabels:
         app: nfs-client-provisioner
     template:
       metadata:
         labels:
           app: nfs-client-provisioner
       spec:
         containers:
           - name: nfs-client-provisioner
             image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
             volumeMounts:
               - name: nfs-client-root
                 mountPath: /persistentvolumes
             env:
               - name: PROVISIONER_NAME
                 value: k8s-sigs.io/nfs-subdir-external-provisioner
               - name: NFS_SERVER
                 value: <YOUR_NFS_SERVER>
               - name: NFS_PATH
                 value: <YOUR_NFS_EXPORT_PATH>
         volumes:
           - name: nfs-client-root
             nfs:
               server: <YOUR_NFS_SERVER>
               path: <YOUR_NFS_EXPORT_PATH>
   

   • Apply nfs-subdir-external-provisioner-deployment.yaml:

   oc create -f nfs-subdir-external-provisioner-deployment.yaml
   

4. Create StorageClass:

   • Define a new StorageClass that uses the NFS Client Provisioner:

   oc create -f <(echo '{
      "apiVersion": "storage.k8s.io/v1",
      "kind": "StorageClass",
      "metadata": {
         "name": "managed-nfs-storage",
         "annotations": {
           "storageclass.kubernetes.io/is-default-class": "true"
         }
      },
      "provisioner": "k8s-sigs.io/nfs-subdir-external-provisioner",
      "parameters": {
         "pathPattern": "${.PVC.namespace}-${.PVC.name}",
         "archiveOnDelete": "false"
      }
   }')
   

5. Verify Provisioning:

   • Create a PersistentVolumeClaim (PVC) to test dynamic provisioning. Or wait until you have completed the "Image Registry" section.
   • Ensure the PVC is bound and the NFS volume is created.

Image Registry

In this section, we will configure the OpenShift Image Registry to use NFS dynamic storage provisioning, as previously set up. The OpenShift Image Registry is essential for storing container images used by your applications.

1. Create Persistent Volume Claim (PVC):

   • Open a terminal and execute the following command to create a PVC for the Image Registry (names are just examples):

   oc create --namespace openshift-image-registry -f <(echo '
   apiVersion: v1
   kind: PersistentVolumeClaim
   metadata:
     name: nfs4registry-pvc-0001
     namespace: openshift-image-registry
   spec:
     accessModes:
       - ReadWriteMany
     resources:
       requests:
         storage: 100Gi
     storageClassName: managed-nfs-storage
     volumeMode: Filesystem
   ')
   

2. Edit Image Registry Configuration:

   • Edit the Image Registry configuration to use the newly created PVC:

     oc edit configs.imageregistry.operator.openshift.io
     

   • In the editor, update the configuration to include the PVC:

     spec:
       storage:
         pvc:
           claim: nfs4registry-pvc-0001
     

3. Set Management State to Managed:

   • Apply the following command to set the management state of the Image Registry to Managed:

     oc patch configs.imageregistry/cluster --patch '{"spec":{"managementState":"Managed"}}' --type=merge
     

4. Verify the Storage Configuration:

   • Ensure that the Image Registry status has changed to Managed and the storage configuration reflects the NFS dynamic storage provisioning.
   • Verify that the PVC is bound to the NFS storage.

5. Check the Registry Pod:

   • Navigate to the Workloads menu and select Pods.
   • Filter the pods by the openshift-image-registry namespace.
   • Verify that the Image Registry pod is running without issues.

Further links about OpenShift 4.15

OpenShift Container Platform 4.15 release notes

https://docs.openshift.com/container-platform/4.15/release_notes/ocp-4-15-release-notes.html