OpenShift · UPI · Air-Gapped

Setting Up an OpenShift Cluster
User-Provisioned Infrastructure
in Air-Gapped Environments

A complete, step-by-step guide to manually provisioning VMs, load balancers, and DNS for an OpenShift cluster in a disconnected network.

Table of Contents

1. What is UPI?
2. Air-Gapped Prerequisites
3. OS Strategy (RHCOS)
4. Architecture & Helper Node
5. Cluster Node Roles
6. Core Deployment Workflow
7. Detailed Setup Steps
8. Configuration Details

1 What is UPI?

With User-Provisioned Infrastructure (UPI), you have maximum control over the cluster setup. You are responsible for manually preparing all virtual machines, load balancers, and DNS records before the OpenShift installer runs. This is the preferred approach for secure, air-gapped, or heavily regulated environments.

2 Air-Gapped Prerequisites

The primary challenge in a disconnected environment is the absence of direct access to the Red Hat Container Registry. You must bridge this gap before initiating the installation.

Mirror Registry

Establish a local container registry (e.g., Red Hat Quay or JFrog Artifactory) within your secure perimeter. Use the oc mirror plugin to sync OpenShift release images, operator catalogs, and Helm charts from the internet to a portable medium, then load them into your local registry.

Internal DNS & NTP

Precise time synchronization and split-horizon DNS are non-negotiable. Every node must be able to resolve the local registry hostname and all internal API endpoints.

3 OS Strategy

Red Hat enforces a specific OS strategy to ensure the self-healing nature of OpenShift.

⚠️

Control Plane (Masters): RHCOS is mandatory. Standard RHEL, Ubuntu, or any other OS cannot be used. Masters are managed by the Machine Config Operator (MCO), which requires an immutable, container-optimized OS to push updates, roll back kernel changes, and manage configurations automatically.

ℹ️

Compute Nodes (Workers): RHCOS is strongly recommended. When you update OpenShift, the OS on the workers updates automatically via rpm-ostree for safe, transactional updates. You have some flexibility here, but RHCOS is the supported default.

4 Architecture & Helper Node

This setup uses a Helper Node as the backbone — it acts as a bridge between the external network and the internal cluster network using two network interfaces.

Network Interfaces

Interface	Zone	Subnet	Role
ens192	External	192.168.0.X	Front-end / internet-facing traffic and corporate LB
ens224	Internal	192.168.22.X	Back-end cluster communication and storage traffic

Infrastructure Services Provided by the Helper Node

DNS (BIND)

Resolves cluster hostnames and API endpoints

DHCP

Assigns static IPs to all cluster nodes

NAT Gateway

Routes internal node traffic through the helper

HAProxy

Load balances API and Ingress traffic

Apache Web Server

Hosts Ignition files for automated installation

NFS Server

Provides persistent storage for the registry

5 Cluster Node Roles

Temporary

Bootstrap Node

Used only during the initial installation to orchestrate Control Plane creation. Decommissioned once the control plane is healthy.

×3 Nodes — HA

Control Plane (Masters)

The "brains" of the cluster. Runs the API server, etcd database, and controllers. Three nodes ensure high availability.

Compute

Worker Nodes

Where your actual applications, containers, and pods run. CSRs must be manually approved in UPI mode.

6 Core Deployment Workflow

Phase I — Configuration & Manifest Generation

On the Bastion host, define the cluster in install-config.yaml, pointing to your local mirror registry. Generate Kubernetes manifests and convert them to Ignition configs (.ign files) that RHCOS nodes execute on first boot.

Phase II — Infrastructure Provisioning

Configure a high-availability load balancer (HAProxy/F5) for the API (port 6443), Machine Config Server (port 22623), and Ingress (ports 80/443). Host the generated .ign files on an internal HTTP server.

Phase III — Bootstrap Sequence

Boot the Bootstrap node → it pulls its config and initiates control plane creation. Boot Masters → they form the etcd quorum. Boot Workers → manually approve their CSRs to join the cluster.

7 Detailed Setup Steps

Execute the following commands on the Helper Node in order.

Step 1 — Install Client Tools

# Extract and install the OpenShift client tools
tar xvf openshift-client-linux.tar.gz
mv oc kubectl /usr/local/bin

# Verify installation
kubectl version
oc version

# Extract the OpenShift Installer
tar xvf openshift-install-linux.tar.gz

Step 2 — Configure Static IP for Internal NIC

Run nmtui-edit ens224 or edit /etc/sysconfig/network-scripts/ifcfg-ens224 with these values:

Address:       192.168.22.1
DNS Server:    127.0.0.1
Search Domain: ocp.lan
Default Route: Disabled
Auto-connect:  Enabled

If changes don't apply, bounce the NIC: nmcli connection down ens224 && nmcli connection up ens224

Step 3 — Configure Firewall Zones

# Assign interfaces to zones
nmcli connection modify ens224 connection.zone internal
nmcli connection modify ens192 connection.zone external

# Enable masquerading (NAT) on both zones
firewall-cmd --zone=external --add-masquerade --permanent
firewall-cmd --zone=internal --add-masquerade --permanent
firewall-cmd --reload

# Verify zones and IP forwarding
firewall-cmd --get-active-zones
firewall-cmd --list-all --zone=internal
firewall-cmd --list-all --zone=external
cat /proc/sys/net/ipv4/ip_forward   # Should return 1

Step 4 — Clone Config Repository

dnf update -y
dnf install git -y
git clone https://github.com/ryanhay/ocp4-metal-install

Step 5 — Install & Configure DNS (BIND)

dnf install bind bind-utils -y
cp ~/ocp4-metal-install/dns/named.conf /etc/named.conf
cp -R ~/ocp4-metal-install/dns/zones /etc/named/

# Open firewall for DNS
firewall-cmd --add-port=53/udp --zone=internal --permanent
firewall-cmd --add-port=53/tcp --zone=internal --permanent  # Required for OCP 4.9+
firewall-cmd --reload

# Enable and start BIND
systemctl enable named && systemctl start named && systemctl status named

Update the external NIC (ens192) to use 127.0.0.1 as its DNS server and enable "Ignore automatically obtained DNS parameters" via nmtui-edit ens192, then restart NetworkManager:

systemctl restart NetworkManager

# Verify DNS resolution
dig ocp.lan
dig -x 192.168.22.200   # Should resolve to ocp-bootstrap.lab.ocp.lan

Step 6 — Install & Configure DHCP

⚠️

Before copying the config, update ~/ocp4-metal-install/dhcpd.conf with the actual MAC addresses of each cluster machine.

dnf install dhcp-server -y
cp ~/ocp4-metal-install/dhcpd.conf /etc/dhcp/dhcpd.conf

firewall-cmd --add-service=dhcp --zone=internal --permanent
firewall-cmd --reload

systemctl enable dhcpd && systemctl start dhcpd && systemctl status dhcpd

Step 7 — Install & Configure Apache Web Server

dnf install httpd -y
# Change Apache to listen on port 8080 (avoids conflicts)
sed -i 's/Listen 80/Listen 0.0.0.0:8080/' /etc/httpd/conf/httpd.conf

firewall-cmd --add-port=8080/tcp --zone=internal --permanent
firewall-cmd --reload

systemctl enable httpd && systemctl start httpd && systemctl status httpd

# Verify it's running
curl localhost:8080

Step 8 — Install & Configure HAProxy

dnf install haproxy -y
cp ~/ocp4-metal-install/haproxy.cfg /etc/haproxy/haproxy.cfg

Open the required firewall ports:

# Control plane API
firewall-cmd --add-port=6443/tcp --zone=internal --permanent
firewall-cmd --add-port=6443/tcp --zone=external --permanent

# Machine Config Server
firewall-cmd --add-port=22623/tcp --zone=internal --permanent

# Application ingress (HTTP/HTTPS)
firewall-cmd --add-service=http --zone=internal --permanent
firewall-cmd --add-service=http --zone=external --permanent
firewall-cmd --add-service=https --zone=internal --permanent
firewall-cmd --add-service=https --zone=external --permanent

# HAProxy stats UI (accessible at http://<helper-ip>:9000/stats)
firewall-cmd --add-port=9000/tcp --zone=external --permanent
firewall-cmd --reload

# Allow HAProxy SELinux binding and start the service
setsebool -P haproxy_connect_any 1
systemctl enable haproxy && systemctl start haproxy && systemctl status haproxy

Step 9 — Install & Configure NFS Server

dnf install nfs-utils -y

mkdir -p /shares/registry
chown -R nobody:nobody /shares/registry
chmod -R 777 /shares/registry

echo "/shares/registry  192.168.22.0/24(rw,sync,root_squash,no_subtree_check,no_wdelay)" > /etc/exports
exportfs -rv

firewall-cmd --zone=internal --add-service=mountd --permanent
firewall-cmd --zone=internal --add-service=rpc-bind --permanent
firewall-cmd --zone=internal --add-service=nfs --permanent
firewall-cmd --reload

systemctl enable nfs-server rpcbind
systemctl start nfs-server rpcbind nfs-mountd

Step 10 — Generate Installation Files

mkdir /var/www/html/ocp4
cp ~/ocp4-metal-install/install-config.yaml /var/www/html/ocp4

ℹ️

Edit install-config.yaml before proceeding: insert your Pull Secret and SSH public key. See the Configuration Details section below for guidance.

# Generate Kubernetes manifests
~/openshift-install create manifests --dir /var/www/html/ocp4

To control whether workloads can run on Control Plane nodes, edit the scheduler manifest:

ls ~/ocp-install/manifests/cluster-scheduler-02-config.yml

# Set mastersSchedulable: true  → allow workloads on masters
# Set mastersSchedulable: false → prevent workloads (default)

# Generate Ignition configs and auth files
~/openshift-install create ignition-configs --dir /var/www/html/ocp4

Step 11 — Host RHCOS Image and Set Permissions

# Move the RHCOS metal image to the web server
mv ~/rhcos-X.X.X-x86_64-metal.x86_64.raw.gz /var/www/html/ocp4/rhcos

# Set correct SELinux context, ownership, and permissions
chcon -R -t httpd_sys_content_t /var/www/html/ocp4/
chown -R apache: /var/www/html/ocp4/
chmod 755 /var/www/html/ocp4/

# Confirm all files are accessible
curl localhost:8080/ocp4/

Step 12 — Boot Cluster Nodes

Boot each node type using the RHCOS ISO or PXE, passing the appropriate Ignition file URL via kernel arguments.

# Bootstrap Node
sudo coreos-installer install /dev/sda \
  -u http://192.168.22.1:8080/ocp4/rhcos \
  -I http://192.168.22.1:8080/ocp4/bootstrap.ign \
  --insecure --insecure-ignition

# Control Plane (Master) Nodes
sudo coreos-installer install /dev/sda \
  -u http://192.168.22.1:8080/ocp4/rhcos \
  -I http://192.168.22.1:8080/ocp4/master.ign \
  --insecure --insecure-ignition

# Worker Nodes
sudo coreos-installer install /dev/sda \
  -u http://192.168.22.1:8080/ocp4/rhcos \
  -I http://192.168.22.1:8080/ocp4/worker.ign \
  --insecure --insecure-ignition

Step 13 — Monitor Bootstrap & Finalize

# Monitor bootstrap progress from the Helper Node
~/openshift-install --dir /var/www/html/ocp4/ wait-for bootstrap-complete --log-level=debug

Once bootstrapping completes, remove the Bootstrap node from HAProxy and shut it down:

# Remove ocp-bootstrap from /etc/haproxy/haproxy.cfg, then reload
systemctl reload haproxy

# Approve Worker CSRs so workers can join the cluster
oc get csr
oc adm certificate approve <csr-name>

# Verify all nodes are Ready
oc get nodes

Step 14 — Post-Installation

# Retrieve console URL and kubeadmin password
cat ~/ocp-install/auth/console-url
cat ~/ocp-install/auth/kubeadmin-password

• Configure Storage: Define StorageClasses (NFS, OCS, or local storage) so applications can persist data.
• Set Up Identity Providers: Replace the temporary kubeadmin user with a permanent solution such as LDAP or OAuth.

8 Configuration Details

The install-config.yaml Blueprint

This is the only file you create manually. It acts as the blueprint for the entire installation. Key fields to populate:

• pullSecret — Authorizes nodes to pull OpenShift images from Red Hat registries.
• sshKey — Allows SSH access into RHCOS nodes as the core user for troubleshooting.
• networking — Defines cluster and service network CIDRs.
• imageContentSources — Points to your local mirror registry (required for air-gapped installs).

How to Get the Pull Secret

1. Log in to the Red Hat OpenShift Cluster Manager at cloud.redhat.com/openshift.
2. Download the pull secret using the "Download pull secret" button.
3. Paste the entire single-line JSON string into your install-config.yaml inside single quotes.

How to Get the SSH Key

# Check for existing keys
ls ~/.ssh/id_rsa.pub || ls ~/.ssh/id_ed25519.pub

# Generate a new key pair (if needed)
ssh-keygen -t ed25519 -f ~/.ssh/id_ocp -C "admin@ocp-cluster"

# Output the public key to copy into install-config.yaml
cat ~/.ssh/id_ocp.pub

---

OpenShift Client vs. Installer — Quick Reference

Feature	OpenShift Client (oc)	OpenShift Installer
Filename	openshift-client-linux.tar.gz	openshift-install-linux.tar.gz
Primary Goal	Managing an existing cluster	Creating or destroying a cluster
Main Binary	oc (and kubectl)	openshift-install
Usage Period	Daily, for the life of the cluster	Primarily during Day 1 setup
Capabilities	Deploy apps, check logs, manage users	Provision VMs, generate Ignition files

Helper Node Interface Roles

Interface	Typical Role	Description
ens192	External / Public	Front-end traffic — connects to the internet or corporate load balancer to serve applications.
ens224	Internal / Private	Back-end traffic — master/worker node communication, storage traffic (CSI/NFS).