A Practical Disaster Recovery Strategy for a Three Node Bare Metal Cluster. In this guide we shall discuss “Backing Up OpenShift Is Not the Same as Recovering It”. I see the same assumption come up in disaster recovery planning: the backup job completed, so it’s recoverable. That assumption breaks down as soon as the team has to rebuild more than the application. Please see Veeam Agent for Windows deployment failing with Access Denied, and Restoring a Physical Ubuntu Server to VMware using Veeam Agent and Export as Virtual Disks.
A backup can be healthy while the recovery plan is incomplete. It’s especially important with a compact OpenShift cluster, where the same three nodes run the control plane and application workloads. After a site loss, hardware and network services come first. OpenShift, storage, and the backup platform follow. Application dependencies still have to be available before a restore helps anyone.
Veeam Kasten is an important part of that design. It can protect Kubernetes resources and persistent data. Site reconstruction and bare metal recovery for Red Hat Enterprise Linux CoreOS nodes remain separate work, outside the documented Kasten application protection scope.
This article explains where Kasten fits, what still needs a separate recovery method, and why the order of recovery matters.
At Recovery Point, we work with these layered recovery problems. The work is rarely about installing one more product. We make sure each part of the service has an owner, a recovery method, and a test that proves it works.
Key Takeaways
- A successful Kasten policy does not prove that an OpenShift site can be rebuilt.
- A three node OpenShift cluster requires two available etcd members for quorum.
- An etcd backup protects cluster state. It is not a replacement for application data protection.
- OpenShift Data Foundation must be available and healthy before persistent application data can be restored to its storage classes.
- A protected ObjectBucketClaim does not, by itself, prove that the objects inside the bucket are protected.
- Local CSI snapshots are useful for fast recovery, but a site loss design needs exported backups outside the production failure domain.
- The only meaningful recovery test ends with a working application and a verified business transaction.
Please see Action required: Update Devolutions Remote Desktop Manager Before July 21, how to Back Up and Restore Microsoft Entra ID with Veeam Backup & Replication v13, and Azure Resource Inventory (ARI) for Engineers: Improving Control and Compliance.
Why a Compact Cluster Changes the Recovery Problem
OpenShift supports a three node cluster where each control plane node can also schedule application workloads. Red Hat installation guidance says not to deploy separate compute machines when installing this topology.
Compact deployment saves hardware and works well where space and capacity are limited. It also combines several failure domains.
If one node is unavailable, the cluster has less control plane resilience and less workload capacity. A three member etcd cluster requires two members for quorum. If two members are unavailable, etcd no longer has the majority it needs.
A full site loss is a bigger problem. There may be no surviving node, no working API, and no storage system available to present persistent volumes. Bare metal recovery adds a long dependency chain. Hardware inventory and remote management must be ready. Firmware, boot configuration, and installation media must match the runbook. Network addressing and DNS have to work before the API and ingress endpoints become useful. Time services, certificates, and registry access finish the path into the platform. None of those items come from an application restore point.
Please see Azure Load Balancer: Configuring for SQL Server Always On Availability Group Listener on Azure Virtual Machines, and Azure Virtual Desktop: Build Custom Session Host Images Using Image Builder Templates [Part 07].
What Kasten Protects
Kasten protects Kubernetes applications through policies. Its current documentation describes snapshot actions with optional export actions. The export converts application metadata and volume snapshot data into a Kasten managed backup stored in an object storage, file storage, or supported Veeam repository location.
At the time of this validation, July 20, 2026, the Veeam Kasten 9.0.1 support table lists Kubernetes 1.31 with OpenShift 4.18. Check the current matrix again before an installation, upgrade, or recovery because support tables change.
For this kind of OpenShift environment, Kasten can protect:
- Namespaced Kubernetes resources included by the policy
- Persistent volume data supported by the configured storage integration
- Application metadata needed for restore
- Selected cluster scoped resources when they are explicitly included and protected
- Application consistent data when a supported hook or Kanister blueprint is configured and tested
- Exported restore points that can be imported by another Kasten instance on a compatible cluster
Kasten supports Ceph RBD and CephFS snapshots and backups through their CSI drivers. Its storage documentation also requires the Ceph VolumeSnapshotClass used by Kasten to carry the k10.kasten.io/is-snapshot-class=true annotation.
That annotation shouldn’t be added blindly. Run the Kasten Primer checks against the intended storage classes first. Primer creates a test volume, writes data, takes a snapshot, creates a new volume from that snapshot, and validates the restored data. That is a much better starting point than assuming an existing snapshot class works because it appears in the API.
Please see Veeam Agent for Windows: VSS Error Caused by Old SQL Server References, Azure Virtual Desktop: Autoscaling Implementing and Monitoring Session Hosts [Part 05], and how to implement Azure Private Link for Azure Virtual Desktop [Part 06].
What Kasten Does Not Replace
Kasten application protection covers applications and their selected data. A bootable OpenShift recovery site requires additional controls. The following items need their own recovery plan:
- Replacement or standby bare metal servers
- Baseboard management controller access and remote console capability
- OpenShift installation assets and version information
- DNS records for the API and application ingress
- Load balancer or virtual IP configuration
- Network routes, firewall rules, and time services
- Pull secrets, registry access, certificates, and identity services
- A supported OpenShift control plane recovery procedure
- OpenShift Data Foundation deployment and validation
- External databases, registries, mail relays, license services, and application integrations
- Object data that is not inside a protected persistent volume
The product is working within its documented scope. Problems start when that boundary is ignored.
CSI Snapshots Need an Independent Backup Copy
CSI snapshots are useful. They can provide fast recovery and avoid reading an entire volume for every local restore. But snapshots depend on the storage system that owns them.
If the OpenShift cluster and its internal Ceph storage are lost together, local snapshot references may no longer be usable. Kasten makes the same distinction in its documentation. Snapshot exports create an infrastructure independent backup format in a configured external location.
For site recovery, I’d treat the local snapshot as the fast copy and the export as the recovery copy. The export location should not share the same physical and administrative failure domain as the production cluster.
There’s another important detail. Kasten exclusively manages the contents of its backup repositories. Veeam warns against manually changing, copying, or otherwise manipulating those repositories outside supported procedures. If immutability or replication is required, it should be implemented through a supported location design and validated against the current Kasten documentation.
Please see Azure Virtual Desktop: Autoscaling Implementing and Monitoring Session Hosts [Part 05], Disaster Recovery Test Checklist: What to Capture Before You Start, and Upgrading Azure AD Connect to Microsoft Entra Connect Sync.
Etcd Backup Solves a Different Problem
OpenShift stores cluster state in etcd. Red Hat describes etcd backup as an important part of control plane disaster recovery.
An etcd backup can support recovery when the cluster loses control plane state or when a critical cluster resource is deleted. Application data stored in persistent volumes or external buckets requires separate protection.
Control plane restoration also needs care. Red Hat warns that restoring an earlier cluster state is a disruptive operation. It can create differences between the state in etcd and the data that still exists on disks or in external systems.
For OpenShift 4.18, Red Hat requires the etcd backup used by the documented cluster state restore procedure to come from the same z stream release. The backup set includes the etcd snapshot and the static pod resources created by the backup process.
OpenShift 4.18 also includes a separate quorum restoration path when at least one suitable control plane node remains. Quorum restoration and restoration from an older etcd snapshot address different conditions. The recovery team must identify the actual failure before choosing a procedure.
Protect etcd for the control plane and protect applications for the services running on the cluster. Each backup has a different job.
Rebuild ODF Before Restoring Application Volumes
OpenShift Data Foundation uses Ceph and Ceph CSI to provide block and file storage. On bare metal, Red Hat requires at least three OpenShift worker or infrastructure nodes with locally attached raw block devices for a standard internal deployment. In a compact cluster, the control plane nodes also carry the worker role.
When a site and its local storage devices are gone, deploying ODF at the recovery site creates a new storage platform with empty target storage. The application data must come from a surviving replication target, a supported storage recovery method, or exported application backups. That distinction matters during both design and testing.
The recovery team should install ODF, present the required raw devices, create or validate the storage system, and confirm Ceph health before restoring persistent applications. The expected storage classes and CSI components must exist before Kasten can create target volumes and write recovered data.
If the original Ceph devices survive, the situation is different. Device, node, and cluster recovery should follow the applicable Red Hat ODF procedure. A site rebuild runbook should not assume that a clean ODF deployment and a recovery of surviving ODF media are the same operation.
Please see PCI Driver Error: Connection to Deployment Share could not be made, Steps to Fix the “Invalid OS GUID” Error in MDT, and Faulting SecHealthUI: Resolve Windows Defender Notification.
An ObjectBucketClaim Is Not the Object Data
An ObjectBucketClaim is a Kubernetes resource that requests a bucket. In an ODF environment, creating an OBC can also create a Secret with credentials and a ConfigMap with connection information.
Protecting those Kubernetes resources preserves useful configuration. Object protection still needs its own evidence.
If an application depends on S3 compatible object data, protect the objects through a supported method. That may be storage replication, bucket versioning and retention, an object backup product, or an application export. The correct method depends on the object platform and the application.
A useful recovery test retrieves representative objects and confirms that the application can read them. An OBC and its generated Secret prove configuration recovery while leaving object recovery untested.
Protect the Kasten Catalog
Kasten is a stateful application. Its own catalog and configuration need protection. Kasten Disaster Recovery, called KDR in the documentation, protects Kasten resources and the restore point catalog through a dedicated policy. Current Kasten releases use Quick DR modes. The selected mode determines whether catalog snapshot data is kept locally, exported, or omitted.
For recovery to another cluster, exported application restore points remain important because local storage snapshot references are usually not available there. Kasten also requires the source cluster ID, the KDR location information, credentials, and the recovery passphrase or supported external secret manager details.
Store that information outside the cluster. If the cluster ID or KDR secret information is lost with the site, catalog recovery can fail even though backup data still exists.
Cold Recovery and Warm Recovery
There are two practical models for this environment.
Cold Recovery
Cold recovery starts with replacement hardware but no running OpenShift cluster. It costs less while everything is healthy. It also takes longer because the team must build the platform before application recovery begins. Hardware readiness and installation assets start the critical path. Network services and ODF come next. Kasten installation, catalog recovery, and application restoration complete it.
Cold recovery can work when the business accepts the longer recovery time and the organization tests the complete build process.
Warm Recovery
Warm recovery uses a separate OpenShift cluster that is already running. It can shorten recovery because several components are prepared before an incident: the OpenShift control plane, network services, storage, and Kasten. The model costs more and creates an ongoing configuration management problem.
The recovery cluster needs compatible OpenShift and Kasten versions, enough ODF capacity, the required operators and custom resource definitions, working registry access, current credentials, and connectivity to the backup location. Storage class and namespace transformations must be tested before a disaster.
A warm cluster that has drifted from production remains unprepared even when its nodes show Ready.
Please see How to Update Your AMD Radeon Display Adapter Driver, how to update Proxmox VE to the latest version 9.2.3 today, and Secure Boot 2023 Compliance Across WinPE, MDT, WDS, and ADK: Boot Chain Alignment and PXE Validation [Final Part].
A Recovery Sequence That Respects Dependencies
The exact runbook will vary, but the major order should remain consistent.
- Declare the incident and select the recovery point.
- Stop automation that could write to both the original and recovery environments.
- Validate recovery site hardware, remote management, network paths, DNS, load balancing, time services, identity, and registry access.
- Install or activate a compatible OpenShift cluster.
- Use the Red Hat control plane recovery procedure that matches the failure. Do not restore etcd by default when a healthy API and quorum still exist.
- Install or recover ODF and confirm Ceph, CSI, storage classes, and capacity are healthy.
- Install Kasten and recover its catalog through the documented KDR process when required.
- Restore explicitly protected cluster scoped resources, operators, and shared services in their documented order.
- Restore application namespaces and persistent data in dependency order.
- Restore or reconnect object data and external services.
- Validate login, APIs, data integrity, background processing, monitoring, and integrations.
- Complete a representative business transaction before traffic is moved.
The order is important. Restoring an application before its operator, storage class, identity provider, or external database is ready creates noise and makes the real failure harder to see.
Common Mistakes
Keeping Every Copy in the Production Cluster
Local snapshots are fast, but they remain tied to production storage. Export critical restore points to a supported external location.
Treating a Green Job as Application Validation
Job completion alone leaves database consistency unproven. Use a supported application hook, blueprint, native dump, or replication method when the application requires it. Then test the recovered data.
Assuming All Cluster Resources Are Included
Kasten application policies primarily protect the resources selected by the policy. Cluster scoped resources require deliberate protection and restore decisions. Review filters carefully because an include or exclude rule can omit something the application needs.
Please see Fix an error occurred while attempting to start selected VM on Hyper-V, Failed to Upgrade VIHR Component: Failed to open deployer Service Management Port, and The Backup Was Safe: The Data Center Was not: A Real-World Lesson About Hidden Data Center Risks and Governance Failures.
Restoring Applications Before Storage
Persistent volume recovery depends on healthy target storage. ODF and CSI validation come first.
Restoring Every Secret Without Review
A security incident may require credentials and certificates to be rotated instead of restored. The runbook should identify which secrets can return and which must be reissued.
Testing Only an In Place Restore
An in place namespace restore proves one recovery path. A clean cluster test covers access to exported backups and reconstruction of the full service.
What I Would Require Before Calling This Recoverable
I’d want to see evidence for each of these items:
- The OpenShift installation assets are current, protected, and usable.
- The exact OpenShift release is recorded with each etcd backup set.
- The Kasten support matrix has been checked for the OpenShift and Kubernetes versions in use.
- Kasten Primer has passed for every storage class used by protected workloads.
- The required VolumeSnapshotClass annotations have been validated.
- Critical policies export restore points outside the production failure domain.
- KDR is enabled and the cluster ID, location details, credentials, and secret information are stored safely outside the cluster.
- Database and object data protection methods have been tested with the application owners.
- A clean or warm recovery cluster can import the required restore points.
- ODF is healthy before persistent application recovery begins.
- The team has completed a service level test, not just a restore task.
- The measured recovery time and data loss are recorded and compared with approved business requirements.
That’s the measure that matters. An estimated RPO or RTO is a planning value. A tested result is evidence.
Please see Enterprise Tape Library Administration: Control Path, Firmware, Media Management and Tape Operations, how to Repair a Corrupt SQL Server Database Without Data Loss, and Azure Application Gateway: Practical Configuration Guide
Final Thoughts
Backing up OpenShift applications is necessary. The rest of the recovery plan covers several independent layers. The platform and etcd need their own procedures. ODF must supply healthy target storage. Application resources, persistent volumes, and object data return through their supported methods. The Kasten catalog and external services complete the recovery path.
At Recovery Point, we look at the full service. We identify what each product can recover, document the dependencies, and test the result. Pretending that one tool covers everything doesn’t help the recovery team. The goal is a process the team can execute when the original environment is unavailable.
Product support and recovery procedures change. Check the current Red Hat and Veeam documentation for the exact versions in use before implementing these steps.
I hope you found this guide on “Backing Up OpenShift Is Not the Same as Recovering It” very useful. Please feel free to leave a comment below.
