AD Recovery: Fix device ran into an issue with error 0xc00002e2

In this article, we shall discuss “AD Recovery: Fix device ran into an issue with error 0xc00002e2”. Recently, my lab environments have been repeatedly restarting due to a faulty switch. After extensive research to solve this issue before performing VM recovery, I discovered that the problem was caused by the Hyper-V host experiencing unplanned restarts. If volumes on virtual hard disks connected to a virtual IDE controller are being used by virtual machines, the virtual hard disks on that server may become inconsistent. Please see How to Reset Services Restore Mode (DSRM) Password, and Active Directory: How to Setup a Domain Controller.

Note: This is a similar error as discussed by Microsoft, but this is no way related to the issue I am facing. You way want to take a look at this “Active Directory Domain Services role was removed from a domain controller without first demoting it before diving deep into this piece to see if this related to the error you are having.

According to Microsoft, when you virtualize your domain controller (DC) on a Hyper-V host server. When the Hyper-V host server crashes or encounters a power outage, the Active Directory database may become corrupted, or the virtual machine fails to start, and you may receive an error message as shown below

Please also see how to fix Windows Update Error Code 0xC1900101 – 0x30018 on Windows 10/11, how to replace a blue screen error with a blank screen for system errors, and how to create blue screen using the Not my Fault tool from Sysinternals.

Troubleshoot via DSRM

Since I know what the end result is due to my resolution, I will walk you through some of the steps I employed to troubleshoot.

Active Directory replication status can be checked using command-line and GUI tools. The REPADMIN command-line tool is available with Windows Server, is the primary tool since 2003. Microsoft’s ADREPLSTATUS GUI tool helps identify replication errors but this has been discontinued. You will find related tools online. Starting with Windows Server 2012, Microsoft offers Windows PowerShell to check replication status.

Note: On a different PC and not Domain controller (Active directory). To use AD replication PowerShell cmdlets, you must import the Active Directory PowerShell modules using the “Import-Module ActiveDirectory” command. Also,  you must install Remote Server Administration Tools (RSAT) for AD DS on non-domain controllers to use these PowerShell cmdlets.

What is Directory Services Repair Mode?

DSRM (Directory Services Repair Mode or Directory Services Restore Mode in versions prior to Windows Server 2012) is a special boot mode for Windows Server domain controllers. It functions similarly to Safe Mode with Networking but does not run Active Directory. Administrators use DSRM to restore Active Directory from a backup. It also helps resolve various issues with AD.

To enter DSRM, press the F8 key immediately after the BIOS POST screen and before the Windows logo appears. In later versions of Windows Servers, use the Advanced Boot Options menu or the Windows Recovery Environment to access DSRM as shown below.

Under Choose an Option, select Troubleshoot

Select “Advanced Options” as shown below

On the Startup Settings, please select Restart.

Select the Directory Services Repair Mode (DSRM), and then log in with the DSRM account.

Note: When you access the VM via DSRM which is a special boot mode designed for Active Directory environments, do not attempt to run Windows Updates as this will not work and will be rolled back. The reason for this is because, Some essential Windows Updates services do not apply in DSRM or Safe Mode. The DSRM environment only allows administrators to repair (restore) the Active Directory database.

Launch PowerShell

If you are unable to get the command prompt via these steps or the command prompt for advanced troubleshooting does not work well. Then you should launch PowerShell as shown below if you are able to login.

Verify Active Directory Replication

There might be issues with Active Directory replication, causing inconsistencies or making the domain controller unavailable. We will use the command below to determine the replication status and summary.

repadmin /replsummary
repadmin /showrepl

As you can see in the images above, we have gotten two errors from the AD Replication Commands “Win32 Error 1355(0x54b) – the specified domain does not exist. Learn more about the System Error Codes (1300-1699).

Let us also run the netdom verify command as shown below. As you can see, the domain does not exist or could not be contacted as well.

Perform Active Directory Database Integrity

Having taking a look at the Event Viewer, you would see that the “DNS server encountered an invalid domain name as well. Launch PowerShell or Command Pompt and type the command below to perform the initial integrity check.

ESENTUTL /g C:\windows\NTDS\ntds.dit /!10240 /8 /o

As you can see, there are inconsistency in the database with error message -1811

This is because of the unplanned shutdown. That is “the Administrator modified logs or lost I/O flush on shutdown”.

According to Microsoft, the Active Directory databases and log files are deployed on suitable hardware, therefore, this does not apply to me. But the below were applicable to me. You can use these methods to troubleshoot Jet database errors:

1. If 0xc00002e1 (c00002e1) and 0xc00002e2 (c00002e2) are virtual guest domain controllers that are running on Windows Server Hyper-V hosts
2. Check whether the event that preceded the LSASS 0xc00002e1 (c00002e1) and 0xc00002e2 (c00002e2) boot errors indicates one of the following issues:
   • Unscheduled power outage.
   • System hang.
   • Installation of Windows updates or service pack installs.
   • Addition or removal of disks, volumes, or partitions to the local system.
   • Hard drive failure.
   • NTDS.DIT or one or more log files were copied from another computer or even from a previous point in this DCs life.

Repair NTDS database

Now let us attempt repair the ntds database. But let us perform the integrity check again once more using the NTDS commands.

ntdsutil.exe
activate instance ntds
files
integrity

As you can see, this failed with a new error -501 JET_errLogFileCorrupt mesaage. I will no longer bother to repair the AD database.

As seen above and in this image, this error is because of the Hardware corrupting the I/O at writing, or the hardware lost flush caused the log to become unusable. This means that the database (DB) is left in a corrupt state.

Note: When an application writes data to a disk, the disk indicates the written operation success. However, when the application tries to read the data that it just wrote, the data does not exist. This issue is called as lost I/O, or lost flush.

Resolution: Perform VM Restore

Microsoft recommends in this case to restore the database from a known good backup, or reinstall the domain controller (DC). In my case, I have a backup of the entire VM machine, therefore, I will restore it.

Note: Note: When performing DC restore, please keep in mind that restoring an older backup could result in some problems, like changing passwords (user/computer). Also, devices in the network may lose the trust relationship with that domain.

I have previously demonstrated various ways to perform VM backup and restore. Please see How to integrate ObjectFirst OOTBI Appliance with VBR, Setup iSCSI Target and Storage LUN on Synology DS923+ for VBR, and Setup DS923+ Synology NAS as a Backup Repository for VBR.

This time, I will perform “Instant Restory” which instantly recover workloads (VMs, EC2 instances, physical servers and so on) directly from compressed and deduplicated backup files as Hyper-V VM. When you perform Instant Recovery, Veeam Backup & Replication mounts recovered VM images to a host directly from backups stored on backup repositories.

Why Use Instant Recovery?

Veeam recommends to use Instant Recovery for tier 1 VMs with little tolerance for business interruption and downtime. Besides disaster recovery matters, Instant Recovery can also be used for testing purposes as it is in this case.

Instant Recovery improves recovery time objectives (RTO) and minimises disruption and downtime of production workloads. However, Instant Recovery offers “temporary spares” for VMs with limited I/O performance.

To give the recovered VMs full I/O performance, you must finalize Instant Recovery by migrating the recovered VMs to the production environment.

Note: If you do not wish to migrate the recovered VM, you can stop publishing it, which removes the recovered VM.

Migration progress

As you can see our VM is available again and useable. This is why backup is very important and ensure to comply with the 3-2-1 backup rule.

As you can see below, there are no longer replication errors. By the way, this is the only DC in this domain at the moment.

I hope you found this article very useful on performing “AD Recovery: Fix device ran into an issue with error 0xc00002e2”. Please feel free to leave a comment below.