Break-Glass Enterprise Ransomware Containment & Forest Isolation Procedure

1. Scope, Activation Authority & Escalation Criteria

This emergency runbook outlines the immediate, drastic containment actions required when active, widespread ransomware execution, lateral credential abuse, or mass volume shadow copy deletion is confirmed within the enterprise.

Immediate Activation Criteria:

Execution of this procedure is authorized IMMEDIATELY without awaiting Change Advisory Board approval upon confirmation of ANY of the following triggers:

1. Two or more servers report encrypted files or ransom notes (e.g., .locked, README_DECRYPT.txt).
2. Detection of unauthorized mass domain-wide commands executing vssadmin delete shadows /all /quiet or wbadmin delete catalog.
3. Simultaneous unauthorized administrative logins detected across all Tier-0 Domain Controllers.

EVERY SECOND COUNTS

In modern ransomware campaigns, human-operated actors encrypt an entire enterprise within 45 to 90 minutes of initial deployment. Execute Phase 1 within 5 minutes of confirmation.

---

2. Containment Sequence Matrix

 ┌────────────────────────────────────────────────────────┐
 │   PHASE 1: NETWORK AIR-GAP (Minutes 0 - 5)             │
 │   - Drop Perimeter Firewall WAN Uplinks                │
 │   - Sever Core Switch Inter-VLAN Routing & Trunks      │
 └─────────────────────────┬──────────────────────────────┘
                           ▼
 ┌────────────────────────────────────────────────────────┐
 │   PHASE 2: STORAGE QUENCH & IMMUTABILITY (Mins 5 - 10) │
 │   - Set ZFS / SAN Datasets to READ-ONLY                │
 │   - Disconnect Hypervisor iSCSI / NFS Mounts           │
 └─────────────────────────┬──────────────────────────────┘
                           ▼
 ┌────────────────────────────────────────────────────────┐
 │   PHASE 3: IDENTITY & KERBEROS FREEZE (Mins 10 - 20)   │
 │   - Reset KRBTGT Password TWICE (Invalidate all TGTs)  │
 │   - Sever Entra ID Sync & Revoke Refresh Tokens        │
 └─────────────────────────┬──────────────────────────────┘
                           ▼
 ┌────────────────────────────────────────────────────────┐
 │   PHASE 4: CLEAN-ROOM RESTORATION (Mins 20+)           │
 │   - Spin up Isolated Clean-Room Recovery Network       │
 │   - Restore from Immutable Backups (SOP-SYS-062)       │
 └────────────────────────────────────────────────────────┘

---

3. Phase 1: Immediate Network Air-Gap (Minutes 0 – 5)

Sever external command-and-control communication and prevent lateral infection spread across VLANs.

Step 1: Drop All Firewall WAN Interfaces

Execute on Perimeter NGFW (FortiGate / pfSense / Cisco ASA):

# FortiGate CLI:
config system interface
    edit "wan1"
        set status down
    next
    edit "wan2"
        set status down
    next
end

# pfSense / OPNsense Console:
# Select Option 2 (Set interface IP address) or pull physical WAN patch cables.

Step 2: Sever Inter-VLAN Core Routing (Cisco Catalyst / Nexus)

Access the core switch via dedicated Out-Of-Band (OOB) console or physical serial:

core-switch# configure terminal
! Shut down inter-VLAN routing interfaces immediately
interface vlan 20
 shutdown
interface vlan 30
 shutdown
interface vlan 40
 shutdown
interface vlan 50
 shutdown
! DO NOT shut down Out-Of-Band Management VLAN 99
end

Result: Endpoints in VLAN 20/30/40 can no longer communicate with each other, Domain Controllers, or storage arrays.

---

4. Phase 2: Storage Quench & Immutability Lock (Minutes 5 – 10)

Prevent ransomware from altering or deleting storage pools, datastores, and snapshots.

Step 1: Force ZFS Datasets to Read-Only (TrueNAS CLI)

Access the TrueNAS storage controller console:

# Set root storage pool to read-only immediately
zfs set readonly=on tank-fast-vms
zfs set readonly=on tank-backup

# Verify write-lock status
zfs get readonly tank-fast-vms

Result: Any in-flight encryption attempting to overwrite virtual disks or file shares will immediately fail with Read-only file system.

Step 2: Sever Hypervisor Shared Datastores

On VMware ESXi / Proxmox VE nodes, unmount or detach network datastores to prevent malicious guest I/O:

# On Proxmox:
pvesm set ceph-vm-pool --disable 1
pvesm set tank-iscsi --disable 1

# On VMware ESXi:
esxcli storage filesystem unmount -u <Volume-UUID>

---

5. Phase 3: Identity & Kerberos Forest Freeze (Minutes 10 – 20)

Attacking actors rely on compromised Kerberos Golden Tickets or cached administrative credentials to control endpoints.

Step 1: Reset the krbtgt Account Password TWICE

Execute on the PDC Emulator Domain Controller via PowerShell:

# Reset 1: Invalidates currently issued TGTs
Set-ADAccountPassword -Identity krbtgt -NewPassword (ConvertTo-SecureString -AsPlainText "TempBreakGlassPwd!2026#Alpha" -Force) -Reset

# Wait 10 minutes for Active Directory replication or force replication:
repadmin /syncall /AdeP

# Reset 2: Invalidates the previous password history hash, destroying all Golden Tickets
Set-ADAccountPassword -Identity krbtgt -NewPassword (ConvertTo-SecureString -AsPlainText "TempBreakGlassPwd!2026#Beta" -Force) -Reset
repadmin /syncall /AdeP

Step 2: Revoke Active Directory Federation & Entra ID Sync

Sever cloud synchronization to protect Microsoft 365 and cloud workloads:

# Stop Entra Connect Sync Service
Stop-Service -Name "ADSync"
Set-Service -Name "ADSync" -StartupType Disabled

Step 3: Evict All Active SMB and RDP Sessions

# Evict all remote SMB connections
Get-SmbSession | Close-SmbSession -Force

# Terminate all active RDP user sessions
qwinsta | ForEach-Object {
    $line = $_.Trim() -split '\s+'
    if ($line[1] -match '^\d+$') {
        rwinsta $line[1]
    }
}

---

6. Phase 4: Clean-Room Environment & Recovery Roadmap

Once the perimeter and domain are frozen:

1. Provision Clean-Room Recovery VLAN:
   • Configure an isolated VLAN on the core switch (e.g., VLAN 888) with no routing outside the isolated lab.
2. Access Immutable Backups:
   • Mount the Linux Hardened Repository or Proxmox Backup Server (per SOP-SYS-062).
   • Validate that backup snapshots were created before the initial compromise timestamp.
3. Forensic Image Preservation:
   • Before restoring or wiping any compromised server, capture forensic memory and triage logs (per SOP-SEC-001).
4. Clean-Room OS Rebuild:
   • Re-deploy Domain Controllers from known clean media.
   • Restore database and business data files incrementally.

---

7. Controlled De-escalation & Rollback Plan

When containment is concluded, forensic analysis is complete, and the Lead Systems Administrator and CISO authorize restoration:

1. Bring Up Management Fabric First:
   • Enable VLAN 10 (Management) and verify clean hypervisors.
2. Restore TrueNAS Storage Write Capabilities:

zfs set readonly=off tank-fast-vms

3. Re-enable Inter-VLAN Routing One Subnet at a Time:

core-switch(config)# interface vlan 10
core-switch(config-if)# no shutdown

4. Monitor SIEM / EDR Dashboard:
   • Watch Wazuh dashboards for any abnormal beaconing or failed authentication spikes before reconnecting WAN links.

---

8. Incident Command Verification Sign-Off

Incident Stage	Required Authority	Time Verified	Signature
Emergency Containment Order	Lead Systems Administrator / CISO		APPROVED
Storage Read-Only Lock	Systems Administration Team		APPROVED
Double KRBTGT Password Reset	Tier-0 Domain Administrator		APPROVED
Clean-Room Recovery Authorize	Incident Commander		APPROVED