TrueNAS Enterprise ZFS Storage Pool Provisioning, iSCSI Target Configuration & NFS Datastore Export

1. Scope, Storage Architecture & Topology

This standard operating procedure specifies the design, configuration, and provisioning of high-availability TrueNAS Enterprise ZFS storage arrays supporting virtualization clusters (VMware vSphere & Proxmox VE).

ZFS VDEV Architecture Baseline

Workload	Recommended ZFS Topology	Volblocksize / Recordsize	SLOG (ZIL) Requirement	L2ARC Cache
VMware / Proxmox iSCSI (Block)	Striped Mirrors (RAID-10)	16K (VMFS6) or 64K (KVM)	Mandatory (Enterprise NVMe with PLP)	Optional (Read-heavy)
File Shares & Archives (NFS/SMB)	RAIDZ2 (Dual Parity)	128K or 1M (Large files)	Optional (if sync writes enforced)	Beneficial
Database Engines (SQL / Postgres)	Striped Mirrors	8K or 16K	Mandatory (Ultra-low latency PLP NVMe)	High RAM priority

 ┌───────────────────────────────────────────────────────────────┐
 │               Dual Redundant Storage Fabric                   │
 │   Fabric A: VLAN 90 (10.10.90.0/24) - Jumbo Frames MTU 9000   │
 │   Fabric B: VLAN 92 (10.10.92.0/24) - Jumbo Frames MTU 9000   │
 └───────────────┬───────────────────────────────┬───────────────┘
                 │                               │
 ┌───────────────┴───────────────────────────────┴───────────────┐
 │               TrueNAS Enterprise Storage Array                │
 │  ┌─────────────────────────────────────────────────────────┐  │
 │  │ ZFS Pool: "tank-fast-vms" (NVMe / SAS SSD Mirror VDEVs) │  │
 │  │  ├─ Dedicated SLOG: Dual Kioxia/Intel NVMe (PLP Mirror) │  │
 │  │  └─ Zvol: "esxi-lun01" (Block iSCSI MPIO via Fabric A/B)│  │
 │  └─────────────────────────────────────────────────────────┘  │
 └───────────────────────────────────────────────────────────────┘

---

2. Pre-Flight Verification & Hardware Sizing

1. ECC RAM: Verify storage controller has minimum 64 GB ECC RAM (ZFS rule of thumb: 1 GB RAM per 1 TB raw storage + ARC sizing).
2. HBA Controller: Ensure SAS controllers operate strictly in IT-Mode (Host Bus Adapter); hardware RAID controller passthrough is strictly prohibited.
3. Power-Loss Protection (PLP): Ensure all write-cache SSDs (SLOG / ZIL) possess hardware Supercapacitors / PLP to guarantee data consistency during catastrophic power cuts.
4. Dual Interfaces: Confirm ens1f0 is mapped to Storage Fabric A (10.10.90.20/24) and ens1f1 to Storage Fabric B (10.10.92.20/24), both configured with MTU 9000.

---

3. Execution: ZFS Storage Pool Provisioning

Step 1: Storage Pool Creation (CLI / Midclt)

Execute via TrueNAS Shell or UI to create a resilient, high-IOPS pool using striped mirrors:

# Verify raw disk serials and non-RAID layout
geom disk list # (CORE) or lsblk -d -o NAME,MODEL,SERIAL,SIZE # (SCALE)

# Create Pool 'tank-fast-vms' with 2-way mirror pairs, dedicated SLOG mirror, and hot spare
zpool create -f -o ashift=12 tank-fast-vms \
    mirror /dev/disk/by-id/nvme-SAMSUNG_MZQL2960HCJR-00A07_S6...1 /dev/disk/by-id/nvme-SAMSUNG_MZQL2960HCJR-00A07_S6...2 \
    mirror /dev/disk/by-id/nvme-SAMSUNG_MZQL2960HCJR-00A07_S6...3 /dev/disk/by-id/nvme-SAMSUNG_MZQL2960HCJR-00A07_S6...4 \
    log mirror /dev/disk/by-id/nvme-INTEL_MEMPEK1W016GA_PH...1 /dev/disk/by-id/nvme-INTEL_MEMPEK1W016GA_PH...2 \
    spare /dev/disk/by-id/nvme-SAMSUNG_MZQL2960HCJR-00A07_S6...5

Step 2: Optimal Pool Property Tuning

# Enable LZ4 compression (CPU-friendly, reduces write amplification)
zfs set compression=lz4 tank-fast-vms

# Disable access time updates to prevent write penalty
zfs set atime=off tank-fast-vms

# Set sync behavior (standard ensures POSIX sync requests are committed to SLOG)
zfs set sync=standard tank-fast-vms

---

4. Execution: iSCSI Target & MPIO Block Storage Provisioning

Step 1: Create Zvol for Block Target

Create a 2 TB Zvol for VMware ESXi VMFS datastore provisioning with 16 KB block size matching hypervisor I/O:

zfs create -V 2T -b 16k tank-fast-vms/esxi-prod-lun01

Step 2: Configure iSCSI Portals (Dual Fabric Listeners)

Navigate to Shares > Block Shares (iSCSI) in TrueNAS or execute via API:

1. Portal ID 1:
   • Listen on IP Fabric A: 10.10.90.20:3260
   • Listen on IP Fabric B: 10.10.92.20:3260
   • Discovery Auth: None (isolated network) or Mutual CHAP

Step 3: Configure Initiator Group & Target

1. Initiator Group: Define authorized ESXi Host IQNs:
   • iqn.1998-01.com.vmware:esxi-host01-abcdef01
   • iqn.1998-01.com.vmware:esxi-host02-abcdef02
2. Target:
   • Target Name: iqn.2026-10.internal.corp:storage.esxi-prod-target
   • Alias: ESXi-SAN-Target
3. Extent Configuration:
   • Extent Name: extent-esxi-prod-lun01
   • Extent Type: Device (Points to tank-fast-vms/esxi-prod-lun01)
   • Logical Block Size: 512 (or 4096 for 4Kn native)
   • Enable T10-DIF / VAAI primitives: Checked (Enables ATS, XCOPY, WRITE SAME)
4. Target / Extent Association:
   • Map Target to Extent with LUN ID 0.

---

5. Execution: NFS v4.1 Datastore Export (Proxmox / VMware)

For shared NFS datastores hosting ISOs, VM templates, or Proxmox shared disks:

Step 1: Create NFS Dataset

zfs create -o recordsize=128k -o compression=zstd tank-fast-vms/nfs-vm-shared

Step 2: Configure NFS Service & Export Rules

1. In TrueNAS Sharing > Unix (NFS) Shares:
   • Path: /mnt/tank-fast-vms/nfs-vm-shared
   • Authorized Networks: 10.10.90.0/24, 10.10.92.0/24
   • Maproot User: root
   • Maproot Group: wheel
   • Security: sys (or krb5p if Active Directory Kerberized)

# Verify export status from Linux shell
showmount -e 10.10.90.20

---

6. Enterprise Data Protection: Automated Snapshots & Remote Replication

Step 1: Automated Local Snapshot Tasks

Prevent ransomware and human error by enforcing granular snapshot schedules:

• Frequency: Every 15 minutes during business hours (retained for 48 hours).
• Daily: 00:00 daily (retained for 30 days).
• Monthly: 1st of month (retained for 12 months).

# Create automated snapshot task via CLI
midclt call pool.snapshottask.create '{
  "dataset": "tank-fast-vms",
  "recursive": true,
  "lifetime_value": 30,
  "lifetime_unit": "DAY",
  "schedule": {"minute": "0", "hour": "0", "dom": "*", "month": "*", "dow": "*"}
}'

Step 2: Offsite ZFS Stream Replication

Configure continuous incremental asynchronous replication to secondary DR TrueNAS array:

# Example manual stream push to DR offsite storage
zfs send -i tank-fast-vms@snap-yesterday tank-fast-vms@snap-today | ssh [email protected] zfs recv dr-pool/backup-vms

---

7. Hypervisor MPIO & Datastore Verification

Step 1: ESXi iSCSI Multipathing Configuration

On ESXi host via SSH:

# Discover target on both IP fabrics
esxcli iscsi adapter discovery sendtarget add --address 10.10.90.20:3260 --adapter vmhba64
esxcli iscsi adapter discovery sendtarget add --address 10.10.92.20:3260 --adapter vmhba64

# Rescan storage adapters
esxcli storage core adapter rescan --adapter vmhba64

# Set Multipath Policy to Round Robin (VMW_PSP_RR)
for device in $(esxcli storage nmp device list | grep -E '^naa\.' | awk '{print $1}'); do
    esxcli storage nmp device set --device $device --psp VMW_PSP_RR
    esxcli storage nmp psp roundrobin deviceconfig set --device $device --type "iops" --iops 1
done

---

8. Rollback & Emergency LUN Restoration

Scenario A: Rapid Snapshot Rollback (Storage Corruption or Ransomware)

If a virtual disk extent is corrupted:

# 1. Unmount datastore on Hypervisors first!
# 2. Rollback dataset to last known good snapshot:
zfs rollback -r tank-fast-vms/esxi-prod-lun01@snap-2026-10-15-1000

# 3. Rescan datastore on Hypervisors

Scenario B: Degraded Disk Replacement

If a mirror drive reports SMART read errors or FAULTED state:

# Identify faulted disk
zpool status tank-fast-vms

# Replace failed physical drive with new drive
zpool replace tank-fast-vms /dev/disk/by-id/old-faulted-drive /dev/disk/by-id/new-healthy-drive

# Monitor resilvering progress
zpool status tank-fast-vms -v

---

9. Operational Audit Trail & Document Approvals

Checkpoint	Requirement	Assigned Role	Sign-off Status
Mirror VDEV Topology	Dual mirror VDEVs + PLP SLOG verified	Systems Administration Team	Approved
MPIO Dual Fabric	Active paths visible across VLAN 90 and 92	Systems Administration Team	Approved
VAAI Hardware Offload	ATS / XCOPY / Zeroing primitives active	Lead Systems Administrator	Approved
ZFS Snapshot Schedule	15-min and daily retention tasks verified	Lead Systems Administrator	Approved