Overview

• This page provides a practical deployment template for NVMe/TCP Boot from SAN on Dell PowerEdge servers
• The overall workflow is:
  • gather required parameters
  • prepare the boot namespace on the storage side
  • configure BIOS/iDRAC boot settings
  • install the OS to the remote NVMe device
  • apply post-install tuning

Goal

• Boot the operating system directly from a remote NVMe/TCP namespace over Ethernet.
• Avoid dependency on local boot disks.
• Keep boot images centralized and easier to replace or recover.
  
  Important note: Compatibility of server models for supporting NVMe Boot from SAN should be verified directly with Dell.

2) Recommended High-Level Flow

• Preparation
  • Collect the host NQN
  • Collect the host IP addresses for each boot NIC / SAN path
  • Collect the target subsystem NQN
  • Collect the discovery IPs or direct target IPs
  • Collect the required ports
  • Collect VLAN details, subnet mask, and gateway information if relevant
• Storage-side setup
  • Create the boot namespace / volume on the storage platform
  • Map the namespace to the host NQN
  • Confirm the target is exposed on the expected IPs and ports
• Server-side setup
  • Enable NVMe-oF boot in BIOS/iDRAC
  • Define one subsystem per SAN path
  • Apply the correct target addressing method
• OS install
  • Boot the installer ISO
  • Verify the remote NVMe device is visible
  • Install the OS onto the remote NVMe disk

3 )Information to Collect Before Starting

3.1) Host-side information

• NVMe-oF Host NQN
• NVMe-oF Host ID

3.2) Target-side information

• NVMe-oF subsystem NQN
• Discovery controller IPs, if using discovery-based configuration
• Direct target / I/O controller IPs, if using static target entries
• NVMe/TCP service port
• Namespace mapping details

4) Steps to Customize for VAST

Please note that a few reboots will be required in order to expose and configure all the BIOS options listed below.

4.1) Configure NVME Settings

• Log in to iDRAC.
• Go to:
  • Configuration > BIOS Settings > NVMe Settings
• Set BIOS NVMe Driver to All Drives
• Expect the values to remain pending until the system reboots.

4.2) Enable NVME-oF

• Log in to iDRAC.
• Go to:
  • Configuration > BIOS Settings > Network Settings > NVMe-oF
• Set to Enabled
• Expect the values to remain pending until the system reboots.

4.3) Configure Network Settings

• Log in to iDRAC.
• Go to:
  • Configuration > BIOS Settings > Network Settings >NVMe-oF SubSystem Settings
• Set NVMe-oF SubSystem1 > Enabled
• Expect the values to remain pending until the system reboots.

4.4) Collect host information from iDRAC

• Log in to iDRAC.
• Go to:
  • Configuration > BIOS Settings > Network Settings
• Record the NVMe-oF Host NQN.
• Record any NIC-specific information that will be used for the boot paths. 

4.5) Define the NVMe-oF boot subsystems

• Configure one boot subsystem entry per SAN path.
• For each subsystem entry, define:
  • interface / NIC port
  • protocol
  • VLAN settings if needed
  • host IP address and subnet mask
  • target subsystem NQN if using direct target mode
  • target address
  • target port

Two common configuration models

• Discovery controller method
  • leave the subsystem NQN empty
  • point BIOS to the discovery controller IP
  • use the discovery service port
• Direct target method
  • enter the subsystem NQN explicitly
  • point BIOS directly to the target IP
  • use the target service port

4.6) Prepare the boot namespace on the storage side

• Create the namespace / volume that will hold the operating system.
• Map it to the host using the host NQN collected from iDRAC.
• Confirm that the namespace is presented through the expected NVMe/TCP target IPs and ports.

4.7) Collect target-side information from VAST

• Gather the target-side values needed by BIOS/iDRAC:
  • subsystem NQN

• discovery IPs or direct target IPs

• NVMe/TCP port – 4420
  • namespace-to-host mapping

4.8) Reboot and boot the OS installer

• Apply the BIOS changes.
• Reboot the host
• Boot from OS installer ISO.
• Confirm that the remote NVMe/TCP disk is visible as an install target.
• Install the OS onto the remote NVMe device.

If the boot disk is not visible

Use the installer shell and verify:

• NVMe subsystem visibility
• NBFT handoff – the process where the system firmware, usually BIOS/UEFI, passes NVMe Boot Firmware Table information to the operating system during boot.
• IP addressing
• target reachability

nvme list-subsys

nvme nbft show

ip addr

ping <target-ip>

These checks help confirm that BIOS handed the boot configuration to the OS correctly and that the target is reachable. 

Choosing Device During Install

During Install

During reboot:

5) Post-Install Validation

Confirm the system is really booting from the remote NVMe device

• After the OS installation completes, verify:
  • the target NVMe namespace is present
  • / and boot-related filesystems reside on the expected NVMe device

[root@localhost ~]# nvme list
Node                  Generic               SN                   Model                                    Namespace  Usage                      Format           FW Rev  
--------------------- --------------------- -------------------- ---------------------------------------- ---------- -------------------------- ---------------- --------
/dev/nvme0n1          /dev/ng0n1            VST0000000e          VASTData                                 0x1          6.54  GB /   2.00  TB    512   B +  0 B   5.5.0   

[root@localhost ~]# nvme list-subsys
nvme-subsys0 - NQN=nqn.2024-08.com.vastdata:4ebc5121-259c-53b4-99b4-4e4a1a3d9f03:default:blockbfs
               hostnqn=nqn.1988-11.com.dell:PowerEdge.R760.8DH3QH4
\
 +- nvme0 tcp traddr=172.21.73.1,trsvcid=4420,host_traddr=172.21.222.214,src_addr=172.21.222.214 live



 [root@localhost ~]# df -h
Filesystem           Size  Used Avail Use% Mounted on
devtmpfs             4.0M     0  4.0M   0% /dev
tmpfs                126G     0  126G   0% /dev/shm
tmpfs                 51G  9.9M   51G   1% /run
efivarfs             304K  192K  108K  64% /sys/firmware/efi/efivars
/dev/mapper/rl-root   70G  5.9G   65G   9% /
/dev/mapper/rl-home  1.8T   13G  1.8T   1% /home
/dev/nvme0n1p2       960M  336M  625M  35% /boot
/dev/nvme0n1p1       599M  7.1M  592M   2% /boot/efi
tmpfs                 26G  4.0K   26G   1% /run/user/0

[root@localhost ~]# nvme nbft show
/sys/firmware/acpi/tables/NBFT:

NBFT Subsystems:

Idx|NQN |Trsp|Address |SvcId|HFIs
—+——————————————————————————+—-+———–+—–+—-
1 |nqn.2024-08.com.vastdata:4ebc5121-259c-53b4-99b4-4e4a1a3d9f03:default:blockbfs|tcp |172.21.73.1|4420 |1

NBFT HFIs:

Idx|Trsp|PCI Addr |MAC Addr |DHCP|IP Addr |Mask|Gateway |DNS
—+—-+———-+—————–+—-+————–+—-+——–+——–
1 |tcp |0:b5:0.0 |b4:5c:b5:c1:c3:94|yes |172.21.222.214|16 |0.0.0.0 |0.0.0.0

NBFT entry detected on the host:

• single VAST NVMe/TCP subsystem is defined in the table
• Target endpoint: 172.21.73.1:4420
• The target is associated with host fabric interface 1
• Host fabric interface 1 maps to NIC 0:b5:0.0
• NIC MAC address: b4:5c:b5:c1:c3:94
• Host IP address: 172.21.222.214/16 obtained via DHCP
• Both host and target reside in the same /16 network
• No gateway is required, matching the 0.0.0.0 gateway value
  
  
  To complete the NVMe configuration setup, please refer to:
  https://kb.vastdata.com/docs/persisting-nvme-of-over-tcp-across-reboots