vSphere 6.x: VMXNET3 and DirectPath I/O issue

vSphere 6.0 and 6.5 have an interesting bug in the vSphere Web Client: a DirectPath I/O option is enabled by default for a new virtual machine with VMXNET3 network adapter provisioned.

A New Virtual Machine dialogue looks good.

vm-provisioning

However, when I open the VM settings, surprise surprise, it is quite different.

vm-settings

I have done my research and found the DirectPath I/O option was enabled regardless the VM hardware version (checked all versions starting from 8 to 13). VMware explains this behaviour in a KB # 2145889 with the recommendation to disable this parameter.

My next thought was to find all VMs that had the same problem. Thanks to VMware and its vast community, I was able to find some useful examples (here, here, and here) for a PowerCLI script below.

# File name: Get-DirectPathIOStatus.ps1
# Description: VMXNET3 - Network DirectPath I/O Status
# VMware KB: https://kb.vmware.com/kb/2145889
#
# 06/07/2018 - Version 1.1
# - Added the PowerCLI version check
# - Improved the host connection block
# - Added an export to CSV
# 26/04/2017 - Version 1.0
# - Initial Release
#
# Author: Roman Dronov (c)
# Initial variables
$outputCollection = @()
$newModule = "VMware.PowerCLI"
$oldModule = "VMware.VimAutomation.Core"
# Check the PowerCLI module availability
Clear-Host
Write-Host "`n Checking if the VMware PowerCLI modules are loaded...`r"
if ($(Get-InstalledModule | ? {$_.Name -like $newModule}) -eq $null) {
if ($(Get-Module | ? {$_.Name -like $oldModule}) -eq $null -and $(Get-Module -Name $oldModule -ListAvailable -ErrorAction Ignore) -eq $null) {
Write-Host "`n You need to install the VMware PowerCLI module to run this script. Exiting...`r" -ForegroundColor Yellow
exit
}
else {
Import-Module -Name $oldModule -ErrorAction Ignore -WarningAction Ignore
$moduleVersion = $(Get-Module -Name $oldModule).Version
Write-Host "`n The VMware PowerCLI module version $moduleVersion is loaded successfuly.`r" -ForegroundColor Green
}
}
else {
$moduleVersion = $(Get-InstalledModule -Name $newModule).Version
Write-Host "`n The VMware PowerCLI module version $moduleVersion is detected.`r" -ForegroundColor Green
}
# Set the VIServer connection mode to Multiple
$connectionMode = $(Get-PowerCLIConfiguration -Scope Session).DefaultVIServerMode
if ($connectionMode -like 'Single') {
Set-PowerCLIConfiguration -Scope Session -DefaultVIServerMode Multiple -Confirm:$false | Out-Null
}
# Connect to the host(s)
function Enter-Credentials {
$defaultUser = $env:USERDOMAIN + "\" + $env:USERNAME
if ($(Read-Host "`n Username [default: $($defaultUser)]") -eq '') {
$viUser = $defaultUser
}
else {
$viUser
}
$viPassword = Read-Host " Password" -AsSecureString
$script:viCredential = New-Object System.Management.Automation.PSCredential ($viUser,$viPassword)
}
function Verify-ViServer {
$viServer = Read-Host -Prompt "`n Input the host name and then press Enter"
Write-Host "`n Checking connection to ""$viServer""..." -ForegroundColor Green
while ($(Test-Connection -ComputerName $viServer -Count 2 -ErrorAction Ignore) -eq $null) {
Write-Host "`n Host ""$viServer"" is not reachable." -ForegroundColor Yellow
$viServer = Read-Host -Prompt "`n Input the correct host name and then press Enter"
}
$script:viServer = $viServer
}
if ($global:DefaultVIServers.Count -ne '0'){
Write-Host "`n Connection has already established with the following host(s):"
$vcsaConnections = $global:DefaultVIServers
foreach ($vcsaServer in $vcsaConnections) {
Write-Host "`t* $vcsaServer`r" -ForegroundColor Yellow
}
$userAnswer = Read-Host "`n Would you like to continue using current connection(s)? [Yes/No]"
while ("Yes","No" -notcontains $userAnswer) {
$userAnswer = Read-Host "`n Please choose correct answer [Yes/No]"
}
if ($userAnswer -eq 'No') {
Verify-ViServer
Enter-Credentials
Write-Host "`n Connecting to ""$viServer""..." -ForegroundColor Green
Connect-VIServer -Server $viServer -Credential $viCredential -ErrorAction Ignore -WarningAction Ignore | Out-Null
}
else {
continue
}
}
else {
Verify-ViServer
Enter-Credentials
Write-Host "`n Connecting to ""$viServer""..." -ForegroundColor Green
Connect-VIServer -Server $viServer -Credential $viCredential -ErrorAction Ignore -WarningAction Ignore | Out-Null
}
# Read all VMs that have VMXNET3
$vmNetworks = Get-VM | Get-NetworkAdapter | Where-Object {$_.Type -eq 'Vmxnet3'}
# Get information about the network card
for ($i=0; $i -lt $vmNetworks.Count; $i++) {
$vm = $vmNetworks[$i].Parent
$vmNetworkName = $vmNetworks[$i].NetworkName
$vmNetworkAdapter = $vmNetworks[$i].ExtensionData.DeviceInfo.Label
$vmNetworkAdapterDPIOStatus = $vmNetworks[$i].ExtensionData.UptCompatibilityEnabled
if ($vmNetworkAdapterDPIOStatus -ne 'True') {
$dpioStatus = "Disabled"
}
else {
$dpioStatus = "Enabled"
}
$properties = @{'Virtual Machine' = $vm.Name;
'HW Version' = $vm.Version;
'Network Adapter' = $vmNetworkAdapter;
'Network Name' = $vmNetworkName;
'DPIO Status' = $dpioStatus}
$object = New-Object –TypeName PSObject –Property $properties
$outputCollection += $object
}
# Disconnect from the host(s) and clear the screen
Disconnect-VIServer * -Confirm:$false
Clear-Host
# Print out the results
$dpioResults = $outputCollection | Sort-Object -Property "Virtual Machine" | select Virtual*,*Version,*Adapter,*Name,*Status
Write-Host "`n DirectPath I/O status`r"
$dpioResults
# Save results to CSV
$csvPath = $env:TEMP + "\DPIOStatus-" + $(Get-Date -Format 'ddMMyyyy') + ".csv"
$dpioResults | Export-CSV -Path $csvPath -NoTypeInformation -Force -Confirm:$false
Invoke-Item $csvPath

Feel free to use this code.

28/11/2016 – Update 1: vSphere Client (C#) and vSphere HTML5 Web Client (Fling) also create VMs with the DirectPath I/O option enabled.

28/11/2016 – Update 2: The workaround is to use PowerCLI to provision a new VM.

26/04/2017 – Update 3: The solution to this problem has been provided by LucD. I am working on an updated script which automate the process of disabling DirectPath I/O for the VMs.

10/07/2018 – Update 4: The script was updated to provide better reporting, including export to the CSV format.

27 thoughts on “vSphere 6.x: VMXNET3 and DirectPath I/O issue

  1. Hi, any idea how I could possibly check this out on HW v10? For some reason creating a new VM from powercli is enabling DirectPath I/O on the network card and I can’t seem to get it disabled (The template is disabled) and can’t find .ExtensionData.UptCompatibilityEnabled. Any help is appreciated!

    Liked by 1 person

    • Hi simnether,

      Thank you for your question.

      I have updated my script to show information about the DirectPath I/O status regardless the VM hardware version.

      Like

  2. Where is the recommendation to disable the parameter? I see that they describe how to change the behavior when creating a VMXNET3 NIC, but not a specific recommendation as to how/why the setting should be set. I’m asking because I just discovered the setting was enabled on my templates and so there are several VMs deployed with this setting enabled now, but the “DirectPath I/O” status is “Inactive” on all the VMs, probably because there’s no backing PCI device configured for DirectPath. So I’m just wondering if anyone has any information about the impact of having the checkbox for DirectPath enabled but DirectPath itself being inactive, whether this causes any issues or not.

    Liked by 1 person

    • Sorry for not answering earlier. It was quite busy month for me. I am still working on the script and let you know when it’s ready.

      Like

  3. hi i tried the script above in VW 6.0 pwrcli 5.5 and happened to see the output is “disabled”. just to cross check i picked a server and saw the directio setting is “enabled”. is there something im missing or do i need to run this script with pwrcli6.0?

    Like

    • Hi Yash,

      I haven’t tested the script with PowerCLI 5.5.

      Just to confirm, what was the hardware version of those VMs and which version of vSphere they were running on?

      Like

  4. just asked simsync to share the version of pcli in the other post (update 3 in your post) as i understand he had same problem and got it fixed.

    Like

  5. Does anyone have a script for changing this setting across multiple VM’s? LucD’s script is good for changing a single VM but it would be good to be able to change this across a selection of VM’s.

    Like

  6. Apologies for the daft question but my PS skills aren’t fantastic.

    What’s the best way to export this report to a CSV/Excel file?

    I’ve used ‘export-csv’ but I get a long string list when trying to do this.

    Final line:
    $OutputCollection | Sort-Object -Property “Virtual Machine” | Format-Table Virtual*,*Version,*Adapter,*Name,*Status -AutoSize | export-csv “C:\export\DPIO_Status.csv” -notypeinformation

    I also put the export into the For loop to append to a text file but it still produces the same result.

    Like

    • Hi Nick, Thank you for providing this feedback. I am working on improving this script and will publish an updated version over the weekend.

      Like

  7. Hello Roman,

    Disabling DirectPath I/O went smoothly for most VM’s. (no eventlog messages)
    However, our Windows clusters were triggered to perform failovers.
    Apparently disabling DirectPath I/O for a short moment disrupts network traffic, eventhough DirectPath I/O is inactive. I did not expect that.

    “Cluster node ‘######’ was removed from the active failover cluster membership. The Cluster service on this node may have stopped. This could also be due to the node having lost communication with other active nodes in the failover cluster. Run the Validate a Configuration wizard to check your network configuration. If the condition persists, check for hardware or software errors related to the network adapters on this node. Also check for failures in any other network components to which the node is connected such as hubs, switches, or bridges.”

    Like

Leave a reply to Yash Cancel reply