Get CPU usage and CPU ready values for all VMs in a cluster

Standard

When working with VMWare it is crucial to monitor the performance of your HA clusters and VMs. It is important to be able to get performance data so you can make sure your VMs have the resources they need and to know if they are sized correctly. To determine this I often use the CPU usage and CPU ready performance counters.

– What are the cpu usage/ready perf. counters and what do they show us?

The CPU usage value is sort of like the value you see in the task manager in windows, the main difference being that this value is not measured by the VM’s OS but by the ESXi host it is running on. As in the task manager this value indicates how much CPU is being used by your VM.
The CPU ready value is the amount of time your VM’s CPU has been “ready”, meaning how much time it has been waiting, doing nothing, for CPU cycles to be assigned to it by the host. If this value is high it is often an indicator of cluster health problems and/or bad VM sizing. I urge you to search for information about these counters and learn about them as they are highly useful when managing a VMWare environment. You can read more about CPU ready time here:
http://blogs.totalcaos.com/understanding-rdy-cpu-ready/

– Why should I use the script found in this blog post?

Gathering data like the performance counters described above can be done with Vcenter Operations Manager or with Veeam if you have this. However these applications are complicated to use and configure (and cost a lot of money). The script I have written below will give you a simple list/csv of the VMs in a given cluster with their CPU usage and CPU ready values in percent. You can then review your VMs and see check for any high/problematic values you find and improve the efficiency of your infrastructure. Any VMs with an average cpu usage over 70 % should probably be assigned more resources. If there are any VMs with ready times of 10% or more it probably means that your cluster/host is overcommitted or that the VM is oversized.

– The script itself and how I wrote it

To simplify the structure of the script I have divided it into three sections variables, functions and script main.

Variables

#VARIABLES
$scriptpath = $MyInvocation.MyCommand.Path
$dir = Split-Path $scriptpath
$CSV = "$dir\Average_CPU_Usage_Peak_Hours.csv"
#Parameters
$vcenter = "Some_Vcenter_server" #Name of your vcenter server
$ClusterName = "Some_cluster" #CPU usage and CPU ready values will be collected for all VMs in this cluster.
$DaysBack = 14 #Number of days back to collect performance counters.
$PeakTimeStart = 8 #hour of the day in 24 hour format 
$PeakTimeEnd = 20 #hour of the day in 24 hour format
$rdy_interval = 7200 #interval of rdy time values aggregation/averaging in seconds. This value should be changed according to your vcenter statistics settings.

Before running the script you will need to change the values in the #parameters section according to your needs/environment. $vcenter should be the name of your vcenter server and $ClusterName the name of the cluster you wish to collect VM performance data from. $DaysBack is the amount of time you wish to collect performance from. E.g if it is set to 14 days you will collect performance data going back 14 days to the time you are running the script. Make sure you set this value to a valid number of days (your vcenter DB must retain the info for the amount of time specified).

I have found that performance data is more useful when it is gathered in business hours so the results show values reflecting a busy system. To gather performance counters from the relevant times of day you must change the values for $peakTimeStart and $peakTimeEnd to suit your needs.

The last parameter is the $rdy_interval. When gathering the cpu ready time performance counters the values are returned as a number of milliseconds. To convert this number correctly to a percent value it is necessary to take into account the aggregation interval (the time between the data points stored in the vcenter database). On my vcenter installation the ready time values are aggregated/averaged every two hours and therefore I have set the $rdy_interval to 7200 seconds. You can read more about this here:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2002181

Now to the functions.

Functions

1. Connect-Vcenter

function Connect-Vcenter {
	param(
		$vcenter
	)
	#Load snap-in
	if (-not (Get-PSSnapin -Name VMware.VimAutomation.Core)) {
		Add-PSSnapin VMware.VimAutomation.Core
	}
	#Connect to vCenter
	if ($global:DefaultVIServers.Count -lt 1) {
		Connect-VIServer $vCenter
	}
}

Connect-Vcenter is a small function that simply adds the snap-in for VMware cmdlets and connects to a vcenter server making the powershell instance ready to fire commands.
2. Get-VMCPUAverage

function Get-VMCPUAverage {
    param (
        $VM
    )
	$Start = (get-date).AddDays(-$DaysBack)
	$Finish = get-date
    
	$stats = get-vm $VM | get-stat -MaxSamples "1000" -Start $Start -Finish $Finish -Stat "cpu.usage.average" | `
    ? { ($_.TimeStamp).Hour -gt $PeakTimeStart -and ($_.TimeStamp).Hour -lt $PeakTimeEnd }
	$aggr_stats = $stats | Measure-Object -Property Value -Average
	$avg = $aggr_stats.Average
	return $avg
}

This function uses the vmware cmdlet Get-Stat to collect the performance counter cpu.usage.average in the specified timeframe. The average value of all the collected data points are then returned.
3. Get-VMCpuRDY

function Get-VMCpuRDY {
	param (
        $VM
    )
	$Start = (get-date).AddDays(-$DaysBack)
	$Finish = get-date
	
	$stats = get-vm $VM | Get-Stat -MaxSamples "1000" -Start $Start -Finish $Finish -Stat Cpu.Ready.Summation | `
    ? { ($_.TimeStamp).Hour -gt $PeakTimeStart -and ($_.TimeStamp).Hour -lt $PeakTimeEnd -and $_.Instance -eq ""}	
	$aggr_stats = $stats | Measure-Object -Property Value -Average	
	$rdy = [Math]::Round(((($aggr_stats.Average)/1000)/$rdy_interval) * 100,1)
	return $rdy
}

Get-VMCpuRDY collects the average values of the performance counter Cpu.Ready.Summation in the specified timeframe. As explained earlier in the post this counter returns a summation of the milliseconds in which the CPU was waiting for resources. In order to convert this into a percentage value I first divide the amount of milliseconds by 1000 to convert to seconds. I divide this amount of seconds by the $rdy_interval, which is the amount of seconds between each data point, and then multiply by 100. I then use [Math]::Round to round off the value to one decimal which is the number returned by the function.

Script Main

#SCRIPT MAIN
clear
#Load the VMWare module and connect to vCenter
Connect-Vcenter -vcenter $vcenter
$AvgCPUValues = @() #Create array to hold the CPU usage and CPU ready values
Get-Cluster $ClusterName | Get-VM | ? {$_.PowerState -eq "PoweredOn"} | % {	#loop through all powered on VMs in the cluster
	$AvgCPUValue = "" | Select "VM","CpuAvg","CpuRdy" #create a custom object with these properties.
	$AvgCPUValue.VM = $_.Name
	$AvgCPUValue.CpuAvg = "{0:N2}" -f $(Get-VMCPUAverage -VM $_) #Get VM CPU usage and round to two decimals
	$AvgCPUValue.CpuRdy = Get-VMCpuRDY -VM $_
	$AvgCPUValues += $AvgCPUValue
}
$AvgCPUValues | Export-Csv $CSV -NoTypeInformation -Force

In the script main I first connect to the vcenter server using my function Connect-Vcenter. I then create an array $AvgCPUValues. The next step is to loop through all the VMs in the cluster using Get-Cluster $ClusterName | Get-VM. In the loop I create a custom object for each VM with the properties VM, CPUAvg and CPURdy. I use the functions Get-VMCPUAverage and Get-VMCpuRDY to get cpu usage and cpu ready values and then assign these to the corresponding properties on the custom object. The custom object is then added to the array $AvgCPUValues which I created in the beginning of the script main.
When the loop has completed I then export this array to CSV in the directory the script was run.

– That’s it! You’re ready to go!

I really hope you find this script useful. Remember to fill out the #parameters section before you run the script.
I have copied in the full script below.

#############################################################################################################
##script:			Get-VMCPUAverage.ps1
##
##Description:		Gets "CPU usage" and "cpu ready" for all VMs in a given cluster and exports the results
#+					to a CSV file in the script directory.
##Created by:		Noam Wajnman
##Creation Date:	March 11, 2014
##Updated on:		May 20, 2014
##############################################################################################################
#VARIABLES
$scriptpath = $MyInvocation.MyCommand.Path
$dir = Split-Path $scriptpath
$CSV = "$dir\Average_CPU_Usage_Peak_Hours.csv"
#Parameters
$vcenter = "Some_Vcenter_server" #Name of your vcenter server
$ClusterName = "Some_cluster" #CPU usage and CPU ready values will be collected for all VMs in this cluster.
$DaysBack = 14 #Number of days back to collect performance counters.
$PeakTimeStart = 8 #hour of the day in 24 hour format 
$PeakTimeEnd = 20 #hour of the day in 24 hour format
$rdy_interval = 7200 #interval of rdy time values aggregation/averaging in seconds. This value should be changed according to your vcenter statistics settings.
#FUNCTIONS
function Connect-Vcenter {
	param(
		$vcenter
	)
	#Load snap-in
	if (-not (Get-PSSnapin -Name VMware.VimAutomation.Core)) {
		Add-PSSnapin VMware.VimAutomation.Core
	}
	#Connect to vCenter
	if ($global:DefaultVIServers.Count -lt 1) {
		Connect-VIServer $vCenter
	}
}
function Get-VMCPUAverage {
    param (
        $VM
    )
	$Start = (get-date).AddDays(-$DaysBack)
	$Finish = get-date
    
	$stats = get-vm $VM | get-stat -MaxSamples "1000" -Start $Start -Finish $Finish -Stat "cpu.usage.average" | `
    ? { ($_.TimeStamp).Hour -gt $PeakTimeStart -and ($_.TimeStamp).Hour -lt $PeakTimeEnd }
	$aggr_stats = $stats | Measure-Object -Property Value -Average
	$avg = $aggr_stats.Average
	return $avg
}
function Get-VMCpuRDY {
	param (
        $VM
    )
	$Start = (get-date).AddDays(-$DaysBack)
	$Finish = get-date
	
	$stats = get-vm $VM | Get-Stat -MaxSamples "1000" -Start $Start -Finish $Finish -Stat Cpu.Ready.Summation | `
    ? { ($_.TimeStamp).Hour -gt $PeakTimeStart -and ($_.TimeStamp).Hour -lt $PeakTimeEnd -and $_.Instance -eq ""}	
	$aggr_stats = $stats | Measure-Object -Property Value -Average	
	$rdy = [Math]::Round(((($aggr_stats.Average)/1000)/$rdy_interval) * 100,1)
	return $rdy
}
#SCRIPT MAIN
clear
#Load the VMWare module and connect to vCenter
Connect-Vcenter -vcenter $vcenter
$AvgCPUValues = @() #Create array to hold the CPU usage and CPU ready values
Get-Cluster $ClusterName | Get-VM | ? {$_.PowerState -eq "PoweredOn"} | % {	#loop through all powered on VMs in the cluster
	$AvgCPUValue = "" | Select "VM","CpuAvg","CpuRdy" #create a custom object with these properties.
	$AvgCPUValue.VM = $_.Name
	$AvgCPUValue.CpuAvg = "{0:N2}" -f $(Get-VMCPUAverage -VM $_) #Get VM CPU usage and round to two decimals
	$AvgCPUValue.CpuRdy = Get-VMCpuRDY -VM $_
	$AvgCPUValues += $AvgCPUValue
}
$AvgCPUValues | Export-Csv $CSV -NoTypeInformation -Force
Advertisements

Powershell – Get DNS A records and export to CSV

Standard

When managing and cleaning up your IP addresses/ranges it can be very useful to get some lists of the records you have in your DNS zones. Here’s a simple script which gets the IP addresses/hostnames of the given DNS zones and exports the results to CSV.
The script is split into two parts Variables and script main. I will walk through and explain both parts below.

Variables

#VARIABLES
$scriptpath = $MyInvocation.MyCommand.Path
$dir = Split-Path $scriptpath
$CSV = "$dir\DNS_A_records.csv"
#parameters
$DNSServer = "Some_DNS_server"
$Zone1 = "myzone.local"
$Zone2 = "me.myzone.local"

I use the $dir object in the paths of my scripts as this object always points to the directory which the script has been run from. This is practical as it allows you to move/copy the script around without having to change any paths. $CSV is the path to the CSV file which will be created at the end of the script run.
Remember to fill out the parameters section before you run the script! $DNSServer is the name of your DNS server. $Zone1, $Zone2… etc are the names of the DNS zones which you want to get A records from. If you have more than 2 zones to get A records from then just add more objects to match the number of zones you need and add the zone names.

Script Main

#SCRIPT MAIN
clear
$DNS_Zones = @()
$DNS_Zones += $Zone1
$DNS_Zones += $Zone2
$hosts = @()
$DNS_Zones | % {
	$zone = $_
	Write-Host "Getting DNS A records from $zone"	
	$DNS_A_records = @(Get-WmiObject -Class MicrosoftDNS_AType -NameSpace Root\MicrosoftDNS -ComputerName $DNSServer -Filter "ContainerName = `'$zone`'")
	$DNS_A_records | % {
		$hostA = "" | select "hostname","IPAddress"
		$hostA.hostname = $_.OwnerName
		$hostA.IPAddress = $_.IPAddress
		$hosts += $hostA
	}
}
$hosts = $hosts | Sort-Object @{Expression={[Version]$_.IPAddress}}
$hosts | Export-Csv $CSV -NoTypeInformation -Force

In the script main I first create the array $DNS_Zones which will hold the different zone names. Then the zone names ($Zone1, $Zone2… etc.) are added to the array. If you created more than two zone objects in the variables section you must add them here too.
I now create the $hosts array which will hold the records we will wish export later. The next thing that happens is that we loop through the $DNS_Zones array. For each DNS Zone we get all A records using WMI and for each of these, a custom object with the properties hostname and IPAddress is created and added to the $hosts array.
I then sort the $hosts array by IP Address using the following code:
$hosts | Sort-Object @{Expression={[Version]$_.IPAddress}}
The @{Expression} argument allows you to add the [version] type declaration on the IPAddress property on the array elements/objects which in turn enables you to easily sort the IPs.
Finally the $hosts array is exported to CSV using the great Export-CSV powershell cmdlet.
I have copied in the full script below.I hope you find it useful. Enjoy!

################################################################################################
##Script:			Get-DNS_A_Records.ps1
##
##Description:		Gets all DNS A records from a given DNS server and exports the information 
#+					to a CSV file.
##Created by:		Noam Wajnman
##Creation Date:	April 07, 2014
################################################################################################
#VARIABLES
$scriptpath = $MyInvocation.MyCommand.Path
$dir = Split-Path $scriptpath
$CSV = "$dir\DNS_A_records.csv"
#parameters
$DNSServer = "Some_DNS_server"
$Zone1 = "myzone.local"
$Zone2 = "me.myzone.local"
#SCRIPT MAIN
clear
$DNS_Zones = @()
$DNS_Zones += $Zone1
$DNS_Zones += $Zone2
$hosts = @()
$DNS_Zones | % {
	$zone = $_
	Write-Host "Getting DNS A records from $zone"	
	$DNS_A_records = @(Get-WmiObject -Class MicrosoftDNS_AType -NameSpace Root\MicrosoftDNS -ComputerName $DNSServer -Filter "ContainerName = `'$zone`'")
	$DNS_A_records | % {
		$hostA = "" | select "hostname","IPAddress"
		$hostA.hostname = $_.OwnerName
		$hostA.IPAddress = $_.IPAddress
		$hosts += $hostA
	}
}
$hosts = $hosts | Sort-Object @{Expression={[Version]$_.IPAddress}}
$hosts | Export-Csv $CSV -NoTypeInformation -Force

Powershell – Get last boot time on remote servers and export results to CSV

Standard

Here’s a script to help you get the last boot time from remote windows servers. If you need to check the uptime of servers or troubleshoot unexpected restarts etc. then this script can be very useful.
The script uses WMI to get the information from the remote servers and then exports the results to a CSV file in the directory where the script was run. I have structured the script in three sections variables, functions and script main. I will go over these sections one by one now.

Variables

#VARIABLES

#$DebugPreference = "continue" #uncomment to get debug info
$scriptpath = $MyInvocation.MyCommand.Path
$dir = Split-Path $scriptpath #path to the directory in which the script is run.
$servers = "$dir\Servers.txt" #last boot time will be retrieved on all the servers given in this file.
$CSVPath = "$dir\LastBootTimes.csv" #results will be exported to csv to a file with this path

The object $dir always points to the directory in which the script is run and is used in the paths to files I work with in the script. This makes the script more robust because I can copy the script wherever I want without having to change any paths at all or edit the script.
$servers is the path to “servers.txt” file. You must create this file before you run the script and enter the names (one per line) of the servers you want to get the last boot time from.
$CSVPath is the path to CSV file which will be created when the script is run.

Functions

1. Get-LastBootTime

#FUNCTIONS
function Get-LastBootTime {
	[CmdletBinding()]
	[OutputType([System.String])]
	param(
		[Parameter(ValueFromPipeline=$true)][System.String[]]$server = $env:COMPUTERNAME
	)
	begin {
		function Test-PortAlive {
			#############################################################################################
			##Function:			Test-PortAlive
			##
			##Description:		Tests connection on a given server on a given port.
			##
			##Created by:		Noam Wajnman
			##Creation Date:	April 02, 2014	
			##############################################################################################
			[CmdletBinding()]
			[OutputType([System.boolean])]
			param(
				[Parameter(ValueFromPipeline=$true)][System.String[]]$server
			)
			$socket = new-object Net.Sockets.TcpClient
			$connect = $socket.BeginConnect($server, 135, $null, $null) #port set to 135 (RPC)
			$NoTimeOut = $connect.AsyncWaitHandle.WaitOne(500, $false) #timeout value set to 500 ms
			if ($NoTimeOut) {
				$socket.EndConnect($connect) | Out-Null
				return $true				
			}
			else {
				return $false
			}
		}
	}
	process {		
		$BootTime = $null
		$server = $($server).toUpper()	
		$alive = Test-PortAlive -server $server
		if ($alive) {
			Write-Debug "connection to $server is open on port $port"			
			$OSInfo = Get-WmiObject win32_operatingsystem -ComputerName $server #get the info with WMI
			$BootTime = $OSInfo.ConvertToDateTime($OSInfo.LastBootUpTime) #convert to datetime
			$BootTime = '{0:yyyy-MM-dd HH:mm:ss}' -f $BootTime #parse time to sortable format
			if ($BootTime) {
				Write-Debug "$server was rebooted last time at:`t$BootTime"					
				$result = "" | select "Server","LastBootTime" #creating a custom object
				$result.Server = $server
				$result.LastBootTime = $BootTime
				$result #return the $result object
			}
			else {
				Write-Debug "couldn't get boot time from server $server"
			}
		}			
		else {
			Write-Debug "Error - connection to $server is not open on port $port"			
		}
	}	
}

the function Get-LastBootTime takes one parameter $server which can also be passed to it via the pipeline.
Pipeline functions usually have three parts “begin”, “process” and “end” (this function only uses “begin” and “process”). In the begin section I have included another function “Test-PortAlive” but I won’t go into details about it as I have covered this in a previous post dedicated to that function. In this script I use it to test if the server is alive and the RPC port 135 is open as this is needed to run the WMI query.
The “process” part of the function is where last boot time information is retrieved using WMI. Basically the function attempts to get the info. If successful then a custom object with the properties “server” and “LastBootTime” is created and returned.

Script Main

#SCRIPT MAIN

clear
$BootTimes = @(gc $servers | Get-LastBootTime) #create and populate array with the last boot times of the given servers.
$BootTimes = $BootTimes | Sort-Object -Property "LastBootTime" #Sort array by last boot time
$BootTimes #Prints the $bootTimes array to the console 
$BootTimes | Export-Csv $CSVPath -NoTypeInformation -Force #Export the results to CSV

In the script main the first thing which happens is to create the array $BootTimes and fill it with the data from Get-LastBootTime function. This is done by running Get-Content on the servers.txt file and then piping the server names to the Get-LastBootTime function.
After this I sort the $BootTimes array by last boot time.
Finally the $BootTimes array is exported to CSV in the script directory.
I have copied in the full script below. I hope you will find it useful.
Enjoy!!

################################################################################################
##Script:			Get-LastBootTime.ps1
##
##Description:		Gets the time of the last boot on the servers given in "servers.txt" and
#+					exports the results to a csv file in the script dir.
##Created by:		Noam Wajnman
##Creation Date:	May 21, 2013
##Updated:			April 02, 2014
################################################################################################
#FUNCTIONS
function Get-LastBootTime {
	[CmdletBinding()]
	[OutputType([System.String])]
	param(
		[Parameter(ValueFromPipeline=$true)][System.String[]]$server = $env:COMPUTERNAME
	)
	begin {
		function Test-PortAlive {
			#############################################################################################
			##Function:			Test-PortAlive
			##
			##Description:		Tests connection on a given server on a given port.
			##
			##Created by:		Noam Wajnman
			##Creation Date:	April 02, 2014	
			##############################################################################################
			[CmdletBinding()]
			[OutputType([System.boolean])]
			param(
				[Parameter(ValueFromPipeline=$true)][System.String[]]$server
			)
			$socket = new-object Net.Sockets.TcpClient
			$connect = $socket.BeginConnect($server, 135, $null, $null) #port set to 135 (RPC)
			$NoTimeOut = $connect.AsyncWaitHandle.WaitOne(500, $false) #timeout value set to 500 ms
			if ($NoTimeOut) {
				$socket.EndConnect($connect) | Out-Null
				return $true				
			}
			else {
				return $false
			}
		}
	}
	process {		
		$BootTime = $null
		$server = $($server).toUpper()	
		$alive = Test-PortAlive -server $server
		if ($alive) {
			Write-Debug "connection to $server is open on port $port"			
			$OSInfo = Get-WmiObject win32_operatingsystem -ComputerName $server #get the info with WMI
			$BootTime = $OSInfo.ConvertToDateTime($OSInfo.LastBootUpTime) #convert to datetime
			$BootTime = '{0:yyyy-MM-dd HH:mm:ss}' -f $BootTime #parse time to sortable format
			if ($BootTime) {
				Write-Debug "$server was rebooted last time at:`t$BootTime"					
				$result = "" | select "Server","LastBootTime" #creating a custom object
				$result.Server = $server
				$result.LastBootTime = $BootTime
				$result #return the $result object
			}
			else {
				Write-Debug "couldn't get boot time from server $server"
			}
		}			
		else {
			Write-Debug "Error - connection to $server is not open on port $port"			
		}
	}	
}
#VARIABLES
#$DebugPreference = "continue" #uncomment to get debug info
$scriptpath = $MyInvocation.MyCommand.Path
$dir = Split-Path $scriptpath #path to the directory in which the script is run.
$servers = "$dir\Servers.txt" #last boot time will be retrieved on all the servers given in this file.
$CSVPath = "$dir\LastBootTimes.csv" #results will be exported to csv to a file with this path
#SCRIPT MAIN
clear
$BootTimes = @(gc $servers | Get-LastBootTime) #create and populate array with the last boot times of the given servers.
$BootTimes = $BootTimes | Sort-Object -Property "LastBootTime" #Sort array by last boot time
$BootTimes #Prints the $bootTimes array to the console 
$BootTimes | Export-Csv $CSVPath -NoTypeInformation -Force #Export the results to CSV