This discussion focuses on Microsoft® Exchange 2007 software performance and scalability on the VMware® vSphere™ 4.0 virtualization platform. Featured speakers, including Todd Muirhead, former member of the Dell TechCenter team and known to the Twitter community as VirtualTodd, discuss how the vSphere 4.0 infrastructure can support different levels of Exchange users and achieve low latency levels in virtualized Exchange configurations. In particular, Todd discusses the results of his latest study before the results get published. Todd is currently a staff performance engineer at VMware.

Technical Community - Background Reading

• Microsoft Exchange and VMware page
  
• Microsoft Exchanage page
• Recent TechTuesday chat: Dell and VMware vSphere 4.0 Performance

Chat Transcript

Dell-KongY	Hi there Virtualtodd!
VirtualTodd	Hi everybody
Dell-KongY	Hi there Jeffs aka Dell TechCenter's very own SANpenguin
Dell-KongY	Welcome Martin!
grealish	Hi Martin!
VirtualTodd	Josh!
Dell-JeffS	Hello
erson	Can we expect any Windows 7 drivers from Dell before 22 October?
Dell-KongY	So far I count two VMware staff engineers: the one and only Virtualtodd and Josh Schnee (jpschnee)
jpschnee	Thanks. I thought I would listen in and see how today's chat went
grealish	So where are we all logging in from?
erson	Sweden!
eczerwin	Chicago
martin	Ireland
VirtualTodd	Windows 7 and Windows Server 2008 R2 will be great guests on vSphere
grealish	Switzerland
eczerwin	Where in Chicago? I am moving there for work
Dell-KongY	Welcome Marcdell and Dustin_lema!
MarcDELL	Good afternoon
grealish	@virtualtodd, I'm testing Windows 7 and Windows 2008 R2 on vSphere already
Dustin_Lema	Thanks
Dell-KongY	Welcome Chris_has!
grealish	@eczerwin, Zurich area
eczerwin	Hmmm…me too. @grealish, you aren't Swiss are you?
Chris_	Thanks and thanks for sending this out today
erson	/me should get a quad-core play box to fool around with VMware
grealish	@eczerwin almost ;)
erson	Hi Scotth
DELL-ScottH	Hello!
grealish	Welcome Scott!
DELL-ScottH	Wait, let me yell a little louder...Hello!!
Dell-KongY	Welcome Scott H!
VirtualTodd	Hey Scott
Dell-KongY	Welcome Angel! Wow, I guess it's about time to begin
DELL-ScottH	Hey Todd, what is up my friend? How was the Dell diamond on the 4th?
Dell-KongY	Thanks everyone for joining the TechCenter and our distinguished guest, Todd Muirhead. Today, Todd will discuss the results of his soon-to-be published white paper on Microsoft Exchange 2007 on VMware vSphere
Dell-KongY	Welcome Depping!
depping	Hey guys
Dell-KongY	We have quite a VMware contingent including Todd, Josh, and Duncan Epping
Dell-KongY	Welcome Meza!
VirtualTodd	It actually will be a blog post on Vroom!
Meza	Hi Kongy
Meza	I have some issues with IT Assistant, which room should I go to? Oh and this is my first time checking out the chat
DELL-ScottH	Vroom! link for those that aren't looking for the vacuum cleaner: http://blogs.vmware.com/performance/
erson	Meza, there is only one chat room
VirtualTodd	Some quick background on Exchange and VMware
DELL-ScottH	Meza, welcome to the chat; what is your issue with IT Assistant?
DELL-ScottH	Chatting with Meza in private
Meza	Okay, thanks Erson
VirtualTodd	Kong and I presented a session at VMworld 2007 on Exchange 2007 on ESX 3.5
Meza	Hi Scott
DELL-ScottH	Hello
VirtualTodd	At the time it was kind of edgy to talk about running Exchange on ESX
Meza	ScottH, I currently have 2 problems: 1. I have two servers recently and they both have amber lights on. I looked at the LCD, and it indicated some sort of power problem. Looked into the Server Administrator, which reveals that both power supplies are said to have predictive failure
VirtualTodd	Our result showed that performance of Exchange on ESX was good; with latencies well below the standard set by Microsoft for good performance
Dell-KongY	www.delltechcenter.com/page/exchange+and+vmware and www.dell.com/downloads/global/solutions/virtexchange2k7_finalfrontier_vmworld2007.pdf
Meza	ScottH, the problem is, in our monitoring the server still shows a green check like nothing is wrong with it
Dell-KongY	Welcome Bm7150, Veeammark, and Ron!
Meza	ScottH, we only found the problem because someone was walking in the data center and saw amber lights
VeeamMark	Thanks @dell-kongy
Meza	ScottH, can we get IT Assistant to send out notifications on these?
VirtualTodd	That portion of the testing was just straight Loadgen testing that measured latency or response time across a range of actions done by Exchange users
DELL-ScottH	Meza, is the SNMP trap destination in OMSA pointing to the IT Assistant box? And, yes, but first you have to make sure that OMSA is set correctly to forward the traps
VirtualTodd	In addition, we tested and measured the performance of Exchange during vMotion and HA failover
Meza	ScottH, yes, if we pull the power plug to simulate a power failure, it will trigger a red X and send out a notification
VirtualTodd	Again, we found good performance—with only limited hits to performance during vMotion
Dell-KongY	Welcome Durkinr!
VirtualTodd	And reasonable recovery time for HA failovers
Dell-KongY	Welcome Delldustin!
DELL-ScottH	Meza, okay, so traps get sent to IT Assistant correctly. On OMSA have you filtered out lower-level events? Let me pull up one of my boxes—what version, which servers?
Meza	ScottH, okay, let me take a look to report to you the version we are running
VirtualTodd	The testing that I did recently focused on using Exchange in a DRS cluster (with vMotion being used to auto load balance)
Dell-KongY	Welcome Nakul1 and Dennis_s!
VirtualTodd	One of the big differences between the earlier tests and the recent test is the size of the workload. In the earlier tests we tested up to 4000 users, but in my DRS testing I used 16,000
Dell-KongY	Welcome Cheddr and Sdeuskar!
VirtualTodd	So it was with a fairly large number of users
Meza	ScottH, we are running Windows Server 2008 standard 32 bit with IT Assistant version 8.3.0
grealish	@virtualtodd, any tests with DPM and comparing power saved over without VMs?
Meza	ScottH, and the server is joined to the domain for authentication
VirtualTodd	Anyway, the results showed that DRS was able to manage the 8 Exchange VMs in the test such that performance improved versus if DRS was not used
Dell-KongY	So a key takeaway is to allow DRS to manage your VMs’ resource needs
VirtualTodd	I haven't done a DPM testing to combine with the DRS tests, but knowing that using DRS does not negatively affect Exchange means that it should work very well
Dell-KongY	@virtualtodd, what was the profile of your Exchange workload?
DELL-ScottH	Meza, on the servers with OMSA, under the Alert Management tab, SNMP Traps, are all the boxes checked for "Information, Warning, Critical"?
VirtualTodd	To get a workload that would change a bit over the course of the test I modeled a company that was spread across the four time zones of the U.S.
Meza	ScottH, okay, let me check that
VirtualTodd	This company just so happened to be exactly evenly spread across the four time zones with 4000 users in East, Central, Mountain, and Pacific
DELL-ScottH	Meza, was the predictive failure a "non-critical" alert? Or what severity did it report?
VirtualTodd	So at the beginning of the test 4000 users start, and then an hour later 4000 more, etc. Toward the end of the test the first 4000 users quit for the day first, then the next 4000, etc.
Meza	ScottH, yes, it's all checked informational, warning, critical
VirtualTodd	So at the beginning and the end there are opportunities for DRS to rebalance the workload. Hardware details: Dell PowerEdge M905 blades and Dell EqualLogic storage
Dell-KongY	So how many of you have deployed Exchange VMs on vSphere? Or are you waiting for reference solutions/architectures?
grealish	Not yet, but planning along with a few other bits
Meza	ScottH, everything has a green check. I had to drill down from System > Main System Chassis to Power Supplies to see the yellow triangles
VirtualTodd	I guess that's not really details, more like very high-level description :)
eczerwin	We did but then we never moved to it—not in production
grealish	But good that you mention PowerEdge M905 blades; we had a design spec done to run vSphere on
DELL-ScottH	Meza, okay, under Alert Management, Platform Events is the "enable Platform Event Filter alerts" box checked?
VirtualTodd	Here is a white paper that details performance of Exchange on vSphere that was done by one of my colleagues on the VMware performance team: http://blogs.vmware.com/performance/2009/05/exchange-2007-performance-on-vsphere-4.html
DELL-ScottH	Meza, hmmm, the yellow triangles didn't bubble up?
VirtualTodd	I have talked to many customers that run Exchange on VMware
Dell-KongY	@eczerwin, if you don't mind sharing, what's keeping you from virtualizing Exchange in production?
Meza	ScottH, no, the higher levels all have green checks
DELL-ScottH	Meza, which server model?
Meza	ScottH, Enable Platform Even Filter Alerts box is not checked
Dell-KongY	How many of you are waiting for Exchange 2010 before you virtualize Exchange?
VirtualTodd	Perhaps one of the most interesting case studies I have seen is the one that details VMware's deployment of Exchange internally (on ESX 3.5): www.vmware.com/files/pdf/solutions/08q4_vm_exchange_server_2007_vi3_wp.pdf
Meza	ScottH, both servers are PowerEdge 6800
DELL-ScottH	Meza, at a minimum, that alert should be showing on the Health page of the main "system" page, is it not?
Meza	ScottH, nope, System Page is showing two green checks
VirtualTodd	I'm really impressed that nobody has clicked a link and accidentally left the chat!
Dell-KongY	Thanks for sharing those links, Virtualtodd
Meza	ScottH, Main System Chassis, check, and Storage, check
DELL-ScottH	Meza, sounds like something is broken in the firmware; latest is on the server?
Meza	ScottH, which firmware? BIOS? BMC?
erson	Virtualtodd, maybe we all are using IE? a buggabuggabooo :)
eczerwin	@dell-kongy, nothing against the solution; the tests we did were just fine...there was just a change of plans, and we decided to consolidate data centers at a later time
DELL-ScottH	ScottH, links have been working better in IE 8 :-)
Dell-KongY	@eczerwin, understood
erson	I can click links with either right- or left-click using IE 7 with no problem whatsoever
VirtualTodd	Some people used to have issues or fears about running Exchange on VMware, but I think that those concerns have gone away almost completely. There are lots of performance papers and case studies showing that the performance is good
Meza	ScottH, Remote Access Controller firmware version 1.60, Baseboard Management Controller version 1.77, Peripheral bay backplane version 1.00, Backplane firmware version 1.00
VirtualTodd	It is also now an officially supported solution. Anybody run into issues or roadblocks lately with running Exchange on ESX?
Meza	ScottH, side note: does IT Assistant not work well with Windows 2008?
DELL-ScottH	Meza, there's DRAC 1.75 and BMC 1.83 available on the support site. I'd start there; seems like a bug, but I'm not in support so not sure if it's a known thing. IT Assistant works fine for me in Windows 2008
Dell-KongY	Just curious, how many Exchange users do you guys build your VMs out for? Todd's study has shown 16k users defined, tested, and supported. Also, do you prefer more VMs with fewer users or fewer VMs with more users supported?
DELL-ScottH	Meza, and there's A06 system BIOS
Dell-KongY	Personally, I like more VMs with smaller numbers of users to take advantage of vSphere features
grealish	One vCPU VMs with FT ;)
Dell-KongY	@grealish, agreed
grealish	And faster vMotion in DRS setups
Meza	ScottH, okay, I'll keep that in mind, since it's a heavy database server they are not going to let me flash firmware without good reason. I'll do some research
VirtualTodd	There has been some work done with Exchange VMs and FT: https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1/en_us/offering_technical/white_paper/h6260-virtual-infrastructure-clariion-cx4-120-vsphere-wp.pdf
Dell-KongY	Anyone using Exchange's HA/DR features over VMware's HA/FT/DR features?
VirtualTodd	Have to have a powerlink account
grealish	I've noticed that in our tests, vCenter really works better with smaller VMs; we could get more performance that way
Meza	ScottH, is version 8.3 the latest?
DELL-ScottH	Meza, yeah, I hear you on that. It seems like a bug; you can call into support and see what they say, but all the errors should bubble to the top health screen
VirtualTodd	I have also seen that 2vCPU VMs will perform better than 4vCPU VMs
Meza	ScottH, gotcha
DELL-ScottH	Meza, there is a version about to release as part of the OpenManage 6.1 release, in a few
VirtualTodd	Although the difference in performance might be imperceptible to most users
DELL-ScottH	Meza, 8.3 is currently the latest; let me pull up my release schedule
VirtualTodd	In general, you have more flexibility with smaller VMs to load balance among hosts
Meza	ScottH, ae the Software Updates not working anymore on IT Assistant? I can't get my Online Repository to sync
VirtualTodd	Is anybody considering using FT for Exchange? Or another workload?
Dell-KongY	So @ virtualtodd, what are your next focus areas as far as vSphere performance testing?
grealish	Yeah, it would make sense for load balancing to work better with smaller slot sizes
Meza	ScottH, lastly, do you know how I can enable scripting permanently on the server? I know that IT Assistant relies on VBS to send out e-mail
DELL-ScottH	Meza, they may have hosed up the catalog file again...arrggh, seems to happen near a release
VirtualTodd	I'm actually working on some Exchange FT testing right now; thanks to all in the Dell TechCenter lab
Meza	ScottH, yeah, I was trying to update it to check all my servers to see which ones are out of date, but can't ever get it to sync...I thought maybe it's because I moved it to Windows 2008
VirtualTodd	The stuff after that is actually under discussion right now, but looking at some multi-workload testing like maybe Exchange and SQL on a vSphere internal cloud
DELL-ScottH	Meza, I have another guy that pinged me about Content Manager not synching and uses the same catalog, so I'm sure something got corrupt on our side—has happened before. Please call that in so we can get more customers logged so they will work on it faster :-) I'm not sure about your question on permanently enabling scripting
Dell-KongY	Very cool. Heterogeneous workload testing in virtualized environments is an awesome topic
Meza	ScottH, this happened to me before in Windows 2003, but one of the IT Assistant gurus got it fixed for me. Basically, when sending out notifications IT Assistant will trigger a VB Script to send out the alert. On my server, I think it's a GPO or something; I'm not sure that disabled any scripting
Dell-KongY	There is always that question of what applications should run on the same system to be the most efficient
Dell-d_glynn	Todd, it sounds like you'll have some great examples for virtualizing the last 5 percent of the data center
grealish	We have a political problem with customers understanding private clouds in heterogeneous setups
Meza	ScottH, I have a script enabler that I can run when I log it and then notifications will work again, but as soon as I log off the scripting is disabled again, so no more notifications
grealish	And that I'm really interesting in hearing about tests
DELL-ScottH	Meza, ah, so a Windows security thing that is not allowing execution of the scripts
Meza	ScottH, yes!
VirtualTodd	I agree that most of what we are looking at now is that last 5 to 10 percent
DELL-ScottH	Meza, hmmm, is it just an executable? I think there are tools that can turn it into a service
Meza	ScottH, I wish I remember what the other guy did to get that to work, but I forgot. It's been a few years
VirtualTodd	grealish, can I quote you on that?
Meza	ScottH, well, about a year, not a few years
DELL-ScottH	Meza, that would be a hack though
Dell-KongY	In that last 5 to 10 percent, we are also looking at HPC workloads :) In virtualized environments
grealish	Sure
VirtualTodd	In theory, thinking of things as a cloud should be easier
Meza	ScottH, oh I see, I thought you might have a more elegant solution, but yeah I guess I can make it run as a service
VirtualTodd	I believe that management of a cloud will be different in some ways than what we are familiar with
Meza	ScottH, it's called noscript.exe
DELL-ScottH	Meza, well, if I were a better Windows guru, I might know how to fix it right :-)
grealish	Yes, but for people this is really a new IT philosophy that they find hard to grasp
DELL-ScottH	Meza, that was just my quick, off-the-cuff, "well this might work"
VirtualTodd	Philosophy and politics: I agree that those things are hard
Dell-d_glynn	HPC in the cloud? Very interesting
Dell-KongY	@grealish, is it more about ownership and utilization of resources?
grealish	Like I have a management meeting tomorrow, and I hope they won't bring this up :(
Meza	ScottH, ha ha, I'm sure you are a guru, but you just don't know about this one. So you haven't heard of anything like this with IT Assistant huh?
grealish	Like in firewall when management sees access policy, user groups, they like these words :)
DELL-ScottH	Meza, the more I learn, the more I learn I need to learn :-) I'm digging through the documents now to see if they have mention of it: http://support.dell.com/support/edocs/software/smsom/6.1/index.htm
grealish	And it makes it very easy to sell this
Meza	ScottH, BTW are they killing IT Assistant?
DELL-ScottH	Meza, 2. Windows Scripting Host (WSH) has disabled the running of VB scripts. By default, WSH is installed during the OS installation. WSH can be configured to prevent scripts with a .vbs extension from being run. On the desktop, right-click My Computer and then go to Open > Tools > Folder Options > File Types and look for the extension .vbs and verify that File Types is set to "vbscript Script File"; if not, click Change and choose "Microsoft Windows-Based Script Host" as the application that runs the script. And, yes, IT Assistant is being replaced by the Dell Management Console
VirtualTodd	But clouds are more difficult to grasp?
DELL-ScottH	Meza, still have a little more than a year
VirtualTodd	(For management)
Meza	ScottH, is that OpenManage 6.1?
grealish	For me, no; but anything new first approach is
DELL-ScottH	Meza, yes, but should be applicable for 6.0.1 too. From the readme: http://support.dell.com/support/edocs/software/smsom/6.1/en/readme/smconsole/readme.txt
grealish	However, will be interesting to see Exchange tested in heterogeneous setups, like with SAP. Most companies have this
Meza	ScottH, awesome with the WSH article...see, you are now a higher guru. But looks like the write up is for 2003. Hmmm...wonder what that is under 2008... I'm looking
VirtualTodd	I actually have been learning a little bit about SAP in the last few weeks; so that would be a good match for my testing
DELL-ScottH	Meza, oh how I love that they change the location of everything...keeps us all employed
Meza	ScottH, yeah, they have to confuse us so we get recertified
DELL-ScottH	Meza, and 6.1 was released on 7/2: http://support.dell.com/support/downloads/download.aspx?c=us&l=en&s=gen&releaseid=r226596&systemid=pwe_6800&servicetag=&os=wnet&osl=en&deviceid=554&devlib=0&typecnt=0&vercnt=11&catid=-1&impid=-1&formatcnt=1&libid=36&fileid=324791
grealish	These big names are for sure the selling test!
DELL-ScottH	Meza, we are all certifiable!
Meza	ScottH, oh, that's a good one!
Dell-KongY	@grealish, so I take it that an Exchange + SAP heterogeneous study would have an audience? ;)
grealish	But can I ask, how do you generate all the user workloads in your tests? @dell-kongy, yeah for sure!
VirtualTodd	For SAP, their SD benchmark is what I would use (that's what I learned how to set up and run). Exchange is the Loadgen tool from Microsoft
Meza	ScottH, no way! In 2008 changing of File Types is moved to Group Policy Management Console?!
grealish	You look at any large organization, they all have purchasing and finance departments, and the two heavy applications are e-mail and SAP. Okay, Citrix too for remote ;) Maybe BlackBerry connector too for the CRAIC ;)
Dell-KongY	My, oh my. How time passes by when you're having fun. Before I forget, thank you to Mr. Todd Muirhead!
VirtualTodd	Thanks Kong for inviting me for today
Dell-KongY	And thanks to the audience for joining us for today's chat!
VirtualTodd	Thanks to everybody for the great discussion and questions
grealish	NP guys, interesting chat
Dell-KongY	As always, the transcript of today's chat will be available tomorrow for your reading pleasure
erson	The much-improved I/O in Exchange 2010 will push people even more into virtualizing Exchange. There isn't much left that's hard to virtualize as long as you have suitable hardware
DELL-ScottH	What'd I miss? Was having great side chat session with Meza
Dell-d_glynn	You can catch up in tomorrow transcript :p
grealish	Sorry Scotth, world blew up and imploded again ;)
DELL-ScottH	Dang! Again?! Where are the transformers when you need them?
erson	Hopefully not in a third movie
DELL-ScottH	Thanks d_glynn :-) Fourth, 5th, and 6th!
Dell-d_glynn	Any time, Scott