Wednesday, June 27, 2007

testing the Sun X4600 M2 and ESX Server 3.0, part II

To reiterate, we visited Sun a few weeks back with visions of virtual machine testing faeries running through our heads:

The plan was to put the Sun X4600 M2 virtualization platform through its paces by toting three virtual machines to a Sun testing center and use ESX Server 3.0 installed on the box to manage a simulation of a highly available, high throughput eCommerce website:
- a Win2K3 web server running our web application (5GB)
- a RHEL3 server running Oracle 10G (34GB)
- a Win2K server to apply load to the web application (5GB)

The plan sounded easy enough, but the execution was somewhat more difficult. Let me tell you a story..

The vms were all stored on an external USB2.0 disk. We had left the two app and testing vms at Sun from the last visit, so they were already installed and ready to go. Since our data model is proprietary, we needed to bring a development version of our database with us each time we visited Sun's facility and delete it when we left. As I built the database on an Intel processor architecture, we needed to convert the database to the Opteron platform of the X4600 using VMware Converter each time we visited Sun. VMware Converter only runs on XP, so it was necessary to copy the db over to an XP workstation and then point VMware Converter to the ESX Server running on the X4600. As you can imagine, this was not an optimal situation, as we spent hours setting this up. By the way, VMware installs its own variant of Linux on the X4600. The Sun box does not run Solaris.

We had two hurdles to overcome:
1) we needed to copy our 34GB database to an XP machine with VMware Converter installed on it..this was a huge time sink
2) we needed to convert the database virtual machine from the Intel platform it was created on to the AMD Opteron platform of the Sun X4600M2

On this second visit to Sun, it took a couple hours to copy the RHEL3 Oracle database vm to the only available XP machine with enough space for it..our hosts' notebook computer! I guess that's what you get for trying to find an XP machine in a Sun testing lab! After the hour and a half long USB transfer, we were then able to start the conversion of the db vm. The conversion took about an hour. Both of these steps ended up eating through the entire morning.

The afternoon allowed me to familiarize myself with the Virtual Center Infrastructure client and the power it has. It bears a more in-depth review, but Virtual Center allows you to setup virtual machines with your desired configuration, start and stop them, and monitor running vms. One quick note if the VMware guys are reading this blog post: a nice addition to the Virtual Center client would be the ability to right-click on the performance charts to switch among the different performance monitor counters (CPU/Memory/Network/etc) immediately, instead of having to laboriously go into a separate menu each time you want to see the running stats of a particular virtual machine. I will try to review the product more in-depth at a future date.

The rest of the afternoon was mostly disappointing, but we did make a bit of headway. Since our last visit, we dumped Microsoft Web Stress Application tool in favor of the QEngine load testing tool. QEngine is much more robust than MS WSAT, is fairly easy to use as load testing tools go and has real time insight into the load on the source and destination servers. For now, we had installed QEngine on the Win2K virtual machine. Unfortunately, due to our limited knowledge of the tool, we were only able to generate a maximum volume of 20 simultaneous users. The tool froze when we tried to apply more load. This was an ignominious end to a generally disappointing day. But QEngine is easy to use and has some advanced functionality, so it bears further evaluation. In the next two weeks, the team is going to resolve the issue and I will try to give a full review of that product in the coming weeks.

One positive note that came out of the session was a chat session on how to resolve our copying/conversion dilemma by using a somewhat clever workaround. The idea is to bring an XP notebook with our database vm installed on it, so we would no longer have to spend the time to wait for the database to transfer over the network. And remember that XP is needed because VMware Converter runs on that OS alone. We could then hook up the notebook to Sun's network in the test lab, start VMware Converter in XP on the notebook and simply point the VMware Converter to the ESX Server installation on the X4600. The interesting thing is that I will use my MacBook Pro with 120GB of space to do this! I've made headway in the last few days on this issue and here's how I'm doing it:

Since we needed an XP machine, I thought I'd take one that was already XP workstation back in the office! But that's not a notebook, it's a dual-processor Xeon box chock full of 140GBs worth of programming tools, abandoned code and bloatware. As I didn't want to start with this fat XP Pro system, I slimmed down my XP Pro workstation's install profile to 8GB by deleting everything I didn't need off of it. Believe me, that was a helluva chore by itself! :)

Now it was time for VMware to the rescue. I used VMware Converter to convert the now trim physical XP system to a virtual machine. The destination of this conversion was another local drive within the workstation. After a slight hiccup, the conversion was successful! Now it was time to put the database in the vm. I started up the vm in VMware Server and copied the 34GB database to the vm via Windows Share. NICE! The file size of the XP vm was still 10GB though..I didn't quite understand that yet.

I stopped the new XP vm for transfer to the MacBook. Aha! Now I see that the vmdk file had ballooned to 45GB. I guess VMware waits until the vm is stopped to take inventory of how much space it uses. I wanted to transfer the XP Pro system with our database on it across to the MacBook, so I zipped up the virtual machine directory with WinZip. WinZip does an excellent job of compressing a virtual machine, so the final size of the zip archive was about 12GB zipped. Nice. I then copied it over the network to my MacBook Pro. I was able to copy the file in about seven minutes (roughly 5000Kbps) because I wired up a crossover cable between my dual Xeon workstation and the MacBook. Sweet!

The important moment was at hand! I had installed the latest 6/21/07 version of VMware Fusion and was excited to see if the 45GB vm would start up running under Mac OSX! I was a little nervous that the 45GB file wouldn't work on Fusion. Its starting slowly..BIOS..XP display..lights are dancing..mouse is moveable..logon prompt..YES! I'm in!! So I am very happy to report that the monster 45GB XP Pro vm with database started up and runs smoothly in VMware Fusion!

So we are good to go for round three at Sun. In the last few weeks, I have had some ups and downs with VMware Converter, but in general, I am still very happy with the rest of the toolset. So we are one step closer to validating our web site using VMware's virtualization technology. And we're using the full suite of VMware's tools to do it:
VMware Server
VMware Converter
VMware ESX Server 3.0

Until next time..

No comments:

Feel free to drop me a line or ask me a question.