Saturday, June 30, 2007

screen or window capture in mac osx

Here's one for the beginners to Mac OSX like myself. I was complaining in an earlier post that the Grab utility in Mac OSX should save to PNG format directly instead of me having to save a TIFF file and then convert the TIFF to PNG in Preview:
/2007/05/viewing-and-editing-html-source-code-in.html

Well, in my haste, I did not see that Preview has a plugin to Grab that allows you to Grab a screen or window capture directly to Preview:


Hence, there is no need for converting a TIFF, because the image is already in Preview and you simply save out to PNG format.

Ah well..I apologize to the Apple Developers. Even though, they still should build a standardized export interface on all image programs that come with the Mac. This interface should be able to export files to a multitude of standard uncompressed or web formats that the user can select.

I have now gotten off my high horse.

Wednesday, June 27, 2007

testing the Sun X4600 M2 and ESX Server 3.0, part II

To reiterate, we visited Sun a few weeks back with visions of virtual machine testing faeries running through our heads:
http://www.techanswerguy.com/2007/06/testing-sun-x4600m2-and-esx-server-30.html

The plan was to put the Sun X4600 M2 virtualization platform through its paces by toting three virtual machines to a Sun testing center and use ESX Server 3.0 installed on the box to manage a simulation of a highly available, high throughput eCommerce website:
- a Win2K3 web server running our web application (5GB)
- a RHEL3 server running Oracle 10G (34GB)
- a Win2K server to apply load to the web application (5GB)

The plan sounded easy enough, but the execution was somewhat more difficult. Let me tell you a story..

The vms were all stored on an external USB2.0 disk. We had left the two app and testing vms at Sun from the last visit, so they were already installed and ready to go. Since our data model is proprietary, we needed to bring a development version of our database with us each time we visited Sun's facility and delete it when we left. As I built the database on an Intel processor architecture, we needed to convert the database to the Opteron platform of the X4600 using VMware Converter each time we visited Sun. VMware Converter only runs on XP, so it was necessary to copy the db over to an XP workstation and then point VMware Converter to the ESX Server running on the X4600. As you can imagine, this was not an optimal situation, as we spent hours setting this up. By the way, VMware installs its own variant of Linux on the X4600. The Sun box does not run Solaris.

We had two hurdles to overcome:
1) we needed to copy our 34GB database to an XP machine with VMware Converter installed on it..this was a huge time sink
2) we needed to convert the database virtual machine from the Intel platform it was created on to the AMD Opteron platform of the Sun X4600M2

On this second visit to Sun, it took a couple hours to copy the RHEL3 Oracle database vm to the only available XP machine with enough space for it..our hosts' notebook computer! I guess that's what you get for trying to find an XP machine in a Sun testing lab! After the hour and a half long USB transfer, we were then able to start the conversion of the db vm. The conversion took about an hour. Both of these steps ended up eating through the entire morning.

The afternoon allowed me to familiarize myself with the Virtual Center Infrastructure client and the power it has. It bears a more in-depth review, but Virtual Center allows you to setup virtual machines with your desired configuration, start and stop them, and monitor running vms. One quick note if the VMware guys are reading this blog post: a nice addition to the Virtual Center client would be the ability to right-click on the performance charts to switch among the different performance monitor counters (CPU/Memory/Network/etc) immediately, instead of having to laboriously go into a separate menu each time you want to see the running stats of a particular virtual machine. I will try to review the product more in-depth at a future date.

The rest of the afternoon was mostly disappointing, but we did make a bit of headway. Since our last visit, we dumped Microsoft Web Stress Application tool in favor of the QEngine load testing tool. QEngine is much more robust than MS WSAT, is fairly easy to use as load testing tools go and has real time insight into the load on the source and destination servers. For now, we had installed QEngine on the Win2K virtual machine. Unfortunately, due to our limited knowledge of the tool, we were only able to generate a maximum volume of 20 simultaneous users. The tool froze when we tried to apply more load. This was an ignominious end to a generally disappointing day. But QEngine is easy to use and has some advanced functionality, so it bears further evaluation. In the next two weeks, the team is going to resolve the issue and I will try to give a full review of that product in the coming weeks.

One positive note that came out of the session was a chat session on how to resolve our copying/conversion dilemma by using a somewhat clever workaround. The idea is to bring an XP notebook with our database vm installed on it, so we would no longer have to spend the time to wait for the database to transfer over the network. And remember that XP is needed because VMware Converter runs on that OS alone. We could then hook up the notebook to Sun's network in the test lab, start VMware Converter in XP on the notebook and simply point the VMware Converter to the ESX Server installation on the X4600. The interesting thing is that I will use my MacBook Pro with 120GB of space to do this! I've made headway in the last few days on this issue and here's how I'm doing it:

Since we needed an XP machine, I thought I'd take one that was already configured..my XP workstation back in the office! But that's not a notebook, it's a dual-processor Xeon box chock full of 140GBs worth of programming tools, abandoned code and bloatware. As I didn't want to start with this fat XP Pro system, I slimmed down my XP Pro workstation's install profile to 8GB by deleting everything I didn't need off of it. Believe me, that was a helluva chore by itself! :)

Now it was time for VMware to the rescue. I used VMware Converter to convert the now trim physical XP system to a virtual machine. The destination of this conversion was another local drive within the workstation. After a slight hiccup, the conversion was successful! Now it was time to put the database in the vm. I started up the vm in VMware Server and copied the 34GB database to the vm via Windows Share. NICE! The file size of the XP vm was still 10GB though..I didn't quite understand that yet.

I stopped the new XP vm for transfer to the MacBook. Aha! Now I see that the vmdk file had ballooned to 45GB. I guess VMware waits until the vm is stopped to take inventory of how much space it uses. I wanted to transfer the XP Pro system with our database on it across to the MacBook, so I zipped up the virtual machine directory with WinZip. WinZip does an excellent job of compressing a virtual machine, so the final size of the zip archive was about 12GB zipped. Nice. I then copied it over the network to my MacBook Pro. I was able to copy the file in about seven minutes (roughly 5000Kbps) because I wired up a crossover cable between my dual Xeon workstation and the MacBook. Sweet!

The important moment was at hand! I had installed the latest 6/21/07 version of VMware Fusion and was excited to see if the 45GB vm would start up running under Mac OSX! I was a little nervous that the 45GB file wouldn't work on Fusion. Its starting slowly..BIOS..XP display..lights are dancing..mouse is moveable..logon prompt..YES! I'm in!! So I am very happy to report that the monster 45GB XP Pro vm with database started up and runs smoothly in VMware Fusion!

So we are good to go for round three at Sun. In the last few weeks, I have had some ups and downs with VMware Converter, but in general, I am still very happy with the rest of the toolset. So we are one step closer to validating our web site using VMware's virtualization technology. And we're using the full suite of VMware's tools to do it:
VMware Server
VMware Converter
VMware ESX Server 3.0

Until next time..

performance note for VMs: they love fast drives

Because virtual machines are essentially big files, you will benefit if the disks they are stored on are fast. So, if you stripe a couple SATA or SAS drives, this should really help speed things up. Enterprise users will obviously have access to better firepower (RAID 1+0, RAID 5, etc), but I'm framing this in the context of the tech dabbler just getting his feet wet with the technology.

I bring this issue up because I put an old, circa 1999 20GB spare drive in my XP workstation and used that drive as the conversion destination of my local XP Professional Workstation to vm. After the conversion, which was successful, I started the vm and the performance was horrible! It was almost like the vm hung or was frozen. In truth, the vm was just extremely slow because the IDE drive was circa 1999 and had little buffer for the 8GB+ vm. I didn't immediately know what was happening, so I used XP's Performance Monitor to view average disk queue length.


Average disk queue length is a rough measure of disk performance, but it is a useful gauge of i/o problems. For me, I find that values of over 20 indicate performance issues. Your mileage may vary.

One note: if you don't like the default scale of 100:1 on the Performance Monitor chart, you can change that default scale by:
1) right-clicking on the statistic in the legend of the chart
2) select Properties
3) click the Data tab
4) choose another scale under the Default Scale dropdown menu

UPDATE 7/6/2007: I've expanded my discussion of performance monitoring VMware Server here:
/2007/07/measuring-performance-while-using.html

Also, I noticed my performance seems to suffer if you use the "Split Disk Into 2GB files" option. As well, I selected "Allocate all disk space now" for better performance.

In regards to this option, I found a bug in the VMware Converter gui. When I initially went to configure the conversion of the local machine, I was not able to deselect the "Split disk into 2GB files" option for the destination drive (a second local hard disk) that I wanted to use.


The checkbox was greyed out and unselectable. Trying to fix the problem, I returned to the destination dialog box, changed my destination drive to a network share and the option suddenly became available. On a hunch, I then went back to the destination location window, changed the destination drive to the original local drive and the "Split disk into 2GB files" option was now available.

I don't know why the option was made unavailable at first, but I was glad to be able to route around the error.

how to get operating system version in Linux/Solaris

A quicky because my brain wasn't working well yesterday. Here are two quick ways to get the operating system version on a Linux box or a Solaris box:

1) sar
Take the top of the output using "sar ¦ head". The output looks like this:
cricketbox cricket >sar ¦ head

SunOS njunpcrkt00 5.8 Generic_108528-29 sun4u 06/27/07

00:01:00 %usr %sys %wio %idle
00:06:00 0 1 1 98
00:11:00 1 2 2 95
00:16:00 1 2 7 90


2) uname -a
cricketbox cricket >uname -a
SunOS njunpcrkt00 5.8 Generic_108528-29 sun4u sparc SUNW,Sun-Fire-280R


On a Linux box like the Fedora Core 6 box below, uname may only print the kernel version:
[user@computer ~]# uname -a
Linux computer.test.com 2.6.18-1.2798.fc6 #1 SMP Mon Oct 16 14:37:32 EDT 2006 i686 i686 i386 GNU/Linux

Here's a related article on how to find the BIOS version of your hardware:
http://www.techanswerguy.com/2007/07/finding-out-bios-version-in-linux.html

Have a good day.

Friday, June 22, 2007

from IIS, convert an SSL cert to PEM format

As we were moving our SSL termination from the web servers to an F5 3400 load balancing switch, I needed to convert our SSL certificate to PEM format. This is a subject that has been the source of much frustration and I thought I'd clarify the process for individuals suffering the same fate as I using step by step instructions.

Update 2010/04/06
On Windows 2003, you can only export the private key via the Certificate Manager Snap-In, not the IIS MMC.
*** end update ***

Requirements
In order to do the conversion, you'll need access to your IIS box with an SSL certificate installed and OpenSSL. I run OpenSSL using cygwin (http://www.cygwin.com/), a set of Unix tools for NT.

Overview
1. Export the certificate from IIS
2. Convert the PKCS#12 certificate
3. Verify the certificate
4. Move the sections in the certificate into the correct order
5. Final test

Detailed Steps

Export the certificate from IIS
1. Open the Internet Information Services (IIS) Manager administration tool.
2. Expand the Web Sites node and locate the SSL-enabled Web site.
3. Right-click this Web site and click Properties.
4. Click the Directory Security tab and in the Secure Communications section of the window, select the View Certificate box.
5. Click the Details tab and click Copy to File


6. Click Next on the Welcome to the Certificate Export Wizard page.


7. Select Yes, export the private key and click Next:


Note: The private key MUST be exported for SSL to work. If the option to export the private key is unavailable, see Microsoft article 232154 – IIS: Export Private Key Option is Grayed When Exporting a Server Certificate.

8. Ensure that the Personal Information Exchange –PKCS #12 radio button is selected and select ONLY the Include all certificates in the certification path if possible check box. Click Next:


9. Enter a password and click Next:


10. Enter a file name and location and click Next. Give the file an extension of .PFX:


11. Click Finish:




Convert the PKCS#12 certificate
1. Move the exported .PFX certificate file to a location from where it may be converted with OpenSSL
2. Convert it with openssl
# openssl.exe pkcs12 -in exportPKCS12.pfx -out cert.pem -nodes

bash-3.00$ openssl.exe pkcs12 -in exportPKCS12.pfx -out cert.pem -nodes
Enter Import Password:
MAC verified OK


Verify the certificate
1. Ensure that the converted certificate is in correct x509 format. Do this by verifying that the following command produces no errors:
# openssl x509 -in cert.pem -text

bash-3.00$ openssl x509 -in cert.pem -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
74:e0:f4:04:1d:af:04:43:aa:89:27:69:ab:74:f9:55
Signature Algorithm: sha1WithRSAEncryption
Issuer: O=VeriSign Trust Network, OU=VeriSign, Inc., OU=VeriSign International Server CA - Class 3, OU=www.veris
ign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
Validity
Not Before: Aug 16 00:00:00 2005 GMT
Not After : Aug 26 23:59:59 2007 GMT
Subject: C=US, ST=New York, L=New York, O=Test, OU=IT eServicesTerms of use at www.verisign.com/rpa (c
)00, OU=Terms of use at www.verisign.com/rpa (c)00, CN=www.test.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (1024 bit)
Modulus (1024 bit):
00:cb:70:5f:ce:ee:d6:8c:5f:13:7a:06:df:51:56:
7d:d9:50:11:dc:69:d7:5d:d8:90:f7:9f:47:df:8d:
cc:56:ae:af:e0:c5:b5:5e:40:a8:8c:5d:03:a8:5e:
49:1c:13:42:2e:bd:d5:29:5e:16:49:18:9b:a3:84:
32:d8:f9:8e:84:2d:14:27:a6:80:b5:19:03:93:39:
44:9f:4f:b8:c5:49:42:9a:ef:0c:9a:cf:05:9e:6f:
02:d7:fb:0b:04:d5:1d:f8:fa:3b:29:d8:ed:bb:fd:
9b:ae:fc:27:1d:d3:ed:d6:86:8a:fb:f0:a3:61:85:
13:a0:84:1c:8a:12:64:0e:17
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
X509v3 Key Usage:
Digital Signature, Key Encipherment
X509v3 CRL Distribution Points:
URI:http://crl.verisign.com/Class3InternationalServer.crl

X509v3 Certificate Policies:
Policy: 2.16.840.1.113733.1.7.23.3
CPS: https://www.verisign.com/rpa

X509v3 Extended Key Usage:
Netscape Server Gated Crypto, Microsoft Server Gated Crypto, TLS Web Server Authentication, TLS Web Clie
nt Authentication
Authority Information Access:
OCSP - URI:http://ocsp.verisign.com

1.3.6.1.5.5.7.1.12:
0_.].[0Y0W0U..image/gif0!0.0...+..............k...j.H.,{..0%.#http://logo.verisign.com/vslogo.gif
Signature Algorithm: sha1WithRSAEncryption
88:ff:10:98:c4:90:62:c2:77:61:33:5e:ba:fa:5f:ca:7e:2d:
7c:1e:b7:64:7c:b8:df:33:4b:92:8a:0a:29:85:b8:9c:c5:93:
db:b2:f0:fc:77:b4:50:db:ec:db:df:c2:36:b9:b6:9e:7c:12:
c4:8d:83:02:43:f1:fc:6a:94:f9:d9:75:e5:66:70:04:74:04:
89:f9:15:8f:1f:45:36:51:b7:a9:7f:7d:94:a9:10:e3:51:c5:
b0:1c:b6:d4:23:cb:66:1d:c0:b0:0d:72:69:dd:49:9d:a8:e4:
33:12:70:8a:b7:84:85:a5:3a:5c:7f:c9:d1:e7:e2:b4:31:57:
ff:e4
-----BEGIN CERTIFICATE-----
MIIErTCCBBagAwIBAgIQdOD0BB2vBEOqiSdpq3T5VTANBgkqhkiG9w0BAQUFADCB
ujEfMB0GA1UEChMwVmVyaVNpZ24gVHJ1c3QgTmV0d29yazEXMBUGA1UECxMOVmVy
aVNpZ24sIEluYy4xfzAxBgNVBAsTKlZlcmlTaWduIEludGVybmF0aW9uYWwgU2Vy
dmVyIENBIC0gQ2xhcsMgMzFJMEcGA1UECxNAd3d3LnZlcmlzaWduLmNvbS9DUFMg
SW5jb3JwLmJ5IFJlZi4gTElBQklMSVRZIExURC4oYyk5NyBWZXJpU2lnbjAeFw0w
NTA4MTYwMDAwMDBaFw0wNzA4MjYyMzU5NTlaMIHlMQswCQYDVQQGEwJVUzETMBEG
A1UECBMKTmV3IEplcnNleTEOMAwGA1UEBxQFVW5pb24xGDAWBgNVBAoUD0JlZCBC
YXRoIEJleW9uZDE/MD0GA1UECxQ2SVQgZVNlcnZpY2VzVGVybXMgb2YgdXNlIGF0
IHd3dy52ZXJpc2lnbi5jb20vcnBhIChjKTAwMTMwMQYDVQQLFCpUZXJtcyBvZiB1
c2UgYXQgd3d3LnZlcmlzaWduLmNvbS9ycGEgKGMpMDAxITAfBgNVBAMUGHd3dy5i
ZWRiYXRoYW5kYmV5b25kLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
y3Bfzu7WjF8TegbfUVZ92VAR3GnXXdiQ959H343MVq6v4MW1XkCojF0DqF5JHBNC
Lr3VKV4WSRibo4Qy2PmOhC0UJ6aAtRkDkzlEn0+4xUlCmu8Mms8Fnm8C1/sLBNUd
+Po7Kdjtu/2brvwnHdPt1oaK+/CjYYUToIQcihJkDhcCAwEAAaOCAYUwggGBMAkG
A1UdEwQCMAAwCwYDVR0PBAQDAgWgMEYGA1UdHwQ/MD0wO6A5oDeGNWh0dHA6Ly9j
cmwudmVyaXNpZ24uY29tL0NsYXNzM0ludGVybmF0aW9uYWxTZXJ2ZXIuY3JsMEQG
A1UdIAQ9MDswOQYLYIZIAYb4RQEHFwMwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93
d3cudmVyaXNpZ24uY29tL3JwYTA0BgNVHSUELTArBglghkgBhvhCBAEGCisGAQQB
gjcKAwMGCCsGAQUFBwMBBggrBgEFBQcDAjA0BggrBgEFBQcBAQQoMCYwJAYIKwYB
BQUHMAGGGGh0dHA6Ly9vY3NwLnZlcmlzaWduLmNvbTBtBggrBgEFBQcBDARhMF+h
XaBbMFkwVzBVFglpbWFnZS9naWYwITAfxAcGBSsOAwIaBBSP5dMahqyNjmvDz4Bq
1EgYLHsZLjAlFiNodHRwOi8vbG9nby52ZXJpc2lnbi5jb20vdnNsb2dvLmdpZjAN
BgkqhkiG9w0BAQUFAAOBgQCI/xzYxJBiwndhM166+l/Kfi18HrdkfLjfM0uSigop
hbicxZPbsvD8d7RQ2+zb38I2ubaefBLEjYMCQ/H8apT52XXlZnAEdASJ+RWPH0U2
Ubepf32UqRDjUcWwHLbUI8tmHcCwDXJp3UmdqOQzEnCKt4SFpTpcf8nR5+K0MVf/
5A==
-----END CERTIFICATE-----


2. Now that we've verified that the certificate is in correct x509 format, check that the certificate file contains a private key:
# cat cert.pem

Look at the output (example shown below) and check for a section that looks similar to this:
-----BEGIN RSA PRIVATE KEY-----
XaBbMFkwVzBVFglpbWFnZS9naWYwITAfxAcGBSsOAwIaBBSP5dMahqyNjmvDz4Bq
1EgYLHsZLjAlFiNodHRwOi8vbG9nby52ZXJpc2lnbi5jb20vdnNsb2dvLmdpZjAN
BgkqhkiG9w0BAQUFAAOBgQCI/xzYxJBiwndhM166+l/Kfi18HrdkfLjfM0uSigop
hbicxZPbsvD8d7RQ2+zb38I2ubaefBLEjYMCQ/H8apT52XXlZnAEdASJ+RWPH0U2
-----END RSA PRIVATE KEY-----

Move the sections in the certificate into the correct order

For some reason, OpenSSL converts the PFX to PEM where the sections of the certificate are in the wrong order. The key sections (shown below) must be in the following order:
a. Private key
b. Intermediate certificate
c. Server certificate

As the PEM is a text file, use your favorite editor (vi or emacs or any editor that does not add or remove line feeds) to reorder the sections as noted above.

Key sections within the certificate

The following is the RSA PRIVATE KEY section:
Bag Attributes
1.3.6.1.4.1.311.17.2:
localKeyID: 01 00 00 00
Microsoft CSP Name: Microsoft RSA SChannel Cryptographic Provider
friendlyName: 4b9cef4cc8c9b849ff5c662fd3e0ef7e_76267e3e-6183-4d45-886e-6e067297b38f
Key Attributes
X509v3 Key Usage: 10
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: QWS-EDE3-CCC,43E7ACA5F4423968
pZJ2SfsSVqMbRRf6ug37Clua5gY0Wld4frPIxFXyJquUHr31dilW5ta3hbIaQ+Rg
... (random characters)
v8dMugeRplkaH2Uwt/mWBk4t71Yv7GeHmcmjafK8H8iW80ooPO3D/ENV8X4U/tlh
5eU6ky3WYZ1BTy6thxxLlwAullynVXZEflNLxq1oX+ZYl6djgjE3qg==
-----END RSA PRIVATE KEY-----

The following is the SERVER CERTIFICATE section:

Bag Attributes
localKeyID: 01 00 00 00
friendlyName: Test Certificate
subject=/C=AU/ST=NSW/L=test/O=Furnishing/OU=Support/CN=test.retail.home
issuer=/DC=ranch/DC=retail/CN=home
-----BEGIN CERTIFICATE-----
MIIFiTCCBHGgAwIBAgIKCGryDgAHzANBgkqhkiG9w0BAQUFADA8MRMwEQYK
... (random characters)
5pLDWYVHhLkA1pSxvFjNJHRSIydWHc5ltGyKqIUcBezVaXyel94pNSUYx07NpPV/
MY2ovQyQZM8gGe3+lGFum0VHbv/y/gB9HhFesog=
-----END CERTIFICATE-----

The following is the INTERMEDIATE CA CERTIFICATE section:

Bag Attributes:
subject=/DC=ranch/DC=retail/CN=home
issuer=/DC=ranch/DC=retail/CN=home
-----BEGIN CERTIFICATE-----
MIIESDCCAzCgAwIBAgIQah20fCRYTY9LRXYMIRaKGjANBgkqhkiG9w0BAQUFADA8
... (random characters)
Nt0nksawDnbKo86rQcNnY5xUs7c7pj2zxj/IOsgNHUp5W6dDI9pQoqFFaDk=
-----END CERTIFICATE-----


Further Intermediate CA certificates may follow depending on the certification path of the exported certificate.

Example of key sections of PEM in wrong order:


Example of key sections of PEM in correct order:


Final test procedure
Use OpenSSL’s verify command to verify that the certificate is properly formed:
# openssl verify cert.pem

bash-3.00$ openssl verify cert.pem
cert2.pem: OK


That's it folks!

Troubleshooting
When I did not move the server certificate portion of the PEM file to the bottom of the PEM file, I received the error:
error 20 at 0 depth lookup:unable to get local issuer certificate

As WordPad does not add control characters to the document, I used WordPad to move the server certificate portion of the PEM file to the bottom. After doing this, I then received the success message:

bash-2.02$ openssl verify certorder.pem
certorder.pem: OK

Thursday, June 07, 2007

VMware Player install on Linux (Fedora Core 6)

As I happen to live in all three worlds (Mac, Linux and Windows), I occasionally need to run XP programs. I thought it would be an even better idea to be able to run XP programs while in Linux, so I recently converted my XP OS to a virtual machine using VMware Converter. If you wish to convert your 2000/XP system to a vm, here is one way to do that:
http://www.techanswerguy.com/2007/06/converting-physical-server-using-vmware.html

Once I had a working virtual copy of my XP system, I then installed VMware Player on my Fedora Core 6 system.

Here are the steps to do that:

1) Get the RPM download from VMware:
http://www.vmware.com/download/player

2) Install the RPM:
[root@computer ~]# rpm -ivh VMware-player-2.0.0-45731.i386.rpm
Preparing... ########################################### [100%]
1:VMwarePlayer ########################################### [100%]

3) Configure VMware Player:
[root@computer ~]# /usr/bin/vmware-config.pl
Making sure services for VMware Player are stopped.

Stopping VMware services:
Virtual machine monitor [ OK ]

Configuring fallback GTK+ 2.4 libraries.

In which directory do you want to install the theme icons?
[/usr/share/icons]

What directory contains your desktop menu entry files? These files have a
.desktop file extension. [/usr/share/applications]

In which directory do you want to install the application's icon?
[/usr/share/pixmaps]

Trying to find a suitable vmmon module for your running kernel.

None of the pre-built vmmon modules for VMware Player is suitable for your
running kernel. Do you want this program to try to build the vmmon module for
your system
(you need to have a C compiler installed on your system)? [yes]

Using compiler "/usr/bin/gcc". Use environment variable CC to override.

What is the location of the directory of C header files that match your running kernel? [/lib/modules/2.6.18-1.2798.fc6/build/include]

Extracting the sources of the vmmon module.

Building the vmmon module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmmon-only'
make -C /lib/modules/2.6.18-1.2798.fc6/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/kernels/2.6.18-1.2798.fc6-i686'
CC [M] /tmp/vmware-config0/vmmon-only/linux/driver.o
CC [M] /tmp/vmware-config0/vmmon-only/linux/hostif.o
CC [M] /tmp/vmware-config0/vmmon-only/common/comport.o
CC [M] /tmp/vmware-config0/vmmon-only/common/cpuid.o
CC [M] /tmp/vmware-config0/vmmon-only/common/hash.o
CC [M] /tmp/vmware-config0/vmmon-only/common/memtrack.o
CC [M] /tmp/vmware-config0/vmmon-only/common/phystrack.o
CC [M] /tmp/vmware-config0/vmmon-only/common/task.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciContext.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciDatagram.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciDriver.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciDs.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciGroup.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciHashtable.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciProcess.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciResource.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmciSharedMem.o
CC [M] /tmp/vmware-config0/vmmon-only/common/vmx86.o
CC [M] /tmp/vmware-config0/vmmon-only/vmcore/moduleloop.o
LD [M] /tmp/vmware-config0/vmmon-only/vmmon.o
Building modules, stage 2.
MODPOST
CC /tmp/vmware-config0/vmmon-only/vmmon.mod.o
LD [M] /tmp/vmware-config0/vmmon-only/vmmon.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.18-1.2798.fc6-i686'
cp -f vmmon.ko ./../vmmon.o
make: Leaving directory `/tmp/vmware-config0/vmmon-only'
The module loads perfectly in the running kernel.

Extracting the sources of the vmblock module.

Building the vmblock module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmblock-only'
make -C /lib/modules/2.6.18-1.2798.fc6/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/kernels/2.6.18-1.2798.fc6-i686'
CC [M] /tmp/vmware-config0/vmblock-only/linux/block.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/control.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/dbllnklst.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/dentry.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/file.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/filesystem.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/inode.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/module.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/stubs.o
CC [M] /tmp/vmware-config0/vmblock-only/linux/super.o
LD [M] /tmp/vmware-config0/vmblock-only/vmblock.o
Building modules, stage 2.
MODPOST
CC /tmp/vmware-config0/vmblock-only/vmblock.mod.o
LD [M] /tmp/vmware-config0/vmblock-only/vmblock.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.18-1.2798.fc6-i686'
cp -f vmblock.ko ./../vmblock.o
make: Leaving directory `/tmp/vmware-config0/vmblock-only'
The module loads perfectly in the running kernel.

Do you want networking for your virtual machines? (yes/no/help) [yes]

Configuring a bridged network for vmnet0.

The following bridged networks have been defined:

. vmnet0 is bridged to eth0

All your ethernet interfaces are already bridged.

Do you want to be able to use NAT networking in your virtual machines? (yes/no)
[yes] yes

Do you want to be able to use host-only networking in your virtual machines?
[no]

Extracting the sources of the vmnet module.

Building the vmnet module.

Using 2.6.x kernel build system.
make: Entering directory `/tmp/vmware-config0/vmnet-only'
make -C /lib/modules/2.6.18-1.2798.fc6/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. modules
make[1]: Entering directory `/usr/src/kernels/2.6.18-1.2798.fc6-i686'
CC [M] /tmp/vmware-config0/vmnet-only/driver.o
CC [M] /tmp/vmware-config0/vmnet-only/hub.o
CC [M] /tmp/vmware-config0/vmnet-only/userif.o
CC [M] /tmp/vmware-config0/vmnet-only/netif.o
CC [M] /tmp/vmware-config0/vmnet-only/bridge.o
CC [M] /tmp/vmware-config0/vmnet-only/filter.o
CC [M] /tmp/vmware-config0/vmnet-only/procfs.o
CC [M] /tmp/vmware-config0/vmnet-only/smac_compat.o
SHIPPED /tmp/vmware-config0/vmnet-only/smac_linux.x386.o
LD [M] /tmp/vmware-config0/vmnet-only/vmnet.o
Building modules, stage 2.
MODPOST
WARNING: could not find /tmp/vmware-config0/vmnet-only/.smac_linux.x386.o.cmd for /tmp/vmware-config0/vmnet-only/smac_linux.x386.o
CC /tmp/vmware-config0/vmnet-only/vmnet.mod.o
LD [M] /tmp/vmware-config0/vmnet-only/vmnet.ko
make[1]: Leaving directory `/usr/src/kernels/2.6.18-1.2798.fc6-i686'
cp -f vmnet.ko ./../vmnet.o
make: Leaving directory `/tmp/vmware-config0/vmnet-only'
The module loads perfectly in the running kernel.

Starting VMware services:
Virtual machine monitor [ OK ]
Blocking file system: [ OK ]
Virtual ethernet [ OK ]
Bridged networking on /dev/vmnet0 [ OK ]
Host network detection [ OK ]

The configuration of VMware Player 2.0.0 build-45731 for Linux for this running
kernel completed successfully.

Enjoy,

--the VMware team

You can now run VMware Player by invoking the following command:
"/usr/bin/vmplayer".


4) Run VMware Player:
You can now run VMware Player by invoking the following command:
"/usr/bin/vmplayer":

[root@computer ~]# /usr/bin/vmplayer
Ah, the lovely splash screen!


5) Open an existing virtual machine:


6) Find your existing virtual machine's .vmx file:


Your existing virtual machine will start:


Don't forget..you cannot start virtual machines within a virtual machine!

Enjoy!

Wednesday, June 06, 2007

VMware Player: no sound / "bad directsound driver"

After I converted my Windows 2000 Professional machine over to a virtual machine, I was disappointed to find that VMware Server doesn't support sound. Here's non-official confirmation of that:
http://kontrawize.blogs.com/kontrawize/2006/03/vmware_server_v_2.html

UPDATE 7/6/2007: I have gotten sound to work using both a Windows 2000 Advanced Server and an XP guest system in VMware Server. You will need to apply the tweak below to enable sound.

Therefore, I downloaded VMware Player, simply to play sound from my VM. Upon installing Player, I saw this error when I tried to initialize my audio application:
"Bad DirectSound Driver"

Apparently, VMware Player sets its config file for the virtual machine to use Sound Blaster emulation. When in truth, the emulation should be Creative AudioPCI (ES1371,ES1373).

So if you have working sound on your host system, but not on the vm running in VMware Player, here are the configuration lines in your VMs .vmx file you need to have in order to enable sound in the virtual machine running in VMware Player:
sound.present = "TRUE"
sound.virtualDev = "es1371"
sound.filename = "-1"
sound.autodetect = "TRUE"

Just make sure to edit/add the lines while your virtual machine is powered off.

It would be nice if VMware could somehow do a pass-through to the actual hardware device installed on the host so that you could do nice things like get low latency for recording software applications like Cubase or Reason.

I will ask the gurus..
UPDATE: Pass through to the real sound card not possible with Workstation or Player. Sound only works as emulated.

VMware Player is a bit of a kids toy, as there is not much to configure and the CD/Floppy/Network/Sound devices are all toggles at the top of the Player window:


Kinda weak. But hey, its free and it works.

Kontrawize also has their own opinions of the differences between VMware Server and Workstation here:
http://kontrawize.blogs.com/kontrawize/2006/03/vmware_server_v.html

Ah, the silliness continues..

converting a physical server using VMware Converter

This morning, I spent some time converting a physical server to a virtual machine using VMware Converter. Here's a good starter page of info to work from, the VMware Converter FAQ, if you do not know the capabilities of VMware Converter:
http://www.vmware.com/products/converter/faqs.html

One detractor from the Converter is that you cannot convert a physical Linux machine to a VMware VM. Oh well.

In this case, however, the physical server was a Windows 2000 Professional machine and is home to my digital audio workstation. This box has a myriad of applications on it, from Cool Edit to Cakewalk and Cubase to Rebirth and Reason. I was planning on building out a newer, more powerful server using the case that the Win2K Professional is in and I don't want to take time and effort to reinstall those applications on my XP box. Also, I don't want to slow the XP box down with a crapload of new applications gobbing up the registry, so it makes sense that I convert my 2000 box to a virtual machine. Also, once virtualized, I will then be able to use the audio workstation in either of my primary environments, XP or Fedora.

VMware Converter is installed on my XP Professional machine. So here is the plan:
-Start VMware Converter on XP
- Convert a physical computer, my Windows 2000 digital audio workstation, to a virtual machine
- Create a Windows share on my XP machine and use the share as the destination for the conversion

The process was relatively pain-free and quick. Here's what I did.
1) on XP, I started VMware Converter. There is a splash screen and then I was given a choice of source computer for the conversion. I chose "Physical Computer:"


2) Converter asks for the name or IP of the remote machine and my authentication credentials:


3) Once logged in, Converter asks to temporarily install Converter Agent on the source machine. I chose to manually uninstall the files later if anything went wrong during the conversion process:


4) Converter then shows the system or active drives on the OS to convert. My setup of one active system drive is the simplest scenario:


5) Converter asked me for a vm name and a location (ie, a network share) to write the destination vm to:


6) Next, I could specify a dynamic virtual disk or a fixed allocation for better performance. In order to save space on my main system, I chose dynamic ("Allow virtual disk files to grow"):


7) Converter asks what type of networking I'd like. Bridged is the default and I accept it:


8) I chose no network or hostname customizations:


9) At this point, Converter displays a summary screen and is ready for me to start the import.


10) One small snag: since I just installed the Converter Agent on the source machine, Converter needs to reboot the source machine to make the Agent active. I click "Yes:"


11) Unfortunately, after the reboot of the source server, the install hangs and does not progress. Figuring that it might be related to the recent Converter Agent install, I stopped the import. I recreated the conversion steps above (steps 1-10). After rebuilding the import project, the conversion started up again successfully and I saw the Progress column "% Done" increment higher:


12) Happily, the conversion works!


Now, after I powered up my new Windows 2000 virtual machine running under VMware Server on my XP box, I see that the underlying motherboard, cdrom and various system devices change. Uh oh!

Be aware of this if you migrate a physical computer to a virtual machine! Virtual machines under VMware are based on the Intel 440BX motherboards and Phoenix BIOS 4.0 Release 6.0:


You'll also see VMware BIOS build 245 if you enable the Boot-Time Diagnostic Screen:


In order to be prepared for automatic system reconfiguration that Windows will do when it sees its new "motherboard", the best idea for you is to have an I386 on your hard drive or a CD with all your system drivers ready to go after the conversion. That way, the process will be pretty painless, outside of reconfiguring your display settings!

Even with the snag, this process took about an hour and a half from start to finish. Now I have my darling digital audio workstation forever enshrined as a virtual machine usable in either of my XP or Fedora environments. Sweet!

UPDATE: VMware server doesn't support sound, but VMware Player does support it. Also, VMware Player sets its config file for the virtual machine to use Sound Blaster emulation. In truth, the emulation should be Creative AudioPCI (ES1371,ES1373). Therefore, if you have working sound on your host system, but not on the vm running in VMware Player, you need to change your virtual machine's .vmx file to enable sound:
sound.present = "TRUE"
sound.virtualDev = "es1371"
sound.filename = "-1"
sound.autodetect = "TRUE"

Just make sure to edit/add the lines while your virtual machine is powered off. It would be nice if VMware could somehow do a pass-through to the actual hardware device installed on the host. Unfortunately, this is currently not possible. So I cannot take advantage of my M-Audio Delta 66 card. BOO HOO! :(

I've detailed this sound driver problem in this post:
/2007/06/vmware-player-no-sound-bad-directsound.html

I hope this is a nice primer for those wishing to undertake the process of converting a physical machine to a virtual one.

Here's a related article on VM performance

Good luck!

testing the Sun X4600M2 and ESX Server 3.0, part I

Outside of a few obstacles, we had a useful and interesting session testing a Sun X4600M2. The plan was to use three virtual machines on ESX Server 3.0 to simulate our eCommerce infrastructure:
- one Win2K3, IIS 6.0 web server running our website application
- one RHEL 3.0 AS running Oracle 10G
- one Win2K server running MS Web Stress Application Tool (MS WSAT) to generate HTTP traffic load against the Win2K3 web server

The X4600M2 we tested was an eight, dual-core 2.4Ghz Opteron RevE cpus connected to a Sun 5310 fiber storage array. The 4600 ran VMware ESX 3.0 server on top of a customized version of Linux built for VMware. I provided the vendor with the three preconfigured virtual machines. The vms were zipped on dual layer DVDs and took a while to copy and unzip, roughly an hour each. Also, the virtual machines were built on an Intel box and as such, needed to be converted specifically to the AMD Opteron architecture of the ESX server (the 4600). This was news to us and took about twenty minutes to convert the 8GB Windows vms and about an hour to convert the 33GB database file.

We started all three vms, did some Windows configuration and verified connectivity between the servers. TNSnames and an ODBC driver needed setup on the web server. The first large hurdle we encountered was that unlike our test system, the RHEL3 vm was not able to find its IP address via DHCP. After trying a few things, we assigned the address statically and the server then became available on the network. Once all three boxes were talking, we then verified that the website could pull data from the database. We did this; however, we saw that the database sequences were not created when we added an item to our cart. I got on the phone with our programmers and after about 45 minutes, resolved the problem using a public synonym. After this problem was solved, we spent a half an hour using the WSAT's recorder function to navigate the website and create the test cases. We were then able to start testing.

As our vendor did not have an Enterprise license for the ESX Server installation, we were limited to assigning up to four cpus per vm. So we assigned each vm the maximum available:
- Oracle vm: four cpus
- IIS vm: four cpus
- MS WSAT vm: four cpus

Since one CPU on the 4600 is dedicated to VMware overhead, this left three CPUs unused in the 4600.

We used MS WSAT to apply load to the Web server instance, slowly increasing load from one session to ten to one hundred virtual users in order to verify that:
1) the stress tool was working correctly,
2) the website was responding appropriately, and
3) we could see data via the VMware Virtual Infrastructure Client management app

We verified that these conditions were met.

It was interesting to view the VMware instrumentation. The VMware Infrastructure management app is a lot like Performance Monitor in Windows. You can view CPU/disk/memory and network stats. We toggled between the three vms and checked out performance stats for each. The most stressed vm was the IIS webserver, as it was serving data to the testing client (the Win2K server running MS WSAT), as well as pulling content from the database.

One interesting metric we saw in the management interface was called Megahertz Used, which is basically the percent of the total megahertz available to a vm. For example, if a vm has one 2.4Ghz cpu dedicated to it and that cpu is 10% busy, you're using 240Mhz of the available CPU power. On our Win2K3 web server vm, we had four cpus available at 2.4Ghz each. This gave us a total of about 10,000 megahertz available to the vm. When we increased the load to the Win2K3 web server, we saw that the webserver was using about 80-90% total CPU available or about 8,800Mhz of CPU. This load was more or less equally divided by the four CPUs assigned to the VM:
cpu0: 2300 mhz used
cpu1: 2200 mhz used
cpu2: 2200 mhz used
cpu3: 2100 mhz used

Utilizing the megahertz available to a vm, VMware is able to balances load to cpus within a vm as well as balance load between vms. ESX server 3.0 can dynamically provision new vms by analyzing this statistic.

Another interesting thing we did was to clone our testing server, the Win2K server with MS WSAT installed on it. As the clone is essentially a file copy, the process is i/o intensive and took about 10 minutes for the 8GB vm. With a configuration tweak and a quick start of the server, the cloned testing server was up and applying load against the website in 15 minutes total from start of clone to finish. Nice!

While testing, we found that the MS stress tool applies load, but has a nagging inability to capture enough information about a users' session so that an order can be completed through the test website. Also, the stress tool seems to quiesce after about 7-10 minutes. This may have been due to some caching on the database and web server layers, but is more likely due to a limitation with MS WSAT. So we are looking to replace this testing tool with one that doesn't have these limitations and can do interesting things like parameterize order and sku numbers in the requested URL. Compuware QALoad is a top candidate and one we're already licensed to have. We are currently researching tools for round 2 and hopefully, we'll have a substitute in the next couple of weeks.

In order to get a more full day of testing on the 4600, we will schedule a second visit to our vendor, with the caveat that I will bring a fully configured database, unzipped on an external USB drive in order to expedite the setup. Also, I hope to persuade the vendor to get an Enterprise license for ESX Server, so that we can assign more than four CPUs to an individual vm. Finally, at the end of the day, I will try to provide some screen shots or scripts of the evaluation session for the blog.

More to come..keep you posted!

Sunday, June 03, 2007

beginner's guide to LVM

After about three weeks of working with it, I have my Fedora Core 6 system very well configured now. There are about 122 software updates waiting in the wings for Core 6, but I haven't wanted to do any updates to it in case the updates break something. So I want to do a backup of the system first. Unfortunately, by default, the standard Core 6 install creates logical volumes for /root and swap instead of good old ext2 or ext3. Argh. So I've had to dust off my poor LVM skills.

A fanastic beginner's guide for LVM is this article by Falko Timme:
http://www.howtoforge.com/linux_lvm

Note that if you do download the Debian Etch virtual machine for this tutorial, you will have to change the default keyboard layout. I believe Falko is German, so the keyboard was very strange. Hunting and pecking, I did find my way to the dash, which had moved to where the ? and / is. So to change your keyboard to your country of origin and layout, you'll need to run through the following command in an SSH window or at the prompt of the virtual machine:
dpkg-reconfigure console-data

Next, I'm going to investigate Falko's second article on how to backup and restore LVMs here:
http://www.howtoforge.com/linux_lvm_snapshots

cheers.

VNC for OS X : Chicken of the VNC

A nice utility I've found to logon to my Fedora box from my MacBook Pro is Chicken of the VNC. Install is simple via DMG file. Here's what the config screen looks like:


Pretty easy! Here are some simple instructions if the graphic doesn't do it for you:
http://www.cs.vassar.edu/SysNews/vnc/osx.html

One tricky part is Full Screen toggle. This is done via CTRL-COMMAND-OPTION-~ (the tilde key).

Have fun!
'sodo
Feel free to drop me a line or ask me a question.