performance problems with HD viewing in Media Center resolved

This is a follow up to my first three posts:
Vista View Saber DA-1N1-I tuner and Media Center
Vista 32-bit, First Impressions
Making Sense of Home Theatre PCs

Background
I had quite a battle over the past couple of days. The goal was to get a working Vista Media Center that did not choke while displaying live, HDTV content. First, I have to say that Microsoft has done a kick ass job with Media Center. It has taken a number of years, but the Media Center interface is really slick and easy to use, the content and functionality deep and the possibilities pretty open, given the way they've designed it. You can even download an SDK for it, which is cool.

In practice, however, getting Media Center to work on older equipment is challenging and frustrating for the newcomer. The interesting thing is the base system I started with ("started with"..yuk, yuk) was no slouch:

• Dell 400SC, 3.2Ghz
• 2GB PC3200 mem
• 128MB ATI All-in-Wonder 9800 Pro AGP 4x card
• 120GB 7200RPM IDE system drive

The Problem
The main problem was that when I viewed live HDTV channels, I would get drop outs and compression artifacts.

As you know, I had already completed the base Vista install, installed aVista View Saber DA-1N1-I tuner card and layered the latest and greatest Media Center with TVPack update on top of it.

There were a couple links to the Media Center TV Pack Unofficial Download on Engadget's site here. I've plucked out a couple of links to download TV Pack 2008 here, but I'm not sure how long they will be valid:
http://rapidshare.com/files/136020246/MediaCenterTVPack2008-KB953272-x64.zip.html
http://www.avsforum.com/avs-vb/showthread.php?s=b523af4d82ccee06f79c49badff6d2a9&p=14451290#post14451290
http://www.mediafire.com/?sharekey=30d15c758a398fcfd2db6fb9a8902bda
http://rapidshare.com/files/138029257/Media_Center_TV_Pack_2008.rar.html

The neat thing about TVPack with the Vista View tuner is that, among many other things it did, was resolve tricky problems with displaying the correct listings in the TV Guide if you have multiple channel sources (ie, cable and antenna signal inputs), as well as add support for clear QAM (quadrature amplitude modulation) channels. QAM channels are the HD channels that a TV provider will send "in the clear" over the cable line. In my case, Media Center sees about 14 clear QAM channels from Comcast.

Tuner Setup
I have a three way splitter in my upstairs loft that splits the cable signal three ways: one to the analog, NTSC connection on the TV tuner card, one to the digital, ATSC connection on the tuner card and one to the cable modem itself. Because I was getting drops, I thought that I was just getting a weak signal. I removed the splitter and plugged the feed directly into the digital connection on the tuner card. No luck, as the signal still broke up.

Initial Performance Stats
Taking a look at the new Reliability and Performance Monitor in Vista (quite nice, really) while an HDTV show was being displayed in MC, I saw that CPU was being driven close to 100% on my hyperthreaded system. Peaking on CPU is never good. I thought the problem could be my video card not offloading enough load from the CPU. Maybe the ATI card was unpowered to the task? I removed it and installed an NVidia 7600GS that I had laying around. Symptoms were the same with the new card.

So far, the evidence showed that the problem was not in the source signal or video card since:
1) the input source signal was clean
2) a capable video card was choking in the same way as an older one

Reviewing Performance Monitor, the disk seemed to be quite busy. I assumed this was due to the live tv buffer that MediaCenter keeps while you watch programs. To explain, MediaCenter has the same ability as a regular consumer DVR, in that it buffers live tv in order to allow the viewer to playback the last few minutes of a program. This rewind feature is supported by the system writing huge files to disk.

I should have pulled up some disk queue stats to confirm, but I had a gut feeling that the problem was in the disk subsystem. Looking at the disk reads and writes in Performance Monitor, I saw that about 4MB of data were being written to and read from the hard drive every second while an HD program was being shown. My current hard drive was a 120GB IDE. IDE throughput is pretty weak by today's standards. Hmmm..this old drive has to go. So, I was off to the nearest store to get a hard drive.

Throwing Money At It Will Solves the Problem!
*** BEGIN DIVERSION ***
In a pinch, Staples is my closest source of computer parts. Of course, it is not the best choice for prices. Oh well, I was on a mission. Seagate 500GB SATA..$130. Eeesh. That is a little too much. I patiently waited for the computer department rep to finish his conversation with another customer. I'm glad I did. I overheard that Staples supported price matching. I scrambled to find a machine with an open internet connection. Finding one, I Googled "site:bestbuy.com seagate 500GB SATA". Lo and behold, they had the very same drive for $90! Great! At the register, the clerk had to confirm the price. I noticed his point of sale system had web browsing directly built in. But he confirmed the price..$40 saved!
*** END DIVERSION ***

I ran home and installed the drive. Reading through the installation manual, I saw that Seagate has Disk Wizard, which allows you to format your new drive. It also had an option I hadn't seen before..to clone an older drives' contents onto a newer one. Psych! I was expecting to have to reinstall Vista, but if this Acronis clone software actually worked, I wouldn't have to go through that pain.

New Drive Fun
I installed the drive in the PC and got Disk Wizard running. Selecting the appropriate clone options, clone was to copy the old drives' content, sets the master boot record and reboot. Hitting "go", I saw that the clone ran very quickly, almost too quick, and then asked me to reboot. I figured the process must do the cloning on reboot, but when the box rebooted, the OS did not boot. Oh sh1t. I hope it hadn't blown away my Vista install. Googling for people with similar problems, I learned from someone else's issue that clone changes the master boot record, so my system trying to read off the new disk. Since the clone process didn't go as expected, there was no operating system for the master boot record to access. Luckily, clone did not touch the original system disk, so after unplugging the new one, I was able to verify my base Vista system was still alive.

Starting Disk Wizard again, I saw it had a bootable install version of the clone process. Since the initial Vista install process did not work, I thought I'd give this a shot. I burned an ISO to a CD and started up the clone process. This time, I saw progress on the clone procedure meter. It took about 17 minutes to clone 24GB of data from a 120GB IDE to my new 500GB Seagate. After the clone process was complete, the box rebooted. While the box rebooted, I made sure to shut down the box entirely and take the original system disk offline. After doing this, I restarted the system and was overjoyed to see Vista start booting. New drive works with my old install of Media Center! Yay!

Did the New Drive Help?
Moment of truth. I pulled up Media Center with some live HDTV. I still had dropouts even with the new, faster drive. Damn! OK. Since you can't turn off Live TV easily in MediaCenter (why would I want to, with Olympic Volleyball coming up?), I decided to meet the burden head on by using my old IDE RAID card and setup a stripe set of old hard drives. I had two 120GB lying around (the one old system disk plus another), so I installed the card, setup the RAID set in the BIOS and formatted the new RAID set when it came up in Vista. Voila! Lots of fast disk capacity on hand! Now to repoint the destination directory for the live hd temporary files to the RAID set.

Move those Temp Files!
In Media Center, I saw that there was no easy way to turn off the Live TV feature. So I figured the location of the temp file directory was in the registry. Reviewing the Disk statistics of Reliability and Performance Monitor more closely, PerfMon actually shows the location of the directory where disk access (reads and writes) are occurring. Sweet! Looking at the details of the offending process, I saw that this directory was the culprit of most of the reads and writes:
C:\Users\Public\Recorded TV\TempRec\TempSBE

Cool. I searched on this string in regedit, found the appropriate location:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Media Center\Service\Video\Tuners\DVR\IO\Writer

Then, I tweaked the values to match a new directory that I setup on the D:. I restarted Media Center for the change to take effect and waited patiently for live HD to come up. Watching it, I saw that HD was definetly smoother, but not perfect. It still hiccupped from time to time. DAMMIT! Looking at the top CPU consumers in Task Manager, I saw that dwm.exe (Desktop Window Manager) kept popping up, albeit with low CPU consumptions of 5-7%. More performance tuning necessary.

Optimizing Media Center Performance
The key to getting Media Center working right is optimizing performance in Vista. Here are a few articles that helped me do that, in order of most useful:
ExtremeTech article
Andrew Grant's blog
APCmag.com article
Jwyse's blog

Also, the following article has some registry tweaks to get your Media Center setup just the way you like it:
MediaCenter.com.au article

One feature that I had kept running in Vista was the Aero interface with a sidebar of gadgets (clock, RSS news feed and rotating pictures). Apparently, the process dwm.exe controls the display of these visual effects and gadgets. In Control Panel -> Personalization -> Window Colors and Appearance, I disabled the Aero interface by selecting "Windows Standard" as the color scheme. After the change, I checked live HD. There were less drops, but still an occasional one every fifty seconds or so. Damn! On a hunch, I disabled the RSS news feed that was in the sidebar under the clock. Restarting Media Center, I watched for any live HD dropouts. One minute went by..no problem. Two minutes..no problem. Three minutes..no drops. Awesome! I think I've got the problem licked!

Right now, I've had live HDTV (the Olympics) running for four hours. There seems to be a barely noticeable lag sometimes when replaying the live HDTV buffer. I believe this is due, again, to disk access. I may add a third 7200RPM 80GB drive to my stripe set to improve performance. I will let you know how that goes.

Conclusions
In general, if you are skirting the edge of recommended hardware for HD viewing, Media Center seems VERY sensitive to any processes taking up CPU or mouse movements. Disabling any CPU performance hogs is essential in order to have live, HDTV in Vista Media Center run properly on an older system. If I want a perfectly clear HD viewing experience with no dropouts, I will not open ANY programs. Also, I don't have any programs running that update the display, like Task Manager or Performance monitor. In fact, I won't even touch the mouse if I want undisturbed video. Media Center a bit touchy, eh?

I assume if I had a new dual or quad core box with a fast CPU, the latest memory, a 1333Mhz front side bus, PCI Express video card and all SATA, I wouldn't have these problems. But I didn't feeling like paying an arm and a leg for a new box. With my trusty old Dell 400SC, I may be able to get by adding a $100 tuner and $100 hard drive only. Time will tell!

In sum, once Media Center is running well, it IS pretty sweet. Now to find a usable remote for VMC. Any suggestions?

:)

Vista View Saber DA-1N1-I tuner and Media Center

This is a follow up to my first two posts:
Vista 32-bit, First Impressions
Making Sense of Home Theatre PCs

Last night, I spent a few hours experimenting with Windows Media Center on Vista and the new TV Pack, with the goal of getting digital cable from Comcast integrated into Media Center. As suggested from the article linked above, I reinstalled Vista (painfully slow). I then installed the TV Pack and associated hotfixes. That day, I had bought a Vista View Saber DA-1N1-I tuner card. It is PCI, to fit in my older Dell 400SC. I let Vista select the driver from the internet and I saw that it found the latest driver.

Setup in WMC was straightforward, similar to Ben's experience. I connected the card's NTSC connection to my comcast line. After the requisite asking of where Ilived, downloading of my area's TV guide and inspecting my tuners' connection, WMC got me to the point where I could select my area's cable provider. Since I live in NY Metro, there were about ten choices. I selected Comcast in Jersey City. So sure enough, channels 1-99 on Comcast appeared.

I was happy to see I had basic analog connectivity, but wondered where the digital channels went. At the time, I didn't realize that this was regular analog cable from Comcast. I went through the discovery process again, and this time I selected Comcast Digital as my provider. Well, this didn't work, because the cable was plugged into the analog, NTSC connection. In my haste, I hadn't realized that the card I bought is only good for ATSC, over the air HD and NTSC analog cable. It won't get me what I wanted, which was the ability to watch and record unencrypted, clear QAM channels via my Comcast connection. Duh.

Taking my idiocy in stride, I moved on to the ATSC output. When I connected my cable connection to ATSC, as expected, WMC didn't find any channels. I thought I might be able to use the cable coming from the back of the PC as an antenna, so I disconnected the opposite end of the cable from the PC (the splitter end), reran the automatic setup and I was nicely surprised to see that the WMC picked up the local over the air HD channel selection, about 14 channels in my case, directly across the river from Manhattan. Performance was a bit choppy, as my signal wasn't very good in my computer closet. I'm sure with a proper antenna, the signal would be great.

In both cases, analog cable and over the air, WMC picked the correct guide. The guide picked up all the shows, was really nice to navigate through and had lots of supplemental information. Very slick.

Today, I went back to the drawing board and picked up a Hauppauge WinTV HVR-1600 . I'll let you know how this install goes tomorrow.

Long story short, though, if you are looking for a card that works for NTSC and ATSC over the air signals out of the box with this latest update for WMC, the Vista View Saber seems to fit the bill.

UPDATE 8/14/2008
Last Tuesday, VistaView released a QAM patch (a simple registry tweak) for MediaCenter with TVpack to allow the Saber DA-1N1-I to allow you to tune QAM channels on ATSC cable TV: http://www.vistaview.tv/index.php?option=com_docman&task=cat_view&gid=31&Itemid=46

I downloaded the patch and rebooted. I split my cable line to go to both the analog NTSC and digital ATSC jacks on the back of the Vista View card. The positive: during the tuner setup, Media Center now sees both the analog tuner and a new digital cable tuner:

ViXS PureTV Analog TV Tuner
ViXS PureTV ATDC/DVBC Tuner

Continuing through the tuner setup, I can select my provider's (Comcast) channel selection and guide. Checking the source of the channels under Guide -> Edit Channels, I get a mix of the OTA HD channels (about 14 channels) on the Digital Cable connection, but the rest (70 or so) are coming through the analog tuner. The digital channels seem to break up a lot. I imagine using an HD antenna instead of the digital cable would help here, since the digital cable is only getting me those 14 channels. If I hook up only one cable to the ATSC input, I only get those 14 channels. I have had no luck seeing any channels I would normally see with the Comcast Motorola box above 99.

From the Guide -> Edit channels, I see that most of the available channels that are disabled have padlocks next to them and that the source of them is the digital tuner. I imagine the padlocks mean that they are encrypted.

Tonight allowed me to get a bit farther, but am perplexed as to why I'm not seeing any channels above 99. I am a bit of a newbie to this MediaCenter/cable integration stuff. It seems as if Comcast locks everything above 99..not sure right now. Looking at the clear qam guide for my zip code here https://www.silicondust.com/hdhomerun/lineupui?Cmd=LocationProgramsWeb&Country=US&Postcode=07302, I notice that the higher range (above 99) channels that I enjoy watching like Military/VH1 classic/Encore/etc are missing from the list. I guess they are not in clear QAM. Maybe I'm just out of luck?

So, a bit of progress, a bit of disappointment.

UPDATE 8/15/2008
One follow up: I'm a bit disappointed in the performance of HDTV in Media Center w/TV Pack. I've got a Dell SC400 (probably four 1/2 years old) P4, 3.2Ghz, 2GB w/120GB IDE drive and ATI all-in-wonder 9800 pro. Not the latest and greatest, sure, but no slouch box either. Viewing standard def, Media Center drives the box to run at 50% CPU without a problem. Running live HDTV (1280x720), MC runs the box to 100%, at which point I get drop outs. Also, I notice SearchIndexer runs constantly, thus sucking up more CPU.

Granted, the system is a few years old, but this is unacceptable. I will look for ways to tune or upgrade the system to see if I can relieve the 100% spiking.

Andrew Grant has an excellent post on optimizing Windows Media Center to improve performance. I will try his suggestions out tonight. A more technical article on the same subject is on APCmag.com.

UPDATE 8/16/2008
Today, I wasted about an hour tracking this high CPU problem down. I accidentally scheduled a recording and wondered why Media Center Receiver service was taking up so much CPU. This was because it was recording a show and I didn't know it! I went into my Recordings and deleted the scheduled recording.

Reference
Getting Started with Windows Media Center in Vista
Getting Started with HDTV in Media Center
Great intro to channel tweaking in Media Center
Tweaking Media Center's HDTV Lineup
Media Center Receiver Server Technical Discussion

'sodo

Vista Ultimate 32-bit first impressions

This is a follow up to my first post
Making Sense of Home Theatre PCs

Last night, I experimented with Vista Ultimate, 32-bit for the first time. My intention was to use Windows Media Center to access all my digital media and to feed Media Center my Comcast HDTV connection. As I was going to install Vista on a 1.8Ghz Pentium IV with 1.5GB of memory, I knew the system was underpowered for Vista and that the install didn't have much of a chance of running smoothly.

I was correct for a number of reasons:
1) First, the Vista install did not recognized my Sil0680 IDE RAID card.
This was a bummer, as I thought I could make up for the lack of CPU horsepower by running a stripe set. During the install, Vista is smart enough to allow you to stop and look for drivers. It also allows you to search for drivers on a USB flash drive, so I found Sil 680 drivers from here:
http://www.driversdown.com/drivers/Silicon-Image-SIL-0680-0680A-IDE-RAID-Driver_75442.shtml

and then copied them to the USB. However, when I loaded them on the box, Vista did not recognize the drivers as the correct drivers for the Sil RAID card. This is doubly disappointing as the Ultimate install DVD I had was SP1. Argh.

So, I had to back off and install the base OS to a regular IDE drive that I had floating around. That was problematic too, as the primary partition was not marked as active. So, I had to boot up with a floppy, mark it as active and give it a base format. Surprisingly, Vista still did not recognize the partition as usable. So, just to see if an older XP install would recognize the partition, I booted with XP. Luckily, XP did see the partition. In fact, though fdisk reported only one partition on the drive, the XP install disk saw two partitions. Strange. Using the XP installer, I deleted both partitions, created a new one and reformatted as NTFS. This seemed to do the trick and the XP install continued and installed XP.

After the XP install finished correctly, I reinserted the Vista install DVD and gave the install another shot. Because I had fixed the drive partition problems with XP, Vista now saw the partition correctly and I was able to install the OS successfully.

2) The install process took interminably long and did not give you adequate feedback about where it was in the process and when that part of the process was going to be finished.

3) ATI's All-in-Wonder Radeon 8500DV does not work with Vista.
This was also a great bummer, as I wanted to try out some of the TV functionality with Windows Media Center. Reading the install README from the Catalyst download section on ATIs website:
http://ati.amd.com/support/drivers/vista32/common-vista32.html

it seems that older card is not supported under Vista. I was hoping to take advantage of my older equipment. With Vista, again, this was not to be.

4) Vista is a resource hog.
Once I got Vista running, the OS did seem to run fairly smoothly on this low-end equipment. However, Vista is heavy:
- base install requires about 10GB with the default install selections
- memory usage is about 540MB with nothing running

Conclusions
So, Vista is a hog. And it is designed for the latest and greatest equipment. I would not recommend the operating system for users who want to take advantage of older equipment they have lying around. Vista just won't run it. This might be Microsoft's way of weaning old equipment off the supported hardware list. Which is here if you need it:
Vista Hardware Compatibility List

As I experiment more with Vista, I will let you know my impressions of the OS. Right now, I'm a bit saddened that Microsoft has kicked the old folks (READ: old equipment) out the door in favor for the new.

Aren't we supposed to care for our elders?
Cacasodo

making sense of media center/home theatre PCs

Impetus
I bought an Apple iPod Touch 32 GB over the weekend (a post is coming) and it has led me down some interesting paths. It is wonderful to be able to scroll through a good chunk of my music, video and photos easily, all in one little handheld device.

Even more impressive than its storage and playback capabilities are all the applications and connectivity options available via the App Store. The apps will really lengthen the life of both the iTouch and iPhone for years down the road.

iPod Touch Extensibility
One app that I particularly liked was Remote, which allows one to control iTunes from the iTouch/iPhone. Plus, its free!
http://www.apple.com/itunes/remote/

While surfing for applications, I also came upon the TouchPad Pro. The handy little application can be used to control a Windows Vista Media Center PC with the iPod Touch!


Unfortunately, as of this writing (08/05/2008), the TouchPad Pro is not available for the iTouch's 2.0 firmware:
http://www.touchpadpro.com/2008/04/touchpad-pro-faq.html

Finally, Mocha VNC Lite is a free app that connects you via to any VNC Server (Mac/PC/Unix) in your environment:
http://www.mochasoft.dk/iphone_vnc.htm

Sweet!

Home Theatre PC (HTPC) Possibilities
In playing around with the iTouch, I started thinking about the possibilities of having the iTouch become my remote for not just my media, but for my HDTV and Home Theatre, too. This got me very excited, as I would love to have all my music, video, recorded HD, podcasts, everything integrated in one solution, with a super easy-to-use remote via the TouchPad Pro. This is the Holy Grail of the so-called "Home Theatre PC!"

From my reading, I seem to have three choices:

• AppleTV
  
• Linux MCE (MythTV)
  
• Windows Vista Media Center
  

Here's one gentleman's comparison of Linux and Windows MCEs. Though biased, it is interesting:


Requirements
Here's what I'm thinking in terms of requirements:
-should be relatively simple to setup: oops, this rules out MythTV
-needs to play cable TV (both std/high def): oops, this rules out Apple TV
-needs to play most common media formats MP3s/MPEGs/DIVX/Quicktimes/DVDs/Blu-ray
-needs to integrate iTunes*

* Yes, you may be saying that Windows Media Center does not integrate iTunes and you'd be correct. However, thinking about how this would work, it should be a relatively minor step to switch applications from Media Center to iTunes via whatever controller or remote you'd choose to use. Also, further investigation showed me that solutions from both sides of the fence (Linux/Windows) seem to have some basic level of iTunes integration or at least being able to access the media files at the system level. More on this later.

From this short list of requirements and some reading that I've done over the past couple of days, I'd say the most likely contender is Media Center running on Windows Vista. I'm no huge Microsoft fan, but if a software's features and functionality fit the bill, then you should use it.

Here is a nice walkthrough to Windows Media Center features:


How to Connect Your TV to a Windows Media Center PC:


Finally, here's five things you didn't know you could do with Windows Media Center:


The Problem
One of the stickiest wickets in that requirements list is how best to integrate your cable SD/HDTV viewing and recording. It would also be nice if this was simple to setup. From my reading so far, it doesn't look like any PC-based home theatre/media centers are easy to setup by any means. At least computer companies are starting to address this by using a PCI card that integrates with your cable TV providers service, also known as a digital cable tuner card. A CableCard is a PCMCIA like card built into new-generation televisions and PCs that allows digital cable reception without a set-top cable box. Here is a nice primer on CableCards:


Here is some basic information about cable TV tuners via HP forum:
http://forums12.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1218736765934+28353475&threadId=1137182

This was a very informative read about the regulatory background of the CableCard:
http://msmvps.com/blogs/chrisl/archive/2007/12/07/1385290.aspx

And Chris also has a good FAQ on the CableCard. Be aware that in its current form, CableCard does not allow you to use On Demand or Pay Per View services. Apparently, this is because those features require two-way communication that is not present in today's CableCard solution. This is unfortunate, because it means that today, in August of 2008, we still do not have a complete Home Theatre solution. (Well, I guess I can live without ONE feature).
:)

Possible Solution: Dell XPS 420
From this article, I found that the Dell XPS 420 might be a solution that integrates a CableCard and might fit the bill.

Here's Dell's full specs:
http://www.dell.com/content/products/productdetails.aspx/xpsdt_420?c=us&cs=19&l=en&s=dhs

An excellent FAQ on the XPS 420:
Chris Lanier's blog

As a cable card is only distributed to consumers if they purchase an entire new system, the unfortunate thing is that the CableCard solution locks individuals into buying a preassembled machine versus building their own. But given some evidence from intrepid individuals like Keith Combs, this might not be all bad. Here is his very informative blog entry:
http://blogs.technet.com/keithcombs/archive/2007/12/15/dell-xps-420-first-impressions.aspx

Also, his six month checkup is a nice follow through:
http://blogs.technet.com/keithcombs/archive/2008/05/08/dell-xps-420-six-month-report.aspx

Another solution: HP m9300t
Another solution is from HP, the HP m9300t Series

Here is a page on HPs site devoted to more general information about the HP m9200 line of products.

Decisions, decisions
Before purchasing any solution, I plan on building out a Vista box, just to see how far I can get, without the cable tv tuner part.

Let you know how I do in a follow-up post,
Cacasodo

Update 8/9/2008
Vista Ultimate install and first impressions

Update 8/8/2008
Digital Tuners
I happen to be perusing blog articles about people who have built their own media centers and I see this Hauppauge digital tuner card has come up a number of times. The FAQ page has some good info on it.
http://www.hauppauge.com/pages/products/data_hvr1800.html
For more basic information on what TV Tuners can and cannot do, take a look at this post from AVS forum/Overclockers.net:
http://www.overclock.net/htpc/294427-hauppauge-wintv-hvr-1600-a.html

Update 8/15/2008
Vista View Saber card, Media Center and TV Pack install
http://www.techanswerguy.com/2008/08/vista-view-saber-da-1n1-i-tuner-and.html

Update 9/15/2008
Viewing YouTube Within VMC
http://www.techanswerguy.com/2008/09/install-and-review-of-yougle-for-vmc.html

best windows shortcut chart

This chart showing Windows keyboard shortcuts is so good it merits its own post:
http://www.scribd.com
Awesome list of windows shortcuts!
'sodo

Verisign Class 1 CA Individual Subscriber cert expires

Yesterday, 5/14/2008, we were rudely awakened to errors in our SSL checkout process. Basically, the dialog box that informs a person browsing the website that a problem has occurred showed up. Investigating our IIS 5.0 web servers, we found that none of the SSL certificates had expired. Digging deeper, we then looked at the certificate store on the machine. In the certificate store, we found that the Intermediate Certificate called Verisign Class 1 CA Individual Subscriber certificate had expired. Yuck!


For Windows 2000
We called Verisign and confirmed that the expired certificate was affecting our site. Verisign was able to give us a link to download the latest certificate:
https://knowledge.verisign.com/support/digital-id-support/index?page=content&id=SO6052



We followed the install and verfication instructions to install the cert on each server in the farm. After making those changes, I verified in Firefox that the SSL dialog box no longer appeared. Thank God!

For Windows 2003
Win2K3 was slightly different. We actually needed to make the certificate appear in our Intermediate Certificate list:


This was accomplished by downloading the latest intermediate cert that works with your chosen level of SSL certification from here:
https://knowledge.verisign.com/support/ssl-certificates-support/index?page=content&id=AR657

For us, we use Non-EV Premium SSL certificates. So we used that link from the main page. Once we installed them by following the directions on the linked page,
http://www.verisign.com/support/verisign-intermediate-ca/secure-site-pro-intermediate/index.html, we saw the intermediate cert appear (date of 10/24/2011) in the proper certificate store:


Finally, a reset of IIS was needed. Yuk!

Afterword
The one thing I do not have clarity on is exactly what certificates does a typical web server running ASP and ASP.net need to run correctly. I know that there is a hierarchy of certs:
1) Root certificates
2) Root cert signs Intermediate
3) Intermediate signs Personal (www.mysite.com)

We had a similar problem back a couple of years ago and I had been too busy to research the question then. I will try to research it and give an update to this page.

Good luck, folks!
'sodo

Ubuntu 7.10 install in VMware Server gotcha

Trying to install Ubuntu 7.10 in VMware Server 1.04, I came upon a lot of these messages as Ubuntu was trying to install itself into the virtual machine's hard drive:
sd 2:0:15:6 rejecting I/O to offline device

Background
A caveat to installing most Linux distributions within VMware Server for XP is that you need to set the SCSI driver to Lsilogic in the configuration file for the virtual machine. The config file will be a file with the extension ".vmx" in the VM's directory. For example, the config file for my Ubuntu install was "Other.vmx"

Solution
Here is the line you need to add to your .vmx file:
scsi0.virtualDev = "lsilogic"

My entire .vmx file ended up looking like this:
config.version = "8"
virtualHW.version = "4"
scsi0.present = "TRUE"
scsi0.virtualDev = "lsilogic"
memsize = "1024"
ide0:0.present = "TRUE"
ide0:0.fileName = "Other.vmdk"
ide1:0.present = "TRUE"
ide1:0.fileName = "auto detect"
ide1:0.deviceType = "cdrom-raw"
floppy0.fileName = "A:"
Ethernet0.present = "TRUE"
displayName = "Ubuntu 7.10"
guestOS = "other"
priority.grabbed = "normal"
priority.ungrabbed = "normal"
ide0:0.redo = ""
ethernet0.addressType = "generated"
uuid.location = "56 4d 32 f1 73 c7 b5 15-26 b3 bf 4b eb 77 4f c6"
uuid.bios = "56 4d 32 f1 73 c7 b5 15-26 b3 bf 4b eb 77 4f c6"
ide1:0.autodetect = "TRUE"
ethernet0.generatedAddress = "00:0c:29:77:4f:c6"
ethernet0.generatedAddressOffset = "0"
numvcpus = "2"


Of course, the line you add may differ if you have more than one SCSI device. Once the line is added, you'll need to restart your virtual machine for the change to take effect.

Good luck!
sodo

black screen/no video in VLC

I noticed a problem in VLC on XP the other day. While watching a music video, I starting getting sound but no video. I thought it was some sort of display problem, but then tried QuickTime and found I could watch the video in QuickTime. Just to see if it would help, I logged off and back on. Still giving the same problem. I didn't change any settings, so I thought it was a hardware problem. So I shutdown the box and turned it back. Same problem. Argh.

I then realized that it must be some issue with VLC itself, so I started digging through the Settings menu; specifically, Preferences. Looking in Preferences -> Audio with "Advanced options" checked, I saw that "Audio desynchronization compensation" had reset to -10000000! Woah! How did that happen? I reset this value to 0 (zero) and played back my original video. Voila! The video played perfectly.

I don't know what key combination or bug I may have triggered, but that is a wacky one. Hope this helps someone in similar distress.

'sodo

authenticated FTP through browser

I always forget the syntax for accessing a password protected FTP site through a web browser

If you know the username and password of the FTP site, you should be able to access the FTP site through your web browser. You need to construct a URL with the following syntax:
ftp://username:password@ftp.ftpsite.com/

For example:
ftp://cacasodo:mypassword@ftp.techanswerguy.com/

Make sure if you are accessing via a proxy server, that FTP is configured in your proxy settings.

'sodo

recording video off Motorola 6412

I had been wanting to copy some unencrypted shows off of my Comcast cable box, a Motorola DCT-6412, for quite some time. By "unencrypted", I mean your basic cable. This does not include premium channels like HBO, SHO or MAX. Also, you won't be able to grab pay-per-view content.

I finally got around to trying this out last week by using my MacBookPro and a standard 6-pin male-to-male firewire cable. GXW on http://www.macosxhints.com/ has provided some excellent instructions for doing this.

Requirements
OS X10.4.9 or above
Apple's FireWire SDK (downloadable from Apple's Developer website)
VLC (Video Lan player for Mac) or MPEG Streamclip for playback

Steps
1) connect Mac to Motorola
2) verify physical connectivity via System Profiler
3) connect and verify firewire connection with AVC Browser
4) replay show and grab stream with Virtual DVHS

You'll need Apple's firewire SDK in order to do this. Create a free Apple ADC Online account from here:
http://developer.apple.com/products/

Evaluation
Once I got the SDK installed and verified my connectivity as above (see GXW's fine article for the details), I was able to successfully pull content off of my Motorola 6412 HD cable tuner. However, the way it works is that you have to capture live content as it is playing or replay a stored recording in order to capture it to your Mac. In other words, this method does not allow you to access the hard drive in the STB to get at the stored recordings directly. Therefore, it will take some time if you have a bunch of shows stored in order to replay and capture them to a file.

Hint: Don't fast-forward or reverse while recording a saved program to your Mac's hard drive. This will screw up the audio and video synchronization of the destination file.

I was able to pull both regular (SD) format and HD format content off the Motorola. HD shows are recorded to MPEG-TS (transport stream) format. Also, I just realized I didn't try recording content off of the analog channels, I only recorded shows off of the digital channels. So I can't speak to that.

Once I get a workflow in place, it will be nice to have my favorite movies and shows as files to stream from a media server. Yeehoo!

enjoy!
TAG

understanding CSS using Firebug

Understanding CSS is not trivial stuff. But the Firebug plugin (http://www.getfirebug.com) for Firefox can help. Here's a video I produced that shows how to inspect your CSS with Firebug. The context is understanding Blogger templates:


Also depicted in the video are cross-hairs from the Orca screen magnifier, an application for disabled people that is installed with the Fedora 7 distribution. I used the cross-hairs because the mouse cursor disappears when I capture the video with Cinelerra.

cheers,
techanswerguy

command line email in Cygwin

As mailx is not part of the Cygwin distribution, you don't have that many choices if you want to send an email from a shell script. In this regard, Jeremy Reed has kindly provided a nice substitute script that can help us out:
http://cygwin.com/ml/cygwin/2005-02/msg00635.html

If you have Cygwin installed, you'll need to do the following things:
1) install ssmtp
2) configure ssmtp with ssmtp-config
3) install Jeremy's mailx shell programs

There are a couple of gotchas with step 2 above.
1) make sure you have the directory /etc/ssmtp created
- for some reason, the ssmtp-config program doesn't create it
2) make sure you define a fully qualified hostname
- if not, you'll get the following error
$ /usr/bin/mailx -s "test" cacasododom@gmail.com < test.txt
3) make sure that you have a symbolic link created for /usr/sbin/sendmail that links to /usr/sbin/ssmtp and that /usr/sbin is in your PATH. Otherwise, you'll get this error:
$ mailx.sh -s "test" joe.user@yahoo.com < test.txt

That's it.

I preferred to use Jeremy's program over nail, because it gives me the same arguments as mailx did.

Thanks Jeremy!
TAG

Site Overlay in Analytics is lame

We can only look a gift horse in the mouth so often. So when Google stumbles with a small feature in the free toolsets that they provide us, we shouldn't yell at them.

Case in point: the Site Overlay feature in Google Analytics.

Site Overlay *should* be very cool, giving us a visual representation of the traffic to our websites by overlaying the percentage of traffic directly on the page itself. For my site, it looks something like this:


Notice all those 0%'s? Unfortunately, Site Overlay only works with static pages. It chokes on dynamic content, CSS, javascript links, redirects and almost any recent technology used to power a website. Here is the company line from a rep I emailed regarding what Site Overlay can and cannot do:

Thank you for your email. I understand you are concerned about zero datain your Site Overlay reports. Please note that the site overlay report uses cURL to request a web page, and displays clicks on standard links that are embedded on that page. Its functionality is currently limited to static pages with unique links to content located elsewhere on the website.

The site overlay is not currently able to work with Javascript links, CSS content, Flash navigation, downloadable files, outbound links, frames, or automatic redirects.

In the above cases, it's common to see missing overlay bars, or bars with zero content. If there are multiple links on a page that all point to the same place, Analytics will total the clicks to on all those links and display the total number for each overlay bar on each of those common links.

If you have a filter that changes your Request URI, the Site Overlay report may display 0 in your report.

Oh well. If Google updates this feature to allow for the use of dynamic content, I'll let you know.
Cacasodo

understanding CSS using Firebug

Blogger templates are dense with Cascading Style Sheets and can pretty difficult to understand if you need to edit them. I didn't really understand them until I picked one apart using Firebug (http://www.getfirebug.com), the developer's plugin to Firefox. I have to tell you, Firebug really kicks ass and makes understand the CSS of any HTML page very clear. Enjoy the video!
'sodo

upgrading Cygwin: dos2unix

I recently converted from a very old version of Cygwin Beta 20 (circa 1999) to the latest 1.5.25-7 version of the Cygwin DLL. I noticed when I tried to execute some of my existing scripts that had executed under the old beta version, I now would receive a lot of these errors:
./script.sh: line 40: $'\r': command not found
or
./script.sh: line 5: syntax error near unexpected token `$'do\r''

Looks to me like the command interpreter on the new version of Cygwin is a lot more picky than the old one. Specifically, Cygwin is choking on the carriage returns that Windows XP Notepad inserts at the end of each line of the file. Yuk.

I wondered if this was something configurable with an environment variable (http://www.cygwin.com/cygwin-ug-net/setup-env.html), so I first tried setting the CYGWIN environment variable to nobinmode (http://www.cygwin.com/cygwin-ug-net/using-cygwinenv.html). Unfortunately, this change had no effect. I did find a bit of background here:
http://cygwin.com/faq/faq-nochunks.html#faq.api.cr-lf

I had a bit more success using one of the nice little utilities Cygwin has called "dos2unix". Dos2Unix strips out any carriage return or line feed characters in text files and automatically saves the file. The command is simple:
dos2unix [filename]

where [filename] is the name of the file you wish to convert. Dos2Unix has no man page, so don't bother looking for it.

Unfortunately, I had quite a few scripts in various directories, so I just went through each directory and typed:
dos2unix *.sh

I could have written a script to interate through the directories, but there were only about fifteen, so I got lazy and just ran the command manually. After running the command, I was able to execute the scripts without error.

Nice!
Cacasodo

tag cloud for Blogger blog labels

Phydeaux3 has posted some very easy-to-follow instructions on how to create a tag cloud for labels on your Blogger blog:
http://phy3blog.googlepages.com/Beta-Blogger-Label-Cloud.html

A tag cloud looks like this:


You can see a working implementation of a label cloud here:
http://crazedmuleproductions.blogspot.com

The one thing that Phydeaux doesn't provide is an overview of the steps necessary in order to create the tag cloud. Here is the overview:

1. Back up your blog template
2. Add the label cloud stylesheet to your blog template (before the end skin tag)
   
3. Add javascript variables for the cloud to your template before the end HEAD tag
4. Replace the template code for your existing Label widget with the custom Label Cloud code

Finally, Phydeaux adds some troubleshooting tips as well as some hints for editing the label cloud text sizes and colors:
http://phy3blog.googlepages.com/Beta-Blogger-Label-Cloud.html

Nice job, Phydeaux!!
'sodo

ps - If you're new to Blogger, I've posted a number of videos here that you may find useful:
/2007/10/introduction-to-blogger-video-tutorial.html

cygwin cron on Win2K/XP

Well, setting up Cygwin (http://www.cygwin.com/) cron on Win2K Server and my XP workstation is a complete bitch. Simply put, my scripts would not start properly. I used the latest version of the Cygwin DLL, v.1.5.25-7.

Why?
In a nutshell, my problems came down to three issues:
1) cron would only work as my local domain account
-lots of good information here: http://www.andrewlouie.com/wordpress/?page_id=88
-and a little bit of info here: http://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-switch
2) cron needed to be installed using "smbntsec" privileges for my SSH-based scripts to work
3) finally, c:\cygwin\bin needed to be the last entry in my PATH enviroment variable

Installation Steps Added 3/1/2008
Most of the following information comes from these gentlemens' fine posts:
-http://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-switch
-http://ist.uwaterloo.ca/~kscully/CygwinSSHD_W2K3.html

The overview is that you'll have to do quite a bit to get cron under cygwin running properly. Please refer to the two docs above, but here is the short list of steps:
1) decide on a user that cron will run under
- we have a domain account used for scheduled tasks, so I used that
2) add the user to /etc/passwd
- setup a proper home directory in /etc/passwd
- if you do not have a proper home directory, you'll see "Can't cd to $HOME" errors from the output of cronevents.exe
- copy valid .bashrc/.bash_profile files to your users' home directory
3) under Control Panel -> Administrative Tools -> Local Security Policy -> Local Policies -> User Rights Assignment -> tweak your Win2K/XP Local Security Policy settings to add the chosen user to
- (Adjust Memory Quotas for a Process) Increase Quotas
- Create a Token Object
- Log on As A Service
- Replace Process Level Token
4) create a crontab for your user
- crontab -e
5) verify that the permissions on the command(s) to be cron'd will allow the user you've chosen to execute the command(s)
6) run cron-config to enable the cron daemon
- you may get execute and write errors related to permissions on /var, /var/run, /var/log, /usr/sbin/sendmail, /usr/bin/cronlog. I took the easy route and fixed them with:
chmod 766 [file or directory]
or
chmod 755 [file or directory]

Also, make sure the user that you will be using to execute cron jobs is setup in /etc/passwd.

6) once the cron daemon is running, validate that your script is running properly by looking at cronevents.exe and $HOME/cron.log
- if you have not applied the Local Policy settings correctly, you may see the "can't switch user context" error in the log output
- at one point, I received this error:
C:\cygwin\usr\sbin\cron.exe: *** fatal error - could not load user32, Win32 error 1114

I resolved this by moving my user from the "guest" group to the "Administrators" group in /etc/passwd. Of course, you may not want to do this based on your security needs.

troubleshooting
I learned a few things along the way:
1) after installing the cron package into Cygwin, you want to use cron-config to setup cron
2) after enabling your cron, cronevents.exe is a good thing
3) occasionally, $HOME/cron.log provides some useful info
4) the CYGWIN environment variable can be a nasty beast:
http://www.cygwin.com/cygwin-ug-net/using-cygwinenv.html
5) the HOME environment variable must be set properly:
http://blog.spikesource.com/crontab.htm suggests running "env" in your cron out to a file to find differences between your user environment and the cron environment. Good idea!
6) don't forget your scripting basics:
/2007/07/why-doesnt-my-shell-script-run-under.html
7) documentation can help:
C:\cygwin\usr\share\doc\Cygwin\cron-4.1-6.README

Detail
1) after you install the cron packages from the CygWin setup program, make sure to use "cron-config" to add or remove the cron daemon as an NT service, and perform configuration of cron.

I needed to run the service as myself, with the security tweaks described in both these places:
-http://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-switch
-http://www.andrewlouie.com/wordpress/?page_id=88

Here's a sample of the installer dialog:
$ cron-config
Cron is already installed as a service under account DOMAIN\USER.
Do you want to remove or reinstall it? (yes/no) yes
OK. The cron service was removed.

Do you want to install the cron daemon as a service? (yes/no) yes
Enter the value of CYGWIN for the daemon: [ntsec smbntsec] smbntsec

The service can run either as yourself or under a privileged account.
Running as yourself allows better access to network drives,
but does not allow to run the crontab of other users.
Do you want to the cron daemon to run as yourself? (yes/no) yes

Please enter the password for user 'SODO':
Reenter:
Running cron_diagnose ...
... no problem found.

Do you want to start the cron daemon as a service now? (yes/no) yes
OK. The cron daemon is now running.

In case of problem, examine the log file for cron,
/var/log/cron.log, and the Windows event log (using /usr/bin/cronevents)
for information about the problem cron is having.

Examine also any cron.log file in the HOME directory
(or the file specified in MAILTO) and cron related files in /tmp.

If you cannot fix the problem, then report it to cygwin@cygwin.com.
Please run the script /usr/bin/cronbug and ATTACH its output
(the file cronbug.txt) to your e-mail.

Occasionally, cron-config may bark at you for permissions errors on /etc/group and /etc/passwd. Fix them and then re-run cron-config.

Event Log Errors
If I ran the cron daemon under the SYSTEM user context, I got the following Application Event log error:
The description for Event ID ( 0 ) in Source ( /usr/sbin/cron ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: /usr/sbin/cron: PID 5300: (CRON) error (can't cd to HOME).

After a proper install, I still see a running set of Information Alerts in the Application logs. These don't seem to hurt, as my jobs still runs:
The description for Event ID ( 0 ) in Source ( /usr/sbin/cron ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: /usr/sbin/cron: PID 5016: (SODO) CMD (/scheduledtasks/scripts/load.sh ).

2) Setup your cron using "crontab -e". This puts you into the edit window of the text editor that you installed with Cygwin.

Once a cron is setup, view the output of cronevents.exe to see what cron jobs executed:
$ cronevents.exe tail
2008/01/31 16:53:01 [SODO] /usr/sbin/cron: PID 5992: (SODO) CMD (/scripts/load.sh >> /scripts/load.txt )
2008/01/31 16:53:01 [SODO] /usr/sbin/cron: PID 6652: (SODO) CMD (/scripts/connScript.sh >> /scripts/conn.txt)
2008/01/31 16:54:01 [SODO] /usr/sbin/cron: PID 8132: (SODO) CMD (/scripts/load.sh >> /scripts/load.txt )
2008/01/31 16:54:01 [SODO] /usr/sbin/cron: PID 4708: (SODO) CMD (/scripts/connScript.sh >> /scripts/conn.txt)
2008/01/31 16:55:01 [SODO] /usr/sbin/cron: PID 8164: (SODO) CMD (/scripts/connDiff.sh)
2008/01/31 16:55:01 [SODO] /usr/sbin/cron: PID 3372: (SODO) CMD (/scripts/connAlert.sh )
2008/01/31 16:55:01 [SODO] /usr/sbin/cron: PID 4460: (SODO) CMD (/scripts/load.sh >> /scripts/load.txt )
2008/01/31 16:55:01 [SODO] /usr/sbin/cron: PID 7976: (SODO) CMD (/scripts/connScript.sh >> /scripts/conn.txt)
2008/01/31 16:56:01 [SODO] /usr/sbin/cron: PID 6424: (SODO) CMD (/scripts/load.sh >> /scripts/load.txt )
2008/01/31 16:56:01 [SODO] /usr/sbin/cron: PID 2676: (SODO) CMD (/scripts/connScript.sh >> /scripts/conn.txt)

3) Viewing the cron logs are important! The cron.log is place in $HOME/cron.log:
$ cat /cygdrive/h/cron.logThis file was written by the /usr/bin/cronlog script on 20080131_165003
From: root (Cron Daemon)To: SODOSubject: Cron SODO@SODO02 /scripts/connDiff.sh
X-Cron-Env: ALLUSERSPROFILE=C:\Documents and Settings\All Users
X-Cron-Env: COMMONPROGRAMFILES=C:\Program Files\Common Files
X-Cron-Env: COMPUTERNAME=SODO02X-Cron-Env: COMSPEC=C:\WINDOWS\system32\cmd.exe
X-Cron-Env: CYGWIN=smbntsecX-Cron-Env: NUMBER_OF_PROCESSORS=4
X-Cron-Env: OS=Windows_NTX-Cron-Env: PATH=/cygdrive/c/Program Files/PHP/:/cygdrive/c/oracle9i/bin:/cygdrive/c/oracle9i/jre/1.4.2/bin/client:/cygdrive/c/oracle9i/jre/1.4.2/bin:/cygdrive/c/Program Files/Oracle/jre/1.1.8/bin:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/Program Files/TeaLeaf:/cygdrive/c/Program Files/Python25:/cygdrive/c/Program Files/QuickTime/QTSystem/:"C:/Program Files/Symantec/Norton Ghost 2003/":/cygdrive/c/Program Files/Common Files/Adaptec Shared/System:/usr/bin:/cygdrive/c/Program Files/SSH Communications Security/SSH Secure Shell:/bin
X-Cron-Env: PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
X-Cron-Env: PROCESSOR_ARCHITECTURE=x86
X-Cron-Env: PROCESSOR_IDENTIFIER=x86 Family 15 Model 4 Stepping 3, GenuineIntel
X-Cron-Env: PROCESSOR_LEVEL=15X-Cron-Env: PROCESSOR_REVISION=0403
X-Cron-Env: PROGRAMFILES=C:\Program FilesX-Cron-Env: SYSTEMDRIVE=C:
X-Cron-Env: SYSTEMROOT=C:\WINDOWS
X-Cron-Env: TMP=/cygdrive/c/DOCUME~1/SODO/LOCALS~1/Temp
X-Cron-Env: TEMP=/cygdrive/c/DOCUME~1/SODO/LOCALS~1/Temp
X-Cron-Env: WINDIR=C:\WINDOWSX-Cron-Env: SHELL=/bin/sh
X-Cron-Env: HOME=//SAN/Users$/SODO
X-Cron-Env: LOGNAME=SODO

4) My scripts do such things as SSH into other boxes and grab HTTP connection counts. The scripts only ran using the CYGWIN environment variable set to smbntsec.

5) the HOME environment variable must be set properly: http://blog.spikesource.com/crontab.htm suggests running "env" in your cron out to a file to find differences between your user environment and the cron environment. Good idea!

6) don't forget your scripting basics:
/2007/07/why-doesnt-my-shell-script-run-under.html

7) documentation can help:
C:\cygwin\usr\share\doc\Cygwin\cron-4.1-6.README

Or whatever the latest version happens to be..

Mail That Cron Sends

Change the email that cron sends error messages to:
http://www.cyberciti.biz/faq/linux-unix-crontab-change-mailto-settings/
or
run cron-config
or
Add/edit the "From:" address in /etc/ssmtp.conf

Hope these tips help!
sodo

couldn't set file permissions in Cygwin

Here's a little tidbit that might help someone. I was configuring my public and private keys to allow secure SSH communications between my workstation and a server (see /2007/08/eliminating-need-to-enter-your-password.html for more details). The workstation is an XP box running Cygwin and the server is a Linux variant. When I went to test ssh, I got the following error in Cygwin from the XP box that denied access to the private key:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
WARNING: UNPROTECTED PRIVATE KEY FILE! @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/cygdrive/h/.ssh/id_rsa' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /cygdrive/h/.ssh/id_rsa
Password:

Woops. I guess that's my fault. Looking at the file permissions, I see that id_rsa perms are too loose:
bash-2.02$ ls -l id_rsa
-rw-r--r-- 1 28235 everyone 1675 Jan 30 15:33 id_rsa

I issued the following chmod command:
bash-2.02$ chmod 600 id_rsa
bash-2.02$ ls -l id_rsa
-rw-r--r-- 1 28235 everyone 1675 Jan 30 15:33 id_rsa

But the permissions stayed the same. What gives? Why aren't the permissions changing?

Apparently, Cygwin has two ways to deal with file permissions. Here's the explanation:
http://www.iu.hio.no/cfengine/docs/cfengine-NT/node24.html

The short answer, though, is that I was able to solve my problem by setting this environment variable in /Cygnus/cygwin-b20/cygnus.bat:
set CYGWIN=ntea

Once I set that environment variable in the batch script and restarted a new Cygwin shell, I was able to set the permissions correctly:
bash-2.02$ chmod 600 id_rsa
bash-2.02$ ls -l id_rsa
-rw------- 1 28235 everyone 1675 Jan 30 15:33 id_rsa

Update 2/8/2008
I also got bollucksed with id_rsa permissions if my user's NT permissions on the file were not Full Control. So watch out for that as well.

Live and learn!
sodo

using regular expressions in Oracle

I needed to select rows of data from a column where some of the records begin with a date in the form:
YYYYMMDD

The other records in the column had random text strings. For example:
N MESSAGE
- -------
1 20080124: Hello. I have written to you today to compliment you
2 I have a problem with one of the products you sell.

I thought it would be perfect to use regular expressions to pluck out only the records I needed. Luckily, Oracle now supports for regular expressions as of version 10G:
Writing Better SQL Using Regular Expressions

I needed to first create the regular expression to match the date. In simple terms, I want to match the eight-digit string.

First, here is a regular expression that matches any digit, 0-9:
[0-9]

Secondly, we can expand this match to include the entire eight digit date:
[0-9]\{8\}

The escaped braces repeat the initial match for any number eight ("8") times. I will then put a carat (^) in front of the regular expression to match the beginning of the pattern buffer. The pattern buffer in this case is the MESSAGE column in our table:
^[0-9]\{8\}

OK! So we have our regular expression. Let's put it in a SELECT statement. We will use Oracle 10Gs new regexp_like condition to perform a regular expression match:
SELECT n,message
FROM message_table
WHERE regexp_like (message, '^[0-9]\{8\}');

N MESSAGE
- -------
1 20080124: Hello. I have written to you today to compliment you

Sweet! This works.

There is a slight problem. I did notice a substantial performance penalty for using regexp_like. For a very small table (<20,000 rows), it took about a minute to return the results of the search. Eeek. A normal WHERE clause came back in less than a second.

If I have any further information regarding performance tuning using regular expressions in Oracle, I will let you know.

Here are some more complex examples

TAG

Belkin F5U503 firewire mishap

Though it may seem so, not all tasks computing are an easily digestible banquet for Cacasodo. Yes, there are high points of success, but like many the intrepid engineer, there are more failures than success. In this light will I tell you my tale of frustration.

It began with a simple goal: to optimize my video editing by installing a firewire card using the PCIX slot of my Dell SC1430. As my one and only PCI slot was used by my Sil680 RAID card, I wanted to install a firewire card in one of the two PCIX slots in the box. And of course, the card had to work under Fedora. As a precursor to the story, I will say that I have a DLink PCI firewire card and that card works without error in the box. But my purpose was to have the RAID card use the one and only PCI slot and the Belkin in one of the PCIX slots so both could live happily in the same home. Alas, I did not know the hell that awaited me in trying to make this happen.

It started as a hasty, unresearched purchase of a Belkin F5U503 firewire card. After my purchase was made, I was told by a friend to avoid the Belkin cards, as Belkin was a known OEM reseller who used substandard parts. Cold comfort that was after the fact, but I digress.

I brought the card home and installed it in the first PCIX slot of the Dell. The Belkin has three connectors that allow it to fit in either PCIX slot of my box:
http://www.staples.com/sbd/img/cat/std/s0057708_std.jpg

Here's a pic of my server's motherboard:
http://dcse.dell.com/IFR/PowerEdge/PESC1430/images/sysbrd_full.jpg

When I installed the card in the PCIX slot, here's what happened. Dmesg gives me this error:
[cacasodo ogre ~]# dmesg | grep 1394
ohci1394: fw-host0: OHCI-1394 1.1 (PCI): IRQ=[38] MMIO=[fc5ff800-fc5fffff] Max Packet=[2048] IR/IT contexts=[4/8]
ohci1394: fw-host0: Get PHY Reg timeout [0x000004c0/0x00000000/100]
ohci1394: fw-host0: Get PHY Reg timeout [0x000004c0/0x00000000/100]

I get a couple more errors in dmesg once I turn the cam on:
ieee1394: hpsb_update_config_rom() is deprecated
ieee1394: Failed to generate Configuration ROM image for host 0

lsmod reads:
[cacasodo ogre ~]# lsmod | grep 1394
ohci1394 43273 0 ieee1394 109081 1 ohci1394

lspci reads:
[cacasodo ogre ~]# lspci | grep 1394
06:06.0 FireWire (IEEE 1394): Texas Instruments TSB43AB23 IEEE-1394a-2000 Controller (PHY/Link)

after probing for raw1394:
[cacasodo ogre ~]# modprobe raw1394
[cacasodo ogre ~]# lsmod | grep 1394
raw1394 37233 0 ohci1394 43273 0 ieee1394 109081 2 raw1394,ohci1394

I am running Fedora 7 and my kernel is the EZPlanet firewire patch kernel. I use this patched kernel because firewire is broken in Fedora 7's standard build:
[cacasodo ogre ~]# uname -r
2.6.22.9-1091.ez.fc7

I do see interrupt activity when I connect and turn on the cam:
[cacasodo ogre ~]# grep 1394 /proc/interrupts
38: 2 0 0 0 1 0 0 0 IO-APIC-fasteoi ohci1394

testlibraw gives me something:
[cacasodo ogre ~]# testlibraw
successfully got handle
current generation number: 1 1 card(s)
found nodes on bus: 0,
card name: ohci1394
using first card found: 0 nodes on bus, local ID is 0, IRM is 63

But plugreport doesn't give me anything:
[cacasodo ogre ~]# plugreport
Host Adapter 0 ==============

Adding "noacpi", "noapic" and "nolapic" to my grub.conf and rebooting didn't help.

To see if it was kernel related, I tried booting using these other distros/kernels, with the same result..the "Get PHY Reg timeout" error:
Ubuntu 6.10 -> 2.6.17-10 kernel
Knoppix -> 2.6.19 kernel
Fedora 6 -> 2.6.18-1.2798 kernel

OK. I am almost spent for the evening. But does the card even work in the PCI slot? I put the card in the PCI slot and lo and behold, the card DOES work. To reiterate, the card does work in the PCI slot, just not in either of the PCIX slots. At this point I'm thinking that the card is not getting proper power, but I'm too tired to continue on. I sleep uneasily.

The next day, I take fifteen minutes to call Belkin. The gentleman on the line is helpful, but when I tell him I'm running Linux, I get the cold shoulder and he dismisses me with the "we don't support the card under Linux" line. Well, if the card works fine in one slot under Linux, why shouldn't it work in the other slot?! No matter, "we don't support the card under Linux."

Argh. So here is where I dive off into the deep end to set out to prove Belkin wrong that it is a power issue and not an OS issue. I went on Belkin's support site and tried to find the voltage requirements for the card. I could not. I searched on google and not so amazingly, I find a link to the voltage requirements ON BELKIN'S OWN SITE!
http://web.belkin.com/support/kb/kb.asp?a=3742&langid=

Great search engine there, Belkin! NOT! As well, I look on Dell's site for the voltage of the PCIX slots:

• 2 64-bit/100MHz PCI-X slots 3.3V

The slots handle 3.3volts. Referring back to the Belkin KB article, it states clearly:
The F5U503 PCI requires a 5 volt PCI slot.

I knew it! Now, in order to prove Belkin wrong, I decide to get a Windows OS running on the Dell and show them that it is a POWER issue and NOT an OS related issue!

To do this, I'd install an older IDE drive in the Dell and apply an old Ghost image I had of XP. Well, the Dell doesn't have a floppy drive. So, I power down my web server, take the floppy out, install it in the Dell and reconfigure the BIOS to allow the floppy to be seen. OK. I try one of my Ghost floppies. It doesn't boot. I try a second and a third. They do not boot as well. Argh.

Wouldn't it be great if Ghost worked on a bootable CD? Unfortunately, Norton doesn't provide that option. But an enterprising person has a lovely install doc written up here:
http://nightowl.radified.com/bootcd/bootcdintro.html

Thank you, Nightowl! Though, it did take me a number of hours to setup the Ghost bootable disk, it was well worth the time as I now have reliable boot media for Ghost. The process itself could use a small blog entry. Nightowl did a great job, but he needs better formatting and less verbiage to make the instructions easier to follow.

Next, I used it to boot and apply a Ghost image of my XP system to that older hard drive I mentioned. But when I bootup, Ghost does not recognize the DVD disc. I try another disc that has an earlier Ghost image. Ghost also does not recognize that disc. Finally, I have a third DVD. Luckily, Ghost recognizes that disc. Unfortunately, when I go to apply the image, Ghost cannot find the IDE hard drive. What the hell? I see the drive in the BIOS, though the BIOS doesn't report back the correct size of the drive. It is the master drive on the IDE controller..what gives?

I tried a few couple configuration options:
-hard drive being slave
-hard drive cable select
-disabling the SATA drives
-changing boot sequence
-refreshing the BIOS version to the latest 1.0.4

None of these worked. Then I thought that if Ghost re-imaging didn't work, I should just install the OS itself. So, I had a bootable copy of the XP disc. I started the system with the disc in the drive, but the install process soon gave me a blue screen with a message regarding pci.sys. Looks like a lot of folks had that one:
http://www.google.com/search?q=xp+professional+install+pci.sys+blue+screen

This problem should be resolved with a disc that integrates XP with SP2 . Which of course, I don't have right now. So for expediancy's sake, I take another route: a Windows 2000 Advanced Server install. I was able to put the extra hard drive in a second machine, start the Win2K install process, but for some reason, the hard drive is not recognized properly by the BIOS. I brought the disk over to my Dell SC400 machine that runs XP. I disconnected the existing XP drive, installed the disk and got Win2K Advanced Server working without a problem. The box boots to that drive, so I know that the Master Boot Record is not corrupt. But when I move the drive back to the SC1430, it refuses to boot.

Not sure what to do at this point. I'd sure like to prove those Belkin support engineers wrong, but I can't get the "Belkin certifed" OS (in this case, Windows 2000 Advanced Server) to boot in the Dell. I assume I will have the same problem with XP SP2, but it might be worth a try.

To sum up:
-card doesn't work in PCIX slot, but I believe it is a simple voltage problem
-I'd like to prove this to Belkin, but I can't get a Windows OS to load via Ghost'ing an existing hard drive or installing a hard drive with a Windows OS on it

Update 1/6/08
Ooops..my bad. Looking in the manual for the Dell SC1430, it shows that pre-Win2003 OSs are not supported. Thereby, I am out $52 for the price I paid for the Belkin card. Just goes to show..DO YOUR RESEARCH before making a purchase of this kind.
!
A Frustrated Cacasodo