Wednesday, February 24, 2010

wake on lan redux

Just like my other wake on lan experience, I spent about eight hours debugging why wake on lan wouldn't work on my newly built Fedora 12 system. I went through a bunch of threads and learned something about sleep states in Linux. I also learned that ACPI has supplanted APM. ACPI is now being replaced by /sys interfaces to power states.

Problem
The network card port was getting disabled by Fedora 12 for wake on lan when the box was shutdown.

Observations
I noticed that the status of the network activity led on the network card went dark when I shut the machine down.  This means that my network card was not active and listening for incoming WOL requests.  If I removed the power cable and plugged the power cable back in, the traffic light blipped on a couple times.  For some reason then, Linux was shutting off or disabling a WOL state for the network card when I would shutdown the machine.

Resolution

After much googling and testing, I found that the sleep state for my network card was disabled.  My network card's identification was found in lspci output:

[sodo@computer ~]$ lspci -tv | grep 1c
           +-1c.0-[01]----00.0  Broadcom Corporation NetXtreme BCM5751 Gigabit Ethernet PCI Express

I then correlated the pci card info to acpitool output.  I found my card (1c) in the output below at #4:

[sodo@computer ~]$ acpitool -w
   Device S-state   Status   Sysfs node
  ---------------------------------------
  1. DWN1   S5 *disabled  pci:0000:03:00.0
  2. PE2X   S5 *disabled  pci:0000:02:00.3
  3. PEX3   S5 *disabled  pci:0000:00:03.0
  4. SBEX   S5 *disabled  pci:0000:00:1c.0
  5. COMP   S5 *disabled  pci:0000:00:1e.0



Seeing it as disabled, I could reenable it by using this command:


[sodo@computer ~]$ acpitool -W 4
   Device S-state   Status   Sysfs node
  ---------------------------------------
  1. DWN1   S5 *disabled  pci:0000:03:00.0
  2. PE2X   S5 *disabled  pci:0000:02:00.3
  3. PEX3   S5 *disabled  pci:0000:00:03.0
  4. SBEX   S5 *enabled  pci:0000:00:1c.0
  5. COMP   S5 *disabled  pci:0000:00:1e.0





I finally resolved my WakeOnLan issue by creating a script to do three things:
1) remove my network card module
modprobe -r tg3
2) enabling sleep states
acpitool -W 1 (and also 2-5)
3) call "shutdown -h now"

The script is simple enough, though it is a bit of a pain to manually call it.  I put an alias to "sd" in my .bashrc:
[sodo@computer ~]$ sudo cat /usr/sbin/sd
modprobe -r tg3
# sleep state for Broadcom card (#4 from acpitool output)
acpitool -W 4
/usr/sbin/shutdown -h now

A variation on the script for my new i7 box:
#!/bin/bash
sudo ethtool -s p5p1 wol g
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
do 
   sudo acpitool -W $i
done
sudo shutdown -h now

A second variation for my latest i9 box (note that I don't need to enable all sleep states, just the one for the card.  Not sure why I thought I needed to enable all sleep states, I was probably just being lazy):
#!/bin/bash
# enable wak-on-lan
sudo ethtool -s enp10s0 wol g

# enable sleep state for Realtek card (#20 from acpitool output)
acpitool -W 20 

/usr/sbin/shutdown -h now

Previous tries
try acpi=off (in grub.conf)
get this error: nvrm failed to register with acpi subsystem
try adding mem to /sys/power/state (in rc.local)
didn't work
try init=0 (each time you shutdown)
didn't work
now that edits are done to rc.local, try resetting card with Broadcom Diagnostic CD
didn't work
try just hitting power button
didn't work

Commands
ethtool eth0
ethtool -i eth0
ethtool -s eth0 wol d
ethtool -s eth0 wol g


WakeOnLan options description
wol pumbagsd...
Sets Wake-on-LAN options. Not all devices support this. The argument to this option is a string of

characters specifying which options to enable.
p Wake on phy activity
u Wake on unicast messages
m Wake on multicast messages
b Wake on broadcast messages
a Wake on ARP
g Wake on MagicPacket(tm)
s Enable SecureOn(tm) password for MagicPacket(tm)
d Disable (wake on nothing). This option clears all previous options.

acpitool -w


[root@ogre ~]$ cat /proc/acpi/sleep
S0 S4 S5

[root@ogre ~]$ cat /sys/power/state
disk
[root@ogre /]# acpitool -w
Device S-state Status Sysfs node
---------------------------------------
1. DWN1 S5 enabled pci:0000:03:00.0
2. PE2X S5 enabled pci:0000:02:00.3
3. PEX3 S5 enabled pci:0000:00:03.0
4. SBEX S5 enabled pci:0000:00:1c.0
5. COMP S5 enabled pci:0000:00:1e.0

[root@ogre ~]# cat /proc/acpi/wakeup
Device S-state Status Sysfs node
DWN1 S5 enabled pci:0000:03:00.0
PE2X S5 enabled pci:0000:02:00.3
PEX3 S5 enabled pci:0000:00:03.0
SBEX S5 enabled pci:0000:00:1c.0
COMP S5 enabled pci:0000:00:1e.0

[root@ogre ~]# sh -c "echo COMP > /proc/acpi/wakeup"

[root@ogre /]# lspci -tv
-[0000:00]-+-00.0 Intel Corporation 5000V Chipset Memory Controller Hub
+-02.0-[02-06]--+-00.0-[03-05]--+-00.0-[04]----00.0 3ware Inc 9650SE SATA-II RAID PCIe
\-01.0-[05]--
\-00.3-[06]--
+-03.0-[07]----00.0 nVidia Corporation GeForce 8800 GT
+-10.0 Intel Corporation 5000 Series Chipset FSB Registers
+-10.1 Intel Corporation 5000 Series Chipset FSB Registers
+-10.2 Intel Corporation 5000 Series Chipset FSB Registers
+-11.0 Intel Corporation 5000 Series Chipset Reserved Registers
+-13.0 Intel Corporation 5000 Series Chipset Reserved Registers
+-15.0 Intel Corporation 5000 Series Chipset FBD Registers
+-16.0 Intel Corporation 5000 Series Chipset FBD Registers
+-1c.0-[01]----00.0 Broadcom Corporation NetXtreme BCM5751 Gigabit Ethernet PCI Express
+-1d.0 Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #1
+-1d.1 Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #2
+-1d.2 Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #3
+-1d.3 Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #4
+-1d.7 Intel Corporation 631xESB/632xESB/3100 Chipset EHCI USB2 Controller
+-1e.0-[08]--
+-1f.0 Intel Corporation 631xESB/632xESB/3100 Chipset LPC Interface Controller
\-1f.1 Intel Corporation 631xESB/632xESB IDE Controller


[root@ogre ~]# lsmod grep tg
tg3 105476 0
[root@ogre ~]# modinfo tg3
filename: /lib/modules/2.6.31.12-174.2.22.fc12.x86_64/kernel/drivers/net/tg3.ko
firmware: tigon/tg3_tso5.bin
firmware: tigon/tg3_tso.bin
firmware: tigon/tg3.bin
version: 3.99
license: GPL
description: Broadcom Tigon3 ethernet driver
author: David S. Miller (davem@redhat.com) and Jeff Garzik (jgarzik@pobox.com)
srcversion: 5B5CB8F28F8498838A4DBA7
alias: pci:v0000106Bd00001645sv*sd*bc*sc*i*
alias: pci:v0000173Bd000003EAsv*sd*bc*sc*i*
alias: pci:v0000173Bd000003EBsv*sd*bc*sc*i*
alias: pci:v0000173Bd000003E9sv*sd*bc*sc*i*
alias: pci:v0000173Bd000003E8sv*sd*bc*sc*i*
alias: pci:v00001148d00004500sv*sd*bc*sc*i*
alias: pci:v00001148d00004400sv*sd*bc*sc*i*
alias: pci:v000014E4d0000168Csv*sd*bc*sc*i*
alias: pci:v000014E4d00001694sv*sd*bc*sc*i*
alias: pci:v000014E4d00001690sv*sd*bc*sc*i*
alias: pci:v000014E4d00001692sv*sd*bc*sc*i*
alias: pci:v000014E4d00001699sv*sd*bc*sc*i*
alias: pci:v000014E4d00001689sv*sd*bc*sc*i*
alias: pci:v000014E4d00001688sv*sd*bc*sc*i*
alias: pci:v000014E4d00001680sv*sd*bc*sc*i*
alias: pci:v000014E4d00001681sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Bsv*sd*bc*sc*i*
alias: pci:v000014E4d00001684sv*sd*bc*sc*i*
alias: pci:v000014E4d00001698sv*sd*bc*sc*i*
alias: pci:v000014E4d00001713sv*sd*bc*sc*i*
alias: pci:v000014E4d00001712sv*sd*bc*sc*i*
alias: pci:v000014E4d000016DDsv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Bsv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001679sv*sd*bc*sc*i*
alias: pci:v000014E4d00001678sv*sd*bc*sc*i*
alias: pci:v000014E4d00001669sv*sd*bc*sc*i*
alias: pci:v000014E4d00001668sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Fsv*sd*bc*sc*i*
alias: pci:v000014E4d00001693sv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Bsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001674sv*sd*bc*sc*i*
alias: pci:v000014E4d00001673sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Bsv*sd*bc*sc*i*
alias: pci:v000014E4d00001672sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Asv*sd*bc*sc*i*
alias: pci:v000014E4d000016FEsv*sd*bc*sc*i*
alias: pci:v000014E4d000016FDsv*sd*bc*sc*i*
alias: pci:v000014E4d000016F7sv*sd*bc*sc*i*
alias: pci:v000014E4d00001601sv*sd*bc*sc*i*
alias: pci:v000014E4d00001600sv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000167Csv*sd*bc*sc*i*
alias: pci:v000014E4d00001677sv*sd*bc*sc*i*
alias: pci:v000014E4d00001676sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Asv*sd*bc*sc*i*
alias: pci:v000014E4d00001659sv*sd*bc*sc*i*
alias: pci:v000014E4d00001658sv*sd*bc*sc*i*
alias: pci:v000014E4d0000166Esv*sd*bc*sc*i*
alias: pci:v000014E4d00001649sv*sd*bc*sc*i*
alias: pci:v000014E4d0000170Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000170Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Dsv*sd*bc*sc*i*
alias: pci:v000014E4d0000169Csv*sd*bc*sc*i*
alias: pci:v000014E4d00001696sv*sd*bc*sc*i*
alias: pci:v000014E4d000016C7sv*sd*bc*sc*i*
alias: pci:v000014E4d000016C6sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A8sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A7sv*sd*bc*sc*i*
alias: pci:v000014E4d000016A6sv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Esv*sd*bc*sc*i*
alias: pci:v000014E4d0000165Dsv*sd*bc*sc*i*
alias: pci:v000014E4d00001654sv*sd*bc*sc*i*
alias: pci:v000014E4d00001653sv*sd*bc*sc*i*
alias: pci:v000014E4d0000164Dsv*sd*bc*sc*i*
alias: pci:v000014E4d00001648sv*sd*bc*sc*i*
alias: pci:v000014E4d00001647sv*sd*bc*sc*i*
alias: pci:v000014E4d00001646sv*sd*bc*sc*i*
alias: pci:v000014E4d00001645sv*sd*bc*sc*i*
alias: pci:v000014E4d00001644sv*sd*bc*sc*i*
depends:
vermagic: 2.6.31.12-174.2.22.fc12.x86_64 SMP mod_unload
parm: tg3_debug:Tigon3 bitmapped debugging message enable value (int)


References
HOWTO: Set your system up for Wake On LAN (WOL)
http://en.wikipedia.org/wiki/Advanced_Configuration_and_Power_Interface
http://acpi.sourceforge.net/documentation/sleep.html
http://ubuntuforums.org/showthread.php?t=1309820
http://forums.fedoraforum.org/archive/index.php/t-63763.html
http://www.gelato.unsw.edu.au/lxr/source/Documentation/power/tricks.txt
http://ubuntuforums.org/showthread.php?t=234588&page=3
http://ubuntuforums.org/showthread.php?t=814939&page=4
https://askubuntu.com/questions/654820/how-to-find-pci-address-of-an-ethernet-interface
http://ask.xmodulo.com/network-card-driver-name-version-linux.html

Fedora List of Kernel Parameters
NOTE: The full list of kernel options is in the file /usr/share/doc/kernel-doc-/Documentation/kernel-parameters.txt, which is installed with the kernel-doc package.

Tuesday, February 23, 2010

how to determine sid of a user

Nice VB script here from Daniel Petri:
http://www.petri.co.il/forums/showthread.php?t=21332

reprinted here for convenience
' www.petri.co.il/forums/showthread.php?p=61469#post61469

Const HKEY_USERS = &H80000003


strComputer = "." '<-- name of remote computer

Set objWMIService = GetObject( _
"winmgmts:{impersonationLevel=impersonate}!\\" _
& strComputer & "\root\cimv2")

Set Users = objWMIService.InstancesOf ("Win32_ComputerSystem")

for each User in Users
logonname = User.UserName
strUsr = Split(logonname,"\")(1)
strDomain = Split(logonname,"\")(0)
exit For
next

set colAccounts = objWMIService.ExecQuery _
("select * From Win32_UserAccount where " _
& "name = '" & strUsr & "' AND domain = '" & strDomain & "'")

For each objAccount in colAccounts
strSID = objAccount.SID
exit For
Next


wsh.echo "HKU-path for", logonname, vbNewLine & "HKEY_USERS\" & strSID & "\..."


OUTPUT
C:\WINDOWS>user.vbs
Microsoft (R) Windows Script Host Version 5.7
Copyright (C) Microsoft Corporation. All rights reserved.

HKU-path for DOMAIN\sodo
HKEY_USERS\S-1-5-11-7177352686-8047241297-547765659-67771

Saturday, February 13, 2010

Fedora 12, x86-64 first impressions

I recently installed Fedora 12, x86-64 on my Dell SC1430. I will say that the new ext4 filesystem is fast, fast. And the kernel loads in about 30 seconds for me:
bootchart.jpg

However, Fedora's latest disappoints with a bunch of minor, yet completely annoying bugs. It's kind of like the Chinese "Death of One Thousand Cuts."

CUT 1
CTRL-ALT-BACKSPACE no longer works
This convenient method of restarting X was disabled for the fat-fingering newbie. Maybe Fedora should have a multicolored penguin (a Mac-Linux hybrid) as its' logo from now on?

At least it was easily solved.

CUT 2
Virtual consoles don't work
You know..the CTRL-ALT-F[1-6] keys. Maybe this one is related to my NVidia card. Still, I'm using the nouveau driver, not the NVidia driver. Don't know why this should have broken.

CUT 3
Nautilus browser doesn't load
I found this error in xsession-errors:
LoadPlugin: failed to initialize shared library ./mozilla/plugins/nppdf.so [./mozilla/plugins/nppdf.so: wrong ELF class: ELFCLASS32]

Turns out, the install of Adobe Reader (32-bit) caused this error, which in turn, stopped Nautilus from loading. Yanking out the Reader resolved the issue. Maybe this isn't the fault of Fedora, but the Adobe RPMs. Still burned about three hours tracking this one down.

CUT 4
Window positions in workspaces not saved
I guess we can blame GNOME here for this annoyance. Devilspie is an interesting workaround, but GNOME should just do the right thing the first time. Currently, it doesn't. Here's another link re:devilspie.

CUT 5
Wake on Lan is now broken
Perhaps this is related to my new 3ware 9650SE RAID card. It was working fine before. I've gone through the Broadcom diagnostics bootable CD (b57udiag.exe) which fixed it the last time. No go this time. And ethtool reports wake on lan should be good to go and the BIOS is enabled for WOL:
[sodo@ogre fedora12]$ sudo ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: g
Current message level: 0x000000ff (255)
Link detected: yes


So I don't know whussup here.

Update 2/24/2010
I spent about eight hours debugging why wake on lan wouldn't work on my newly built system. When through a million threads, learned something about sleep states in Linux..finally got it by putting a line in the Linux halt script (called by "shutdown -h now" to remove the kernel module for the ethernet driver. Self-evident, right? NOT! What the F?

full story here
*** end update ***

CUT 6
GRUB takes 3.5 minutes to boot!
Man, this one is a bugger. Grub loads and then the system hangs for 3.5 minutes until kernel starts up. Probably related to the new GPT partitioning that I'm using on the new 4.5TB RAID5 set. Drag having to wait that long for boot. I've got a few leads on the issue, nothing solved yet.

Update 2/28/2010
Looks like I was barking up the wrong tree with my assumption that it was Grub and GPT causing my long boot times. It seems to be an interaction with the two PCI Express cards that I have in the box: the NVidia 8800GT and the 3Ware 9650SE card. Once I removed the 8800GT, the box booted up in about 40 seconds. Of course, I can't put the 8800GT in the lower slow because it is does not fit with the 8x-16x PCIe adapter. Now I have to figure out what the heck is going on..
*** end update ***

Update 3/2/2010
So whaddya do when you have a sticky problem? YOU DREMEL THE F*CKER!

I must say I did a much nicer job dremeling the lower slot (the one on the right in the picture) this time.

Now the box boots in 40 seconds instead of 4 minutes..wahoo!
*** end update ***

CUT 7
Preferences -> Windows control has been removed by default
Another irritation. I like my Windows to come to the foreground by simply mousing over them. The Windows Preference control is not installed by default. So, you'll need to install control-center-extra to return the ability to tweak your window focus settings. Argh.

CUT 8
No Xorg.conf file
The Fedora 12 Deployment Guide makes no mention of the possibility of having no xorg.conf file after a new install:
http://docs.fedoraproject.org/deployment-guide/f12/en-US/html/s1-x-server-configuration.html

But it is mentioned in the list of Fedora 12 bugs.

CUT 9
Disabling the nouveau video driver in favor of the NVidia proprietary driver is a pain
Related to #8. If you'd rather use NVidia's driver instead of the open source nouveau video driver, this can cause some pain too. Here's info on that process. The x86-64 NVidia install per those instructions may break with the following error in Xorg.0.log:
(II) Loading /usr/lib64/xorg/modules/extensions/nvidia/libglx.so
dlopen: libGLcore.so.1: cannot open shared object file: No such file or directory
(EE) Failed to load /usr/lib64/xorg/modules/extensions/nvidia/libglx.so
(II) UnloadModule: "glx"
(EE) Failed to load module "glx" (loader failed, 7)


This is due to the installer either installing the NVidia 32-bit libraries first, or just not working properly. So the fix is to install the 64-bit libraries first and then the 32-bit ones second, as per my post on FedoraForum.org:
http://forums.fedoraforum.org/showpost.php?p=1332653&postcount=961


Related, here's info on video modes you can select upon boot using a kernel switch. More importantly, how the nouveau driver gets loaded in the kernel here and in the context of Mandriva, here (via initrd).

I was interested in disabling the nouveau driver altogether, just in case it fixed issue #2 above, so I:
1) added nouveau to /etc/modprobe.d/blacklist.conf
2) rebuilt my mkinitrd file (had to "yum install mkinitrd" on F12)
mkinitrd -v -f /root/initrd-`uname -r`.img `uname -r`
3) added a new entry in my grub.conf pointing to that mkinitrd, no longer using an initial ram filesystem:
title Fedora (2.6.31.12-174.2.3.fc12.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.31.12-174.2.3.fc12.x86_64 ro root=/dev/mapper/vg_ogre-lv_root scsi_mod.scan=sync
initrd /initramfs-2.6.31.12-174.2.3.fc12.x86_64.img


but using old school initrd file:
title Fedora (2.6.31.12-174.2.3.fc12.x86_64_minusNouveau)
root (hd0,0)
kernel /vmlinuz-2.6.31.12-174.2.3.fc12.x86_64 ro root=/dev/mapper/vg_ogre-lv_root scsi_mod.scan=sync
initrd /initrd-2.6.31.12-174.2.3.fc12.x86_64.img


Sankaran Raman provides an alternate method of disabling the nouveau driver by creating and editing an xorg.conf file.


CUT 10
Mplayer keeps resetting volume to maximum
This one is not good for my ears or my speakers. Thankfully, there is an easy fix:
http://www.linuxquestions.org/questions/linux-software-2/mplayer-keeps-resetting-volume-to-max-773269/

CUT 11
AAC audio encoding not included in ffmpeg non-free distro
Now, this isn't a cut on Fedora. Rightly so, RPM Fusion doesn't package FAAC support as it is a proprietary codec. So you'll get errors in FFMPEG like "unsupported audio codec: libfaac" if you try to convert an audio or video file to AAC audio format. Ugh. Here is a solution to this problem:
http://fozzolog.fozzilinymoo.org/tech/2009/11/recompiling-ffmpeg-for-fedora-12-to-add-faac-support.html

CUT 12
lose control of left mouse button
The left mouse button stops functioning. This is an incredible pain in the a$$. From a link I found I Google (of course), under Keyboard Preferences -> Mouse Keys, I disabled "Pointer can be controlled using the keypad". This seems to have resolved the issue. Yarg!!

Pardon my evident frustration on these issues folks, but I'm spending too much time on install problems and not enough time creating. It's brutal just trying to get work done. I'm not leaving Fedora, but everytime I upgrade, this is what happens.

I'll keep you abreast of the various remaining situations..
TAG

Friday, February 05, 2010

3ware 9650SE support experience

I recently bought a RAID card and new drives for video editing:
3WARE Pci-e 9650SE RAID card with Battery Backup
four Western Digital 1.5 TB Green SATA 32MB Cache Hard Drive

I setup a RAID5 array with the four 1.5TB drives and installed Fedora 12, x86-64 on top. The RAID array is the boot drive and I've setup a GPT partition. However, I have a slight problem as the boot times on my Dell SC1430 have increased to about four minutes! This is an unacceptably long time for a boot sequence to finish.

Update 2011/10/24
The long-boot problem I experienced was because during the process of the 3ware install, I had to move the GPU (NVidia 8800GT) to a different PCIe (PCI Express) slot (the second one from the end).  For some reason, the Dell SC1430 that I have slows the boot time considerably when using that slot for the video card.  Moving the 8800GT from the second slot to the first PCIe slot resolved the boot time issue.
*** end update ***

Online Chat System
I opened a ticket with 3ware support via their online chat and after an hour of waiting, an engineer finally got on the chat and told me that I need to upgrade my controller card firmware version and Linux driver version to the latest and greatest. The online chat system was OK; however, I waited an hour, stepped away for five minutes and in those five minutes, the engineer responded to my query and promptly dropped off the line.

Luckily, there is a feature to have the chat session emailed to you. I did that and saw the response from the engineer. So that was good. But here's a note for 3ware..if your engineer's are too busy to leave their chat session open for more than five minutes, either retrain them to keep the session open for at least fifteen minutes or hire more support engineers!! It is frustrating for the customer to be dropped like that.

Secondly, I need confirmation about the availability of an rpm for F12, x86-64, so I hopped on the online chat again and waited 45 minutes. After waiting, the online chat just dropped me altogether without any answer from an engineer. Frustrating. 3ware needs to work on this system.

Multiple Software Versions
In any case, I needed to find my current driver version and controller card version. Just finding this information was a task in itself. The support information on 3ware's site is all there..it is just a big task just wading through it all. Also, there seems to be a lot of redundant and confusing information in the Linux section.

Here's another suggestion for 3ware..for 3ware newbies, there should be a simple introduction to the product explaining that the product includes multiple software versions:
-the firmware on the controller card
-the operating system driver
-the command line interface software version
-the software api version
-the web gui version


Basic Install on Linux
You want to install just two items:
1) the command line interface (tw_cli)
2) the 3DM2 GUI

As of this writing, the most recent install packages are these:

tw_cli-linux-x86_64-9.5.3.tgz
3DM2_CLI-Linux-x86_64-9.5.3.tgz



You'll unpack those files (tar xvf ).  If you run any flavor of Linux, that 3DM2 installer will drop a "setupLinux*.bin" file in the unpack directory.  You'll want to "chmod a+x setupLinux*.bin" and then execute that file.  In my case:
./setupLinux_x64.bin

Here are some screens to help you see the process:







Web GUI Gotcha
One of the other gotchas is that once you install the web GUI (3DM2), you'll need to set a browser exception of "localhost".  If you don't do that, you'll get "Webpage is not available" when you login to https://localhost:888.

Good Information from the Command Line client
9650SE controller card firmare version
[root@ogre ~]# tw_cli
//ogre> show

Ctl Model (V)Ports Drives Units NotOpt RRate VRate BBU
------------------------------------------------------------------------
c4 9650SE-4LPML 4 4 1 0 1 1 OK

//ogre> /c4 show

Unit UnitType Status %RCmpl %V/I/M Stripe Size(GB) Cache AVrfy
------------------------------------------------------------------------------
u0 RAID-5 OK - - 256K 4190.92 RiW ON

VPort Status Unit Size Type Phy Encl-Slot Model
------------------------------------------------------------------------------
p0 OK u0 1.36 TB SATA 0 - WDC WD15EADS-00P8B0
p1 OK u0 1.36 TB SATA 1 - WDC WD15EADS-00P8B0
p2 OK u0 1.36 TB SATA 2 - WDC WD15EADS-00P8B0
p3 OK u0 1.36 TB SATA 3 - WDC WD15EADS-00P8B0

Name OnlineState BBUReady Status Volt Temp Hours LastCapTest
---------------------------------------------------------------------------
bbu On Yes OK OK OK 255 06-Feb-2010

//ogre> /c4 show firmware
/c4 Firmware Version = FE9X 4.08.00.006


9650SE controller card serial number
[root@ogre ~]# tw_cli
//ogre> /c4 show serial
/c4 Serial Number = LXXXX10A9420XXX


9650SE Linux driver version
[root@ogre ~]# cat /sys/class/scsi_host/host4/stats
3w-9xxx Driver version: 2.26.02.012
Current commands posted: 0
Max commands posted: 29
Current pending commands: 0
Max pending commands: 0
Last sgl length: 2
Max sgl length: 32
Last sector count: 24
Max sector count: 256
SCSI Host Resets: 0
AEN's: 0


9650SE application software version
[root@ogre ~]# tw_cli
//ogre> show ver
CLI Version = 2.01.09.004
API Version = 2.06.01.006


Ticketing System
Also, there is a separate system for ticketing (Siebel CRM provided by lsi.com) so you have to create two logins, which is dumb:
-one login for the support site
-one login for ticketing

They need to fix this.

I will update this post when I have further information and hopefully, a solution to my problem.
'sodo

3ware Resources
3ware Support Contacts
http://www.3ware.com/support/support.asp
3ware Software Version Checker
http://www.3ware.com/support/versionchecker.asp
3ware Software Download Center
http://www.3ware.com/support/download.asp
Documentation
http://www.3ware.com/support/userdocs.asp
Direct Downloads for all 3ware products
http://www.3ware.com/support/downloadpage.asp
Direct Downloads for the 9650SE
http://www.3ware.com/support/downloadpageprod.asp?pcode=19&path=Escalade9650SE-Series&prodname=3ware%209650SE-%202LP/4LPML/8LPML/%2012ML/16ML
Online Support Ticketing System
https://www.3ware.com/

Other Resources
Common 3ware /var/log/messages

Friday, January 15, 2010

shell scripting best practices

I put together this spreadsheet of some hints, remembrances and foggy thoughts about shell scripting. More as a jog to my cob-webbed brain then anything else:
Shell Scripting Best Practices PDF

Reference
Exit Codes

Monday, December 07, 2009

playstation store poor business practice

So, I recently bought a Sony Playstation 3. Very nice unit with lots of features:
-play games
-watch blurays
-surf the internet
-connect to media servers

Not bad for $300. So tonight, I tried out another feature of the Playstation Store: purchasing a movie. I saw that Terminator Salvation was recently released. Cool! Let's check it out.

This was the first time I used the Playstation Store to buy a movie. I navigated through the menus and saw that the movie was available to rent in HD. Rather than rent it in HD, I wanted to simply buy the movie outright. I navigated down to the movie and purchased the movie. Unfortunately, in the excitement of the purchase, I didn't realize that the movie I purchased was in Standard Definition. Uh oh! What to do?

Can I redownload the movie in High Definition? No.

Can I cancel the order and repurchase the movie in HD? No.
Well, why not? You can rent the movie in HD, but you can't BUY the movie in HD? How stupid is that?!

Since there was no option to cancel the order, I had but one choice. I'm going to have to call the Playstation Store. Ugh.

Calling the Playstation Store (877) 971-7669, I learned from Bruce that there are no refunds on movie purchases even if:
1) it is your first time purchasing a movie
2) you didn't download the movie
3) you simply made a mistake and want to return it for the High Definition version

After talking to Bruce, I needed confirmation from someone in management that this is truly their business practice:
- to not refund you even if you didn't use their product

I got someone named Richard on the line. Richard answered and said he was from "Tech Support." After that introduction, I didn't have much confidence that he'd be very customer friendly. Indeed, Richard laid out the same story as Bruce..no leniency for a first time user..no refund for my $14.99 + $1.05 undownloaded, unwatched movie. This experience just soured me on the whole Playstation 3.

The Playstation Store is about the only store in the whole United States that doesn't allow you to return something you didn't use. Thanks Sony! That's mighty white of you.

Sometimes people just make mistakes. After coming home from a long day, they needn't be penalized if they happen to overlook the letter H for the letter S! And I didn't even use the product! I didn't download it and I didn't watch it. It's like buying some t-shirts, not opening the package and wanting to return them. Should be no problem, right? WRONG! Not for Sony, because even if you DON'T use their product and want to return it, you CAN'T. Shame on Sony's return policy. Shame shame shame!

After this experience, I might just sell the Playstation because it is quite evident that all Sony wants is my money. Since I don't play many games, it will be easy for me to unload this thing and get another device to be the receiver for my audio and video servers. I have no problem with that.

Word to Sony..work on your return policy for movies. It really sucks.

Saturday, November 14, 2009

streaming 1080P using MoCA, ethernet over cable

As you guys know, I recently bought a new AV receiver. In support of that, I setup an audio server that streams lossless AAC from the MacBook to the receiver. This works well and sounds great. FYI - the lossless audio streams at about 1411Kbps according to this spec:

Secondly, I setup TwonkyMedia server on my Linux video editing box to stream video. I tested streaming 1080P content from that server over wireless using a Linksys by Cisco WET610N Wireless-N Ethernet Bridge to the Playstation 3. It's cool, as the very high def video plays back, but unfortunately, I get stutters. Connecting the PS3 via wired eliminates the stutters. So, the best solution to the problem would be to run an ethernet line from upstairs to the downstairs. However, I live in a condo and it is impractical for me to run ethernet. What to do?

It seems that a standards group has been working on this same problem and that a solution has been around for a few years now:

I guess I've been sleeping at the wheel! Anyway, MoCA allows me to use the coax cable within my house to stream video. That's great! Here are a few devices that do what I wanted:

Motorola NIM100
Actiontec MI424WR (used in Verizon FIOS)

After reading the below articles, I decided the Moto NIM100s were the best fit for me, as they were an inexpensive and solid solution:

I purchased two from eBay for $110 (shipping included).

After a few missteps, I got those b1tches working last night. I had trouble early on, as:
1) the devices don't default to DHCP by default
2) I didn't realize you couldn't get to the admin GUI when the coax connection was active

So I fumbled around for two hours until I figured that stuff out.

But now I am able to stream glorious 1080P videos from my Twonky Media Server to my Playstation 3! Of course, its not perfect, as the highest bitrate streams are coming in at greater than 60Mbps! Woah! Thus, I need to gain an understanding of the capacity of the MoCA network within my condo.

To do this, I found a nifty utility called netio that allows you to measure bandwidth between two hosts on your network:

You setup a server component on a PC at one end of your network and then a client PC at the other end of the network. The program then sends packets of different size from the client to the server and reports the send and receive rates in between. The output looks like this:

CLIENT SIDE

C:\temp\netio\bin>win32-i386.exe -t 192.168.1.97

NETIO - Network Throughput Benchmark, Version 1.26
(C) 1997-2005 Kai Uwe Rommel

TCP connection established.
Packet size 1k bytes: 10345 KByte/s Tx, 10908 KByte/s Rx.
Packet size 2k bytes: 10700 KByte/s Tx, 10954 KByte/s Rx.
Packet size 4k bytes: 11209 KByte/s Tx, 11342 KByte/s Rx.
Packet size 8k bytes: 11157 KByte/s Tx, 10953 KByte/s Rx.
Packet size 16k bytes: 11122 KByte/s Tx, 11044 KByte/s Rx.
Packet size 32k bytes: 11256 KByte/s Tx, 11536 KByte/s Rx.
Done.


SERVER
C:\temp\netio\bin>win32-i386.exe -t -s
NETIO - Network Throughput Benchmark, Version 1.26
(C) 1997-2005 Kai Uwe Rommel
TCP server listening.
TCP connection established ...
Receiving from client, packet size 1k ...
Sending to client, packet size 1k ...
Receiving from client, packet size 2k ...
Sending to client, packet size 2k ...
Receiving from client, packet size 4k ...
Sending to client, packet size 4k ...
Receiving from client, packet size 8k ...
Sending to client, packet size 8k ...
Receiving from client, packet size 16k ...
Sending to client, packet size 16k ...
Receiving from client, packet size 32k ...
Sending to client, packet size 32k ...
Done.
TCP server listening.

hope folks find this useful,
TAG



Twonky Media Install notes
(from the HowTo)
Copy all files (keeping the directory structure) into your most 
favourite installation directory, e.g. /usr/local/twonkymedia.

Make sure that the server file, plugins and the cgi scripts have the exe bit set.
If not, try:
"chmod 700 twonkym* cgi-bin/* plugins/*"
Eventually it is necessary to have a multicast route set for the
server by issuinge, e.g.:
"route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0"

If you are running a Debian distribution on a 64 bit architecture, it
is possible that you need to install the 32 bit compatibility
libraries. In this case invoke:
"apt-get install ia32-libs"

Now start the server with a full qualified path (never just "./twonkymedia"),
but "/usr/local/twonkymedia/twonkymedia" . The server will display a startup
message. The server is running now and you can find and use it with
your UPnP client device. Before you continue, you should point the
server to one or more content directories via the configuration
web-page, unless the content is available from subdirectories of the
server' working directory.

To open the server's configuration web-page, open the following url in a web
browser:

http://127.0.0.1:9000/config

Saturday, October 31, 2009

network sniffing with a MacBook as intermediary

I bought a new network aware AV receiver for my home theatre the other week:
Pioneer SC-25

And I was trying to figure out how the Pioneer was communicating over the internet. Since there is no console to the receiver, the only way I could figure out what is was doing was to capture packets as they entered and exited the ethernet port of the device. Unfortunately, I didn't have a hub. A hub would allow me to connect the receiver and a laptop with Wireshark running in order to see the packets in and out of the receiver:
http://www.smallnetbuilder.com/lanwan/lanwan-howto/30305-packet-captures-and-network-devices

What to do?

Then I remembered that my MacBook has an Internet Sharing feature. You can share your Internet connection from either the AirPort or the ethernet port on the MacBook:
http://docs.info.apple.com/article.html?path=Mac/10.5/en/8156.html

The Internet connection on my MacBook is provided by wireless, so theoretically, I should be able to:
1) share my MacBook's internet connection
2) connect the receiver to the ethernet port on the laptop for the shared 'net connection
3) have Wireshark sniff the ethernet port as the receiver's packets pass through it

After enabling Internet Sharing on the ethernet port, I connected the ethernet cable coming from the receiver to the MacBook's ethernet port. I fired up Wireshark, started streaming music from the Pioneer and sure enough, there were packets flying through the Interface monitor in Wireshark! Very cool.


Note: one problem you might see when you first start using Wireshark is this error:
/dev/bpf0: Permission denied

Just do give read permissions to that ethernet device:
chmod 644 /dev/bpf0

cheers,
TAG

Monday, October 26, 2009

new home theatre, courtesy of Pioneer SC-25

As I had a ten year old Denon 3801, it was high time I replaced my AVR with one with more modern features. Based on its feature set as compared to comparable Denon's or Onkyos, I concluded that the SC-25 was my best choice. I put together this short video of the SC-25 setup process for those who might be interested.


enjoy,
sodo

Tuesday, October 06, 2009

win7 rc1 and mac time machine fun

It had been a pretty fun-less summer round the Guy's house, with lots of work, a tough economy and aging parent problems. So, I finally said to myself, "Man, I haven't played a video game in a long time. Maybe I'll pick up Wolfenstein." This simple act led me down a very busy path the last couple weekends..

Aging Box Won't Cut It
My six year old Dell SC400 would not do the game justice, so I needed an alternative. Should I invest in a new core i7 box? Pricing one out on newegg with a ASRock board and an i7 920 chip came to almost $1000. I don't want to spend $1000 right now. But maybe I could install Vista 64-bit on the dual, quad core SC1430? Last time I tried installing a new OS on that box about a year ago, I learned that you can't install on IDE drives. So I needed a SATA drive. Luckily, I've since bought a couple of SATA drives and had a 500 gigger laying around. I also had a Vista 64-bit install disk, so it looks like I had all I needed to get started..

I could bore you with a lot of details about what has transpired over the last two weeks. Instead, I'll just bullet point the high points. Feel free to comment or ask where appropriate. Suffice it to say that the last two weekends have not been all that relaxing! :)

Last weekend
- installed Vista 64-bit and Wolfenstein. The OS ran fast on the dual, quad beast. Also, Wolfenstein's fun..just like the last one with better graphics.
- upgraded to Windows 7 RC1 (7057)
- Win7s performance went in the toilet, something must have been wrong. I blew it away and reinstalled Vista 64-bit
- backed up Vista to 500GB via Acronis (Seagate) Disk Wizard (six hours). Successfully test booted the backup.
- tried the Win 7 compatibility test. Just silly..it only looks at CPU/mem/video card specs. I was hoping it would look at your hardware and actually do a full compatability test that would tell you if the drivers for a specific card were available or not (listen up, Microsoft!). But that makes too much sense. Oh well.
- did Windows 7 RC1 (7147) full install/overwrite. Worked fine, as opposed to the upgrade, which did not.

This past week
- spent days this week trying to get Media Center working my tuner card.
- Analog cable works over NTSC
- ATSC with an antenna doesn't work, though it did in Vista Media Center. Weird.

This weekend
Win 7 work
- I can't install Win7 on my SATA RAID cause I ran out of drive bays in the SC1430
- alternative would be to move the drive to an external DVD, thus freeing up a bay
- Win 7 suddenly stops working and blue screen of death with wimFsf.sys PAGE_FAULT_IN_NONPAGED_AREA error (not sure why)
- did all sorts of things to figure out what was happening (tried rebooting with DVD, removed PCI cards one by one, performed memtest)
- had to remove NVidia card and connect onboard video to get boot DVD to work and repair system
- repair actually worked and Win7 booted again
- not sure why this happened
- backed up Win7 RC1 to my old 120MB notebook drive via Acronis & tested the backup (took 20 hours)
- while removing and replacing cards, I accidentally cut the wire responsible for the intrusion detection switch.
- Thus, I had to head to Home Depot to get a wire stripper, strip the wire ends, thread them back together and wrap with electrical tape. Irritating.

I think I must have opened my SC1430 and rebooted over 50 times these past couple of weekends

Mac Work
- prepped Mac for second profile for girlfriend to use Mac for her iPod syncing
- reused the 500GB to do a Time Machine backup of Macbook
- Time Machine worked well. It completed the initial back up of 170GB in about four hours.
- created a user profile for her on the Mac
- pulled the files off her iPod Nano and network copied them to her new profile
- sync'd her iPod with the Mac
- my profile seems corrupt (permissions errors, can't do SW updates from it), so I used her profile to install the latest SW updates (10.5.8, new iTunes, etc)
- Mac OSX VNC still preventing me from logging out of my profiles. Had to disable sharing
- upgraded her iPod's firmware
- noticed my Macbook was full of crap, so I deleted 50GB worth of shat.
- redid the Time Machine backup
- noticed mcd on the mac was taking up 90% of the CPU (stopping Spotlight and unplugging the TM backup drive seemed to stop this)

Still to do
One last try this week: move box down near windows instead of up in loft.

Looking at the work I've done over the past couple weeks, a number of trends appear:
- got my systems' backed up (Mac/Win7)
- tried a new OS (Win7)
- built a sandbox on the Mac for my girlfriend to start using the Mac
- played a game for about two hours :)

Not bad for two weekends of struggle.
TAG

Thursday, June 11, 2009

wake on lan on the SC1430

Wake On Lan..she is a working on the SC1430 now, after a bit of sturm unt drang:
http://en.community.dell.com/forums/p/19278497/19501270.aspx#

Tuesday, March 31, 2009

Fedora 10, xrandr and InFocus projector

So, I'm doing this presentation at the Trenton Computer Festival next month and couldn't get the InFocus IN2102EP Projector to work on my dual boot MacBook Pro (2.33Ghz Core 2 Duo) that runs both Leopard and Fedora 10, x86-64 using ReFit as the boot loader. I tried to connect the InFocus to the DVI output of the Mac while running under Fedora.

The InFocus is connected this way:
MacBook external DVI output > DVI-VGA dongle > VGA - M1-DA Cable
> InFocus M1-DA connector

The problem is that though xrandr sees the DVI-0 port as connected, but the InFocus doesn't get the signal or spring to life through the setup listed above.

I've verified that the DVI-0 output works with a regular DVI cable connected to the DVI input on my Dell 1907FP 19-inch Flat Panel LCD. So this means a digital signal is being output from the port. Now, I've also verified that the DVI-0 output from Fedora does NOT work with the DVI-VGA dongle connected to the VGA input on the Dell. When connected with the DVI-VGA setup listed above, xrandr sees that the InFocus is connected (shown below). Also, Gnome sees the InFocus in the connected displays.

On the flipside, I've verified that the InFocus projector works using the DVI-VGA cable setup when booted from the Mac OS. So that's odd.

Since I can see the output while connected to the Dell's DVI input, I don't think this is a configuration issue. But maybe Fedora just doesn't like the DVI-VGA dongle. I'm sure I could buy an M1-DA direct to DVI adapter and be done with it, but I'd rather save the $55 and figure out why the Mac sees the Infocus, but why Fedora has troubles.

Here is the output of xrandr:
[cm@cm ~]$ xrandr
Screen 0: minimum 320 x 200, current 2464 x 900, maximum 2720 x 1924
LVDS connected 1440x900+0+0 (normal left inverted right x axis y axis)
331mm x 207mm
1440x900 60.0*+
2048x1536 60.0
1920x1440 60.0
1856x1392 60.0
1792x1344 75.0 60.0
1600x1200 85.0 75.0 70.0 65.0 60.0
1400x1050 74.8 60.0
1280x1024 85.0 75.0 60.0
DVI-0 connected 1024x768+1440+0 (normal left inverted right x axis y axis)
0mm x 0mm
1024x768 60.0*+ 85.0 75.1 75.0 70.1 60.0*
1400x1050 60.0
1280x1024 75.0 60.0
1024x768_60.00 60.0
1280x960 60.0
1152x864 75.0
832x624 74.6
800x600 85.1 72.2 75.0 60.3 56.2
640x480 85.0 72.8 75.0 72.8 75.0 60.0
720x400 85.0
640x400 85.1
640x350 85.1
0x0 0.0


Here is my xorg.conf:
Section "ServerLayout"
Identifier "Default Layout"
Screen 0 "Screen0" 0 0
InputDevice "Keyboard0" "CoreKeyboard"
InputDevice "Synaptics Touchpad" "CorePointer"
EndSection

Section "Module"
Load "synaptics"
Load "extmod"
Load "dbe"
Load "glx"
EndSection

Section "InputDevice"
Identifier "Synaptics Touchpad"
Driver "synaptics"
Option "SendCoreEvents" "true"
Option "Device" "/dev/input/mice"
Option "Protocol" "auto-dev"
Option "SHMConfig" "true"
Option "LeftEdge" "10"
Option "RightEdge" "1200"
Option "TopEdge" "10"
Option "BottomEdge" "370"
Option "FingerLow" "10"
Option "FingerHigh" "20"
Option "MaxTapTime" "180"
Option "MaxTapMove" "220"
Option "SingleTapTimeout" "100"
Option "MaxDoubleTapTime" "180"
Option "LockedDrags" "off"
Option "MinSpeed" "1.10"
Option "MaxSpeed" "1.30"
Option "AccelFactor" "0.08"
Option "TapButton1" "1"
Option "TapButton2" "3"
Option "TapButton3" "2"
Option "RTCornerButton" "0"
Option "RBCornerButton" "0"
Option "LTCornerButton" "0"
Option "LBCornerButton" "0"
Option "VertScrollDelta" "20"
Option "HorizScrollDelta" "50"
Option "HorizEdgeScroll" "0"
Option "VertEdgeScroll" "0"
Option "VertTwoFingerScroll" "1"
Option "HorizTwoFingerScroll" "1"
EndSection

Section "InputDevice"
Identifier "Keyboard0"
Driver "kbd"
Option "XkbModel" "pc105"
Option "XkbLayout" "us"
EndSection

Section "Device"
Identifier "ATI Technologies Inc M56P [Radeon Mobility X1600]"
Driver "radeon"
Option "backingstore" "true"
Option "RenderAccel" "true"
Option "AccelMethod" "EXA"
Option "Monitor-LVDS" "Internal Monitor"
Option "Monitor-DVI-0" "External DVI"
BusID "PCI:1:0:0"
EndSection

Section "Monitor"
Identifier "Internal Monitor"
HorizSync 30.0 - 110.0
VertRefresh 50.0 - 150.0
Option "DPMS"
EndSection

Section "Monitor"
Identifier "External DVI"
Modeline "1024x768_60.00" 108.88 1280 1360 1496 1712 1024 1025 1028
1060 -HSync +Vsync
Option "Enable" "true"
EndSection

Section "Screen"
Identifier "Screen0"
Device "ATI Technologies Inc M56P [Radeon Mobility X1600]"
DefaultDepth 24
SubSection "Display"
Depth 24
Virtual 2720 1924 # <--- This is the newly added entry
EndSubSection
EndSection

Section "DRI"
Mode 0666
EndSection


Alex Deucher on the Xorg mailing list was able to help me:
> > > If you are using F10 with kernel modesetting (KMS) it doesn't have the
> > > proper quirks do deal with the broken connector table on the macbooks.
> > > I think the non-KMS code shipped with F10 should have the proper
> > > quirk handling, if not, you may need to upgrade your ddx.

Kernel modesetting on Fedora systems allows the different graphics modes that are specified in the xorg.conf file to be set by the kernel, rather than the Device Dependent X (DDX) driver in the X server. More info here:
http://fedoraproject.org/wiki/Features/KernelModesetting
http://dri.freedesktop.org/wiki/DDX

So, as per Alex's comment, I did a workaround and used the "nomodeset" kernel parameter option on kernel command line in grub.conf to disable KMS. So now, the display wakes up when Gnome starts.

Hooray!
TAG

Xorg/Xrandr References
Xorg Info Page
XRandr Wiki
Open Solaris on MacBook Pro
XRandr on Debian

Saturday, March 07, 2009

upgrading my macbook pro system drive

Lately, I've been preparing a talk on Linux video production in linux that I will be giving at the end of April. More or less, it will be a technology demo that shows how to accomplish various video related tasks in Linux:
-capturing a video (acquisition)
-importing the output from the camera via firewire
-basics of file formats
-editing
-rendering
-scripting
-saving to a final destination (iPod/DVD/Vimeo)

So there will be a lot of ground to cover in the hour or so I will be speaking. I considered the mechanics of my presentation and first thought it would be cool to use my JVC HD10U to capture video of the conference which I'd then edit in Cinelerra. I had Fedora 10 already running in a virtual machine via VMware Fusion on my MacBook Pro 2.33Ghz Intel Core 2 Duo.

This method posed one big hurdle: VMware does not support firewire. VMware supports USB, but my camera only transmits high def video over firewire. Thus, I would not be able to capture live video from the camera during the talk. Conceivably, I could use a USB camera, but the quality would be so piss poor as to be embarrassing. So the VMware solution was nixed. The best solution was to dual boot my MacBook. I had my marching orders: I had to figure out how to dual boot my MacBook.

As expected, I hit a few speed bumps along the way, as the TechAnswerGuy always does. And as usual, you get to learn from my mistakes. Here we go..

Summary
Here was the high level plan:
1) upgrade my system drive to a 320GB 7200RPM Seagate Momentus
I was running out of space on the original 120GB drive, so this would be fun.
2) partition that fat new system drive to have a 240GB partition for Mac OSX and leave the 60GB unused space for Fedora
3) use SuperDuper to backup all important files to the Mac partition
4) use either Boot Camp or rEFIt to present a nice dual boot menu when the MacBook boots
5) configure the Fedora 10, x86-64 instance with a working copy of Cinelerra, dvgrab and all the media goodies necessary to make a slam-bang presentation come alive

Devil in the Details
In order to upgrade my system drive, I needed to migrate the data from the old drive to the new drive. Lucky for Mac users, David Nanian has written a wonderfully simple app called SuperDuper that creates bootable backups of Mac system drives. I prefer the word clone, as it refers to the fact that the backup also makes the drive bootable. Semantics aside, the result is the same.

Since my system drive is the drive that came installed in the MacBook, I needed to hook up the new 320GB Seagate I had bought. This was done using a very neat little device, a non enclosed disk enclosure called the Thermaltake BlacX eSATA Hard Drive Docking Station that I had bought at the same time for this express purpose.

Thermaltake BlacX..Sweet!
What a simple, easy little device to use! It is like a little USB disk drive bucket for both 2.5" and 3.5" disks. You just insert a disk connector side down into this bad boy, plug it in via USB into your PC or MAC, turn it on and voila, instant access to your storage! No fussing with screws or metal sleeves. It's cheap too, low $40s. After using enclosures for years, it is nice to not have to pick up a screwdriver to get access to my storage.


Once I plugged in the new drive into the BlacX and turned it on, it was time to 1) partition the empty drive and 2) do the SuperDuper backup. Partitioning the drive with the OSX Disk Utility was fairly easy. I accepted the defaults for the partition:
* Mac OS Extended partition (Journaled)
* install Mac OS 9 drivers

Partition Hell
Of the 300GB usable out of the drive, I allocated 240GB in the Mac partition and 60GB Free Space to land my Fedora 10 system. One very important thing that I needed to select is under Options:
select GUID Partition Table. Initially, I had taken the defaults settings and the defaults create a drive that uses the older Apple Partition Map drive partitioning scheme:


Do NOT use APM because you will get errors in both Boot Camp and rEFIt. The Boot Camp error went something like this:
Startup disk cannot be partitioned or restored to a single partition.
Backup the disk and use Disk Utility to format it as a single Mac OS Extended (Journaled) Volume.


The above error is very misleading, because when you look at the format of your partition, the partition is formatted in the correct way. But at the bottom of Disk Utility's Partition screen for the drive, the information about the drive will be listed and the key piece is "Partition Map Scheme". If Apple Partition Map is there, you won't be able to use Boot Camp or rEFIt. In my case, I had to repartition the new drive again to create a partition that had a Partition Map Scheme of GUID Partition Table. Here's a short guide that explains how to convert a drive from APM to GUID.

Long story short, you'll have to blow away whatever is on the disk to do the conversion. So be advised.

SooperDooper Indeed!
Once I had resolved the Partition Map problem, I moved onto backing up my sensitive data to the new drive with SuperDuper. There are a number of guides out there and SuperDuper is really easy to use. It took about three hours to copy 100GB of data from my currently internal old system drive to the new 320GB, 7200RPM hard drive that was plugged into the Thermaltake.

Quid Pro Quo
The next step was to take apart the MacBook and switch out the old drive for the new. This was non-trivial, required a delicate touch and took about an hour.
http://www.extremetech.com/article2/0,2845,2119529,00.asp

rEFIt
As I am all for open source solutions, I decided to use rEFIt as my boot manager. rEFIt also provides a clean way to install another OS. In point of fact, you don't need rEFIt to boot into a second operating system on your MacBook. Simply hold down the Option button as your MacBook is booting, and you will see OSX's built in system volume chooser.

Once I installed rEFIt, I rebooted the system and I saw the rEFIt boot manager appear. I booted a second time and inserted my Fedora 10, x86-64 install DVD and when the rEFIt boot manager appeared, I now had a second choice appear in the menu. That of Tux!


Familiar Territory
After that, the Fedora install process was the usual. One note: make sure to look at the disk properties in the disk configuration section of the install. Even though Fedora deselects any Mac partitions, it is good to double check this. The Fedora installer make good choices for the partitioning, given that my newly installed system disk in the Mac was found as /dev/sda:
* Fedora automatically deselected the first two partitions in /dev/sda as they were Mac partitions
* Fedora correctly assigned /dev/sda3 as the boot partition

Nice. After confirming Fedora's correct selected defaults for the partitioning layout, I then selected the base installation plus Developer Tools. This selection, plus the dependencies for Cinelerra build resulted in about 5.1GB used on the disk. Not too porky. I went to breakfast and the install finished in about 30 minutes.

After the system restarted, I found some troubles with the Mac's touchpad. I resolved them with the synaptics driver and some xorg.conf config, but I will detail that in a second post.

I will try to fill in some of the details of this long winded post with pictures to break up the monotony of the text.

TAG

Other References
http://www.macgeekery.com/tips/cli/nondestructively_resizing_volumes
http://ubuntuforums.org/archive/index.php/t-766172.html
http://www.shirt-pocket.com/forums/showthread.php?t=3282
http://forum.onmac.net/showthread.php?t=2793

Sunday, February 22, 2009

extending a logical volume in Fedora

I have a Fedora 10, x86-64 installation running under VMware Fusion on my Mac. Today, I ran out of disk space on my root partition. My root partition sits on a logical volume. In the "df" output below, note that I only have 92MB left on my / drive:
[root@ogre ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00 5.7G 5.4G 92M 99% /
/dev/sda1 190M 30M 151M 17% /boot
tmpfs 543M 0 543M 0% /dev/shm
/dev/sr0 96M 96M 0 100% /media/GParted-live


Oh no!

Research Necessary
Now, I haven't worked with logical volumes in quite some time, so I had to do a bunch of reading to figure out what the heck to do. These documents were of immeasurable help:
A Beginner's Guide to LVM
VMware Fusion - Expanding a Disk
Resizing LVM Volumes in Linux

Summary
In a nutshell, here are the steps necessary to expand the amount of space in your logical volume on Fedora. The first step is the allocation of new space in the context of VMware, but you could easily substitute the installation of a new hard drive:
1) In VMware Fusion, under Virtual Machine -> Settings -> Hard Disk, expand the size of your disk to the desired amount. I increased my disk space allocation from 8GB to 10GB:


2) Make a new partition using unallocated space
3) Create a new physical volume from the new partition that was just created
4) Extend the volume group into the new physical volume that was just created
5) Note the amount of free space in the volume group
6) Extend the amount of space in the logical volume using the value of free space
7) Resize the filesystem on the logical volume
8) Note the increased available space in the filesystem on the logical volume
9) Drink beer, you've earned it!

Details
##########
#2) MAKE A NEW PARTITION USING UNALLOCATED SPACE
##########
[root@ogre ~]# parted
GNU Parted 1.8.8
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sda: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 206MB 206MB primary ext3 boot
2 206MB 8587MB 8382MB primary lvm

(parted) mkpart primary 8588 10700
(parted) print
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sda: 10.7GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 206MB 206MB primary ext3 boot
2 206MB 8587MB 8382MB primary lvm
3 8587MB 10.7GB 2113MB primary

(parted) quit
Information: You may need to update /etc/fstab.

##########
#3) CREATE A NEW PHYSICAL VOLUME FROM THE NEW PARTITION THAT WAS JUST CREATED
##########
[root@ogre ~]# lvm
lvm> pvcreate /dev/sda3
Physical volume "/dev/sda3" successfully created
lvm> pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 VolGroup00 lvm2 a- 7.78G 32.00M
/dev/sda3 lvm2 -- 1.97G 1.97G

##########
#4) EXTEND THE VOLUME GROUP INTO THE NEW PHYSICAL VOLUME THAT WAS JUST CREATED
##########
lvm> vgextend VolGroup00 /dev/sda3
Volume group "VolGroup00" successfully extended
lvm> pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 VolGroup00 lvm2 a- 7.78G 32.00M
/dev/sda3 VolGroup00 lvm2 a- 1.94G 1.94G

##########
#5) NOTE THE AMOUNT OF FREE SPACE IN THE VOLUME GROUP
##########
lvm> vgdisplay VolGroup00
--- Volume group ---
VG Name VolGroup00
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 4
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 2
Act PV 2
VG Size 9.72 GB
PE Size 32.00 MB
Total PE 311
Alloc PE / Size 248 / 7.75 GB
Free PE / Size 63 / 1.97 GB
VG UUID H89xDs-yTqE-y2YZ-ORVq-idgW-HdGf-hHU8m3

##########
#6) EXTEND THE AMOUNT OF SPACE IN THE LOGICAL VOLUME USING THE VALUE OF FREE SPACE
##########
lvm> lvextend -l+63 /dev/VolGroup00/LogVol00
Extending logical volume LogVol00 to 7.75 GB
Logical volume LogVol00 successfully resized
lvm> lvs
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
LogVol00 VolGroup00 -wi-ao 7.75G
LogVol01 VolGroup00 -wi-ao 1.97G
lvm> quit
Exiting.

##########
#7) RESIZE THE FILESYSTEM ON THE LOGICAL VOLUME
##########
[root@ogre ~]# resize2fs /dev/VolGroup00/LogVol00
resize2fs 1.41.3 (12-Oct-2008)
Filesystem at /dev/VolGroup00/LogVol00 is mounted on /; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/VolGroup00/LogVol00 to 2031616 (4k) blocks.
The filesystem on /dev/VolGroup00/LogVol00 is now 2031616 blocks long.

##########
#8) NOTE THE INCREASED AVAILABLE SPACE IN THE FILESYSTEM ON THE LOGICAL VOLUME
##########
[root@ogre ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
7.7G 5.4G 2.0G 74% /
/dev/sda1 190M 30M 151M 17% /boot
tmpfs 543M 0 543M 0% /dev/shm
/dev/sr0 96M 96M 0 100% /media/GParted-live

##########
#9) DRINK BEER!
##########


Hooray! Now I won't run out of space in my VM!
TAG

vmware tools install in Fedora 10

Here are some quick notes to installing VMware Tools in Fedora 10 in Fusion. My particular architecture is x86-64, but these should work for regular x86 installs:
1) yum groupinstall "Development Tools"
2) yum install kernel-devel* kernel-headers*
3) run ./vmware-install.pl as root

Also, I had one problem where the kernel-devel* and kernel-headers* package did not match my kernel, which was one revision lower than my installed kernel. The vmware-install.pl program told me this when I tried to run it. The solution to this was to simply update my kernel:
yum install kernel

TAG

pulseaudio and Fedora 10 don't get along

Working with my Fedora 10, x86-64 VM running under Fusion on my MacBook Pro was an exercise in frustration today:
- pulseaudio was spitting out noise and I couldn't uninstall it
- VMware Tools wouldn't compile
- I was running out of disk space in the VM

PulseAudio Problem
The problem with PulseAudio and Fedora 10, x86-64 is that when you use any multimedia application that has sound, the pulseaudio driver just creates noise in your speakers. Also, any video stream associated with the audio stream will be corrupted or at a minimum, sped up or slowed down and generally unwatchable.

Solutions that should work, don't
Under System -> Preferences -> Personal -> Sessions, I unchecked the PulseAudio Sound System. Even with this change, my apps in Gnome kept using Pulse. I'm wondering if this problem is related to Gnome in the last three Fedoras not saving session information properly?

Also, reading the man pages for pulse-daemon.conf, pulse-client.conf and pulseaudio wasn't much help as it did not yield a simple solution for disabling it.

I couldn't remove pulseaudio, because the beast has hooks into everything:
[sodo@ogre ~]$ sudo yum remove pulseaudio*
compiz-gnome x86_64 0.7.8-7.fc10

control-center x86_64 1:2.24.0.1-12.fc10
gdm x86_64 1:2.24.0-12.fc10
gdm-user-switch-applet x86_64 1:2.24.0-12.fc10
gnome-applets x86_64 1:2.24.3.1-1.fc10
gnome-panel x86_64 2.24.3-1.fc10
gnome-session x86_64 2.24.3-1.fc10
gnome-session-xsession x86_64 2.24.3-1.fc10
gnome-settings-daemon x86_64 2.24.1-7.fc10
gstreamer-plugins-good x86_64 0.10.11-4.fc10
libcanberra x86_64 0.10-3.fc10
libcanberra-gtk2 x86_64 0.10-3.fc10
mencoder x86_64 1.0-0.103.20080903svn.fc10
mjpegtools x86_64 1.9.0-0.6.rc3.fc10
mjpegtools-gui x86_64 1.9.0-0.6.rc3.fc10
mplayer x86_64 1.0-0.103.20080903svn.fc10
orca x86_64 2.24.3-1.fc10
plymouth-gdm-hooks x86_64 0.6.0-0.2008.11.17.3.fc10
totem x86_64 2.24.3-1.fc10
totem-gstreamer x86_64 2.24.3-1.fc10
totem-mozplugin x86_64 2.24.3-1.fc10
totem-nautilus x86_64 2.24.3-1.fc10
vlc x86_64 0.9.8a-1.fc10
vlc-devel


A Working Solution!
The pulseaudio* removal wasn't going to work, as it was going to remove a helluva lot of dependent programs. However, a different syntax for the removal of pulseaudio seemed to work a little better:
[sodo@ogre ~]$ sudo yum remove pulseaudio

Without the asterisk (*), yum removed only pulseaudio programs and not any dependencies. So it was just a tweak to the yum remove syntax that did the trick. Thank God. Now my videos don't break up!

FYI - As a last resort, if none of these removals work, just rename the binary to something else:
sudo mv /usr/bin/pulseaudio /usr/bin/paudio

That way, the system won't be able to find it. Of course, this isn't the preferred method of disabling pulseaudio!
;)
TAG

Reference
http://forums.fedoraforum.org/showthread.php?t=213711
An untested solution:
http://forums.fedoraforum.org/showthread.php?t=206868
Feel free to drop me a line or ask me a question.