Wednesday, February 28, 2007

installing a Perl module

So shoot me. I haven't done this in a long time. I was trying to install XML::Writer and forgot how to install a Perl module. Here are the steps:

1) download the module as a gzipped tar
2) uncompress the file:
[root@computer Desktop]# tar zxvf XML-Writer-0.602.tar.gz
XML-Writer-0.602/
XML-Writer-0.602/Writer.pm
XML-Writer-0.602/META.yml
XML-Writer-0.602/Changes
XML-Writer-0.602/t/
XML-Writer-0.602/t/pod-coverage.t
XML-Writer-0.602/t/pod.t
XML-Writer-0.602/t/01_main.t
XML-Writer-0.602/MANIFEST
XML-Writer-0.602/TODO
XML-Writer-0.602/README
XML-Writer-0.602/Makefile.PL

Change into the install directory and create the Makefile
[root@computer Desktop]# cd XML-Writer-0.602
[root@computer XML-Writer-0.602]# perl Makefile.PL
Checking if your kit is complete...
Looks good
Writing Makefile for XML::Writer

Run "make" to create the module
[root@computer XML-Writer-0.602]# make
cp Writer.pm blib/lib/XML/Writer.pm
Manifying blib/man3/XML::Writer.3pm

Run "make test" to test your new module
[root@computer XML-Writer-0.602]# make test
PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/01_main.........ok
t/pod-coverage....skipped
all skipped: Test::Pod::Coverage required for testing pod coverage
t/pod.............skipped
all skipped: Test::Pod 1.00 required for testing POD
All tests successful, 2 tests skipped.
Files=3, Tests=213, 1 wallclock secs ( 0.54 cusr + 0.43 csys = 0.97 CPU)

Run "make install" to install the module in its proper place
[root@computer XML-Writer-0.602]# make install
Installing /usr/lib/perl5/site_perl/5.8.6/XML/Writer.pm
Installing /usr/share/man/man3/XML::Writer.3pm
Writing /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi/auto/XML/Writer/.packlist
Appending installation info to /usr/lib/perl5/5.8.6/i386-linux-thread-multi/perllocal.pod

that's it!

Sunday, February 18, 2007

MediaGate Ximeta NDAS drivers for Linux .. what crap!

For the past two weekends, I have been doing a lot of experimentation lately trying to get the Ximeta NDAS drivers to work. As I do my video editing in Linux, the goal was to be able to wirelessly mount the hard drive in the MediaGate while it sits next to my home theatre. This would allow me to be able to script the entire edit, copy and distribute process within Linux. The end result of this process is to be able to view my HDV video creations on my HDTV.

To be impolite, what a cluster f*** these drivers are. I first tried to install the RPMs and build from source on my main Fedora Core 4 system. I got this error:
- unable to handle kernel NULL pointer dereference at virtual address

I submitted the error (http://code.ximeta.com/trac-ndas/ticket/434)
and an engineer was kind enough to respond back; however, he stated that they won't support FC4. OK. No help there. So, I went on a quest to find a Linux distro that would work by either installing the prepackaged RPMs that Ximeta provides or by compiling the drivers myself. During my quest, I went through five Linux distros (Ubuntu 6.06, 6.1.0, Red Hat Enterprise Edition AS v3.0, Fedora Core 4 and Core 6). This involved the usual: download 600MB+ iso, burn to CD or DVD and install.

In short, NONE of the RPMs Ximeta provided worked with these OSs. They failed for various reasons:
- failure to mount the NTFS drive
- failure to load sal module

The trial-and-error with the various Linux distros was brutal. However, I got a bit further yesterday as I compiled from source on Fedora Core 6 and found this error:
- linux/config.h no such file or directory

Luckily, there was a tech tip on http://code.ximeta.com/trac-ndas/wiki/FedoraCore6 in the wiki that suggested to remove that line. I remove the line, recompiled and custom RPMs were created here:
[gagazote@demon i686]# pwd
/usr/src/redhat/RPMS/i686
[gagazote@demon i686]# ls
ndas-admin-1.0.3-101.i686.rpm
ndas-kernel-1.0.3-2.6.18_1.2798.fc6.101.i686.rpm

Once I had the compiled drivers installed, I examined and mounted the device through these series of commands:
[root@demon ~]# ndasadmin register XXXXX-XXXXX-XXXXX-XXXXX-YYYYY -n MediaGateWireless
where XXXXX is the device id, YYYYY is the write key and -n is the NDAS name

One of the two following commands will show that the device is recognized:
[root@demon ~]# cat /proc/ndas/devices/MediaGateWireless/slots
1
[root@demon ~]# cat /proc/ndas/devs
Name ID Key Serial Version Online Slots
MediaGateWireless MRLDQAJV9UVKDD6***** Yes 00003630 1 Yes 1


Then, I enabled the drive properly using the ndasadmin commands. Here is the command to enable the device as read/write:
[root@demon ~]# ndasadmin enable -s 1 -o w
Block device /dev/ndas-0000363032811569:0 is ready to use.


By the way, if you have contention conflicts with another computer trying to access the MediaGate, you'll receive a message like this in Linux:
[root@localhost ~]# ndasadmin enable -s 1 -o w
enable: You did not provide a valid write key or slot 1 is exclusively accessed by other host.
Execute the following command to send the request for the write permission.
ndasadmin request -s 1


(As a sidebar, in order to disable the device, use this command:
[root@localhost ~]# ndasadmin disable -s 1
/dev/ndas-0000363032811569:0 is disabled
)

Finally, I created a mount point and mounted the wirelessly connected MediaGate:
[root@demon ~]# mkdir /mnt/ndas
[root@demon ~]# mount -t ext2 /dev/ndas-00003630328115 /mnt/ndas


From the fdisk -l output showing the mounted disk, Linux did not seem to be seeing the drive partitions correctly. The output (not shown) basically showed a blank partition table. What is wrong there? I thought that might be part of the problem, so I went ahead and reformatted the entire 80GB drive as a Linux ext2 primary partition.

Luck! I was able to connect to the drive and I saw a "lost+found" directory. Then, I copied a small file to the device and waited with bated breath as I turned on my entertainment center to see if the device recognized the hard drive. Alas, no dice. I re-read the printed documentation and it said that only FAT and NTFS drives work with the MediaGate. OK. I'm a fathead, I should have read those directions beforehand.

So, back to fdisk I go. This time, I took the simple route and used the XP Ximeta drivers to mount and format the disk. Again, I did this task over the wireless connection. The format took about an hour. After the format, I went back to my Fedora 6 install in order to mount the NTFS partition read/write. Write access to NTFS in Linux is something fairly new and it is supported by using the ntfs-3g (http://www.ntfs-3g.org/) and fuse (http://fuse.sourceforge.net/) libraries. Mounting the drive gave me the following error:
[root@demon ~]# ntfs-3g /dev/ndas-00003630328115 /mnt/ndas
Failed to open inode $MFTMirr: Input/output error
Failed to load $MFTMirr: Input/output error
Failed to startup volume: Input/output error
Failed to mount '/dev/ndas-00003630328115': Input/output error NTFS is inconsistent. Run chkdsk /f on Windows then reboot it TWICE! The usage of the /f parameter is very IMPORTANT! No modification was made to NTFS by this software.


OK. Well, when I ran fdisk -l again to find out what partition the OS thinks it saw. Fedora reported four separate partitions on that one drive! What is going on? Is this simply the way an XP formatted NTFS partition looks to Linux? Or maybe something weird happened with formatting the drive wirelessly? In any case, I yanked the drive from the MediaGate, plugged it in to an available box and formatted it as NTFS directly via XP. I then put the newly formatted drive back into the MediaGate. My fingers were crossed when I used my Fedora 6 install to mount the drive again. Again, I was confronted with disappointment. Same error as above! Argh! I then opened a ticket with Ximeta on this error, but I am not hopeful given their previous response and responses to other people on the wiki. (It is nice that they provide Linux drivers for us, but for God's sake Ximeta, test your drivers!!)

After reading the manual again, I found that the MediaGate should work with both FAT32 and NTFS drives. Now, the real purpose of this exercise was to be able to copy large video files (sometimes 4GB+) over to the MediaGate to view on my HDTV directly from Linux. I understand that formatting the drive as FAT32 would limit me to 4GB files. But at this point, I just wanted something to work. Please, Lord!!

Since the drive and partition was recognized in Fedora, I was able to use fdisk to remove the current NTFS partition and create a new FAT32 partition. I then used mkdosfs to format the partition. Here are the commands:
mkdosfs
-c (check for bad sectors)
-F 32 (FAT size in bits)
-S (sector size)
-n (volume name)
-v (verbose)


My command line ended up looking like this:
mkdosfs -c -F 32 -S 16384 -n MediaGate -v

Once I created and formatted the partition, I tried to mount it as type vfat in linux. Fedora complained:
[root@localhost]# mount -t vfat /dev/ndas-00003630328115 /mnt/ndas
mount: wrong fs type, bad option, bad superblock on /dev/hdb1, or too many mounted file systems


What the F? Again, the format and partitioning was done via the MediaGate's wireless connection. This is the second format to go awry, so something must be wrong with this wireless method. So then I thought I would format the disk FAT32 the good old fashioned way..direct connect with a Windows 98 boot floppy disk! Simple, right? Wrong!!

In the fashion of this long-winded tale, this became a mini-nightmare in itself:
1) the floppy was disabled in the BIOS on my secondary machine, I needed to reset
2) after that reset, the floppy drive was either broken or just did not recognize the boot floppy. I tried multiple boot floppies. None worked.
3) booting the floppy using my primary PC was successful. I was able to use fdisk to create the FAT partition. Thank goodness. In order to format the drive, I needed to reboot. But after rebooting, the boot floppy stopped worked. No help via multiple warm and cold boots. YARG!!

Therefore, I resorted to the salvation of the harried admin, the Linux SysRescCD. But my drive was set to master and the DVD drive was also set to master. So I pulled the jumper on the hard drive and successfully booted up the SysRescCD. I then ran mkdosfs and formatted the drive as FAT32. This process seemed to work and I tested a mount right then and there. The mount worked and a file copy to the mount worked!! So it looked like I was good to go.

I pulled the drive, put it in the MediaGate, took it down to my home theatre and hooked it up. No dice. YEAAARGH! What the hell?! I re-read the manual and saw that the drive needed to be master. Ah so. Fathead rears his ugly noggin again. I had pulled the jumper in order for the drive to work with the DVD, so this should be a simple fix. I pulled the drive from the case, switched the jumper and plugged the unit back up to the TV. Thank goodness, the MediaGate recognized the hard drive this time. Now for the big test..would the Ximeta Linux drivers recognize and mount the drive wirelessly? More importantly, would I be able to copy files to it?!

I went back upstairs to my tech lair and watch with bated breath as I tried to mount the wirelessly attached NDAS drive:
[root@localhost]# mount -t vfat /dev/ndas-00003630328115 /mnt/ndas

My God! It worked! No errors as before! (This seems to indicate something is definetly awry when you try to format a disk wirelessly or with Fedora's fdisk and FAT formatting capabilities. Maybe both conspiring together?)

In any case, I then copied a 60MB file. That worked! I went downstairs to the home theatre, navigated to the hard drive and found the file there. I played the file. It worked!! FAN-TAS-TIC!! I went back upstairs and copied over an 800MB file. After a few minutes, I trudged downstairs and found that the file was listed on the hard drive. I pressed Play. It worked! Hooray! Now for the big test. I copied a 4.7GB file wirelessly from Linux to the FAT formatted drive on the MediaGate. This took about an hour. I saw that only 4.0GB was copied over, the FAT32 filesize limit. At this point, who cared? At least most of the file copied over. And since it was an MPEG2, partial files should play back just fine. But will it play? I pressed play and my HDV file played in gorgeous full resolution color! With a 4.0GB limitation, using the FAT partition may not be the optimum situation, but at least it works!

It has taken me two weeks to muddle through this. Why can't these device manufacturers TEST their code? Is that too much to ask? Infuriating.

After all, I must ask, why do I put myself through all of this?
:

Update 1/14/2008
On the brighter side, I was able to successfully upgrade the device firmware to version 1.1.7. This resolved some video playback problems for MPEG-TS files and includes an FTP server:
/2008/01/mediagate-mg350hd-v117-firmware.html

I did try to install 64-bit Linux NDAS drivers from Ximeta. But guess what? You go to the site and there are no drivers at all in the Linux 64 directory:
http://code.ximeta.com/dev/current/

Boo! Hiss! And just plain stupid.

Is it too much to ask that these device manufacturers make solid Linux drivers?

Update 1/30/2008
As well, using the NDAS drivers on my MacBook Pro, I tried formatting the drive as a Mac compatible disk. The format worked; however, the disk was not recognized when I restarted the device. Going into the Mediagate menus, I was only able to connect wirelessly. The hard drive icon was greyed out.

Update 2/6/2008
I've decided to forego the NDAS drivers entirely and use firmware version 1.1.7's FTP server. GOODBYE, NDAS!! Here's a link to a blog entry on v1.1.7:
/2008/01/mediagate-mg350hd-v117-firmware.html

Sunday, February 04, 2007

weeding out some unnecessary services from FC4

This post is mainly for my own notes. In trying to reduce the amount of programs that are loading on my FC4 system, I've removed the following:

bluetooth
captive
haldaemon * PUT BACK..BROKE GNOME
iptables
isdn
mDNSResponder
mdmpd
netfs
nifd
pcmcia
portmap * PUT BACK..BROKE NFS
rhnsd
rpcgssd
rpcidmapd
rpcsvcgssd
wine
xfs * PUT BACK..BROKE FONT SERVER

Saturday, February 03, 2007

ntfs-3g .. write reliably from Linux to NTFS partitions

My task today was to backup my videos, in case my bloody RAID stripe set goes kaput. I found this program ntfs-3g (http://www.ntfs-3g.org/) that allows me to write files from Linux partitions to NTFS partitions. In addition to installing ntfs-3g, you will also need to have FUSE (Filesystem in User Space) libraries installed. FUSE allows you to "implement a fully functional filesystem from a userspace program." Well, OK. I'm a little thick to get the purpose of that one right now. I just want my files to be backed up.

I got both pieces installed through the tarball configure/make/make install process. After the install, I copied a 6GB video file from my ext3 partition to an NTFS partition as a test. I then rebooted into XP and replayed it back in my XP based video editing software. I could still play the entire file, so I assume that ntfs-3g is doing its job. Hooray!

At the end of the evening, I copied 200GB of videos from my ext2 system to a backup NTFS partition. She works!
Feel free to drop me a line or ask me a question.