Showing posts with label 3ware 9650SE. Show all posts
Showing posts with label 3ware 9650SE. Show all posts

Friday, April 12, 2013

i7 3930K (LGA2011) build: Asus P9X79

I spent a good few weeks researching and reading up on my latest monster video editing box.  Though it was expensive, I decided to go for the i7 3930K chip, a pricey $539 on sale from NewEgg.  Listed below are the parts that I assembled for the box.  The mainboard and the chipset were my primary concerns, but the other parts were chosen because they conformed to my build specs and they were on sale or discounted in one way or another during one of NewEgg's Build sales.

The cost of the items weren't bad, but they certainly added up:










1 x ($-75.00) DISCOUNT FOR AUTOADD #76165
$-75.00

1 x ($-85.80) DISCOUNT FOR PROMOTION CODE
$-85.80

Subtotal:$1707.10
Tax:$120.06
Shipping and Handling:$7.99
Total Amount:$1835.15


Here's a pic of them fresh out of the shipping box:
Because I don't build too often, it took about a day to assemble the main board and the components in the above picture.  It took another 1/2 day to port my four drive RAID 1+0 set from my old video editing workstation to this one.  Of course, I had to create a new RAID set on the new box.  I will no longer be using the trusty 3Ware 9650SE RAID card, as the new mobo has both Intel and Marvell RAID chipsets on board.  The Marvell only supports two drives, so I used the Intel to connect up my four 1.5TB Western Digital Green drives.

As all my content (photos/docs/videos/music/virtual machines/etc) is on this redundant RAID set, I made sure to backup the ext4 filesystem that sits on top it with fsarchiver.  Secondly, I bought a 3TB drive as a utility drive to try out my fsarchive restores.  This is a good system, as it confirms my backup is valid:
1) unmount content filesystem (the filesystem that's on top of RAID)
2) backup content using fsarchiver to a system drive with enough space to hold the 1.4TB of content
3) as my livelihood depends on my content, test the fsarchiver restore process by restoring the archive to a third drive, a 3TB Seagate partitioned with GPT and formatted using ext4
4) when the restore is validated as good, copy the files from the restore over to my new RAID 1+0 set. 

I love the Corsair 400R case.  With the nice cable tunnels, I was able to keep the mainboard area pretty free of cables, though it doesn't look like it from the below pic.  You can see the pipes to the Corsair H100i in the center of the photo.





































I ran most of the cables through the tunnels.  You can see how many cables were hidden using these tunnels:





































I'm not a great cabler and I just wanted most of the cables out of the way.  Given that there are five 3.5" drives, one 2.5" SSD and a DVD player in the box, I think I did alright.

The real test was firing the box up for the first time.  I was rather shocked when it did come up, mainly for the worry that the memory wouldn't be compatible.  But it was.  Also, I had invested a great deal of time before the build in reading the 176-page manual and watching a bunch of build videos listed at the bottom of this post.  So the base build with a single SSD hard drive powered up successfully.  Hooray!

The next hurdle was installing a basic operating system, Fedora 18 64-bit on my single SSD in the machine.  Again, surprisingly, this worked without a hitch.  I had spent a good deal of time reading the FedoraProject's list of UEFI bugs and I was suspect of problems, but was very happy that I didn't encounter any.

The final hurdle was migrating my four drives from my old workstation to run a RAID 1+0 configuration on the new box using the ASUS's Intel RAID chipset.  Once I moved the drives over, I  configured the BIOS in the Asus P9X79 to run RAID.  I then created the RAID set and rebooted the box.  I then saw three main screens on bootup:
1) The Marvell RAID BIOS bootup screen:


















2) The Intel RAID BIOS bootup screen:


















3) The American Megatrends BIOS screen:


















The American Megatrends BIOS and ASUS's UEFI BIOS screens are completely configurable and nicely laid out.  I won't be using many of the options, but it is just a pleasure to have a system that is so well stocked, but boots up quickly.  I'd say it takes about 30 seconds to get from cold start to my initial Fedora 18 grub2 prompt.

I haven't overclocked the mobo yet, but according to what I've been reading and the fact that I have good chip cooling via the H100i, I should be able to push the i7 from 3.2 to at least 4.6Ghz.

Wattage Used for Typical Tasks
Here is a chart of the tasks running in Fedora 18 x86-64 and the wattage used








Gnome System Monitor graphic
On the top row, the CPU History chart, you can see that my H264 video encode took about 40% of the CPU.  You can see all twelve CPUs being used.  On the bottom row, the Network History, you can see my upload to Vimeo.  The upload of my 530MB file took about 150s at a peak upload rate of 40Mbps.  That 40Mbps upload is courtesy of FIOS.





























Next up: installing Windows 7 Professional so that I can do some baseline performance measurements.

ciao!
TAG

UPDATE 2013/11/23
I've been so busy at work for the last six months that I haven't had time to use the new box.  Turns out that I made a mistake in copying over my files to my new drive..I hadn't preserved the time/date stamp of all my files, normally accomplished with a "cp -Rp .."!  This means that I then can no longer tell when I edited any of my scripts, took a picture, etc, etc.  Bollucks!  So I had to grab the timestamps from my original drive and apply them via "touch".  The procedure looked something like this:

sudo find  -printf "%t %h/%f\n" > ~/temp.txt
awk '{print "touch -d \""$1,$2,$3,$4,$5"\"",$6}' temp.txt > fixDate.sh

temp.txt looked like this:
Thu Nov 22 12:37:26.0000000000 2012 ./sodo
Sat Jan 22 10:29:32.0000000000 2011 ./sodo/.netbeans
Sat Jan 22 10:29:32.0000000000 2011 ./sodo/.netbeans/.superId
Sat Jan 22 11:58:29.0000000000 2011 ./sodo/.netbeans/6.7

fixDate.sh looked like this:
#!/bin/bash -xv
/usr/bin/touch -d "Thu Nov 22 12:37:26.0000000000 2012" ./sodo
/usr/bin/touch -d "Sat Jan 22 10:29:32.0000000000 2011" ./sodo/.netbeans
/usr/bin/touch -d "Sat Jan 22 10:29:32.0000000000 2011" ./sodo/.netbeans/.superId
/usr/bin/touch -d "Sat Jan 22 11:58:29.0000000000 2011" ./sodo/.netbeans/6.

Then I plopped a #!/bin/bash at the top of the shell script, chmod a+x fixDate.sh, and off I went.  But the script worked, thankfully!  Unfortunately, I wasted about two hours dealing with this today.  Super drag!

A second problem I see is when I use "mv" to move files to my new RAID set, the mv command gives me:
mv: setting attribute ‘security.selinux’ for ‘security.selinux’: Operation not permitted

Haven't figured out why tho.  Argh.
*** end update ***


References
ASUS P9X79 Home Page: https://www.asus.com/Motherboards/P9X79_DELUXE/
Manuals/Qualified Vendor Lists/Firmware Updates: http://usa.asus.com/Motherboards/Intel_Socket_2011/P9X79_DELUXE/#download

JJ of ASUS Republic of Gamers (asusrog) was immeasurably helpful in the builds.  In fact, I probably wouldn't have bought the board if I hadn't seen these videos.  Here are the core videos that helped me:
X79 Overview, Part I: http://www.youtube.com/watch?v=qGgQG3ANtaA
X79 Overview, Part II: http://www.youtube.com/watch?v=iE-YJafM65k
P9X79 Deluxe Hands-On Review: http://www.youtube.com/watch?v=xBPgv8aiL0Y
Build, Part I: http://www.youtube.com/watch?v=j9aY1HGYIyM
Build, Part II: http://www.youtube.com/watch?v=W8jogtOzw6Y

Overclocking on Sandy Bridge
Auto Overclocking: http://www.youtube.com/watch?v=Ct6tQaEsWYY
Part I: http://www.youtube.com/watch?v=Kx2z07sFM2I
Part II: http://www.youtube.com/watch?v=seVPIR06ZY4
Advanced Overclocking: http://www.youtube.com/watch?v=CZfOEs5n0jo

Non-JJ vid on overclocking: http://www.youtube.com/watch?v=8qz8olA8bRM

RAM Cache/SSD Cache
How to Setup a RAM Cache on a P7X79: http://www.youtube.com/watch?v=qGrqMllHVVY
RamCache/Ramdisk: http://www.youtube.com/watch?v=5cqfhZvyE80
SSD Caching: http://www.youtube.com/watch?v=uhCGo0LWaTY
SSD Caching (short version): http://www.youtube.com/watch?v=hvfiRA6hDhE


Cable Management
http://www.youtube.com/watch?v=5xKIiAu4rio

Friday, February 05, 2010

3ware 9650SE support experience

I recently bought a RAID card and new drives for video editing:
3WARE Pci-e 9650SE RAID card with Battery Backup
four Western Digital 1.5 TB Green SATA 32MB Cache Hard Drive

I setup a RAID5 array with the four 1.5TB drives and installed Fedora 12, x86-64 on top. The RAID array is the boot drive and I've setup a GPT partition. However, I have a slight problem as the boot times on my Dell SC1430 have increased to about four minutes! This is an unacceptably long time for a boot sequence to finish.

Update 2011/10/24
The long-boot problem I experienced was because during the process of the 3ware install, I had to move the GPU (NVidia 8800GT) to a different PCIe (PCI Express) slot (the second one from the end).  For some reason, the Dell SC1430 that I have slows the boot time considerably when using that slot for the video card.  Moving the 8800GT from the second slot to the first PCIe slot resolved the boot time issue.
*** end update ***

Online Chat System
I opened a ticket with 3ware support via their online chat and after an hour of waiting, an engineer finally got on the chat and told me that I need to upgrade my controller card firmware version and Linux driver version to the latest and greatest. The online chat system was OK; however, I waited an hour, stepped away for five minutes and in those five minutes, the engineer responded to my query and promptly dropped off the line.

Luckily, there is a feature to have the chat session emailed to you. I did that and saw the response from the engineer. So that was good. But here's a note for 3ware..if your engineer's are too busy to leave their chat session open for more than five minutes, either retrain them to keep the session open for at least fifteen minutes or hire more support engineers!! It is frustrating for the customer to be dropped like that.

Secondly, I need confirmation about the availability of an rpm for F12, x86-64, so I hopped on the online chat again and waited 45 minutes. After waiting, the online chat just dropped me altogether without any answer from an engineer. Frustrating. 3ware needs to work on this system.

Multiple Software Versions
In any case, I needed to find my current driver version and controller card version. Just finding this information was a task in itself. The support information on 3ware's site is all there..it is just a big task just wading through it all. Also, there seems to be a lot of redundant and confusing information in the Linux section.

Here's another suggestion for 3ware..for 3ware newbies, there should be a simple introduction to the product explaining that the product includes multiple software versions:
-the firmware on the controller card
-the operating system driver
-the command line interface software version
-the software api version
-the web gui version


Basic Install on Linux
You want to install just two items:
1) the command line interface (tw_cli)
2) the 3DM2 GUI

As of this writing, the most recent install packages are these:

tw_cli-linux-x86_64-9.5.3.tgz
3DM2_CLI-Linux-x86_64-9.5.3.tgz



You'll unpack those files (tar xvf ).  If you run any flavor of Linux, that 3DM2 installer will drop a "setupLinux*.bin" file in the unpack directory.  You'll want to "chmod a+x setupLinux*.bin" and then execute that file.  In my case:
./setupLinux_x64.bin

Here are some screens to help you see the process:







Web GUI Gotcha
One of the other gotchas is that once you install the web GUI (3DM2), you'll need to set a browser exception of "localhost".  If you don't do that, you'll get "Webpage is not available" when you login to https://localhost:888.

Good Information from the Command Line client
9650SE controller card firmare version
[root@ogre ~]# tw_cli
//ogre> show

Ctl Model (V)Ports Drives Units NotOpt RRate VRate BBU
------------------------------------------------------------------------
c4 9650SE-4LPML 4 4 1 0 1 1 OK

//ogre> /c4 show

Unit UnitType Status %RCmpl %V/I/M Stripe Size(GB) Cache AVrfy
------------------------------------------------------------------------------
u0 RAID-5 OK - - 256K 4190.92 RiW ON

VPort Status Unit Size Type Phy Encl-Slot Model
------------------------------------------------------------------------------
p0 OK u0 1.36 TB SATA 0 - WDC WD15EADS-00P8B0
p1 OK u0 1.36 TB SATA 1 - WDC WD15EADS-00P8B0
p2 OK u0 1.36 TB SATA 2 - WDC WD15EADS-00P8B0
p3 OK u0 1.36 TB SATA 3 - WDC WD15EADS-00P8B0

Name OnlineState BBUReady Status Volt Temp Hours LastCapTest
---------------------------------------------------------------------------
bbu On Yes OK OK OK 255 06-Feb-2010

//ogre> /c4 show firmware
/c4 Firmware Version = FE9X 4.08.00.006


9650SE controller card serial number
[root@ogre ~]# tw_cli
//ogre> /c4 show serial
/c4 Serial Number = LXXXX10A9420XXX


9650SE Linux driver version
[root@ogre ~]# cat /sys/class/scsi_host/host4/stats
3w-9xxx Driver version: 2.26.02.012
Current commands posted: 0
Max commands posted: 29
Current pending commands: 0
Max pending commands: 0
Last sgl length: 2
Max sgl length: 32
Last sector count: 24
Max sector count: 256
SCSI Host Resets: 0
AEN's: 0


9650SE application software version
[root@ogre ~]# tw_cli
//ogre> show ver
CLI Version = 2.01.09.004
API Version = 2.06.01.006


Ticketing System
Also, there is a separate system for ticketing (Siebel CRM provided by lsi.com) so you have to create two logins, which is dumb:
-one login for the support site
-one login for ticketing

They need to fix this.

I will update this post when I have further information and hopefully, a solution to my problem.
'sodo

3ware Resources
3ware Support Contacts
http://www.3ware.com/support/support.asp
3ware Software Version Checker
http://www.3ware.com/support/versionchecker.asp
3ware Software Download Center
http://www.3ware.com/support/download.asp
Documentation
http://www.3ware.com/support/userdocs.asp
Direct Downloads for all 3ware products
http://www.3ware.com/support/downloadpage.asp
Direct Downloads for the 9650SE
http://www.3ware.com/support/downloadpageprod.asp?pcode=19&path=Escalade9650SE-Series&prodname=3ware%209650SE-%202LP/4LPML/8LPML/%2012ML/16ML
Online Support Ticketing System
https://www.3ware.com/

Other Resources
Common 3ware /var/log/messages
Feel free to drop me a line or ask me a question.