Hello, Wednesday

I woke up at 3:50am this morning unable to sleep and by the time I got to work, I figured that was the worst thing that could happen today. Then I logged in to our CentOS kickstart machine to see how my “yum update” to CentOS 5.2 went last night and found all kinds of problems. Joy. So I did some manual patching to get things operational again and went to re-run the update to get the system into a known good state. That’s when things went from bad to worse:

Updating : yum ####################### [42/56]
Installing: yum-fastestmirror ####################### [43/56]
Installing: kernel ####################### [44/56]
DAC960#0: Physical Device 1:0 killed because of timeout on SCSI command
DAC960#0: Physical Device 1:1 killed because of timeout on SCSI command
DAC960#0: Physical Device 1:2 killed because of timeout on SCSI command
DAC960#0: Physical Device 1:3 killed because of timeout on SCSI command
DAC960#0: Physical Device 1:4 killed because of timeout on SCSI command
DAC960#0: Physical Device 1:5 killed because of timeout on SCSI command
DAC960#0: Physical Device 1:8 killed because of timeout on SCSI command
DAC960#0: Physical Device 1:0 is now DEAD
DAC960#0: Physical Device 1:1 is now DEAD
DAC960#0: Physical Device 1:2 is now DEAD
DAC960#0: Physical Device 1:3 is now DEAD
DAC960#0: Physical Device 1:4 is now DEAD
DAC960#0: Physical Device 1:5 is now DEAD
DAC960#0: Physical Device 1:8 is now DEAD
DAC960#0: Logical Drive 0 (/dev/rd/c0d0) is now OFFLINE
DAC960#0: Physical Device 1:0 killed because it was removed
DAC960#0: Physical Device 1:1 killed because it was removed
DAC960#0: Physical Device 1:3 killed because it was removed
DAC960#0: Physical Device 1:4 killed because it was removed
Updating : apr-util-devel ####################### [45/56]
Updating : postgresql-libs ####################### [46/56]
Installing: hal #### [47DAC960#0: Physical Device 1:5 killed because it was removed
56]

And the system stopped responding. *sigh* Off to the colo again…