The Never Ending Server Update Thread - Updated 2/12/07

[Jonah Weiland]

Hey Guys,

Well, the server upgrade didn't go exactly as planned, but there's definitely some good news and bad news.

The bad news -- the heat syncs we ordered are not compatible with the mother board, meaning we don't yet have quad processors on both the CBR server and CBR Forums server. Oh well. Going to have to do some research and find another set of heat syncs.

The good news -- We took out the old 2.4 Ghz dual-core processor from the CBR Forums server and put in the new 3.2 Ghz dual-core processor. So, the Forums now have about 30-40% more processor speed, which is good, but we'd really like to get that up to a 140% increase. Once we get the proper heat syncs, we'll put in those other processors.

Hopefully this faster processor helps a bit, at least as a hold over until we can find the right heat syncs.

That's it for now. More details as soon as I have them.

[howyadoin]

Incidentally (or not), is it just me, or are the CBR clocks an hour fast? I posted this at 11:37 AM, but the timestamp on the post reads 12:37 PM.

[Jonah Weiland]

The clock on the server is fine. Might be the way vBulletin handles daylight savings changes ... will have to take a look later.

[Jonah Weiland]

OK, new heat syncs are on their way. Hopefully by weeks end we'll have this server operating quad CPUs and it'll be blazingly fast.

As is, the upgrade to the faster CPU has made a noticeable difference, but it definitely needs additional power.

Will keep you posted with more as soon as I have it.

[Jonah Weiland]

Well, another trip to the data center, another couple of small issues.

So, to catch everyone up. CBR is run off two servers -- Lois (the CBR and CBR Ads Server) and Kara (the Forums and Blog server). Previously, each machine was running a single Pentium 4 2.4 Ghz Xeon processor (they're hyper threaded, so it's almost like having dual core CPUs) with the ability to expand to two processors. The plan was to move the CPU from Kara into the Lois machine, then put two new Pentium 4 3.2 Ghz Xeon processors in Kara to really give her a boat load of power. That was the plan.

Last Sunday, my Admin Mark and I went down to the data center only to discover that the new heat syncs we purchased for the CPUs were incorrect, so we couldn't finish the job. Instead, we swapped out the 2.4 Ghz CPU in Kara for one of the new 3.2 Ghz CPUs. That's helped make Kara faster and more responsive, but we're still experiencing the occasional delays. Not as bad as before, but still an issue.

So, last week I got the right heat syncs. Mark and I went down to the data center today and installed the second 2.4 Ghz CPU in Lois and everything went A-OK. Lois is now running dual 2.4 Ghz CPUs and is blazing fast. Beauty. Time to work on Kara.

We installed the second 3.2 Ghz CPU in Kara, powered her up but nothing would happen. No boot screen, just a long beep coming out of the server case. Not a good sign. After playing around with things a bit we determined that the 350 watt power supply in Kara was not quite powerful enough to run 2 3.2 Ghz CPUs, two EIDE 120 Gig hard discs with two gigs of on board memory. Drat!

So, upgrades to Lois are fine, but Kara still needs a bit of work. I'm ordering a new power supply Monday morning for Kara that will hopefully get here by Tuesday afternoon. Tuesday night we should finally get a chance to upgrade Kara completely!

Not a totally wasted trip today, but annoying that one more will be needed.

Will update further as I have information.

[Jonah Weiland]

Spoke with Supermicro, the makers of the server case, and they suggested a 420 Watt power supply. It's been ordered and should arrive on Tuesday. If it does, then Mark will install it Tuesday night. If not, it probably won't get installed until next weekend as both Mark and I are busy Wednesday through Friday.

There's your latest update. Hopefully Tuesday night we'll be golden.

[Jonah Weiland]

Good news! The new power supply came today, as scheduled. So, the plan is sometime tonight to have Mark replace the power supply in the server, then install the second CPU in the server. With a little luck, the machine will come online without any problems.

Swapping out a power supply, especially on a 1U server, can take some time, so we might see between 60-120 minutes of downtime tonight. This will affect both the forums and the blog, but hopefully it'll go quickly and you'll all be back posting and reading in short order.

Thanks, folks! Keep your fingers crossed!

[Spike-X]

*pssst* It's "heat sink".

[Jonah Weiland]

OK, so last week we got the new power supply, installed it Tuesday night, put both CPUs in the machine and, well ... nada.

Something more serious is wrong with this server or we just need to do more investigating, the time for which I don't have right now and the investigating would require that this server be offline for an extended period of time (probably 6-8 hours), which is something I don't like the idea of.

So, here's what I'm currently thinking. I need to buy a new client server anyway. What I may do is buy the new server and make it the new CBR Forums server. Then, pull the old CBR Forums server, work on it from home where we don't have time pressures, get it working properly and let it become the new client server.

I've got a lot going on this week prepping for NYCC, so I'm not sure when I'll exactly be able to get to this. I'm talking with my admin Tuesday afternoon about a number of items and will have this on the agenda.

Man, this is so frustrating! A simple upgrade has turned into a real headache.

[howyadoin]

So, any news? I haven't seen CBR as wonky as it is tonight in years. Replies at Rita's are actually appearing ahead of the original posts.

[dingo]

So, any news? I haven't seen CBR as wonky as it is tonight in years. Replies at Rita's are actually appearing ahead of the original posts.

Oooooh, time travel effects, don't fix it!

[shaxper]

I'm having a lot of trouble with load time, as well. Sometimes I can't get threads to load at all.

[thehod]

The boards do seem to have been playing silly buggers for a little while now.

[Tami]

It looks like the server is going a little buggy again today. :(

[Jonah Weiland]

The last couple of days have been bad. Sorry, folks.

The second CPU would not take, even after we got the new and recommended power supply. GRRR!

Since then, I've just not had time to put into it. One Sunday afternoon we'll have to take the server offline for a good four hours or so to determine exactly why it can't be upgraded before I go spending more money on upgrading this thing. Problem is, most of the past month worth of Sundays have seen conventions and this weekend is no different.

All this, on top of which the blog and forums that are hosted on this server have seen huge spikes in traffic (Did anyone read something about "Captain America" #25?). That extra traffic has really crippled this machine.

On a positive note, the upgrades to the CBR server allowed it to hold up well under the pressure of the Digg effect today:

http://www.digg.com/television/Linde...ip_Description

Love that!

So, if we can get these upgrades completed, I'm relatively confident life will be better. But it's all about time. I'll keep you posted.