HD Recoding.... Still TONS of Work

[Beck38]

Like many others (uh... I guess everyone here), I've been sucking on the HD tit for quite a while, mostly recoded products.

There's 2-3 major 'packages' out there to pull together the tools and such to accomplish this, most common is the MeGUI, Handbreak, and a very early beta version of HD-Rebuilder.

What's bad about all of them, is that the programmers haven't really kept the programs up with the pieces of the puzzle, in that as the various 'bits' have been improved (or not!), they haven't exactly been quick on the ball to upgrade their programs to take advantage of the new pieces.

But, be that as it may, anyone who's dragged themselves through the process(es), gets a real education as to how much effort, and how much computing horsepower is needed to grind out a 'product'.

Now, I was there 'at the beginning' of SD/DVD recoding, circa 2001-3. I 'lived' on Doom9 for quite a while picking up tips. I built a machine (Athlon 64) to do the work, and ran it 24/7 for several years (including a CPU upgrade somewhere around 2004), until it finally simply gave up in 2006, and I built a replacement.

But the computational power needed to do the recoding is nothing short of jaw-dropping. A 4 or even 8 core fast CPU takes upwards of 4+ DAYS (at maximum quality) to grind through the bits. No wonder the number of recodes out their is actually pretty low.

I have a 'next to highest' speed 4core AMD machine (Vista), where the newest available cpu actually came out after I had the machine built. I have looked into going with the newest 3Ghz i7 6core/12thread machine (Win7), and the base machine runs around $5K, again specifically to do this 'work'. And, there are a couple motherboards out there that do TWO of those monsters (so that would be 12core/24thread), for a couple thousand more.

Quickly, one gets some real respect for those tackling this.

[whiteboy]

Some of your numbers are way off. 4+ days for an encode wtf???? You may be using some ridiculous unnecessary settings or something.

Also 5000 is much more then that system should cost, yes it will be outrageous but you could build a nice system with that cpu for 2000 bucks.

[Beck38]

I would suggest that your numbers are WAY off from reality.

A single i7 processor system, utilizing the i7 980X processor (6cpu/12threads), would be well over $2K simply by the fact the processor is $1K+, and a decent motherboard (single again) is another $300. Add to that a decent amount of RAM, and of course the case and even a cheapo graphics card, and you're nibbling at over $2K without talking about a decent drive system OR paying Mr. Bill (gates) for the OS.

Put in a RAID1 for the OS drive (say, twin 750GB), and a 2TB for mass storage, and that adds a good $300+. Also, as you're running 24/7, a solid power supply for any case (and to power the mb/cpu combo) is another $100-150 depending on how good a case/ps deal your can wrangle.

So.... what's the toll? at least $3500 low-balling everything, and you'd end up what I had 5 years or so ago, a dead machine after 4+ years of constant use. Note I didn't add a good UPS, either, so that's an additional $250-300+.

Quality recodes take time/cpu cycles. Yep, you can cut corners (like a lot of folks) and do 720P instead of 1080P, and can use 'FAST' settings all around. You get what you pay for, and it that case, pretty slimy results. On my current main HT display, I can tell 720P recodes from 20 feet across the room while standing in my dining room looking though to my living room/ht setup. And this is with a early generation (but broadcast grade) HD display. Any way you want to contribute to my fund for a 3D upgrade when decent equipment becomes available? (i.e., 2011+)

BTW, add in a mb with dual CPU capability (although at $1K that would be an interesting tug of war with mr. plastic!), and that adds more.

So, $5K is very realistic. My original single processor (AMD x64) machine I built in 2002 cost over $2500, and it had only some 600GB of disk space, 2GB of ram, and again, was low-ball quality wise. And it didn't really last, although it's power was at best a few percentage points of what this new system would be.

[Beck38]

In the 'well, but...' department:

The AMD Phenom2 x6 and it's attendant 890FX Chipset have been out for couple of months, and there are a host of motherboards that support the AM3 socket (in fact, my newest machine built a year ago has it, but not the 890fx chipset).

About 1/4 the cost of the Intels at approx. the same speed and such, although all the reviews I've read so far put it just a squid below the Intel. $750+ being the 'squid'.

Unfortunately, I've been unable to find any dual CPU motherboards for the AMD; and a couple of message board that make the statement that there probably never will be, and the only way to get more than 6 cores on an AMD platfor is to go with the more expensive Opteron's (up to 12 core), but very expensive and low Mhz compared to the Intels.

Or, wait a few more months before things 'catch up'. I'll try 'pricing out' a single CPU machine this coming week, and decide how I can shuttle things around to get the most 'bang for the buck'. I'd probably put my current 4core machine into recode service, but although it's pretty fast (again, only one step down in speed from currently available 4core AMD's), it's running Vista32 due to maintaining s/w compatibility with a couple of programs I need to use several times a week.

We will see what my local shop can do...

[zot]

Beck, would one of these babies be of any use to you for "sucking on the HD tit" ;)

http://www.elementaltechnologies.com/products/server


By combining GPU and CPU processing one Elemental Server replaces up to seven dual quad core 3GHz CPU-only servers.

I remember a couple of years ago, the big story across the internet was that soon the GPU would supplement -or even largely replace- the CPU for video encoding(no surprise, Intel did not exactly agree). No mention of the price, but I'd be surprised if these servers are not in the range of a new car.

[Beck38]

That was the 'future track' by some a few years back, before all this xXx (super-multi-core) CPU's took off like a rocket.

Way back in the 3D rendering stone age (oh, about 1990+), I designed and built several multi-machine rendering 'farms', both for my own company and others. The horsepower available today in a single chip design (4x through 12x+) is mind boggling.

Then again, the amount of work we expect these things to do has exploded as well. Just look at recoding as an example:

Cira 2003:
8GB/DVD9 SD-DVD recoded to 4.5GB/DVD5
Top CPU available (2003): P4/Athlon, ~2Ghz, 2GB ram, 500GB hard drive
Time to recode: ~24hrs+

Cica 2009:
8GB/DVD9 SD-DVD recoded to 4.5GB/DVD5
Top CPU: Athlon64x2/Intel Core2, ~3Ghz, 4-8GB ram, 2TB drive
Time to recode: ~6+ hours

I've done a full series of tests with the various HD recoding schemes, all of them are based on x264 engines, though. Unlike the multi-pass generation with SD-DVD, all the x264 types are 2-pass, with the profile/presets (as defined by 'Sharktooth') are the 'standard' for driving the recode engine.

Using my current Phenom2/4x machine, the recode time varies between ~5hours in 'FAST' mode (all settings at the lowest level), and upwards of ~110 hours in 'INSAME' mode where all the settings are at maximum.
(FYI, as I think I noted earlier, this is a Vista32 machine, 3GB usable RAM, tons of HD space). But obviously slow by 64bit with more ram standards.

(If I had Vista64 and more memory (but that would render several other programs/projects unusable on that machine), it'd be faster.)

So, it's time to get serious about building yet another box, this with Win7-64, ton of superfast/cheap ram, and the best CPU available. I'm a Asus MB person, and they do have a new 890FX based AM3 board (the M4A89TD PRO/USB3 and one other I believe), but like I said, no 'dual' systems.

Well, I'll try and hunt up some of the decent testing results on this, and how much it will crank out. If I can get about double the performance I get now, it will be worth it, bring those time down by half would be nice, without breaking the bank.

BTW, a couple of folks I've talked to that were on the 'bleeding edge' of HD recoding 3-4 years ago, built multi-CPU (and multi-core) Xeon machines, costing upwards of 6-8K at the time of their build.

Things are moving fast now though. The new AMD's (with the 890FX chipset), should start changing the scene at breakneck speed.