do you log today...?
When hi-tech meets high fantasy
When hi-tech meets high fantasy
By Mark Ward
BBC News Online technology correspondent
Whether you like the three Lord of the Rings films or not there is little doubt that they are a triumph for technology.
Running the huge technological resources behind the films was Weta Digital, a firm formed by Rings director Peter Jackson and others in 1993 to do the effects for the Heavenly Creatures movie.
Back then Weta had only one computer, which it leased, to do special effects work.
Now it runs the third largest supercomputer on the planet if you count the number of processors, 3300, it can call on, says Scott Houston, chief technical officer at Weta.
The ones that beat Weta are the Japanese Earth Simulator (5120 processors) and Los Alamos National Laboratory's supercomputer (8192 processors).
Big database, big problems
The reason Weta's data centre in Miramar New Zealand is not on the Top 500 supercomputer list, said Mr Houston, was because of the way computing power is calculated for those rankings.
The Weta executive was in London as part of a project to promote New Zealand as a place to find skilled technology workers and get programming jobs done.
Weta, said Mr Houston, was a standard bearer for what was possible.
"We have done some remarkable things," he said.
By any measure the amount of data prepared and processed for the movies is staggering.
Weta's data store is 500 terabytes in size spread among 220 million files.
Some of that data is a few years old because some sequences, such as the Balrog from the Mines of Moria, appeared in more than one film.
Finding and moving the data for that sequence out of storage so it could be reworked for the second film took about three days.
A database of that size takes a lot of looking after.
Currently Weta is working on better ways to archive and organise it so that it can potentially be re-purposed for other projects.
This work is needed, said Mr Houston, because the current set up means that it would take months to recover from a double drive failure in one of its disk stores.
But most of the data is from the final movie, The Return of the King, which had more than 1500 special effects shots in it. By contrast the first movie had only 400 and the second 900.
Battle to build
One of the longest special effects sequences was the Battle of the Pelennor fields that took place in front of Minas Tirith.
The battle pitted 120,000 digital orcs, each with its own AI fighting style and armour against 8,000 digital Riders of Rohan.
It was also almost the sequence that derailed the entire picture.
In mid-August last year, Mr Houston had to work out how long it would take to prepare this sequence for the finished film with the resources they had.
He came back with an estimate of 347 days. Given that the immovable release date for the final film was 18 December, that was a few hundred days too long.
"There was never a question of putting any barriers in the way to Peter Jackson's vision," said Mr Houston.
Instead they just had to get more computers, even though Weta's four data centres were chock full of machines at that point.
"We needed another 1,000 processors and we had nowhere to put them," he said.
The answer was to build another data centre nearby, lay a 10gigabit per second cable to it to connect it to the other four and ask IBM to set up a production line just to build the computers Weta needed to stock it.
The result was that within two weeks the new data centre was finished and helping prepare the battle for the big screen.
Since then it, and other Weta resources, have been used for effects in the forthcoming I, Robot movie and will help with future releases such as The Lion, The Witch and the Wardrobe.
For future productions that suddenly need more processing power Mr Houston wants to do things differently.
"We are a real peaks and troughs business," he said.
Much better would be to use a Grid-type approach that lets Weta call on processing power where-ever it is and when it needs it.
"I do not want to have to build the computer room in two weeks again," he said.
It is likely that Weta will have to call on more processing power than it has as its disposal.
"I know my boss and he's not going to be stopped," said Mr Houston. "He has been totally vindicated by the success of Lord of the Rings."
"King Kong is going to be bigger," he said.
He is confident of this, even though King Kong only has one, albeit big, monster and many sequences in Lord of the Rings featured hundreds of thousands of them.
"King Kong is covered in hair," he said, "we could be animating that."
Story from BBC NEWS:
Published: 2004/04/30 10:31:45 GMT
© BBC MMIV
05/12/2002 - 05/19/2002
05/02/2004 - 05/09/2004
05/09/2004 - 05/16/2004
05/16/2004 - 05/23/2004
08/08/2004 - 08/15/2004