Callers coming into the SPAN office were worried. People at the other end of the phone were scared. Many of the calls came from network managers who took care of a piece of SPAN at a specific NASA site, such as the Marshall s.p.a.ce Flight Center. Some were panicking; others spoke in a sort of monotone, flattened by a morning of calls from 25 different hysterical system administrators. A manager could lose his job over something like this.
Most of the callers to the SPAN head office were starved for information. How did this rogue worm get into their computers? Was it malicious? Would it destroy all the scientific data it came into contact with? What could be done to kill it?
NASA stored a great deal of valuable information on its SPAN computers. None of it was supposed to be cla.s.sified, but the data on those computers is extremely valuable. Millions of man-hours go into gathering and a.n.a.lysing it. So the crisis team which had formed in the NASA SPAN project office, was alarmed when reports of ma.s.sive data destruction starting coming in. People were phoning to say that the worm was erasing files.
It was every computer manager"s worst nightmare, and it looked as though the crisis team"s darkest fears were about to be confirmed.
Yet the worm was behaving inconsistently. On some computers it would only send anonymous messages, some of them funny, some bizarre and a few quite rude or obscene. No sooner would a user login than a message would flash across his or her screen:
Remember, even if you win the rat race--you"re still a rat.
Or perhaps they were graced with some bad humour:
Nothing is faster than the speed of light...
To prove this to yourself, try opening the refrigerator door before the light comes on.
Other users were treated to anti-authoritarian observations of the paranoid:
The FBI is watching YOU.
or
Vote anarchist.
But the worm did not appear to be erasing files on these systems.
Perhaps the seemingly random file-erasing trick was a portent of things to come--just a small taste of what might happen at a particular time, such as midnight. Perhaps an unusual keystroke by an unwitting computer user on those systems which seemed only mildly affected could trigger something in the worm. One keystroke might begin an irreversible chain of commands to erase everything on that system.
The NASA SPAN computer team were in a race with the worm. Each minute they spent trying to figure out what it did, the worm was pushing forward, ever deeper into NASA"s computer network. Every hour NASA spent developing a cure, the worm spent searching, probing, breaking and entering. A day"s delay in getting the cure out to all the systems could mean dozens of new worm invasions doing G.o.d knows what in vulnerable computers. The SPAN team had to dissect this thing completely, and they had to do it fast.
Some computer network managers were badly shaken. The SPAN office received a call from NASA"s Jet Propulsion Laboratories in California, an important NASA centre with 6500 employees and close ties to California Inst.i.tute of Technology (Caltech).
JPL was pulling itself off the network.
This worm was too much of a risk. The only safe option was to isolate their computers. There would be no SPAN DEC-based communications with the rest of NASA until the crisis was under control. This made things harder for the SPAN team; getting a worm exterminating program out to JPL, like other sites which had cut their connection to SPAN, was going to be that much tougher. Everything had to be done over the phone.
Worse, JPL was one of five routing centres for NASA"s SPAN computer network. It was like the centre of a wheel, with a dozen spokes branching off--each leading to another SPAN site. All these places, known as tailsites, depended on the lab site for their connections into SPAN. When JPL pulled itself off the network, the tailsites went down too.
It was a serious problem for the people in the SPAN office back in Virginia. To Ron Tencati, head of security for NASA SPAN, taking a routing centre off-line was a major issue. But his hands were tied.
The SPAN office exercised central authority over the wide area network, but it couldn"t dictate how individual field centres dealt with the worm. That was each centre"s own decision. The SPAN team could only give them advice and rush to develop a way to poison the worm.
The SPAN office called John McMahon again, this time with a more urgent request. Would he come over to help handle the crisis?
The SPAN centre was only 800 metres away from McMahon"s office. His boss, Jerome Bennett, the DECNET protocol manager, gave the nod.
McMahon would be on loan until the crisis was under control.
When he got to Building 26, home of the NASA SPAN project office, McMahon became part of a core NASA crisis team including Todd Butler, Ron Tencati and Pat Sisson. Other key NASA people jumped in when needed, such as Dave Peters and Dave Stern. Jim Green, the head of the National s.p.a.ce Science Data Center at G.o.ddard and the absolute boss of SPAN, wanted hourly reports on the crisis. At first the core team seemed only to include NASA people and to be largely based at G.o.ddard.
But as the day wore on, new people from other parts of the US government would join the team.
The worm had spread outside NASA.
It had also attacked the US Department of Energy"s worldwide High-Energy Physics" Network of computers. Known as HEPNET, it was another piece of the overall SPAN network, along with Euro-HEPNET and Euro-SPAN. The NASA and DOE computer networks of DEC computers crisscrossed at a number of places. A research laboratory might, for example, need to have access to computers from both HEPNET and NASA SPAN. For convenience, the lab might just connect the two networks.
The effect as far as the worm was concerned was that NASA"s SPAN and DOE"s HEPNET were in fact just one giant computer network, all of which the worm could invade.
The Department of Energy keeps cla.s.sified information on its computers. Very cla.s.sified information. There are two groups in DOE: the people who do research on civilian energy projects and the people who make atomic bombs. So DOE takes security seriously, as in "threat to national security" seriously. Although HEPNET wasn"t meant to be carrying any cla.s.sified information across its wires, DOE responded with military efficiency when its computer managers discovered the invader. They grabbed the one guy who knew a lot about computer security on VMS systems and put him on the case: Kevin Oberman.
Like McMahon, Oberman wasn"t formally part of the computer security staff. He had simply become interested in computer security and was known in-house as someone who knew about VMS systems and security.
Officially, his job was network manager for the engineering department at the DOE-financed Lawrence Livermore National Laboratory, or LLNL, near San Francisco.
LLNL conducted mostly military research, much of it for the Strategic Defense Initiative. Many LLNL scientists spent their days designing nuclear arms and developing beam weapons for the Star Wars program.9 DOE already had a computer security group, known as CIAC, the Computer Incident Advisory Capability. But the CIAC team tended to be experts in security issues surrounding Unix rather than VMS-based computer systems and networks. "Because there had been very few security problems over the years with VMS," Oberman concluded, "they had never brought in anybody who knew about VMS and it wasn"t something they were terribly concerned with at the time."
The worm shattered that peaceful confidence in VMS computers. Even as the w.a.n.k worm coursed through NASA, it was launching an aggressive attack on DOE"s Fermi National Accelerator Laboratory, near Chicago. It had broken into a number of computer systems there and the Fermilab people were not happy. They called in CIAC, who contacted Oberman with an early morning phone call on 16 October. They wanted him to a.n.a.lyse the w.a.n.k worm. They wanted to know how dangerous it was. Most of all, they wanted to know what to do about it.
The DOE people traced their first contact with the worm back to 14 October. Further, they hypothesised, the worm had actually been launched the day before, on Friday the 13th. Such an inauspicious day would, in Oberman"s opinion, have been in keeping with the type of humour exhibited by the creator or creators of the worm.
Oberman began his own a.n.a.lysis of the worm, oblivious to the fact that 3200 kilometres away, on the other side of the continent, his colleague and acquaintance John McMahon was doing exactly the same thing.
Every time McMahon answered a phone call from an irate NASA system or network manager, he tried to get a copy of the worm from the infected machine. He also asked for the logs from their computer systems. Which computer had the worm come from? Which systems was it attacking from the infected site? In theory, the logs would allow the NASA team to map the worm"s trail. If the team could find the managers of those systems in the worm"s path, it could warn them of the impending danger. It could also alert the people who ran recently infected systems which had become launchpads for new worm attacks.
This wasn"t always possible. If the worm had taken over a computer and was still running on it, then the manager would only be able to trace the worm backward, not forward. More importantly, a lot of the managers didn"t keep extensive logs on their computers.
McMahon had always felt it was important to gather lots of information about who was connecting to a computer. In his previous job, he had modified his machines so they collected as much security information as possible about their connections to other computers.
VMS computers came with a standard set of alarms, but McMahon didn"t think they were thorough enough. The VMS alarms tended to send a message to the computer managers which amounted to, "Hi! You just got a network connection from here". The modified alarm system said, "Hi!
You just got a network connection from here. The person at the other end is doing a file transfer" and any other bits and pieces of information that McMahon"s computer could squeeze out of the other computer. Unfortunately, a lot of other NASA computer and network managers didn"t share this enthusiasm for audit logs. Many did not keep extensive records of who had been accessing their machines and when, which made the job of chasing the worm much tougher.
The SPAN office was, however, trying to keep very good logs on which NASA computers had succ.u.mbed to the worm. Every time a NASA manager called to report a worm disturbance, one of the team members wrote down the details with paper and pen. The list, outlining the addresses of the affected computers and detailed notations of the degree of infection, would also be recorded on a computer. But handwritten lists were a good safeguard. The worm couldn"t delete sheets of paper.
When McMahon learned DOE was also under attack, he began checking in with them every three hours or so. The two groups swapped lists of infected computers by telephone because voice, like the handwritten word, was a worm-free medium. "It was a kind of archaic system, but on the other hand we didn"t have to depend on the network being up,"
McMahon said. "We needed to have some chain of communications which was not the same as the network being attacked."
A number of the NASA SPAN team members had developed contacts within different parts of DEC through the company"s users" society, DECUS.
These contacts were to prove very helpful. It was easy to get lost in the bureaucracy of DEC, which employed more than 125000 people, posted a billion-dollar profit and declared revenues in excess of $12 billion in 1989.10 Such an enormous and prestigious company would not want to face a crisis such as the w.a.n.k worm, particularly in such a publicly visible organisation like NASA. Whether or not the worm"s successful expedition could be blamed on DEC"s software was a moot point. Such a crisis was, well, undesirable. It just didn"t look good.
And it mightn"t look so good either if DEC just jumped into the fray.
It might look like the company was in some way at fault.
Things were different, however, if someone already had a relationship with a technical expert inside the company. It wasn"t like NASA manager cold-calling a DEC guy who sold a million dollars worth of machines to someone else in the agency six months ago. It was the NASA guy calling the DEC guy he sat next to at the conference last month.
It was a colleague the NASA manager chatted with now and again.
John McMahon"s a.n.a.lysis suggested there were three versions of the w.a.n.k worm. These versions, isolated from worm samples collected from the network, were very similar, but each contained a few subtle differences. In McMahon"s view, these differences could not be explained by the way the worm recreated itself at each site in order to spread. But why would the creator of the worm release different versions? Why not just write one version properly and fire it off? The worm wasn"t just one incoming missile; it was a frenzied attack. It was coming from all directions, at all sorts of different levels within NASA"s computers.
McMahon guessed that the worm"s designer had released the different versions at slightly different times. Maybe the creator released the worm, and then discovered a bug. He fiddled with the worm a bit to correct the problem and then released it again. Maybe he didn"t like the way he had fixed the bug the first time, so he changed it a little more and released it a third time.
In northern California, Kevin Oberman came to a different conclusion.
He believed there was in fact only one real version of the worm spiralling through HEPNET and SPAN. The small variations in the different copies he dissected seemed to stem from the worm"s ability to learn and change as it moved from computer to computer.
McMahon and Oberman weren"t the only detectives trying to decipher the various manifestations of the worm. DEC was also examining the worm, and with good reason. The w.a.n.k worm had invaded the corporation"s own network. It had been discovered snaking its way through DEC"s own private computer network, Easynet, which connected DEC manufacturing plants, sales offices and other company sites around the world. DEC was circ.u.mspect about discussing the matter publicly, but the Easynet version of the w.a.n.k worm was definitely distinct. It had a strange line of code in it, a line missing from any other versions. The worm was under instructions to invade as many sites as it could, with one exception. Under no circ.u.mstances was it to attack computers inside DEC"s area 48. The NASA team mulled over this information. One of them looked up area 48. It was New Zealand.