Power outages felt across campus


The director of Physical Plant said this week she hopes to meet with members of the research community shortly about recent power outages at the Institute tied to the cogeneration project.

Victoria V. Sirianni acknowledged that the outages have caused "serious and costly damage to research experiments" and "great inconvenience" to the rest of the community.

An "installation failure" caused the power outage that cut off all but emergency power to about two-thirds of MIT's buildings a week ago on Wednesday, March 23. It followed two others that occurred on February 25 and February 28.

Ms. Sirianni said all three resulted from the present configuration of MIT's electrical system as the Institute switches over to its new cogeneration facility.

MIT is in the process of installing a 22-megawatt combustion turbine, in conjunction with a heat recovery boiler, which will provide about 90 percent of the Institute's electrical and thermal energy requirements when the work is completed. The projected date is January 1995.Until then, MIT is continuing to use power supplied by Cambridge.

The project was undertaken to stabilize the Institute's utility bills and provide a more reliable electrical supply.

Ms. Sirianni said the outages in February were caused by faulty factory wiring of a component in the protective relaying circuits of the new switch gear located in Building 42. "The cause for the most recent outage again involved the switch gear, where a bolted connection between breakers overheated and subsequently flashed," she said.

"We have determined that this week's outage was the result of an installation failure," Ms. Sirianni said in a message to the community. "I assure you this equipment was tested again and again, but the complexity of what is occurring in these critical stages as we switch over makes occurrences like this not as uncommon as one would like. Please note that the backups we would normally have to prevent these occurrences from causing outages or even power dips are severely compromised by the present electrical configuration."

"We are caught in the proverbial Catch-22," she continued. "Until we complete the installation of the B bus in June, we will not be able to achieve the redundancy and protection we need; but to finish the job we must eliminate and modify the protection we have right now on a daily basis.

"The fact that we are engaged in this project principally to increase the reliability of our electrical service must be of little comfort by this time. I can only hope it will be so in the future."

Ms. Sirianni said she was asking department and center heads "to encourage members of the research community to meet with some of us in the Plant so that we can address their specific concerns. I believe it might help for us to explain in greater detail exactly what has happened, why we are so vulnerable, how we plan to

manage the risk for the remaining weeks until we are fully operational, and lastly and most importantly, to see if there are any interim measures we might use to provide some relief to this community."

In the latest incident, it was the A bus that overheated, explained Roger Moore, superintendent of utilities. Once that happened, he said, "we had to open up the main breaker serving the bus, and that essentially cut off all the power to all buildings connected to that bus." The affected buildings were on the east side of Massachusetts Avenue. A bus is a conductor for transferring electrical current and distributing it to outgoing feeders.

Power went off at 9:13am and was restored to most of the campus by noon, although some buildings had to wait until late afternoon for electricity. It took longer to restore normal power to Buildings 44, 45 and 48 because the cable feeding those buildings had to be kept out of service in order to work on the affected bus. About 15 buildings lost power a second time for about two minutes at 7pm during a planned outage that enabled utility personnel to reconnect the buildings to the repaired bus.

Because of reconfiguration associated with cogeneration, it took longer than usual to restore power, Mr. Moore said. Many of the individual building load switches had to be switched from the new supply (the overheated bus) to the old supply (substations 2 and 3), which will be taken out of service when the new substation is completed, he said.

"In order to tie individual buildings into the old substations, we had to disconnect from the new A bus and then switch back into the old supply, which meant there were about 75 distribution switches that had to be operated," Mr. Moore explained.

The problem was complicated by the fact that the drawings of the system at present "represent half of the old system and half of the new system" because of work in progress, he said.

Between 10 and 15 electricians were assigned to the task of restoring power, he said.

Emergency lighting cut in automatically when the outages occurred, as did emergency power for elevators, fire alarms, life safety and essential research.

Generally, the outage occurred at a propitious time-halfway into the spring break.

Still, concern about the affects of such outages was expressed by several faculty members and researchers, including those working in the areas of biology, toxicology and metallurgy.

"When an experiment is interrupted," a biologist said, "what we lose in time is much longer than the actual outage. It can be as much as a week or more." He also mentioned the possibility of destroying valuable biological specimens.

Morton Berlan, director of telecommunications systems, said analog telephones, accounting for about half of the telephones on campus, were unaffected, as was the voice mail system. Digital telephones remained in service if they had emergency backup, he said.

Cecilia R. D'Oliveira, director of Distributed Computing and Network Services, noted that both MITnet and Athena services are distributed across the campus.

"In general, when we experience a power outage, the impact of the outage is a function of exactly where power went out," she said. "We put critical equipment on backup power whenever possible. Backup power typically provides us with 10-30 minutes of battery backup. Wednesday's power hit was especially troublesome because the outage was prolonged and caused batteries on the backup units to become drained to the point where power dropped and equipment failed."

Within the MITnet and Athena computing and communications environments, she said, there were three key impacts:

  • MITnet service: An MITnet router in Building 11 went down for 4-5 minutes after its backup battery power ran out. This affected local area networks in the main group around 9:30am. In addition, local area network service was out for much longer in buildings where the power stayed out much of the day. This included Buildings 6, 14, 16, 48, 54, 56 and 66.
  • Athena file service: User file space within the Athena environment is provided by file servers located in Buildings 11, W20 and E40. These servers are all on backup power. On Wednesday, the file servers located in Building 11 went out for about 45 minutes between 9 and 10am after the batteries in their backup power supplies ran out. This affected a limited number of faculty and students (about 30) who were not able to access their files during this time.
  • ������Athena public workstation facilities: Athena's public clusters where students work at Athena workstations are located throughout the campus. Those clusters in buildings where power went out were affected.

A version of this
article appeared in the
March 30, 1994

issue of MIT Tech Talk (Volume
38, Number
27).


Topics: Campus services

Comments

Back to the top