Despite our best efforts, the Data Center maintenance on July 7 was unsuccessful and must be attempted again on Aug. 11. Berkeley IT collaborated with Facilities Services, suppliers, and campus stakeholders to select this new date to ensure maintenance is completed before fall instruction begins.
We will be performing mechanical and electrical system upgrades which will require a complete shutdown of our campus Data Center on Sunday, Aug. 11. This work is required to support the campus research mission and to provide improved resilience for the Data Center to run on generator power during campus outages. Berkeley IT will be working with Facilities Management to make mechanical and electrical system upgrades that will reduce the impact of unplanned campus power outages in the future.
Since the Data Center houses many of our mission-critical servers on campus, powering things off will impact access to many systems including CalTime, Cal Central, Student Information Systems / Berkeley Campus Solutions (SIS/BCS), Cal Answers, Berkeley Financial System (BFS), BearBuy, and Savio Condo/Berkeley Research Cluster, to name a few. We understand this may disrupt your ability to work and we apologize for any inconvenience.
Frequently Asked Questions
What is being done?
Maintenance will be performed on components that provide emergency power to the data center when there is a campus power outage. To accomplish this work, power must be shut off to all systems hosted in the Data Center. As a result, systems located in the Data Center will be unavailable during the shutdown periodset from 2 a.m. through 10:30 p.m. on Sunday, Aug. 11, 2024.
Why is this being done?
The data center is running directly on campus power with a backup power system that provides up to 45 minutes of power in the event of a campus power failure. However, the generator that provides sustained power during a power outage cannot be engaged due to a failed ATS (automatic transfer switch). This switch must be repaired to enable automated, sustained emergency power if a power outage occurs.
Which systems will be impacted?
The following systems will not be available during the Data Center shutdown period. The systems listed below will be down from 2 a.m. through 10:30 p.m. on Sunday, Aug. 11 (times are estimated and subject to change). Updates will be provided as the work is completed and systems are brought back online, please check the system status page which will be updated regularly.
|
|
|
Which systems will be available?
The following systems will remain available during the shutdown period on Aug. 11, 2024. If the system you are looking for is not in the list below and you are unsure whether it runs in the Data Center, please check the system status page which will be updated regularly.
|
|
|
Why couldn't the work be completed earlier, on July 7?
The scheduled Data Center outage on July 7 began as planned. In the process of doing the repair work, the vendor determined that the necessary parts required to assemble the Automatic Transfer Switch (ATS) in the cabinet were not included in the order. After discussing possible alternatives with Facilities Services and Data Center teams and determining that there were no viable options, the decision was made to abort the repair and proceed with power-up of the data center and systems. The Data Center power-up was completed by 8:30 p.m. on Sunday, with final system checks extending into Monday and Tuesday.
I have work to do during the outage, will my system be available?
We understand that a shutdown of the Data Center may be disruptive to the campus and to your ability to work. However, this shutdown is necessary to perform the ATS repairs (automatic transfer switch). This switch must be repaired to enable automated, sustained emergency power if a power outage occurs. To ensure a graceful shutdown of systems, services that run in the Data Center will be taken offline up to 6 hours in advance of the required electrical/power work. The ATS repair work will then take up to 6 hours to complete, and we will need approximately 8 hours to re-establish all services and systems after the power work is completed. As we bring systems back up, it is not possible to pinpoint the exact timing for each. Therefore, it is best to plan ahead and know that systems will be downfrom 2 a.m. through 10:30 p.m. on Sunday, Aug. 11 (times are estimated and subject to change) unless listed in the “Which systems will be available?” section. To provide further clarification on the availability of systems, service providers will be sending communications regarding the availability of the systems they support as the date for this work approaches.
Why doesn't the Data Center use backup generators for this work?
The repairs to the ATS (automatic transfer switch) require everything to be down in the Data Center, even those systems that the backup generator would power.
I have a virtual machine (VM) I manage myself, do I have to shut it down myself before the power goes off?
Yes. Those who manage their own VM’s need to shut them down to avoid a forced shutdown in advance of the underlying infrastructure being powered down.
Who can I contact for more information?
We are collaborating on this project with several IT service providers across campus. Please contact Sam Griffin at samgriffin@berkeley.edu if you have questions.