The general information is aimed at providing groups interested in the facilities with background for technical planning.
The proposal preparation information covers budgeting support for research computing in proposals.
The Bates facility has an onsite staff who are available to help with small tasks, such as occasionally power cycling systems or receiving shipping. This service is provided on an as available basis. Bates tenants can request remote assistance through email to staff at the facility.
The MGHPCC facility hosts a resident MIT subcontractor who is able to provide remote assistance to groups with equipment at the facility. The subcontractor is typically onsite twice a week, usually on Tuesdays and Thursdays.
Bates staff are usually available to power cycle equipment if needed.
Tenants at MGHPCC have access to an online interface to the power outlets in their allocated space. The interface can be used to remotely monitor and control power to equipment.
All the facilities are connected to the main MIT ``18.'' network. The links to the facilities currently run over dedicated 10 Gbit/s circuits. These circuits are presently being upgraded to 100Gbit/s service. This means that systems at the shared facilities can be, for the most part, effectively the same ``network distance'' away as facilities on campus.
The facilities do have different designs, reflecting the evolution of
equipment. Prospective tenants typically do need to work with facility
representatives to ensure equipment and plans are compatible with a facility.
Factors to consider include • the physical shape dimensions of equipment;
• the compatibility of equipment with the power and cooling designs of the
facilities; • UPS short coverage power versus fully redundant, uninterrupted power.
All the facilities have racks pre-installed, some of the key differences
are listed below.
The W91 racks can accommodate
shelves for holding equipment that is not designed for a rack.
The W91 space has the lower power density and is limited to about
6KW of power draw in each rack.
The W91 facilities do not include UPS, tenants may supply battery
UPS themselves.
Bates racks use rack-door water heat-exchangers to capture exhaust heat
efficiently. The Bates racks are designed to accommodate
rack mounted computers.
Bates racks support medium power draw. They are designed for
an average electrical load of 10KW per rack. Individual racks can
exceed this slightly, but higher than 10KW rack loads need to
be planned in coordination with facility staff.
The Bates rack-door design does limit the depth of rack equipment
slightly, and prospective tenants should include this consideration
in their check list.
Bates has a small amount of flexible space that can accommodate odd
sized or pre-purchased racks.
Bates has a dedicated set of racks for UPS backed circuits that provide
10-15 minutes of coverage in the event of a power outage. The installation
of extra UPS hardware elsewhere in the facility is discouraged.
Managing off campus systems is simpler with systems that support remote,
out-of-band "IPMI" management. Most modern hardware does provide support
for this.
The MGHPCC racks use a hot-aisle containment design that captures exhaust
heat as it passes through in-row chillers. The design is optimized for rack
mount compatible hardware.
MGHPCC racks are grouped into pods that must average 14KW load, but that
can accommodate 25KW loads in individual racks. Each MGHPCC pod has between
20 and 24 racks.
MGHPCC provides a set of 10 generator backed racks that can maintain
service in the event of a power outage. These are designed for use in lieu
of individual tenant battery based, finite duration UPS units. The MGHPCC facility
currently strongly discourages the use of battery UPS.
The use of rails with rack mounted computers at MGHPCC can reduce
the need to carry out service working in the hot aisle areas of the facility.
The hot aisle regions can be uncomfortable to work in on a filled out
pod.
Most on campus buildings have a 10 gigabit/s circuit in their network entrance room. Network infrastructure from entrance rooms to desktops is either 1 gigabit/s or 100 megabit/s. For the most part there is no discernible difference in network performance between any of the facilities.
Both Bates and MGHPCC facilities are accessible to researchers
for visiting. The facilities are operational resources with
a certain amount of high-voltage circuitry, so visits do
require coordinating with an escort. Please feel free to
contact either Bob Redwine or Chris Hill if you wish to
arrange a visit.
Tenants at the facility can be authorized for unescorted
access.
Yes. Both Bates and MGHPCC facilities have tenant spaces that can be used for safely storing standby spare parts and space to hold items prior to installation and/or removal.
For large equipment moves the research computing moving program may be able to help. The research computing program is working with several computer equipment movers already. Small items going to Bates can use a shuttle service that runs between MIT and Bates as needed. Small items going to MGHPCC can be shipped by Fedex or UPS or can be transported by a subcontractor as needed.
Each of the W91, Bates and MGHPCC set of racks are controlled by MIT. At this stage, as with many other MIT facilities, there is no need for a special commitment in typical situations. Proposal budgets should include allowances for any direct cost charges by the facilities and should budget for any sizable use of remote subcontractor services. Proposals that may result in more than 10 racks of equipment or that may require facility or engineering enhancements should contact the facilities to develop an appropriate budget and plan.
The following text can be incorporated into proposals that need to include a facility statement regarding the research computing spaces.
MIT operates three shared research computing locations with a total of more than 170 racks of capacity. The locations can support a total of 2MW of compute load collectively. Each location is accessible from the main MIT campus via dedicated 10 gigabit/s links. The locations provide environmentally controlled rooms for housing computers together with a cohort of facilities maintenance and operations staff. This ensures a quality environment for housing computers that is actively monitored for performance and for security on a 24/7 basis. Technical assistance is available at each location. The three locations consist of an on campus data center (W91), a facility at the MIT Bates Linear Accelerator Campus in Middleton, MA and the MIT portion of the Massachusetts Green High Performance Computing Center (MGHPCC). Both the Bates and the MGHPCC locations house high up-time Tier 2 data analysis and distribution centers for the CERN Large Hadron Collider experiment.
Rack rental budget requests should be entered as Management and Services (M&S) items. The budgeted amounts are subject to full F&A charges.
It is anticipated that the W91 rack rental will be eliminated in the future.
Rack rental budget amounts should be entered as Management and Services (M&S) items.
Virtual hands budget amounts should be entered as Management and Services (M&S) items.
The remaining rack rental at W91 is billed quarterly.