General Structure

The next generation supercomputer system, Shirokane3, began operation from April 2015. Shirokane3 is not just equipped with high-speed processing abilities and IO functionality, but a system that both possesses in-operation extensibility and conserves electric power.

Furthermore, one priority segment began operation from January 2015.

Shirokane3 General Structure

Shirokane3 performance

Cooling equipment

System name	Number of nodes	Rpeak (Tflops)	CPU	OS	CPU Cores	memory /node	Disk
Compute node Fat	6	8.17	Xeon E7-8857 v2 (3.0 GHz)	Red Hat Enterprise Linux	288	2 TB	600 TiB
Compute node Thin	469	414	Xeon E5-2670 v3 (2.3 GHz)		11,160	128 GB	-
Home Disk	-	-	-	-	-	-	12 PiB
Archive Disk	-	-	-	-	-	-	1 PiB + 21 PB

Compute node Fat

One node has roughly 2 TB of memory; this group of computenodes have 1.6 TB internal flash memory disks built-in. The home disk is separate, and connects to an external 600 TiB disk setup.

Compute node Thin

With 24-core Intel Xeon E5-2670 c3 CPUs and equipped with 128 GB of memory, they are Shirokane3's largest group of compute nodes. The group is composed of 469 total nodes, and of those, 24 are equipped with an Xeon Phi 7120P for enhanced operating performance.

Home Disk

The home disk has approximately five times the disk capacity of Shirokane2, and is endowed with a high speed of 12 PiB, with a high-capacity disk domain. Per file system, it can create upwards of 130,000 files a second, and upwards of 150,000 directories a second.

Archive Disk

Compared to the home disk, it is a lower-priced and high-capacity domain. The high-speed storage domain consists of 1 PiB, while the low-speed high-capacity tape domain is 21 PB. Thanks to the hierarchical storage management system, the access time, file size and other factors are taken into account and data is automatically moved to both domains. Cutting-edge devices and technology have been adopted in order to make best use of a limited power supply for the Shirokane3.

Shirokane3 employs an indirect evaporation cooling system. Warm air inside the room is released to the Oasis outside, then passes through a polymer heat exchanger. Water is sprinkled on the heat exchanger, and the resulting vaporization heat is used to cool the air inside the room. Typically fans are used for cooling year-round, without using a compressor. Theoretical PUE is estimated to be 1.06.

Through the use of a 6 TB HDD filled with helium, the electricity consumption per 1 TB has been cut in half. The maximum electricity consumption of the huge 12 PB file system is a mere 65 kW.

The 469 nodes in the Compute node Thin group are equipped with the latest CPU available, the Intel Xeon E5-2670 v3. This CPU, codenamed Haswell during development, can perform theoretical floating-point arithmetic at a rate twice that of existing CPUs, with highly efficient power use.

The system also takes other various considerations in account, so as to use the minimum amount of power necessary for large-scale calculations: automatically suspending servers in the absence of jobs, using the high-capacity tape library, and so forth.

Top of Page