​Linux totally dominates supercomputers

It finally happened. Today, all 500 of the world's top 500 supercomputers are running Linux.

Linux rules supercomputing. This day has been coming since 1998, when Linux first appeared on theTOP500 Supercomputer list. Today it finally happened: All 500 of the world's fastest supercomputers are running Linux.

The last two non-Linux systems, a pair of Chinese IBM POWER computers running AIX, dropped off the November 2017 TOP500 Supercomputer list.

Overall, China now leads the supercomputing race with 202 computers to the US' 144. China also leads the US in aggregate performance. China's supercomputers represent 35.4 percent of the Top500's flops, while the US trails with 29.6 percent. With an anti-science regime in charge of the government, America will only continue to see its technological lead decline.

When the first Top500 supercomputer list was compiled in June 1993, Linux was barely more than a toy. It hadn't even adopted Tux as its mascot yet. It didn't take long for Linux to start its march on supercomputing

In 1993/1994, at NASA's Goddard Space Flight Center, Donald Becker and Thomas Sterling designed a Commodity Off The Shelf (COTS) supercomputer: Beowulf. Since they couldn't afford a traditional supercomputer, they built a cluster computer made up of 16 Intel 486 DX4 processors, which were connected by channel bonded Ethernet. This Beowulf supercomputer was an instant success.

To this day, the Beowulf design remains a popular, inexpensive way of designing supercomputers. Indeed, in the latest Top500 list, 437 of the world's fastest computers are using cluster designs that owe a debt of gratitude to Beowulf.

Linux first appeared on the Top500 in 1998. Before Linux took the lead, Unix was supercomputing's top operating system. Since 2003, the Top500 was on its way to Linux domination. By 2004, Linux had taken the lead for good.

"Linux [became] the driving force behind the breakthroughs in computing power that have fueled research and technological innovation," as reported by The Linux Foundation. In other words, Linux is dominant in supercomputing, at least in part, because it is helping researchers push the limits on computing power.

This happened for two reasons: First, because most of the world's top supercomputers are research machines built for specialized tasks, each machine is a standalone project with unique characteristics and optimization requirements. To save costs, no one wants to develop a custom operating system for each of these systems. With Linux, however, research teams can easily modify and optimize Linux's open-source code to their one-off designs.

For example, the new Linux 4.14 enables supercomputers to use Heterogeneous Memory Management (HMM). This enables GPUs and CPUs to access a process's shared address space. Exactly 102 of the Top500 are using now using GPU accelerator/co-processor technology. All of these will perform better, thanks to HMM.

And, just as importantly, as the Linux Foundation pointed out, "The licensing cost of a custom, self-supported Linux distribution is the same, whether you're using 20 nodes or 20-million nodes." Thus, "by tapping into the vast open-source Linux community, projects had access to free support and developer resources to help keep developer costs on par with, or below other operating systems."

Now that Linux has reached the apex of supercomputing, I can't imagine it losing its leadership. It will take a hardware revolution, such as quantum computing, to shake Linux's supercomputing grip. Of course, Linux may still reign since IBM developers are already working on porting Linux to quantum computers.