Beowulf clusters are made from commodity off-the-shelf, standard desktop hardware with a few tweaks. In Medusa's case, the whole cluster was networked via one giant 300+ port meshed Ethernet switch.
I was hired as part of a team to administer the hardware and write custom software- the precursor to modern DevOps tooling. Today's observability and configuration tools that we take for granted hadn't been invented yet, so we wrote many low-level tools ourselves. These are projects I contributed to:
smartmontools: We spent a lot of time tweaking and testing new hard drives not released to the public yet.
Our lab was the main contributor to the open source project.
I wrote configuration management tools that could be considered precursors to systems like Chef or Puppet.
We pushed the linux kernel to the limit. I tuned everything from hard drive sector alignment to the number of concurrent processes and the number of open files allowed by the kernel. I hacked on file systems, and ethernet frame sizes to optimize for our specific data.
I helped solve low level problems with the JFS filesystem by contributing the the linux kernel driver code in collaboration with its lead author.
I made lifelong friends, learned about Linux internals, and fell in love with distributed computing at scale.
