This lesson is in the early stages of development (Alpha version)

Why Use a Cluster?

Overview

Teaching: 15 min
Exercises: 5 min
Questions
  • Why would I be interested in High Performance Computing (HPC)?

  • What can I expect to learn from this course?

Objectives
  • Be able to describe what an HPC system is

  • Identify how an HPC system could benefit you.

Frequently, research problems that use computing can outgrow the desktop or laptop computer where they started:

In all these cases, what is needed is access to more computers than can be used at the same time.

And what do you do?

Talk to your neighbour, office mate or rubber duck about your research. How does computing help you do your research? How could more computing help you do more or better research?

Doing Analysis or Running Code

A standard Laptop for standard tasks

Today, people coding or analysing data typically work with laptops.

/hpc-intro/A%20standard%20laptop

Let’s dissect what resources programs running on a laptop require:

Schematically, this can be reduced to the following:

/hpc-intro/Schematic%20of%20how%20a%20computer%20works

When tasks take too long

When the task to solve become heavy on computations, the operations are typically out-sourced from the local laptop or desktop to elsewhere. Take for example the task to find the directions for your next business trip. The capabilities of your laptop are typically not enough to calculate that route spontaneously. So you use website, which in turn runs on a server that is almost exclusively not in the same room as you are.

/hpc-intro/A%20rack%20half%20full%20with%20servers

Note here, that a server is mostly a noisy computer mounted into a rack cabinet which in turn resides in a data center. The internet made it possible that these data centers do not require to be nearby your laptop. What people call the cloud is mostly a web-service where you can rent such servers by providing your credit card details and by clicking together the specs of this remote resource.

The server itself has no direct display or input methods attached to it. But most importantly, it has much more storage, memory and compute capacity than your laptop will ever have. In any case, you need a local device (laptop, workstation, mobile phone or tablet) to interact with this remote machine, people typically call ‘a server’.

When one server is not enough

If the computational task or analysis to complete is daunting for a single server, larger agglomerations of servers are used. These go by the name of clusters or super computers.

/hpc-intro/A%20rack%20with%20servers

The methodology of providing the input data, communicating options and flags as well as retrieving the results is quite opposite to using a plain laptop. Moreover, using a GUI style interface is often discarded in favor of using the command line. This imposes a double paradigm shift for prospect users:

  1. they work with the command line (not a GUI style user interface)
  2. they work with a distributed set of computers (called nodes)

I’ve never used a server, have I?

Take a minute and think about which of your daily interactions with a computer may require a remote server or even cluster to provide you with results.

Key Points

  • High Performance Computing (HPC) typically involves connecting to very large computing systems elsewhere in the world.

  • These other systems can be used to do work that would either be impossible or much slower or smaller systems.

  • The standard method of interacting with such systems is via a command line interface called Bash.