Supercomputing at Virginia Tech


By Sarah Stewart, a senior in industrial and systems engineering

Because of movies like Transcendence and Lucy, the public is becoming more acquainted with the idea of supercomputing. Supercomputing, or high performance computing (HPC), is the use of a supercomputer to perform advanced calculations that cannot be solved on a regular computer, or would take an infeasible amount of time to solve — perhaps months or years. With this resource, researchers, students and faculty no longer have to limit their problems and simulations to only what one computer can handle. The Advanced Research Computing (ARC) group spearheads Virginia Tech’s research in supercomputing, which is broken into two areas: high performance computing (HPC) and visualization.

Virginia Tech currently houses four supercomputers, three of which are directly accessible to students. Brian Marshall, one of the computational scientists on the HPC side, referred to the supercomputers as clusters. Cluster refers to connecting several computers in parallel to form one supercomputer, rather than creating one large, complex single machine. The four supercomputers are BlueRidge, HokieSpeed, HokieOne and Ithaca.

BlueRidge is the supercomputer that has received the most press. It is the fastest cluster available, but in order for students to run jobs on it, they must be sponsored by their advisor.

HokieSpeed is a graphic processing unit-accelerated (GPU) cluster that uses the same graphics processing units originally targeted to the video game and PC display market. These GPUs are especially useful for accelerating visualization packages. HokieSpeed is also the most efficient, or “green,” of the four supercomputers.

HokieOne has shared memory. This means it appears to be one large supercomputer, rather than a cluster, to the program you are running on it. Underneath, it is still a cluster of smaller computers.

Lastly, Ithaca targets new users. It can run Matlab software, a programming language many engineering and math students are familiar with. All jobs submitted to the clusters are monitored, so if you attempt to do something like mine bitcoins, you will lose all of your supercomputer privileges permanently.

One of Marshall’s projects involves the Virginia Department of Environmental Quality (DEQ). The DEQ had been running their simulations on Athena, a supercomputer that Virginia Tech has retired. Marshall is currently working to change the simulation code so that it can be run on BlueRidge, while striving to make the simulations run faster and better than before.

Another HPC project involves NVIDIA Jetson development boards, which are typically used for automotive applications. Marshall, James McClure (another computational scientist at ARC) and an undergraduate student are working with network traffic monitoring on campus. Currently, the IT Security Office is only capable of monitoring a small amount of network traffic since their methods are not automated. Marshall, McClure and the undergraduate student are looking to place several of the NVIDIA Jetson boards around campus for the purpose of filtering traffic so that only important or suspect internet traffic is sent to the IT Security Office to monitor.

ARC’s Visual Computing group provides high-performance visualization infrastructure and services to take the simulation results, up to terabytes of information, and turn them into interactive 3-D visuals that we can understand. Visualization at ARC includes the Visionarium Lab in Torgerson Hall, and high performance clusters at the Andrews Information Systems Building. In the Visionarium, students and faculty have access to a number of high-resolution, stereoscopic and immersive displays. For example, the VisCube is a room consisting of three 10 by 10-foot walls and floor with 1920 by 1920 pixels each.

When entering the VisCube, special glasses are worn to transform two 2-D images into immersive 3-D virtual reality. There is a virtual camera rendering a slightly different view of the scene for each eye, just like regular human vision. The user navigates the virtual world with a handheld controller that is tracked and the image adjusts appropriately to the user’s perspective. This allows you to travel through and around objects, with the virtual image appearing to float in the center of the VisCube.

High performance computing and virtual reality are commonly used for applications in fluid dynamics and molecular dynamics as well as design, architecture and mechanics. Computational science and HPC simulation are providing new insights into real (and proposed) systems. For example, Professors Burns, Borggaard, Cliff, Zietsman, Gugercin and Iliescu in the Mathematics Department and the Interdisciplinary Center for Applied Mathematics (ICAM) are building improved control systems for energy-efficient buildings (see images). In these images, the thick tubes represent the air flow; the colors demonstrate temperature; and the wire-like cages represent physical boundaries.

Another area of visualization research is using commodity virtual reality hardware, like the Xbox Kinect and stereo televisions, to improve spatial visualization skills. Incoming freshmen engineering students are given a series of tests, one of which tests spatial visualization skills. Spatial visualization describes the act of manipulating an object in your mind. This includes turning a numbered die and cutting a cross-section of an object. Strength in spatial visualization skills has been linked to success in engineering. Studentswho need extra practice to strengthen those skills will soon have the opportunity to have a more interactive and fun experience.

The next frontier is scaling up to bigger data and systems, and connecting visual environments to live running simulations for computational steering. Recent work in ARC is enabling remote, clustered visualization on HokieSpeed and BlueRidge so that users can visualize their data without moving it. Managing large data and maintaining interactive frame rates for analysis at the speed of thought is an ongoing research challenge.

ARC is always looking to reach out to students and to publicize the free resources and classes that they offer. Although freshmen students may not be capable of jumping right into HPC or virtual reality, certainly juniors and seniors from various science and engineering majors are suitably prepared. Marshall described HPC as having a “steep learning curve,” but also found the learning process to be fun.