Prof. Meyer explains how big data is changing astronomy

UMBC’s Eileen Meyer, assistant professor of physics, argues that recent technological advances in the big data era are changing how astronomers learn about the universe.  In an article published in outlets from Popular Science to Smithsonian Magazine, Meyer describes recent research that discovered thousands of black holes near the center of the Milky Way. The research team found them not with a new telescope, but “by digging through old, long-archived data,” she writes. “Astronomers are gathering an exponentially greater amount of data every day—so much that it will take years to uncover all the hidden signals buried in the archives.”

Over the past several decades, advances in communications have allowed astronomy research to shift away from being conducted largely by individual scientists working with the instruments available at their own institutions. Today, large, international research teams collaborate on projects, often using instruments shared by several institutions across multiple countries.

The research instruments themselves are also vastly more powerful today than in the past. Older instruments could only detect visible light (with wavelengths from about 400 to 700 billionths of a meter). New instruments examine everything from gamma rays (extremely high-energy waves with wavelengths less than a trillionth of a meter) to radio waves (with wavelengths as big as a meter).

Newer equipment can also quickly process more data—a lot more. Meyer writes that the Hubble Space Telescope has made 1.3 million observations since its launch in 1990, and it transmits 20 GB of data per week, “which is impressive for a telescope first designed in the 1970s.” Compare that to the Square Kilometer Array, scheduled for completion in 2020. “In just one year of activity,” Meyer notes, “it will generate more data than the entire internet.”

“These ambitious projects will test scientists’ ability to handle data,” Meyer writes. Archives will need to have capacity to store, and then make accessible, a million times more data than can be stored on a typical 1 terabyte external hard drive.

Although it poses challenges, “the data deluge will make astronomy become a more collaborative and open science than ever before,” Meyer says. Whereas before you had to be “an academic or eccentric rich person with access to a good telescope” to do this kind of research, Meyer says, that’s no longer the case. “Thanks to internet archives, robust learning communities and new outreach initiatives, citizens can now participate in science.”

Meyer has taken advantage of huge, stored data sets in her own work studying the jets of high-speed plasma that spew from black holes. She collected 400 raw images taken over 13 years by Hubble to reveal the structure of these jets for the first time.

“This kind of work was only possible because other observers, for other purposes, just happened to capture images of the source I was interested in, back when I was in kindergarten,” she says. “As astronomical images become larger, higher resolution and ever more sensitive, this kind of research will become the norm.”