This article was originally published in the Jan. 15, 2018 issue of SpaceNews magazine.

The European Space Agency team operating the star-mapping Gaia space telescope is preparing for its most comprehensive data release to date while defending its overtaxed data-processing network against budget cuts.

Since science operations began in mid-2014, the 700-million-euro ($843 million) Gaia mission has collected more than a trillion measurements, capturing star characteristics such as brightness, position and motion to create a 3D map of the Milky Way galaxy. The return on investment, measured in peer-reviewed papers, is strong at 250 and climbing. The processing of Gaia’s raw observations into meaningful measurements involves hundreds of astronomers spread across 20 countries. Gaia’s Data Processing and Analysis Consortium (DPAC) has been in place since 2006, or seven years before Gaia’s launch. Funded primarily by France, Germany, Spain and the U.K., it also gets a sliver of the 19 million euros ESA spends annually on Gaia operations.

Fred Jansen, the Gaia mission manager at ESA, said astronomers are generating meaningful research from Gaia, producing “something like one paper every day and a half,” but continued biennial delivery of refreshed Gaia data troves are not a given. Other astronomy missions are competing for the limited resources going into the DPAC, which has the arduous task of preparing Gaia data for dissemination.

“There are a number of highly ambitious projects running in Europe at the moment which use a similar paradigm as Gaia [for] processing the data because it’s too complex for a single organization to do it,” Jansen said, mentioning ESA’s Euclid dark matter mission launching in 2020 and the exoplanet hunter Plato launching in 2026. Those missions will need their own data-processing consortiums, he said, and their needs could put pressure on Gaia funding.

Astronomers got their first taste of Gaia data in September 2016: position and brightness info for 1.14 billion stars and a more advanced set of measurements for 2 million of the brightest of those stars thanks to a combination of Gaia measurements with archival data from Hipparcos, a four-year ESA astrometry mission launched in 1989. Early hardware problems, including one that let stray light into Gaia’s telescope, contributed to a nine-month delay in Data Release-1.

Another contributor was the underestimated complexity of processing Gaia’s star data. Prior to launch, Jansen said, the Gaia team thought it could release a new data set annually. As it turns out, the best Gaia can promise is once every two years. Data Release-2, scheduled for April, will included the more-advanced measurements for over a billion stars — a 500-fold increase over the 2016 release.

Jansen said a third release planned for late 2020 will build on the first two.

DPAC chairman Anthony Brown said ESA member states funding Gaia research are putting pressure on DPAC to cut costs by shedding people. Around 450 people calibrate and process Gaia data, Jansen said. Close to 180 work full time, said Brown.

Gaia officials see no easy way around the mission’s processing needs, which includes use of one of the world’s most powerful computers, the MareNostrum supercomputer in Barcelona. Brown and Jansen estimate that a 10 percent cut of DPAC’s budget — which they declined to quantify —would be bearable, but any more than that would jeopardize research.

Gaia’s data-processing needs stem less from the quantity involved — just under 50 terabytes collected as of December — and more from the fact that nearly every pixel in every image has scientific meaning.

“Gaia is ’big data’ not so much in the sense of the amount of data — CERN, Google, Facebook, etc., handle much larger amounts — but certainly when it comes to the complexity,” Brown said. “Every bit in our raw data counts, and the design of the measurements necessitates processing all data together in order achieve the ultimate performance, where all calibrations have to be derived from the same data.”

Furthermore, with each cycle, Gaia’s 3D map of the Milky Way becomes much more detailed, increasing its science value but putting a heavier burden on the pan-European team of experts creating and correcting the catalog.

“Normally, you would think that by repeating processing sufficiently you get into a routine and it becomes easier so you can divest resources, but in our case for the next three to four years things will be getting more complex because we are getting more precise,” Brown said. “Once we are well into an extended phase of the mission, it should be able to do with less resources.”

Prior to Gaia’s launch, the DPAC gave input on the telescope’s payload design (which carries sensors for astrometry, photometry and spectroscopy), worked on data processing requirements and software, and researched scientific algorithms, according to Brown. The DPAC also spent “a lot of effort” producing simulated Gaia telemetry, which supported large operational rehearsals ahead of launch, he said.

Gaia’s five-year mission ends next year, but Jansen said the spacecraft is healthy enough to reach 2024. If Gaia’s mission is extended the full five years, it would need three additional years of DPAC processing. Euclid should be in orbit by then and Plato will be much closer to launch. Should Gaia lose some DPAC resources by then, it would slow — but not stop — data harvests.

“We may have to sacrifice a few things, but at the moment it looks like we can keep the bulk of the necessary and guaranteed outputs,” Jansen said.

Extending Gaia would be well worth the effort, said Roger Davies, president of the European Astronomical Society.
“It is a mission which makes its impact by looking at how the sky changes over time,” he said. “Obviously it’s greatest impact comes at the end of the mission when it’s finished an it has the maximum baseline. We can expect things to get better and better.”

Davies described it as “a central mission for understanding galaxies,” and a mission that gave ESA its own research niche, building off Hipparcos, which he credited with reinvigorating astrometry, “making it essential to modern astrophysics.”

“What Gaia is doing is now capitalizing on that position,” he said.