From USB sticks to terabyte data transfers: how astronomers are making the universe feel smaller

Those telescopes are hosted in ‘dark skies’ areas of countries such as Australia, South Africa and Chile, where light pollution is limited and astronomers can enjoy the clearest views.

Researchers are accustomed to travelling from all over the world to spend weeks at a time at these research stations, studying the skies and gathering data to take back to their home institutions for further scrutiny.

But what happens when all the physical, non-digital routes to the data are cut off?

Adapting when travel is not an option

That’s the position Michael Burton, the director of Armagh Observatory and Planetarium (AOP) in Northern Ireland, found himself in over the past 15 months. Formerly a professor at the University of New South Wales, he’d been working on datasets from the Australian telescope when the COVID pandemic struck.

Ciaran Magee, ICT manager at AOP, takes up the story:

“Michael couldn’t fly to Australia. Even the postal service in and out of Australia was shut down for a very long time. So we couldn’t even get the data put onto a USB stick and posted to us, which is how we used to transfer data, even though the risk of damage was high.”

Fortunately, there was another option.

Transferring massive datasets

AOP uses the Janet Network, which is linked to other European and worldwide national research and education networks (NRENs) through the GÉANT data network. This kind of research data transfer requirement is one of the raison d’etres of the global NREN community, making it eminently feasible to transfer a massive dataset from a telescope in Australia to Armagh, to be computed on AOP’s own servers.

Ciaran says:

“We’re currently moving four terabytes of data from Australia to Armagh.

“It’s faster and more secure and stable to transfer it across the dedicated data network, even where the host network is sending it at a throttled rate due to the size of the upload, than it would be to try to do that over the internet. And, of course, it’s much faster than waiting for the COVID restrictions to ease enough for Michael to be allowed to travel to Australia, or for them to post it up on USB drives.

"It will allow him to continue his research in these COVID times – and beyond – in a more secure and timely way.”

Being able to transfer datasets across a network rather than physically doesn’t just benefit AOP’s director. AOP is a research centre, carrying out cutting edge work in several key areas of astrophysics, contributing to the international astronomical community. Its researchers need regular telescope time at international facilities.

“Australia is the most extreme scenario, but every year we would have up to five people going down to the telescope station in Majorca for six-week periods

“Forget beaches – the telescopes are at the very top of a mountain and it's actually really, really cold. It's not a lot of fun. The last researcher who went out before the pandemic was on his own there, in a telescope, for several weeks.”

Saving travel and time

Aside from the environmental benefits of reducing the number of flights astronomers have to take in order to access the data they need, condensing a lonely six-week stint to a simple wait for networked data levels the field for those who may not be able to head to the other side of the world for weeks at a time, due to family situations or other commitments.

“To be able to transfer the data across networks is infinitely better and the links are always improving. With the Janet Network we are on a one-gig network and from next year we’ll be upgrading that. We're going to have better and better bandwidth, and we wouldn't be able to do these transfers without it."

The extra bandwidth is also enabling some homegrown solutions. Operation of AOP’s own working telescope used to be a very manual process, with astronomers reluctant to sit in it all night when there was a good chance they would see little thanks to Armagh’s cloudy skies. With the improved bandwidth provided by Janet, astronomers can now control the dome remotely, from opening it up, to pointing the telescope and even getting alerts to close it quickly when the inevitable rain arrives.

“The better connection and bandwidth that we have is helping us with that service. But that's only a very small scenario compared to the much bigger time-saving that we get by sending data sets from overseas to here,”

It all adds to the appeal of Armagh Observatory, which was formed in 1790, and its Planetarium, the longest running in the British Isles and the largest in Ireland. Together, AOP continues to excite new generations about the vastness of space, from schoolchildren who travel from all over Ireland to have their imaginations stimulated in the Planetarium to the graduates from Queens, Belfast and Trinity, Dublin who work alongside AOP’s staff astronomers on their PhDs in astrophysics.