QIMR Berghofer is a Brisbane-based medical research institute working to better understand, prevent, and treat disease. Bioinformatics Manager John Pearson oversees the institute’s high-performance computing (HPC) platform and works across genomics, cancer research and research computing. 

He understands first hand the challenge of managing cancer genomics data, where a single human genome can generate about 100 gigabytes of compressed data. In cancer studies, researchers often need around three genomes’ worth of sequencing for each patient, meaning a study of 100 patients can quickly grow to around 30 terabytes of data.

To support sharing of these large datasets, QIMR Berghofer developed SeqHaven, a dedicated infrastructure project designed to make genomic data available for research. SeqHaven uses AARNet’s high-speed research and education network and Globus to transfer large genomic datasets securely and reliably with research organisations in Australia and overseas.

Building SeqHaven for secure genomic data sharing

Developed with support from the National Collaborative Research Infrastructure Strategy (NCRIS) and Australian BioCommons, SeqHaven provides QIMR Berghofer with separate network, security, server, and storage infrastructure for sharing data securely at scale.

For John, the challenge was not only managing these large volumes of data, but finding a reliable way to move it. The team was initially looking at using a tool such as File Transfer Protocol (FTP), or to potentially build something themselves, but ultimately adopted Globus.

“What brought us to Globus was knowing it was developed by people like us who understand academic research,” John said. 

Globus was already widely used by research organisations, national infrastructure providers, and existing collaborators. Combined with local support from AARNet, this meant QIMR Berghofer could connect into an established research data-sharing ecosystem without starting from scratch. Globus also supports users with different levels of technical capability through web, command line, and API options, while allowing large transfers to be managed in the background.

From data distribution to collaboration

QIMR Berghofer has completed three pilot transfers using Globus. In two cases, the receiving organisations were already using Globus, which meant administrators at both ends could manage the transfer directly, making the process significantly easier for the researchers involved. 

“The data moved just like water through a garden hose,” John said. “Globus takes the complexity out of the transfer. I could talk to the researchers at the other end, while the technical teams moved the data in the background.”

The pilots also revealed an unexpected benefit. What began as a data distribution exercise has also become a collaboration tool. In one instance, a smaller organisation was unable to receive an 18-terabyte dataset in full. Using Globus, the teams were able to exchange smaller subsets of the dataset, test formats, refine outputs, and move information back and forth multiple times.

“This is exactly the kind of challenge AARNet and Globus are designed to support,” said Greg D’Arcy, Senior Manager of Research and Innovation Engagement at AARNet. “When researchers are working with datasets at this scale, reliable data movement is not just a technical requirement. It helps teams collaborate, reuse data, and make the most of the investment already made in generating it.”

Opening cancer datasets to more researchers

Having control over data distribution is also changing how QIMR Berghofer thinks about the value of the datasets it produces. In the past, the institute primarily shared raw data, often by sending it to large international genomic repositories. With Globus and SeqHaven, John sees an opportunity to share data at multiple stages of the analysis process.

Large research groups with the infrastructure to handle tens or hundreds of terabytes will still be able to access raw data. Smaller labs, however, may benefit more from processed analysis datasets, such as lists of variants, insertions, deletions, and substitutions, which are easier to use without requiring the same computing resources.

QIMR Berghofer is also exploring whether researchers who use the data could contribute new analysis outputs back into SeqHaven, allowing useful results to be shared with others and building on the original sequencing work.

By sharing data at multiple stages of the analysis process, QIMR Berghofer could help more researchers use cancer genomics data, increasing the impact of datasets that are expensive and time-consuming to generate. 

“Having control of the means of distribution, and having a system like Globus, has completely changed the way I think about sharing data,” John said.