With the AARNet Mirror hosting over 140TB of data, the latest redesign makes it much easier for users to find the files they’re looking for. This new iteration has increased overall reliability and functionality of the repository to better cater for the large number of users that use the mirror each day.

The AARNet Mirror hosts popular freely available content. It exists to provide on-net and geographically local copies of much-requested data, supporting projects that don’t have the resources to employ commercial content delivery networks.

The data available via the Mirror includes popular open source operating systems, academic research tools, publishing tools, and obscure data sets. Highlights include a data set that contains the first 100 billion digits of Pi as well as the entire OpenStreetMap archive.

The AARNet Mirror is run for the benefit of the researchers, educators and administrative groups of our customer institutions, but we do make it open access for all globally. As such it is accessed by a broad spectrum of users across the Australian internet. The Mirror is rarely commented upon until there’s an issue with it as it’s just thought of as a defacto part of the Australian internet landscape.

The Mirror is the primary Australian host for many projects, such as Ubuntu Linux, OpenBSD and FreeBSD. When a user identifies themselves as being in Australia to the installers of these operating systems, the AARNet Mirror is where the tools fetch data from.

Having this data available locally allows researchers and other users to access the content quickly, cost effectively and reliably, without having to find terabytes of storage themselves. This encourages ad-hoc access.

The AARNet Mirror is on its 7th iteration, having originally started at the Cooperative Research Centre for Distributed Systems Technology (DSTC) at the University of Queensland in the late 1990s. It was the first 10Gbps data service AARNet offered, and now runs at 25Gbps with data stored in both Canberra and Melbourne.

The current design of the AARNet Mirror leverages the snapshot functionality of the ZFS filesystem, ensuring that data stored is valid and correct, and every presented data repository is valid and consistent with the origin archives. Beaming over a hundred remote repositories into one place means there can be synchronisation and other failures outside AARNet’s control, and the snapshot functionality ensures we don’t present inconsistent archives that cause difficulties for users. Combined with a mature synchronisation reporting setup, we’re able to act quickly to deal with any archives being out of date.

On a quiet day, the AARNet mirror will routinely deliver between 10 and 20 terabytes of data to users. There are the days when a hosted archive delivers a new version, and we’ll see traffic volumes spike substantially. The key benefit continues to be the locality and ready fast access in an ad-hoc nature for our users.

The primary purpose of the AARNet Mirror is to support our customers. This means customers can request data to be mirrored and, as long as the data is freely distributable and we can organise a reasonable method of synchronisation, content can be added quickly and without difficulty.

Author: David Jericho, AARNet Systems Architect