The Australian War Memorial has been growing its digital asset repository for the last five years as it continues to digitise its archive of photos, videos, and other memorabilia.
At the end of each day at 4.55pm, a bulger and piper farewell visitors at the Australian War Memorial's Pool of Reflection with a moving Last Post ceremony. This is broadcast live daily via webcam on the Memorial's website.
The streaming of the service on a Dell-Aruba wireless network forms only a small part of the large body of work the technology team at the Australian War Memorial has been working on for more than a decade.
Daryl Winterbottom, head of information technology at the Australian War Memorial, explained that the core function of his department is to run a collection management database. A large part of that requires the team to digitise the memorial's collections, a project which the group officially started about five years ago.
"Naturally we have material that is historical and so we're digitising that. We have paper records such as samples of official diaries and private records, and to make them available to the public we do a lot of page scanning and digitising that material to make it available on the web. It also preserves the original file because people don't have to handle them to be able to access them," he said.
"Photographs are a major area as well. Donations are also coming in digital format, such as from Iraq and Afghanistan whether it comes off from somebody's mobile phone or official photographer, we have to preserve and store that; not only stills but also sound and video."
Annually, the memorial is offered between 40,000 to 50,000 donated items; where on average people donate 13 items. Winterbottom explained how the memorial is seeking to be more selective in the donations it accepts based on provenance and the relevancy the donation would have in helping tell stories.
"We are interested in collecting medals and to have as many Victoria Crosses in our collection, but we're limited to how many photographs you could have of a particular vessel leaving Sydney; for example, you try to get the best and most representative shot, but you don't necessarily need to collect all of them. Trying to manage that donation rate at the moment is a challenge," he said.
Film is also another medium that is being digitised, which Winterbottom said is proving to be the most difficult given the volume of data that is required to store historic films. In fact, the Australian War Memorial will soon be kickstarting a program that will be focused on digitising 5,000 pieces of footage using a recently purchased film scanner to preserve the detail of the original films at high resolution.
Winterbottom has predicted that when the film scanning program starts, the data storage level will eclipse, if not double, the existing 150 terabytes of data currently stored on-site in the memorial's EMC Isilon digital asset repository.
"The platform we've bought its quite expandable certainly within the anticipated five-year life before one needs to consider replacing the whole lot," he said.
Winterbottom acknowledged, however, the cost to replace the increasingly growing storage will become a challenge down the track, particularly given that the annual IT budget is less than a million dollars. He said the memorial is reserving funding in anticipation of the potential cost it would have in the future.
To further maintain the preservation of the digitised files in case of a system failure or catastrophic event, Winterbottom explained the memorial keeps four copies of every individual file: Two copies on disk and two copies on tape all of which are stored across three physical sites, including two located nearby the Australian War Memorial.
As part of best practice, the memorial also carries out regular check summing of the quality of its files, as well as rolling random testing against the database of check sums to monitor if there are any evolving problems.
"We go to a lot of trouble to regularly check the integrity of those objects that are stored to ensure they haven't changed and for that we make a heavy reliance on check summing the file," he said.
When asked whether the memorial would consider moving its digitised assets into the cloud — an environment in which many federal government departments are moving into — Winterbottom said at a later date the memorial may consider it for disaster recovery, but the main copies of any files will be maintained in the memorial's own systems.
"At this point in time the curators are sensitive about ensuring the integrity of these objects. We have yet to see a provider that can service us and meet those requirements in full. It's not something immediately obvious to us yet. I think you would find the other agencies feel the same way," he said.