Abstract
AusCover (www.auscover.org.au) is a facility of the Terrestrial Ecosystem Research Network providing a national expert network and data delivery service for Australian biophysical remote sensing data time-series, continental-scale map products, and selected high-resolution data sets. There are ten partner agencies, located in all the Australian mainland capitals, participating in the AusCover program. The AusCover data system underpins the facility by providing enabling infrastructure for geographically distributed management and sharing of large data sets and by presenting a unifying view of the data collections through a virtual data centre. The data system addresses three of the main problems encountered by non-expert prospective users of remote sensing data: what data exist, where are they, and what do they mean? Simultaneously, it seeks to assist data producers and providers by providing frameworks, protocols and tools that make it easier to organise, describe and serve data sets. The volume of modern remote sensing data sets combined with that of historical archives, together with the range of custodians and providers, means that a distributed approach to data storage is essential. In order that diverse data sets can be identified, located and accessed within such a system, the adoption of open standards for describing and representing data has been critical to the development. For high-level descriptions of data sets, the ANZLIC profile of ISO 19115 was chosen so as to 1) simplify the metadata, 2) enable the use of existing tools such as Geonetworks and ANZmetlite for management, and 3) maximise the chances of its discovery through related data catalogues. An effort has been made, wherever practical, to store data sets in self-describing and architecture independent file formats such as NetCDF and HDF. This confers the advantages of easily storing data and metadata (and provenance information) together, allowing uniform access across platforms, and providing for a future migration path that can be fully automated. AusCover does not however require the use of these formats; if it is impractical to store data this way, AusCover can still host the data files and a metadata record to enable data discovery, or even only the metadata record if custodians require that the data be stored outside the AusCover system. The use of open file representation standards permits several value-adding services to be built on top of the data system. Advanced data server software, such as THREDDS and the OPeNDAP Hyrax systems, support the OGC web service protocols and the Data Access Protocol (DAP) on NetCDF, HDF and some other format files. The exposure of file-level metadata by these servers enables automatic harvesting of granule metadata for catalogue population and indexing. Additionally, the OGC and DAP protocols support user-customised on-demand subsetting and representation of data, both for visualisation and delivery. On top of the physical data system and server infrastructure, the AusCover Data Systems team is developing frameworks and protocols to assist management of the data content. The team provides guidance on data formatting and organisation, licencing and custodianship responsibilities. By working with the AusCover Data Products team, guidelines are being developed for documentation and quality assurance and are being incorporated into a governance framework, which defines the processes and mechanisms by which data are managed and the roles and relationships of the various participants.
The governance framework reduces the data discovery, access and interpretation burden on users by ensuring consistency across the data collection.