CHORDS

The Cloud-Hosted Real-time Data Services for the Geosciences (CHORDS), a real-time data management infrastructure will provide a system to archive, navigate and distribute real-time data streams via the Internet, and employ data and metadata formats that adhere to standards, which simplify the user experience.

The importance of real-time scientific data is ever increasing, but due to a lack of interoperability, real-time decision-making is infeasible in many cases that require streaming scientific data to be coupled with complex models. While EarthCube will provide an unprecedented framework for disseminating historical data sources, the use of real-time data raises a complex set of challenges. The CHORDS project’s goal is to investigate and further enhance the role of real-time data within the geosciences, by investigating the potential of streaming data sources in enabling adaptive experimentation and real-time hypothesis testing.

Most of the current real-time data acquisition processes are either proprietary, feed into mission-specific tools, or are otherwise not available to broader stakeholders within the geosciences. Standards, such as the OGC Sensor Web Enablement (SWE) specifications, have created interoperability interfaces and metadata encodings to fuse real-time sensor streams into information infrastructures, but methods to couple these streams with workflows and models have yet to be developed.

Public dissemination of real-time observations is uncommon, which constrains the experiments that can be conducted and limits the use of data in a variety of geoscientifc domains. For example, detecting faulty sensors or changing the direction of an ongoing sampling campaign is often unfeasible due to the need to conduct an analysis of streaming data. This often results in missed observations and wasted investments. Proprietary data distribution systems are often only directed toward visualizations. Tools used by mission scientists (e.g. vessel operators or sensor network technicians), are typically suited towards are specific application, expensive to deploy, brittle, and painful to maintain. CHORDS’s user-friendly, simple and inexpensive web-based infrastructure will address these challenges, lowers the barriers to entry, and enables real-time observational data collection, management and distribution.

CHORDS stakeholders form an amalgam of scientists and university researchers who need to manage and distribute real-time data streams, but who do not have the time or other resources to develop networking, software and computational infrastructure. With minimal cost and effort, an instance of CHORDS will be created on a cloud server, and simply configured through a web interface. The researcher can then add simple functionality to their distributed data systems, which transmit observations (or any other real-time artifact) to the CHORDS instance, using straightforward HTTP transactions. Through simple steps, the researcher will create a real-time networked data distribution system, without writing any software, purchasing any hardware, downloading and installing software, or employing specialized software engineering and IT staff.

Once configured, through simple HTTP queries the CHORDS instance will be available to spool the data artifacts to many clients. Researchers and other data consumers can then easily incorporate access methods into their research workflows. These workflows will include systems ranging from highly integrated command and control systems, data assimilation into models, field project control centers, standalone applications, or even web browsers and spreadsheets. A major advantage of CHORDS will be that the end user can work in whatever environment is most effective for them.

A CHORDS instance, running in the cloud, will provide core capability for data aggregation and dissemination. A configurable data store will provide a structured data holding, which supports queries, notifications and data export. Using cloud infrastructure will permit the system to easily scale to fit the changing needs of the project. CHORDS will provide a viable building block for a large population of users who need a simple and inexpensive capability for real-time data distribution.