... We created a system that allows the user to easily create and share workflows, data and algor... more ... We created a system that allows the user to easily create and share workflows, data and algorithms. ... The scientific workflow system connects to shared content repositories, where users can easily share data, workflows, algorithms and annotations. ...
2008 IEEE Fourth International Conference on eScience, 2008
Abstract As scientific work becomes more complex, contextual information is critical to enable it... more Abstract As scientific work becomes more complex, contextual information is critical to enable its use. We have developed the concept of Knowledge Spaces that manage such information and scope the space of data, metadata, and algorithms relevant to the work of ...
ABSTRACT As researchers work with larger and higher dimensional computational data and perform mo... more ABSTRACT As researchers work with larger and higher dimensional computational data and perform more complex chains of analysis, it becomes increasingly difficult to share and reproduce analyses and scientific results with fellow researchers and interested stakeholders. A framework for the dynamic web publication of data products and computational services has been developed to address these issues and to enable rich, focused interactions with computational research products. The framework integrates state-of-the-art web application technologies, semantic content management capabilities and an exploratory scientific workflow system. Its goal is to support the full scientific life cycle from the design and development of computational procedures, analyses and visualisations to the publication of annotated results and executable services on the web. The published data are available within web pages displaying live maps, graphs, tables and other visualisations that enable significant in situ exploration and further analyses. These capabilities are particularly applicable in areas such as hydrology where geospatial data from different sources and at different spatial and temporal scales must be synthesised. The contribution of this work to the water resource community is in expediting the process of learning and enabling multi-faceted collaborations based on science-related electronic objects.
ABSTRACT The lack of rainfall data in many watersheds is one of the major barriers for modeling a... more ABSTRACT The lack of rainfall data in many watersheds is one of the major barriers for modeling and studying many environmental and hydrological processes and supporting decision making. There are just not enough rain gages on the ground. To overcome this data scarcity issue, a Web 2.0 digital watershed is developed at NCSA(National Center for Supercomputing Applications), where users can point-and-click on a web-based google map interface and create new precipitation virtual sensors at any location within the same coverage region as a NEXRAD station. A set of scientific workflows are implemented to perform spatial, temporal and thematic transformations to the near-real-time NEXRAD Level II data. Such workflows can be triggered by the users' actions and generate either rainfall rate or rainfall accumulation streaming data at a user-specified time interval. We will discuss some underlying components of this digital watershed, which consists of a semantic content management middleware, a semantically enhanced streaming data toolkit, virtual sensor management functionality, and RESTful (REpresentational State Transfer) web service that can trigger the workflow execution. Such loosely coupled architecture presents a generic framework for constructing a Web 2.0 style digital watershed. An implementation of this architecture at the Upper Illinois Rive Basin will be presented. We will also discuss the implications of the virtual sensor concept for the broad environmental observatory community and how such concept will help us move towards a participatory digital watershed.
ABSTRACT Recent advances in sensor technology are facilitating the deployment of real-time sensor... more ABSTRACT Recent advances in sensor technology are facilitating the deployment of real-time sensor networks into the environment that can produce measurements at high spatial and/or temporal resolutions. At the same time, advances in cyberinfrastructure technologies are allowing researchers to use large quantities of heterogeneous sensor data for improving understanding of large-scale environmental processes, as well as in monitoring and modeling the environments in which the sensors are deployed. In their endeavors, researchers often seek the "raw" sensor data in order to ensure that their analyses will be free from bias, but usually such data have already been processed in ways that may not meet the needs of the researchers. Furthermore, acquiring sensor data suitable for a particular application (e.g. at an appropriate spatiotemporal resolution) may require the re-purposing of sensor data that might be beyond the scope of the original sensor design and deployment. Similar to the recent revolution of computer server virtualization, it is now necessary to consider virtualization of sensors and sensor networks so that existing sensor networks and their measurements can be easily re-purposed and used in new ways. This study explores the creation of virtual sensors and their potential impacts on national environmental observatories such as the WATERS Network including how virtual sensors can facilitate broad community collaborations. Additionally, the process of validating virtual sensors through community participation and review will be discussed. To demonstrate how virtual sensors will benefit the real-time modeling of environmental systems, a case study will be presented that addresses the real-time prediction of combined sewer overflows in the Chicago Metropolitan area. Combined sewer overflow (CSO) forecasts rely heavily on measurements regarding the quantity and spatial distribution of rainfall. To improve the accuracy of CSO forecasts, data from virtual rainfall sensors are employed. The virtual sensors are created by performing three transformations on real-time reflectivity data measured by the Next Generation Weather Radar (NEXRAD) system. The first transformation registers the NEXRAD data grid with geographic coordinate system used by the CSO model. Second, the reflectivity data is converted to rainfall rates using a radar-rainfall relationship, and finally, the data are aggregated in time to provide rainfall accumulation estimates that correspond with the temporal discretization of the CSO model.
This paper addresses the problem of designing a highly interactive scientific meta-workflow syste... more This paper addresses the problem of designing a highly interactive scientific meta-workflow system that aims at building complex problem-solving environments. The meta-workflow is viewed as a framework that integrates heterogeneous workflow engines, ...
... We created a system that allows the user to easily create and share workflows, data and algor... more ... We created a system that allows the user to easily create and share workflows, data and algorithms. ... The scientific workflow system connects to shared content repositories, where users can easily share data, workflows, algorithms and annotations. ...
2008 IEEE Fourth International Conference on eScience, 2008
Abstract As scientific work becomes more complex, contextual information is critical to enable it... more Abstract As scientific work becomes more complex, contextual information is critical to enable its use. We have developed the concept of Knowledge Spaces that manage such information and scope the space of data, metadata, and algorithms relevant to the work of ...
ABSTRACT As researchers work with larger and higher dimensional computational data and perform mo... more ABSTRACT As researchers work with larger and higher dimensional computational data and perform more complex chains of analysis, it becomes increasingly difficult to share and reproduce analyses and scientific results with fellow researchers and interested stakeholders. A framework for the dynamic web publication of data products and computational services has been developed to address these issues and to enable rich, focused interactions with computational research products. The framework integrates state-of-the-art web application technologies, semantic content management capabilities and an exploratory scientific workflow system. Its goal is to support the full scientific life cycle from the design and development of computational procedures, analyses and visualisations to the publication of annotated results and executable services on the web. The published data are available within web pages displaying live maps, graphs, tables and other visualisations that enable significant in situ exploration and further analyses. These capabilities are particularly applicable in areas such as hydrology where geospatial data from different sources and at different spatial and temporal scales must be synthesised. The contribution of this work to the water resource community is in expediting the process of learning and enabling multi-faceted collaborations based on science-related electronic objects.
ABSTRACT The lack of rainfall data in many watersheds is one of the major barriers for modeling a... more ABSTRACT The lack of rainfall data in many watersheds is one of the major barriers for modeling and studying many environmental and hydrological processes and supporting decision making. There are just not enough rain gages on the ground. To overcome this data scarcity issue, a Web 2.0 digital watershed is developed at NCSA(National Center for Supercomputing Applications), where users can point-and-click on a web-based google map interface and create new precipitation virtual sensors at any location within the same coverage region as a NEXRAD station. A set of scientific workflows are implemented to perform spatial, temporal and thematic transformations to the near-real-time NEXRAD Level II data. Such workflows can be triggered by the users' actions and generate either rainfall rate or rainfall accumulation streaming data at a user-specified time interval. We will discuss some underlying components of this digital watershed, which consists of a semantic content management middleware, a semantically enhanced streaming data toolkit, virtual sensor management functionality, and RESTful (REpresentational State Transfer) web service that can trigger the workflow execution. Such loosely coupled architecture presents a generic framework for constructing a Web 2.0 style digital watershed. An implementation of this architecture at the Upper Illinois Rive Basin will be presented. We will also discuss the implications of the virtual sensor concept for the broad environmental observatory community and how such concept will help us move towards a participatory digital watershed.
ABSTRACT Recent advances in sensor technology are facilitating the deployment of real-time sensor... more ABSTRACT Recent advances in sensor technology are facilitating the deployment of real-time sensor networks into the environment that can produce measurements at high spatial and/or temporal resolutions. At the same time, advances in cyberinfrastructure technologies are allowing researchers to use large quantities of heterogeneous sensor data for improving understanding of large-scale environmental processes, as well as in monitoring and modeling the environments in which the sensors are deployed. In their endeavors, researchers often seek the "raw" sensor data in order to ensure that their analyses will be free from bias, but usually such data have already been processed in ways that may not meet the needs of the researchers. Furthermore, acquiring sensor data suitable for a particular application (e.g. at an appropriate spatiotemporal resolution) may require the re-purposing of sensor data that might be beyond the scope of the original sensor design and deployment. Similar to the recent revolution of computer server virtualization, it is now necessary to consider virtualization of sensors and sensor networks so that existing sensor networks and their measurements can be easily re-purposed and used in new ways. This study explores the creation of virtual sensors and their potential impacts on national environmental observatories such as the WATERS Network including how virtual sensors can facilitate broad community collaborations. Additionally, the process of validating virtual sensors through community participation and review will be discussed. To demonstrate how virtual sensors will benefit the real-time modeling of environmental systems, a case study will be presented that addresses the real-time prediction of combined sewer overflows in the Chicago Metropolitan area. Combined sewer overflow (CSO) forecasts rely heavily on measurements regarding the quantity and spatial distribution of rainfall. To improve the accuracy of CSO forecasts, data from virtual rainfall sensors are employed. The virtual sensors are created by performing three transformations on real-time reflectivity data measured by the Next Generation Weather Radar (NEXRAD) system. The first transformation registers the NEXRAD data grid with geographic coordinate system used by the CSO model. Second, the reflectivity data is converted to rainfall rates using a radar-rainfall relationship, and finally, the data are aggregated in time to provide rainfall accumulation estimates that correspond with the temporal discretization of the CSO model.
This paper addresses the problem of designing a highly interactive scientific meta-workflow syste... more This paper addresses the problem of designing a highly interactive scientific meta-workflow system that aims at building complex problem-solving environments. The meta-workflow is viewed as a framework that integrates heterogeneous workflow engines, ...
Uploads