Tuesday, November 6, 2007 - 2:05 PM
227d

10E00 Emerging Cyberinfrastructure Capabilities and Opportunities

Du Du1, R. Badia2, Patrick Linke1, and Antonis Kokossis1. (1) Chemical and Process Engineering, University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom, (2) Barcelona Supercomputing Center, Barcelona, Spain

The systematic development of new products and processes relies upon the extensive experimentation with simulation and optimization models. Process and product development combine stages for experimental data generation and analysis, stages for knowledge acquisition and pre-conceptual screening, as well as a repeated application of conceptual and detailed models required to run in either sequence or parallel mode. Integration between stages is currently manual as resources are heterogeneous (experimental databases, PDFs, computer models, property and cost data, software, technical reports, and images) and widely distributed (separate or distant R&D groups sharing complementary or similar knowledge and expertise). Practices are limited due to the lack of computing power, and a similar lack of integration in the underlying methods and resources. Grids and hyper-infrastructure environments offer particularly attractive technologies with a potential to enable integrated applications and the design of distributed experiments in industrial environments. Grid platforms support peer-to-peer communication and are designed to coordinate activities across different locations. The systematic employment of Grids, however, requires an efficient design of the experiments and a partial re-deployment of the algorithms to fully exploit the technological advantages of the Grids. The paper addresses the design of synthesis experiments in hyper-infrastructure clusters that support GT4.0. The synthesis studies make use of stochastic optimization techniques that coordinate simulation experiments through a novel optimization algorithm tailored for distributed applications. The research findings report on two separate fronts: the features of a new (gridified) optimization algorithm as well as the deployment of lower and upper middleware used to monitor and control the performance of the computer experiments. The algorithm addresses large-scale applications and enables parallel data analysis and knowledge acquisition. The experiments are designed around homogeneous sections of the Markov chains and are distributed over separate pools (repositories) that could be accessed and analyzed externally. Periodical redistribution of solutions offers external controls and could lead to significant acceleration. The middleware tools embedded include Superscalar and Gridway. The former is a user-friendly framework that provides an abstract layer to program applications. It increases the performance of parallelization and is composed by graphical interfaces useful for the design of Grid configurations. Superscalar is useful for the automatic deployment of the experiments and is particularly relevant to the application as it is specifically designed for problems composed by several (coarse) grain tasks. As the tasks are called repeatedly, Superscalar is able to execute the coarse grain tasks in independent Grid servers (sequentially to those with data dependencies and concurrently otherwise). GridWay is a workload management tool and a programming framework for grids based on the Globus Toolkit components, both pre-ws and ws. It allows an unattended and efficient execution of jobs on dynamic Grid environments. GridWay automatically performs all the job scheduling steps, provides fault recovery mechanisms, and adapts job scheduling and execution to the changing Grid conditions. GridWay supports the Distributed Resource Management Application API, specified by the Global Grid Forum to develop distributed applications. The implementation enables the visualization of results and the analysis of solutions without a need to interrupt the optimization. The paper reports on the efficiency of the implementation, explains accelerations achieved against different applications and summarizes recommendations for future work.