WSC 2005 Final Abstracts

Monday 1:30:00 PM 3:00:00 PM
Metamodels
Chair: Russell Barton (The Pennsylvania State University)

Abstract:
Many simulation experiments require considerable computer time, so interpolation is needed for sensitivity analysis and optimization. The interpolating functions are ‘metamodels’ (or ‘response surfaces’) of the underlying simulation models. For sensitivity analysis and optimization, simulationists use different interpolation techniques (e.g. low-order polynomial regression or neural nets). This paper, however, focuses on Kriging interpolation. In the 1950’s, D.G. Krige developed this technique for the mining industry. Currently, Kriging interpolation is frequently applied in Computer Aided Engineering. In discrete-event simulation, however, Kriging has just started. This paper discusses Kriging for sensitivity analysis in simulation, including methods to select an experimental design for Kriging interpolation.

Issues in Development of Simultaneous Forward-inverse Metamodels
Russell R. Barton (The Pennsylvania State University)

Abstract:
Metamodels provide estimates of simulation outputs as a function of design parameters. Often in the design of a system or product, one has performance targets in mind, and would like to identify system design parameters that would yield the target performance vector. Typically this is handled iteratively through an optimization search procedure. As an alternative, one could map system performance requirements to design parameters via an inverse metamodel. Inverse metamodels can be fitted ‘for free,’ given an experiment design to fit several forward models for multiple performance measures. This paper discusses this strategy, and some of the issues that must be resolved to make the approach practical.

International Collaborations in Web-based Simulation: A Focus on Experimental Design and Optimization
William E. Biles (University of Louisville) and Jack P. C. Kleijnen (Tilburg University)

Abstract:
This paper summarizes several years of research conducted by the authors to investigate the use of the world-wide web in conducting large-scale simulation studies. The initial efforts, at Tilburg University in 1999, were directed toward accessing several computer processors via the web and assigning each processor a portion of the simulation workload in a parallel replications simulation format. This early work utilized models coded in the Java-based Silk simulation language. By 2001, this research had extended the web-based simulation approach to more widely used simulation languages such as Arena. The present state of this research is that large-scale simulation studies can be conducted on a set of computers, accessed through the web, in a fraction of the time needed using a single processor.

Kriging Metamodeling in Discrete-Event Simulation: An Overview
Wim C.M. Van Beers (Tilburg University)

A Two-Phase Screening Procedure for Simulation Experiments
Susan M. Sanchez and Thomas W. Lucas (Naval Postgraduate School) and Hong Wan (Purdue University)

Abstract:
Analysts examining complex simulation models often conduct screening experiments to identify the most important factors. Controlled sequential bifurcation (CSB) is a screening procedure, developed specifically for simulation experiments, that uses a sequence of hypothesis tests to classify the factors as either important or unimportant. CSB controls the probability of Type I error for each factor, and the power at each bifurcation step, under heterogeneous variance conditions. CSB does, however, require the user to correctly state the directions of the effects prior to running the experiments. Experience indicates that this can be problematic with complex simulations. We propose a hybrid two-phase approach, FF-CSB, to relax this requirement. Phase 1 uses an efficient fractional factorial experiment to estimate the signs and magnitudes of the effects. Phase 2 uses these results in controlled sequential bifurcation. We describe this procedure and provide an empirical evaluation of its performance.

Multiple Predictor Smoothing Methods for Sensitivity Analysis
Curtis B. Storlic (Colorado State University) and Jon C. Helton (Arizona State University)

Abstract:
The use of multiple predictor smoothing methods in sampling-based sensitivity analyses of complex models is investigated. Specifically, sensitivity analysis procedures based on smoothing methods employing the stepwise application of the following nonparametric regression techniques are described: (i) locally weighted regression (LOESS), (ii) additive models (GAMs), (iii) projection pursuit regression (PP_REG), and (iv) recursive partitioning regression (RP_REG). The indicated procedures are illustrated with both simple test problems and results from a performance assessment for a radioactive waste disposal facility (i.e., the Waste Isolation Pilot Plant). As shown by the example illustrations, the use of smoothing procedures based on nonparametric regression techniques can yield more informative sensitivity analysis results than can be obtained with more traditional sensitivity analysis procedures based on linear regression, rank regression or response surface regression when nonlinear relationships between model inputs and model predictions are present.

Bootstrapping Simultaneous Confidence Bands
Russell C.H. Cheng (University of Southampton)

Abstract:
Once a regression has been fitted to data, it is usually necessary to add confidence intervals to indicate the accuracy of the fitted regression line. This can easily be done for individual explanatory variable values. However some-times confidence limits are needed simultaneously for the whole range of explanatory variable values of interest. In other words the problem is to construct a confidence band within which the entire unknown true regression line lies with given confidence. This article discusses computer intensive methods for doing this. The advantage of such methods compared with classical asymptotic methods is that accurate coverages can be obtained quite easily using bootstrap resampling.