WSC 2008 Final Abstracts

Comparing Designs for Computer Simulation Experiments
Rachel Terese Johnson and Douglas C Montgomery (Arizona State University), Bradley Jones (SAS Institute) and John W Fowler (Arizona State University)

Abstract:
The use of simulation as a modeling and analysis tool is wide spread. Simulation is an enabling tool for experi-menting virtually on a validated computer environment. Often the underlying function for the results of a computer simulation experiment has too much curvature to be adequately modeled by a low order polynomial. In such cases finding an appropriate experimental design is not easy. This research uses prediction variance over the volume of the design region to evaluate computer simulation experiments assuming the modeler is interested in fitting a second order polynomial or a Gaussian Process model to the response data. Both space-filling and optimal designs are considered.

Using Simulation Early in the Design of a Fuel Injector Production Line
Mustafa H. Tongarlak, Bruce Ankenman, and Barry L Nelson (Northwestern University) and Laurent Borne and Kyle Wolfe (Delphi Corporation)

Abstract:
Delphi Corporation decided to use simulation from concept development to installation of a new multimillion dollar fuel injector production line. In this paper we describe how simulation was employed in the concept development phase to assess whether production targets required for financial viability were feasible and to identify the critical features of the line on which to focus design-improvement efforts.

Design of Experiments: Overview
Jack P.C. Kleijnen (Tilburg University)

Abstract:
Design Of Experiments (DOE) is needed for experiments with real-life systems, and with either deterministic or random simulation models. This contribution discusses the different types of DOE for these three domains, but focusses on random simulation. DOE may have two goals: sensitivity analysis and optimization. This contribution starts with classic DOE including 2(k-p) and Central Composite Designs (CCDs). Next, it discusses factor screening through Sequential Bifurcation. Then it discusses Kriging including Latin Hypercube Sampling and sequential designs. It ends with optimization through Generalized Response Surface Methodology and Kriging combined with Mathematical Programming, including Taguchian robust optimization

Large Deviations Perspective on Ordinal Optimization of Heavy-Tailed Systems
Jose Blanchet (Columbia University), Jingchen Liu (Harvard University) and Bert Zwart (Georgia Tech)

Abstract:
We consider the problem of selecting the best among several heavy-tailed systems from a large deviations perspective. In contrast to the light-tailed setting studied by Glynn and Juneja (2004), in the heavy-tailed setting, the probability of false selection is characterized by a rate function that does not require as detailed information about the probability distributions of the system's performance. This motivates the question of studying static policies that could potentially provide convenient implementations in heavy-tailed settings. We concentrate on studying sharp large deviations estimates for the probability of false detection which suggest precise optimal allocation policies when the systems have comparable heavy-tails. Additional optimality insights are given for systems with non-comparable tails.

Mathematical Programming Representations for State-Dependent Queues
Wai Kin Victor Chan (Rensselaer Polytechnic Institute) and Lee Schruben (University of California, Berkeley)

Abstract:
Discrete-event dynamic systems with feedback, where the behavior of the system depends on the system state, are difficult to model due to the uncertainties and dependencies of system performance on the system state. Service systems, in particular, tend to exhibit this behavior where servers may work faster (or slower) when facing an increasingly long line of impatient customers. A common example is a state-dependent queue where the service rate depends on the queue size, which can change during service. In this paper, we present a mathematical programming representation for the sample path dynamics of a state-dependent queue, and illustrate its application in sensitivity analysis.

Discrete Stochastic Optimization Using Linear Interpolation
Honggang Wang and Bruce W Schmeiser (Purdue University)

Abstract:
We consider discrete stochastic optimization problems where the objective function can only be estimated by a simulation oracle; the oracle is defined only at the discrete points. We propose a method using continuous search with simplex interpolation to solve a wide class of problems. A retrospective framework provides a sequence of deterministic approximating problems that can be solved using continuous optimization techniques that guarantee desirable convergence properties. Numerical experiments show that our method finds the optimal solutions for discrete stochastic optimization problems orders of magnitude faster than existing random search algorithms.

Max-Min Optimality of Service Rates in Queueing Systems with Customer-Average Performance Criterion
Li Xia, Ming Xie, Wenjun Yin, and Jin Dong (IBM China Research Laboratory)

Abstract:
In this paper, we consider the optimization of service rates in queueing systems, especially in closed Jackson networks. The optimization criterion is the customer-average performance, which is another important performance metric compared with the traditional time-average performance. Based on the methodology of perturbation analysis, we can derive a performance difference equation when the service rates are changed. With this difference equation, we find the optimal service rates have a Max-Min property, i.e., the optimal service rates can be chosen from its maximal or minimal value. This property can reduce the complexity of this type of optimization problems. Moreover, we also prove the max-min optimality is valid for both state-dependent service rates and load-dependent service rates in queueing systems.

Evaluating the Transient Behavior of Queueing Systems via Simulation and Transfer Function Modeling
Jingang Liu and Feng Yang (West Virginia University)

Abstract:
Characterizing the transient behavior of queueing systems is a difficult problem, which has been addressed by either simplified analytical models or simulation. We seek to capture the transient performance of systems from a new perspective: based on high-fidelity simulation experiments, we estimate a number of transfer function models (the discrete approximations of those ODEs provided by an analytical approach) which characterizes the evolution of the system's dynamic behavior.

On Step Sizes, Stochastic Shortest Paths, and Survival Probabilities in Reinforcement Learning
Abhijit Gosavi (Missouri University of Science and Technology)

Abstract:
Reinforcement Learning (RL) is a simulation-based technique useful in solving Markov decision processes if their transition probabilities are not easily obtainable or if the problems have a very large number of states. We present an empirical study of (i) the effect of step-sizes (learning rules) in the convergence of RL algorithms, (ii) stochastic shortest paths in solving average reward problems via RL, and (iii) the notion of survival probabilities (downside risk) in RL. We also study the impact of step sizes when function approximation is combined with RL. Our experiments yield some interesting insights that will be useful in practice when RL algorithms are implemented within simulators.

Automating Warm-Up Length Estimation
Kathryn Hoad, Stewart Robinson, and Ruth Davies (Warwick Business School)

Abstract:
There are two key issues in assuring the accuracy of estimates of performance obtained from a simulation model. The first is the removal of any initialisation bias, the second is ensuring that enough output data is produced to obtain an accurate estimate of performance. This paper is concerned with the first issue, and more specifically warm-up estimation. A continuing research project is described that aims to produce an automated procedure, for inclusion into commercial simulation software, for estimating the length of warm-up and hence removing initialisation bias from simulation output data.

Stationarity Tests and MSER-5: Exploring the Intuition Behind Mean-Squared-Error-Reduction in Detecting and Correcting Initialization Bias
William W Franklin and K. Preston White (University of Virginia)

Abstract:
We explore the reasoning behind MSER-5, an efficient and effective truncation heuristic for reducing initialization bias in steady-state simulation. We also compare MSER-5 with the KPSS stationarity test as one means of investigating the possibility that MSER’s effectiveness is the result of its utility as a stationarity measure. Conversely, this comparison also lets us explore whether or not a stationarity test from the time-series literature can be used as an effective initialization bias-control heuristic. Finally, we investigate the use of an alternative form of MSER-5 that uses a variance estimator that adjusts for serial correlation.

Using Slithers of Simulation in a New Approach for Intelligent Initialization of Non-Terminating Systems
Philip G Brabazon (Nottingham University Business School)

Abstract:
Reduction and even avoidance of the initial transient is known to be possible using intelligent initialization but it is not an approach that has been actively researched for some time. Using ideas from complexity science and informed by the recently developed equation-free technique a new method for identifying initializing conditions is developed. Many short bursts of simulation, called slithers, are used to construct the dynamic function of a system. Importantly, the function reveals the point attractor of the dynamic system to which it will evolve and the conditions at the attractor define the initializing conditions for future simulation runs. The method is demonstrated by application to a queuing network.