Results 1  10
of
28
Stochastic Gradient Estimation
, 2006
"... We consider the problem of efficiently estimating gradients from stochastic simulation. Although the primary motivation is their use in simulation optimization, the resulting estimators can also be useful in other ways, e.g., sensitivity analysis. The main approaches described are finite differences ..."
Abstract

Cited by 39 (6 self)
 Add to MetaCart
We consider the problem of efficiently estimating gradients from stochastic simulation. Although the primary motivation is their use in simulation optimization, the resulting estimators can also be useful in other ways, e.g., sensitivity analysis. The main approaches described are finite differences (including simultaneous perturbations), perturbation analysis, the likelihood ratio/score function method, and the use of weak derivatives.
On choosing parameters in retrospectiveapproximation algorithms for simulationoptimization
 Proceedings of the 2006 Winter Simulation Conference. Institute of Electrical and Electronics Engineers: Piscataway
"... The Stochastic RootFinding Problem is that of finding a zero of a vectorvalued function known only through a stochastic simulation. The SimulationOptimization Problem is that of locating a realvalued function’s minimum, again with only a stochastic simulation that generates function estimates. ..."
Abstract

Cited by 20 (8 self)
 Add to MetaCart
(Show Context)
The Stochastic RootFinding Problem is that of finding a zero of a vectorvalued function known only through a stochastic simulation. The SimulationOptimization Problem is that of locating a realvalued function’s minimum, again with only a stochastic simulation that generates function estimates. Retrospective Approximation (RA) is a samplepath technique for solving such problems, where the solution to the underlying problem is approached via solutions to a sequence of approximate deterministic problems, each of which is generated using a specified sample size, and solved to a specified error tolerance. Our primary focus, in this paper, is providing guidance on choosing the sequence of sample sizes and error tolerances in RA algorithms. We first present an overview of the conditions that guarantee the correct convergence of RA’s iterates. Then we characterize a class of errortolerance and samplesize sequences that are superior to others in a certain precisely defined sense. We also identify and recommend members of this class, and provide a numerical example illustrating the key results. 1
Perturbation analysis and optimization of stochastic hybrid systems
 European Journal of Control
, 2010
"... Abstract We present a general framework for carrying out perturbation analysis in Stochastic Hybrid Systems (SHS) of arbitrary structure. In particular, Infinitesimal Perturbation Analysis (IPA) is used to provide unbiased gradient estimates of performance metrics with respect to various controllab ..."
Abstract

Cited by 13 (5 self)
 Add to MetaCart
(Show Context)
Abstract We present a general framework for carrying out perturbation analysis in Stochastic Hybrid Systems (SHS) of arbitrary structure. In particular, Infinitesimal Perturbation Analysis (IPA) is used to provide unbiased gradient estimates of performance metrics with respect to various controllable parameters. These can be combined with standard gradientbased algorithms for optimization purposes and implemented on line with little or no distributional information regarding stochastic processes involved. We generalize an earlier concept of "induced events" for this framework to include system features such as delays in control signals or modeling multiple user classes sharing a resource. We apply this generalized IPA to two SHS with different characteristics. First, we develop a gradient estimator for the performance of a linear switched system with control signal delays and a safety constraint and show that it is independent of the random delay distributional characteristics. Second, we derive closedform unbiased IPA estimators for a Stochastic Flow Model (SFM) of systems executing tasks subject to either hard or soft realtime constraints. These estimators are incorporated in a gradientbased algorithm to optimize performance by controlling a task admission threshold parameter. Simulation results are included to illustrate this optimization approach.
OnLine IPA Gradient Estimators in Stochastic Continuous Fluid Models
 Journal of Optimization Theory and Applications
, 2002
"... This paper applies Infinitesimal Perturbation Analysis (IPA) to lossrelated and workloadrelated metrics in a class of Stochastic Flow Models (SFM). It derives closedform formulas for gradient estimators of these metrics with respect to various parameters of interest, such as bu#er size, service ra ..."
Abstract

Cited by 9 (2 self)
 Add to MetaCart
(Show Context)
This paper applies Infinitesimal Perturbation Analysis (IPA) to lossrelated and workloadrelated metrics in a class of Stochastic Flow Models (SFM). It derives closedform formulas for gradient estimators of these metrics with respect to various parameters of interest, such as bu#er size, service rate and inflow rate. The IPA estimators derived are simple and fast to compute, and are further shown to be unbiased and nonparametric in the sense that they can be computed directly from observed data without any knowledge of the underlying probability law. These properties hold the promise of utilizing IPA gradient estimates as an ingredient of online management and control of telecommunications networks. While this paper considers singlenode SFMs, the analysis method developed is amenable to extensions to networks of SFM nodes with more general topologies. Key words and phrases. Stochastic Fluid Models (SFM), Infinitesimal Perturbation Analysis (IPA), network management and control. # Supported in part by the National Science Foundation under grant DMI0085659 and by DARPA under contract F306020020556.
WeC01.2 Fluid Approximation and Perturbation Analysis of a Dynamic Priority Call Center
"... Abstract — We analyze a call center with multiclass calls and dynamic priority service discipline, in which a lower priority customer becomes high priority when its waiting time exceeds a given service level threshold. For each priority queue, the service discipline is first come, first served. Base ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract — We analyze a call center with multiclass calls and dynamic priority service discipline, in which a lower priority customer becomes high priority when its waiting time exceeds a given service level threshold. For each priority queue, the service discipline is first come, first served. Based on a fluid approximation of the system, we apply infinitesimal perturbation analysis (IPA) to derive estimators for the derivative of the queue lengths with respect to the threshold parameter. We establish unbiasedness of the estimators, and report numerical results via simulation. I.
Infinitesimal Perturbation Analysis and Optimization for MaketoStock Manufacturing Systems Based on Stochastic Fluid Models. Discrete Event Dynamic System
, 2006
"... In this paper we study MakeToStock manufacturing systems and seek online algorithms for determining optimal or near optimal buffer capacities (hedging points) that balance inventory against stockout costs. Using a Stochastic Fluid Model (SFM), we derive sample derivatives (sensitivities) which, u ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
In this paper we study MakeToStock manufacturing systems and seek online algorithms for determining optimal or near optimal buffer capacities (hedging points) that balance inventory against stockout costs. Using a Stochastic Fluid Model (SFM), we derive sample derivatives (sensitivities) which, under very weak structural assumptions on the defining demand and service processes, are shown to be unbiased estimators of the sensitivities of a cost function with respect to these capacities. When applied to discretepart systems, we show that these estimators are greatly simplified and become nonparametric. Thus, they can be easily implemented and evaluated on line. Though the implementation on discretepart systems does not necessarily preserve the unbiasedness property, simulation results show that stochastic approximation algorithms that use such estimates do converge to optimal or near optimal hedging points. 1
Multiintersection Traffic Light Control Using Infinitesimal Perturbation Analysis?
"... Abstract: We address the traffic light control problem for multiple intersections in tandem by viewing it as a stochastic hybrid system and developing a Stochastic Flow Model (SFM) for it. Using Infinitesimal Perturbation Analysis (IPA), we derive online gradient estimates of a cost metric with res ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
Abstract: We address the traffic light control problem for multiple intersections in tandem by viewing it as a stochastic hybrid system and developing a Stochastic Flow Model (SFM) for it. Using Infinitesimal Perturbation Analysis (IPA), we derive online gradient estimates of a cost metric with respect to the controllable green and red cycle lengths. The IPA estimators obtained require counting traffic light switchings and estimating car flow rates only when specific events occur. The estimators are used to iteratively adjust light cycle lengths to improve performance and, in conjunction with a standard gradientbased algorithm, to obtain optimal values which adapt to changing traffic conditions. Simulation results are included to illustrate the approach.
Quasidynamic Traffic Light Control for a Single Intersection
, 2013
"... We address the traffic light control problem for a single intersection by viewing it as a stochastic hybrid system and developing a Stochastic Flow Model (SFM) for it. We adopt a quasidynamic control policy based on partial state information defined by detecting whether vehicle backlog is above ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
We address the traffic light control problem for a single intersection by viewing it as a stochastic hybrid system and developing a Stochastic Flow Model (SFM) for it. We adopt a quasidynamic control policy based on partial state information defined by detecting whether vehicle backlog is above or below a certain threshold, without the need to observe an exact vehicle count. The policy is parameterized by green and red cycle lengths which depend on this partial state information. Using Infinitesimal Perturbation Analysis (IPA), we derive online gradient estimators of an average traffic congestion metric with respect to these controllable green and red cycle lengths when the vehicle backlog is above or below the threshold. The estimators are used to iteratively adjust light cycle lengths so as to improve performance and, in conjunction with a standard gradientbased algorithm, to seek optimal values which adapt to changing traffic conditions. Simulation results are included to illustrate the approach and quantify the benefits of quasidynamic traffic light control over earlier static approaches. 1.
Infinitesimal perturbation analysis for a single stochastic fluid model node with a class of feedback controlled traffic
 in Proceedings of the 2004 American Control Conference
, 2004
"... Abstract — In this paper we adopt the Stochastic Fluid Modeling framework for management and control of communication networks and attempt to explicitly model feedback controlled sources. Specifically, the inflow process consists of a feedback controlled source which is modelled as a hybrid automat ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
Abstract — In this paper we adopt the Stochastic Fluid Modeling framework for management and control of communication networks and attempt to explicitly model feedback controlled sources. Specifically, the inflow process consists of a feedback controlled source which is modelled as a hybrid automaton (it has both, timedriven as well as eventdriven dynamics). This paper derives Infinitesimal Perturbation Analysis (IPA) derivative estimators for the buffer occupancy and throughput with respect to a node parameter (i.e., the buffer size). As also shown in earlier work [1], [2], [3], such estimators are used together with stochastic approximation techniques to dynamically determine the optimal operating point. I.
Application of IPA to Fluid Petri Nets
"... Infinitesimal Perturbation Analysis (IPA) recently has been extensively investigated in the setting of fluid queues, where it was shown to yield simple algorithms for computing the gradients of several performance functions. More lately, efforts have been made to extend its application domain from f ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
(Show Context)
Infinitesimal Perturbation Analysis (IPA) recently has been extensively investigated in the setting of fluid queues, where it was shown to yield simple algorithms for computing the gradients of several performance functions. More lately, efforts have been made to extend its application domain from fluid queueing networks to other kinds of stochastic hybrid systems. In this vein, the present paper inaugurates a study of the application of IPA to a class of hybrid Petri nets. The main point of concern is the modeling element of the fluid transition with multiple input places, representing concurrency and synchronization in Petri nets, and not yet studied in the context of IPA. We first derive the IPA gradient of the throughput with respect to fluid flow parameters at the input places, and then consider an example of optimizing throughput in a forkjoin system. Simulation experiments are presented in support of the theoretical results. We point out that the main purpose of the paper is to initiate a study of IPA in the setting of hybrid Petri nets, and not to consider application examples. Published as: