#### DMCA

## SANDIA REPORT LDRD Final Report: Robust Analysis of Large-scale Combinatorial Applications NOTICE: LDRD Final Report: Robust Analysis of Large-scale Combinatorial Applications

### BibTeX

@MISC{Hart_sandiareport,

author = {William E Hart and Robert D Carr and Cynthia A Phillips and Jean-Paul Watson and Nicholas L Benavides and Harvey Greenberg and Todd Morrison and William E Hart and Robert D Carr and Cynthia A Phillips and Jean-Paul Watson and Nicolas L Benavides and Harvey Greenberg and Todd Morrison},

title = {SANDIA REPORT LDRD Final Report: Robust Analysis of Large-scale Combinatorial Applications NOTICE: LDRD Final Report: Robust Analysis of Large-scale Combinatorial Applications},

year = {}

}

### OpenURL

### Abstract

Abstract Discrete models of large, complex systems like national infrastructures and complex logistics frameworks naturally incorporate many modeling uncertainties. Consequently, there is a clear need for optimization techniques that can robustly account for risks associated with modeling uncertainties. This report summarizes the progress of the Late-Start LDRD "Robust Analysis of Largescale Combinatorial Applications". This project developed new heuristics for solving robust optimization models, and developed new robust optimization models for describing uncertainty scenarios. 3 Acknowledgments We thank Regan Murray for collaborating on the application of robust optimization techniques to water security applications. Executive Summary Many real-world problems are concerned with maximizing or minimizing an objective (e.g. maximizing profit, minimizing costs, or lowering of risk). Optimization methods are commonly used for these applications to find a best possible solution to a problem mathematically, which improves or optimizes the performance of the system. Many real-world optimization problems involve discrete decisions, such as selecting investments, allocating resources and scheduling activities. These discrete optimization problems arise in many application areas like infrastructure surety, military inventory and transportation logistics, production planning and scheduling, and informatics. Discrete models of large, complex systems like national infrastructures and complex logistics frameworks naturally incorporate many modeling uncertainties. Model factors like transportation times and demands in water networks are inherently variable. Further, other information like logistical costs and infrastructure capacity limitations may only be known at a coarse, aggregate level of precision. Although such models can be optimized using average or estimated data, solutions found in this manner often fail to reflect the risks associated with these modeling uncertainties. Consequently, there is a clear need for discrete optimization methods that can robustly account for risks associated with modeling uncertainties. So called robust optimization techniques find solutions that optimize a performance objective while accounting for these modeling uncertainties. Sandia's discrete optimization group has developed robust optimization methods for a variety of real-world problems, but a consistent challenge has been that existing robust optimization approaches cannot be reliably used on large-scale applications. This report summarizes the progress of the Late-Start LDRD "Robust Analysis of Large-scale Combinatorial Applications". This project's accomplishments can be grouped into three areas: • Robust Optimization with CVaR: Conditional value-at-risk (CVaR) is a risk metric that is commonly used in financial models. Discrete optimization formulations of CVaR have recently been developed for water security and facility location applications, but only small-scale problems can be practically solved with existing optimization solvers. We describe experimental analyses of CVaR problems that (a) characterize computational bottlenecks and (b) evaluate the performance of heuristic optimization solvers. • Minimizing Regret: The regret of a decision made under uncertainty refers to the impact of not having made an optimal decision without uncertainties. Uncertainty in discrete optimization models can often be analyzed by minimizing the maximum regret. We present a new minimax regret formulation that is mathematically stronger than previous approaches. 9 • Enabling Technologies: Several software development efforts were initiated to enable the solution of robust optimization applications. A new search strategy for the PICO integer programming solver was developed to avoid over-constraining the search in risk-constrained applications. Further, a Python module was developed to flexibly model and solve the nonlinear formulations that commonly arise in discrete optimization problems. We expect these new capabilities to directly impact Sandia's ability to address modeling uncertainties in a variety of new and ongoing efforts. For example, the following projects will leverage these capabilities in FY08: • Water Security (EPA): The EPA has been funding Sandia to develop contaminant warning systems for water distribution systems. A key element of this is the placement of sensors, which involves uncertain data. The EPA is interested in Sandia's robust optimization capabilities, and the new formulation developed in this LDRD addresses a key scalability challenge in this work: accounting for seasonal demand varitions. We do not expect this model to be used within the current SNL-EPA WFO project, but in FY08 we will leverage this when formulating a new WFO project with the EPA. • Scheduling Investments in Future Energy Supplies (LDRD): An ongoing LDRD project will leverage these robust optimization capabilities to address uncertainties in models used to plan investments our nation's energy infrastructure. Addressing uncertainties is a fundamental aspect of these models, but robust optimization is not the technical focus. However, our robust optimization results can be immediately applied to these models. • Aircraft Fleet Planning (LMSV): An ongoing Shared Vision project with Lockheed Martin will leverage the Pyomo software developed in this project to support a new application initiative. This software will be used to formulate and solve an aircraft fleet planning logistics model. In particular, Pyomo's ability to interface with aircraft design sub-models is critical to this project. Further, there is significant uncertainty in these logistics problems, and thus robust optimization techniques are particularly valuable for these planning activities. 2 Robust Optimization with CVaR The conditional value-at-risk (CVaR) metric is a risk measure that has been widely used in the finance community. The CVaR risk measure can be applied to applications in which problem uncertainty can be characterized by a set of scenarios. For example, in infrastructure security applications, scenarios might consider possible failures of infrastructure components due to attacks. In logistics models, scenarios might characterize the time needed to transport materials. These types of scenario-based optimization formulations are very flexible. They can integrate complex uncertainty models. For example, they allow optimization methods to be used effectively with data generated by more detailed simulation models. Further, they allow for the characterization of the impacts of uncertainties in a generic manner. For example, we have recently used CVaR to model risk in a water security applications In this section we describe an integer programming (IP) model for CVaR for sensor placement. Preliminary computational results highlight challenges with solving this IP on real-world sensor placement applications. We address this challenge in several ways. First, we consider bottlenecks in the solution of the CVaR integer program (IP); specifically, we consider the runtime of the root linear programming relaxation. Although the cost of this relaxation can be reduced, the total cost of the CVaR IP remains large. Consequently, we consider the application of several IP heuristics. A CVaR Integer Programming Formulation Value-at-Risk (VaR) is a percentile-based metric usually defined as the maximal allowable loss within a certain confidence level γ ∈ (0, 1) For example, suppose f ( x, ξ ) = ξ and Pr(ξ = 10) = Pr(ξ = 100) = 1 2 . Then, 11 If we think of f as some measure of risk, the chance constraint is a confidence level (γ) that the risk not exceed some level, which we minimize. (We can equivalently write Pr[ f ( x, ξ ) > u] ≤ 1 − γ, which says that we want the probability of exceeding some level (u) to be less than 1 − γ, say 5%. The objective is to minimize that level subject to that chance constraint.) We generally want to know what the VaR is at various values of γ, ranging from 90% to 99%. The Conditional Value-at-Risk (CVaR) is a related metric which measures the conditional expectation of losses exceeding VaR at a given confidence level. Technically, this expectation is the Tail Conditional Expectation (TCE), and CVaR is linearization of TCE investigated by Uryasev and Rockafellar To see this relation, let Ω be the set of scenarios, let ω i the probability of realizing scenario i, and f i ( x) be the value of f in scenario i ∈ Ω. Observe that by defining an indicator variable, ρ i , to be 1 if f i ≥ VaR and 0 otherwise, we can write TCE as, Next we define a vector of auxiliary variables y = (y i ) such that Then we can write, Noting that ∑ i ω i ρ i ≈ γ, we define CVaR as the approximation, Graphically, CVaR holds with equality at those points where γ = ∑ i ω i ρ i (the points where the usual function steps) and joins the steps with a linear overestimate of CVaR. Thus, CVaR serves as an approximation of the bilinear form found in the formulation of TCE and is continuous in γ. 12 We illustrate the use of CVaR by developing an IP for minimizing the CVaR of impacts in sensor placement problem. We consider the sensor placement formulation described Berry et al. [6]: This IP minimizes the expected impact of a set of contamination scenarios defined by A . For each scenario a, L a ⊆ L defines the set of locations that can be contaminated in the scenario, α a defines the weight of the scenario, and d ai defines the impact of the contamination; Berry et al. [6] consider a water security application, where typical impacts are population exposure, extent of contamination, and time to detection. The s i variables indicate where sensors are placed in the network, subject to a budget p, and the x ia variables indicate whether scenario a is witness at location i by a sensor. A limitation of this model is that is considers only the weighted average of scenario impacts, the expected impact. Thus rare, but potentially catastrophic, contamination scenarios will be essentially ignored. To address these possibly disastrous extremes we need to include some measure of the risk associated with a particular solution. As a risk metric that is sensitive to large tails CVaR is well suited to this task. Hence we have formulated the sensor placement problem with restricted risk: Alternately, we can formulate the risk adverse sensor placement problem as a goal programming problem by dropping the maximum CVaR constraint (13) and replacing the objective 13 function (7) with, Either formulation allows us to explore the efficient frontier of trade-offs between minimizing the expected case versus reducing our exposure to risky outliers. In the first formulation this can be done by solving the problem for a number of values, maxCVaR, assuming we know a reasonable range to work in. The second formulation avoids the need to know this range, instead we explore the efficient frontier by varying the parameter λ . Computational issues and the difficulties inherent in exploring the efficient frontier of non-convex, multiple-objective problems (such as integer programs) will probably require the use of both formulations or a combination of the two. Preliminary Computational Results Our preliminary computational analysis of CVaR considers a small sensor placement application for water security for which we can solve many CVaR optimization problems to optimality. We explore the efficient frontier by solving for the minimum expected value subject to an upper bound ("maxCVaR") on CVaR. Since we would like to investigate only the non-trivial points we seed our search by solving a weighted sum formulation ("MinBoth") with a very small weight (λ = .01) on CVaR. The data points from these experiments are plotted in Appendix A includes an article submitted for publication to the Journal of Infrastructure Systems. This article summarizes the use of this CVaR IP for real-world sensor placement applications; Watson, Hart and Murray [13] describes a preliminary version of this article, and a journal submission was completed as part of this LDRD. This article shows the difficulty associated with optimizing large-scale CVaR models. We were not able to solve the IP formulation for real-world sensor placement applications. Further, it was difficult to assess whether heuristic optimization methods provided near-optimal solutions. Thus, Minimizing CVaR From our previous results, a clear challenge is the analysis of IP models with highly constrained CVaR. In particular, simply minimizing CVaR can be a very challenging problem, even for small distribution networks. IP solvers have not proven effective at minimizing CVaR because the LP bounds used in this search process are very weak. Consequently, we have focused on the application of heuristic optimizers, which work quickly but do not guarantee that an optimal is found. In previous work, we have developed a GRASP heuristic for CVaR [6], and in this project we contrast this heuristic with two new IP solvers: a feasibility pump heuristic, and a fractional decomposition tree (FDT) heuristic. The feasibility pump heuristic is a recently developed strategy for quickly finding solutions to general integer programs. This heuristic starts with the optimal LP solution, and then solves a series of LP subproblems that attempt to drive this solution towards a feasible discrete solution. This heuristic has recently been integrated into the PICO IP solver, where it generates solutions used to prune the branch-and-bound tree used during search. FDT starts with the optimal LP solution, which is often not discrete. It then considers a series of decompositions of the fractional solution, into a convex combination of two solutions that are integral in one-or-more variables. Thus FDT iteratively fixes discrete solutions, but in a manner that is guided by the LP fractional solution. FDT has been recently developed by Carr and Phillips [7], and our application of FDT to CVaR is one of the first evaluations of this heuristic on real-world applications. An FDT implementation for minimizing CVaR was implemented in the AMPL modeling language. The GRASP and feasibility pump results were generated within a few minutes. The FDT heuristic generates a series of solutions; the first solution is generated within a few minutes, and generating all subsequent solutions required up to an hour. These results indicate that feasibility pump and FDT are not an improvement over the GRASP heuristic, either in final solution value or in terms of the required runtime. A clear limitation of heuristic methods is that they fail to provide a confidence bound in the final solution. One strategy for providing a bound is to compute the linear relaxation of the IP formulation, which relaxes the integrality constraints. The values of the LP relaxations are provided in A New Formulation for Minimizing Regret The regret of a decision made under uncertainty refers to the impact of not having made an optimal decision without uncertainties. For example, consider the context of selecting a facility location to meet "customer" demand (e.g. locating a fire station or waste dump). Facility demands are uncertain, and these uncertainties can often be characterized as a set of potential demand scenarios. In practice, one or more of these scenarios is likely to dominate future usage of the facility, but further information is unavailable when a decision maker selects the facility location. A minimax regret formulation minimizes the regret of a decision across all scenarios. Here, regret can be characterized as the difference between a solution and the solution value optimized for a particular scenario: so the canonical minimax formulation is This is also called the worst-case regret, since we are minimizing the worst regret overall. Chen et al. [8] consider a minimax regret model that minimizes the worst regret over the best 100α% (say 95%) of the scenarios. This is better than minimizing the worst case regret when one does not want an answer that is dominated by few scenarios (which may occur with low probability). For example, airports should not cater only to Thanksgiving and Christmas travel, but a worst-case regret formulation could do just that. The following sections critique this model and present an alternative formulation that can be used to optimize this modified regret formulation. In particular, this reformulation is motivated by the fact that the technique described by Chen et al. [8] is not practical for large facility location applications. The final section below discusses the relationship between this facility location model and the water security application discussed above. A Minimax Regret Formulation The original idea behind the facility location problem is that there is a network of customer node locations with a demand at each node and potential sites for p facilities that service these demands at a cost that increases with the distance from the facility to the customer. The problem then is to place the facilities so that the total demand is met at minimum cost. We start with demand nodes i numbered from 1 to m so that the first n nodes (from 1 to n) are the potential sites to place a facility, of which p of these sites will be chosen for putting facilities. We are given scenarios 1 to K. For scenario k we specify a demand h ik for each customer i and distance d i jk between each customer i and potential facility location j. We assign a probability q k that scenario k will occur and the minimum costV k for satisfying 19 the total demand given one knows a head of time that scenario k will occur;V k is obtained by solving a facility location problem for scenario k alone. The robust model proposed by Chen et al. [8] finds set of facility locations that minimize a regret measure. So, if x j for j = 1..n are binary variables that indicate where we will put our facilities and y i, j,k are binary variables that are 1 when customer i gets serviced by facility j under scenario k, the total cost of meeting the demand for scenario k is Since the lowest possible cost would beV k , there is a regret of R k from scenario k given by We could minimize a weighted sum of regrets or a worst-case regret, but in both cases outlier scenarios can significantly skew our solution. A worst-case regret may only be relevant for a small number of extreme scenarios, and a weighted sum can be similarly skewed by large outlier values. Although we would want a solution to work well in principle for all scenarios, that fact that we have uncertainties in the relevance of these scenarios motivates the decision maker to ignore these extreme scenarios for the analysis. Our robust measure based on regrets is to minimize the worst regret in the best 95% of the outcomes. If we have a binary variable z k indicating whether scenario k is in the best 100α% of the regrets, then we will minimize the maximum regret W where for each k we have the constraint which makes W larger than any regret R k in the best 100α% (that is for which z k = 1). Notice that these constraints are designed to say nothing we didn't already know when k is not one of the selected scenarios (z k = 0), but say exactly what we want when k is a selected scenario. To ensure that the z variables are set correctly, we need the constraint Unfortunately, the constraints that give lower bounds for W are non-linear, so we cannot solve this formulation with an integer programming solver in a standard manner. Chen et al. [8] resolve this by guessing the constants m k to be as close to but bigger than the actual regrets R k as possible. Then, the constraints can be used to bound W . The problem with this approach is that if these guesses for m k are off, one may have to make new guesses and solve the IP all over again. 20 Finally, we have the normal facility location constraints for the x variables (indicating facilities) and the y variables (indicating which facility services each customer). Since we are placing p facilities, we have Since in each scenario k, only one facility services any customer i, we have n ∑ j=1 y i jk = 1 ∀i ∈ {1, .., m}∀k ∈ {1, .., K}. Since a facility cannot service a customer if it were never built, we have As was stated earlier, our objective is to minimize W subject to the constraints of this section. A New Minimax Regret Formulation We have come up with several ideas for improving the formulation discussed in the previous section. The first idea is to turn the variable W into a constant by guessing its value to be some W * . We will soon see that this makes the cost of a single scenario easier to model as a linear function, and besides we had to make guesses in the previous IP as well. In keeping with our robust modeling ideas, we should be able to model a truncated cost of a scenario k that is its actual cost if z k = 1, but only W * +V k if k were an outlier (z k = 0). Then, the cost of scenario k can be given by an almost linear function The only non-linearity is the product R k z k , but we will explain later that this product can be closely approximated by linear variables F k , turning the single scenrio cost into the linear function Going back to equation We show the basic modeling idea behind the variables F k that are used to determine F k , and are the product of the cost (optimum single scenario plus regret) times z k . Now, we can impose constraints that bound the variables F k : These constraints ensure that if the cost exceeds the threshold, that is then z k and F k would be set to 0, which one can afford to do up to 1 − α of the time, so that Our next idea is to define scaled versions of the LP relaxation for the facility location formulation so that we could effectively model F k and F k , the versions of the cost and regret of the solution scaled by z k , for each scenario k. To achieve this, we create variables t jk and v i jk that we wish to satisfy v i jk := y i jk z k t jk := x j z k . The usual LP relaxation for facility location is Our constraints to scale this LP by z k are: If we take the above formulation and divide each of t, v, and F by z k , we get the usual LP relaxation for facility location, which means that these scaled models create no additional error compared with the LP relaxation other than that from relaxing the binary variables of the problem. Our third idea is to enforce t to be z k multiplied by the same x vector for all k while allowing v to be z k multiplied by a different y i jk vector for each scenario k. This makes sense since we want to make a placement of facilities that does not depend on scenario while which facility services a customer could depend on the scenario. In fact, we do not use a k subscript for the x variables, but do use such a subscript for the y variables. This modeling distinction leads us to form constraints analogous to the single constraint 22 We may now have a constraint for each j ∈ {1, .., n} stating: Also, we can form another constraint analogous to based on the idea that x j − t jk = x j (1 − z k ), so we can multiply 0 ≤ x ≤ 1 by 1 − z k as well as by z k . Hence, we get 0 These are similar to the constraints except that we take advantage of x not depending on the scenario k. Our robust model in its entirety, except that the objective function is left out, is as follows: As for the objective function, we have choices. One idea is for this IP to simply be a feasibility problem with no objective function. Another idea is to add up the cost of each scenario k truncated by W * +V k , and minimize. Thus an objective function could be One can see by examining this IP model when the z variables all have 0, 1 values that its LP relaxation is as tight as that of the facility location problem for each scenario when this integality condition is satisfied. This indicates that ours is a better LP relaxation than that of the previous section. This also indicates that branching on the z variables is particularly important when solving our robust IP with branch-and-bound. Facility Location and Sensor Placement The sensor placement model (SP) discussed in Section 2 is closely related to the standard p-median formulation used for facility location. There is some additional structure that can be exploited in (SP), but otherwise it uses the same set of constraints. However, the CVaR formulation and our minimax regret formulation address different aspect of modeling uncertainties in sensor placement applications. The CVaR model was developed to characterize the risk associated with different contamination events. In general, we wish to optimize expected performance, while constraining such risk to acceptable level. But when drawing a correspondence with facility location, the CVaR model considers only a single scenario; the different contamination events correspond to different demands on a facility. To better understand this correspondence, consider the placement of sensors to protect against contamination events at different seasons of the year. Water usage patterns will be quite different between summer and winter, and thus contamination events could propagate in very different manners and have different consequences. However, for each season there is a set of possible contamination events that need to be considered, for different locations and times of day for contamination. Thus, the seasons correspond to the scenarios that we have considered for facility location. This generalization of the sensor placement problem addresses one of the key limitations of existing sensor placement formulations. There are a large number of possible scenarios that account for different conditions in the network and different characteristics of contamination events. However, existing approaches lump all of the contamination events in these scenarios into one set of events. As we noted earlier, this could lead to sensor placement designs that are skewed towards particular contamination events in particular scenarios. 4 Enabling Technologies Two enabling technologies were developed as part of our research efforts to facilitate the solution of robust optimization applications. A new modeling tool, Pyomo, was developed to provide a more flexible environment for modeling and solving complex formulations like robust optimization problems. Also, we developed a new search strategy for the PICO integer programming solver that can manage constraint violations in a flexible manner and cache nearly feasible solutions. The Pyomo Modeling Tool Appendix B includes a technical report that describes the Python Optimization Modeling Objects (Pyomo) package. Pyomo is a Python package that can be used to define abstract problems, create concrete problem instances, and solve these instances with standard solvers. Pyomo provides a capability that is commonly associated with algebraic modeling languages like AMPL and GAMS. However, Pyomo can leverage Python's programming environment to support the development of complex models and optimization solvers in the same modeling environment. Algebraic Modeling Languages (AMLs) are high-level programming languages for describing and solving mathematical problems, particularly optimization-related problems An alternative strategy for modeling mathematical problems is to use a standard programming language in conjunction with a software library that uses object-oriented design to support similar mathematical concepts. Although these modeling libraries sacrifice the intuitive mathematical syntax of an AML, they allow the user to leverage the greater flexibility of standard programming languages. For example, modeling libraries like FLOPC++ [3] and OPL [5] enable the solution of large, complex problems within a user-defined application. Pyomo is a Python package that can be used to define abstract problems, create concrete problem instances, and solve these instances with standard solvers. Like other modeling libraries, Pyomo can generate problem instances and apply optimization solvers with a fully expressive programming language. Further, Python is a noncommercial language with a very large user community, which will ensure robust support for this language on a wide range of compute platforms. Python is a powerful dynamic programming language that has a very clear, readable syntax and intuitive object orientation. Python's clean syntax allows Pyomo to express 25 mathematical concepts in a reasonably intuitive manner. Further, Pyomo can be used within an interactive Python shell, thereby allowing a user to interactively interrogate Pyomobased models. Thus, Pyomo has many of the advantages of both AML interfaces and modeling libraries. Pyomo was developed as part of this project to facilitate the development of heuristic optimizers for complex applications like robust optimization problems. Specifically, our goal was to develop heuristics like FDT in Python using Pyomo's modeling objects. Unfortunately, this goal was not realized due to time constraints; instead, we implemented FDT with a rather awkward AMPL model. However, our prototype of Pyomo can be used to model and solve simple integer programming applications using Sandia's PICO IP solver. We expect Pyomo to mature as we use it for applications, and that it will play a key role in the development of new applications. In FY08, we plan to use Pyomo to analyze aircraft fleet planning applications under Lockheed Martin Shared Vision funding, including robust planning models. We currently plan to release Pyomo under an open-source license to encourage its use by external collaborators. Goal Programming in PICO In practice, satisfying a risk constraint exactly in a robust optimization formulation is less crucial than finding an effective compromise between the optimization objective and the performance risk. Thus, a risk constraint is better described as a goal that we want to meet, and risk-constrained robust optimization formulations can be effectively cast as goal programming models. Solution of goal programming models for robust optimization differs from standard discrete optimization in at least two important ways. First, the outcome of robust optimization is a set of solutions that represent trade-offs between the optimization objective and risk. Thus, the optimizer needs to maintain this set of solutions, and filter out solutions that are dominated by other solutions (i.e. they are not better in either the objective or risk value than at least one other solution). This is an example of a bi-criteria optimization problem, and standard sorting techniques can be used to maintain a set of undominated solutions. Second, the search process needs to be adapted to explicitly recognize goals. Standard discrete optimization techniques do not allow the search to focus on infeasible solutions; in fact, the efficiency of a discrete optimization solver is often related to how well it eliminates infeasible solutions. When considering goal constraints, we need to allow for infeasible solutions. This can be done by biasing search towards solutions that meet our goals. This is a natural extension of many heuristic solvers, which simply augment the objective with a penalty associated with how much the goal constraint is violated. Support for goal constraints is being added to Sandia's PICO integer programming solver. The heuristic solvers that PICO supports for integer programming formulations can recognize goals and treat them appropriately, and PICO maintains a pool of solutions 26 that represent different trade-offs between the optimization objective and these goal values. This capability will be included in an forthcoming release of PICO (planned for fall of 2007). 27 References [1] AIMMS home page. [2] AMPL home page. [3] FLOPC++ home page. [4] GAMS home page. [5] OPL home page. [6] J. BERRY, W. E. HART, C. E. PHILLIPS, J. G. UBER, AND J. Abstract The sensor placement problem in contamination warning system design for water distribution networks involves maximizing the protection level afforded by limited numbers of sensors, typically quantified as the expected impact of a contamination event; the issue of how to mitigate against high-impact events is either handled implicitly or ignored entirely. Consequently, expected-case sensor placements run the risk of failing to protect against high-impact, 9/11-style attacks. In contrast, robust sensor placements address this concern by focusing strictly on high-impact events and placing sensors to minimize the impact of these events. We introduce several robust variations of the sensor placement problem, distinguished by how they quantify the potential damage due to high-impact events. We explore the nature of robust versus expected-case sensor placements on three real-world, large-scale networks. We find that robust sensor placements can yield large reductions in the number and magnitude of high-impact events, for modest increases in expected impact. The resulting ability to trade-off between robust and expected-case impacts is a key, unexplored dimension in contamination warning system design.