### Table 1{1. Speci cation for a simple stochastic process

"... In PAGE 13: ... The experimentinvolves a stochastic process described in (Howard, 1960), consisting of three states, f1;; 2;; 3g, three actions, fa;; b;; cg, and a dis- countrate, ,of0:4. The state transition probabilities and rewards are shown in Table1 {1, where R(i;; u;; j) indicates the reward on performing action, u, starting in state, i, and ending in state, j.For this exper- iment, we use the stochastic value iteration algorithm, SVI, shown in Figure 1{1, and weinvoke SVI with =0:1, and varying between 0 and 1.... ..."

### Table 1: Illustration of Stochastic Process

2005

"... In PAGE 7: ... That is, we model a stochastic process with one absorptive state that might be termed quot;falling off the chart. quot; Table1 details the stochastic process for the first three periods. More formally, let Ck,i indicate the number of albums that appear on the kth week apos;s Billboard Chart and have appeared for i weeks (i = 1,.... ..."

### Table 8. The Stochastic-Offline-Rect-Dp algorithm for o ine, region-based dynamic pro- gramming in stochastic problems.

1995

"... In PAGE 21: ....2.7. Stochastic Region-Based O ine Dynamic Programming (Stochastic-Offline-Rect-Dp) Table8 describes an algorithm for prioritized sweeping for region-based dynamic programming. It is essentially the same as the point-based algorithm except that the process of performing stochastic region-based backups is much more complex in line 9.... ..."

Cited by 39

### Table 2: Non-convex quarticly constrained optimization problem for hierarchy and policy discovery in bounded stochastic recursive controllers.

"... In PAGE 5: ... 3.3 Algorithms Since the problem in Table2 has non-convex (quartic) constraints in Eq. 5 and 6, it is difficult to solve.... In PAGE 5: ... 5 and 6, it is difficult to solve. We consider three approaches inspired from the techniques for non-hierarchical controllers: Non-convex optimization: Use a general non-linear solver, such as SNOPT, to directly tackle the optimization problem in Table2 . This is the most convenient approach, however a globally optimal solution may not be found due to the non-convex nature of the problem.... In PAGE 7: ... 4 Experiments We report on some preliminary experiments with three toy problems (paint, shuttle and maze) from the POMDP repository3. We used the SNOPT package to directly solve the non-convexoptimization problem in Table2 and bounded hierarchical policy iteration (BHPI) to solve it iteratively. Table 3 reports the running time and the value of the hierarchical policies found.... ..."

### Table 2: Non-convex quarticly constrained optimization problem for hierarchy and policy discovery in bounded stochastic recursive controllers.

in Abstract

"... In PAGE 5: ... 3.3 Algorithms Since the problem in Table2 has non-convex (quartic) constraints in Eq. 5 and 6, it is difficult to solve.... In PAGE 5: ... 5 and 6, it is difficult to solve. We consider three approaches inspired from the techniques for non-hierarchical controllers: Non-convex optimization: Use a general non-linear solver, such as SNOPT, to directly tackle the optimization problem in Table2 . This is the most convenient approach, however a globally optimal solution may not be found due to the non-convex nature of the problem.... In PAGE 7: ... 4 Experiments We report on some preliminary experiments with three toy problems (paint, shuttle and maze) from the POMDP repository3. We used the SNOPT package to directly solve the non-convex optimization problem in Table2 and bounded hierarchical policy iteration (BHPI) to solve it iteratively. Table 3 reports the running time and the value of the hierarchical policies found.... ..."

### Table 2: Non-convex quarticly constrained optimization problem for hierarchy and policy discovery in bounded stochastic recursive controllers.

in Abstract

"... In PAGE 5: ... 3.3 Algorithms Since the problem in Table2 has non-convex (quartic) constraints in Eq. 5 and 6, it is difficult to solve.... In PAGE 5: ... 5 and 6, it is difficult to solve. We consider three approaches inspired from the techniques for non-hierarchical controllers: Non-convex optimization: Use a general non-linear solver, such as SNOPT, to directly tackle the optimization problem in Table2 . This is the most convenient approach, however a globally optimal solution may not be found due to the non-convex nature of the problem.... In PAGE 7: ... 4 Experiments We report on some preliminary experiments with three toy problems (paint, shuttle and maze) from the POMDP repository3. We used the SNOPT package to directly solve the non-convex optimization problem in Table2 and bounded hierarchical policy iteration (BHPI) to solve it iteratively. Table 3 reports the running time and the value of the hierarchical policies found.... ..."

### Table 2.- Schematic classification of stochastic optimization methods. Seminal references and selected examples of their application in process engineering are also given.

"... In PAGE 12: ...riori assumptions or pre-processing work. . There are at least four different classes of approaches which were apparently generated independently. A schematic classification, together with key seminal references and selected examples of their application in process engineering are given in Table2 . Some more details about each type follows: g183g32 Adaptive stochastic methods were developed in the domains of electrical and control engineering and applied mathematics (e.... In PAGE 13: ...stable configuration as slow cooling of a metal takes place). Apart from those methods presented in Table2 , during recent years a number of other (so called) meta-heuristics have been presented, mostly based on other biological or physical phenomena, and with combinatorial optimization as their original domain of application. Examples of these more recent methods are Taboo Search (TS), Ant Colony Optimization (ACO) (Dorigo, Maniezzo amp; Colorni, 1996; Bonabeau, Dorigo amp; Theraulaz, 2000; Jayaraman, Kulkarni amp; Shelokar, 2000) and particle swarm methods (Bonabeau, Dorigo amp; Theraulaz, 1999).... ..."

### Table 3 uses this approach to develop an interaction matrix that is symmetrical about the main diagonal. Consistent with the concept of correlation, the matrix assumes that the interaction of CTs is mutually equivalent. For example, studies in Stochastic processes will produce a strong positive impact on

"... In PAGE 4: ...rows. Table3 . Interaction Matrix for Curricular Topics Note: High interaction = 9/13=0.... ..."

### Table 2: Estimates of Variance Minimizing Growth Rate and Risk Return Trade-O , Stochastic Frontier and Random E ects Models from Table 1

"... In PAGE 30: ... For this reason, too, the random e ects estimates of the model from Table 1 are closer to the DEA estimates than are the stochastic frontier estimates. In addition, the variance minimizing rate of growth using DEA is just over 3%, while the corresponding estimates from Table2 are higher, at more than 4%. Again, the random e ects model estimate from Table 2 is closer to the DEA estimates than the stochastic frontier estimate.... In PAGE 30: ... In addition, the variance minimizing rate of growth using DEA is just over 3%, while the corresponding estimates from Table 2 are higher, at more than 4%. Again, the random e ects model estimate from Table2 is closer to the DEA estimates than the stochastic frontier estimate. Ultimately, the choice of method depends on whether, in layperson apos;s terms, the growth-volatility observations are believed to come from the same distribution or not, where the choice of DEA assumes a greater degree of sameness than does random e ects, and random e ects assumes a greater degree of sameness than does stochastic... In PAGE 32: ...wo speci cations are very close, at 2.61% and 2.63%. An interesting di erence between these estimates and the estimates of the variance minimizing rate of growth for total GSP in Table2 is the decrease of between 1% and 2% in the growth rate when the per capita data are used. The main reason for this downward \shift quot; of the per capita GSP growth-volatility frontier compared to the total GSP frontier is that we are controlling for the e ect of population growth on the rate of growth of GSP.... ..."