We present an introduction to a new class of derivative free methods for unconstrained optimization. We start by discussing the motivation for such methods and why they are in high demand by practitioners. We then review the past developments in this field, before introducing the features that characterize the newer algorithms. In the context of a trust region framework, we focus on techniques that ensure a suitable "geometric quality" of the considered models. We then outline the class of algorithms based on these techniques, as well as their respective merits. We finally conclude the paper with a discussion of open questions and perspectives. 1 Motivation In this paper, we consider the problem of minimizing a nonlinear smooth objective function of several variables when the derivatives of the objective function are unavailable and when no constraints are specified on the problem's variables. More formally, we consider the problem min x2R n f(x); where we assume that f i...

Introduction The ever expanding gamut of medical imaging techniques provides the clinician an increasingly multifaceted view of brain function and anatomy. The information provided by the various imaging modalities is often complementary (i.e. provides separate but useful information) and synergistic (i.e. the combination of information provides useful extra information). For example, X-ray computed tomography (CT) and magnetic resonance (MR) imaging exquisitely demonstrate brain anatomy but provide little functional information. Positron emission tomography (PET) and single photon emission computed tomography (SPECT) scans display aspects of brain function and allow metabolic measurements but poorly delineate anatomy. Furthermore, CT and MR images describe complementary morphologic features. For example, bone and calcifications are best seen on CT images, while soft-tissue structures are better differentiated by MR imaging. Clinical diagnosis and therapy planning and evaluatio

Abstract. During the last few years, conjugate-gradient methods have been found to be the best available tool for large-scale minimization of nonlinear functions occurring in geophysical applications. While vectorization techniques have been applied to linear conjugate-gradient methods designed to solve symmetric linear systems of algebraic equations, arising mainly from discretization of elliptic partial differential equations, due to their suitability for vector or parallel processing, no such effort was undertaken for the nonlinear conjugate-gradient method for large-scale unconstrained minimization. Computational results are presented here using a robust memoryless quasi-Newton-like conjugate-gradient algorithm by Shanno and Phua applied to a set of large-scale meteorological problems. These results point to the vectorization of the conjugate-gradient code inducing a significant speed-up in the function and gradient evaluation for the nonlinear conjugate-gradient method, resulting in a sizable reduction

Finding the global optimum on a large, multimodal, complex, and discontinuous (or nondifferentiable) landscape is usually very hard, even using the evolutionary approach. However, some of these complex landscapes can be approximated and smoothened without changing the nature of the problem, i.e., without modifying the global optimum and its location. The approximated and smoothened landscape is often much easier to search than the original one. In this paper, we propose a new algorithm using landscape approximation and hybrid evolutionary and local search. We also list several algorithm design principles. Following the basic algorithm, an example algorithm is given from our previous work of the combination of landscape approximation and local search (LALS). Furthermore, we develop a novel evolutionary algorithm with n-dimensional approximation (EANA), which shares the same rules as the basic algorithm, but remedies some of the drawbacks found in the LALS. Comparisons with evo...

www.niss.org

A novel automatic registration algorithm for the alignment of medical imaging data was developed. The algorithm measures alignment by comparison of dense feature sets (tissue labels) and optimum alignment is found by minimizing the mismatch of tissue segmentations. A parallel implementation that distributes resampling and comparison operations across a cluster of symmetric multiprocessors achieves execution times in a clinically compatible range (5--10 minutes). Each node executes a parallelized resample and compare operation implemented with POSIX threads, and work is dynamically load balanced across the cluster with communication implemented with MPI. The quality of the registration algorithm and the performance characteristics of the parallel implementation were investigated for typical registration problems. The algorithm has been used to successfully achieve intrapatient and interpatient registration of tissue segmentations without any manual intervention for over thre...

Abstract. This paper describes an effort to model a student’s changing knowledge state during skill acquisition. Dynamic Bayes Nets (DBNs) provide a powerful way to represent and reason about uncertainty in time series data, and are therefore well-suited to model student knowledge. Many general-purpose Bayes net packages have been implemented and distributed; however, constructing DBNs often involves complicated coding effort. To address this problem, we introduce a tool called BNT-SM. BNT-SM inputs a data set and a compact XML specification of a Bayes net model hypothesized by a researcher to describe causal relationships among student knowledge and observed behavior. BNT-SM generates and executes the code to train and test the model using the Bayes Net Toolbox [1]. Compared to the BNT code it outputs, BNT-SM reduces the number of lines of code required to use a DBN by a factor of 5. In addition to supporting more flexible models, we illustrate how to use BNT-SM to simulate Knowledge Tracing (KT) [2], an established technique for student modeling. The trained DBN does a better job of modeling and predicting student performance than the original KT code (Area Under Curve = 0.610> 0.568), due to differences in how it estimates parameters. 1

This paper presents a covariance function analysis of mature weight records of beef cows, fitting a random regression model. It attempts to demonstrate the kind of calculations, results and problems which might be encountered with this `new' type of analysis, but throughout the paper, familiarity with covariance function models is assumed. Material and methods

... these new methods, we develop a prototype, called Novel (Nonlinear Optimization Via External Lead), that solves nonlinear constrained and unconstrained problems in a unified framework. We show experimental results in applying Novel to solve nonlinear optimization problems, including (a) the learning of feedforward neural networks, (b) the design of quadrature-mirror-filter digital filter banks, (c) the satisfiability problem, (d) the maximum satisfiability problem, and (e) the design of multiplierless quadrature-mirror-filter digital filter banks. Our method achieves better solutions than existing methods, or achieves solutions of the same quality but at a lower cost.

Abstract—We present a new approach to camera calibration as a part of a complete and practical system to recover digital copies of sculpture from uncalibrated image sequences taken under turntable motion. In this paper, we introduce the concept of the silhouette coherence of a set of silhouettes generated by a 3D object. We show how the maximization of the silhouette coherence can be exploited to recover the camera poses and focal length. Silhouette coherence can be considered as a generalization of the well-known epipolar tangency constraint for calculating motion from silhouettes or outlines alone. Further, silhouette coherence exploits all the geometric information encoded in the silhouette (not just at epipolar tangency points) and can be used in many practical situations where point correspondences or outer epipolar tangents are unavailable. We present an algorithm for exploiting silhouette coherence to efficiently and reliably estimate camera motion. We use this algorithm to reconstruct very high quality 3D models from uncalibrated circular motion sequences, even when epipolar tangency points are not available or the silhouettes are truncated. The algorithm has been integrated into a practical system and has been tested on more than 50 uncalibrated sequences to produce high quality photo-realistic models. Three illustrative examples are included in this paper. The algorithm is also evaluated quantitatively by comparing it to a state-of-the-art system that exploits only epipolar tangents. Index Terms—Silhouette coherence, epipolar tangency, image-based visual hull, focal length estimation, circular motion, 3D modeling. 1