Results 1  10
of
19
Assessor error in stratified evaluation
 In Proceedings of the 19th ACM International Conference on Information and Knowledge Management (CIKM
, 2010
"... Several important information retrieval tasks, including those in medicine, law, and patent review, have an authoritative standard of relevance, and are concerned about retrieval completeness. During theevaluationofretrievaleffectivenessinthesedomains,assessors make errors in applying the standard o ..."
Abstract

Cited by 5 (4 self)
 Add to MetaCart
(Show Context)
Several important information retrieval tasks, including those in medicine, law, and patent review, have an authoritative standard of relevance, and are concerned about retrieval completeness. During theevaluationofretrievaleffectivenessinthesedomains,assessors make errors in applying the standard of relevance, and the impact oftheseerrors,particularlyonestimatesofrecall,isofcrucialconcern. UsingdatafromtheinteractivetaskoftheTRECLegalTrack, thispaperinvestigateshowreliablytheyieldofrelevantdocuments can be estimated from sampled assessments in the presence of assessor error, particularly where sampling is stratified based upon the results of participating retrieval systems. We show that assessorerrorisin general a greater source of inaccuracy thansampling error. A process of appeal and adjudication, such as used in the interactive task, is found to be effective at locating many assessment errors; but the process is expensive if complete, and biased if incomplete. An unbiased doublesampling method for resolving assessment error is proposed, and shown on representative data to bemore efficient and accurate than appealbased adjudication.
Bayesian Analysis of Binary Data Subject to Misclassification
 In Bayesian Analysis in Statistics and Econometrics: Essays in Honor of Arnold Zellner
, 1996
"... This paper considers estimation of success probabilities of categorical binary data subject to misclassification errors from the Bayesian point of view. It has been shown by Bross (1954) that sample proportions are in general biased estimates. This bias is a function of the amount of misclassificati ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
This paper considers estimation of success probabilities of categorical binary data subject to misclassification errors from the Bayesian point of view. It has been shown by Bross (1954) that sample proportions are in general biased estimates. This bias is a function of the amount of misclassification and can be substantial. Tenenbein (1970) proposed to eliminate the bias by subjecting a portion of the sample to both true and fallible classifiers, resulting in a 2 x 2 table, from which the misclassification rates can be estimated. The rationale is that fallible classifiers are inexpensive relative to infallible ones. Hence if only a part of the sample is measured by the infallible classifier one can obtain a more efficient estimate, for a given sampling budget, than by measuring the whole sample using the infallible classifier. In many contexts an infallible classifier is unavailable or prohibitively expensive. Bayesian methods then provide a useful approach for dealing with the conseq...
Assessing accuracy of a continuous screening test in the presence of verification bias
 Journal of the Royal Statistical Society: Series C (Applied Statistics
"... Summary. In studies to assess the accuracy of a screening test, often definitive disease assessment is too invasive or expensive to be ascertained on all the study subjects. Although it may be more ethical or cost effective to ascertain the true disease status with a higher rate in study subjects w ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Summary. In studies to assess the accuracy of a screening test, often definitive disease assessment is too invasive or expensive to be ascertained on all the study subjects. Although it may be more ethical or cost effective to ascertain the true disease status with a higher rate in study subjects where the screening test or additional information is suggestive of disease, estimates of accuracy can be biased in a study with such a design.This bias is known as verification bias. Verification bias correction methods that accommodate screening tests with binary or ordinal responses have been developed; however, no verification bias correction methods exist for tests with continuous results. We propose and compare imputation and reweighting biascorrected estimators of true and false positive rates, receiver operating characteristic curves and area under the receiver operating characteristic curve for continuous tests. Distribution theory and simulation studies are used to compare the proposed estimators with respect to bias, relative efficiency and robustness to model misspecification. The bias correction estimators proposed are applied to data from a study of screening tests for neonatal hearing loss.
Misclassification in Logistic Regression with Discrete Covariates
 Biometrical Journal
, 2003
"... We study the effect of misclassification of a binary covariate on the parameters of a logistic regression model. In particular we consider 2 2 2 tables. We assume that a binary covariate is subject to misclassification that may depend on the observed outcome. This type of misclassification is know ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
We study the effect of misclassification of a binary covariate on the parameters of a logistic regression model. In particular we consider 2 2 2 tables. We assume that a binary covariate is subject to misclassification that may depend on the observed outcome. This type of misclassification is known as (outcome dependent) differential misclassification. We examine the resulting asymptotic bias on the parameters of the model and derive formulas for the biases and their approximations as a function of the odds and misclassification probabilities. Conditions for unbiased estimation are also discussed. The implications are illustrated numerically using a case control study. For completeness we briefly examine the effect of covariate dependent misclassification of exposures and of outcomes.
Use of Screening Tests to Assess Cancer Risk and to Estimate the Risk of Adult T
"... We developed methods to assess the cancer risks by screening tests. These methods estimate the size of the high risk group adjusted for the characteristics of screening tests and estimate the incidence rates of cancer among the high risk group adjusted for the characteristics of the tests. A method ..."
Abstract
 Add to MetaCart
(Show Context)
We developed methods to assess the cancer risks by screening tests. These methods estimate the size of the high risk group adjusted for the characteristics of screening tests and estimate the incidence rates of cancer among the high risk group adjusted for the characteristics of the tests. A method was also developed for selecting the cutoff point of a screening test. Finally, the methods were applied to estimate the risk of the adult Tcell leukemia/lymphoma.
The Canadian Journal of Statistics 1
"... La revue canadiennede statistique Semiparametric efficient estimation for the auxiliary outcome problem with the conditional mean model Jinbo CHEN and Norman E. BRESLOW Key words and phrases: Auxiliary outcome; conditional mean model; Horvitz–Thompson estimator; missing at random; semiparametric eff ..."
Abstract
 Add to MetaCart
(Show Context)
La revue canadiennede statistique Semiparametric efficient estimation for the auxiliary outcome problem with the conditional mean model Jinbo CHEN and Norman E. BRESLOW Key words and phrases: Auxiliary outcome; conditional mean model; Horvitz–Thompson estimator; missing at random; semiparametric efficient estimation. MSC 2000: Primary: 62D05, 62J12; secondary: 62H12. Abstract: The authors consider semiparametric efficient estimation of parameters in the conditional mean model for a simple incomplete data structure in which the outcome of interest is observed only for a random subset of subjects but covariates and surrogate (auxiliary) outcomes are observed for all. They use optimal estimating function theory to derive the semiparametric efficient score in closed form. They show that when covariates and auxiliary outcomes are discrete, a Horvitz–Thompson type estimator with empirically estimated weights is semiparametric efficient. The authors give simulation studies validating the finitesample behaviour of the semiparametric efficient estimator and its asymptotic variance; they demonstrate the efficiency of the estimator in realistic settings. Estimation semiparamétriquement efficace pour le problème du résultat auxiliaire dans le modèle à moyenne conditionnelle Résumé: Les auteurs s’intéressent à l’estimation semiparamétriquement efficace de paramètres dans le modèle à moyenne conditionnelle pour une structure de données incomplète simple dans laquelle l’événement d’intérêt n’est observé que pour un sousensemble aléatoire de sujets alors que les covariables et les variables de substitution (auxiliaires) sont observées pour tous. Ils font appel àlathéorie des fonctions d’estimation optimales pour déterminer le score semiparamétriquement efficace de façon explicite. Ils montrent que lorsque les covariables et les variables auxiliaires sont discrètes, un estimateur de type Horvitz–Thompson à poids estimés empiriquement est semiparamétriquement efficace. Les auteurs présentent des études de simulation validant le comportement à taille finie de l’estimateur semiparamétriquement efficace et de sa variance asymptotique; ils démontrent en outre l’efficacité de cet estimateur dans des contextes réalistes. 1.
Size for CaseControl Genetic Association Studies in the Presence of Phenotype and/or Genotype Misclassification Errors ∗
"... ..."
(Show Context)
A Double Sampling Scheme Model for . . .
, 1974
"... A general double sampling scheme model which employs a combination of an errorfree measurement process and a faulty measurement process is developed. The model allows estimation of measurement error variance and elimination of measurement process bias. The model is applied to two specific survey s ..."
Abstract
 Add to MetaCart
A general double sampling scheme model which employs a combination of an errorfree measurement process and a faulty measurement process is developed. The model allows estimation of measurement error variance and elimination of measurement process bias. The model is applied to two specific survey situations, a selfenumeration survey and an interviewer conducted survey. Using a cost function which reflects the relative cost of the errorfree measurement process and the faulty measurement process, optimum values for the sample sizes are derived and the optimum number of interviewers is indicated. For various values of the parameters the DSS model is compared to using only the faulty measurement process or only the errorfree measurement process and the preferred sampling scheme is indicated.
Artículo panorámico / Survey Misclassified multinomial data: a Bayesian approach
"... Abstract. In this paper, the problem of inference with misclassified multinomial data is addressed. Over the last years there has been a significant upsurge of interest in the development of Bayesian methods to make inferences with misclassified data. The wide range of applications for several sampl ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. In this paper, the problem of inference with misclassified multinomial data is addressed. Over the last years there has been a significant upsurge of interest in the development of Bayesian methods to make inferences with misclassified data. The wide range of applications for several sampling schemes and the importance of including initial information make Bayesian analysis an essential tool to be used in this context. A review of the existing literature followed by a methodological discussion is presented in this paper.