Bootstrap

The bootstrap method estimates the standard error of the median by repeatedly drawing ``bootstrap samples" from the original data, reevaluating the median for each bootstrap sample, and estimating the standard error of the original median by the observed variability in the bootstrap medians.

Bootstrap technique was invented by Bradley Efron (1979, 1981, 1982) and further developed by Efron and Tibshirani (1993). "Bootstrap" means that one available sample gives rise to many others by resampling (a concept reminiscent of pulling yourself up by your own bootstrap). While the original objective of cross-validation is to verify replicability of results and that of jackknife is to detect outliers, Efron developed bootstrap with inferential purposes.

In bootstrap, the original sample could be duplicated as many times as the computing resources allow, and then this expanded sample is treated as a virtual population. Then samples are drawn from this population to verify the estimators. Obviously the "source" for resampling in bootstrap could be much larger than that in the other two. In addition, unlike cross validation and jackknife, the bootstrap employs sampling with replacement. Indeed, sampling with replacement in a bootstrap is more accurate than sampling without replacement in terms of simulating chance. Further, in cross-validation and Jackknife, the n in the sub-sample is smaller than that in the original sample, but in bootstrap every resample has the same number of observations as the original sample. Thus, the bootstrap method has the advantage of modeling the impacts of the actual sample size (Fan & Wang, 1996).