bb.science
Class Bootstrap.UnitTest

java.lang.Object
  extended by bb.science.Bootstrap.UnitTest
Enclosing class:
Bootstrap

public static class Bootstrap.UnitTest
extends Object

See the Overview page of the project's javadocs for a general description of this unit test class.


Nested Class Summary
private static class Bootstrap.UnitTest.CauchyStandard
          Implements the standard Cauchy distribution (i.e. x0 = 0, gamma = 1).
private static class Bootstrap.UnitTest.CiResult
          Accumulates the results of running many individual Bootstrap.UnitTest.CiTasks.
private static class Bootstrap.UnitTest.CiTask
          This class is NOT multithread safe: it expects to only be touched by a single thread.
private static class Bootstrap.UnitTest.CoverageResult
          Accumulates the results of running many individual Bootstrap.UnitTest.CoverageTasks.
private static class Bootstrap.UnitTest.CoverageTask
          This class is NOT multithread safe: it expects to only be touched by a single thread.
private static interface Bootstrap.UnitTest.Distribution
          Specifies the api for classes that model a probability distribution.
private static class Bootstrap.UnitTest.DistributionAbstract
          Implements some common functionality used by concrete subclasses.
private static class Bootstrap.UnitTest.ExponentialStandard
          Implements the standard exponential distribution (i.e. lambda = 1).
private static class Bootstrap.UnitTest.GaussianStandard
          Implements the standard gaussian distribution (i.e. mean = 0, sd = 1).
 
Field Summary
private static double confidenceLevel
           
private static Bootstrap.UnitTest.Distribution[] distributions
           
private static int numberResamples
           
private static int numberTrials
           
private  PrintWriter pw
          A PrintWriter for method output.
private static int sampleLength
           
 
Constructor Summary
Bootstrap.UnitTest()
           
 
Method Summary
 void compareBootstrapCiWithTheory()
          Ouput is the relevant files in the doc/bootstrap/testResults directory of this project.
private  void compareBootstrapCiWithTheory(Bootstrap.UnitTest.Distribution distribution)
           
private  void determineCoverage(Bootstrap.UnitTest.Distribution distribution, int sampleLength, int numberResamples, double confidenceLevel, int numberTrials, String label)
           
 void determineNumberOfTrials()
          Explores how the CI coverage relative precision changes as a function of the number of trials.
private
<T> void
executeCode(Callable<T> task, String filename, String execTimePrefix)
           
private  double precisionOfCoverage(int n, double p, double q)
          Assume that you perform a CI generating process n times, and that each CI has the exact same coverage probabilty (i.e. confidenceLevel) specified by p.
static void setupClass()
           
static void teardownClass()
           
private static String toPercent(double d)
           
 void vary_numberResamples()
          Ouput is the relevant files in the doc/bootstrap/testResults directory of this project.
private  void vary_numberResamples(Bootstrap.UnitTest.Distribution distribution)
           
 void vary_sampleLength()
          Ouput is the relevant files in the doc/bootstrap/testResults directory of this project.
private  void vary_sampleLength(Bootstrap.UnitTest.Distribution distribution)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

distributions

private static final Bootstrap.UnitTest.Distribution[] distributions

sampleLength

private static final int sampleLength
See Also:
Constant Field Values

numberResamples

private static final int numberResamples
See Also:
Constant Field Values

confidenceLevel

private static final double confidenceLevel
See Also:
Constant Field Values

numberTrials

private static final int numberTrials
See Also:
Constant Field Values

pw

private PrintWriter pw
A PrintWriter for method output. Defaults to the console, but may be changed to write to a file.

Constructor Detail

Bootstrap.UnitTest

public Bootstrap.UnitTest()
Method Detail

setupClass

public static void setupClass()
                       throws InterruptedException
Throws:
InterruptedException

teardownClass

public static void teardownClass()
                          throws IOException
Throws:
IOException

determineNumberOfTrials

public void determineNumberOfTrials()
                             throws Exception
Explores how the CI coverage relative precision changes as a function of the number of trials. Execute this method to get an idea of how many trials you should execute to achieve a desired precision.

See the file .../doc/bootstrap/testResults/determineNumberOfTrials.txt for sample output. Summary: to get ~1 digit of precison, need ~100 trials, to get ~2 digits of precison, need ~10,000 trials, and to get each sucessive digit requires 100X more trials (e.g. 1,000,000 trials for 3 digits) because of the very slow 1/sqrt(n) convergence.

Throws:
Exception

precisionOfCoverage

private double precisionOfCoverage(int n,
                                   double p,
                                   double q)
Assume that you perform a CI generating process n times, and that each CI has the exact same coverage probabilty (i.e. confidenceLevel) specified by p. Let N be defined as the number of those n CIs which actually contained the true value Then the probability distribution for N is the binomial distribution B(n, p).

The mean value of N is n * p.

To get a measure for a reasonable range of values for N, define the "central interval" to be the sole interval [lower, upper] which satisfies cdf(upper) - cdf(lower) = q, and 1 - cdf(upper) = cdf(lower) = q/2 where cdf is the cumulative distribution function of B(n, p). In other words, N lies within [lower, upper] with a probabilty of q, and has equal probability of 1 - q/2 of falling below or above this "central interval". Define the "central range" to be the length of this interval, i.e. upper - lower.

Then this method returns the "central range" divided by the mean, i.e. (upper - lower) / (n * p). This is the relative precision that we can expect with probability q.


vary_sampleLength

public void vary_sampleLength()
                       throws Exception
Ouput is the relevant files in the doc/bootstrap/testResults directory of this project.

Throws:
Exception

vary_sampleLength

private void vary_sampleLength(Bootstrap.UnitTest.Distribution distribution)
                        throws Exception
Throws:
Exception

vary_numberResamples

public void vary_numberResamples()
                          throws Exception
Ouput is the relevant files in the doc/bootstrap/testResults directory of this project.

Throws:
Exception

vary_numberResamples

private void vary_numberResamples(Bootstrap.UnitTest.Distribution distribution)
                           throws Exception
Throws:
Exception

determineCoverage

private void determineCoverage(Bootstrap.UnitTest.Distribution distribution,
                               int sampleLength,
                               int numberResamples,
                               double confidenceLevel,
                               int numberTrials,
                               String label)
                        throws Exception
Throws:
Exception

compareBootstrapCiWithTheory

public void compareBootstrapCiWithTheory()
                                  throws Exception
Ouput is the relevant files in the doc/bootstrap/testResults directory of this project.

Throws:
Exception

compareBootstrapCiWithTheory

private void compareBootstrapCiWithTheory(Bootstrap.UnitTest.Distribution distribution)
                                   throws Exception
Throws:
Exception

executeCode

private <T> void executeCode(Callable<T> task,
                             String filename,
                             String execTimePrefix)
                  throws Exception
Throws:
Exception

toPercent

private static String toPercent(double d)