
@chapsummary
Probability is the only calculus of uncertainty.
@endchapsummary

vpdfl contains several classes for representing and using
multivariate probability density functions (@acronym{PDF}s).

Each @acronym{PDF} comes as a builder-model-sampler triplet of objects.

e.g. The abstract @acronym{PDF} base classes
@itemize
@item vpdfl_builder_base
@item vpdfl_pdf_base
@item vpdfl_pdf_sampler
@end itemize

The multivariate Gaussian @acronym{PDF} classes
@itemize
@item vpdfl_axis_gaussian_builder
@item vpdfl_axis_gaussian
@item vpdfl_axis_gaussian_sampler
@end itemize

The builders are used to compute the model data (eg the means, variances
etc) for the models. The models are used to calculate densities,
cumulative probabilities, gradients, etc. Samplers are used to generate samples
from a given model. The samples are produced with a @acronym{PDF} given by its model.


@subsection Strategy Pattern
This code is an example of the strategy pattern
(@cite{Gamma, et al. Design Patterns, Addison Wesley, 1995}.) It is possible to
write code that builds and uses a @acronym{PDF}, where your code does not itself
know what sort of @acronym{PDF} is being used. Both builders and @acronym{PDF}s
can be saved and loaded by base class pointer.

@section vpdfl_pdf_base
This is the base class for Multi-Variate @acronym{PDF} classes.
It also records the mean and variance in each dimension. Derived classes may
record more information (eg covariance matrix etc).
For cases where the distributions of parameters are multi-modal,
the number and centres of each peak can be recorded. This is particularly
useful for non-linear and mixture model representations of the parameter
distributions.


The @acronym{PDF}s are assumed to be continuous, and defined over an @math{|R^n} vector space.
The type of these vectors is @code{vnl_vector<double>}.

@subheading Main functions
@table @code

@item vpdfl_pdf_sampler* sampler()
Create appropriate sampler class on heap and return pointer

@item int n_dims() const
Dimensionality of vector space represented

@item const vnl_vector<double>& mean() const
Mean vector for distribution

@item const vnl_vector<double>& variance()
Vector giving variance along each dimension

@item int n_peaks() const
Number of peaks of distribution (1 unless multimodal)

@item const vnl_vector<double>& peak(int i) const
Position of the i'th peak (useful for multi-modal @acronym{PDF}s)

@item double log_p(x)
Log of probability density at x

@item double operator(x)
Probability density at x.

@item void gradient(g,  x, p)
Gradient of @acronym{PDF} at x

@item double log_prob_thresh(pass_proportion)
Compute threshold for @acronym{PDF} to pass a given proportion

@item void nearest_plausible(x; log_p_min)
Compute nearest point to x which has a log density above a threshold
@end table

@section vpdfl_builder_base
A base class for objects which build Multivariate @acronym{PDF} models from sets
of vectors supplied in a @code{mbl_data_wrapper<vnl_vector<double> >} derived iterator.

@subheading Main functions
@table @code
@item vpdfl_pdf_base* new_model()
Create a new model of appropriate type on heap and return pointer

@item void build(model, data)
Build the model from the vectors supplied in data

@item void weighted_build(model, data, weights)
Build the model from the supplied vectors, allowing for given weights
@end table

@section vpdfl_base_sampler
This generates samples drawn from the distribution described by a @code{vpdfl_pdf_base}.
Note that unlike other types in the library, the samplers don't contain useful
state, and cannot be saved.

@subheading  Main functions
@table @code
@item const vpdfl_pdf_base& model() const
The @code{vpdfl_pdf_base} for which this is a sampler

@item void set_model(pdf)
Set model for which this is a sampler

@item void sample(x)
Draw random sample from distribution
@end table


@section Derived Classes

@table @code
@item vpdfl_axis_gaussian
Gaussian @acronym{PDF} aligned with the cartesian axes.

@item vpdfl_gaussian
General gaussian at arbitrary orientation. Axes are
defined by a set of eigenvectors, @code{eigenvecs()}, and their associated eigenvalues,
@code{eigenvals()} of the covariance matrix of the original data.

@item vpdfl_pc_gaussian
A gaussian at arbitrary orientation. The covariance in the direction of the smaller
eigenvalues are assumed for efficiency to be circular.

@item vpdfl_mixture_pdf
Mixture model containing two or more components (each a @code{vpdfl_pdf_base} derived class)
@end table

@section Examples

Suppose we wish to compute a multivariate gaussian from a set of
vectors, then estimate the probability that each vector was generated by
the distribution.

@example
vcl_vector<vnl_vector<double> > data(n);
// Load in the vectors
....

// Create an iterator object to pass the data in
mbl_data_wrapper<vnl_vector<double> > v_data(data);

// Define what type of builder to use.  In this case we want a Gaussian
vpdfl_gaussian_builder builder;

// Generate model to build
vpdfl_pdf_base *model = builder.new_model();
// I could have created it directly using
// vpdfl_gaussian model;

// Build the model from the data
builder.build(*model, v_data);
vsl_print_summary(vcl_cout, model);

// Now test each vector
for (int i=0;i<data.size();++i)
    vcl_cout<<"log(P(v(i))) = "<<model->log_p(data[i])<< vcl_endl;

// Tidy up
delete model;
@end example

To generate a collection of 4d Normally distributed samples.
@example
// Create PDF
vpdfl_axis_gaussian model;
vnl_vector mean(4, 0.0), var(4, 1.0);
model.set(mean, var);

// Create a sampler.
vpdfl_sampler_base *sampler = model.new_sampler();

// Somewhere to store the data
vcl_vector<vnl_vector<double> > data(n);

// Generate 100 samples.
for (unsigned i =0; i < n; ++i)
    sampler->sample(data[i]);

// Tidy-up
delete sampler;
@end example


It is straightforward to merge the above examples, demonstrating how to
use a @acronym{PDF} and sampler
to generate some data, and then building a new @acronym{PDF} from that data.
This is exactly what the test programs do.


@section Further Development

@subsection Discrete Probability Distributions.
There is nothing fundamental in the design that means that a @acronym{PDF}
has to be defined over a real vector space. It should be possible to template
the base classes, so that the basic design could be used for discrete distributions
or for continuous univariate distributions.
