| Title: | Generate Samples from Multivariate Truncated Normal Distributions |
|---|---|
| Description: | Efficient sampling from high-dimensional truncated Gaussian distributions, or multivariate truncated normal (MTN). Techniques include zigzag Hamiltonian Monte Carlo as in Akihiko Nishimura, Zhenyu Zhang and Marc A. Suchard (2024) <doi:10.1080/01621459.2024.2395587>, and harmonic Monte Carlo in Ari Pakman and Liam Paninski (2014) <doi:10.1080/10618600.2013.788448>. |
| Authors: | Zhenyu Zhang [aut, cre], Andrew Chin [aut], Akihiko Nishimura [aut], Marc A. Suchard [aut], John W. Ratcliff et al. [cph, ctb] (authors and copyright holders of see2neon.h under an MIT license) |
| Maintainer: | Zhenyu Zhang <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.3.4 |
| Built: | 2026-05-12 09:31:21 UTC |
| Source: | https://github.com/cran/hdtg |
Compute Cholesky decomposition of a matrix.
cholesky(A)cholesky(A)
A |
matrix to decompose |
upper triangular matrix R such that A = U'U.
# Larger example set.seed(123) B <- matrix(rnorm(16), 4, 4) B <- t(B) %*% B # Make symmetric positive definite U <- cholesky(B) U # Verify decomposition all.equal(B, t(U) %*% U)# Larger example set.seed(123) B <- matrix(rnorm(16), 4, 4) B <- t(B) %*% B # Make symmetric positive definite U <- cholesky(B) U # Verify decomposition all.equal(B, t(U) %*% U)
Create the C++ object to set up SIMD vectorization for speeding up calculations for Zigzag-HMC ("Zigzag-HMC engine").
createEngine( dimension, lowerBounds, upperBounds, seed, mean, precision, flags = 128L )createEngine( dimension, lowerBounds, upperBounds, seed, mean, precision, flags = 128L )
dimension |
the dimension of MTN. |
lowerBounds |
a vector specifying the lower bounds. |
upperBounds |
a vector specifying the upper bounds. |
seed |
random seed. |
mean |
the mean vector. |
precision |
the precision matrix. |
flags |
which SIMD instruction set to use. 128 = SSE, 256 = AVX. |
a list whose only element is the Zigzag-HMC engine object.
setMean(), setPrecision(), zigzagHMC(), markovianZigzag()
# Create a 2D engine with simple bounds dimension <- 2 lowerBounds <- c(-1, -1) upperBounds <- c(1, 1) mean <- c(0, 0) precision <- matrix(c(1, 0.5, 0.5, 1), nrow = 2) engine <- createEngine(dimension, lowerBounds, upperBounds, seed = 123, mean, precision, flags = 128) # Check the engine structure str(engine)# Create a 2D engine with simple bounds dimension <- 2 lowerBounds <- c(-1, -1) upperBounds <- c(1, 1) mean <- c(0, 0) precision <- matrix(c(1, 0.5, 0.5, 1), nrow = 2) engine <- createEngine(dimension, lowerBounds, upperBounds, seed = 123, mean, precision, flags = 128) # Check the engine structure str(engine)
Create the C++ object to set up SIMD vectorization for speeding up calculations for Zigzag-NUTS ("Zigzag-NUTS engine").
createNutsEngine( dimension, lowerBounds, upperBounds, seed, stepSize, mean, precision, flags = 128L )createNutsEngine( dimension, lowerBounds, upperBounds, seed, stepSize, mean, precision, flags = 128L )
dimension |
the dimension of MTN. |
lowerBounds |
a vector specifying the lower bounds. |
upperBounds |
a vector specifying the upper bounds. |
seed |
random seed. |
stepSize |
the base step size for Zigzag-NUTS. |
mean |
the mean vector. |
precision |
the precision matrix. |
flags |
which SIMD instruction set to use. 128 = SSE, 256 = AVX. |
a list whose only element is the Zigzag-NUTS engine object.
setMean(), setPrecision(), zigzagHMC(), createEngine()
# Create a Zigzag-NUTS engine for a 2D problem dimension <- 2 lowerBounds <- c(-2, -2) upperBounds <- c(2, 2) stepSize <- 0.1 mean <- c(0.5, -0.5) precision <- matrix(c(2, 0.3, 0.3, 2), nrow = 2) nuts_engine <- createNutsEngine(dimension, lowerBounds, upperBounds, seed = 456, stepSize, mean, precision) str(nuts_engine)# Create a Zigzag-NUTS engine for a 2D problem dimension <- 2 lowerBounds <- c(-2, -2) upperBounds <- c(2, 2) stepSize <- 0.1 mean <- c(0.5, -0.5) precision <- matrix(c(2, 0.3, 0.3, 2), nrow = 2) nuts_engine <- createNutsEngine(dimension, lowerBounds, upperBounds, seed = 456, stepSize, mean, precision) str(nuts_engine)
Generate a d-dimensional momentum where the density of each element is proportional to exp(-|pi|).
drawLaplaceMomentum(d)drawLaplaceMomentum(d)
d |
dimension of the momentum. |
a d-dimensional Laplace-distributed momentum.
# Draw a 3-dimensional Laplace momentum with reproducible results set.seed(3) momentum <- drawLaplaceMomentum(3) momentum# Draw a 3-dimensional Laplace momentum with reproducible results set.seed(3) momentum <- drawLaplaceMomentum(3) momentum
One-step Harmonic HMC Sampler (Whitened Coordinates)
getHarmonicSample( whitenedPosition, whitenedConstraints, integrationTime, diagnosticMode = FALSE, seed = NULL )getHarmonicSample( whitenedPosition, whitenedConstraints, integrationTime, diagnosticMode = FALSE, seed = NULL )
whitenedPosition |
Position in whitened coordinates |
whitenedConstraints |
List from |
integrationTime |
Time for dynamics simulation |
diagnosticMode |
Return bounce diagnostics |
seed |
random seed |
# Basic usage with whitened coordinates set.seed(123) whitened_pos <- c(0.1, -0.2, 0.3) # Create example whitened constraints whitened_constraints <- list( direc = matrix(c(1, 0, 0, 0, 1, 0), nrow = 2, byrow = TRUE), direcRowNormSq = c(1, 1), bound = c(-0.5, -0.5) ) result <- getHarmonicSample( whitenedPosition = whitened_pos, whitenedConstraints = whitened_constraints, integrationTime = pi/4 ) result # With diagnostics enabled result_diag <- getHarmonicSample( whitenedPosition = whitened_pos, whitenedConstraints = whitened_constraints, integrationTime = pi/4, diagnosticMode = TRUE ) str(result_diag)# Basic usage with whitened coordinates set.seed(123) whitened_pos <- c(0.1, -0.2, 0.3) # Create example whitened constraints whitened_constraints <- list( direc = matrix(c(1, 0, 0, 0, 1, 0), nrow = 2, byrow = TRUE), direcRowNormSq = c(1, 1), bound = c(-0.5, -0.5) ) result <- getHarmonicSample( whitenedPosition = whitened_pos, whitenedConstraints = whitened_constraints, integrationTime = pi/4 ) result # With diagnostics enabled result_diag <- getHarmonicSample( whitenedPosition = whitened_pos, whitenedConstraints = whitened_constraints, integrationTime = pi/4, diagnosticMode = TRUE ) str(result_diag)
For a given MTN the function returns an initial vector whose elements are one of:
(1) middle point of the truncation interval if both lower and upper bounds are
finite (2) lower (upper) bound +0.1 (-0.1) if only the lower (upper) bound is finite
(3) the corresponding mean value if lower bound = -Inf are upper bound = Inf.
getInitialPosition(mean, lowerBounds, upperBounds)getInitialPosition(mean, lowerBounds, upperBounds)
mean |
a d-dimensional mean vector. |
lowerBounds |
a d-dimensional vector specifying the lower bounds. |
upperBounds |
a d-dimensional vector specifying the upper bounds. |
an eligible d-dimensional initial vector.
harmonicHMC(), zigzagHMC(), markovianZigzag()
# Example 1: Bounded interval mean <- c(0, 0) lower <- c(-1, -2) upper <- c(1, 2) getInitialPosition(mean, lower, upper) # Example 2: Mixed bounds (some finite, some infinite) mean <- c(0, 0, 0) lower <- c(-Inf, 0, -1) upper <- c(Inf, 5, Inf) getInitialPosition(mean, lower, upper) # Example 3: All unbounded (returns mean) mean <- c(1, 2, 3) lower <- c(-Inf, -Inf, -Inf) upper <- c(Inf, Inf, Inf) getInitialPosition(mean, lower, upper)# Example 1: Bounded interval mean <- c(0, 0) lower <- c(-1, -2) upper <- c(1, 2) getInitialPosition(mean, lower, upper) # Example 2: Mixed bounds (some finite, some infinite) mean <- c(0, 0, 0) lower <- c(-Inf, 0, -1) upper <- c(Inf, 5, Inf) getInitialPosition(mean, lower, upper) # Example 3: All unbounded (returns mean) mean <- c(1, 2, 3) lower <- c(-Inf, -Inf, -Inf) upper <- c(Inf, Inf, Inf) getInitialPosition(mean, lower, upper)
Simulate the Markovian zigzag dynamics for a given position over a specified travel time.
getMarkovianZigzagSample(position, velocity = NULL, engine, travelTime)getMarkovianZigzagSample(position, velocity = NULL, engine, travelTime)
position |
a d-dimensional position vector. |
velocity |
optional d-dimensional velocity vector. If NULL, it will be generated within the function. |
engine |
an object representing the Markovian zigzag engine, typically containing settings and state required for the simulation. |
travelTime |
the duration for which the dynamics are simulated. |
A list containing the position and velocity after simulating the dynamics.
# First create an engine set.seed(123) engine <- createEngine( dimension = 2, lowerBounds = c(-1, -1), upperBounds = c(1, 1), seed = 123, mean = c(0, 0), precision = diag(2) ) # Draw a single Markovian zigzag sample position <- c(0.1, -0.2) travel_time <- 0.5 sample_result <- getMarkovianZigzagSample( position = position, engine = engine, travelTime = travel_time ) sample_result# First create an engine set.seed(123) engine <- createEngine( dimension = 2, lowerBounds = c(-1, -1), upperBounds = c(1, 1), seed = 123, mean = c(0, 0), precision = diag(2) ) # Draw a single Markovian zigzag sample position <- c(0.1, -0.2) travel_time <- 0.5 sample_result <- getMarkovianZigzagSample( position = position, engine = engine, travelTime = travel_time ) sample_result
Simulate the Zigzag-HMC or Zigzag-NUTS dynamics on a given MTN.
getZigzagSample(position, momentum = NULL, nutsFlg, engine, stepSize = NULL)getZigzagSample(position, momentum = NULL, nutsFlg, engine, stepSize = NULL)
position |
a d-dimensional initial position vector. |
momentum |
a d-dimensional initial momentum vector. |
nutsFlg |
logical. If |
engine |
list. Its |
stepSize |
step size for Zigzag-HMC. If |
one MCMC sample from the target MTN.
getZigzagSample is particularly efficient when the target MTN has a random
mean and covariance/precision where one can reuse the Zigzag-HMC engine object while
updating the mean and covariance. The following example demonstrates such a use.
zigzagHMC(), drawLaplaceMomentum()
set.seed(1) n <- 1000 d <- 10 samples <- array(0, c(n, d)) # initialize MTN mean and precision m <- rnorm(d, 0, 1) prec <- rWishart(n = 1, df = d, Sigma = diag(d))[, , 1] # call createEngine once engine <- createEngine(dimension = d, lowerBounds = rep(0, d), upperBounds = rep(Inf, d), seed = 1, mean = m, precision = prec) HZZtime <- sqrt(2) / sqrt(min(mgcv::slanczos( A = prec, k = 1, kl = 1 )[['values']])) currentSample <- rep(0.1, d) for (i in 1:n) { m <- rnorm(d, 0, 1) prec <- rWishart(n = 1, df = d, Sigma = diag(d))[,,1] setMean(engine = engine, mean = m) setPrecision(engine = engine, precision = prec) currentSample <- getZigzagSample(position = currentSample, nutsFlg = FALSE, engine = engine, stepSize = HZZtime) samples[i,] <- currentSample }set.seed(1) n <- 1000 d <- 10 samples <- array(0, c(n, d)) # initialize MTN mean and precision m <- rnorm(d, 0, 1) prec <- rWishart(n = 1, df = d, Sigma = diag(d))[, , 1] # call createEngine once engine <- createEngine(dimension = d, lowerBounds = rep(0, d), upperBounds = rep(Inf, d), seed = 1, mean = m, precision = prec) HZZtime <- sqrt(2) / sqrt(min(mgcv::slanczos( A = prec, k = 1, kl = 1 )[['values']])) currentSample <- rep(0.1, d) for (i in 1:n) { m <- rnorm(d, 0, 1) prec <- rWishart(n = 1, df = d, Sigma = diag(d))[,,1] setMean(engine = engine, mean = m) setPrecision(engine = engine, precision = prec) currentSample <- getZigzagSample(position = currentSample, nutsFlg = FALSE, engine = engine, stepSize = HZZtime) samples[i,] <- currentSample }
Generate MCMC samples from a d-dimensional truncated Gaussian distribution with constraints Fx+g >= 0 using the Harmonic Hamiltonian Monte Carlo sampler (Harmonic-HMC).
harmonicHMC( nSample, burnin = 0, mean, choleskyFactor, constrainDirec, constrainBound, init, time = c(pi/8, pi/2), precFlg, seed = 1, extraOutputs = c() )harmonicHMC( nSample, burnin = 0, mean, choleskyFactor, constrainDirec, constrainBound, init, time = c(pi/8, pi/2), precFlg, seed = 1, extraOutputs = c() )
nSample |
number of samples after burn-in. |
burnin |
number of burn-in samples (default = 0). |
mean |
a d-dimensional mean vector. |
choleskyFactor |
upper triangular matrix R from Cholesky decomposition of precision or covariance matrix into R^TR. |
constrainDirec |
the k-by-d F matrix (k is the number of linear constraints). |
constrainBound |
the k-dimensional g vector. |
init |
a d-dimensional vector of the initial value. |
time |
HMC integration time for each iteration. Can either be a scalar value for a fixed time across all samples, or a length 2 vector of a lower and upper bound for uniform distribution from which the time is drawn from for each iteration. |
precFlg |
logical. whether |
seed |
random seed (default = 1). |
extraOutputs |
vector of strings. "numBounces" and/or "bounceDistances" can be requested, with the latter containing the distances in-between bounces for each sample and hence incurring significant computational and memory costs. |
When extraOutputs is empty (default), returns an nSample-by-d matrix of samples.
When extraOutputs contains "numBounces" and/or "bounceDistances", returns a list with elements:
samples |
|
numBounces |
Vector of bounce counts per sample (if requested) |
bounceDistances |
List of bounce distances per sample (if requested) |
Pakman, A. and Paninski, L. (2014). Exact Hamiltonian Monte Carlo for Truncated Multivariate Gaussians. Journal of Computational and Graphical Statistics. doi:10.1080/10618600.2013.788448
getHarmonicSample(), cholesky(), getInitialPosition()
set.seed(1) d <- 10 A <- matrix(runif(d^2)*2 - 1, ncol=d) precMat <- t(A) %*% A R <- cholesky(precMat) mu <- rep(0, d) constrainDirec <- diag(d) constrainBound <- rep(0,d) initial <- rep(1, d) results <- harmonicHMC(1000, 1000, mu, R, constrainDirec, constrainBound, initial, precFlg = TRUE)set.seed(1) d <- 10 A <- matrix(runif(d^2)*2 - 1, ncol=d) precMat <- t(A) %*% A R <- cholesky(precMat) mu <- rep(0, d) constrainDirec <- diag(d) constrainBound <- rep(0,d) initial <- rep(1, d) results <- harmonicHMC(1000, 1000, mu, R, constrainDirec, constrainBound, initial, precFlg = TRUE)
Sample from a truncated multivariate normal distribution using the Markovian Zigzag process, a continuous-time, non-reversible Markov chain Monte Carlo method based on piecewise deterministic Markov processes (PDMPs).
markovianZigzag( nSample, burnin = 0, mean, prec, lowerBounds, upperBounds, init = NULL, stepSize = NULL, seed = 1, diagnosticMode = FALSE, nStatusUpdate = 0L )markovianZigzag( nSample, burnin = 0, mean, prec, lowerBounds, upperBounds, init = NULL, stepSize = NULL, seed = 1, diagnosticMode = FALSE, nStatusUpdate = 0L )
nSample |
Number of samples after burn-in. |
burnin |
Number of burn-in samples (default = 0). |
mean |
A d-dimensional mean vector. |
prec |
A d-by-d precision matrix of the Gaussian distribution. |
lowerBounds |
A d-dimensional vector specifying the lower bounds.
|
upperBounds |
A d-dimensional vector specifying the upper bounds.
|
init |
A d-dimensional vector of the initial value. |
stepSize |
Step size for the Markovian Zigzag sampler. Default value
is the empirically optimal choice: |
seed |
Random seed (default = 1). |
diagnosticMode |
Logical. |
nStatusUpdate |
Number of status updates to print during sampling. If 0 (default), no updates are printed. |
An nSample-by-d matrix of samples. If diagnosticMode is TRUE,
a list with additional diagnostic information is returned.
Bierkens, J., Roberts, G. O., and Zitt, P.-A. (2019). Ergodicity of the zigzag process. The Annals of Applied Probability, 29(4): 2266-2301.
getMarkovianZigzagSample(), createEngine()
set.seed(1) d <- 5 A <- matrix(runif(d^2)*2-1, ncol=d) precMat <- t(A) %*% A initial <- rep(1, d) results <- markovianZigzag( nSample = 1000, burnin = 1000, mean = rep(0, d), prec = precMat, lowerBounds = rep(0, d), upperBounds = rep(Inf, d) )set.seed(1) d <- 5 A <- matrix(runif(d^2)*2-1, ncol=d) precMat <- t(A) %*% A initial <- rep(1, d) results <- markovianZigzag( nSample = 1000, burnin = 1000, mean = rep(0, d), prec = precMat, lowerBounds = rep(0, d), upperBounds = rep(Inf, d) )
Set the mean vector for a given Zigzag-HMC engine object.
setMean(engine, mean)setMean(engine, mean)
engine |
A Zigzag-HMC engine container object. |
mean |
the mean vector. |
createEngine(), createNutsEngine()
# First create an engine engine <- createEngine(dimension = 2, lowerBounds = c(-1, -1), upperBounds = c(1, 1), seed = 123, mean = c(0, 0), precision = diag(2)) # Update the mean setMean(engine, mean = c(0.5, 0.5))# First create an engine engine <- createEngine(dimension = 2, lowerBounds = c(-1, -1), upperBounds = c(1, 1), seed = 123, mean = c(0, 0), precision = diag(2)) # Update the mean setMean(engine, mean = c(0.5, 0.5))
Set the precision matrix for a given Zigzag-HMC engine object.
setPrecision(engine, precision)setPrecision(engine, precision)
engine |
A Zigzag-HMC engine container object. |
precision |
the precision matrix. |
createEngine(), createNutsEngine()
# First create an engine engine <- createEngine(dimension = 2, lowerBounds = c(-1, -1), upperBounds = c(1, 1), seed = 123, mean = c(0, 0), precision = diag(2)) # Update with a correlated precision matrix new_precision <- matrix(c(2, 0.8, 0.8, 2), nrow = 2) setPrecision(engine, precision = new_precision)# First create an engine engine <- createEngine(dimension = 2, lowerBounds = c(-1, -1), upperBounds = c(1, 1), seed = 123, mean = c(0, 0), precision = diag(2)) # Update with a correlated precision matrix new_precision <- matrix(c(2, 0.8, 0.8, 2), nrow = 2) setPrecision(engine, precision = new_precision)
Generate MCMC samples from a d-dimensional truncated Gaussian distribution with element-wise truncations using the Zigzag Hamiltonian Monte Carlo sampler (Zigzag-HMC).
zigzagHMC( nSample, burnin = 0, mean, prec, lowerBounds, upperBounds, init = NULL, stepSize = NULL, nutsFlg = FALSE, precondition = FALSE, seed = 1, diagnosticMode = FALSE )zigzagHMC( nSample, burnin = 0, mean, prec, lowerBounds, upperBounds, init = NULL, stepSize = NULL, nutsFlg = FALSE, precondition = FALSE, seed = 1, diagnosticMode = FALSE )
nSample |
number of samples after burn-in. |
burnin |
number of burn-in samples (default = 0). |
mean |
a d-dimensional mean vector. |
prec |
a d-by-d precision matrix of the Gaussian distribution. |
lowerBounds |
a d-dimensional vector specifying the lower bounds. |
upperBounds |
a d-dimensional vector specifying the upper bounds. |
init |
a d-dimensional vector of the initial value. |
stepSize |
step size for Zigzag-HMC or Zigzag-NUTS (if |
nutsFlg |
logical. If |
precondition |
logical. If |
seed |
random seed (default = 1). |
diagnosticMode |
logical. |
When diagnosticMode = FALSE (default), returns an nSample-by-d matrix of samples.
When diagnosticMode = TRUE, returns a list with elements:
samples |
|
stepsize |
The step size used for sampling |
Nishimura, A., Zhang, Z., and Suchard, M. A. (2024). Zigzag path connects two Monte Carlo samplers: Hamiltonian counterpart to a piecewise deterministic Markov process. Journal of the American Statistical Association, 1-13.
Nishimura, A., Dunson, D. B., and Lu, J. (2020). Discontinuous Hamiltonian Monte Carlo for discrete parameters and discontinuous likelihoods. Biometrika, 107(2): 365-380.
getZigzagSample(), createEngine(), createNutsEngine(), setMean(), setPrecision()
set.seed(1) d <- 10 A <- matrix(runif(d^2)*2-1, ncol=d) precMat <- t(A) %*% A initial <- rep(1, d) results <- zigzagHMC(nSample = 1000, burnin = 1000, mean = rep(0, d), prec = precMat, lowerBounds = rep(0, d), upperBounds = rep(Inf, d))set.seed(1) d <- 10 A <- matrix(runif(d^2)*2-1, ncol=d) precMat <- t(A) %*% A initial <- rep(1, d) results <- zigzagHMC(nSample = 1000, burnin = 1000, mean = rep(0, d), prec = precMat, lowerBounds = rep(0, d), upperBounds = rep(Inf, d))