Bayesian Methods for Distributional and Semiparametric Regression: Our Satellite Event at the Bayes Comp 2025

This satellite workshop aims to bridge the gap between computational and theoretical advancements and modern applications in Bayesian methods for distributional and semiparametric regression by bringing together leading experts in the field. Participants will benefit from talks that cover key tasks, such as model formulation, variable selection, inference techniques and associated computational challenges and practical implications. By highlighting the latest developments, this workshop will provide an overview of current research advancements, fostering discussions that inspire collaboration and innovation in advanced Bayesian regression.

The satellite event is a part of Bayes Comp 2025, taking place from June 16–20, 2025, in Singapore. Registration is available through the conference Website.

Tentative Programme

Monday 16 June

• 11:15-11:45 Welcome & Opening Address
• 11:45-12:30 Michael Stanley Smith - Deep Distributional Time Series Models and the Probabilistic Forecasting of Intraday Electricity Prices
• 12:30-14:00 Lunch
• 14:00-14:45 Marina Vannucci - Varying-coefficients Bayesian models for inference of networks and covariate effects
• 14:45-15:30 Zhidi Lin - Towards Flexibility and Efficiency of Gaussian Process State-Space Models
• 15:30-16:00 Coffee
• 16:00-16:45 Robert Kohn - ProDAG: Projection-Induced Variational Inference for Directed Acyclic Graphs
• 16:45-17:30 Francesca Labanca - Copula-based models for spatially dependent cylindrical data

Tuesday 17 June

• 09:00-09:45 Cheng Li - Radial neighbours for provably accurate scalable approximations of Gaussian processes
• 09:45-10:30 Maria De Iorio - TBD
• 10:30-11:00 Coffee
• 11:00-11:45 Mattias Villani - Time-varying multi-seasonal ARMA processes with semiparametric evolution over time
• 11:45-12:30 Bohan Wu - Monte Carlo Inference for Semiparametric Bayesian Regression
• 12:30-14:00 Lunch
• 14:00-14:45 David Rossell - Semi-parametric local variable selection under misspecification
• 14:45-15:30 Jaewoo Park - Bayesian Function-on-Function Regression for Spatial Functional Data
• 15:30-16:00 Coffee
• 16:00-16:45 Scott A. Sisson - Regression with random rectangle summaries and variational transdimensional inference
• 16:45-17:30 Antik Chakraborty - Orthogonal calibration via posterior projections with applications to the Schwarzschild model
• We plan to organize a joint dinner in the early evening for further networking and to collectively unwind after the workshop

Organizers

The satellite event is organized by

• Nadja Klein, Scientific Computing Center, Karlsruhe Institute of Technology
• Lucas Kock, Department of Statistics and Data Science, National University of Singapore

Abstracts

• Antik Chakraborty, Department of Statistics, Purdue University

Orthogonal calibration via posterior projections with applications to the Schwarzschild model

Abstract: The orbital superposition method originally developed by \cite{schwarzschild1979numerical} is used to study the dynamics of growth of a black hole and its host galaxy, and has uncovered new relationships between the galaxy's global characteristics. Scientists are specifically interested in finding optimal parameter choices for this model that best match physical measurements along with quantifying the uncertainty of such procedures. This renders a statistical calibration problem with multivariate outcomes. In this article, we develop a Bayesian method for calibration with \emph{multivariate outcomes} using orthogonal bias functions. Our approach is based on projecting the posterior to an appropriate space which allows the user to choose any nonparametric prior on the bias function(s) instead of having to model it (them) with Gaussian processes. We develop a functional projection approach using the theory of Hilbert spaces. A finite-dimensional analog of the projection problem is also considered. We illustrate the proposed approach using a BART prior and apply it to calibrate the Schwarzschild model illustrating how a multivariate approach may resolve discrepancies resulting from a univariate calibration.

• Maria De Iorio, Department of Paediatrics, Yong Loo Lin School of Medicine, National University of Singapore

TBD

Abstract: TBD

• Robert Kohn, School of Economics, UNSW Business School, University of New South Wales

ProDAG: Projection-Induced Variational Inference for Directed Acyclic Graphs

Abstract: Directed acyclic graph (DAG) learning is a rapidly expanding field of research. Though the field has witnessed remarkable advances over the past few years, it remains statistically and computationally challenging to learn a single (point estimate) DAG from data, let alone provide uncertainty quantification. Our paper addresses the difficult task of quantifying graph uncertainty by developing a Bayesian variational inference framework based on novel distributions that have support directly on the space of DAGs. The distributions, which we use to form our prior and variational posterior, are induced by a projection operation, whereby an arbitrary continuous distribution is projected onto the space of sparse weighted acyclic adjacency matrices (matrix representations of DAGs) with probability mass on exact zeros. Though the projection constitutes a combinatorial optimization problem, it is solvable at scale via recently developed techniques that reformulate acyclicity as a continuous constraint. We empirically demonstrate that our proposed method, \texttt{ProDAG}, can perform higher quality Bayesian inference than possible with existing state-of-the-art alternatives.

• Francesca Labanca, Dipartimento di Statistica, Informatica, Applicazioni "G. Parenti" (DiSIA), University of Florence

Copula-based models for spatially dependent cylindrical data

Abstract: Cylindrical data frequently arise across various scientific disciplines, including meteorology (e.g., wind direction and speed), oceanography (e.g., marine current direction and speed or wave heights), ecology (e.g., telemetry), and medicine (e.g., seasonality and intensity in disease onset). Such data often occur as spatially correlated series of intensities and angles, thereby representing dependent bivariate response vectors of linear and circular components. To accommodate both the circular-linear dependence and spatial autocorrelation, while remaining flexible in their marginal specifications, copula-based models for cylindrical data have been developed in the literature. However, existing approaches typically treat the copula parameters as constants not related to covariates, and regression specifications for the marginal distributions are frequently restricted to linear predictors. In this work, we propose a structured additive conditional copula regression model for cylindrical data. In our approach, the circular component is modeled using a wrapped Gaussian process, and the linear component follows a distributional regression, and both components allow for the inclusion of linear and non-linear covariates effects. Furthermore, by leveraging the empirical equivalence between Gaussian random fields (GRFs) and Gaussian Markov random fields (GMRFs), our approach avoids the computational bur- den typically associated with GRFs, while simultaneously allowing for non-stationarity in the covariance structure. Posterior estimation is performed via Markov chain Monte Carlo simulation. We evaluate the performance of the proposed model in a simulation study and subsequently in an analysis on wind directions and speed in Germany. This is joint work with Anna Gottard and Nadja Klein.

• Cheng Li, Department of Statistic and Data Science, National University of Singapore

Radial neighbours for provably accurate scalable approximations of Gaussian processes

Abstract: In geostatistical problems with massive sample size, Gaussian processes can be approximated using sparse directed acyclic graphs to achieve scalable O(n) computational complexity. In these models, data at each location are typically assumed conditionally dependent on a small set of parents that usually include a subset of the nearest neighbours. These methodologies often exhibit excellent empirical performance, but the lack of theoretical validation leads to unclear guidance in specifying the underlying graphical model and sensitivity to graph choice. We address these issues by introducing radial-neighbour Gaussian processes, a class of Gaussian processes based on directed acyclic graphs in which directed edges connect every location to all of its neighbours within a predetermined radius. We prove that any radial-neighbour Gaussian process can accurately approximate the corresponding unrestricted Gaussian process in the Wasserstein-2 distance, with an error rate determined by the approximation radius, the spatial covariance function and the spatial dispersion of samples. We offer further empirical validation of our approach via applications on simulated and real-world data, showing excellent performance in both prior and posterior approximations to the original Gaussian process.

• Zhidi Lin, Department of Statistic and Data Science, National University of Singapore

Towards Flexibility and Efficiency of Gaussian Process State-Space Models

Abstract: Gaussian Process State-Space Models (GPSSMs) provide a robust framework for modeling complex dynamical systems by combining probabilistic inference with nonparametric flexibility. However, their practical adoption is often constrained by computational challenges and limited flexibility to diverse real-world scenarios. This talk introduces recent advancements that significantly improve the flexibility and learning efficiency of GPSSMs. We present scalable methodologies leveraging Gaussian process approximations and ensemble Kalman filters, enabling efficient training on large-scale, high-dimensional datasets. Additionally, we discuss innovations in model design incorporating normalizing flows, which enhance GPSSMs’ ability to capture intricate temporal dependencies and adapt to evolving system dynamics. Empirical results on both synthetic and real-world datasets illustrate the potential of these techniques, demonstrating improved accuracy, reduced computational demands, and greater model capacity. These contributions highlight a promising pathway for deploying GPSSMs in practical, real-world applications.

• Jaewoo Park, Department of Statistics and Data Science, Yonsei University

Bayesian Function-on-Function Regression for Spatial Functional Data

Abstract: Spatial functional data arise in many settings, such as particulate matter curves observed at monitoring stations and age population curves at each areal unit. Most existing functional regression models have limited applicability because they do not consider spatial correlations. Although functional kriging methods can predict the curves at unobserved spatial locations, they are based on variogram fittings rather than constructing hierarchical statistical models. In this manuscript, we propose a Bayesian framework for spatial function-on-function regression that can carry out parameter estimations and predictions. However, the proposed model has computational and inferential challenges because the model needs to account for within and between-curve dependencies. Furthermore, high-dimensional and spatially correlated parameters can lead to the slow mixing of Markov chain Monte Carlo algorithms. To address these issues, we first utilize a basis transformation approach to simplify the covariance and apply projection methods for dimension reduction. We also develop a simultaneous band score for the proposed model to detect the significant region in the regression function. We apply our method to both areal and point-level spatial functional data, showing the proposed method is computationally efficient and provides accurate estimations and predictions.

• David Rossell, Department of Business and Economics, Universitat Pompeu Fabra

Semi-parametric local variable selection under misspecification

Abstract: Local variable selection aims to test for the effect of covariates on an outcome within specific regions. We outline a challenge that arises in the presence of non-linear effects and model misspecification. Specifically, for common semi-parametric methods even slight model misspecification can result in a high false positive rate, in a manner that is highly sensitive to the chosen basis functions. We propose a methodology based on orthogonal cut splines that avoids false positive inflation for any choice of knots, and achieves consistent local variable selection. Our approach offers simplicity, handles both continuous and categorical covariates, and provides theory for high-dimensional covariates and model misspecification. We discuss settings with either independent or dependent data. Our proposal allows including adjustment covariates that do not undergo selection, enhancing the model's flexibility. Our examples describe salary gaps associated with various discrimination factors at different ages, and the effects of covariates on functional data measuring brain activation at different times.

• Scott A. Sisson, School of Mathematics and Statistics, University of New South Wales

Regression with random rectangle summaries and variational transdimensional inference

Abstract: The first concerns how to implement regression models y=f(x) when individual (x_i,y_i) are not observed, but where instead we observe random rectangles bounding a group of (x_i,y_i). This can be useful when there is a large number of observations, or when there are privacy concerns. The approach constructs an integrated likelihood and develops exact and approximate pseudo-marginal MCMC algorithms for inference. The second concerns how to implement a variational approximation to the posterior distribution of a Bayesian robust variable selection problem. Rather than construct a standard variational approximation of each model, and across the distribution of model probabilities, we instead attempt this using a single variational target that spans the entire transdimensional model space. The approach introduces a CoSMIC normalising flow, and embeds this within a variational transdimensional inference (VTI) algorithm.

• Michael Stanley Smith, Melbourne Business School, University of Melbourne

Deep Distributional Time Series Models and the Probabilistic Forecasting of Intraday Electricity Prices

Abstract: Recurrent neural networks (RNNs) with rich feature vectors of past values can provide accurate point forecasts for series that exhibit complex serial dependence. We propose two approaches to constructing deep time series probabilistic models based on a variant of RNN called an echo state network (ESN). The first is where the output layer of the ESN has stochastic disturbances and a Bayesian prior for regularization. The second employs the implicit copula of an ESN with Gaussian disturbances, which is a Gaussian copula process on the feature space. Combining this copula process with a non-parametrically estimated marginal distribution produces a distributional time series model. The resulting probabilistic forecasts are deep functions of the feature vector and marginally calibrated. In both approaches, Markov chain Monte Carlo methods are used to estimate the models and compute forecasts. The proposed models are suitable for the complex task of forecasting intraday electricity prices. Using data from the Australian market, we show that our deep time series models provide accurate short term probabilistic price forecasts, with the copula model dominating. Moreover, the models provide a flexible framework for incorporating probabilistic forecasts of electricity demand, which increases upper tail forecast accuracy from the copula model significantly.

• Marina Vannucci, Department of Statistics, Rice University

Varying-coefficients Bayesian models for inference of networks and covariate effects

Abstract: New methods for the simultaneous inference of graphical models and covariates effects in the Bayesian framework will be discussed. I will consider regression settings where the interest is in the estimation of sparse networks among a set of primary variables, and where covariates may impact the strength of edges. The proposed models utilize spike-and-slab priors to perform edge selection, and Gaussian process priors to allow for flexibility in the covariate effects. Efficient and scalable algorithms for posterior inference will be employed for the estimation of the models. Simulation studies will demonstrate how the proposed models improve on the accuracy of existing methods, in both network recovery and covariate selection. I will show applications of the proposed models to neuroimaging and genomic datasets.

• Mattias Villani, Department of Statistics, Stockholm University

Time-varying multi-seasonal ARMA processes with semiparametric evolution over time

Abstract: We develop multi-seasonal ARMA processes where the time evolution of the parameters follow a flexible semiparametric process over the stability and invertibility regions. Computational algorithms are developed to sample from the posterior distribution. The model and computational methods are illustrated on simulated and real data.

• Bohan Wu, Department of Statistics, Columbia University

Monte Carlo Inference for Semiparametric Bayesian Regression

Abstract: Data transformations are essential for broad applicability of parametric regression models. However, for Bayesian analysis, joint inference of the transformation and model parameters typically involves restrictive parametric transformations or nonparametric representations that are computationally inefficient and cumbersome for implementation and theoretical analysis, which limits their usability in practice. We introduce a simple, general, and efficient strategy for joint posterior inference of an unknown transformation and all regression model parameters. The proposed approach directly targets the posterior distribution of the transformation by linking it with the marginal distributions of the independent and dependent variables, and then deploys a Bayesian nonparametric model via the Bayesian bootstrap. Crucially, this approach delivers (1) joint posterior consistency under general conditions, including multiple model misspecifications, and (2) efficient Monte Carlo (not Markov chain Monte Carlo) inference for the transformation and all parameters for important special cases. These tools apply across a variety of data domains, including real-valued, positive, and compactly-supported data. Simulation studies and an empirical application demonstrate the effectiveness and efficiency of this strategy for semiparametric Bayesian analysis with linear models, quantile regression, and Gaussian processes. The R package SeBR is available on CRAN.