Marques, Mateus Maia
(2024)
Smoothness and covariance structure modelling
in Bayesian machine learning models.
PhD thesis, National University of Ireland Maynooth.
Abstract
Bayesian additive regression trees (BART) is a Bayesian tree-based model which can
provide high predictive accuracy in both classification and regression problems. Within
the Bayesian paradigm, regularisation is achieved by defining priors which ensure that
each tree contributes modestly to the overall ensemble, thereby enhancing generalisation.
Consequently, BART has proven to be very useful in a wide array of applications.
However, the standard BART model is limited in certain respects. This thesis introduces
some novel extensions to the BART framework to address certain key shortcomings. The
inherent lack of smoothness, which is intrinsic to the piecewise-constant nature of the
decision trees, is the motivation behind two of our proposals. The first involves the incorporation
of Gaussian processes while the second uses penalised splines in the terminal
nodes. Both of these novel approaches yield demonstrable improvements from the points
of view of predictive accuracy and uncertainty calibration in extensive simulations and
real-world applications.
Another drawback of the standard BART model is that it is designed for predicting
univariate outcomes. We introduce a third extension to embed BART in the seemingly
unrelated regression framework to deal with multiple outcomes and model the covariance
structure arising from their joint distribution. The method is applied in a causal setting
in order to determine the cost-effectiveness of a novel medical intervention.
The incorporation of penalised splines is designed to introduce smoothness to BART’s
predictions. Concurrently, the extension to model multivariate outcomes within a seemingly
unrelated regression framework enhances BART by structuring the covariance
among responses. The synthesis of Gaussian processes with BART exemplifies this
dual enhancement, simultaneously facilitating smooth predictive surfaces and capturing
structured dependency, although the latter is within the feature space.
Item Type: |
Thesis
(PhD)
|
Keywords: |
Smoothness; covariance; structure modelling;
Bayesian machine learning models; |
Academic Unit: |
Faculty of Science and Engineering > Research Institutes > Hamilton Institute |
Item ID: |
18853 |
Depositing User: |
IR eTheses
|
Date Deposited: |
10 Sep 2024 14:55 |
URI: |
|
Use Licence: |
This item is available under a Creative Commons Attribution Non Commercial Share Alike Licence (CC BY-NC-SA). Details of this licence are available
here |
Repository Staff Only(login required)
|
Item control page |
Downloads per month over past year
Origin of downloads