Statistical Methods in Finance 2023

                                

Abstract

On Bayesian Analysis with Jacobi Priors for Big Data

By Sourish Das
Chennai Mathematical Institute, India

Abstract This paper introduces an alternative Bayesian method, termed the 'Jacobi prior,' aiming to address the computational challenges inherent in traditional techniques. The Jacobi prior, characterized by its innovative approach, is specifically designed to enhance predictive accuracy while concurrently reducing time complexities, surpassing the performance of established methods. Our objective is to provide an efficient solution for intricate predictive modelling tasks, particularly in the realm of big data challenges within cloud-based environments. The Jacobi prior proves instrumental in mitigating computational complexities associated with conventional approaches, making it particularly relevant for organizations seeking to minimize their carbon footprint and achieve long-term compliance with environmental, social, and governance (ESG) standards. This paper demonstrates the superior performance of the alternate Bayesian approach embodied by the Jacobi prior in comparison to well-established methods such as Lasso, Ridge, Elastic Net, and MCMC-based Bayesian methods, particularly in the context of predictive accuracy. To illustrate the effectiveness of the alternate Bayesian method, we present a comprehensive evaluation focused on Credit Risk estimation, specifically assessing default probability. The dataset utilized for this analysis is sourced from the U.S. Small Business Administration (SBA). Through rigorous experimentation and analysis, we substantiate the efficacy of the Jacobi prior as a cutting-edge solution for predictive modelling, showcasing its superiority in addressing contemporary computational challenges. Moreover, the Jacobi prior exhibits remarkable versatility by efficiently handling partitioned data distributed over different servers across continents, further establishing its applicability in diverse and distributed computing environments. This is a joint work with Abhishek Chakraborty and Shouvik Sardar.

---

---