Advances in High-Dimensional Statistics: Sparsity, Rough Path Learning, and Geometric Invariance

Abstract

High-dimensional statistics addresses the challenges of analyzing data where the number of parameters exceeds the number of observations, a scenario increasingly common in modern applications such as deep learning, image analysis, and time series modeling. Traditional estimators like least squares tend to overfit and become computationally infeasible in such regimes. This thesis presents methodological contributions that address both statistical and computational challenges in high-dimensional settings by introducing novel regularization techniques, exploring path signatures for modeling irregular data, and leveraging geometric invariances in neural networks. The first contribution lays the foundation with the introduction of cardinality sparsity, a generalized sparsity concept encouraging models with few unique parameter values rather than merely a few non-zero ones. This framework, detailed in Chapter 2, leads to new regularization schemes that promote structured simplicity and provide sharp theoretical guarantees on estimation and generalization. Chapter 3 builds on this by exploring the computational benefits, where the restricted number of distinct parameter values enhances efficiency, particularly in matrix multiplication speed and memory usage, offering an approach to statistically robust and computationally scalable models. The second part of the thesis, presented in Chapter 4, explores high-dimensional time series models through the lens of signature transforms, particularly in the context of fractional Brownian motion (fBm). Using the path signature representation and Lasso regularization, the work proposes the Signature Lasso method for learning from rough paths. Theoretical analysis, supported by simulations, shows that this approach can outperform traditional regression techniques, especially when data exhibits irregular temporal patterns and dependency structures. The third contribution, covered in Chapter 5, focuses on group-convolutional neural networks (G-CNNs) and their ability to generalize convolutional operations to handle a broader class of geometric transformations. While classical CNNs are limited to translation invariance, G-CNNs introduced in this work are shown to be stable under affine transformations generated by the general linear group. This significantly expands the class of symmetries neural networks can effectively learn, while simplifying complex group convolutions to standard real-valued integrations. Collectively, these contributions propose principled approaches to taming overfitting, enhancing model interpretability, and reducing computational burdens in high-dimensional learning. The methods developed span across statistical theory, time series analysis, and neural architecture design, offering a robust framework for modern data science applications.