The paper “Adapting to Noise Tails in Private Linear Regression”, co-authored by Prof. Jinyuan Chang, postdoctoral researcher Lin Yang, PhD student Mengyue Zha of the Hong Kong University of Science and Technology, and Professor Wenxin Zhou of the University of Illinois Chicago, has been officially accepted by the Journal of the American Statistical Association, ,one of the leading international journals in the field of statistics.
Abstract
While the traditional goal of statistics is to infer population parameters, modern practice increasingly demands protection of individual privacy. One way to address this need is to adapt classical statistical procedures into privacy-preserving algorithms. In this paper, we develop differentially private tail-robust methods for linear regression. The trade-off among bias, privacy, and robustness is controlled by a tunable robustification parameter in the Huber loss. We implement noisy clipped gradient descent for low-dimensional settings and noisy iterative hard thresholding for high-dimensional sparse models. Under sub-Gaussian errors, our method achieves near-optimal convergence rates while relaxing several assumptions required in earlier work. For heavy-tailed errors, we explicitly characterize how the non-asymptotic convergence rate depends on the moment index, privacy parameters, sample size, and intrinsic dimension. Our analysis shows how the moment index influences the choice of robustification parameters and, in turn, the resulting statistical error and privacy cost. By quantifying the interplay among bias, privacy, and robustness, we extend classical perspectives on privacy-preserving robust regression. The proposed methods are evaluated through simulations and two real datasets.
Author Introduction
Jinyuan Chang is the Executive Director of the Joint Laboratory of Data Science and Business Intelligence at Southwestern University of Finance and Economics. He is a Guanghua Chair Professor and a recipient of the National Science Fund for Distinguished Young Scholars of China. He primarily engaged in research related to complex data analysis.
Lin Yang is a postdoctoral fellow at the Joint Laboratory of Data Science and Business Intelligence, Southwestern University of Finance and Economics. His research mainly focuses on high-dimensional data analysis and functional data analysis.
Mengyue Zha is a PhD student in the Department of Mathematics at the Hong Kong University of Science and Technology. Her research mainly focuses on large language models, adversarial attack and defense for intelligent agents, and high-dimensional statistics.
Wenxin Zhou is an Associate Professor at the College of Business Administration, the University of Illinois Chicago. His research mainly focuses on high-dimensional statistics and robust inference, nonparametric learning and modern machine learning methods, deep learning and neural network theory, as well as quantile regression and expected shortfall regression.
