An ℓ1-oracle inequality for the Lasso in finite mixture Gaussian regression models
Laboratoire de Mathématiques, Faculté des Sciences d’Orsay,
Université Paris-Sud, 91405
Revised: 17 July 2012
We consider a finite mixture of Gaussian regression models for high-dimensional heterogeneous data where the number of covariates may be much larger than the sample size. We propose to estimate the unknown conditional mixture density by an ℓ1-penalized maximum likelihood estimator. We shall provide an ℓ1-oracle inequality satisfied by this Lasso estimator with the Kullback–Leibler loss. In particular, we give a condition on the regularization parameter of the Lasso to obtain such an oracle inequality. Our aim is twofold: to extend the ℓ1-oracle inequality established by Massart and Meynet  in the homogeneous Gaussian linear regression case, and to present a complementary result to Städler et al. , by studying the Lasso for its ℓ1-regularization properties rather than considering it as a variable selection procedure. Our oracle inequality shall be deduced from a finite mixture Gaussian regression model selection theorem for ℓ1-penalized maximum likelihood conditional density estimation, which is inspired from Vapnik’s method of structural risk minimization  and from the theory on model selection for maximum likelihood estimators developed by Massart in .
Mathematics Subject Classification: 62G08 / 62H30
Key words: Finite mixture of Gaussian regressions model / Lasso / ℓ1-oracle inequalities / model selection by penalization / ℓ1-balls
© EDP Sciences, SMAI, 2013