Central limit theorems for stochastic approximation with controlled Markov chain dynamics

LTCI, CNRS & TELECOM ParisTech, 46 rue Barrault, 75634 Paris cedex 13, France
gersende.fort@telecom-paristech.fr

Received: 11 September 2013
Revised: 5 March 2014

Abstract

This paper provides a Central Limit Theorem (CLT) for a process {θ_n, n ≥ 0} satisfying a stochastic approximation (SA) equation of the form θ_{n + 1} = θ_n + γ_{n + 1}H(θ_n,X_{n + 1}); a CLT for the associated average sequence is also established. The originality of this paper is to address the case of controlled Markov chain dynamics {X_n, n ≥ 0} and the case of multiple targets. The framework also accomodates (randomly) truncated SA algorithms. Sufficient conditions for CLT’s hold are provided as well as comments on how these conditions extend previous works (such as independent and identically distributed dynamics, the Robbins−Monro dynamic or the single target case). The paper gives a special emphasis on how these conditions hold for SA with controlled Markov chain dynamics and multiple targets; it is proved that this paper improves on existing works.