Talagrand's concentration inequality

In the probability theory field of mathematics, Talagrand's concentration inequality is an isoperimetric-type inequality for product probability spaces.^[1][2] It was first proved by the French mathematician Michel Talagrand.^[3] The inequality is one of the manifestations of the concentration of measure phenomenon.^[2]

Roughly, the product of the probability to be in some subset of a product space (e.g. to be in one of some collection of states described by a vector) multiplied by the probability to be outside of a neighbourhood of that subspace at least a distance ${\displaystyle t}$ away, is bounded from above by the exponential factor ${\displaystyle e^{-t^{2}/4}}$. It becomes rapidly more unlikely to be outside of a larger neighbourhood of a region in a product space, implying a highly concentrated probability density for states described by independent variables, generically. The inequality can be used to streamline optimisation protocols by sampling a limited subset of the full distribution and being able to bound the probability to find a value far from the average of the samples.^[4]

Statement

The inequality states that if ${\displaystyle \Omega =\Omega _{1}\times \Omega _{2}\times \cdots \times \Omega _{n}}$ is a product space endowed with a product probability measure and ${\displaystyle A}$ is a subset in this space, then for any ${\displaystyle t\geq 0}$

${\displaystyle \Pr[A]\cdot \Pr \left[{A_{t}^{c}}\right]\leq e^{-t^{2}/4}\,,}$

where ${\displaystyle {A_{t}^{c}}}$ is the complement of ${\displaystyle A_{t}}$ where this is defined by

${\displaystyle A_{t}=\{x\in \Omega ~:~\rho (A,x)\leq t\}}$

and where ${\displaystyle \rho }$ is Talagrand's convex distance defined as

${\displaystyle \rho (A,x)=\max _{\alpha ,\|\alpha \|_{2}\leq 1}\ \min _{y\in A}\ \sum _{i~:~x_{i}\neq y_{i}}\alpha _{i}}$

where ${\displaystyle \alpha \in \mathbf {R} ^{n}}$, ${\displaystyle x,y\in \Omega }$ are ${\displaystyle n}$-dimensional vectors with entries ${\displaystyle \alpha _{i},x_{i},y_{i}}$ respectively and ${\displaystyle \|\cdot \|_{2}}$ is the ${\displaystyle \ell ^{2}}$-norm. That is,

${\displaystyle \|\alpha \|_{2}=\left(\sum _{i}\alpha _{i}^{2}\right)^{1/2}}$

References