Banach space

A Banach space is a complete normed vector space. That is, it’s a vector space equipped with a norm $\left|\cdot \right|$ which induces a distance function in the obvious way. Every Cauchy sequence converges to some point in the space.

A major obstacle to working in Banach spaces is that an inner product does not necessarily exist.

In statistics, we usually assume the space is separable, meaning that it admits a countable dense subset. This makes sure that probability measures, Borel $\sigma$-algebras, and empirical means are well-defined.

We also often place a smoothness assumption on the space. Without such a smoothness assumption, roughly speaking, values that are close to each other in the space may behave so differently that statistical inference becomes impossible.

A Banach space $(B,\left|\cdot \right|)$ is $(2,\beta )$ smooth if

$${\left|x+y\right|}^2+{\left|x-y\right|}^2\le 2{\left|x\right|}^2+2{\beta }^2{\left|y\right|}^2,\forall x,y\in B.$$

An equivalent condition is that the Banach space $(B,\left|\cdot \right|)$ is $\beta$-smooth, where we say the space is $\beta$-smooth if the squared norm ${\left|\cdot \right|}^2$ is $\beta$-smooth with respect to $\left|\cdot \right|$, where we say that a function $f:B\to \mathbb{R}$ is $\beta$-smooth with respect to $\left|\cdot \right|$ if for any $x,y\in B$,

$$f(y)\le f(x)+D_{x}f(x)(y-x)+\frac{\beta }{2}{\left|y-x\right|}^2,$$

where $D_{x}f(x)(y-x)$ is the Gateaux derivative of $f$ at $x$ in the direction $y-x$.

Smooth, separable Banach spaces include any separable Hilbert space $(\beta =1)$ and Lp spaces ($\beta \le p\wedge 2$).

See concentration in Banach spaces.