Convex_conjugate : definition of Convex_conjugate and synonyms of Convex

In mathematics, convex conjugation is a generalization of the Legendre transformation. It is also known as Legendre–Fenchel transformation or Fenchel transformation (after Adrien-Marie Legendre and Werner Fenchel).

1 Definition
2 Examples
- 2.1 Connection with average value at risk
- 2.2 Ordering
3 Properties
4 Table of selected convex conjugates
5 See also
6 References
7 External links

Definition

Let $X$ be a real normed vector space, and let $X^{*}$ be the dual space to $X$ . Denote the dual pairing by

$\langle \cdot , \cdot \rangle : X^{*} \times X \to \mathbb{R}.$

For a functional

$f : X \to \mathbb{R} \cup \{ + \infty \}$

taking values on the extended real number line, the convex conjugate

$f^\star : X^{*} \to \mathbb{R} \cup \{ + \infty \}$

is defined in terms of the supremum by

$f^{\star} \left( x^{*} \right) := \sup \left \{ \left. \left\langle x^{*} , x \right\rangle - f \left( x \right) \right| x \in X \right\},$

or, equivalently, in terms of the infimum by

$f^{\star} \left( x^{*} \right) := - \inf \left \{ \left. f \left( x \right) - \left\langle x^{*} , x \right\rangle \right| x \in X \right\}.$

This definition can be interpreted as an encoding of the convex hull of the function's epigraph in terms of its supporting hyperplanes. [1]

Examples

The convex conjugate of an affine function

$f(x) = \left\langle a,x \right\rangle - b,\, a \in \mathbb{R}^n, b \in \mathbb{R}$

$f^\star\left(x^{*} \right) = \begin{cases} b, & x^{*} = a \\ +\infty, & x^{*} \ne a. \end{cases}$

The convex conjugate of a power function

$f(x) = \frac{1}{p}|x|^p,\,1<p<\infty$

$f^\star\left(x^{*} \right) = \frac{1}{q}|x^{*}|^q,\,1<q<\infty$

where $\tfrac{1}{p} + \tfrac{1}{q} = 1.$

The convex conjugate of the absolute value function

$f(x) = \left| x \right|$

$f^\star\left(x^{*} \right) = \begin{cases} 0, & \left|x^{*} \right| \le 1 \\ \infty, & \left|x^{*} \right| > 1. \end{cases}$

The convex conjugate of the exponential function $f(x)=\,\! e^x$ is

$f^\star\left(x^{*} \right) = \begin{cases} x^{*} \ln x^{*} - x^{*} , & x^{*} > 0 \\ 0 , & x^{*} = 0 \\ \infty , & x^{*} < 0. \end{cases}$

Convex conjugate and Legendre transform of the exponential function agree except that the domain of the convex conjugate is strictly larger as the Legendre transform is only defined for positive real numbers.

Connection with average value at risk

Let F denote a cumulative distribution function of a random variable X. Then

$f(x):= \int_{-\infty}^x F(u)\,du = \operatorname{E}\left[\max(0,x-X)\right] = x-\operatorname{E} \left[\min(x,X)\right]$

has the convex conjugate

$\begin{align} f^\star(p)=\int_0^p F^{-1}(q) \, dq & = (p-1)F^{-1}(p)+\operatorname{E}\left[\min(F^{-1}(p),X)\right] \\ & = p F^{-1}(p)-\operatorname{E}\left[\max(0,F^{-1}(p)-X)\right].\end{align}$

Ordering

A particular interpretation has the transform

$f^\text{inc}(x):= \arg \sup_t \,t\cdot x-\int_0^1 \max\{t-f(u),0\} \, \mathrm d u,$

as this is a nondecreasing rearrangement of the initial function f; in particular, $f^\text{inc}= f$ for ƒ nondecreasing.

Properties

The convex conjugate of a closed convex function is again a closed convex function. The convex conjugate of a polyhedral convex function (a convex function with polyhedral epigraph) is again a polyhedral convex function.

Convex-conjugation is order-reversing: if $f \le g$ then $f^* \ge g^*$ . Here

$(f \le g ) :\iff (\forall x, f(x) \le g(x)).$

Biconjugate

The convex conjugate of a function is always lower semi-continuous. The biconjugate $f^{**}$ (the convex conjugate of the convex conjugate) is also the closed convex hull, i.e. the largest lower semi-continuous convex function with $f^{**}\le f$ . For proper functions f, f = f^** if and only if f is convex and lower semi-continuous by Fenchel–Moreau theorem.

Fenchel's inequality

For any function f and its convex conjugate f^* Fenchel's inequality (also known as the Fenchel–Young inequality) holds for every $x \in X$ and $p \in X^*$ :

$\left\langle p,x \right\rangle \le f(x) + f^\star(p).$

Maximizing argument

It is interesting to observe that the derivative of the function is the maximizing argument to compute the convex conjugate:

$f^\prime(x) = x^*(x):= \arg\sup_{x^\star} \langle x, x^\star\rangle -f^\star(x^\star)$ and

$f^{\star\prime}(x^\star) = x(x^\star):= \arg\sup_x \langle x, x^\star\rangle-f(x);$

whence

$x= \nabla f^{\star}(\nabla f(x)),$

$x^\star= \nabla f(\nabla f^{\star}(x^\star)),$

and moreover

$f^{\prime\prime}(x) \cdot f^{\star\prime\prime} (x^\star(x))=1,$

$f^{\star\prime\prime}(x^\star)\cdot f^{\prime\prime}(x(x^\star))=1.$

Scaling properties

If, for some $\beta>0$ , $\,g(x)=\alpha+ \beta \cdot f(\gamma x+\delta)$ , then

$g^\star(x^\star)= -\alpha- \frac\delta\gamma x^\star +\beta \cdot f^\star \left(\frac {x^\star}{\beta \gamma}\right).$

In case of an additional parameter (α, say) moreover

$f_\alpha(x)=-f_\alpha(\tilde x),$

where $\tilde x$ is chosen to be the maximizing argument.

Behavior under linear transformations

Let A be a bounded linear operator from X to Y. For any convex function f on X, one has

$\left(A f\right)^\star = f^\star A^\star$

where

$(A f)(y) = \inf\{ f(x) : x \in X , A x = y \}$

is the preimage of f w.r.t. A and A^* is the adjoint operator of A.^[1]

A closed convex function f is symmetric with respect to a given set G of orthogonal linear transformations,

$f\left(A x\right) = f(x), \; \forall x, \; \forall A \in G$

if and only if its convex conjugate f^* is symmetric with respect to G.

Infimal convolution

The infimal convolution of two functions f and g is defined as

$\left(f \star_\inf g\right)(x) = \inf \left \{ f(x-y) + g(y) \, | \, y \in \mathbb{R}^n \right \}.$

Let f₁, …, f_m be proper convex functions on Rⁿ. Then

$\left( f_1 \star_\inf \cdots \star_\inf f_m \right)^\star = f_1^\star + \cdots + f_m^\star.$

The infimal convolution of two functions has a geometric interpretation: The (strict) epigraph of the infimal convolution of two functions is the Minkowski sum of the (strict) epigraphs of those functions.^[2]

Table of selected convex conjugates

The following table provides Legendre transforms for many common functions as well as a few useful properties.^[3]

$f(x)$	$\operatorname{dom}(f)$	$f^(x^)$	$\operatorname{dom}(f^*)$
$f(ax)$ (where $a \neq 0$ )	$X$	$f^\left(\frac{x^}{a}\right)$	$X^*$
$f(x + b)$	$X$	$f^(x^) - \langle b,x^* \rangle$	$X^*$
$a f(x)$ (where $a > 0$ )	$X$	$a f^\left(\frac{x^}{a}\right)$	$X^*$
$\alpha+ \beta \cdot f(\gamma x+\delta)$	$X$	$-\alpha- \frac\delta\gamma x^\star +\beta \cdot f^\star \left(\frac {x^\star}{\beta \gamma}\right)\quad (\beta>0)$	$X^*$
$\frac{\|x\|^p}{p}$ (where $p > 1$ )	$\mathbb{R}$	$\frac{\|x^*\|^q}{q}$ (where $\frac{1}{p} + \frac{1}{q} = 1$ )	$\mathbb{R}$
$\frac{-x^p}{p}$ (where $0 < p < 1$ )	$\mathbb{R}_+$	$\frac{-(-x^*)^q}q$ (where $\frac 1 p + \frac 1 q = 1$ )	$\mathbb{R}_{--}$
$\sqrt{1 + x^2}$	$\mathbb{R}$	$-\sqrt{1 - (x^*)^2}$	$[-1,1]$
$-\log(x)$	$\mathbb{R}_{++}$	$-(1 + \log(-x^*))$	$\mathbb{R}_{--}$
$e^x$	$\mathbb{R}$	$\begin{cases}x^* \log(x^) - x^ & \text{if }x^* > 0\\ 0 & \text{if }x^* = 0\end{cases}$	$\mathbb{R}_{+}$
$\log\left(1 + e^x\right)$	$\mathbb{R}$	$\begin{cases}x^* \log(x^) + (1 - x^) \log(1 - x^) & \text{if }0 < x^ < 1\\ 0 & \text{if }x^* = 0,1\end{cases}$	$[0,1]$
$-\log\left(1 - e^x\right)$	$\mathbb{R}$	$\begin{cases}x^* \log(x^) - (1 + x^) \log(1 + x^) & \text{if }x^ > 0\\ 0 & \text{if }x^* = 0\end{cases}$	$\mathbb{R}_+$

References

^ Ioffe, A.D. and Tichomirov, V.M. (1979), Theorie der Extremalaufgaben. Deutscher Verlag der Wissenschaften. Satz 3.4.3
^ Bauschke, Heinz H.; Goebel, Rafal; Lucet, Yves; Wang, Xianfu (2008). "The Proximal Average: Basic Theory". SIAM Journal on Optimization 19 (2): 766. DOI:10.1137/070687542.
^ Borwein, Jonathan; Lewis, Adrian (2006). Convex Analysis and Nonlinear Optimization: Theory and Examples (2 ed.). Springer. pp. 50–51. ISBN 978-0-387-29570-1.

Arnol'd, Vladimir Igorevich (1989). Mathematical Methods of Classical Mechanics (Second ed.). Springer. ISBN 0-387-96890-3. MR 997295.
Rockafellar, R. Tyrell (1970). Convex Analysis. Princeton: Princeton University Press. ISBN 0-691-01586-4. MR 0274683.

External links

Touchette, Hugo (2005-07-27). "Legendre-Fenchel transforms in a nutshell" (PDF). http://www.maths.qmw.ac.uk/~ht/archive/lfth2.pdf. Retrieved 2007-07-24. ^{[dead link]}

Touchette, Hugo (2006-11-21). "Elements of convex analysis" (PDF). http://www.maths.qmul.ac.uk/~ht/archive/convex1.pdf. Retrieved 2008-03-26.

Arabic (ar)	Bulgarian (bg)	Chinese (zh)	Czech (cs)
Danish (da)	Dutch (nl)	English (en)	Estonian (et)
Finnish (fi)	French (fr)	German (de)	Greek (el)
Hungarian (hu)	Italian (it)	Japanese (ja)	Korean (ko)
Norwegian (no)	Polish (pl)	Portuguese (pt)	Romanian (ro)
Russian (ru)	Serbian (sr)	Slovenian (sl)	Spanish (es)
Swedish (sv)	Turkish (tr)	Latvian (lv)	Persian (fa)
Lithuanian (lt)	Icelandic (is)	Slovak (sk)	Croatian (hr)
Hindi (hi)	Vietnamese (vi)	Hebrew (he)	Thai (th)
Indonesian (id)

Arabic (ar)	Bulgarian (bg)	Chinese (zh)	Czech (cs)
Danish (da)	Dutch (nl)	English (en)	Estonian (et)
Finnish (fi)	French (fr)	German (de)	Greek (el)
Hungarian (hu)	Italian (it)	Japanese (ja)	Korean (ko)
Norwegian (no)	Polish (pl)	Portuguese (pt)	Romanian (ro)
Russian (ru)	Serbian (sr)	Slovenian (sl)	Spanish (es)
Swedish (sv)	Turkish (tr)	Latvian (lv)	Persian (fa)
Lithuanian (lt)	Icelandic (is)	Slovak (sk)	Croatian (hr)
Hindi (hi)	Vietnamese (vi)	Hebrew (he)	Thai (th)
Indonesian (id)

Wikipedia

Convex conjugate

Contents