Proving Pinsker's inequality
up vote
1
down vote
favorite
I wanna show that for $a$ and $b >0$ and $<1$. Then
$$2(a-b)^2leq (a) log (a) + (1-a)log(1-a) -(a)log(b)-(1-a)log(1-b).$$
If I wanna go from left to right, then I think I gotta use Taylor's but I don't know how. Also, the right hand side I figured out is equal to
$$(a)logfrac ab+(1-a)logfrac{1-a}{1-b}.$$
real-analysis inequality logarithms taylor-expansion
add a comment |
up vote
1
down vote
favorite
I wanna show that for $a$ and $b >0$ and $<1$. Then
$$2(a-b)^2leq (a) log (a) + (1-a)log(1-a) -(a)log(b)-(1-a)log(1-b).$$
If I wanna go from left to right, then I think I gotta use Taylor's but I don't know how. Also, the right hand side I figured out is equal to
$$(a)logfrac ab+(1-a)logfrac{1-a}{1-b}.$$
real-analysis inequality logarithms taylor-expansion
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I wanna show that for $a$ and $b >0$ and $<1$. Then
$$2(a-b)^2leq (a) log (a) + (1-a)log(1-a) -(a)log(b)-(1-a)log(1-b).$$
If I wanna go from left to right, then I think I gotta use Taylor's but I don't know how. Also, the right hand side I figured out is equal to
$$(a)logfrac ab+(1-a)logfrac{1-a}{1-b}.$$
real-analysis inequality logarithms taylor-expansion
I wanna show that for $a$ and $b >0$ and $<1$. Then
$$2(a-b)^2leq (a) log (a) + (1-a)log(1-a) -(a)log(b)-(1-a)log(1-b).$$
If I wanna go from left to right, then I think I gotta use Taylor's but I don't know how. Also, the right hand side I figured out is equal to
$$(a)logfrac ab+(1-a)logfrac{1-a}{1-b}.$$
real-analysis inequality logarithms taylor-expansion
real-analysis inequality logarithms taylor-expansion
edited Nov 16 at 18:32
Angelo Lucia
578213
578213
asked Nov 15 at 0:16
Jack
1017
1017
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
up vote
4
down vote
accepted
As already mentioned, this is a special case of Pinsker's inequality.
The following proof for this special case is taken from Pinsker’s inequality and its applications to lower bounds:
For fixed $a in (0, 1)$ define $f:(0, 1) to Bbb R$ as
$$
f(b) = alogfrac ab+(1-a)logfrac{1-a}{1-b} - 2(a-b)^2 , .
$$
Then
$$
f'(b) = -frac ab + frac{1-a}{1-b} + 4(a-b) = (b-a) left(frac{1}{b(1-b)} - 4right) , .
$$
The second factor is $ge 0$, with equality only at a single point ($b= frac 12$), so that $f$ is strictly decreasing on $(0, a]$ and
strictly increasing on $[a, 1)$. It follows that
$$
f(b) ge f(a) = 0 , ,
$$
which is the desired inequality. Equality holds only if and only if $b=a$.
add a comment |
up vote
3
down vote
You are trying to prove Pinsker's inequality. Since both $a$ and $b$ are between 0 and 1, we can think of $(a,1-a)$ and $(b,1-b)$ as binary probability distributions. Let me call them $P$ and $Q$ respectively, so that $P(0)=a$, $P(1)=1-a$, $Q(0)=b$ and $Q(1)=1-b$.
Now it is easy to see that your left hand side is $2delta(P,Q)^2$, where $delta$ is the total variation distance between $P$ and $Q$, while the right hand side is the relative entropy, or Kullback-Leibler divergence, $D(P||Q)$.
You can find the proof of Pinsker's inequality in many places, for example here: Proof of Pinsker's inequality.
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
accepted
As already mentioned, this is a special case of Pinsker's inequality.
The following proof for this special case is taken from Pinsker’s inequality and its applications to lower bounds:
For fixed $a in (0, 1)$ define $f:(0, 1) to Bbb R$ as
$$
f(b) = alogfrac ab+(1-a)logfrac{1-a}{1-b} - 2(a-b)^2 , .
$$
Then
$$
f'(b) = -frac ab + frac{1-a}{1-b} + 4(a-b) = (b-a) left(frac{1}{b(1-b)} - 4right) , .
$$
The second factor is $ge 0$, with equality only at a single point ($b= frac 12$), so that $f$ is strictly decreasing on $(0, a]$ and
strictly increasing on $[a, 1)$. It follows that
$$
f(b) ge f(a) = 0 , ,
$$
which is the desired inequality. Equality holds only if and only if $b=a$.
add a comment |
up vote
4
down vote
accepted
As already mentioned, this is a special case of Pinsker's inequality.
The following proof for this special case is taken from Pinsker’s inequality and its applications to lower bounds:
For fixed $a in (0, 1)$ define $f:(0, 1) to Bbb R$ as
$$
f(b) = alogfrac ab+(1-a)logfrac{1-a}{1-b} - 2(a-b)^2 , .
$$
Then
$$
f'(b) = -frac ab + frac{1-a}{1-b} + 4(a-b) = (b-a) left(frac{1}{b(1-b)} - 4right) , .
$$
The second factor is $ge 0$, with equality only at a single point ($b= frac 12$), so that $f$ is strictly decreasing on $(0, a]$ and
strictly increasing on $[a, 1)$. It follows that
$$
f(b) ge f(a) = 0 , ,
$$
which is the desired inequality. Equality holds only if and only if $b=a$.
add a comment |
up vote
4
down vote
accepted
up vote
4
down vote
accepted
As already mentioned, this is a special case of Pinsker's inequality.
The following proof for this special case is taken from Pinsker’s inequality and its applications to lower bounds:
For fixed $a in (0, 1)$ define $f:(0, 1) to Bbb R$ as
$$
f(b) = alogfrac ab+(1-a)logfrac{1-a}{1-b} - 2(a-b)^2 , .
$$
Then
$$
f'(b) = -frac ab + frac{1-a}{1-b} + 4(a-b) = (b-a) left(frac{1}{b(1-b)} - 4right) , .
$$
The second factor is $ge 0$, with equality only at a single point ($b= frac 12$), so that $f$ is strictly decreasing on $(0, a]$ and
strictly increasing on $[a, 1)$. It follows that
$$
f(b) ge f(a) = 0 , ,
$$
which is the desired inequality. Equality holds only if and only if $b=a$.
As already mentioned, this is a special case of Pinsker's inequality.
The following proof for this special case is taken from Pinsker’s inequality and its applications to lower bounds:
For fixed $a in (0, 1)$ define $f:(0, 1) to Bbb R$ as
$$
f(b) = alogfrac ab+(1-a)logfrac{1-a}{1-b} - 2(a-b)^2 , .
$$
Then
$$
f'(b) = -frac ab + frac{1-a}{1-b} + 4(a-b) = (b-a) left(frac{1}{b(1-b)} - 4right) , .
$$
The second factor is $ge 0$, with equality only at a single point ($b= frac 12$), so that $f$ is strictly decreasing on $(0, a]$ and
strictly increasing on $[a, 1)$. It follows that
$$
f(b) ge f(a) = 0 , ,
$$
which is the desired inequality. Equality holds only if and only if $b=a$.
edited Nov 18 at 10:26
answered Nov 17 at 17:34
Martin R
26.1k33046
26.1k33046
add a comment |
add a comment |
up vote
3
down vote
You are trying to prove Pinsker's inequality. Since both $a$ and $b$ are between 0 and 1, we can think of $(a,1-a)$ and $(b,1-b)$ as binary probability distributions. Let me call them $P$ and $Q$ respectively, so that $P(0)=a$, $P(1)=1-a$, $Q(0)=b$ and $Q(1)=1-b$.
Now it is easy to see that your left hand side is $2delta(P,Q)^2$, where $delta$ is the total variation distance between $P$ and $Q$, while the right hand side is the relative entropy, or Kullback-Leibler divergence, $D(P||Q)$.
You can find the proof of Pinsker's inequality in many places, for example here: Proof of Pinsker's inequality.
add a comment |
up vote
3
down vote
You are trying to prove Pinsker's inequality. Since both $a$ and $b$ are between 0 and 1, we can think of $(a,1-a)$ and $(b,1-b)$ as binary probability distributions. Let me call them $P$ and $Q$ respectively, so that $P(0)=a$, $P(1)=1-a$, $Q(0)=b$ and $Q(1)=1-b$.
Now it is easy to see that your left hand side is $2delta(P,Q)^2$, where $delta$ is the total variation distance between $P$ and $Q$, while the right hand side is the relative entropy, or Kullback-Leibler divergence, $D(P||Q)$.
You can find the proof of Pinsker's inequality in many places, for example here: Proof of Pinsker's inequality.
add a comment |
up vote
3
down vote
up vote
3
down vote
You are trying to prove Pinsker's inequality. Since both $a$ and $b$ are between 0 and 1, we can think of $(a,1-a)$ and $(b,1-b)$ as binary probability distributions. Let me call them $P$ and $Q$ respectively, so that $P(0)=a$, $P(1)=1-a$, $Q(0)=b$ and $Q(1)=1-b$.
Now it is easy to see that your left hand side is $2delta(P,Q)^2$, where $delta$ is the total variation distance between $P$ and $Q$, while the right hand side is the relative entropy, or Kullback-Leibler divergence, $D(P||Q)$.
You can find the proof of Pinsker's inequality in many places, for example here: Proof of Pinsker's inequality.
You are trying to prove Pinsker's inequality. Since both $a$ and $b$ are between 0 and 1, we can think of $(a,1-a)$ and $(b,1-b)$ as binary probability distributions. Let me call them $P$ and $Q$ respectively, so that $P(0)=a$, $P(1)=1-a$, $Q(0)=b$ and $Q(1)=1-b$.
Now it is easy to see that your left hand side is $2delta(P,Q)^2$, where $delta$ is the total variation distance between $P$ and $Q$, while the right hand side is the relative entropy, or Kullback-Leibler divergence, $D(P||Q)$.
You can find the proof of Pinsker's inequality in many places, for example here: Proof of Pinsker's inequality.
answered Nov 15 at 0:54
Angelo Lucia
578213
578213
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2999007%2fproving-pinskers-inequality%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown