What is $frac{partial}{partial d_A} f$ where $d_A$ is matrix $A$'s diagonal?
up vote
0
down vote
favorite
Let us assume that we know $frac{partial}{partial A} f$, where $f$ is a scalar function, and $A$ any matrix.
Now suppose we are interested in the special case when $A$ is diagonal, and we want to know what's $$frac{partial}{partial d_A} f$$ where $d_A$ is matrix $A$'s diagonal.
Also, what would be $frac{partial}{partial d_A} A$?
matrix-calculus
asked Nov 17 at 14:15
An old man in the sea.
1,60411031
add a comment |
up vote
0
down vote
favorite
Let us assume that we know $frac{partial}{partial A} f$, where $f$ is a scalar function, and $A$ any matrix.
Now suppose we are interested in the special case when $A$ is diagonal, and we want to know what's $$frac{partial}{partial d_A} f$$ where $d_A$ is matrix $A$'s diagonal.
Also, what would be $frac{partial}{partial d_A} A$?
matrix-calculus
asked Nov 17 at 14:15
An old man in the sea.
1,60411031
1
It's totally unclear what you have in mind. Give a more detailed description of the envisaged mathematical situation.
– Christian Blatter
Nov 17 at 14:55
you cannot define the differential for just diagonal matrices, if not for the variable of $f$. In any case a matrix is isomorphic to a vector space so you can apply the multivariable derivative
– Masacroso
Nov 17 at 15:00
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
Let us assume that we know $frac{partial}{partial A} f$, where $f$ is a scalar function, and $A$ any matrix.
Now suppose we are interested in the special case when $A$ is diagonal, and we want to know what's $$frac{partial}{partial d_A} f$$ where $d_A$ is matrix $A$'s diagonal.
Also, what would be $frac{partial}{partial d_A} A$?
matrix-calculus
asked Nov 17 at 14:15
An old man in the sea.
1,60411031
Let us assume that we know $frac{partial}{partial A} f$, where $f$ is a scalar function, and $A$ any matrix.
Now suppose we are interested in the special case when $A$ is diagonal, and we want to know what's $$frac{partial}{partial d_A} f$$ where $d_A$ is matrix $A$'s diagonal.
Also, what would be $frac{partial}{partial d_A} A$?
matrix-calculus
matrix-calculus
asked Nov 17 at 14:15
An old man in the sea.
1,60411031
asked Nov 17 at 14:15
An old man in the sea.
1,60411031
asked Nov 17 at 14:15
An old man in the sea.
1,60411031
asked Nov 17 at 14:15
An old man in the sea.
1,60411031
asked Nov 17 at 14:15
An old man in the sea.
1,60411031
1,60411031
1
It's totally unclear what you have in mind. Give a more detailed description of the envisaged mathematical situation.
– Christian Blatter
Nov 17 at 14:55
you cannot define the differential for just diagonal matrices, if not for the variable of $f$. In any case a matrix is isomorphic to a vector space so you can apply the multivariable derivative
– Masacroso
Nov 17 at 15:00
add a comment |
1
It's totally unclear what you have in mind. Give a more detailed description of the envisaged mathematical situation.
– Christian Blatter
Nov 17 at 14:55
you cannot define the differential for just diagonal matrices, if not for the variable of $f$. In any case a matrix is isomorphic to a vector space so you can apply the multivariable derivative
– Masacroso
Nov 17 at 15:00
1
1
It's totally unclear what you have in mind. Give a more detailed description of the envisaged mathematical situation.
– Christian Blatter
Nov 17 at 14:55
It's totally unclear what you have in mind. Give a more detailed description of the envisaged mathematical situation.
– Christian Blatter
Nov 17 at 14:55
you cannot define the differential for just diagonal matrices, if not for the variable of $f$. In any case a matrix is isomorphic to a vector space so you can apply the multivariable derivative
– Masacroso
Nov 17 at 15:00
you cannot define the differential for just diagonal matrices, if not for the variable of $f$. In any case a matrix is isomorphic to a vector space so you can apply the multivariable derivative
– Masacroso
Nov 17 at 15:00
add a comment |
1 Answer
1
active
oldest
votes
up vote
2
down vote
accepted
You have calculated the gradient of the function $f(A)$
$$G=frac{partial f}{partial A}$$
with no constraints on the matrix variable, and now you wish to constrain $A$ to be diagonal, i.e.
$$A={rm Diag}(a)$$
Start with a differential in terms of $dA$, then change the variable to $da$
$$eqalign{
df &= G:dA cr
&= G:{rm Diag}(da) cr
&= {rm diag}(G):da cr
frac{partial f}{partial a}
&= {rm diag}(G) cr
&= {rm diag}Big(frac{partial f}{partial A}Big) cr
}$$
where
$,,,:,,$ is a product notation for the trace $,,A:B={rm Tr}(A^TB)$
$,,,{rm Diag}()$ generates a diagonal matrix from the input vector
${,,,rm diag}()$ extracts the diagonal of a matrix into an output vector
answered Nov 17 at 15:32
lynn
1,746177
Lynn, thanks for the answer. The result it's what I expected it to be, but I'm not sure the equality $G:Diag(da)=diag(G):da$? is Diag ->diagonal matrix generated from a vector and diag ->main diagonal of matrix ? ?
– An old man in the sea.
Nov 17 at 17:08
Decompose the gradient into a sum of its diagonal and off-diagonal parts. Only the diagonal part will contribute to the above trace product; the product with the off-diagonal part is zero.
– lynn
Nov 18 at 18:07
Lynn, many thanks ;)
– An old man in the sea.
Nov 18 at 18:24
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
You have calculated the gradient of the function $f(A)$
$$G=frac{partial f}{partial A}$$
with no constraints on the matrix variable, and now you wish to constrain $A$ to be diagonal, i.e.
$$A={rm Diag}(a)$$
Start with a differential in terms of $dA$, then change the variable to $da$
$$eqalign{
df &= G:dA cr
&= G:{rm Diag}(da) cr
&= {rm diag}(G):da cr
frac{partial f}{partial a}
&= {rm diag}(G) cr
&= {rm diag}Big(frac{partial f}{partial A}Big) cr
}$$
where
$,,,:,,$ is a product notation for the trace $,,A:B={rm Tr}(A^TB)$
$,,,{rm Diag}()$ generates a diagonal matrix from the input vector
${,,,rm diag}()$ extracts the diagonal of a matrix into an output vector
answered Nov 17 at 15:32
lynn
1,746177
Lynn, thanks for the answer. The result it's what I expected it to be, but I'm not sure the equality $G:Diag(da)=diag(G):da$? is Diag ->diagonal matrix generated from a vector and diag ->main diagonal of matrix ? ?
– An old man in the sea.
Nov 17 at 17:08
Decompose the gradient into a sum of its diagonal and off-diagonal parts. Only the diagonal part will contribute to the above trace product; the product with the off-diagonal part is zero.
– lynn
Nov 18 at 18:07
Lynn, many thanks ;)
– An old man in the sea.
Nov 18 at 18:24
add a comment |
up vote
2
down vote
accepted
You have calculated the gradient of the function $f(A)$
$$G=frac{partial f}{partial A}$$
with no constraints on the matrix variable, and now you wish to constrain $A$ to be diagonal, i.e.
$$A={rm Diag}(a)$$
Start with a differential in terms of $dA$, then change the variable to $da$
$$eqalign{
df &= G:dA cr
&= G:{rm Diag}(da) cr
&= {rm diag}(G):da cr
frac{partial f}{partial a}
&= {rm diag}(G) cr
&= {rm diag}Big(frac{partial f}{partial A}Big) cr
}$$
where
$,,,:,,$ is a product notation for the trace $,,A:B={rm Tr}(A^TB)$
$,,,{rm Diag}()$ generates a diagonal matrix from the input vector
${,,,rm diag}()$ extracts the diagonal of a matrix into an output vector
answered Nov 17 at 15:32
lynn
1,746177
Lynn, thanks for the answer. The result it's what I expected it to be, but I'm not sure the equality $G:Diag(da)=diag(G):da$? is Diag ->diagonal matrix generated from a vector and diag ->main diagonal of matrix ? ?
– An old man in the sea.
Nov 17 at 17:08
Decompose the gradient into a sum of its diagonal and off-diagonal parts. Only the diagonal part will contribute to the above trace product; the product with the off-diagonal part is zero.
– lynn
Nov 18 at 18:07
Lynn, many thanks ;)
– An old man in the sea.
Nov 18 at 18:24
add a comment |
up vote
2
down vote
accepted
up vote
2
down vote
accepted
You have calculated the gradient of the function $f(A)$
$$G=frac{partial f}{partial A}$$
with no constraints on the matrix variable, and now you wish to constrain $A$ to be diagonal, i.e.
$$A={rm Diag}(a)$$
Start with a differential in terms of $dA$, then change the variable to $da$
$$eqalign{
df &= G:dA cr
&= G:{rm Diag}(da) cr
&= {rm diag}(G):da cr
frac{partial f}{partial a}
&= {rm diag}(G) cr
&= {rm diag}Big(frac{partial f}{partial A}Big) cr
}$$
where
$,,,:,,$ is a product notation for the trace $,,A:B={rm Tr}(A^TB)$
$,,,{rm Diag}()$ generates a diagonal matrix from the input vector
${,,,rm diag}()$ extracts the diagonal of a matrix into an output vector
answered Nov 17 at 15:32
lynn
1,746177
You have calculated the gradient of the function $f(A)$
$$G=frac{partial f}{partial A}$$
with no constraints on the matrix variable, and now you wish to constrain $A$ to be diagonal, i.e.
$$A={rm Diag}(a)$$
Start with a differential in terms of $dA$, then change the variable to $da$
$$eqalign{
df &= G:dA cr
&= G:{rm Diag}(da) cr
&= {rm diag}(G):da cr
frac{partial f}{partial a}
&= {rm diag}(G) cr
&= {rm diag}Big(frac{partial f}{partial A}Big) cr
}$$
where
$,,,:,,$ is a product notation for the trace $,,A:B={rm Tr}(A^TB)$
$,,,{rm Diag}()$ generates a diagonal matrix from the input vector
${,,,rm diag}()$ extracts the diagonal of a matrix into an output vector
answered Nov 17 at 15:32
lynn
1,746177
edited Nov 17 at 16:01
answered Nov 17 at 15:32
lynn
1,746177
answered Nov 17 at 15:32
lynn
1,746177
answered Nov 17 at 15:32
lynn
1,746177
1,746177
Lynn, thanks for the answer. The result it's what I expected it to be, but I'm not sure the equality $G:Diag(da)=diag(G):da$? is Diag ->diagonal matrix generated from a vector and diag ->main diagonal of matrix ? ?
– An old man in the sea.
Nov 17 at 17:08
Decompose the gradient into a sum of its diagonal and off-diagonal parts. Only the diagonal part will contribute to the above trace product; the product with the off-diagonal part is zero.
– lynn
Nov 18 at 18:07
Lynn, many thanks ;)
– An old man in the sea.
Nov 18 at 18:24
add a comment |
Lynn, thanks for the answer. The result it's what I expected it to be, but I'm not sure the equality $G:Diag(da)=diag(G):da$? is Diag ->diagonal matrix generated from a vector and diag ->main diagonal of matrix ? ?
– An old man in the sea.
Nov 17 at 17:08
Decompose the gradient into a sum of its diagonal and off-diagonal parts. Only the diagonal part will contribute to the above trace product; the product with the off-diagonal part is zero.
– lynn
Nov 18 at 18:07
Lynn, many thanks ;)
– An old man in the sea.
Nov 18 at 18:24
Lynn, thanks for the answer. The result it's what I expected it to be, but I'm not sure the equality $G:Diag(da)=diag(G):da$? is Diag ->diagonal matrix generated from a vector and diag ->main diagonal of matrix ? ?
– An old man in the sea.
Nov 17 at 17:08
Lynn, thanks for the answer. The result it's what I expected it to be, but I'm not sure the equality $G:Diag(da)=diag(G):da$? is Diag ->diagonal matrix generated from a vector and diag ->main diagonal of matrix ? ?
– An old man in the sea.
Nov 17 at 17:08
Decompose the gradient into a sum of its diagonal and off-diagonal parts. Only the diagonal part will contribute to the above trace product; the product with the off-diagonal part is zero.
– lynn
Nov 18 at 18:07
Decompose the gradient into a sum of its diagonal and off-diagonal parts. Only the diagonal part will contribute to the above trace product; the product with the off-diagonal part is zero.
– lynn
Nov 18 at 18:07
Lynn, many thanks ;)
– An old man in the sea.
Nov 18 at 18:24
Lynn, many thanks ;)
– An old man in the sea.
Nov 18 at 18:24
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3002403%2fwhat-is-frac-partial-partial-d-a-f-where-d-a-is-matrix-as-diagonal%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
It's totally unclear what you have in mind. Give a more detailed description of the envisaged mathematical situation.
– Christian Blatter
Nov 17 at 14:55
you cannot define the differential for just diagonal matrices, if not for the variable of $f$. In any case a matrix is isomorphic to a vector space so you can apply the multivariable derivative
– Masacroso
Nov 17 at 15:00