Is there a constraint on the sum of the type-I & type II error probabilities?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty{ margin-bottom:0;
}
up vote
3
down vote
favorite
Is it true that if $H_0$ and $H_a$ are complementary hypotheses of the Binomial trial, i.e., the negation of $H_0$ is $H_a$ then the type-I error $alpha$ plus type-II error $beta$ equals 1? Or is that sum always less then 1, or can it be sometimes even greater then 1?
hypothesis-testing statistical-significance mathematical-statistics p-value type-i-and-ii-errors
|
show 4 more comments
up vote
3
down vote
favorite
Is it true that if $H_0$ and $H_a$ are complementary hypotheses of the Binomial trial, i.e., the negation of $H_0$ is $H_a$ then the type-I error $alpha$ plus type-II error $beta$ equals 1? Or is that sum always less then 1, or can it be sometimes even greater then 1?
hypothesis-testing statistical-significance mathematical-statistics p-value type-i-and-ii-errors
If you mean probability of the errors, then no. Type 1 error and Type 2 error are not complementary events in general.
– StubbornAtom
Nov 29 at 14:51
OK. Is that sum always less then 1, or can it be sometimes even greater then 1?
– user2925716
Nov 29 at 14:52
Please mention 'that sum' of the probabilities if that's what you mean, instead of the sum of the two errors.
– StubbornAtom
Nov 29 at 14:59
I really mean $alpha+beta$ as usually denoted these two errors. $alpha$ is the probability that we reject $H_0$ if $H_0$ is true and the other one accept $H_0$ if $H_1$ is true.
– user2925716
Nov 29 at 15:02
Okay. But that's not what your post says - that's what I am saying.
– StubbornAtom
Nov 29 at 15:04
|
show 4 more comments
up vote
3
down vote
favorite
up vote
3
down vote
favorite
Is it true that if $H_0$ and $H_a$ are complementary hypotheses of the Binomial trial, i.e., the negation of $H_0$ is $H_a$ then the type-I error $alpha$ plus type-II error $beta$ equals 1? Or is that sum always less then 1, or can it be sometimes even greater then 1?
hypothesis-testing statistical-significance mathematical-statistics p-value type-i-and-ii-errors
Is it true that if $H_0$ and $H_a$ are complementary hypotheses of the Binomial trial, i.e., the negation of $H_0$ is $H_a$ then the type-I error $alpha$ plus type-II error $beta$ equals 1? Or is that sum always less then 1, or can it be sometimes even greater then 1?
hypothesis-testing statistical-significance mathematical-statistics p-value type-i-and-ii-errors
hypothesis-testing statistical-significance mathematical-statistics p-value type-i-and-ii-errors
edited Nov 29 at 15:47
gung♦
105k34255517
105k34255517
asked Nov 29 at 14:44
user2925716
1335
1335
If you mean probability of the errors, then no. Type 1 error and Type 2 error are not complementary events in general.
– StubbornAtom
Nov 29 at 14:51
OK. Is that sum always less then 1, or can it be sometimes even greater then 1?
– user2925716
Nov 29 at 14:52
Please mention 'that sum' of the probabilities if that's what you mean, instead of the sum of the two errors.
– StubbornAtom
Nov 29 at 14:59
I really mean $alpha+beta$ as usually denoted these two errors. $alpha$ is the probability that we reject $H_0$ if $H_0$ is true and the other one accept $H_0$ if $H_1$ is true.
– user2925716
Nov 29 at 15:02
Okay. But that's not what your post says - that's what I am saying.
– StubbornAtom
Nov 29 at 15:04
|
show 4 more comments
If you mean probability of the errors, then no. Type 1 error and Type 2 error are not complementary events in general.
– StubbornAtom
Nov 29 at 14:51
OK. Is that sum always less then 1, or can it be sometimes even greater then 1?
– user2925716
Nov 29 at 14:52
Please mention 'that sum' of the probabilities if that's what you mean, instead of the sum of the two errors.
– StubbornAtom
Nov 29 at 14:59
I really mean $alpha+beta$ as usually denoted these two errors. $alpha$ is the probability that we reject $H_0$ if $H_0$ is true and the other one accept $H_0$ if $H_1$ is true.
– user2925716
Nov 29 at 15:02
Okay. But that's not what your post says - that's what I am saying.
– StubbornAtom
Nov 29 at 15:04
If you mean probability of the errors, then no. Type 1 error and Type 2 error are not complementary events in general.
– StubbornAtom
Nov 29 at 14:51
If you mean probability of the errors, then no. Type 1 error and Type 2 error are not complementary events in general.
– StubbornAtom
Nov 29 at 14:51
OK. Is that sum always less then 1, or can it be sometimes even greater then 1?
– user2925716
Nov 29 at 14:52
OK. Is that sum always less then 1, or can it be sometimes even greater then 1?
– user2925716
Nov 29 at 14:52
Please mention 'that sum' of the probabilities if that's what you mean, instead of the sum of the two errors.
– StubbornAtom
Nov 29 at 14:59
Please mention 'that sum' of the probabilities if that's what you mean, instead of the sum of the two errors.
– StubbornAtom
Nov 29 at 14:59
I really mean $alpha+beta$ as usually denoted these two errors. $alpha$ is the probability that we reject $H_0$ if $H_0$ is true and the other one accept $H_0$ if $H_1$ is true.
– user2925716
Nov 29 at 15:02
I really mean $alpha+beta$ as usually denoted these two errors. $alpha$ is the probability that we reject $H_0$ if $H_0$ is true and the other one accept $H_0$ if $H_1$ is true.
– user2925716
Nov 29 at 15:02
Okay. But that's not what your post says - that's what I am saying.
– StubbornAtom
Nov 29 at 15:04
Okay. But that's not what your post says - that's what I am saying.
– StubbornAtom
Nov 29 at 15:04
|
show 4 more comments
3 Answers
3
active
oldest
votes
up vote
4
down vote
accepted
For an arbitrarily chosen decision rule -- meaning that the decision rule can be anything that you make up just for the heck of it, it doesn't need to be sensible in any sense of the word -- the arithmetic sum of the Type I and Type II error probabilities can be any number in $[0,2]$ as the answer by @Bjorn points out.
Example 1: The observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $0$ and so their sum is also $0$.
Example 2: As in Example 1, the observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. But now the decision rule is
$$X begin{array}{c} H_a\gtrless\{H_0}end{array} 0$$
which is exactly bass ackwards from the decision rule in Example 1 (Hey, I said upfront that we are going to consider arbitrary decision rules, not necessarily only sensible ones!!). Now, since the OP asks in a comment for an explanation of my assertion (in a previous version of this answer) that "both the Type I and Type II error probabilities have value $1$ and so their sum is $2$.", here goes.
If the null hypothesis is true, then in our model, the observation $X$ is always a positive number. But the decision rule is that whenever the observation has positive value, we are going to reject the null hypothesis. Continuing to remember that the null hypothesis is true, we see that our decision rule tells us to always reject the null hypothesis (when it is true). So, what is the probability that we reject the null hypothesis when in fact the null hypothesis is true? $100%$, right? A similar argument applies to the Type II error probability. If the null hypothesis is actually false, the observation must have negative value in our model. But the decision rule perversely insists that we must refuse to reject the null when the observation has negative value which happens only when the null hypothesis is false. So, the probability of failing to reject the null hypothesis when in fact the null hypothesis is false (which is what a Type II error is) must be $100%$ too, right?
Thus, both the Type I error probability and the Type II error probability have value 1 for this (admittedly contrived) example of a decision rule, and so their arithmetic sum must be 2.
Hopefully, the above is enough of a "basic computation of this fact" as the OP desires, or is necessary to resort to the Peano axioms to prove that $1+1=2$?
Example 3: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $frac{2}{3}$ and so their sum is $frac 43$.
Example 4: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. But now the decision rule is
$$X begin{array}{c}H_a\gtrless\{H_0}end{array} 0$$
which is more sensible than the decision rule in Example 3, and it leads to both the Type I and Type II error probabilities having value $frac{1}{3}$ and so their sum is $frac 23 < 1$.
So why @AdamO says: what's wrong with his observation? > P(decomposing corpse | dead) + P(looking alright | alive) > 1 Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
– user2925716
Nov 29 at 18:01
3
@user2925716 You are the one interested in asking what the arithmetic sum of the Type I and Type II error probabilities is. This arithmetic sum has no meaning at all in decision making; it is a weighted sum of the two error probabilities that is of interest in Bayesian decision making, and so AdamO is quite correct in his assertion that the fact that sum that you are interested in can have value greater than $1$ is neither surprising nor interesting.
– Dilip Sarwate
Nov 29 at 18:11
Yes, but what about his argument that one of these two probabilities is always zero so it never can be the case that that sum is larger than $1$ ? In fact, I do not understand why your example yields number $2$, can you explain this to a beginner (me)?
– user2925716
Nov 29 at 18:23
@DilipSarwate: Strictly, if you're going into Bayesianism, then you don't have "probability of Type I/II error" at all, because those terms presuppose a Frequentist approach. But then OP's question is meaningless in the first instance, so I fear we have to stay on the Frequentist side of the fence.
– Kevin
Nov 29 at 18:23
2
@user2925716: AdamO is not claiming that "either $alpha =0$ or $beta = 0$." He is claiming (correctly) that both $alpha$ and $beta$ are conditional probabilities that cannot be legally summed.
– Kevin
Nov 29 at 18:25
|
show 1 more comment
up vote
5
down vote
Case 1: the null hypothesis is true. The type II error is 0. The type I error is less than the nominal size of the test unless the test is biased. It can be as high as 1 if the test decision is "reject the null every time".
Case 2: the null hypothesis is false. The type I error is 0. the type II error can be as high as 1 if the test decision is "do not reject the null any time".
To conflate Bayesian and frequentist terminology : you can't speak of the Pr(Type 1 error) without "conditioning" or knowing H_0 is true. A nice bit of frequentist notation is this: $P_{H_0}(text{Event})$ to refer to probabilities of events or outcomes under the probability model where the null is true, or $P_{theta = theta_0}(text{Event})$ equivalently.
If you want to be crazy and sum together probabilities that don't make sense, you can conceive of two values of $theta ne theta_0$ and $theta=theta_0$ for which the Type 1 and Type 2 errors add to more than 1. For instance:
P(decomposing corpse | dead) + P(looking alright | alive) > 1
Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
Is it so irrational to add $alpha$ and $beta$ and seek for their sum minimum?
– user2925716
Nov 29 at 15:54
1
Not necessarily, provided you specify (a) what the specific alternative hypothesis is at which $beta$ will be evaluated and (b) what you are varying in order to minimize the sum. The sum could be viewed as a proxy for a combination of losses and prior probabilities associated with two hypotheses.
– whuber♦
Nov 29 at 16:01
@user2925716 I wouldn't. Even in the context of a power analysis, where we speculate as to the possible value(s) of $theta$ where the alternative may hold, this "probability of error" statement only makes sense when the costs of Type 1 and Type 2 errors are the same. But they are not. I report them separately, and discuss the implications of each error.
– AdamO
Nov 29 at 16:23
2
@whuber what does it mean to "specify the alternative hypothesis"? if that is the probability model for the outcome then there is no type I error because the null is wrong. The point is being explicit about what you're comparing. You can add the probabilities together, but as I argue the sum does not represent a probability.
– AdamO
Nov 29 at 18:16
Adam, I'm not trying to suggest the sum represents a probability. I am only responding constructively to the OP's query concerning "is it so irrational." Within the limited space permitted by a comment, I was trying to suggest that such a sum can be interpreted as (proportional to) a posterior expected loss. "Specify the alternative hypothesis" means to stipulate a specific distribution within the set known as "$H_A,$" which is usually not just one distribution. Unless one does so, $beta$ is undefined.
– whuber♦
Nov 29 at 20:18
|
show 2 more comments
up vote
2
down vote
It is true that with a standard hypothesis test you either reject the null hypothesis or you do not. I.e. type II error + power = 1 under $H_A$ and non-rejection probability + type I error = 1 under $H_0$.
However, the statement the way you phrase it is not true. Type I and type II errors cannot happen under the same scenario within the traditional frequentist hypothesis testing paradigm. I.e. either $H_0$ is true, in which can you can either wrongly reject (type I error) or not reject the null hypothesis, or $H_a$ is true, in which case you can either correctly reject $H_0$ (how often you do this on average is the power) or wrongly not reject (type II error).
Please see the last added line in the question.
– user2925716
Nov 29 at 14:57
Regarding the last line: The sum can be any number in (0, 2). It even depends on where exactly you are within $H_A$.
– Björn
Nov 29 at 16:05
In the other answer they've just proved that the sum $alpha+beta$ is in $[0,1]$...
– user2925716
Nov 29 at 16:07
It's unclear to which answer you refer--I cannot find any such proof.
– whuber♦
yesterday
add a comment |
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
4
down vote
accepted
For an arbitrarily chosen decision rule -- meaning that the decision rule can be anything that you make up just for the heck of it, it doesn't need to be sensible in any sense of the word -- the arithmetic sum of the Type I and Type II error probabilities can be any number in $[0,2]$ as the answer by @Bjorn points out.
Example 1: The observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $0$ and so their sum is also $0$.
Example 2: As in Example 1, the observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. But now the decision rule is
$$X begin{array}{c} H_a\gtrless\{H_0}end{array} 0$$
which is exactly bass ackwards from the decision rule in Example 1 (Hey, I said upfront that we are going to consider arbitrary decision rules, not necessarily only sensible ones!!). Now, since the OP asks in a comment for an explanation of my assertion (in a previous version of this answer) that "both the Type I and Type II error probabilities have value $1$ and so their sum is $2$.", here goes.
If the null hypothesis is true, then in our model, the observation $X$ is always a positive number. But the decision rule is that whenever the observation has positive value, we are going to reject the null hypothesis. Continuing to remember that the null hypothesis is true, we see that our decision rule tells us to always reject the null hypothesis (when it is true). So, what is the probability that we reject the null hypothesis when in fact the null hypothesis is true? $100%$, right? A similar argument applies to the Type II error probability. If the null hypothesis is actually false, the observation must have negative value in our model. But the decision rule perversely insists that we must refuse to reject the null when the observation has negative value which happens only when the null hypothesis is false. So, the probability of failing to reject the null hypothesis when in fact the null hypothesis is false (which is what a Type II error is) must be $100%$ too, right?
Thus, both the Type I error probability and the Type II error probability have value 1 for this (admittedly contrived) example of a decision rule, and so their arithmetic sum must be 2.
Hopefully, the above is enough of a "basic computation of this fact" as the OP desires, or is necessary to resort to the Peano axioms to prove that $1+1=2$?
Example 3: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $frac{2}{3}$ and so their sum is $frac 43$.
Example 4: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. But now the decision rule is
$$X begin{array}{c}H_a\gtrless\{H_0}end{array} 0$$
which is more sensible than the decision rule in Example 3, and it leads to both the Type I and Type II error probabilities having value $frac{1}{3}$ and so their sum is $frac 23 < 1$.
So why @AdamO says: what's wrong with his observation? > P(decomposing corpse | dead) + P(looking alright | alive) > 1 Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
– user2925716
Nov 29 at 18:01
3
@user2925716 You are the one interested in asking what the arithmetic sum of the Type I and Type II error probabilities is. This arithmetic sum has no meaning at all in decision making; it is a weighted sum of the two error probabilities that is of interest in Bayesian decision making, and so AdamO is quite correct in his assertion that the fact that sum that you are interested in can have value greater than $1$ is neither surprising nor interesting.
– Dilip Sarwate
Nov 29 at 18:11
Yes, but what about his argument that one of these two probabilities is always zero so it never can be the case that that sum is larger than $1$ ? In fact, I do not understand why your example yields number $2$, can you explain this to a beginner (me)?
– user2925716
Nov 29 at 18:23
@DilipSarwate: Strictly, if you're going into Bayesianism, then you don't have "probability of Type I/II error" at all, because those terms presuppose a Frequentist approach. But then OP's question is meaningless in the first instance, so I fear we have to stay on the Frequentist side of the fence.
– Kevin
Nov 29 at 18:23
2
@user2925716: AdamO is not claiming that "either $alpha =0$ or $beta = 0$." He is claiming (correctly) that both $alpha$ and $beta$ are conditional probabilities that cannot be legally summed.
– Kevin
Nov 29 at 18:25
|
show 1 more comment
up vote
4
down vote
accepted
For an arbitrarily chosen decision rule -- meaning that the decision rule can be anything that you make up just for the heck of it, it doesn't need to be sensible in any sense of the word -- the arithmetic sum of the Type I and Type II error probabilities can be any number in $[0,2]$ as the answer by @Bjorn points out.
Example 1: The observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $0$ and so their sum is also $0$.
Example 2: As in Example 1, the observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. But now the decision rule is
$$X begin{array}{c} H_a\gtrless\{H_0}end{array} 0$$
which is exactly bass ackwards from the decision rule in Example 1 (Hey, I said upfront that we are going to consider arbitrary decision rules, not necessarily only sensible ones!!). Now, since the OP asks in a comment for an explanation of my assertion (in a previous version of this answer) that "both the Type I and Type II error probabilities have value $1$ and so their sum is $2$.", here goes.
If the null hypothesis is true, then in our model, the observation $X$ is always a positive number. But the decision rule is that whenever the observation has positive value, we are going to reject the null hypothesis. Continuing to remember that the null hypothesis is true, we see that our decision rule tells us to always reject the null hypothesis (when it is true). So, what is the probability that we reject the null hypothesis when in fact the null hypothesis is true? $100%$, right? A similar argument applies to the Type II error probability. If the null hypothesis is actually false, the observation must have negative value in our model. But the decision rule perversely insists that we must refuse to reject the null when the observation has negative value which happens only when the null hypothesis is false. So, the probability of failing to reject the null hypothesis when in fact the null hypothesis is false (which is what a Type II error is) must be $100%$ too, right?
Thus, both the Type I error probability and the Type II error probability have value 1 for this (admittedly contrived) example of a decision rule, and so their arithmetic sum must be 2.
Hopefully, the above is enough of a "basic computation of this fact" as the OP desires, or is necessary to resort to the Peano axioms to prove that $1+1=2$?
Example 3: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $frac{2}{3}$ and so their sum is $frac 43$.
Example 4: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. But now the decision rule is
$$X begin{array}{c}H_a\gtrless\{H_0}end{array} 0$$
which is more sensible than the decision rule in Example 3, and it leads to both the Type I and Type II error probabilities having value $frac{1}{3}$ and so their sum is $frac 23 < 1$.
So why @AdamO says: what's wrong with his observation? > P(decomposing corpse | dead) + P(looking alright | alive) > 1 Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
– user2925716
Nov 29 at 18:01
3
@user2925716 You are the one interested in asking what the arithmetic sum of the Type I and Type II error probabilities is. This arithmetic sum has no meaning at all in decision making; it is a weighted sum of the two error probabilities that is of interest in Bayesian decision making, and so AdamO is quite correct in his assertion that the fact that sum that you are interested in can have value greater than $1$ is neither surprising nor interesting.
– Dilip Sarwate
Nov 29 at 18:11
Yes, but what about his argument that one of these two probabilities is always zero so it never can be the case that that sum is larger than $1$ ? In fact, I do not understand why your example yields number $2$, can you explain this to a beginner (me)?
– user2925716
Nov 29 at 18:23
@DilipSarwate: Strictly, if you're going into Bayesianism, then you don't have "probability of Type I/II error" at all, because those terms presuppose a Frequentist approach. But then OP's question is meaningless in the first instance, so I fear we have to stay on the Frequentist side of the fence.
– Kevin
Nov 29 at 18:23
2
@user2925716: AdamO is not claiming that "either $alpha =0$ or $beta = 0$." He is claiming (correctly) that both $alpha$ and $beta$ are conditional probabilities that cannot be legally summed.
– Kevin
Nov 29 at 18:25
|
show 1 more comment
up vote
4
down vote
accepted
up vote
4
down vote
accepted
For an arbitrarily chosen decision rule -- meaning that the decision rule can be anything that you make up just for the heck of it, it doesn't need to be sensible in any sense of the word -- the arithmetic sum of the Type I and Type II error probabilities can be any number in $[0,2]$ as the answer by @Bjorn points out.
Example 1: The observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $0$ and so their sum is also $0$.
Example 2: As in Example 1, the observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. But now the decision rule is
$$X begin{array}{c} H_a\gtrless\{H_0}end{array} 0$$
which is exactly bass ackwards from the decision rule in Example 1 (Hey, I said upfront that we are going to consider arbitrary decision rules, not necessarily only sensible ones!!). Now, since the OP asks in a comment for an explanation of my assertion (in a previous version of this answer) that "both the Type I and Type II error probabilities have value $1$ and so their sum is $2$.", here goes.
If the null hypothesis is true, then in our model, the observation $X$ is always a positive number. But the decision rule is that whenever the observation has positive value, we are going to reject the null hypothesis. Continuing to remember that the null hypothesis is true, we see that our decision rule tells us to always reject the null hypothesis (when it is true). So, what is the probability that we reject the null hypothesis when in fact the null hypothesis is true? $100%$, right? A similar argument applies to the Type II error probability. If the null hypothesis is actually false, the observation must have negative value in our model. But the decision rule perversely insists that we must refuse to reject the null when the observation has negative value which happens only when the null hypothesis is false. So, the probability of failing to reject the null hypothesis when in fact the null hypothesis is false (which is what a Type II error is) must be $100%$ too, right?
Thus, both the Type I error probability and the Type II error probability have value 1 for this (admittedly contrived) example of a decision rule, and so their arithmetic sum must be 2.
Hopefully, the above is enough of a "basic computation of this fact" as the OP desires, or is necessary to resort to the Peano axioms to prove that $1+1=2$?
Example 3: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $frac{2}{3}$ and so their sum is $frac 43$.
Example 4: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. But now the decision rule is
$$X begin{array}{c}H_a\gtrless\{H_0}end{array} 0$$
which is more sensible than the decision rule in Example 3, and it leads to both the Type I and Type II error probabilities having value $frac{1}{3}$ and so their sum is $frac 23 < 1$.
For an arbitrarily chosen decision rule -- meaning that the decision rule can be anything that you make up just for the heck of it, it doesn't need to be sensible in any sense of the word -- the arithmetic sum of the Type I and Type II error probabilities can be any number in $[0,2]$ as the answer by @Bjorn points out.
Example 1: The observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $0$ and so their sum is also $0$.
Example 2: As in Example 1, the observation $X$ always has positive value when $H_0$ is true and always has negative value when $H_a$ is true. But now the decision rule is
$$X begin{array}{c} H_a\gtrless\{H_0}end{array} 0$$
which is exactly bass ackwards from the decision rule in Example 1 (Hey, I said upfront that we are going to consider arbitrary decision rules, not necessarily only sensible ones!!). Now, since the OP asks in a comment for an explanation of my assertion (in a previous version of this answer) that "both the Type I and Type II error probabilities have value $1$ and so their sum is $2$.", here goes.
If the null hypothesis is true, then in our model, the observation $X$ is always a positive number. But the decision rule is that whenever the observation has positive value, we are going to reject the null hypothesis. Continuing to remember that the null hypothesis is true, we see that our decision rule tells us to always reject the null hypothesis (when it is true). So, what is the probability that we reject the null hypothesis when in fact the null hypothesis is true? $100%$, right? A similar argument applies to the Type II error probability. If the null hypothesis is actually false, the observation must have negative value in our model. But the decision rule perversely insists that we must refuse to reject the null when the observation has negative value which happens only when the null hypothesis is false. So, the probability of failing to reject the null hypothesis when in fact the null hypothesis is false (which is what a Type II error is) must be $100%$ too, right?
Thus, both the Type I error probability and the Type II error probability have value 1 for this (admittedly contrived) example of a decision rule, and so their arithmetic sum must be 2.
Hopefully, the above is enough of a "basic computation of this fact" as the OP desires, or is necessary to resort to the Peano axioms to prove that $1+1=2$?
Example 3: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. The decision rule is
$$X begin{array}{c}H_0\gtrless\{H_a}end{array} 0$$
leading to both the Type I and Type II error probabilities having value $frac{2}{3}$ and so their sum is $frac 43$.
Example 4: The observation $X sim U[-2,1]$ whenever $H_0$ is true while $X sim U[-1,2]$ whenever $H_a$ is true. But now the decision rule is
$$X begin{array}{c}H_a\gtrless\{H_0}end{array} 0$$
which is more sensible than the decision rule in Example 3, and it leads to both the Type I and Type II error probabilities having value $frac{1}{3}$ and so their sum is $frac 23 < 1$.
edited Nov 29 at 19:28
answered Nov 29 at 17:37
Dilip Sarwate
29.5k252146
29.5k252146
So why @AdamO says: what's wrong with his observation? > P(decomposing corpse | dead) + P(looking alright | alive) > 1 Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
– user2925716
Nov 29 at 18:01
3
@user2925716 You are the one interested in asking what the arithmetic sum of the Type I and Type II error probabilities is. This arithmetic sum has no meaning at all in decision making; it is a weighted sum of the two error probabilities that is of interest in Bayesian decision making, and so AdamO is quite correct in his assertion that the fact that sum that you are interested in can have value greater than $1$ is neither surprising nor interesting.
– Dilip Sarwate
Nov 29 at 18:11
Yes, but what about his argument that one of these two probabilities is always zero so it never can be the case that that sum is larger than $1$ ? In fact, I do not understand why your example yields number $2$, can you explain this to a beginner (me)?
– user2925716
Nov 29 at 18:23
@DilipSarwate: Strictly, if you're going into Bayesianism, then you don't have "probability of Type I/II error" at all, because those terms presuppose a Frequentist approach. But then OP's question is meaningless in the first instance, so I fear we have to stay on the Frequentist side of the fence.
– Kevin
Nov 29 at 18:23
2
@user2925716: AdamO is not claiming that "either $alpha =0$ or $beta = 0$." He is claiming (correctly) that both $alpha$ and $beta$ are conditional probabilities that cannot be legally summed.
– Kevin
Nov 29 at 18:25
|
show 1 more comment
So why @AdamO says: what's wrong with his observation? > P(decomposing corpse | dead) + P(looking alright | alive) > 1 Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
– user2925716
Nov 29 at 18:01
3
@user2925716 You are the one interested in asking what the arithmetic sum of the Type I and Type II error probabilities is. This arithmetic sum has no meaning at all in decision making; it is a weighted sum of the two error probabilities that is of interest in Bayesian decision making, and so AdamO is quite correct in his assertion that the fact that sum that you are interested in can have value greater than $1$ is neither surprising nor interesting.
– Dilip Sarwate
Nov 29 at 18:11
Yes, but what about his argument that one of these two probabilities is always zero so it never can be the case that that sum is larger than $1$ ? In fact, I do not understand why your example yields number $2$, can you explain this to a beginner (me)?
– user2925716
Nov 29 at 18:23
@DilipSarwate: Strictly, if you're going into Bayesianism, then you don't have "probability of Type I/II error" at all, because those terms presuppose a Frequentist approach. But then OP's question is meaningless in the first instance, so I fear we have to stay on the Frequentist side of the fence.
– Kevin
Nov 29 at 18:23
2
@user2925716: AdamO is not claiming that "either $alpha =0$ or $beta = 0$." He is claiming (correctly) that both $alpha$ and $beta$ are conditional probabilities that cannot be legally summed.
– Kevin
Nov 29 at 18:25
So why @AdamO says: what's wrong with his observation? > P(decomposing corpse | dead) + P(looking alright | alive) > 1 Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
– user2925716
Nov 29 at 18:01
So why @AdamO says: what's wrong with his observation? > P(decomposing corpse | dead) + P(looking alright | alive) > 1 Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
– user2925716
Nov 29 at 18:01
3
3
@user2925716 You are the one interested in asking what the arithmetic sum of the Type I and Type II error probabilities is. This arithmetic sum has no meaning at all in decision making; it is a weighted sum of the two error probabilities that is of interest in Bayesian decision making, and so AdamO is quite correct in his assertion that the fact that sum that you are interested in can have value greater than $1$ is neither surprising nor interesting.
– Dilip Sarwate
Nov 29 at 18:11
@user2925716 You are the one interested in asking what the arithmetic sum of the Type I and Type II error probabilities is. This arithmetic sum has no meaning at all in decision making; it is a weighted sum of the two error probabilities that is of interest in Bayesian decision making, and so AdamO is quite correct in his assertion that the fact that sum that you are interested in can have value greater than $1$ is neither surprising nor interesting.
– Dilip Sarwate
Nov 29 at 18:11
Yes, but what about his argument that one of these two probabilities is always zero so it never can be the case that that sum is larger than $1$ ? In fact, I do not understand why your example yields number $2$, can you explain this to a beginner (me)?
– user2925716
Nov 29 at 18:23
Yes, but what about his argument that one of these two probabilities is always zero so it never can be the case that that sum is larger than $1$ ? In fact, I do not understand why your example yields number $2$, can you explain this to a beginner (me)?
– user2925716
Nov 29 at 18:23
@DilipSarwate: Strictly, if you're going into Bayesianism, then you don't have "probability of Type I/II error" at all, because those terms presuppose a Frequentist approach. But then OP's question is meaningless in the first instance, so I fear we have to stay on the Frequentist side of the fence.
– Kevin
Nov 29 at 18:23
@DilipSarwate: Strictly, if you're going into Bayesianism, then you don't have "probability of Type I/II error" at all, because those terms presuppose a Frequentist approach. But then OP's question is meaningless in the first instance, so I fear we have to stay on the Frequentist side of the fence.
– Kevin
Nov 29 at 18:23
2
2
@user2925716: AdamO is not claiming that "either $alpha =0$ or $beta = 0$." He is claiming (correctly) that both $alpha$ and $beta$ are conditional probabilities that cannot be legally summed.
– Kevin
Nov 29 at 18:25
@user2925716: AdamO is not claiming that "either $alpha =0$ or $beta = 0$." He is claiming (correctly) that both $alpha$ and $beta$ are conditional probabilities that cannot be legally summed.
– Kevin
Nov 29 at 18:25
|
show 1 more comment
up vote
5
down vote
Case 1: the null hypothesis is true. The type II error is 0. The type I error is less than the nominal size of the test unless the test is biased. It can be as high as 1 if the test decision is "reject the null every time".
Case 2: the null hypothesis is false. The type I error is 0. the type II error can be as high as 1 if the test decision is "do not reject the null any time".
To conflate Bayesian and frequentist terminology : you can't speak of the Pr(Type 1 error) without "conditioning" or knowing H_0 is true. A nice bit of frequentist notation is this: $P_{H_0}(text{Event})$ to refer to probabilities of events or outcomes under the probability model where the null is true, or $P_{theta = theta_0}(text{Event})$ equivalently.
If you want to be crazy and sum together probabilities that don't make sense, you can conceive of two values of $theta ne theta_0$ and $theta=theta_0$ for which the Type 1 and Type 2 errors add to more than 1. For instance:
P(decomposing corpse | dead) + P(looking alright | alive) > 1
Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
Is it so irrational to add $alpha$ and $beta$ and seek for their sum minimum?
– user2925716
Nov 29 at 15:54
1
Not necessarily, provided you specify (a) what the specific alternative hypothesis is at which $beta$ will be evaluated and (b) what you are varying in order to minimize the sum. The sum could be viewed as a proxy for a combination of losses and prior probabilities associated with two hypotheses.
– whuber♦
Nov 29 at 16:01
@user2925716 I wouldn't. Even in the context of a power analysis, where we speculate as to the possible value(s) of $theta$ where the alternative may hold, this "probability of error" statement only makes sense when the costs of Type 1 and Type 2 errors are the same. But they are not. I report them separately, and discuss the implications of each error.
– AdamO
Nov 29 at 16:23
2
@whuber what does it mean to "specify the alternative hypothesis"? if that is the probability model for the outcome then there is no type I error because the null is wrong. The point is being explicit about what you're comparing. You can add the probabilities together, but as I argue the sum does not represent a probability.
– AdamO
Nov 29 at 18:16
Adam, I'm not trying to suggest the sum represents a probability. I am only responding constructively to the OP's query concerning "is it so irrational." Within the limited space permitted by a comment, I was trying to suggest that such a sum can be interpreted as (proportional to) a posterior expected loss. "Specify the alternative hypothesis" means to stipulate a specific distribution within the set known as "$H_A,$" which is usually not just one distribution. Unless one does so, $beta$ is undefined.
– whuber♦
Nov 29 at 20:18
|
show 2 more comments
up vote
5
down vote
Case 1: the null hypothesis is true. The type II error is 0. The type I error is less than the nominal size of the test unless the test is biased. It can be as high as 1 if the test decision is "reject the null every time".
Case 2: the null hypothesis is false. The type I error is 0. the type II error can be as high as 1 if the test decision is "do not reject the null any time".
To conflate Bayesian and frequentist terminology : you can't speak of the Pr(Type 1 error) without "conditioning" or knowing H_0 is true. A nice bit of frequentist notation is this: $P_{H_0}(text{Event})$ to refer to probabilities of events or outcomes under the probability model where the null is true, or $P_{theta = theta_0}(text{Event})$ equivalently.
If you want to be crazy and sum together probabilities that don't make sense, you can conceive of two values of $theta ne theta_0$ and $theta=theta_0$ for which the Type 1 and Type 2 errors add to more than 1. For instance:
P(decomposing corpse | dead) + P(looking alright | alive) > 1
Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
Is it so irrational to add $alpha$ and $beta$ and seek for their sum minimum?
– user2925716
Nov 29 at 15:54
1
Not necessarily, provided you specify (a) what the specific alternative hypothesis is at which $beta$ will be evaluated and (b) what you are varying in order to minimize the sum. The sum could be viewed as a proxy for a combination of losses and prior probabilities associated with two hypotheses.
– whuber♦
Nov 29 at 16:01
@user2925716 I wouldn't. Even in the context of a power analysis, where we speculate as to the possible value(s) of $theta$ where the alternative may hold, this "probability of error" statement only makes sense when the costs of Type 1 and Type 2 errors are the same. But they are not. I report them separately, and discuss the implications of each error.
– AdamO
Nov 29 at 16:23
2
@whuber what does it mean to "specify the alternative hypothesis"? if that is the probability model for the outcome then there is no type I error because the null is wrong. The point is being explicit about what you're comparing. You can add the probabilities together, but as I argue the sum does not represent a probability.
– AdamO
Nov 29 at 18:16
Adam, I'm not trying to suggest the sum represents a probability. I am only responding constructively to the OP's query concerning "is it so irrational." Within the limited space permitted by a comment, I was trying to suggest that such a sum can be interpreted as (proportional to) a posterior expected loss. "Specify the alternative hypothesis" means to stipulate a specific distribution within the set known as "$H_A,$" which is usually not just one distribution. Unless one does so, $beta$ is undefined.
– whuber♦
Nov 29 at 20:18
|
show 2 more comments
up vote
5
down vote
up vote
5
down vote
Case 1: the null hypothesis is true. The type II error is 0. The type I error is less than the nominal size of the test unless the test is biased. It can be as high as 1 if the test decision is "reject the null every time".
Case 2: the null hypothesis is false. The type I error is 0. the type II error can be as high as 1 if the test decision is "do not reject the null any time".
To conflate Bayesian and frequentist terminology : you can't speak of the Pr(Type 1 error) without "conditioning" or knowing H_0 is true. A nice bit of frequentist notation is this: $P_{H_0}(text{Event})$ to refer to probabilities of events or outcomes under the probability model where the null is true, or $P_{theta = theta_0}(text{Event})$ equivalently.
If you want to be crazy and sum together probabilities that don't make sense, you can conceive of two values of $theta ne theta_0$ and $theta=theta_0$ for which the Type 1 and Type 2 errors add to more than 1. For instance:
P(decomposing corpse | dead) + P(looking alright | alive) > 1
Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
Case 1: the null hypothesis is true. The type II error is 0. The type I error is less than the nominal size of the test unless the test is biased. It can be as high as 1 if the test decision is "reject the null every time".
Case 2: the null hypothesis is false. The type I error is 0. the type II error can be as high as 1 if the test decision is "do not reject the null any time".
To conflate Bayesian and frequentist terminology : you can't speak of the Pr(Type 1 error) without "conditioning" or knowing H_0 is true. A nice bit of frequentist notation is this: $P_{H_0}(text{Event})$ to refer to probabilities of events or outcomes under the probability model where the null is true, or $P_{theta = theta_0}(text{Event})$ equivalently.
If you want to be crazy and sum together probabilities that don't make sense, you can conceive of two values of $theta ne theta_0$ and $theta=theta_0$ for which the Type 1 and Type 2 errors add to more than 1. For instance:
P(decomposing corpse | dead) + P(looking alright | alive) > 1
Is this surprising or interesting? No. IRL one of those error probabilities will always be 0 and the other is less than or equal to 1 depending on how good or stupid the test is.
edited Nov 29 at 15:43
answered Nov 29 at 15:21
AdamO
32k257136
32k257136
Is it so irrational to add $alpha$ and $beta$ and seek for their sum minimum?
– user2925716
Nov 29 at 15:54
1
Not necessarily, provided you specify (a) what the specific alternative hypothesis is at which $beta$ will be evaluated and (b) what you are varying in order to minimize the sum. The sum could be viewed as a proxy for a combination of losses and prior probabilities associated with two hypotheses.
– whuber♦
Nov 29 at 16:01
@user2925716 I wouldn't. Even in the context of a power analysis, where we speculate as to the possible value(s) of $theta$ where the alternative may hold, this "probability of error" statement only makes sense when the costs of Type 1 and Type 2 errors are the same. But they are not. I report them separately, and discuss the implications of each error.
– AdamO
Nov 29 at 16:23
2
@whuber what does it mean to "specify the alternative hypothesis"? if that is the probability model for the outcome then there is no type I error because the null is wrong. The point is being explicit about what you're comparing. You can add the probabilities together, but as I argue the sum does not represent a probability.
– AdamO
Nov 29 at 18:16
Adam, I'm not trying to suggest the sum represents a probability. I am only responding constructively to the OP's query concerning "is it so irrational." Within the limited space permitted by a comment, I was trying to suggest that such a sum can be interpreted as (proportional to) a posterior expected loss. "Specify the alternative hypothesis" means to stipulate a specific distribution within the set known as "$H_A,$" which is usually not just one distribution. Unless one does so, $beta$ is undefined.
– whuber♦
Nov 29 at 20:18
|
show 2 more comments
Is it so irrational to add $alpha$ and $beta$ and seek for their sum minimum?
– user2925716
Nov 29 at 15:54
1
Not necessarily, provided you specify (a) what the specific alternative hypothesis is at which $beta$ will be evaluated and (b) what you are varying in order to minimize the sum. The sum could be viewed as a proxy for a combination of losses and prior probabilities associated with two hypotheses.
– whuber♦
Nov 29 at 16:01
@user2925716 I wouldn't. Even in the context of a power analysis, where we speculate as to the possible value(s) of $theta$ where the alternative may hold, this "probability of error" statement only makes sense when the costs of Type 1 and Type 2 errors are the same. But they are not. I report them separately, and discuss the implications of each error.
– AdamO
Nov 29 at 16:23
2
@whuber what does it mean to "specify the alternative hypothesis"? if that is the probability model for the outcome then there is no type I error because the null is wrong. The point is being explicit about what you're comparing. You can add the probabilities together, but as I argue the sum does not represent a probability.
– AdamO
Nov 29 at 18:16
Adam, I'm not trying to suggest the sum represents a probability. I am only responding constructively to the OP's query concerning "is it so irrational." Within the limited space permitted by a comment, I was trying to suggest that such a sum can be interpreted as (proportional to) a posterior expected loss. "Specify the alternative hypothesis" means to stipulate a specific distribution within the set known as "$H_A,$" which is usually not just one distribution. Unless one does so, $beta$ is undefined.
– whuber♦
Nov 29 at 20:18
Is it so irrational to add $alpha$ and $beta$ and seek for their sum minimum?
– user2925716
Nov 29 at 15:54
Is it so irrational to add $alpha$ and $beta$ and seek for their sum minimum?
– user2925716
Nov 29 at 15:54
1
1
Not necessarily, provided you specify (a) what the specific alternative hypothesis is at which $beta$ will be evaluated and (b) what you are varying in order to minimize the sum. The sum could be viewed as a proxy for a combination of losses and prior probabilities associated with two hypotheses.
– whuber♦
Nov 29 at 16:01
Not necessarily, provided you specify (a) what the specific alternative hypothesis is at which $beta$ will be evaluated and (b) what you are varying in order to minimize the sum. The sum could be viewed as a proxy for a combination of losses and prior probabilities associated with two hypotheses.
– whuber♦
Nov 29 at 16:01
@user2925716 I wouldn't. Even in the context of a power analysis, where we speculate as to the possible value(s) of $theta$ where the alternative may hold, this "probability of error" statement only makes sense when the costs of Type 1 and Type 2 errors are the same. But they are not. I report them separately, and discuss the implications of each error.
– AdamO
Nov 29 at 16:23
@user2925716 I wouldn't. Even in the context of a power analysis, where we speculate as to the possible value(s) of $theta$ where the alternative may hold, this "probability of error" statement only makes sense when the costs of Type 1 and Type 2 errors are the same. But they are not. I report them separately, and discuss the implications of each error.
– AdamO
Nov 29 at 16:23
2
2
@whuber what does it mean to "specify the alternative hypothesis"? if that is the probability model for the outcome then there is no type I error because the null is wrong. The point is being explicit about what you're comparing. You can add the probabilities together, but as I argue the sum does not represent a probability.
– AdamO
Nov 29 at 18:16
@whuber what does it mean to "specify the alternative hypothesis"? if that is the probability model for the outcome then there is no type I error because the null is wrong. The point is being explicit about what you're comparing. You can add the probabilities together, but as I argue the sum does not represent a probability.
– AdamO
Nov 29 at 18:16
Adam, I'm not trying to suggest the sum represents a probability. I am only responding constructively to the OP's query concerning "is it so irrational." Within the limited space permitted by a comment, I was trying to suggest that such a sum can be interpreted as (proportional to) a posterior expected loss. "Specify the alternative hypothesis" means to stipulate a specific distribution within the set known as "$H_A,$" which is usually not just one distribution. Unless one does so, $beta$ is undefined.
– whuber♦
Nov 29 at 20:18
Adam, I'm not trying to suggest the sum represents a probability. I am only responding constructively to the OP's query concerning "is it so irrational." Within the limited space permitted by a comment, I was trying to suggest that such a sum can be interpreted as (proportional to) a posterior expected loss. "Specify the alternative hypothesis" means to stipulate a specific distribution within the set known as "$H_A,$" which is usually not just one distribution. Unless one does so, $beta$ is undefined.
– whuber♦
Nov 29 at 20:18
|
show 2 more comments
up vote
2
down vote
It is true that with a standard hypothesis test you either reject the null hypothesis or you do not. I.e. type II error + power = 1 under $H_A$ and non-rejection probability + type I error = 1 under $H_0$.
However, the statement the way you phrase it is not true. Type I and type II errors cannot happen under the same scenario within the traditional frequentist hypothesis testing paradigm. I.e. either $H_0$ is true, in which can you can either wrongly reject (type I error) or not reject the null hypothesis, or $H_a$ is true, in which case you can either correctly reject $H_0$ (how often you do this on average is the power) or wrongly not reject (type II error).
Please see the last added line in the question.
– user2925716
Nov 29 at 14:57
Regarding the last line: The sum can be any number in (0, 2). It even depends on where exactly you are within $H_A$.
– Björn
Nov 29 at 16:05
In the other answer they've just proved that the sum $alpha+beta$ is in $[0,1]$...
– user2925716
Nov 29 at 16:07
It's unclear to which answer you refer--I cannot find any such proof.
– whuber♦
yesterday
add a comment |
up vote
2
down vote
It is true that with a standard hypothesis test you either reject the null hypothesis or you do not. I.e. type II error + power = 1 under $H_A$ and non-rejection probability + type I error = 1 under $H_0$.
However, the statement the way you phrase it is not true. Type I and type II errors cannot happen under the same scenario within the traditional frequentist hypothesis testing paradigm. I.e. either $H_0$ is true, in which can you can either wrongly reject (type I error) or not reject the null hypothesis, or $H_a$ is true, in which case you can either correctly reject $H_0$ (how often you do this on average is the power) or wrongly not reject (type II error).
Please see the last added line in the question.
– user2925716
Nov 29 at 14:57
Regarding the last line: The sum can be any number in (0, 2). It even depends on where exactly you are within $H_A$.
– Björn
Nov 29 at 16:05
In the other answer they've just proved that the sum $alpha+beta$ is in $[0,1]$...
– user2925716
Nov 29 at 16:07
It's unclear to which answer you refer--I cannot find any such proof.
– whuber♦
yesterday
add a comment |
up vote
2
down vote
up vote
2
down vote
It is true that with a standard hypothesis test you either reject the null hypothesis or you do not. I.e. type II error + power = 1 under $H_A$ and non-rejection probability + type I error = 1 under $H_0$.
However, the statement the way you phrase it is not true. Type I and type II errors cannot happen under the same scenario within the traditional frequentist hypothesis testing paradigm. I.e. either $H_0$ is true, in which can you can either wrongly reject (type I error) or not reject the null hypothesis, or $H_a$ is true, in which case you can either correctly reject $H_0$ (how often you do this on average is the power) or wrongly not reject (type II error).
It is true that with a standard hypothesis test you either reject the null hypothesis or you do not. I.e. type II error + power = 1 under $H_A$ and non-rejection probability + type I error = 1 under $H_0$.
However, the statement the way you phrase it is not true. Type I and type II errors cannot happen under the same scenario within the traditional frequentist hypothesis testing paradigm. I.e. either $H_0$ is true, in which can you can either wrongly reject (type I error) or not reject the null hypothesis, or $H_a$ is true, in which case you can either correctly reject $H_0$ (how often you do this on average is the power) or wrongly not reject (type II error).
answered Nov 29 at 14:53
Björn
9,0531834
9,0531834
Please see the last added line in the question.
– user2925716
Nov 29 at 14:57
Regarding the last line: The sum can be any number in (0, 2). It even depends on where exactly you are within $H_A$.
– Björn
Nov 29 at 16:05
In the other answer they've just proved that the sum $alpha+beta$ is in $[0,1]$...
– user2925716
Nov 29 at 16:07
It's unclear to which answer you refer--I cannot find any such proof.
– whuber♦
yesterday
add a comment |
Please see the last added line in the question.
– user2925716
Nov 29 at 14:57
Regarding the last line: The sum can be any number in (0, 2). It even depends on where exactly you are within $H_A$.
– Björn
Nov 29 at 16:05
In the other answer they've just proved that the sum $alpha+beta$ is in $[0,1]$...
– user2925716
Nov 29 at 16:07
It's unclear to which answer you refer--I cannot find any such proof.
– whuber♦
yesterday
Please see the last added line in the question.
– user2925716
Nov 29 at 14:57
Please see the last added line in the question.
– user2925716
Nov 29 at 14:57
Regarding the last line: The sum can be any number in (0, 2). It even depends on where exactly you are within $H_A$.
– Björn
Nov 29 at 16:05
Regarding the last line: The sum can be any number in (0, 2). It even depends on where exactly you are within $H_A$.
– Björn
Nov 29 at 16:05
In the other answer they've just proved that the sum $alpha+beta$ is in $[0,1]$...
– user2925716
Nov 29 at 16:07
In the other answer they've just proved that the sum $alpha+beta$ is in $[0,1]$...
– user2925716
Nov 29 at 16:07
It's unclear to which answer you refer--I cannot find any such proof.
– whuber♦
yesterday
It's unclear to which answer you refer--I cannot find any such proof.
– whuber♦
yesterday
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f379423%2fis-there-a-constraint-on-the-sum-of-the-type-i-type-ii-error-probabilities%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
If you mean probability of the errors, then no. Type 1 error and Type 2 error are not complementary events in general.
– StubbornAtom
Nov 29 at 14:51
OK. Is that sum always less then 1, or can it be sometimes even greater then 1?
– user2925716
Nov 29 at 14:52
Please mention 'that sum' of the probabilities if that's what you mean, instead of the sum of the two errors.
– StubbornAtom
Nov 29 at 14:59
I really mean $alpha+beta$ as usually denoted these two errors. $alpha$ is the probability that we reject $H_0$ if $H_0$ is true and the other one accept $H_0$ if $H_1$ is true.
– user2925716
Nov 29 at 15:02
Okay. But that's not what your post says - that's what I am saying.
– StubbornAtom
Nov 29 at 15:04