Why does the Kolmogorov-Smirnov test work?





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty{ margin-bottom:0;
}






up vote
23
down vote

favorite
8












In reading about the 2-sample KS test, I understand exactly what it is doing but I don't understand why it works.



In other words, I can follow all the steps to compute the empirical distribution functions, find the maximum difference between the two to find the D-statistic, calculate the critical values, convert the D-statistic to a p-value etc.



But, I have no idea why any of this actually tells me anything about the two distributions.



Someone could have just as easily told me that I need jump over a donkey and count how fast it runs away and the if the velocity is less than 2 km/hr then I reject the null-hypothesis. Sure I can do what you told me to do, but what does any of that have to do with the null-hypothesis?



Why does the 2-sample KS test work? What does computing the maximum difference between the ECDFs have to do with how different the two distributions are?



Any help is appreciated. I am not a statistician, so assume that I'm an idiot if possible.










share|cite|improve this question


















  • 4




    Welcome to CV, Darcy! Great question!
    – Alexis
    Nov 28 at 18:00






  • 1




    Jump over a donkey... :)
    – Richard Hardy
    Nov 29 at 12:48

















up vote
23
down vote

favorite
8












In reading about the 2-sample KS test, I understand exactly what it is doing but I don't understand why it works.



In other words, I can follow all the steps to compute the empirical distribution functions, find the maximum difference between the two to find the D-statistic, calculate the critical values, convert the D-statistic to a p-value etc.



But, I have no idea why any of this actually tells me anything about the two distributions.



Someone could have just as easily told me that I need jump over a donkey and count how fast it runs away and the if the velocity is less than 2 km/hr then I reject the null-hypothesis. Sure I can do what you told me to do, but what does any of that have to do with the null-hypothesis?



Why does the 2-sample KS test work? What does computing the maximum difference between the ECDFs have to do with how different the two distributions are?



Any help is appreciated. I am not a statistician, so assume that I'm an idiot if possible.










share|cite|improve this question


















  • 4




    Welcome to CV, Darcy! Great question!
    – Alexis
    Nov 28 at 18:00






  • 1




    Jump over a donkey... :)
    – Richard Hardy
    Nov 29 at 12:48













up vote
23
down vote

favorite
8









up vote
23
down vote

favorite
8






8





In reading about the 2-sample KS test, I understand exactly what it is doing but I don't understand why it works.



In other words, I can follow all the steps to compute the empirical distribution functions, find the maximum difference between the two to find the D-statistic, calculate the critical values, convert the D-statistic to a p-value etc.



But, I have no idea why any of this actually tells me anything about the two distributions.



Someone could have just as easily told me that I need jump over a donkey and count how fast it runs away and the if the velocity is less than 2 km/hr then I reject the null-hypothesis. Sure I can do what you told me to do, but what does any of that have to do with the null-hypothesis?



Why does the 2-sample KS test work? What does computing the maximum difference between the ECDFs have to do with how different the two distributions are?



Any help is appreciated. I am not a statistician, so assume that I'm an idiot if possible.










share|cite|improve this question













In reading about the 2-sample KS test, I understand exactly what it is doing but I don't understand why it works.



In other words, I can follow all the steps to compute the empirical distribution functions, find the maximum difference between the two to find the D-statistic, calculate the critical values, convert the D-statistic to a p-value etc.



But, I have no idea why any of this actually tells me anything about the two distributions.



Someone could have just as easily told me that I need jump over a donkey and count how fast it runs away and the if the velocity is less than 2 km/hr then I reject the null-hypothesis. Sure I can do what you told me to do, but what does any of that have to do with the null-hypothesis?



Why does the 2-sample KS test work? What does computing the maximum difference between the ECDFs have to do with how different the two distributions are?



Any help is appreciated. I am not a statistician, so assume that I'm an idiot if possible.







distributions statistical-significance nonparametric kolmogorov-smirnov






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked Nov 28 at 17:05









Darcy

16618




16618








  • 4




    Welcome to CV, Darcy! Great question!
    – Alexis
    Nov 28 at 18:00






  • 1




    Jump over a donkey... :)
    – Richard Hardy
    Nov 29 at 12:48














  • 4




    Welcome to CV, Darcy! Great question!
    – Alexis
    Nov 28 at 18:00






  • 1




    Jump over a donkey... :)
    – Richard Hardy
    Nov 29 at 12:48








4




4




Welcome to CV, Darcy! Great question!
– Alexis
Nov 28 at 18:00




Welcome to CV, Darcy! Great question!
– Alexis
Nov 28 at 18:00




1




1




Jump over a donkey... :)
– Richard Hardy
Nov 29 at 12:48




Jump over a donkey... :)
– Richard Hardy
Nov 29 at 12:48










3 Answers
3






active

oldest

votes

















up vote
9
down vote













Basically, the test is consistent as a direct result of the Glivenko Cantelli theorem, one of the most important results of empirical processes and maybe statistics.



GC tells us that the Kolmogorov Smirnov test statistic goes to 0 as $n rightarrow infty$ under the null hypothesis. It may seem intuitive until you grapple with real analysis and limit theorems. This is a revelation because the process can be thought of as an uncountably infinite number of random processes, so the laws or probability would lead one to believe that there is always one point which could exceed any epsilon boundary but no, the supremum will converge in the long run.



How long? Mmyyeeaa I don't know. The power of the test is kind of dubious. I'd never use it in reality.



http://www.math.utah.edu/~davar/ps-pdf-files/Kolmogorov-Smirnov.pdf






share|cite|improve this answer

















  • 2




    +1 Hi AdamO! Got a one to two sentence take on the power being "kind of dubious?" I would love that perspective (I have gathered that the test is considered easily "overpowered").
    – Alexis
    Nov 28 at 17:59








  • 1




    @Alexis The test is not overpowered, IRL we almost never expect the null to be true, rather we just don't care whether the 99.999-th percentile differs by 0.1 between $F_1$ and $F_2$., so whenever I see $p > 0.05$ from the KS test, all I think is, "that's a false negative" and whenever I see $p < 0.05$ I think "whoop-dee-do so what can you say about that?". Tests of the strong null hypothesis $F_1 = F_2$ aren't a compelling way of presenting scientific evidence.
    – AdamO
    Nov 28 at 19:18








  • 1




    Ok. I get yer concern with hypothesis tests for difference. But does your concern about power arise from the simple ontological belief that $F_{1}$ almost surely $ne F_{2}$? or is there something more mathy about asymptotics or something else in there?
    – Alexis
    Nov 28 at 19:49






  • 1




    @Alexis no, I have no concerns with the mathematics of the test. In fact, I think it's quite elegant and the limit theorem result is very impressive.
    – AdamO
    Nov 29 at 4:15






  • 2




    @Alexis I will say, in settings where it is possible for $F_1$ to be exactly equal to $F_2$, the test can be pretty handy. I agree that not a lot of substantive scientific applications fit that bill, but in a statistical computing context where you want to validate that some software you've written is generating pseudo random numbers from some known distribution, it's quite useful. It effectively codifies the intuition you'd get from looking at probability plots.
    – bamts
    2 days ago


















up vote
9
down vote













We have two independent, univariate samples:



begin{align}
X_1,,X_2,,...,,X_N&overset{iid}{sim}F\
Y_1,,Y_2,,...,,Y_M&overset{iid}{sim}G,
end{align}

where $G$ and $F$ are continuous cumulative distribution functions. The Kolmogorov-Smirnov test is testing
begin{align}
H_0&:F(x) = G(x)quadtext{for all } xinmathbb{R}\
H_1&:F(x) neq G(x)quadtext{for some } xinmathbb{R}.
end{align}

If the null hypothesis is true, then ${X_i}_{i=1}^N$ and ${Y_j}_{j=1}^M$ are samples from the same distribution. All it takes for the $X_i$ and the $Y_j$ to be draws from different distributions is for $F$ and $G$ to differ by any amount at at least one $x$ value. So the KS test is estimating $F$ and $G$ with the empirical CDFs of each sample, honing in on the largest pointwise difference between the two, and asking if that difference is "big enough" to conclude that $F(x)neq G(x)$ at some $xinmathbb{R}$.






share|cite|improve this answer






























    up vote
    8
    down vote













    An intuitive take:



    The Kolmogorov-Smirnov test relies pretty fundamentally on the ordering of observations by distribution. The logic is that if the two underlying distributions are the same, then—dependent on sample sizes—the ordering should be pretty well shuffled between the two.



    If the sample ordering is "unshuffled" in an extreme enough fashion (e.g., all or most the observations in distribution $Y$ come before the observations in distribution $X$, which would make the $D$ statistic much larger), that is taken as evidence that the null hypothesis that the underlying distributions are not identical.



    If the two sample distributions are well shuffled, then $D$ won't have an opportunity to get very big because the ordered values of $X$ and $Y$ will tend to track along with one another, and you won't have enough evidence to reject the null.






    share|cite|improve this answer





















      Your Answer





      StackExchange.ifUsing("editor", function () {
      return StackExchange.using("mathjaxEditing", function () {
      StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
      StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
      });
      });
      }, "mathjax-editing");

      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "65"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f379235%2fwhy-does-the-kolmogorov-smirnov-test-work%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      3 Answers
      3






      active

      oldest

      votes








      3 Answers
      3






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      9
      down vote













      Basically, the test is consistent as a direct result of the Glivenko Cantelli theorem, one of the most important results of empirical processes and maybe statistics.



      GC tells us that the Kolmogorov Smirnov test statistic goes to 0 as $n rightarrow infty$ under the null hypothesis. It may seem intuitive until you grapple with real analysis and limit theorems. This is a revelation because the process can be thought of as an uncountably infinite number of random processes, so the laws or probability would lead one to believe that there is always one point which could exceed any epsilon boundary but no, the supremum will converge in the long run.



      How long? Mmyyeeaa I don't know. The power of the test is kind of dubious. I'd never use it in reality.



      http://www.math.utah.edu/~davar/ps-pdf-files/Kolmogorov-Smirnov.pdf






      share|cite|improve this answer

















      • 2




        +1 Hi AdamO! Got a one to two sentence take on the power being "kind of dubious?" I would love that perspective (I have gathered that the test is considered easily "overpowered").
        – Alexis
        Nov 28 at 17:59








      • 1




        @Alexis The test is not overpowered, IRL we almost never expect the null to be true, rather we just don't care whether the 99.999-th percentile differs by 0.1 between $F_1$ and $F_2$., so whenever I see $p > 0.05$ from the KS test, all I think is, "that's a false negative" and whenever I see $p < 0.05$ I think "whoop-dee-do so what can you say about that?". Tests of the strong null hypothesis $F_1 = F_2$ aren't a compelling way of presenting scientific evidence.
        – AdamO
        Nov 28 at 19:18








      • 1




        Ok. I get yer concern with hypothesis tests for difference. But does your concern about power arise from the simple ontological belief that $F_{1}$ almost surely $ne F_{2}$? or is there something more mathy about asymptotics or something else in there?
        – Alexis
        Nov 28 at 19:49






      • 1




        @Alexis no, I have no concerns with the mathematics of the test. In fact, I think it's quite elegant and the limit theorem result is very impressive.
        – AdamO
        Nov 29 at 4:15






      • 2




        @Alexis I will say, in settings where it is possible for $F_1$ to be exactly equal to $F_2$, the test can be pretty handy. I agree that not a lot of substantive scientific applications fit that bill, but in a statistical computing context where you want to validate that some software you've written is generating pseudo random numbers from some known distribution, it's quite useful. It effectively codifies the intuition you'd get from looking at probability plots.
        – bamts
        2 days ago















      up vote
      9
      down vote













      Basically, the test is consistent as a direct result of the Glivenko Cantelli theorem, one of the most important results of empirical processes and maybe statistics.



      GC tells us that the Kolmogorov Smirnov test statistic goes to 0 as $n rightarrow infty$ under the null hypothesis. It may seem intuitive until you grapple with real analysis and limit theorems. This is a revelation because the process can be thought of as an uncountably infinite number of random processes, so the laws or probability would lead one to believe that there is always one point which could exceed any epsilon boundary but no, the supremum will converge in the long run.



      How long? Mmyyeeaa I don't know. The power of the test is kind of dubious. I'd never use it in reality.



      http://www.math.utah.edu/~davar/ps-pdf-files/Kolmogorov-Smirnov.pdf






      share|cite|improve this answer

















      • 2




        +1 Hi AdamO! Got a one to two sentence take on the power being "kind of dubious?" I would love that perspective (I have gathered that the test is considered easily "overpowered").
        – Alexis
        Nov 28 at 17:59








      • 1




        @Alexis The test is not overpowered, IRL we almost never expect the null to be true, rather we just don't care whether the 99.999-th percentile differs by 0.1 between $F_1$ and $F_2$., so whenever I see $p > 0.05$ from the KS test, all I think is, "that's a false negative" and whenever I see $p < 0.05$ I think "whoop-dee-do so what can you say about that?". Tests of the strong null hypothesis $F_1 = F_2$ aren't a compelling way of presenting scientific evidence.
        – AdamO
        Nov 28 at 19:18








      • 1




        Ok. I get yer concern with hypothesis tests for difference. But does your concern about power arise from the simple ontological belief that $F_{1}$ almost surely $ne F_{2}$? or is there something more mathy about asymptotics or something else in there?
        – Alexis
        Nov 28 at 19:49






      • 1




        @Alexis no, I have no concerns with the mathematics of the test. In fact, I think it's quite elegant and the limit theorem result is very impressive.
        – AdamO
        Nov 29 at 4:15






      • 2




        @Alexis I will say, in settings where it is possible for $F_1$ to be exactly equal to $F_2$, the test can be pretty handy. I agree that not a lot of substantive scientific applications fit that bill, but in a statistical computing context where you want to validate that some software you've written is generating pseudo random numbers from some known distribution, it's quite useful. It effectively codifies the intuition you'd get from looking at probability plots.
        – bamts
        2 days ago













      up vote
      9
      down vote










      up vote
      9
      down vote









      Basically, the test is consistent as a direct result of the Glivenko Cantelli theorem, one of the most important results of empirical processes and maybe statistics.



      GC tells us that the Kolmogorov Smirnov test statistic goes to 0 as $n rightarrow infty$ under the null hypothesis. It may seem intuitive until you grapple with real analysis and limit theorems. This is a revelation because the process can be thought of as an uncountably infinite number of random processes, so the laws or probability would lead one to believe that there is always one point which could exceed any epsilon boundary but no, the supremum will converge in the long run.



      How long? Mmyyeeaa I don't know. The power of the test is kind of dubious. I'd never use it in reality.



      http://www.math.utah.edu/~davar/ps-pdf-files/Kolmogorov-Smirnov.pdf






      share|cite|improve this answer












      Basically, the test is consistent as a direct result of the Glivenko Cantelli theorem, one of the most important results of empirical processes and maybe statistics.



      GC tells us that the Kolmogorov Smirnov test statistic goes to 0 as $n rightarrow infty$ under the null hypothesis. It may seem intuitive until you grapple with real analysis and limit theorems. This is a revelation because the process can be thought of as an uncountably infinite number of random processes, so the laws or probability would lead one to believe that there is always one point which could exceed any epsilon boundary but no, the supremum will converge in the long run.



      How long? Mmyyeeaa I don't know. The power of the test is kind of dubious. I'd never use it in reality.



      http://www.math.utah.edu/~davar/ps-pdf-files/Kolmogorov-Smirnov.pdf







      share|cite|improve this answer












      share|cite|improve this answer



      share|cite|improve this answer










      answered Nov 28 at 17:18









      AdamO

      32k257136




      32k257136








      • 2




        +1 Hi AdamO! Got a one to two sentence take on the power being "kind of dubious?" I would love that perspective (I have gathered that the test is considered easily "overpowered").
        – Alexis
        Nov 28 at 17:59








      • 1




        @Alexis The test is not overpowered, IRL we almost never expect the null to be true, rather we just don't care whether the 99.999-th percentile differs by 0.1 between $F_1$ and $F_2$., so whenever I see $p > 0.05$ from the KS test, all I think is, "that's a false negative" and whenever I see $p < 0.05$ I think "whoop-dee-do so what can you say about that?". Tests of the strong null hypothesis $F_1 = F_2$ aren't a compelling way of presenting scientific evidence.
        – AdamO
        Nov 28 at 19:18








      • 1




        Ok. I get yer concern with hypothesis tests for difference. But does your concern about power arise from the simple ontological belief that $F_{1}$ almost surely $ne F_{2}$? or is there something more mathy about asymptotics or something else in there?
        – Alexis
        Nov 28 at 19:49






      • 1




        @Alexis no, I have no concerns with the mathematics of the test. In fact, I think it's quite elegant and the limit theorem result is very impressive.
        – AdamO
        Nov 29 at 4:15






      • 2




        @Alexis I will say, in settings where it is possible for $F_1$ to be exactly equal to $F_2$, the test can be pretty handy. I agree that not a lot of substantive scientific applications fit that bill, but in a statistical computing context where you want to validate that some software you've written is generating pseudo random numbers from some known distribution, it's quite useful. It effectively codifies the intuition you'd get from looking at probability plots.
        – bamts
        2 days ago














      • 2




        +1 Hi AdamO! Got a one to two sentence take on the power being "kind of dubious?" I would love that perspective (I have gathered that the test is considered easily "overpowered").
        – Alexis
        Nov 28 at 17:59








      • 1




        @Alexis The test is not overpowered, IRL we almost never expect the null to be true, rather we just don't care whether the 99.999-th percentile differs by 0.1 between $F_1$ and $F_2$., so whenever I see $p > 0.05$ from the KS test, all I think is, "that's a false negative" and whenever I see $p < 0.05$ I think "whoop-dee-do so what can you say about that?". Tests of the strong null hypothesis $F_1 = F_2$ aren't a compelling way of presenting scientific evidence.
        – AdamO
        Nov 28 at 19:18








      • 1




        Ok. I get yer concern with hypothesis tests for difference. But does your concern about power arise from the simple ontological belief that $F_{1}$ almost surely $ne F_{2}$? or is there something more mathy about asymptotics or something else in there?
        – Alexis
        Nov 28 at 19:49






      • 1




        @Alexis no, I have no concerns with the mathematics of the test. In fact, I think it's quite elegant and the limit theorem result is very impressive.
        – AdamO
        Nov 29 at 4:15






      • 2




        @Alexis I will say, in settings where it is possible for $F_1$ to be exactly equal to $F_2$, the test can be pretty handy. I agree that not a lot of substantive scientific applications fit that bill, but in a statistical computing context where you want to validate that some software you've written is generating pseudo random numbers from some known distribution, it's quite useful. It effectively codifies the intuition you'd get from looking at probability plots.
        – bamts
        2 days ago








      2




      2




      +1 Hi AdamO! Got a one to two sentence take on the power being "kind of dubious?" I would love that perspective (I have gathered that the test is considered easily "overpowered").
      – Alexis
      Nov 28 at 17:59






      +1 Hi AdamO! Got a one to two sentence take on the power being "kind of dubious?" I would love that perspective (I have gathered that the test is considered easily "overpowered").
      – Alexis
      Nov 28 at 17:59






      1




      1




      @Alexis The test is not overpowered, IRL we almost never expect the null to be true, rather we just don't care whether the 99.999-th percentile differs by 0.1 between $F_1$ and $F_2$., so whenever I see $p > 0.05$ from the KS test, all I think is, "that's a false negative" and whenever I see $p < 0.05$ I think "whoop-dee-do so what can you say about that?". Tests of the strong null hypothesis $F_1 = F_2$ aren't a compelling way of presenting scientific evidence.
      – AdamO
      Nov 28 at 19:18






      @Alexis The test is not overpowered, IRL we almost never expect the null to be true, rather we just don't care whether the 99.999-th percentile differs by 0.1 between $F_1$ and $F_2$., so whenever I see $p > 0.05$ from the KS test, all I think is, "that's a false negative" and whenever I see $p < 0.05$ I think "whoop-dee-do so what can you say about that?". Tests of the strong null hypothesis $F_1 = F_2$ aren't a compelling way of presenting scientific evidence.
      – AdamO
      Nov 28 at 19:18






      1




      1




      Ok. I get yer concern with hypothesis tests for difference. But does your concern about power arise from the simple ontological belief that $F_{1}$ almost surely $ne F_{2}$? or is there something more mathy about asymptotics or something else in there?
      – Alexis
      Nov 28 at 19:49




      Ok. I get yer concern with hypothesis tests for difference. But does your concern about power arise from the simple ontological belief that $F_{1}$ almost surely $ne F_{2}$? or is there something more mathy about asymptotics or something else in there?
      – Alexis
      Nov 28 at 19:49




      1




      1




      @Alexis no, I have no concerns with the mathematics of the test. In fact, I think it's quite elegant and the limit theorem result is very impressive.
      – AdamO
      Nov 29 at 4:15




      @Alexis no, I have no concerns with the mathematics of the test. In fact, I think it's quite elegant and the limit theorem result is very impressive.
      – AdamO
      Nov 29 at 4:15




      2




      2




      @Alexis I will say, in settings where it is possible for $F_1$ to be exactly equal to $F_2$, the test can be pretty handy. I agree that not a lot of substantive scientific applications fit that bill, but in a statistical computing context where you want to validate that some software you've written is generating pseudo random numbers from some known distribution, it's quite useful. It effectively codifies the intuition you'd get from looking at probability plots.
      – bamts
      2 days ago




      @Alexis I will say, in settings where it is possible for $F_1$ to be exactly equal to $F_2$, the test can be pretty handy. I agree that not a lot of substantive scientific applications fit that bill, but in a statistical computing context where you want to validate that some software you've written is generating pseudo random numbers from some known distribution, it's quite useful. It effectively codifies the intuition you'd get from looking at probability plots.
      – bamts
      2 days ago












      up vote
      9
      down vote













      We have two independent, univariate samples:



      begin{align}
      X_1,,X_2,,...,,X_N&overset{iid}{sim}F\
      Y_1,,Y_2,,...,,Y_M&overset{iid}{sim}G,
      end{align}

      where $G$ and $F$ are continuous cumulative distribution functions. The Kolmogorov-Smirnov test is testing
      begin{align}
      H_0&:F(x) = G(x)quadtext{for all } xinmathbb{R}\
      H_1&:F(x) neq G(x)quadtext{for some } xinmathbb{R}.
      end{align}

      If the null hypothesis is true, then ${X_i}_{i=1}^N$ and ${Y_j}_{j=1}^M$ are samples from the same distribution. All it takes for the $X_i$ and the $Y_j$ to be draws from different distributions is for $F$ and $G$ to differ by any amount at at least one $x$ value. So the KS test is estimating $F$ and $G$ with the empirical CDFs of each sample, honing in on the largest pointwise difference between the two, and asking if that difference is "big enough" to conclude that $F(x)neq G(x)$ at some $xinmathbb{R}$.






      share|cite|improve this answer



























        up vote
        9
        down vote













        We have two independent, univariate samples:



        begin{align}
        X_1,,X_2,,...,,X_N&overset{iid}{sim}F\
        Y_1,,Y_2,,...,,Y_M&overset{iid}{sim}G,
        end{align}

        where $G$ and $F$ are continuous cumulative distribution functions. The Kolmogorov-Smirnov test is testing
        begin{align}
        H_0&:F(x) = G(x)quadtext{for all } xinmathbb{R}\
        H_1&:F(x) neq G(x)quadtext{for some } xinmathbb{R}.
        end{align}

        If the null hypothesis is true, then ${X_i}_{i=1}^N$ and ${Y_j}_{j=1}^M$ are samples from the same distribution. All it takes for the $X_i$ and the $Y_j$ to be draws from different distributions is for $F$ and $G$ to differ by any amount at at least one $x$ value. So the KS test is estimating $F$ and $G$ with the empirical CDFs of each sample, honing in on the largest pointwise difference between the two, and asking if that difference is "big enough" to conclude that $F(x)neq G(x)$ at some $xinmathbb{R}$.






        share|cite|improve this answer

























          up vote
          9
          down vote










          up vote
          9
          down vote









          We have two independent, univariate samples:



          begin{align}
          X_1,,X_2,,...,,X_N&overset{iid}{sim}F\
          Y_1,,Y_2,,...,,Y_M&overset{iid}{sim}G,
          end{align}

          where $G$ and $F$ are continuous cumulative distribution functions. The Kolmogorov-Smirnov test is testing
          begin{align}
          H_0&:F(x) = G(x)quadtext{for all } xinmathbb{R}\
          H_1&:F(x) neq G(x)quadtext{for some } xinmathbb{R}.
          end{align}

          If the null hypothesis is true, then ${X_i}_{i=1}^N$ and ${Y_j}_{j=1}^M$ are samples from the same distribution. All it takes for the $X_i$ and the $Y_j$ to be draws from different distributions is for $F$ and $G$ to differ by any amount at at least one $x$ value. So the KS test is estimating $F$ and $G$ with the empirical CDFs of each sample, honing in on the largest pointwise difference between the two, and asking if that difference is "big enough" to conclude that $F(x)neq G(x)$ at some $xinmathbb{R}$.






          share|cite|improve this answer














          We have two independent, univariate samples:



          begin{align}
          X_1,,X_2,,...,,X_N&overset{iid}{sim}F\
          Y_1,,Y_2,,...,,Y_M&overset{iid}{sim}G,
          end{align}

          where $G$ and $F$ are continuous cumulative distribution functions. The Kolmogorov-Smirnov test is testing
          begin{align}
          H_0&:F(x) = G(x)quadtext{for all } xinmathbb{R}\
          H_1&:F(x) neq G(x)quadtext{for some } xinmathbb{R}.
          end{align}

          If the null hypothesis is true, then ${X_i}_{i=1}^N$ and ${Y_j}_{j=1}^M$ are samples from the same distribution. All it takes for the $X_i$ and the $Y_j$ to be draws from different distributions is for $F$ and $G$ to differ by any amount at at least one $x$ value. So the KS test is estimating $F$ and $G$ with the empirical CDFs of each sample, honing in on the largest pointwise difference between the two, and asking if that difference is "big enough" to conclude that $F(x)neq G(x)$ at some $xinmathbb{R}$.







          share|cite|improve this answer














          share|cite|improve this answer



          share|cite|improve this answer








          edited Nov 29 at 0:52

























          answered Nov 28 at 17:18









          bamts

          623310




          623310






















              up vote
              8
              down vote













              An intuitive take:



              The Kolmogorov-Smirnov test relies pretty fundamentally on the ordering of observations by distribution. The logic is that if the two underlying distributions are the same, then—dependent on sample sizes—the ordering should be pretty well shuffled between the two.



              If the sample ordering is "unshuffled" in an extreme enough fashion (e.g., all or most the observations in distribution $Y$ come before the observations in distribution $X$, which would make the $D$ statistic much larger), that is taken as evidence that the null hypothesis that the underlying distributions are not identical.



              If the two sample distributions are well shuffled, then $D$ won't have an opportunity to get very big because the ordered values of $X$ and $Y$ will tend to track along with one another, and you won't have enough evidence to reject the null.






              share|cite|improve this answer

























                up vote
                8
                down vote













                An intuitive take:



                The Kolmogorov-Smirnov test relies pretty fundamentally on the ordering of observations by distribution. The logic is that if the two underlying distributions are the same, then—dependent on sample sizes—the ordering should be pretty well shuffled between the two.



                If the sample ordering is "unshuffled" in an extreme enough fashion (e.g., all or most the observations in distribution $Y$ come before the observations in distribution $X$, which would make the $D$ statistic much larger), that is taken as evidence that the null hypothesis that the underlying distributions are not identical.



                If the two sample distributions are well shuffled, then $D$ won't have an opportunity to get very big because the ordered values of $X$ and $Y$ will tend to track along with one another, and you won't have enough evidence to reject the null.






                share|cite|improve this answer























                  up vote
                  8
                  down vote










                  up vote
                  8
                  down vote









                  An intuitive take:



                  The Kolmogorov-Smirnov test relies pretty fundamentally on the ordering of observations by distribution. The logic is that if the two underlying distributions are the same, then—dependent on sample sizes—the ordering should be pretty well shuffled between the two.



                  If the sample ordering is "unshuffled" in an extreme enough fashion (e.g., all or most the observations in distribution $Y$ come before the observations in distribution $X$, which would make the $D$ statistic much larger), that is taken as evidence that the null hypothesis that the underlying distributions are not identical.



                  If the two sample distributions are well shuffled, then $D$ won't have an opportunity to get very big because the ordered values of $X$ and $Y$ will tend to track along with one another, and you won't have enough evidence to reject the null.






                  share|cite|improve this answer












                  An intuitive take:



                  The Kolmogorov-Smirnov test relies pretty fundamentally on the ordering of observations by distribution. The logic is that if the two underlying distributions are the same, then—dependent on sample sizes—the ordering should be pretty well shuffled between the two.



                  If the sample ordering is "unshuffled" in an extreme enough fashion (e.g., all or most the observations in distribution $Y$ come before the observations in distribution $X$, which would make the $D$ statistic much larger), that is taken as evidence that the null hypothesis that the underlying distributions are not identical.



                  If the two sample distributions are well shuffled, then $D$ won't have an opportunity to get very big because the ordered values of $X$ and $Y$ will tend to track along with one another, and you won't have enough evidence to reject the null.







                  share|cite|improve this answer












                  share|cite|improve this answer



                  share|cite|improve this answer










                  answered Nov 28 at 17:55









                  Alexis

                  15.6k34595




                  15.6k34595






























                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Cross Validated!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      Use MathJax to format equations. MathJax reference.


                      To learn more, see our tips on writing great answers.





                      Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                      Please pay close attention to the following guidance:


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f379235%2fwhy-does-the-kolmogorov-smirnov-test-work%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      AnyDesk - Fatal Program Failure

                      How to calibrate 16:9 built-in touch-screen to a 4:3 resolution?

                      QoS: MAC-Priority for clients behind a repeater