Results 1 to 5 of 5
  1. - Top - End - #1
    Ogre in the Playground
     
    BardGuy

    Join Date
    Jan 2009

    Default F statistic/distribution question

    When doing a hypothesis test on an F-statistic, I know you generally reject the null hypothesis if the F-statistic is a lot bigger than 1. (How much bigger depending on your alpha.)

    If F is really small, like .0001, would that also generally be a good reason to reject the null hypothesis?
    This seems to come up less in the examples I've seen, but theory-wise that seems just as problematic as a big F-value.

  2. - Top - End - #2
    Titan in the Playground
    Join Date
    May 2007
    Location
    Tail of the Bellcurve
    Gender
    Male

    Default Re: F statistic/distribution question

    A small F value is in no way problematic under the null hypothesis. Consider the case of testing whether a single parameter is zero; in this case the F statistic is exactly the square of the t statistic for the same test. Is a small t value a problem under the null?
    Blood-red were his spurs i' the golden noon; wine-red was his velvet coat,
    When they shot him down on the highway,
    Down like a dog on the highway,
    And he lay in his blood on the highway, with the bunch of lace at his throat.


    Alfred Noyes, The Highwayman, 1906.

  3. - Top - End - #3
    Ogre in the Playground
     
    BardGuy

    Join Date
    Jan 2009

    Default Re: F statistic/distribution question

    But I thought that, if the parameter is what it is under Ho, the F-value would be 1. Thus, wouldn't small be bad (at least in a 2-sided test)?

    I'm coming from ANOVA and regression viewpoint, if that adds any useful context. So I'm really looking at the test if any of the variables are relevant to the model or not. I think, in that, the F-statistic is derived by some division being squares of errors or something else put out in the ANOVA table output (though I never fully memorized the details.)
    Thus, between a big F and a really small F, in one the denominator is way bigger and in the other the numerator is way bigger. But either should be an issue... right?

  4. - Top - End - #4
    Titan in the Playground
    Join Date
    May 2007
    Location
    Tail of the Bellcurve
    Gender
    Male

    Default Re: F statistic/distribution question

    If I'm doing a t-test of mu_1 - mu_2 = 0, then although I'm talking about two parameters, the difference between mu_1 and mu_2 is one dimensional. Thus we can say it has sides, because mu_1 - mu_2 falls somewhere on the real line, either less than zero, equal to zero (H0) or greater than zero. The sides of the hypothesis test describe which of those deviations I care about.

    I can also test this with an F distribution; as I said before the F statistic is exactly the square of the t statistic. Now I'm making a hypothesis about (mu_1 - mu_2)^2, which is either zero (H0) or not (HA). There's no sides left, because I'm looking at squared differences in means, and those are always going to be greater than or equal to zero. The F test can't tell me anything about how mu_1 and mu_2 differ, because I take the square. The t-test can, and is therefore generally preferable when doing comparisons of only two means.

    In practice, most things you test with an F distribution have complex hypotheses, you can't really say they're one or two sided because they are statements about the relative values of parameters in high dimensional space. The t-test can't do this, it can only test a single comparison at at a time.

    Consider the simplest ANOVA scenario that doesn't reduce to a t test. I've got three treatments, and I want to know if they have different means; does mu_1 = mu_2 = mu_3 or not? Now I care about two differences; mu_1 - mu_2 and mu_1 - mu_3*. Under H0, both these differences are zero. The F test tests whether (mu_1 - mu_2)^2 = 0 and (mu_1 - mu_3)^2 = 0. Again, there are no sides, both because I've squared things, and because it's not really a meaningful concept in the two-dimensional space of differences between three means.


    edit: Under H0, the expectation of MSTR, the mean squared treatment error, is sigma^2. The expectation of SSE, sum of squares error is also sigma^2. The test statistic is F = MSTR/MSE, but E(F) = E(MSTR/MSE) != E(MSTR)/E(MSE) under H0. The expectation is actually df_denominator/(df_denominator - 2) when df_denominator is > 2, and otherwise the distribution fails to have an expectation**. If you have a lot of error degrees of freedom, this gets very close to one, but it is not one.

    *These particular differences are arbitrary. I could also pick mu_1 - mu_2 and mu_2 - mu_3. Point is, there's only two, and any other difference of treatment means is a linear combination of these two.

    **Which is vaguely surprising, until one recalls that a t with df=1 is Cauchy. One need not have a mean to have a valid test statistic and p-value, because p-values are probabilities, not statements about the mean of the test statistic.
    Last edited by warty goblin; 2018-05-11 at 02:16 PM.
    Blood-red were his spurs i' the golden noon; wine-red was his velvet coat,
    When they shot him down on the highway,
    Down like a dog on the highway,
    And he lay in his blood on the highway, with the bunch of lace at his throat.


    Alfred Noyes, The Highwayman, 1906.

  5. - Top - End - #5
    Titan in the Playground
     
    Jay R's Avatar

    Join Date
    Oct 2010
    Location
    Dallas, TX
    Gender
    Male

    Default Re: F statistic/distribution question

    It depends on what you're testing. Since most F-tests are ANOVA, it's a one-sided test, and your rejection region is only the upper tail. But there are some two-tails F-tests.

    An F statistic is technically comparing two variances. It's F = s12 / s22. If you want to test H0: σ12 = σ22 vs. Ha: σ12 ≠ σ22, then it's a two-tailed test, and your rejection region includes both tails at the α/2 level.

    But if you are testing H0: σ12 ≤ σ22 vs. Ha: σ12 > σ22, then it's a right-tailed test, and the rejection region is just the upper tail at the α level.

    In ANOVA, you are testing to see if the variance between groups is large enough that it cannot be explained away by the variance within groups. Therefore it's a one-tailed test.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •