Results 1 to 5 of 5

20180507, 02:15 PM (ISO 8601)
 Join Date
 Jan 2009
F statistic/distribution question
When doing a hypothesis test on an Fstatistic, I know you generally reject the null hypothesis if the Fstatistic is a lot bigger than 1. (How much bigger depending on your alpha.)
If F is really small, like .0001, would that also generally be a good reason to reject the null hypothesis?
This seems to come up less in the examples I've seen, but theorywise that seems just as problematic as a big Fvalue.

20180507, 05:19 PM (ISO 8601)
 Join Date
 May 2007
 Location
 Tail of the Bellcurve
 Gender
Re: F statistic/distribution question
A small F value is in no way problematic under the null hypothesis. Consider the case of testing whether a single parameter is zero; in this case the F statistic is exactly the square of the t statistic for the same test. Is a small t value a problem under the null?
Bloodred were his spurs i' the golden noon; winered was his velvet coat,
When they shot him down on the highway,
Down like a dog on the highway,And he lay in his blood on the highway, with the bunch of lace at his throat.
Alfred Noyes, The Highwayman, 1906.

20180508, 09:54 AM (ISO 8601)
 Join Date
 Jan 2009
Re: F statistic/distribution question
But I thought that, if the parameter is what it is under Ho, the Fvalue would be 1. Thus, wouldn't small be bad (at least in a 2sided test)?
I'm coming from ANOVA and regression viewpoint, if that adds any useful context. So I'm really looking at the test if any of the variables are relevant to the model or not. I think, in that, the Fstatistic is derived by some division being squares of errors or something else put out in the ANOVA table output (though I never fully memorized the details.)
Thus, between a big F and a really small F, in one the denominator is way bigger and in the other the numerator is way bigger. But either should be an issue... right?

20180511, 02:08 PM (ISO 8601)
 Join Date
 May 2007
 Location
 Tail of the Bellcurve
 Gender
Re: F statistic/distribution question
If I'm doing a ttest of mu_1  mu_2 = 0, then although I'm talking about two parameters, the difference between mu_1 and mu_2 is one dimensional. Thus we can say it has sides, because mu_1  mu_2 falls somewhere on the real line, either less than zero, equal to zero (H0) or greater than zero. The sides of the hypothesis test describe which of those deviations I care about.
I can also test this with an F distribution; as I said before the F statistic is exactly the square of the t statistic. Now I'm making a hypothesis about (mu_1  mu_2)^2, which is either zero (H0) or not (HA). There's no sides left, because I'm looking at squared differences in means, and those are always going to be greater than or equal to zero. The F test can't tell me anything about how mu_1 and mu_2 differ, because I take the square. The ttest can, and is therefore generally preferable when doing comparisons of only two means.
In practice, most things you test with an F distribution have complex hypotheses, you can't really say they're one or two sided because they are statements about the relative values of parameters in high dimensional space. The ttest can't do this, it can only test a single comparison at at a time.
Consider the simplest ANOVA scenario that doesn't reduce to a t test. I've got three treatments, and I want to know if they have different means; does mu_1 = mu_2 = mu_3 or not? Now I care about two differences; mu_1  mu_2 and mu_1  mu_3*. Under H0, both these differences are zero. The F test tests whether (mu_1  mu_2)^2 = 0 and (mu_1  mu_3)^2 = 0. Again, there are no sides, both because I've squared things, and because it's not really a meaningful concept in the twodimensional space of differences between three means.
edit: Under H0, the expectation of MSTR, the mean squared treatment error, is sigma^2. The expectation of SSE, sum of squares error is also sigma^2. The test statistic is F = MSTR/MSE, but E(F) = E(MSTR/MSE) != E(MSTR)/E(MSE) under H0. The expectation is actually df_denominator/(df_denominator  2) when df_denominator is > 2, and otherwise the distribution fails to have an expectation**. If you have a lot of error degrees of freedom, this gets very close to one, but it is not one.
*These particular differences are arbitrary. I could also pick mu_1  mu_2 and mu_2  mu_3. Point is, there's only two, and any other difference of treatment means is a linear combination of these two.
**Which is vaguely surprising, until one recalls that a t with df=1 is Cauchy. One need not have a mean to have a valid test statistic and pvalue, because pvalues are probabilities, not statements about the mean of the test statistic.Last edited by warty goblin; 20180511 at 02:16 PM.
Bloodred were his spurs i' the golden noon; winered was his velvet coat,
When they shot him down on the highway,
Down like a dog on the highway,And he lay in his blood on the highway, with the bunch of lace at his throat.
Alfred Noyes, The Highwayman, 1906.

20180517, 11:16 AM (ISO 8601)
 Join Date
 Oct 2010
 Location
 Dallas, TX
 Gender
Re: F statistic/distribution question
It depends on what you're testing. Since most Ftests are ANOVA, it's a onesided test, and your rejection region is only the upper tail. But there are some twotails Ftests.
An F statistic is technically comparing two variances. It's F = s_{1}^{2} / s_{2}^{2}. If you want to test H_{0}: σ_{1}^{2} = σ_{2}^{2} vs. Ha: σ_{1}^{2} ≠ σ_{2}^{2}, then it's a twotailed test, and your rejection region includes both tails at the α/2 level.
But if you are testing H_{0}: σ_{1}^{2} ≤ σ_{2}^{2} vs. Ha: σ_{1}^{2} > σ_{2}^{2}, then it's a righttailed test, and the rejection region is just the upper tail at the α level.
In ANOVA, you are testing to see if the variance between groups is large enough that it cannot be explained away by the variance within groups. Therefore it's a onetailed test.