Wald test when one coefficient is significant and another is insignificant ?

[Mike Kraft]

Hi
I am regressing abnormal returns on set of accounting variables. The regression and the output in Stata are below:

reg AR1 a b c d ,r

Linear regression Number of obs = 56328
F( 4, 56323) = 108.73
Prob > F = 0.0000
R-squared = 0.0251
Root MSE = .8932

------------------------------------------------------------------------------
| Robust
AR1 | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
a .6628295 .0428066 15.48 0.000 .5789282 .7467307
b .4635614 .338008 1.37 0.170 -.1989364 1.126059
c .46176 .0571501 8.08 0.000 .3497455 .5737745
d .1211691 .0120079 10.09 0.000 .0976336 .1447045
_cons .040062 .0038999 10.27 0.000 .0324181 .0477059



It is clear that the coefficient on b is insignificant. I use the following Wald test:

test c=b


F( 1, 56323) = 0.00
Prob > F = 0.9960 // I conclude that both b and c coeff are indifferent


test b=0


F( 1, 56323) = 1.88
Prob > F = 0.1702 // I conclude that b is insignificantly different from zero

test a=c

F( 1, 56323) = 6.28
Prob > F = 0.0122 // I conclude that both a and c and significantly different

test a=b

F( 1, 56323) = 0.34
Prob > F = 0.5591 //// ? ////

Here comes my question; how can I interpret the results that the coeff on b and c are not different and that the coeff on b (which is statistically significant) is different from a, while c (which is statistically insignificant but indifferent from b) is indifferent from a ( that is highly significant in the model) ?? If the coeff on b is statistically insignificant why the Wald test results seems to support its equality with a, while the coefficient on c is statistically significant and similar magnitude to b , but statistically different from a ?

[Carlo Lazzaro]

Mike:
as a temptative answer, I would also consider that your comparisons are not independent. Hence, statistical (in)significance should be read in the light of multiple comparisons.

[Mike Kraft]

Carlo; I am not sure what you mean ! Do you suggest any particular test.
I also do not know why the table format changed...Anyone knows how to report my codes and output more professionally .

[Maarten Buis]

The key issue here is that you failed to reject the hypothesis that the effects of c and b are the same. This is something different than saying that the efects of c and b are the same. With statistical testing we can only reject or fail to reject a hypothesis, but we can never confirm a hypothesis. This is a difference between absense of evidence (fail to reject) and evidence of absense (confirm). So it is very well possible to fail to reject the hypotheses that c and b are the same and a and c are the same and reject the hypothesis that a and b are the same.

[Clyde Schechter]

The problem is the way you are thinking about "significantly different." There is actually nothing at all surprising in these results. (Well, with an N of more than 56,000 it is somewhat surprising that you managed to find any variable whose coefficient is not significantly different from zero--but that's not the issue here.)

The test of c=b doesn't tell you much at all about c and b. In particular, it does not really tell you in a yes/no way whether c = b, or even whether c is approximately equal to b. What it does tell you is that the difference, c-b, is estimated with enough lack of precision that data like this are consistent with its being zero. Similarly, the difference between a and b, b-a is estimated with enough lack of precision that data like this are consistent with its being zero. Note that b's coefficient falls between a and c. So the distance between a and c is the sum of the distance between a and b, and that between b and c. Now, neither of the last two differences is measured precisely enough to say whether they are zero or not. But the distance between a and c is the total of those, and is large enough that despite the imprecision of estimation of a, b, and c, we have enough precision to infer (probabilistically) that a-c is not zero. (That is, these particular data would be unusual if a-c were zero.)

It is always important to bear in mind that "statistical significance" does not mean that something is necessarily big, and it is not about equaling or not equaling zero. It is about whether the data can estimate that something precisely enough to support a probabilistic inference about its being zero.

[Mike Kraft]

If this is the case, how can I conclude these results in wording? I wanted to show that both b and c are seen as indifferent in relation to returns on the LHS, but not sure how to report these results?
Thanks

[Clyde Schechter]

To what kind of audience are you conveying your results? Are they statistically sophisticated, relatively naïve, what?

[Carlo Lazzaro]

Mike:
- in my tempative reply I meant that you might have considered correcting p-values for multiple comparisons, as you constrasted some of your coefficients against each other in more that one instance;
-an intersting reference covering one of the issue that Clyde and Maarten pointed out (the absence of evidence is not the evidence of absence) is: http://www.ncbi.nlm.nih.gov/pmc/arti...0606-0027.pdf;
- the formatting issues you reported can be easily worked around by posting wht you tyoed and what Stata gave you back via code delimiters (#-button among Advanced editor [A-button] options).

[Mike Kraft]

Hi Clyde
The audience are sophisticated finance scholars. The aim is to report that the market seems to price both b and c as being the same (though one is significant and the other is not, but they have similar coeff magnitude).Also, I wanted to say that the market differentiates between both both a in one side and b and c in another side. However, the fact that a and b and different but a and c are not makes me uncomfortable and can not really proceed !
Do you suggest any way to report these ?

[Clyde Schechter]

Well, I think what you are aiming to report goes a bit beyond what the data substantiate. The main difficulty arises because b is estimated with rather low precision. You can certainly say that in terms of magnitude of coefficient, a looks rather different from b and c, whereas b and c are close to each other. But I think you have to be frank about the vagueness of the information about b. Certainly the data are not very consistent with a and c being equal. But the data are quite consistent with b being anywhere between a and c, or even on the far side of either of them. We just don't learn much about b from this data.

If there is some reasonably strong prior information about b from other sources, then you might consider a Bayesian approach. If not, I'm afraid you simply can't draw the conclusion that you would like to draw here.

You have a very large sample size, so it is unlikely that getting more data of the same kind will help you. Perhaps there is a different source of data in which the variable b is measured more accurately, or perhaps there are some other covariates that could be added to the model that might lead to a more precise estimate of the b coefficient. I can't think of much else to suggest.