Benfords Law and chi2 test

[noTh1ng]

Hi all,

I have a question concerning benfords law and chi2 test. I want to check a dataset for conformity to benfords law and so far have helped myself using "firstdigit" and "digdis". However those two do not allow me to plot a graph that shows the logarithmic distribution benford laws follow, but only the result of my variable.

furthermore i just want to generally know how to perform chi2 test in that respect. I do not know how to test "against" benfords law, as i dont have a specific variable for it and i am very unsure in how to create such?

Thanks for your help! If I did not express myself clearly feel free to ask!

[Nick Cox]

Some matters of form, please:

1. Full real names are requested. Names like "noThing" are not impressive or amusing here.

2. You are asked to explain where user-written programs you refer to come from. Mentioning firstdigit and digdis is one thing, but explaining that they come from SSC allows experienced users to go directly to a download.

3. Attention to spelling and punctuation is also requested; thus Benford's Law is the standard form.

All these are explained in the advice guide under the FAQ.

Here is a silly example but more serious technique:

sysuse auto, clear
set scheme s1color 
gen firstdigit = real(substr(string(price), 1, 1))
contract firstdigit
set obs 9 
gen x = _n 
gen expected = log10(1 + 1/x) 
twoway histogram firstdigit [fw=_freq], barw(0.5) bfcolor(ltblue) blcolor(navy) discrete fraction ///
|| connected expected x, xla(1/9) ytitle("observed and expected" "probabilities under Benford's Law") ///
yla(, ang(h) format("%02.1f")) legend(off)

[noTh1ng]

Hi Nick,

thanks for your answer. I am sorry, I obviously should have read the FAQ first. During registration it did not state any information about using my "real-name", but suggested to use any nick I favor.

I will look at your code, but honestly believed that there is an easier way to test variables for conformity of Benford's Law using chi2-test, especially for beginners like me!
Ben

[Nick Cox]

Thanks for your promise of attention to procedure. The welcome page advises reading the FAQ before posting.

Otherwise I don't understand this answer: firstdigit (SSC)has the chi-square test wired in, and that's one of its main purposes, and digdis (SSC) is much more versatile yet. So, all the work you need is long since done.

[Nick Cox]

Attention to spelling requires that I apologise for misreading your name. It's noTh1ng, not noThing.

[noTh1ng]

Attention to spelling requires that I apologise for misreading your name. It's noTh1ng, not noThing.

I understand your issue with my nickname, however I can not change it.

My general problem is that digdis and firstdigit results are not handy, as I want to show the significance of my results. If I could use the regular chi2 test my results would be convertable to outreg, which would be able to present me a nicer output.
Ben

[Nick Cox]

You can change your identifier by contacting the administrators. See the home page, bottom right-hand corner ("Contact us").

I claim responsibility for firstdigit (SSC). My contribution, such as it was, was programming the test. I like to think that users should and can take care of exactly how they want to present results. As the help for firstdigit indicates, you can retrieve its results as saved results (type return list to see what they are). I don't use outreg (SJ) but a program such as mine that leaves results in its wake is consistent with many programs that collate results, or with your own code looping over possibilities and producing combined tables or graphs.

Very similar comments should apply to digdis (SJ).

Ultimately the program doexactlywhatiwant has to be one you write yourself!

[Nick Cox]

digdis is on SSC, not SJ.

[Maarten Buis]

Ultimately the program doexactlywhatiwant has to be one you write yourself!

This is an important reason why many user-written programs get written in the first place; someone wants Stata to do something and writes the program, (s)he then also makes that program available to others because (s)he is a nice person and wants to safe others some time, but this release to the public is just a secondary result.

[Alan Riley (StataCorp)]

During registration it did not state any information about using my "real-name", but suggested to use any nick I favor.

The registration page, immediately under the field where you enter your username, requests that you use your real name.

I understand your issue with my nickname, however I can not change it.

Please go to the Contact Us page and request a username change. An administrator will be happy to change it for you.

[Stephanie Retz]

Dear Nick,

I am also working with your program firstdigit (SSC). Could you tell me which chi-squared test the program uses and what the null hypothesis H0 is?

I am assuming H0 is "the first digits are Benford-distributed" and the alternate hypothesis Ha is "the first digits are not Benford-distributed"? Hence, if the p-value is above a certain value (e.g. 0.05), the H0 cannot be rejected and I can assume that my data satisfies Benford's Law?

Is it possible to do a chi-squared test of the law based on your example in comment #2? E.g. with the command chitest? I am curious on what exactly firstdigit does and would like to try it out myself "manually".

Best regards,
Stephanie

[Nick Cox]

You can always study the code as well as the help.

The help says

firstdigit tabulates and analyses the first digits of numeric variables. It also tests
Benford's law that digits d = 1,..,9 occur with probabilities log10(1 + 1/d). Thus
given data of 12, 345, 6789, etc., it would extract 1, 3, 6, etc., tabulate the
frequencies of the digits 1 to 9 and give a chi-square test of the law. Use return list
to see returned results.

and I am sorry if that seems unclear. But the hypotheses are exactly as you state and for a manual comparison you can use Mata as in the code. Note that firstdigit includes all digits 1 to 9 regardless of whether they are observed.

It's a common-or-garden Pearson-style chi-square test (not likelihood ratio).

viewsource firstdigit.ado

shows you the code.

[Stephanie Retz]

Thanks Nick!

I had read the help but was not familiar with mata and the viewsource command - that's great.