Test frameworks/best-practices for Stata commands

[Kristoffer Bjarkefur]

Hi Statalist,

I work with a team that generates Stata tools for our team that we are also sharing publicly, either through SSC like this or directly through GitHub like this. As both those packages and the team maintaining them gets bigger, we are looking into some best practices for testing code and perhaps even more general code-testing frameworks to help ensure that new updates to large commands do not cause issues in other features of the command.

There are many tools like this in other programming languages, but those are often language-specific. We cannot be the first ones to have thought of this in the Stata community. Do any of you have some best practices for code testing you could share? If you do not have any input but this might be something you have been thinking about as well, feel free to just hit the subscribe button and we will be happy to post material or links to material that we find.

Any tips and advice will be appreciated! Thanks!

[daniel klein]

See Gould (2001). See also

help cscript

Best
Daniel


Gould, W. 2001. Statistical software certification. The Stata Journal, 1(1), pp. 29-50.

[Nick Cox]

As with much advice, note what I recommend rather than whether it's what I have always done in the past. (It isn't!)

0. Authors get to stipulate their own expectations and users just need to respect that, or move on and use something else, or indeed write their own code.

1. Every command should have a certification script that should run without error after each revision and be based on datasets supplied with the script or available to any Stata user.

2. All code should be accessible to all users. Even if a user has no interest in studying the code, it is often the case that the community gets asked questions about code from authors who are no longer active or even visible and programmers have little or no chance of commenting usefully if any code is hidden. This rules out circulating compiled Mata code without the source.

3. All dependencies on other community-contributed commands must be explicit.

4. The documentation requires as much effort as the code. In some quite different communities the tacit preference is give the minimum the user needs to know and little else and the tacit guideline is that it's not a programmer's job to explain the statistics (or whatever else it is)! In contrast, it can help the programmer as much as users to use the help file to include discussions, a wealth of examples and detailed references.

[Kristoffer Bjarkefur]

Thank you Daniel and Nick! This is really useful! We will read up and implement!