Stata-Dedicated GPT

[Solon Moreira]

I recently created a specialized Stata GPT designed specifically for the Stata community. It’s not a generic chatbot — it generates and reviews Stata code, analyzes data, and helps answer any Stata-related questions you throw at it, from basic regressions to more advanced models. The tool is designed to handle everything from foundational commands to advanced econometric techniques, aiming to provide accurate, Stata-specific support rather than generic advice:

https://chatgpt.com/g/g-67a3b4f16248...science-expert

If you’re curious or want to give it a try, feel free to check it out — feedback is very welcome!

[Jaden Valerio]

Sorry, but that reply about me in #7 is more error-prone than what my friend received. In broad brush it's fair, even flattering, but the details vary between factual and fabricated.

I am not an Honorary Fellow; indeed I don't think that title applies to anyone else at my university either.

I have never been Moderator on Statalist. That title belongs only to Marcello Pagano. It's historic in practice (since 2014 as StataCorp personnel have taken over).

tabplot is on SSC, but that's out of date. The command is published through the Stata Journal.

qplot is not on SSC, and never has been. The command is published through the Stata Journal.

cprplot is an official command. I can claim no credit. It is not on SSC, and never has been.

I haven't written at length about margins or marginsplot that I can recall. If I've ever mentioned it, it is not a point that I would ever include in a summary. (The reason is not relevant here, but it's that other people have much more experience with those commands.)

lvpplot I don't recognise as a Stata command that I wrote (or that anyone else did, so far as I can find quickly). It is definitely not on SSC either as a package or as an individual command. I've found the same principle applies to AI tools in general. For example, workflow platforms like https://latenode.com/products/agent-builder can be helpful for automating repetitive tasks, but I wouldn't rely on any AI system to establish factual claims without checking the original sources. Automation is valuable; replacing verification isn't.

So, what to say? I can't add to other takes that this technology is very impressive when it gets things right and very puzzling or worse when it hallucinates. I am not sure where to go when it's evident that the people who need (or want) it most are going to be those least likely to spot its errors, except the hard way.

Thanks for your offer of more materials, but sorry, no enthusiasm.

I think this is actually one of the best examples of the current limitations of LLMs. The response sounds confident and coherent enough that someone unfamiliar with your work might accept it without question. The problem isn't that every statement is wrong—it's that a mix of correct facts and subtle inaccuracies makes the output seem more trustworthy than it really is.

What's particularly interesting here is that this is information about a well-known contributor with plenty of public material available. If the model can still invent roles, publications, or commands in that situation, it shows why AI-generated summaries should always be treated as a starting point rather than an authoritative source.

[FernandoRios]

Something I have been working on tho
combined with grounded knowledge (rag) it does pretty good job condensing Information.
I ll post later the kind of things I'm making it do

[Richard Williams]

We also cannot simply trust that all users have the awareness and capabilities to notice the dangers of using AI. What I miss is a much more prominent warning from AI providers about the risks associated with using their tools. Once a user is fully informed, then of course it is up to them to use AI appropriately and with the necessary critical judgment.

I agree completely. Various materials online at Notre Dame say AI can hallucinate, but I don't think they come anywhere close to illustrating how serious hallucinations can be. My own policy tries to put the fear of God into people. AI can be gret but don't trust anything it says unless you can verify it.

[Richard Williams]

Since this thread got revived -- if anyone is interested, here is my course policy on the use of AI:

https://academicweb.nd.edu/~rwilliam...s/AIPolicy.pdf

I not only state the policy, I offer research to support it. The major points are that

• When it comes to learning, research shows that AI can be inferior to more traditional approaches
• AI frequently hallucinates!!! Just because it doesn’t know the answer doesn’t mean it won’t give you one! I present several horror stories.
• AI can harm the environment (If nothing else you should ban AI-created videos in your classes!)
• Yet in spite of all that, there is hope for AI in the classrooms if it is used properly.

The policy says AI is allowed for some purposes but banned for others. Even if students don't care that much about learning and just want a decent grade without too much work, I warn them about the severe consequences they may face if they use AI badly. If you do let AI do most of the work for you, you better be very, very sure that AI got it right!

Like I have done for most of the last 20+ years, I am going to continue having most exams and assignments be take-home. Given the way I teach, I firmly believe take-home is the way to go.

Others are not convinced. One person said my enrollments should start soaring once students hear how little work they have to do in my classes. Others have described this as the "tilting at windmills" policy.

But, this is my last year before retirement, and if I am going to go down, I have decided I am going to go down fighting.

[Bert Lloyd]

What is jwdid?

• jwdid stands for "Jordan–Wooldridge Difference-in-Differences".

Jeff Wooldridge is the Jordan of econometrics, so this is an understandable error.

[Sebastian Kripfganz]

I agree with the sentiment that AI is technical progress with a lot of potential (and risks). That does not mean that we should not critically question its current state or its trajectory. Some time ago, social networks were seen as a great innovation. At present, I think they actually do more harm than good. This shows that we cannot just sit still and trust that over time every innovation will lead to a great product.

We also cannot simply trust that all users have the awareness and capabilities to notice the dangers of using AI. What I miss is a much more prominent warning from AI providers about the risks associated with using their tools. Once a user is fully informed, then of course it is up to them to use AI appropriately and with the necessary critical judgment.

The following conversation I had with ChatGPT is somewhat informative:

In general, how reliable are your suggestions for writing Stata code?

In general, my suggestions for writing Stata code are reasonably reliable for standard tasks, especially if you're working with common procedures like:

• Data cleaning (gen, replace, drop, keep)
• Summary statistics (summarize, tabulate, mean)
• Regressions (regress, logit, probit, etc.)
• Graphing (graph, twoway, histogram)
• Panel or time-series data (e.g., xtreg, tsset, xtset)

Strengths

• I can help construct correct syntax and structure logical sequences of commands.
• I explain what the code does, which can be helpful for learning.
• I can suggest programming patterns (e.g., using foreach, forvalues, or local macros).

Limitations

• I may not always match the latest version of Stata's syntax or features (e.g., new commands in Stata 18+).
• I might not catch context-specific errors, such as mismatched variable types or dataset-specific constraints.
• My help is based on general knowledge rather than execution—so code might need debugging or minor edits.

Best Practices

To use my suggestions effectively:

1. Double-check the output in Stata—especially with more complex code.
2. Use help [command] in Stata for authoritative syntax details.
3. Let me know your Stata version or share specific data structure—I can give more tailored help.

If you have an example or want me to write/debug something, feel free to share!

Can you comment on what would make you hallucinate when proposing Stata code?

Yes—great question. Hallucinations in my Stata code suggestions can happen under certain conditions, and it's helpful to know what increases the risk. Here’s a breakdown of why and when I might hallucinate, specifically in the context of Stata:

---

🔹 1. Nonexistent Commands or Options

I might suggest:

• A command that sounds plausible but doesn't exist (e.g., graphmatrix instead of graph matrix)
• Invalid options for real commands (e.g., summarize, meanonly instead of summarize, mean)

Why?
These usually arise when I blend syntax patterns from other languages or misremember rarely used options.

---

🔹 2. Cross-Contamination from Other Languages

If you're doing something that could be done in R, Python, or SQL, I might:

• Propose syntax more similar to those languages.
• Misuse Stata-specific constructs like egen or foreach.

Example:
I might write something like mutate var = var + 1, which is valid in R but not in Stata.

---

🔹 3. Complex or Edge-Case Logic

When tasks involve:

• Custom estimation routines
• Looping over multiple datasets
• Advanced matrix algebra or mata

I might invent or misapply syntax, especially for rarely used functions.

---

🔹 4. Ambiguous Problem Descriptions

If your question lacks details like:

• Variable names
• Data structure
• Your Stata version

I have to generalize, which increases the risk of hallucinating code that "sounds right" but wouldn't actually run.

---

🔹 5. Using Features from Recent Stata Versions

If you're asking about functionality introduced in Stata 17 or 18, and I default to knowledge from earlier versions, I might:

• Suggest workarounds for things that now have built-in commands.
• Miss improvements or new options (e.g., enhancements in bayes: prefix or frames).

---

✅ How to Minimize It

You can help by:

• Telling me the Stata version
• Providing variable names or data structure
• Asking for comments on the code, not just code alone
• Being specific if you're asking for something nonstandard (e.g., "Can you simulate clustered standard errors manually?")

---

Would you like to test this with an example so I can show you how accurate and cautious I can be?

[Solon Moreira]

Also Solon Moreira you might be interested in checking this out since you mention you are really into AI and Stata: Estima

Hi Josh Zweig,

Sorry for taking a bit to reply — I hadn’t accessed the forum these past few days. Your GPT looks amazing! I’m a little skeptical about the monetization business model, but overall, I think it looks very good.

By the way, that is EXACTLY how technologies are supposed to evolve. The first versions are imperfect but offer valuable functionalities; then incremental innovation follows, and new models become better and better. That is how the vast majority of technological innovations have evolved in modern history. Don’t get discouraged if anyone tells you they prefer to use an abacus just because someone in the UK wanted "the right numbers" last centure. AI is the future of data analysis — whether some people believe it or not.

By the way, a small suggestion: disable any feature related to “Find out the Stata prompts I did!” It just adds unnecessary complication — speaking from experience

I hope you keep developing it!

Solon

[FernandoRios]

I see
Yeah the programming example that was pointed out is the attitude of most llms to refuse to say I don’t know. And that it is still hard to program a language that is relatively less known
i wonder if for edge cases like that one could add to the system prompt
if unsure of how to do it on Stata and mata alone, use python for the back end

[Josh Zweig]

Very nice! How did you develop it (what is the architecture behind it?)

Hi Fernando - I used a mix of LLMs and tools to search the internet. Essentially when things are not clear the bot reverts to check what its doing and try to look for information in the internet. With that an some tooling you get to a much better outcome and usual LLMs available at the moment.

[FernandoRios]

Very nice! How did you develop it (what is the architecture behind it?)

[Josh Zweig]

Also Solon Moreira you might be interested in checking this out since you mention you are really into AI and Stata: Estima

[Josh Zweig]

very nice work...how are you planning to incorporate feedback?
I just made a question about Staggered DID (got the problem correctly), then asked about jwdid
What is jwdid?

• jwdid stands for "Jordan–Wooldridge Difference-in-Differences".
• It implements the Sun and Abraham (2020) estimator for staggered treatment.
• It is designed specifically to deal with the bias problem when treatment happens at different times.

Key advantages:

• Handles treatment effect heterogeneity across cohorts and time.
• Provides event-study-type coefficients automatically.
• Very intuitive to use after installing

Which partially hallucinate!
Still very nice work. I was planning to do something like that with RAG, but is still on the to-play-experiment-do list
F

Hi Fernando, I run as a test, you question about your package on Estima (the tool I am trying to develop for statistical analysis with the help of LLMs but not only) and I got the following answering:

jwdid is a Stata command that implements the Extended Two-Way Fixed Effects (ETWFE) estimator proposed by Jeffrey Wooldridge (2021) for Difference-in-Differences (DiD) analysis. This command is particularly important because it addresses some of the key challenges in modern DiD analysis, especially with staggered treatment adoption.

Key features of jwdid:

1. It's based on the Mundlak approach to panel data
2. It's designed to handle staggered DiD designs (where treatment happens at different times)
3. It provides correct identification of Average Treatment Effects (ATT) where traditional two-way fixed effects (TWFE) might fail

You can try the tool here

[Richard Williams]

Here is another problem I have with AI - the huge amount of energy it requires. Some say that AI will eventually create efficiencies and be well worth it. If so, I hope it does so before it makes climate change even worse.

https://www.npr.org/2024/07/10/nx-s1...or-electricity

[daniel klein]

I asked for a solution to what seems to me like a fairly simple problem. Here's my experience.

I (think I) specifically asked for returned results:

I'm working with Stata. I want to add the four smallest and largest values to the returned results of -summarize-. Can you suggest code to do that?

The first answer pointed to list, which doesn't get the job done. I omit the code suggestion. I followed up with

OK, but I want those values as returned scalars or in a matrix.

Well, here is the suggested code:

program define extremal_values, rclass
    syntax varname

    tempvar sorted_id
    tempname minvals maxvals

    * Drop missings and sort
    preserve
        keep if !missing(`varlist`)
        
        gen `sorted_id' = _n
        sort `varlist'
        gen double __min = `varlist'
        keep __min
        keep in 1/4
        matrix define `minvals' = __min'

        restore
        preserve
        keep if !missing(`varlist`)
        
        gen `sorted_id' = _n
        gsort -`varlist'
        gen double __max = `varlist'
        keep __max
        keep in 1/4
        matrix define `maxvals' = __max'

    restore

    * Combine into a single matrix
    matrix define r(extreme) = (`minvals', `maxvals')

    * Optionally return each value as scalar too
    forvalues i = 1/4 {
        return scalar min`i' = r(extreme)[1, `i']
        return scalar max`i' = r(extreme)[1, `=`i'+4']
    }

    * Label the matrix rows and columns
    matrix colnames r(extreme) = min1 min2 min3 min4 max1 max2 max3 max4
end

I won't even try to point out all the problems. For starters, none of the matrix commands will work. matrrix define doesn't take variable names as input; that's mkmat. Obviously, r(extreme) is not a valid matrix name for matrix define, either. Even if the code worked, the approach is generally as inefficient as it gets. Why is the temporary variable sorted_id generated? Twice? It's never used. I won't explain in detail how not explicitly using double, long, or better yet c(obs_t) will quietly fail to create a unique identifier in large datasets; it's one of the most common mistakes trying to create identifiers in Stata. Back to inefficiency. Why keep the same subset of data twice? Why (g)sort twice? We can directly refer to the end of the sorted data: in -4/L; that's no obscure secret trick; it's an example in the documentation. By the way, what happens when fewer than 4 observations are left? Moving on. Why copy `varlist' to __min and __max? The original data is preserved; we can refer to `varlist' directly.

By the way, this is the response from "standard" ChatGPT in a newly stared chat with no memory given the exact same prompt (note: singular!):

program define summarize_extremes, rclass
    syntax varname

    quietly summarize `varlist'
    
    // Save the summarize results
    return scalar mean = r(mean)
    return scalar sd   = r(sd)
    return scalar min  = r(min)
    return scalar max  = r(max)
    return scalar N    = r(N)

    // Get smallest 4 values
    preserve
    keep if !missing(`varlist')
    gsort `varlist'
    gen _obs = _n
    quietly forvalues i = 1/4 {
        quietly su `varlist' if _obs == `i', meanonly
        return scalar smallest`i' = r(mean)
    }

    // Get largest 4 values
    gsort -`varlist'
    gen _obs = _n
    quietly forvalues i = 1/4 {
        quietly su `varlist' if _obs == `i', meanonly
        return scalar largest`i' = r(mean)
    }
    restore
end

Not better by any means, but not worse either. That's not a knock on the Stata-dedicated implementation. I'm also not saying that AI isn't helpful when writing code. But it seems to me the limitations are quite obvious -- and severe.