[Jesse Wursten]

Jesse Wursten
As to your second question, if you have Stata 14 or later, you already have a Java instance available on your machine (it is bundled with Stata from that point forward since it is needed to support the Java API).

Ah, that's good to know! libjson seems to do what I need, looks like it can even submit structured API calls.

[wbuchanan]

Marc Kaulisch
I can appreciate that and it was what I tried to do with jsonio. There could potentially be other ways of abstracting things, but the structure of JSON doesn't make it the most easy/useful for transmitting rectangularized datasets.

[Marc Kaulisch]

Marc Kaulisch
Which user-written JSON packages do you have difficulties with?

It is years ago and I was frustrated - as far as I remember I tried -insheetjson-, -jsonio- and -libjson-. If I remember correctly I was successful with -insheetjson- to do the part I needed, but I avoid JSON as much as possible. I know it is difficult to parse JSON but nonethless I - as user - would love to see some maybe automatic JSON-structure analysis and then an opportunity to transform it into a Stata-like data structure.

[wbuchanan]

Jesse Wursten
Regarding your first question, it really depends on the payload itself. That will be true of any JSON parser though.

As to your second question, if you have Stata 14 or later, you already have a Java instance available on your machine (it is bundled with Stata from that point forward since it is needed to support the Java API).

[Jesse Wursten]

Jesse Wursten

https://wbuchanan.github.io/StataJSON (e.g., -jsonio-) is cross platform (since all the underlying code is written in Java). The package also allows you to make API calls, though it does not handle OAuth or other authentication methods.

Is there a way to use StataJSON that does not store the key-value in the dataset? And this still requires that the user has Java installed, right?

[wbuchanan]

Jesse Wursten

https://wbuchanan.github.io/StataJSON (e.g., -jsonio-) is cross platform (since all the underlying code is written in Java). The package also allows you to make API calls, though it does not handle OAuth or other authentication methods.

[wbuchanan]

Marc Kaulisch
Which user-written JSON packages do you have difficulties with?

[Marc Kaulisch]

I don't know if it's been mentioned yet, but a native way to handle APIs would be great. More and more data providers are moving towards providing data through an API, but Stata doesn't really offer much in terms of handling them. You can always go through shell>curl or python, but neither makes for a very robust, easy and cross-platform approach.

It also implies a better Stata-native handling of JSON, or? When used I have some difficulties while using the user-written JSON readers...

[Jesse Wursten]

I don't know if it's been mentioned yet, but a native way to handle APIs would be great. More and more data providers are moving towards providing data through an API, but Stata doesn't really offer much in terms of handling them. You can always go through shell>curl or python, but neither makes for a very robust, easy and cross-platform approach.

[wbuchanan]

Christopher Bratt
I understand that there is a user-written command, but the full reading of my suggestions (including #3) implies extensions to the existing append methodology as well. I'm also aware of the different Stan interfaces (including StataStan - developed by Robert Grant), but I am talking more about native integration. Integrating Stan within Stata would also provide additional capabilities via the StanMath library (e.g., auto-differentiation in addition to the user written package from Paul Lambert for autodiff functionality).

On an unrelated note, one other request that I forgot to list is:

8. Ability to save/restore do-file editor preferences in Linux environments.
9. Avoid help window auto-resizing in RHEL environments.

I've been able to save preferences to the general Stata window and get them to load, but for whatever reason, it seems the do-file editor always resorts to the defaults and it'd be really nice to have the same preference saving/restoring applied to that part of the environment as well. (In case anyone is interested it happens in RHEL but seems to function fine in Ubuntu). For #9, it may be something specific to the individual RHEL instance, but opening a help file in Stata has a bizarre effect where the window will open one size and then automatically start shrinking itself down to a size that requires the user to expand the window to read things fairly comfortably.

[Christopher Bratt]

Support appending with frames.

search xframeappend

Integrating Stan for Bayesian models

You may consider going via python.

My view now is that anyone who wants to use Stan from Stata should do so through PyStan or CmdStanPy.

https://github.com/stan-dev/statastan

[wbuchanan]

Time for some of my recurring requests:

1. Allow mapping aesthetic properties of graphs to data elements/values (e.g., instead of defining a scatterplot twice with auto.dta to show the foreign and domestic cars with different colors/symbols you could pass the variable to an argument for the fill color and/or marker symbols and the colors would be selected from the scheme).
2. Document all of the properties used in Scheme files
3. Adopt SQL approaches to some of the data management tasks (e.g., have a distinct option for append that would be comparable to UNION compared to the current UNION ALL behavior, allow arbitrary relations to define the matches between data sets, etc...)
4. Model fit statistics for gsem
5. Infit/Outfit statistics for IRT models (Rasch models in particular)

And some newer requests

1. Provide newer optimization methods, options for newer optimization methods, and/or options to create adaptive optimization functions for models that use optimization in someway (e.g., defining varying step sizes to hopefully speed up model convergence, ADABoost, ADAM, etc...).
2. Speed up reshape, sorting, and other routine data management tasks (working in a shared computing environment can mean waiting several hours for a single reshape command to execute)
3. Support appending with frames.
4. Provide a way to translate and/or document standard SVG elements so existing scheme files could be translated into CSS.
5. IRT related linking/equating
6. Expanded -bayes- prefix coverage
7. Integrating Stan for Bayesian models

[paulvonhippel]

Stata MP 18 should have the native ability to bootstrap in parallel

[Christopher Bratt]

Inspired by #274

Suggestion: Fit indices in -gsem-

A few days ago, I saw Stata's webcast on survey data. It included a -gsem- model that was unlikely to have good fit, i.e. unlikely to represent/explain the data well. The estimated association between the estimated factor and the predictor would thus be biased.

This reminded me of the lack of fit estimates in Stata's -gsem-. Of course, alternatives to -gsem- are available, some for free (such as R's lavaan). But since -gsem- is around and may be used by many, would it be worth considering adding the most important fit indices available in -sem- even to -gsem-?

In case Stas Kolenikov reads this: I believe you use Stata quite a lot for SEM. Any thoughts on this issue?

[Dirk Enzmann]

The implementation of Satorra-Bentler adjustments for models with cross-group constraints (see -sem-) -- as far as I know this is currently only available in programs such as EQS, MPlus, or Lavaan (R).