New ML package: pystacked for Stacking Regression and Classification

[Achim Ahrens]

I am happy to announce a new Stata package which I wrote together with Mark Schaffer and Chris Hansen.

pystacked implements stacked generalization (Wolpert, 1992) via scikit-learn’s sklearn.ensemble.StackingRegressor and sklearn.ensemble.StackingClassifier. Stacking is a way of combining predictions from multiple supervised machine learners (the “base learners”) into a final prediction to improve performance. The currently-supported base learners are:

• Linear regression
• Logistic regression
• Lasso, ridge and elastic net
• Support vector machines
• Gradient boosted trees
• Random forest
• Neural nets (Multi-layer Perceptron)

pystacked can also be used with a single base learner and, thus, provides an easy-to-use API for scikit-learn’s machine learning algorithms.



Plenty of examples are provided on our website: https://statalasso.github.io/docs/pystacked/

This is the first publicly released version of pystacked -- Feedback and bug reports are very much welcome.

pystacked is not yet on SSC. You can install it from github:

net install pystacked, from(https://raw.githubusercontent.com/aahrens1/pystacked/main) replace

See also installation instructions here.

[Pete Upham]

Thank you for this package! I have been using pystacked without incident for regression. I recently attempted to use it for a multi-class classification problem, but the returned class predictions indicate the program is treating it as a binary classification task. Does pystacked support multi class classification? I could not find any references addressing this question in the help documentation.

[Achim Ahrens]

Hi Pete, I am afraid pystacked only supports regression and binary classification. I will make this clear in the help file. Best, Achim

[Pete Upham]

Many thanks for the clarification, Achim.

[Xiaobo Shen]

Achim, could you please tell me how I can download and install the ddml package in Stata? Thanks!

[Achim Ahrens]

Link is above:

 
 net install pystacked, from(https://raw.githubusercontent.com/aahrens1/pystacked/main) replace

[Xiaobo Shen]

Achim, many thanks! I have installed pystacked , but when I type "ddml init partial" in command window, it returns "ddml not found". Why?

[Xiaobo Shen]

it says "command ddml unrecognized"

[Achim Ahrens]

ddml is not part of the pystacked package. pystacked is a separate program from ddml. ddml is not yet available. Send me an email and I will share a prelim version with you.

[Xiaobo Shen]

That's great! my email is [email protected]. Many thanks Achim!

[Achim Ahrens]

An updated version of pystacked is available from SSC thanks to Kit Baum:

ssc install pystacked

You can always get the latest version from github. We update the github version more frequently:

net install pystacked, from(https://raw.githubusercontent.com/aahrens1/pystacked/main)

Finally, we have a working paper that you can find here: https://arxiv.org/abs/2208.10896

Please check the working paper and help file for plenty of examples and demonstrations.