Problems installing packages that include ancillary files

[William Lisowski]

There seems to be a problem with installation of software packages that have auxiliary files. I am not sure if what I describe below, and alluded to in a recent topic, is new behavior, or just something that's becoming more apparent (to me?) over time.

Auxiliary files obtained via a net get command (rather than as part of a net install command) are copied into Stata's "net other" directory specified by net set other rather than into Stata's PLUS directory. This makes sense when the auxiliary files are sample data, but not so much when they're necessary for the functioning of the package, as they are for markstat (from SCC) and zanthro (from SJ) to name two I've seen problems with in the last few weeks. The default for the "net other" directory is the user's current directory, and any setting to another directory is not retained from session to session of Stata.

Here's an example using markstat installation. (Note that this example is functionally equivalent to using search markstat, clicking to open the description in the Viewer window, and clicking on the two "click here" links in the Viewer window.)

. net query
from    http://www.stata.com
ado     /Users/lisowskiw/Library/Application Support/Stata/ado/plus/
other   (current directory)

. net describe markstat, from(http://fmwww.bc.edu/RePEc/bocode/m)

------------------------------------------------------------------------------------------------
package markstat from http://fmwww.bc.edu/RePEc/bocode/m
------------------------------------------------------------------------------------------------

TITLE
      'MARKSTAT': module to support literate data analysis using Stata and Markdown

...

INSTALLATION FILES                             (type net install markstat)
      markstat.ado
      markstat.sthlp

ANCILLARY FILES                                (type net get markstat)
      markstat.css
      markstat.docx
      markstats5.zip
------------------------------------------------------------------------------------------------

. net query
from    http://fmwww.bc.edu/RePEc/bocode/m/
ado     /Users/lisowskiw/Library/Application Support/Stata/ado/plus/
other   (current directory)

. net install markstat
checking markstat consistency and verifying not already installed...
installing into /Users/lisowskiw/Library/Application Support/Stata/ado/plus/...
installation complete.

. net get markstat
checking markstat consistency and verifying not already installed...

copying into current directory...
      copying  markstat.css
      copying  markstat.docx
      copying  markstats5.zip
ancillary files successfully copied.

.
. pwd
/Users/lisowskiw/Research/Statalist

. dir markstat*

-rw-r--r--  1 lisowskiw  staff   1645 Dec 12 20:10 markstat.css
-rw-r--r--  1 lisowskiw  staff  14871 Dec 12 20:10 markstat.docx
-rw-r--r--  1 lisowskiw  staff  27758 Dec 12 20:10 markstats5.zip

.

So you see that the auxiliary files have been installed into my current working directory, rather than into the PLUS directory, where a findfile command would locate them (a technique used by zanthro).

I'm not a developer, so I can't take this any farther - I'm past the limits of my knowledge here, and am only writing this because I've seen users stumble over this twice in recent weeks.

[German Rodriguez]

Thanks for bringing this up, William. By default Stata determines where files go based on the suffix: .ado and .sthlp are considered "installation files" and go in the system directory, and practically everything else is considered an "ancillary file", made available through a separate get, and put in the current working directory. It is possible, however, to force other files to be installed rather than treated as ancillary by designating them as F rather than f in the package file, see help usersite for more information.

This is in fact the strategy I used with markstat. In 1.0 we designated markstat.css as F, and in 2.0 we extended the designation to markstat.docx and markstats5.zip, which has JavaScript and CSS files for HTML slide shows, so these files are found by findfile no matter where the command is run from. These files had the correct designation last November 14, but as you note they are now listed as ancillary at SSC. Not sure what might have happened, but will follow up with Kit Baum and report when I know more.

[German Rodriguez]

Thanks to Kit Baum for his speedy response. He has already updated the pkg file at SSC and ssc describe markstat now returns in part

INSTALLATION FILES                             (type net install markstat)
      markstat.ado
      markstat.css
      markstat.sthlp
      markstat.docx
      markstats5.zip

So all files will go in the installation folder as required. This resolves the issue for markstat.

Regarding the related zanthro discussed here, the .dta files are all designated as ancillary. Maybe they should be designated as installation files. But as Clyde Schechter noted in that thread, you can always put the files in ado/plus/z yourself.

[William Lisowski]

Thank you for your resolution to this problem. I have brought these threads to the attention of the zanthro support person to see if in their case it was intentional, since in the dm0004 release the ancillary files were (and are) included in the installation, while in the update dm0004_1 they are not - as confirmed by looking at the .pkg file.

I appreciate your further comment that the problem with markstat was due to an issue at SSC. While zanthro is distributed through SJ, and the most recent release is from 2013, I am wondering if a similar issue exists/existed there at some point. This seemed too systematic - two instances of the same issue on two different packages coming to light within a few weeks of each other.

By the way, thanks too for the reference to help usersite - I knew I'd seen a discussion of package construction, but was not able to pull up the magic search term (hint: search package is useless, as is search pkg ) from memory when I was writing last night.

[Suzanna Vidmar]

Thank-you William for contacting me and bringing this to my attention. I contacted Stata Journal and they've fixed this. The growth reference .dta files are now included in the installation.