Parallel computing with Stata/MP 14.1 and parallel package: invalid internship and stata(): 3598 Stata returned error

[Benjamin Beckers]

Dear Statalisters,

first, apologies for my rudimentary understanding of Stata as I only recently started using it. I am working with an unbalanced panel dataset of monthly observations over the horizon of 10 years. For each month, I would like to run a logit regression and subsequently obtain the predictive probabilities across a grid of the independent variable. To speed things up, I would like to parallelize the job using George Vega's and Brian Quistorff's parallel package.

Unfortunately, I obtain the following error message:

invalid 'internship'
                        stata():  3598  Stata returned error
parallel_export_programs():     -  function returned error
          parallel_write_do():     -  function returned error
                         <istmt>:     -  function returned error

All hints that I found suggest that observations - or the complete dataset - are missing (not passed through the parallel command?). Does anyone know what may be wrong?

This is my code which I adopted from http://www.statalist.org/forums/foru...setting-memory.

clear all
set processors 2
parallel setclusters 4

use random_data

program define logit_program, byable(recall)
    marksample ym
    logit `1' `2' if `ym'
    forvalues k = 10(10)10 {
        margins , at((p`k') `2')
    }
end

sort ym
parallel, by(ym) programs(logit_program) : logit_program dummy_buyer inc

The defined program itself works and does its job without the parallel command, i.e. using:

by ym : logit_program dummy_buyer inc

My first try placing the self-written program into a for-loop within a separate do-file, and then calling parallel do "filename" failed, due to the unbalanced nature of the dataset I suspect.

Thank you very much for any advice!

[George Vega]

Dear Benjamin,

If you could provide us with a toy dataset so we could reproduce the error we might be able to help you better. Either way, your approach might not be the best since the program would be running within each instance of stata but would not be returning anything useful to the current (mother) session since it only takes back the dataset, hence your program should somehow write the results from the margins command in some sort of log (for example). A quick-n-dirty example would be

program define logit_program, byable(recall)
    marksample ym
    logit `1' `2' if `ym'

    log using log_number_$pll_instance.txt, text replace
    forvalues k = 10(10)10 {
        margins , at((p`k') `2')
    }
    log close
end

Best,