Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Stata 15.1 crashes Windows 10

    Because I did not receive an acknowledgement of my e-mail from the StataCorp technical support team, I post the problem here again. Maybe someone else can replicate it.

    I noticed that occassionally Stata 15.1 reserves more and more RAM which eventually lets my Windows 10 operating system crash (black screen). I can replicate the problem with a Monte Carlo simulation based on the code below. (WARNING: Executing this code might crash your computer and might lead to the loss of unsaved data!) Already after a few reps Stata 15.1 consumes almost all of my 16 GB RAM. The same code runs smoothly without any problem in Stata 14.2. (It does not seem to be a specific problem of the simulate command, as it just happened to me again in a different situation which I cannot replicate. Instead, it might be related to Mata's optimize() and moptimize() routines which I am using frequently, for example inside my xtdpdgmm program.)

    Code:
    ssc install xtdpdgmm
    
    program define xtdpdgmm_simul, rclass
        version 14.2
        syntax , [AR(real 0.8) T(integer 6) N(integer 100)]
    
        *--------------------------------------------------*
        *** data generation ***
        drop _all
        loc obs                = (`t' + 1) * `n'                // number of observations
        set obs `obs'
        egen int id = seq(), b(`= `t' + 1')                    // panel identifier
        egen byte time = seq(), f(0) t(`t')                    // time identifier
        mata: xtdpdgmm_simul_rgen(`ar', `t', `n')            // random-number generation
        xtset id time
    
        *--------------------------------------------------*
        *** GMM estimation ***
        xtdpdgmm y L.y x, noser two vce(r) gmm(y, lag(2 6) c m(d)) gmm(x, lag(0 4) c m(d))
        tempname b
        mat `b'                = e(b)
        loc b_y                = el(`b', 1, 1)
    
        *--------------------------------------------------*
        *** simulation results ***
        eret clear
        drop _all
        mata: mata drop xtdpdgmm_opt1_1 xtdpdgmm_opt2_1
        ret sca b_y            = `b_y'
    end
    
    version 14.2
    mata:
    mata set matastrict off
    
    // random-number generation
    void xtdpdgmm_simul_rgen(real scalar ar, real scalar T, real scalar N)
    {
        // pseudo-random number generation
        u    = rnormal(T, N, 0, sqrt(1 - ar^2))
        e    = rnormal(T, N, 0, sqrt(1 - ar^2))
        a    = rnormal(1, N, 0, 1)
    
        // initialization
        X    = J(1, N, 0)
        Y    = J(1, N, 0)
    
        // data-generating process
        for (t = 1; t <= T; t++) {
            xt    = ar * X[t, .] + e[t, .]
            yt    = ar * Y[t, .] + sqrt(1 - ar^2) * xt + a + u[t, .]
            X    = (X \ xt)
            Y    = (Y \ yt)
        }
    
        // variable generation
        var1 = st_addvar("double", "y")
        var2 = st_addvar("double", "x")
        st_store((1, (1 + T) * N), var1, vec(Y[|1, 1 \ (1 + T), N|]))
        st_store((1, (1 + T) * N), var2, vec(X[|1, 1 \ (1 + T), N|]))
    }
    end
    
    simulate, seed(33820) r(500) nol: xtdpdgmm_simul
    Stata version information:
    Code:
    . about
    
    Stata/SE 15.1 for Windows (64-bit x86-64)
    Revision 08 May 2018
    Copyright 1985-2017 StataCorp LLC
    
    Total physical memory:    16628560 KB
    Available physical memory: 13277124 KB
    https://www.kripfganz.de/stata/

  • #2
    Update: Here is an even simpler example. After running the following estimation twice in a row, Stata 15.1 already requires the whole 16 GB RAM. I never had this problem before and I have not done any changes to the xtdpdqml command in quite a while. Something must have happened in the latest Stata update (or Windows update?).
    Code:
    ssc install xtdpdqml
    
    webuse abdata
    
    quietly xtdpdqml n w k, conc initi(2) vce(r)
    Notice the small amount of remaining physical memory after running this command three times:
    Code:
    . about
    
    Stata/SE 15.1 for Windows (64-bit x86-64)
    Revision 08 May 2018
    Copyright 1985-2017 StataCorp LLC
    
    Total physical memory:    16628560 KB
    Available physical memory:  358332 KB
    Last edited by Sebastian Kripfganz; 29 May 2018, 08:22.
    https://www.kripfganz.de/stata/

    Comment


    • #3
      Another update:
      The developers at StataCorp could replicate this abnormal behavior but they need to investigate it further.

      (Apparently, there have been official holidays in Texas on Friday and yesterday. My grievance about the missing acknowledgement was thus unjustified.)
      https://www.kripfganz.de/stata/

      Comment


      • #4
        I just ran the code from post #1 in Stata 15.1 on a machine with 64 GB of memory. At first it seemed to work fine but then I too had problems. After running your code, I got

        Code:
        . about
        
        Stata/MP 15.1 for Windows (64-bit x86-64)
        Revision 08 May 2018
        Copyright 1985-2017 StataCorp LLC
        
        Total physical memory:    67036996 KB
        Available physical memory:  271352 KB
        So yes, available memory does seem to take a hit. And when I next typed "clear all" Stata stopped responding.

        I tried 14.2 and yes, it seems to work much better after running your code.

        Code:
        . about
        
        Stata/MP 14.2 for Windows (64-bit x86-64)
        Revision 29 Jan 2018
        Copyright 1985-2015 StataCorp LLC
        
        Total physical memory:    67036996 KB
        Available physical memory: 62347444 KB
        No lockups with subsequent commands.

        In short, it isn't just you. I have the same problems on a machine that has much more memory,

        Oh, and I am using Win7.
        Last edited by Richard Williams; 29 May 2018, 12:07.
        -------------------------------------------
        Richard Williams, Notre Dame Dept of Sociology
        StataNow Version: 19.5 MP (2 processor)

        EMAIL: [email protected]
        WWW: https://www3.nd.edu/~rwilliam

        Comment


        • #5
          In the 08mar2018 update (item 5.) we fixed a crashing bug in Mata's
          subview code. Subview's are commonly used in intermediate calculations
          within panel-data estimation commands like xtdpdqml. As part of this
          fix, the subview now keeps track of extra information about the variable
          list of the parent view when that variable list contains factor
          variables notation or time-series operators. The memory used to keep
          track of this extra information was not being freed properly, so tight
          loops over the observations or groups of observations that use subviews
          to make intermediate calculations in a likelihood evaluator could
          eventually consume a disproportionate amount of memory. We have
          determined the source of the problem and intend to have this fixed very
          soon.

          Comment


          • #6
            Originally posted by Jeff Pitblado (StataCorp) View Post
            We have determined the source of the problem and intend to have this fixed very soon.
            That's good news. Thanks for the detailed explanation.
            https://www.kripfganz.de/stata/

            Comment


            • #7
              I am happy to confirm that the problem is now fixed with the 06 Jun 2018 update. Thanks to StataCorp for the quick reaction.
              https://www.kripfganz.de/stata/

              Comment

              Working...
              X