Can you give me a crash course on Inflation forecasting, with the appropriate Stata tools?

[Joro Kolev]

Good afternoon,

I am working on monthly data. I would like to produce from 1 month, to 36 months ahead Inflation forecasts, without look ahead bias. That is, when I generate the forecasts on say January 2000, I want in my forecast only data from January 2000 or before to enter my forecast, that is, as if I were doing it in real time.

1. Which is the state of the art Inflation forecasting model? Do you have a reference for a recent paper on how this should be done?
2. My associate tells me that ARIMA models are used in this. If I were to go for an ARIMA model

1. Is there some Stata tool for automatic model selection? (I spent some time gazing at Autocorrelation and Partial autocorrelation functions and choosing the best ARIMA model based on those, but it was many years ago, maybe 18, maybe 20 years ago… I have lost this skill now.)
2. Which are the tools in Stata for generating 1month – 36 months forecasts without look ahead bias? I would guess -rolling-, and -forecast- would be useful here, but I have never worked with them, and any advice will be helpful.

[Joro Kolev]

I am attaching the CPI for the US, all urban consumers, if somebody wants to show code with examples. CPIallUrbanConsumersUSA.dta

[Jared Greathouse]

What you are likely interested in is cvlasso. It performs rolling origin cross validation, where you walk forward by a given amount of steps. The cross validation prevents overfitting, and is quite a robust technique to not use tomorrow to predict today, as a colleague of mine says.


In short, machine learning will be your best friend here, most likely. But note that I'm not a time series researcher, so it's likely others who do real forecasting work will have a much more informed take than me.

[Andrew Musau]

Stock and Watson illustrate a variety of methods in the linked PDF and also provide access to the data and replication files: https://scholar.harvard.edu/files/st...ginflation.pdf

[Justin Niakamal]

I am attaching the CPI for the US, all urban consumers, if somebody wants to show code with examples.

Here's an example in the spirit of #2 using forecast and US CPI. Note;I use the community-contributed mylabels and cvlasso. Hopefully the example code points you in a the right direction.

import delimited using  "https://fred.stlouisfed.org/series/CPIAUCSL/downloaddata/CPIAUCSL.csv", clear

gen month = mofd(date(date, "YMD"))
format month %tmMon-YY
gen gr = 100*(value/value[_n-12]-1)

keep if yofd(dofm(month)) >= 1990

local end = month[_N]
tsset month, monthly
tsappend, add(24)

cvlasso gr L(1/24).gr if month <=`end', rolling  h(24)
local Vs = "`e(varXmodel)'"

reg gr `Vs'

estimates store ols
forecast create myols
forecast estimates ols
forecast solve, prefix(d_) begin(`=`end'+1') simulate(betas, statistic(stddev, prefix(sd_)) reps(300))

gen lb = .
gen ub = .
gen ulb = .
gen uub = .

cvlasso gr L(1/24).gr if month <=`end', rolling  h(24)

local T = month[_N]

forvalues i = `=`end'+1'(1)`T' {
    predict double xhat if month==`i', xb lse
    replace gr = xhat if month == `i'
    drop xhat
    replace lb = gr - invnormal(0.975)*sd_gr if month ==`i'
    replace ub = gr + invnormal(0.975)*sd_gr if month ==`i'
    replace ulb = gr - 1*sd_gr if month ==`i'
    replace uub = gr + 1*sd_gr if month ==`i'    
}

gen realized = cond(month <= `end', gr, .)
gen fcasted  = cond(month > `end', gr, . )

replace lb = gr if month == `end'
replace ub = gr if month == `end'

replace ulb = gr if month == `end'
replace uub = gr if month == `end'

// search mylabels
mylabels -5(5)10, local(L) suffix(%)

levelsof month if mi(value), local(XX) clean

twoway  rarea   uub  ulb month,  fcolor(red%90)  lcolor(red%1) ||              ///
        rarea   ub   lb month,   fcolor(red%50)  lcolor(red%1) ||              ///
            line gr month, color(red)   lwidth(medthick) leg(off) xtitle("")     ///
         xlab(`=month[1]'(36)`=month[_N]', labsize(small)) title(" ")         ///
         ylab(`L') note("Here's an Example, Joro")                             ///
         caption("Source: U.S. Bureau of Labor Statistics", size(vsmall))     ///
         xline(`XX', lpattern(solid . .) lwidth(vthick) lcolor(gs14))

[Jared Greathouse]

Let me give an example, now that I've had time to think about this a little more (note that I use labvars). Let's use the Kaggle dataset on Walmart sales. Note that my estimator for this is forthcoming to SSC as soon as it's approved by Professor Baum, but here's an example of how we can use causal inference methods for forecasting. Precisely, we'd be using my scul command, or synthetic controls using LASSO, using the cvlasso command under the hood (although in a different setup than Justin Niakamal)

If we download that dataset, we notice sales data for Wal-Mart stores, weekly figures. I think the number of Walmarts here is 44. Anyways, suppose you're an econometrician who works at your local Walmart (which I suspect doesn't happen) and the store owner boss person comes in your office and says "I'd like you to produce a model that forecasts how much we'll be making next year." You say "Okay."

clear
input float(date weekly_sales25) double cf float(relative diff_ te_ub te_lb cf_ub cf_lb)
2605  .6772316  .7435651219975844 -47   -.06633351  -.065502696   -.067164324         .         .
2606   .583364  .7213227648333396 -46   -.13795875   -.13712794    -.13878956         .         .
2607  .6762607  .7348933411938653 -45   -.05863263   -.05780182    -.05946345         .         .
2608  .6285166  .6488559303251823 -44  -.020339375   -.01950856    -.02117019         .         .
2609  .6657501  .6874460505418711 -43  -.021695964   -.02086515    -.02252678         .         .
2610    .66062   .682503779228865 -42  -.021883804   -.02105299    -.02271462         .         .
2611  .6597958  .6742155122693413 -41  -.014419692  -.013588877   -.015250506         .         .
2612  .6966876  .6732774748053546 -40   .023410164    .02424098     .02257935         .         .
2613  .8224864  .7863949110427525 -39   .036091488   .036922302    .035260674         .         .
2614   .712648  .7125403799631337 -38 .00010759453  .0009384087  -.0007232196         .         .
2615  .7153116   .708208735648502 -37   .007102911   .007933725    .006272097         .         .
2616  .6945317  .7012636623954754 -36  -.006731924   -.00590111   -.007562738         .         .
2617   .706924  .6530195788191722 -35    .05390444    .05473526     .05307363         .         .
2618   .724469  .7316387155586508 -34   -.00716971  -.006338895   -.008000524         .         .
2619  .6980739  .6752227006426822 -33   .022851223    .02368204     .02202041         .         .
2620  .7041132  .6860416413482252 -32   .018071603    .01890242    .017240789         .         .
2621  .7924426  .7846250381793286 -31   .007817522   .008648336    .006986707         .         .
2622  .7641554  .7586192086109286 -30   .005536239   .006367053    .004705424         .         .
2623  .7371632  .7020883205873193 -29   .035074867    .03590568    .034244053         .         .
2624  .7804449  .7253240421374395 -28    .05512088    .05595169     .05429006         .         .
2625  .7375692  .7308215485533556 -27   .006747605   .007578419     .00591679         .         .
2626  .7594079  .7630035388281322 -26  -.003595661  -.002764847   -.004426475         .         .
2627  .7195911  .7452242728253868 -25   -.02563313   -.02480232   -.026463946         .         .
2628  .7269978   .696644726818822 -24   .030353066    .03118388     .02952225         .         .
2629   .665291  .6685103895087733 -23  -.003219438 -.0023886235   -.004050252         .         .
2630  .6821243   .682747035760088 -22  -.000622719 .00020809507  -.0014535332         .         .
2631  .6994644  .7014384635385481 -21 -.0019740232  -.001143209  -.0028048374         .         .
2632  .6860723  .6915591865616376 -20  -.005486837  -.004656023   -.006317651         .         .
2633  .7244998  .6972684984916419 -19    .02723132   .028062137     .02640051         .         .
2634   .711462  .7053319040731604 -18   .006130057   .006960872    .005299243         .         .
2635  .6857001  .7119822214290035 -17  -.026282163   -.02545135    -.02711298         .         .
2636   .655812  .6990049246893044 -16   -.04319296   -.04236215    -.04402377         .         .
2637  .6289899   .658351093566611 -15   -.02936122  -.028530404    -.03019203         .         .
2638  .6078193  .6259670622953658 -14  -.018147744   -.01731693    -.01897856         .         .
2639  .6586401   .643743348722116 -13   .014896798   .015727611    .014065984         .         .
2640  .6742839  .6725129589854164 -12   .001770903  .0026017174    .000940089         .         .
2641  .6160948  .6555743227680342 -11   -.03947955   -.03864874    -.04031037         .         .
2642  .6616442  .6717422559922666 -10  -.010098095  -.009267281    -.01092891         .         .
2643   .674458  .6709618921600441  -9   .003496135   .004326949   .0026653204         .         .
2644  .6963145  .6921721972354901  -8  .0041423165   .004973131    .003311502         .         .
2645    .71325  .6990547507663066  -7    .01419529   .015026105    .013364476         .         .
2646  .7180567  .6832629861334494  -6   .034793753   .035624567     .03396294         .         .
2647 1.1152406  1.120631002213039  -5  -.005390428  -.004559614   -.006221243         .         .
2648  .8855729  .8123059655575744  -4   .073266946    .07409776     .07243613         .         .
2649  .9647292  .9062463507256129  -3    .05848284    .05931365     .05765203         .         .
2650 1.0477076 1.0206541791939328  -2    .02705338    .02788419    .026222564         .         .
2652  .5587946  .6017222336819971   0   -.04292762    -.0420968    -.04375843  .6454807 .55962545
2653  .5723608  .5787037991059448   1  -.006342986  -.005512171     -.0071738  .5858776 .57319164
2654 .56809354  .5808789099126095   2   -.01278537  -.011954557   -.013616186  .5944951 .56892437
2655  .6004487  .5702144271111482   3    .03023424   .031065054    .029403426  .5996178 .53914934
2656  .6398304  .6595170871067486   4  -.019686677   -.01885586    -.02051749  .6800346  .6406612
2657  .6156667  .6752889934665909   5   -.05962225   -.05879143    -.06045306   .735742  .6164976
2658   .634637  .7276104143200354   6   -.09297342    -.0921426    -.09380423  .8214146  .6354678
2659 .57081634  .6374975128964782   7   -.06668118   -.06585036    -.06751199  .7050095 .57164717
2660  .6409122  .6659451172982181   8   -.02503294  -.024202127   -.025863755  .6918089   .641743
2661  .5998284  .6304839720686941   9   -.03065561  -.029824795    -.03148642  .6619704  .6006592
2662  .6033936  .6411772190883206  10    -.0377836   -.03695279    -.03861442  .6797916  .6042244
2663  .6163242  .6193139276860348  11  -.002989682 -.0021588674   -.003820496  .6231344  .6171551
2664  .6183778  .6306464005193483  12  -.012268595   -.01143778    -.01309941  .6437458  .6192086
2665  .6486061  .6699229109670226  13   -.02131679  -.020485975   -.022147603  .6920705   .649437
2666  .6745625  .6868437532962195  14    -.0122813  -.011450484   -.013112113  .6999559  .6753933
2667  .7565885  .7351366812199074  15    .02145178   .022282593    .020620964  .7557576  .7128541
2668   .649245  .6538371248044198  16  -.004592101  -.003761287   -.005422915    .65926  .6500759
2669  .6594465  .7111477628535119  17   -.05170123   -.05087041    -.05253204  .7636798  .6602774
2670  .6847831  .7059293748849225  18  -.021146273   -.02031546    -.02197709  .7279065  .6856139
2671  .6779713  .6698693589404816  19   .008101945   .008932759     .00727113  .6771405  .6609366
2672   .718374   .711587837808318  20   .006786116    .00761693    .005955302  .7175431  .7039709
2673  .7375517  .7658888885969478  21   -.02833714  -.027506325   -.029167954  .7950568  .7383826
2674  .7402596  .7226386554914724  22   .017620992   .018451806    .016790178  .7394288  .7041869
2675  .7173734  .7170091788575838  23  .0003642519   .001195066 -.00046656225  .7174757  .7158141
2676  .6992701  .7166794333972114  24  -.017409304   -.01657849   -.018240118  .7349195   .700101
2677  .7062069  .7367276454788588  25   -.03052079  -.029689973     -.0313516  .7680792  .7070377
2678  .6985296  .7069519847076479  26  -.008422384   -.00759157   -.009253198  .7162052  .6993604
2679  .6805102  .6952656305640442  27  -.014755407  -.013924593   -.015586222  .7108518  .6813411
2680  .6708549  .6819493032574652  28  -.011094377  -.010263563   -.011925192  .6938745  .6716858
2681  .6683908  .6537268673217961  29   .014663943   .015494757     .01383313    .66756  .6382321
2682   .679706  .6931966966808724  30   -.01349072  -.012659905   -.014321533  .7075182  .6805368
2683  .6671305  .6719616281416074  31  -.004831158  -.004000344   -.005661972  .6776236  .6679613
2684  .6889588   .674492260685112  32   .014466504   .015297318     .01363569  .6881279  .6591949
2685  .7264225  .6937618417164477  33   .032660708   .033491522    .031829894  .7255917  .6602703
2686   .699779   .711569184503413  34   -.01179021  -.010959396   -.012621025  .7241902  .7006098
2687  .6732485  .6839555822088428  35  -.010707052  -.009876238   -.011537867  .6954935  .6740794
2688  .6287205  .6361496473500562  36  -.007429185   -.00659837   -.008259999  .6444097  .6295513
2689  .6208859   .633543489645774  37   -.01265758  -.011826767   -.013488395  .6470319  .6217167
2690  .6391603  .6162901931865676  38    .02287008   .023700897     .02203927  .6383294  .5925893
2691  .6715229   .669401934339734  39  .0021209214  .0029517356   .0012901073   .670692  .6664502
2692  .6469155   .634308813877868  40    .01260668   .013437496    .011775867  .6460847  .6208713
2693  .6906755  .6871394348013535  41   .003536062  .0043668766    .002705248  .6898447  .6827726
2694   .724444  .6796328211243209  42    .04481115    .04564197     .04398034  .7236131  .6339909
2695  .7183936  .7107128110411064  43   .007680813   .008511627    .006849999  .7175628  .7022012
2696  .7192351  .7085181653961508  44   .010716897   .011547712    .009886083  .7184042  .6969705
2697  .7285256  .6865852606705903  45    .04194038    .04277119     .04110956  .7276948  .6438141
2698 1.1162114 1.0879050382234714  46   .028306376    .02913719     .02747556 1.1153806 1.0587678
2699  .8783146  .7717025472798061  47      .106612    .10744282      .1057812  .8774837  .6642597
2700   .916446   .840466826030604  48    .07597914    .07680996     .07514833  .9156151  .7636569
2701  .9975025  .9380735490546754  49    .05942896    .06025977     .05859814  .9966717  .8778138
2702  1.290533 1.2426088174813004  50    .04792413    .04875494     .04709331  1.289702  1.193854
2703  .6836654  .6986226223984836  51  -.014957228  -.014126414   -.015788041  .7144107  .6844962
2704  .6364191  .6159693904708663  52    .02044973    .02128054    .019618914  .6355883 .59468883
2705  .6147643  .5675151462250014  53    .04724919       .04808     .04641837  .6139335 .51943517
end
format %tw date

tsset date, w

labvars week date cf "Sales (in Millions)" "Week" "SCUL Predictions"
cls
twoway (tsline weekly_sales25, lcolor(black) lwidth(medium)) ///
(tsline cf if rel >=0, lcolor(blue) lwidth(thin) lpattern(dash_dot)), ///
legend(ring(0) pos(9))

Without going into the technical details, I selected a random store (25) at a random treatment time, in this case week 1 of 2011, and honestly, my results surprised me. Note that no intervention in fact happened at this time (that I'm aware of !!) so if my estimator performs well, ideally it won't detect a treatment effect and it'll simply predict the sales of real Walmart 25 as though nothing happened (because in fact, nothing happened). To be clear what's going on, the "forecasts" here are based exclusively on the pre-intervention outcomes of other Wal-Marts- I don't need any other predictors here.

The figure here presents the results. I don't show the pre-intervention line, but the pre-intervention RMSPE is 0.035 (which I suppose in millions isn't so big of a deal). The estimated treatment effect is -0.001, and again without doing the math and thinking of the details, I suspect that this is an acceptable effect size. Put another way, if I hired an economist to predict the next 52 weeks of sales and they came out this good, I would be quite satisfied with the results of this.

Would you agree? Joro Kolev

[Jared Greathouse]

To give another example (this time using Walmart Store 20)

* Example generated by -dataex-. For more info, type help dataex
clear
input float(date weekly_sales20) double cf float relative
2605 2401395.5 2256928.8786294796 -47
2606   2109108 2106419.5627530543 -46
2607 2161549.8 2177673.1331814197 -45
2608   1898194 1973724.6182036411 -44
2609 2119213.8 2072176.6018043745 -43
2610   2010975 2013902.1506962571 -42
2611 1951848.4 1981668.4983276066 -41
2612   1894743 1932281.9221196286 -40
2613 2405395.3 2228532.0426434367 -39
2614 2007796.3 2138957.8524056105 -38
2615 1985784.6 2046089.4800336254 -37
2616 1878862.4 1984229.2357607645 -36
2617 1919053.3 1931301.7287765748 -35
2618 2137202.5 2138099.4898041952 -34
2619 2033211.6  2047117.685627485 -33
2620   1893737 1965674.8995749177 -32
2621   2065985 2099905.3427827074 -31
2622 2203619.3 2220215.1256900528 -30
2623 2100489.8 2104569.0548692467 -29
2624 2091903.6  2095584.160204602 -28
2625   1973136 2013815.1516180595 -27
2626 2143676.8 2096629.1908883008 -26
2627 2107285.8 2057426.3097857393 -25
2628   2031852 2019210.6493304793 -24
2629   1900536 1913817.9041480443 -23
2630 1955896.6 1927124.8564594383 -22
2631 1910177.4 2085547.6942420972 -21
2632 2071022.5 2007073.4900329073 -20
2633 1975374.5  2030721.257877512 -19
2634 1946369.6 2044024.0924828763 -18
2635 2121561.5 2068838.4092592525 -17
2636 2014954.8  1970989.590860501 -16
2637 1948359.8 1943767.2048881862 -15
2638 1789687.6 1842433.7001813632 -14
2639 1933719.3 1909086.9209127196 -13
2640 2060389.3 2037992.2713946195 -12
2641 1950676.4 1919508.9680943186 -11
2642 1893955.3 1953006.5937263183 -10
2643   1891816 1956134.0627654346  -9
2644 2184316.8  2117022.282608982  -8
2645 2187765.3 2093801.2966552714  -7
2646   2105059 2049148.0093478633  -6
2647   2811634 2941129.2245124383  -5
2648 2416051.3  2323387.372202224  -4
2649   2752122  2534448.086678571  -3
2650   2819193  2795519.820157801  -2
2652   1843031 1926127.3597814948   0
2653   1884345 1865716.5315428835   1
2654 1781805.6 1848907.6631177708   2
2655 1761506.6  1827443.322169575   3
2656   2351143 2106977.0211564684   4
2657 2211388.3 2156168.0583616113   5
2658 2258616.3 2249066.3581997836   6
2659 1938608.5 2014175.0634997936   7
2660 2143424.5  2112805.423399349   8
2661 1990932.8 2012420.9293847838   9
2662 1931668.6 2003382.7490867868  10
2663 1824711.3 1927447.1811403797  11
2664   1927993 1969746.6810294753  12
2665 2027056.4 2087234.7084436305  13
2666 2057406.4  2050002.687267223  14
2667 2313861.8  2171878.502347499  15
2668 1881788.3  1986481.168555376  16
2669 2090838.5 2128032.3215143415  17
2670 2036748.5  2073052.316910805  18
2671   1953416 1971127.1428634687  19
2672   1944433  2052437.602613398  20
2673 2182246.8  2193565.429603298  21
2674   2135062 2122068.2374318154  22
2675 2065191.3 2134519.8109899764  23
2676 1950826.4 2040582.2926994902  24
2677 2053165.4 2079452.4752492674  25
2678 2123787.8 2077541.6460256774  26
2679 2039875.8 2012718.1487268368  27
2680   1950905 1979254.1510982884  28
2681   1858441 1929377.4096337145  29
2682 2189353.8  2088601.825398091  30
2683 2052246.4 2015407.0709348868  31
2684   1990018 2055869.0971870252  32
2685 1933577.3 2122198.7181630866  33
2686   2141766 2037691.7498127364  34
2687 2050542.5 2030411.9333762391  35
2688 1979009.5 1980401.2142708756  36
2689 1888119.8 1910226.3839335432  37
2690 1945808.3   1918016.02465959  38
2691 2135982.8   2110634.51833571  39
2692 2010107.6 1993959.0365070107  40
2693   2104242 2085716.0995203136  41
2694 2065421.5   2077148.11258761  42
2695 2284106.5  2209581.296617582  43
2696 2269975.8 2146603.2232350074  44
2697 2169933.8  2102358.160586511  45
2698   2906233  2911804.361906894  46
2699 2298776.8 2340913.6081403447  47
2700   2546124 2477690.8424711605  48
2701   2762817 2662765.7898659697  49
2702   3555371  3390290.971433864  50
2703   2043245 2081698.7053137247  51
2704   1964702  2001179.980176376  52
2705 1911510.6  1895304.266263699  53
2706   1892776 1889421.3737781884  54
2707 1761016.5 1806324.9522780625  55
2708 2203523.3 2088624.3876176486  56
2709 2462978.3  2250786.089757569  57
2710 2309025.3 2329579.7960310616  58
2711 2045837.5 2077369.4210477152  59
2712 2148822.8  2142044.787785865  60
2713 2139265.5 2147104.7052015765  61
2714 2064991.8 2082545.5857324344  62
2715   1992437  2053333.000008729  63
2716 2074721.8 2081682.6168110485  64
2717   2565260 2407624.8084499035  65
2718   2045396  2088275.556112432  66
2719   1884428 2052563.4208852863  67
2720   1886504  2008584.142419789  68
2721   2163511  2149230.018335039  69
2722   2168097 2151356.4006817113  70
2723 2039222.3 2131071.9805964944  71
2724   2114989  2162141.796189582  72
2725 2143126.5 2165129.9223711425  73
2726 2231962.3  2179746.042952236  74
2727 2165160.3  2181527.418325735  75
2728 2060588.8  2143363.387459722  76
2729 2055952.6 2105844.4576511127  77
2730 2358055.3  2221945.230754105  78
2731   2134680 2100316.0673973393  79
2732 1970170.3 2089657.7378558333  80
2733   1911559 1982087.6325705436  81
2734 2094515.8 2104457.6560244313  82
2735 2144245.5 2130054.8379171505  83
2736 2045061.3  2096435.792700404  84
2737 2005341.4 2141300.8312830976  85
2738 2062481.5 2163723.5667367815  86
2739   2080529  2126920.559755352  87
2740   2047950 2025123.0650306006  88
2741 2028587.3 2017550.0877933886  89
2742 2008350.6 2032619.2215914018  90
2743   2246412  2187527.242232185  91
2744 2162951.3 2120167.6670080377  92
2745 1999363.5 2071027.8578589933  93
2746 2031650.5 2074906.8585628944  94
end
format %tw date

loc int_time = tw(2011w1)


labvars we date cf "Sales (in Millions)" "Week" "SCUL Predictions"
cls
tsset date, w

twoway (tsline we, lcolor(black) lwidth(medium)) ///
(tsline cf if rel >=0, lcolor(blue) lwidth(thin) lpattern(dash_dot)), ///
legend(ring(0) pos(10)) xli(`int_time')

Same game as described above- pick a random store, random intervention time and predict the next 94 weeks of sales. We have a pre-intervention RMSPE of 76,000 (which again for millions is simply nothing impactful) and a treatment effect of -6869 dollars lost on average over the course of the next 94 weeks, which is pretty meaningless if you ask me.

Given how it performs here, this isn't a bad way of doing time series prediction, much as other folks have discussed. My algorithm isn't as fancy as theirs, but it does a pretty darn good job.