Hi,
I have a question regarding two of Stata's filtering commands; "tsfilter bw..." and "butterworth...".
I am working with raw tri-axial accelerometer data files which are quite large (each approximately 1.5GB in size and ~ 90,000,000 observations over 3 variables (3 axes of accelerometer)). The raw data is unfiltered and thus needs to be "cleaned" - which I believe what the 'tsfilter' and 'butterworth' commands do. I also have a time variable which runs from 1 to the end of the dataset (again ~ 90,000,000).
Prior to running any of these filtering commands, I tsset the time variable.
But after running any of the filtering functions, I get the following errors;
COMMAND: tsfilter bw axis1_bw = axis1, order(4) maxperiod(4)
ERROR:
J(): 3900 unable to allocate real <tmp>[91433996,91434000]
Toeplitz(): - function returned error
_TSFILTER_getQ(): - function returned error
_TSFILTER_bwFilter(): - function returned error
<istmt>: - function returned error
COMMAND: butterworth axis1, freq(4) order(4) stub(a1);
ERROR:
I(): 3900 unable to allocate real <tmp>[91434000,91434000]
bwmata(): - function returned error
<istmt>: - function returned error
r(3900);
I know filtering data is supposed to be computationally very expensive/time consuming but I am not exactly sure what these errors mean - is this related to MATA? Furthermore, I am unable to find anything useful online (except this webpage where they get the same error for a different command - http://www.stata.com/statalist/archi.../msg00481.html).
In addition, when I drop most of the data and use these functions on a smaller subset of the data (~15,000 observations) it filters the data and works well but very slowly. I don't understand what to do.
Any suggestions and thanks in advance,
Kindest Regards,
Kishan Bakrania
I have a question regarding two of Stata's filtering commands; "tsfilter bw..." and "butterworth...".
I am working with raw tri-axial accelerometer data files which are quite large (each approximately 1.5GB in size and ~ 90,000,000 observations over 3 variables (3 axes of accelerometer)). The raw data is unfiltered and thus needs to be "cleaned" - which I believe what the 'tsfilter' and 'butterworth' commands do. I also have a time variable which runs from 1 to the end of the dataset (again ~ 90,000,000).
Prior to running any of these filtering commands, I tsset the time variable.
But after running any of the filtering functions, I get the following errors;
COMMAND: tsfilter bw axis1_bw = axis1, order(4) maxperiod(4)
ERROR:
J(): 3900 unable to allocate real <tmp>[91433996,91434000]
Toeplitz(): - function returned error
_TSFILTER_getQ(): - function returned error
_TSFILTER_bwFilter(): - function returned error
<istmt>: - function returned error
COMMAND: butterworth axis1, freq(4) order(4) stub(a1);
ERROR:
I(): 3900 unable to allocate real <tmp>[91434000,91434000]
bwmata(): - function returned error
<istmt>: - function returned error
r(3900);
I know filtering data is supposed to be computationally very expensive/time consuming but I am not exactly sure what these errors mean - is this related to MATA? Furthermore, I am unable to find anything useful online (except this webpage where they get the same error for a different command - http://www.stata.com/statalist/archi.../msg00481.html).
In addition, when I drop most of the data and use these functions on a smaller subset of the data (~15,000 observations) it filters the data and works well but very slowly. I don't understand what to do.
Any suggestions and thanks in advance,
Kindest Regards,
Kishan Bakrania
