Efficient Rolling Window Identification in Matlab -


i have time-series data missing data running estimation function on rolling windows. windows not uniform length , each variable has differing start , end dates. want remove windows have missing data. windows overlap, single missing observation remove great many windows consideration. want mapping each date windows contain it.

currently, have logical matrix has row every possible day , each column represents 1 of windows true values days of window. can subset matrix rows representing missing data, , whichever columns contain true values invalid windows. problem logical matrix gets large (10k x 10k ~100mb) , there can many. can convert sparse solves size problem, calculation of windows remove becomes slow when windows long.

this doesn't smell problem should resource intensive (either memory or computation), there better way?

edit: let me add example may little clearer. full set of dates range 1 100. windows 1:10, 2:11, 3:12 , on through 91:100 (these uniform, doesn't matter example). have series runs 5 25, has nan 17.

that 1 nan knocks out ten windows (8:17 through 17:26). want efficient mapping observation 17 windows 8:17. clearly, it's pretty easy when windows uniform length, efficient method when windows irregular?

logical comparisons cost little time , resources. sure bottleneck?

if window creation time-consuming, may want in while-loop, can skip few entries if encounter nan inside window.

if window creation fast, uniform lengths, can do

%# startend created according example, can whatever quick method startend = [(1:(100-windowsize+1))',(windowsize+1:100)']; nanidx = find(isnan(data))';  %'#  %# line temporarily creates logical array of size nwindows-by-numberofnans %# smaller nwindows-by-nwindows badwindows = any(bsxfun(@le,startend(:,1),nanidx) & bsxfun(@ge,startend(:,2),nanidx),2);  startend(badwindows,:) = []; 

Comments

Popular posts from this blog

apache - Add omitted ? to URLs -

redirect - bbPress Forum - rewrite to wwww.mysite prohibits login -

php - How can I stop spam on my custom forum/blog? -