c# - Storing related timed-based data with variable logging frequencies -
i work data logging system in car racing. developing application aids in analysis of logged data , have found of query functionality with, datasets, datatables , linq useful, i.e. minimums, averages, etc.
currently, extracting data native format data table , post-processing data. working data channels logged @ same rate, i.e. 50 hz (50 samples per second). start writing logged data database platform independent, , extraction process doesn't have happen everytime want analyze dataset.
which leads me main question... have recommendation best way store data related time, logged @ different rates? have approximately 200 channels logged , rates vary 1 hz 500 hz.
some of methods have thought of far are:
- creating datatable data @ 500 hz using double.nan values between actual logged samples
- creating separate tables each logging frquency, i.e. 1 table 1 hz, 10 hz, , 500 hz.
- creating separate table each channel relationship time table. each time step indexed, , table of data each channel not dependent on fixed time frequency
i think i'm leaning towards index time stamp separate table each channel, wanted find out if has advice on best practice.
for record, datasets can range 10 mb 200-300 mb depending on duration of time car on track.
i have single data store houses entire season, or @ least entire race event, considering well.
thanks advice!
can create table like:
channel, timestamp, measurement
?
the database structure doesn't need depend on frequency; frequency can determined amount of time between timestamps.
this gives more flexibility can write 1 piece of code handle calculations on channels, give channel name.
Comments
Post a Comment