python - What is the most performant way to store a list of Tuples in App-Engine? -


when storing , retrieving datastore entity contains list of tuples efficient way of storing list?

when have encountered problem tuples key value pairs, datetime , sample results, (x, y) coordinates.
number of tuples variable , ranges 1 few hundred.

the entity containing these tuples, need referenced quickly/cheaply, , tuple values not need indexed.

i have had problem few times, , have solved number of different ways.

method 1:

convert tuple values string , concatenate them delimiter.

def putentity(entity, tuples):   entity.tuples = ['_'.join(tuple) tuple in tuples]   entity.put() 

advantages: results readable in datastore viewer, fetched in 1 get. disadvantages: potential precision loss, programmer required deserialize/serialize, more bytes required store data in string format.

method 2:

store each tuple value in list , zip / unzip tuple.

def putentity(entity, tuples):   entity.keys = [tuple[0] tuple in tuples]   entity.values = [tuple[1] tuple in tuples]   entity.put() 

advantages: no loss of precision, confusing still possible view data in datastore viewer, able enforce types, fetched in 1 get.
disadvantage: programmer needs zip / unzip tuples or maintain order in lists.

method 3:

serialize list of tuples in manor json, pickle, protocol buffers , store in blob or text property.

advantages: usable objects, , more complex objects, less risk of bug miss matching tuple values.
disadvantages: blob store access requires , additional fetch?, can not view data in datastore viewer.

method 4:

store tuples in entity , keep list of keys.

advantages: more obvious architecture. if entity view, no longer need keep 2 copies of tuple data.
disadvantages: 2 fetches required 1 entity , key list , 1 tuples.

i wondering if knows 1 performs best , if there way haven't thought about?

thanks, jim

i use method 3. blobstore may require fetch, db.blobproperty not. objects important comes out of storage put in use pickleproperty (which can found in tipfy, , other utility libraries).

for objects need state stored wrote jsonproperty function works pickleproperty (but uses simplejson, obviously).

for me getting data in single fetch, , being idiot-proof, more important cpu performance (in app engine). according google i/o talk on appstats, trip datastore going more expensive bit of local parsing.


Comments

Popular posts from this blog

apache - Add omitted ? to URLs -

redirect - bbPress Forum - rewrite to wwww.mysite prohibits login -

php - How can I stop spam on my custom forum/blog? -