python - Tracking Django/FastCGI Process Errors -


i run django based site on nginx fastcgi server. site works great. every 2-3 days, site run unknown problem , stop responding requests.

munin graphs shows io blocks read & write per second increases 500% during problem.

i wrote python script record the following stats every 1 minute.

load averages cpu usage (user, nice, system, idle, iowait) ram usage swap usage number of fastcgi processes ram used fastcgi processes 

the record shows during problem, number of fastcgi processes doubled (from normal value of 10-15 25-30). , ram usage fastcgi processes doubled (from 17% 35% of total ram on server). memory usage increase required more swap used slow down disk io made server unresponsive.

fastcgi parameters used:

maxspare=10 minspare=5 maxchildren=25 maxrequests=1000  

i guess problem due poorly written python code in part of site. don't know how find out part of code froze existing fastcgi processes , forking new instances.

you've limited number of children 25 when there 25 processes running , processing requests further ones block , site appear not responding.

it sounds me have infinite (or long) loop causing processes block. suggest add idle-timeout fastcgi script. have effect of allowing site continue killing long running queries, , let debug problem sending tracebacks processes killed.


Comments

Popular posts from this blog

apache - Add omitted ? to URLs -

redirect - bbPress Forum - rewrite to wwww.mysite prohibits login -

php - How can I stop spam on my custom forum/blog? -