php - Should I be using message queuing for this? -


i have php application has 5k users , keep increasing forseeable future. once week run script that:

  • fetches users database
  • loops through users, , performs upkeep each 1 (this includes adding new db records)

the last time script ran, processed 1400 users before dieing due 30 second maximum execute time error. 1 solution thought of have main script still fetch users, instead of performing upkeep process itself, make asynchronous curl call (1 each user) new script perform upkeep particular user.

my concern here 5k+ curl calls bring down server. remedied using messaging queue instead of curl calls? have no experience using one, i've read seems might help. if so, message queuing system recommend?

some background info:

  • this symfony project, using doctrine orm , mysql db
  • the server windows machine, , i'm using windows' task scheduler , wget run script automatically once per week.

any advice , appreciated.

if it's possible, make scheduled task (cron job) run more , use limit 100 (or other number) process limited number of users @ time.


Comments

Popular posts from this blog

apache - Add omitted ? to URLs -

redirect - bbPress Forum - rewrite to wwww.mysite prohibits login -

php - How can I stop spam on my custom forum/blog? -