How does a typical medium to large web application "grow up" infrastructure-wise? -

the skinny: when facebook / twitter / youtube (whatever) went basic idea in software to... bigger (maybe 100,000 users?), how did grow?

is there "best practices" growth path medium sized web applications?

the real question: when specifying or bidding on medium sized web application project, biggies? in case in question, use php framework, seems these generalize language.

so programmers core application (to me) obvious part. user management, user interface, , special classes made handle application. seems me less half of real project.

ultimately, growth, infrastructure , meta-ui issues main focus, right?

1) infrastructure: cloud application space, data storage, db synchronization multi-datacenter situations.

2) language , cultural issues: making app seem "likeable" or @ least useable in major "culture markets"

3) data indexing issues

4) api / interoperability issues (both embedded apps ala facebook , external access data both end users , major players search engines, etc.)

...so, sure missing half of them, , have little idea how prioritize.

the accepted answer here pretty starting point answer seek.

expanding on item number 1 list seem pretty key, , decide kind of scaling issues going face, , illude classes of technology might useful. doing touch on items 3 , 4, kind of interrelated.

below large mess of questions might in exploratory mind set expand on scalability thoughts, not direct answer question starting point:

whats features in app like? read heavy or write heavy, or maybe both? when viewing data need realtime newest possible state time? or can delayed? how far can delayed? can caching help? caching easy part, think of how expire caches, thats hard stuff is. data working like? highly relational, or more separate documents?

what performance requirements? - app generate reports in background , email them out? or need display perfect realtime map of current tweets off of twitter happen? updates users need propagate other users? every user or subset of them? how fast have that? page need load in under 300ms or under 2s?

do external services have limitations of kind, maybe request limits, or rate limits? means you'll need queue , batch requests. of external data providers sending data faster can process it? might need queue up, might need make part of system scalable on own, variable number of "workers" can move , down. consider applying "individually scalable" principal other parts of system, pays dividends in large installations.

hope helps :)