vanilla with a large userbase

kl

We are going to replace our existing forum with Vanilla. The stats are 80k daily posts and 30M pageviews a month. The users table has 2M rows, though most of them are inactive. The comments table has nearly 30M rows. We expect these figures to grow at a steady rate. I would like to make sure our install can handle a database of this size. I've followed the directions here: http://markosullivan.ca/blog/?p=491 , but I worry about what other problems might be lurking around the corner. Does anyone have any pointers about what I might tweak to deal with a forum of this size?

headxot

wow 80k posts per day? damn..
i'm very interested in knowing how this turns out in terms of performance

i have limited knowledge in terms of performance since i haven't deployed the forum yet. But that was also my #1 concern and i did heavy research here on the forums.

Some issues have been brought regarding heavy use of extensions, in particular a few have been pointed (like stats, tags, etc). With that amount of traffic you are most likely running already a stellar server, so that would also be a key aspect.

kl

It seems to me like I will need to add FULLTEXT indices on columns like LUM_Comment.body and LUM_Discussion.name (and maybe a few others). I don't see search working otherwise...unless there's something I'm missing?

x00

There are so many ways to reduce load, and speed up thing outside of the database. There are already extensions that exist like LowCal Vanilla. One of the tricks you can use is write an extension that at peek loads will turn off all the string formatters. Then put a notice up and leave it up to them to turn them back per user session. One of things that slows a site down is all the parsing like BBcode. Sometimes people just want to read the text, and get the speed benefit. Database wise there isn't much available yet. However Optimize Tables does two important things. Defrags the database, and prunes the IP address table which can grow pretty large Databases are not great search engines. Fulltext indexing usually isn't enough. I may sphynx binding extension at some point. I also believe you can implement with a MySQL extension, though not as good.

x00

I recommend caching, although grahacks simple cache is a bit rudimentary. However it is adequate for basic page caching and you can treat users and non-user differently, etc. I might write one, however I think that caching is one of the few thing that is best done in the core. That is because there are various types of caching like data, page, fragment, the list goes on. However I would like the idea of having extensions to implement different caching engines, and also come up with a rules system for non-programmers. Query caching in the MySQL is possible but quite limited.

Dinoboff

(Low-Cal Vanilla doesn't help with the server load, it even gave more work to the server. It help the page loading faster on the client side)

x00

Me.....and speed up things

TomTester

I was initially going to whisper this... but... tip #1 for vanilla is to turn off the whispering system, which enables you to turn on SimpleCache http://lussumo.com/addons/index.php?PostBackAction=AddOn&AddOnID=331 . Mark had a post on his personal blog a while back that may give some insight into optimizations: http://markosullivan.ca/blog/?p=491 Low-cal vanilla may increase server load because it employs GZIP, but... the reduced connections should allow you to handle many more concurrent users. I'm finalizing the launch of a new site. Came to the conclusion that 3-line CSS files added to the header by useful but silly little add-ons makes no sense, so I've manually moved the CSS into vanilla.css and blocked the respective 'add to header' code in the add-on, then turned on Apache's Mod_Deflate for CSS/JS/via htaccess. A big difference reduction in connections per page (and is in line with YSlow recommendations). TT PS IMHO simple is always better (think twitter, think TCP/IP, think the Internet) At the risk of pissing off some people here, might I call attention to an older discussion on simple approaches to speed up vanilla: http://lussumo.com/community/discussion/6014/release-canceled-but-code-below-vzippy-vanilla-packer-alternative-occams-razor-rules/

x00

Like I said I mentioned sphynx. However you can also turn off search altogether in peek time, and just refer them to a Google site search (but don’t do this permanently as user will be up in arms). Also many counts aren’t necessary. Some of the queries in vanilla 1 aren't that efficient. You can patch queries without modifying the core, via extensions. It is trickier to make optimisation that is to say add a column. Vanilla 2 should have more efficient queries.

x00

I can't stand css tit bits. that is why I never put them in my extentions.