Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Try Vanilla Forums Cloud product
Vanilla 2.6.1 is here with critical security patches. One of them has been publicly disclosed.

Vanilla Performance Discussion

zserozsero New
edited September 2013 in Feedback

I've just done some tests, comparing Vanilla vs. MyBB vs. FluxBB and I found out that although I love the concept and the community features of Vanilla, it's 10x slower compared to FluxBB when under heavy load. Is there anything I can do to make it faster? I'm using APC what is used by FluxBB hence why it can be so fast. Isn't there a mod or patch to make Vanilla use APC?

I've written my detailed results here:
http://www.lowendtalk.com/discussion/5323/vanilla-vs.-fluxbb-vs.-mybb-siege-test

«13

Comments

  • x00x00 MVP
    edited October 2012

    APC you just turn on in php, you can use memecache. I recommend clearing activity periodically (this is something fixed in 2.1). Apache is slow, Vanilla performs better with nginx.

    Performance is a complicated subject so I'm not going to get into huge detail, but when you are doing benchmarking all you are doing is accessing how it performs (using that request method) on that setup. You could have performance acceleration software that would actually slow down your site if you don't have enough memory or capacity.

    grep is your friend.

  • zserozsero New
    edited October 2012

    I am on nginx / php-fpm / APC. Normally the website isn't slow, it's just slow compared to other forums in siege-test.

    Vanilla response -c100: 1.94 secs

    Vanilla response -c200: 2.03 secs

    MyBB response -c100: 0.25 secs

    MyBB response -c200: 0.73 secs

    FluxBB response -c100: 0.04 secs

    FluxBB response -c200: 0.08 secs

  • 50sQuiff50sQuiff ✭✭
    edited January 2013

    Are you running this benchmark from a clean install of each forum? I've just finished working on my box and the plain Vanilla 2.0 -c100 test is giving me 80.7 transactions per second on a Linode 512, which is three times faster than your results.

    In your test on similar hardware Flux delivers 193 trans/sec which is superb, but it's not radically (10x) better than Vanilla. Considering the functionality of Vanilla and Garden it's very fast indeed, provided you've got a properly baked server stack.

  • Activity tend to slow thing down as it grows, prune careful but periodically with a cron job, especially with 2.0

    You really don't need old activity that is transient. Just some messages.

    grep is your friend.

  • Just for the sake of evaluating the benchmark posted in this thread I used Vanilla straight out of the box. This is without even minifying the CSS/JS, let alone disabling Activity or other software optimisations.

  • phreakphreak VanillaAPP - White label iOS and Android App MVP

    A "Prune Activity" Plugin would be great. With some timeframe options ("Keep last 365 days", "Keep last 6 months").

  • LincLinc Director of Development Detroit Vanilla Staff

    What page of each site were you hitting, and was the same amount of data (or any?) loaded into each forum before these benchmarks were done?

  • LincLinc Director of Development Detroit Vanilla Staff
    edited January 2013

    My analogy is a post saying "Guys, I just bought Car A and Car B. When I floored them both, Car A got to the end of the road faster." My first questions would be: Did you put either car past first gear? Was there turning involved? Can you tell us about this road? Can you offer us any context or insight about your results beyond "this thing went faster when I typed in this command"?

    I'm obviously keenly interested in Vanilla's performance (and frankly very prepared to accept it isn't nearly the fastest kid in town at stock config - that hasn't been our foremost concern since we provide customized hosting for it which speeds it up considerably) but I don't see useful levels of info in what's been presented so far.

  • AnonymooseAnonymoose ✭✭
    edited January 2013

    @50sQuiff said:
    This is without even minifying the CSS/JS, let alone disabling Activity or other software optimisations.

    Unfortunately, activities can't be disabled in Vanilla. I wish they could be.

  • Im moving a couple of my big forums to vanilla. Love the architecture, but it is indeed slow compared to other forums :( Im going to bite the bullet and move to vanilla anyway.
    I hope version 3.x is all about performance improvements.

    I just imported one of my big forums into vanilla and apache benchmarked it against the original old phpbb 2.0 version of the forum. Used 100 concurrent clients for the test.

    phpbb 2.0: 303 requests per second
    vanilla: 22 requests per second

    I have tuned the hell out of my MySql database running on an SSD drive for this move, and I still get these low numbers. This is a pretty hefty dedicated server. Quad core xeon processors, lots of ram, etc.

    Vanillas queries look super heavy and beat the hell out of the MySql engine.
    I may have to custom hack to get the performance I want.

    I hope the vanilla dev team really take this seriously. It just doesnt scale well right now.
    Thats a big deal breaker for a lot of sites.

  • LincLinc Director of Development Detroit Vanilla Staff

    @jackjitsu What version did you test?

    I recommend Varnish.

  • Version 2.0.18.8

    Varnish will not help. Im already running on a highly tuned NGINX install with fast_cgi caching.

    When I enable nginx caching on my wordpress sites I get 25,000 requests per second.

    When I try this with Vanilla Forums it fails because the home page pushes out a Cookie and a no-cache header, both of which prevents caching by caching engines like nginx and varnish.

  • I did some more load testing with slow queries enabled in MYSQL to catch what goes slow under load. To my surprise, no query exceeded 3 seconds.

    I was running HTOP and watching. Looks like Vanilla Forums is really heavy on the CPU's, not so much the DB which was my original assumption.

    I NEED MORE POWER! LOL.

    UnderDog
  • x00x00 MVP
    edited September 2013

    yes vanilla's queries aren’t that bad.

    Never go on what a query looks like, optimised queries can look pretty heavy, and the optimisation is not just in the query itself.

    grep is your friend.

    LincUnderDog
  • businessdadbusinessdad Stealth contributor MVP

    If you would like to know what code is causing the performance issues, you could try using XDebug profiler and then a tool like KCacheGrind to inspect it, but beware that it may take a significant amount of time to both to collect and inspect the data.

    hgtonightUnderDog
  • LincLinc Director of Development Detroit Vanilla Staff

    @jackjitsu You will find 2.1 makes the necessary adjustments to run Varnish and probably fixes many of the queries you're concerned about. Varnish has been a tremendous asset to us.

    UnderDogShadowdare
  • I havent dug into the code yet to review the architecture yet. I guessed the rendering/templating was where most of the time is spent.

    Everything is a trade off, and im willing to trade some CPU for all the awesome hooks and plugin architecture Vanilla provides. I installed a bunch of plugins, then deleted the plugin directory and the forum still ran. PhpBB would explode instantly.

    A loosely coupled plugin architecture is an awesome thing, and worth some trade offs, but 10X CPU requirement is a bit much.

    Hopefully this will get better with time.

    UnderDog
  • @Lincoln said:
    jackjitsu You will find 2.1 makes the necessary adjustments to run Varnish and probably fixes many of the queries you're concerned about. Varnish has been a tremendous asset to us.

    Looked into Varnish.... now I see the trick used to make it work.
    Varnish is being configured to IGNORE no-cache headers for at least 60 seconds.
    Basically micro-caching. Which you can do in nginx as well.

    sigh In any case, I will have to use nginx or varnish as a no-cache header ignoring proxy. I dont like this additional complexity :(

    # Adjusting Varnish's caching - hold on to all cachable objects fo 24 hours.
    # Objects declared explicibly as uncacheable are held for 60 seconds, which
    # helps in the event of a sudden ridiculous rush of traffic.
    if (beresp.ttl < 24h) {
    if (beresp.http.Cache-Control ~ "(private|no-cache|no-store)") {
    set beresp.ttl = 60s;
    }
    else {
    set beresp.ttl = 24h;
    }
    }

    hgtonightUnderDog
  • @jackjitsu said:
    OK.... so I did the equivalent of Varnish but with NGINX. No need for more software ;-)

    Implemented a microcache using NGINX (Varnish replacement), proxying to the real NGINX instance on the back end port 8080. Used a tiny 2 second cache.

    Went from 22 requests per second to 20,000. :) Feeling better about this now.

    NGINX users really should not use Varnish. Using NGINX<--->NGINX natively is a lot easier/faster.

    @jackjitsu

    You sound like you know a heck of a lot about tuning. It would be cool it you wrote up a brief synopsis of what you did with some config, etc. examples, if you have the time. I think it would be enlightening to all of us.

    I may not provide the completed solution you might desire, but I do try to provide honest suggestions to help you solve your issue.

    hgtonightUnderDogAaronWebstey
  • I'll write something up after im done building, testing, and releasing the new forum.
    Caching with session cookies can be tricky, I want to shake out any weird proxy/cache bugs with real users on a live site before I post anything. The NGINX configuration is fairly simple, and ill post the setup after im confident it works. Im not an expert at configuring NGINX, and could easily make a mistake, but I am a scalability architect with 25 years IT experience ;-)

    hgtonight50sQuiffUnderDog
  • @jackjitsu

    anything you write up, will be great as a guide, i'm sure. looking forward to it.

    Im not an expert at configuring NGINX, and could easily make a mistake

    perfection although desired is not necessary, you probably know more than alot of us. :).

    I may not provide the completed solution you might desire, but I do try to provide honest suggestions to help you solve your issue.

    UnderDog
  • Well, as I suspected, there IS a better way.

    I ditched the NGINX PROXY to NGINX, and got the same results using a single instance of NGINX. Where is the mosh emote when you need one???

    The setup just became trivial, fast, with low complexity. Yes!

  • R_JR_J Cheerleader & Troubleshooter Munich Moderator
  • Would you mind sharing your nginx config with us @jackjitsu ?

  • I will. Let me finish my testing. Im real excited about the re-launch of my site using Vanilla right now.

  • meshugymeshugy Musician/Hacker ✭✭

    I was experiencing load times on 2.1b2 significantly slower than my old php forum. I enabled Xcache on my apache server and it sped things up dramatically!

    UnderDog
  • hbfhbf wiki guy? MVP

    @Todd said:
    We've done a lot of scaling optimizations which took load off of io, but has raised the load on cpu. This is due to having more calculation in the app rather than the database.

    We will be improving the cpu performance in coming versions. We do have some inefficiencies due to some options in the code that don't get much use and we will be removing support for some of these edge cases. I'll be talking about this after 2.1 gets its final release.

    One of our cpu bottlenecks though is rendering and that has to do with our theming flexibility. This is just going to be a fact of a Vanilla forum and I think it's a fair trade-off. We have a lot more power than phpbb or mybb and we're designed to work on more modern hardware.

    We build our software for scale. This means that our software won't perform as well on lesser hardware or on small communities. However, when the community is large we have the ability to parallelize a lot of the code and as a result we can run massive communities.

    not to sound recalcitrant but shifting load from backend to front end or vice versa only buys you a performance when you have an application that is woefully unbalanced to begin with.

    true optimization for performance comes only when you redefine the architecture such that the mundane or repetitive calculations and tasks are no longer necessary.

    as for generalized event management and other flexibility focused capabilities, i'd like to profile vanilla in my production environment to really see where the cycles are being consumed. My site was performing god awful a few months ago, I traced it down to some really poorly defined SQL statements. But at its best, Vanilla seems to be a bit sluggish.

    to be fair none of this is easy, i've designed more than my fair share of "enterprise class" solutions with poor performance due to architectural choices.

  • hbfhbf wiki guy? MVP

    @meshugy said:
    I was experiencing load times on 2.1b2 significantly slower than my old php forum. I enabled Xcache on my apache server and it sped things up dramatically!

    on this note, and building on my previous post. my 2.1b2 site was running about 5-10x slower than my 2.0.18.x site (loads that normally take 2sec were taking 10-20sec). This would be with the same content but actually fewer plugins, due to the problems with so many plugins an the new version.

«13
Sign In or Register to comment.