Cache in Vanilla

Grahack

Klod suggested a cache system for Feed Publisher.
1) I found nothing in our forums about caching in Vanilla. Is there something I missed ?
2) I wonder if caching could be done at a higher level (through an extension) and if this could be a better idea than cache only the feeds...
3) Could someone point me on some guidelines or free classes to see where to begin ?

As I imagine all those feeds clients pinging 24 hours a day, caching things for just 1 hour could be a significant gain. But maybe I'm wrong...


EDIT: found ob_start() in init_vanilla.php and settings.php, but I didn't found (yet) the end of the buffering

WallPhone

As far as a core ob_end function, I don't believe there is one. I think it just dumps to the screen when the script terminates.

I was considering a caching extension also, but would have limited it to cache only the first page of the discussions grid for guests (cached for guests only, since this is different for every logged in user, depending on which discussions have been read, whispers etc.) and caching discussion threads that don't contain any whispers.

I never have enough time to play around programming, so I'll let you know some pretty basic Vanilla caching ideas:

• You can nest ob_start() calls to grab just a portion of the page.
• Cache only the database-intensive portions of the page--the actual CommentGrid & DiscussionGrid. You probably could use the PreRender delegate to insert a ob_start() and PostRender to use ob_get_contents(). (this may not actually work... but its worth a try)
• The cache should be invalidated by both a time limit and an update to the data that made that cache--so delete the discussion grid cache file when someone posts anything new, and delete the comment grid file if someone edits or posts in that discussion
• The hard part may be skipping the normal execution of Vanilla's comment grid and discussions grid if a cache file is found and displayed. Easiest approach might be to cache through the page end and just exit the script right after displaying the cache. A better and more flexible approach would be to extend the CommentGrid and DiscussionGrid classes to skip their execution so that things like the comment form will still execute normally and do things like remember the user's Text/HTML selection
• Timestamps will be a problem: "1 hour ago" could actually turn out to be several hours ago.
  My thought on this would be to store the raw timestamp in the cache file, then parse them out and run a timediff() against that when you display the cache so that the timestamps stay relevant.

And for something completely different--instead of caching the HTML output, cache the data that comes back from the database. You would only have to extend one core Vanilla class (library/framework/Framework.Class.MySql.php), and wouldn't have to worry about timestamps, user settings, etc.

An easy way to do this would be to take a MD5() of the query as it is sent to the database, if you have a cache file that is named the same, just return that instead of going to the database. If the file is too old or doesn't exist, create one with the data that comes back from the database. Have the script delete all cache files older than some time limit before checking for a match.

The hard part of this approach is invalidating the cache file when the underlying data is updated.

Grahack

thanks Wallphone for these useful thoughts
WallPhone:I think it just dumps to the screen when the script terminates.
This is right, buffer is flushed even with no ob_xxx() function at the end of the PHP execution.

WallPhone:The cache should be invalidated by both a time limit and an update to the data that made that cache.
This is clearly what I think I'm not capable of (especially after reading your post!!!). I only thought about lagging the update by a few minutes, for quiet forums.

What I could try to do is to make a very simple cache system in an extension, that Feed Publisher could use.
If I set a kind of filter system on it (what to be cached or not, depending on different criteria: user or guest, page, extension in use...), this could be quite flexible and usable by other extensions or the whole forum.

But you didn't answer one of my questions: do you think feeds are a big part of a server load ? I don't have a "big traffic" web site, so I have no logs to study...

Thanks again.

WallPhone

Feeds are interesting. I never gave much thought to feeds. I presume they would be much simpler to cache then Vanilla itself--no timestamps, no whispers, no user preferences etc. You can name the cache file the same as the DiscussionID and just delete it whenever there is a comment posted to that DiscussionID. You could even do some .htaccess trickery to point the feed requests directly to the cache file so that PHP isn't even involved in serving a feed file--unless it doesn't exist. (this is known as 'Baked' server side caching) For another approach, you could presumably track and return last-modified and e-tag information (HTTP 1.1 client-side caching) for each discussion and just send that back to the feed reader when it asks. I don't know how much difference that would make as opposed to full blown caching. I'm afraid I don't know how many feed readers in the wild respect HTTP 1.1 caching, or even how much traffic they generate. It probably differs depending on audience. Maybe someone else could give some sample numbers?

adrian.

what about using memcache on the database class?

adrian.

I know yayhooray implemented it as some point and it seemed to speed up the database calls...

Grahack

@adrian
I didn't find yayhooray's work on memcached on our forums. It could be nice, but I won't work on this now, maybe in a dedicated extension...

@ WallPhone
I didn't know about .htaccess trickery or HTTP 1.1 client-side caching... I'll try to find docs about that.
I won't get into your parsing timestamps trick for now. Anyway, isn't it a bit CPU intensive ?
But my biggest problem is skipping the normal execution of Vanilla, as you mentionned before. I need skipping code both for this cache extension, and (just realised lately) for the Feed Publisher extension.
Using exit; is not a very good idea !!! You suggested to extend some classes, ok but how to replace the original ones by the new ones. There is the $Page->AddRenderControl(...); method, but is there a RemoveRenderControl method ?

EDIT: maybe the trick is with

$Context->ObjectFactory->SetReference("MyNeatClass", "AnotherClass");

sushilkar

Time is not working properly