Best Of

I did one of these ~6 months ago and I'd like to followup on it and give out some additional roadmap context.

How'd we stick to our previous roadmap

Accomplished Goals

• Shipped new Embed System. (Documentation)
• Alt-text for images in Rich Editor.
• Emptied the Security Backlog (Over 300 security fixes since launching our HackerOne program!)
• Testing Pull Requests on CircleCI. (CircleCI)
• Improved automated testing - Code Coverage reporting & Visual regression testing.
• Refactored the text formatting pipeline (Gdn_Format is mostly deprecated. see FormatService, and DateTimeFormatter for new methods. )
• Splitting the frontend into different independant packages. (See /packages directory).

Still In Progress

• New Controllers & Routing. The new structure has matured significantly as we've worked on new projects, and documentation is forthcoming.
• Rich Editor mobile improvements.

Abandoned

• Builtin Docker Images - I was working on this personally and got burnt-out of it before I could get it polished enough for code review. Unfortunately I don't have much motivation for it. It looks like the community is starting to take it up though. I'm available for questions of anyone working on projects like this.

New Roadmap Objectives

Warning: These notes are not a guarantee of any particular version number or release date. It just represents some of our development goals.

Complete the 4.0 release

The 4.0 release is currently in progress. Main improvements will include:

• New Base Theme & New theming system.
• Hero Image plugin no longer a plugin. Instead it is built-in to core.
• 100s of bug-fixes
• Performance improvements.
• New PHP version requirements.

Documentation Overhaul

As we move our developer documentation over to it's new home in our Success Knowledge Base, we are updating outdated articles, adding new ones, and moving some new technical documentation from our internal documentation to become public.

Items that are set to receive new documentation:

I'll link the documentation here as it becomes available.

• New Controllers & Routing.
• Using Twig for Views.
• Using React for Views.
• Using the new Theming system.
• Using the new FormatService.
• Advanced custom embeds for Rich Editor.
• Improved local installation instructions.
• i18n & Localization.
• New Model Class & Pipelines.
• Dependency Injection.
• New Javascript & Style loading for addons.

As we write these we will primarily looking to provide "quickstart" guides with code examples. For for more technical understanding it's best to look at the key source code files, which will be linked at the bottom of documentation, where possible.

I've been using Vanilla for about 5 years, and have developed a maintenance process along the way that has broken down. My goal was to version control our setup (customisations, theme, plugins), and allow for automated deployments like the rest of our stack. I did this by forking the repo and adding to it: for plugins that were on GitHub, I added them as submodules; when they weren't on GitHub, I added their contents to our repo.

This worked for the most part, but upgrading the forums was always a bit of a pain, as we had to rebase and deal with any merge conflicts. We now have a huge number of merge conflicts, to the point that I'd like to start over and make a better setup.

The goal is to create a version controlled forum setup, with reproducible builds, that can run on an ephemeral filesystem (even heroku) so we can load balance it and even blow it away and reconstruct it if necessary. The twelve-factor app provides some background and general principles for this.

To do this, I figure I'll need to:

1. Start from a specific Vanilla release: either a distributed package (zip) or the GitHub codebase (in which case I'd need to run composer).
2. Fill in config.php with settings and secrets, without putting secrets into version control.
3. Pull in our theme, either from a separate git repo or from a directory within this repo, and move it to the themes directory of the Vanilla codebase.
4. Do the same for 10+ plugins, some of which are on GitHub, others which are just .zip files on the addons website.
5. Set conf , cache , and uploads to be writable by PHP.
6. Somehow make uploads a separate volume (like S3) so it can be shared across multiple instances and can persist (even be backed up).

I've started implementing this in a Dockerfile. You can see my progress so far at this repo.

Where I need help

1. config.php

I'm having some trouble with config.php because Vanilla writes to it when an admin makes a change in the admin dashboard: it replaces getenv(...) with the plaintext value. Therefore, if you want to commit any changes made in the dashboard back to source control (say, by mounting config.php as a volume and then making the changes), you'd be committing secrets unless you remember to go back in and change them back to getenv . Solutions to this I've thought of so far are: (a) make it read-only, so the application cannot change it, or (b) consider it a secrets file, outside of source control. But then how would you do an automated deployment? Is there a way to stop Vanilla from writing to config.php ? I would think the database would be better suited for some of these settings, no?

2. Pulling in the theme and plugins

The simplest approach here is for the Dockerfile to just curl and unzip the theme and plugins. I can either put them in the appropriate place in the Vanilla codebase, or symlink them; it doesn't make much difference, I don't think. But it starts to feel like the job of a dependency manager. I could use composer here, but I'd have to either move the installed directory or symlink it to the appropriate place, and I don't think composer can do that, can it? Does anything else come to mind here?

Let me know if there's been any prior work on this, or if anyone wants to collaborate. I'm sure there's ways we can make the repo generic.

There is no tutorial on that and the pager is quite complex, thow you do not need to do much to get it going. After you have set up a working pager, I recommend taking a closer look at the class.pagermodule.php to find out more about its possibilities.

The best way to get going is to look at a page where you can find a pager, e.g. one of the discussions lists (recent discussions, bookmarked, mine etc.) You can find that code in /applications/vanilla/controllers/class.discussionscontroller.php

Let's look at the mine() method, which is quite short, but I stripped out what is not necessary to get a working example:

   public function mine($page = 'p1') {
...
       // Set criteria & get discussions data
       list($offset, $limit) = offsetLimit($page, c('Vanilla.Discussions.PerPage', 30));
...
       $countDiscussions = $this->setData('CountDiscussions', $discussionModel->getCount($wheres));
...
       // Build a pager
       $pagerFactory = new Gdn_PagerFactory();
       $this->EventArguments['PagerType'] = 'MorePager';
...
       $this->Pager = $pagerFactory->getPager($this->EventArguments['PagerType'], $this);
       $this->Pager->MoreCode = 'More Discussions';
       $this->Pager->LessCode = 'Newer Discussions';
       $this->Pager->ClientID = 'Pager';
       $this->Pager->configure(
           $offset,
           $limit,
           $countDiscussions,
           'discussions/mine/%1$s'
       );

       $this->setData('_PagerUrl', 'discussions/mine/{Page}');
       $this->setData('_Page', $page);
       $this->setData('_Limit', $limit);
...
       // Deliver JSON data if necessary
       if ($this->_DeliveryType != DELIVERY_TYPE_ALL) {
           $this->setJson('LessRow', $this->Pager->toString('less'));
           $this->setJson('MoreRow', $this->Pager->toString('more'));
           $this->View = 'discussions';
       }
...
   }

The main element in there is the following block:

       $this->Pager->configure(
           $offset,
           $limit,
           $countDiscussions,
           'discussions/mine/%1$s'
       );

The pager module needs to be configured with those information: where does the current page starts (offset), how much elements does it show (limit), how much elements are there in total (countDiscussions) and where should the previous and next page links should point to.

But look at method code step by step:

   public function mine($page = 'p1') {

In your plugin this will most probably be

   public function pluginController_foo_create($sender, $args) {

The $args array will contain everything that comes after yourforum.com/plugin/foo/...

Therefore, if your url scheme for e.g. the third page should be /plugin/foo/p3, you should have the following code for getting the page for building the pager: $page = $args[0] ?? 'p1';

If your url scheme is /plugin/foo/bar/baz/p3, the required code would be $page = $args[2] ?? 'p1';

       list($offset, $limit) = offsetLimit($page, c('Vanilla.Discussions.PerPage', 30));

This line does a lot! You should look up the function offsetLimit in functions.general.php. What's happening here is that the variables $offset and $limit are filled based on the $page variable and the config value for Vanilla.Discussions.PerPage. I would encoure to make your view configurable in the same way.

       list($offset, $limit) = offsetLimit($page, Gdn::config('Plugins.FooBar.PerPage', 30));

And if it makes sense, you can directly use a Vanilla PerPage setting as default!

        list($offset, $limit) = offsetLimit(
           $page,
           Gdn::config(
               'Plugins.FooBar.PerPage',
                Gdn::config(
                    'Vanilla.Discussions.PerPage',
                    30
                )
            )
        );

The following lines in the snippet are split because that way a plugin author can influence the look and feel of the pager of the mine() method.

       $this->EventArguments['PagerType'] = 'MorePager';
...
       $this->Pager = $pagerFactory->getPager($this->EventArguments['PagerType'], $this);

You can condense it to this line

       $this->Pager = $pagerFactory->getPager('MorePager', $this);

By the way: in most cases when you look at Vanilla code and see $this, you will have to replace that with $sender (if you stick to Vanillas conventions)

Frankly spoken, I'm not sure if you need this block:

       $this->setData('_PagerUrl', 'discussions/mine/{Page}');
       $this->setData('_Page', $page);
       $this->setData('_Limit', $limit);

ANd you have to decide by yourself if that makes sense for you:

       // Deliver JSON data if necessary
       if ($this->_DeliveryType != DELIVERY_TYPE_ALL) {
           $this->setJson('LessRow', $this->Pager->toString('less'));
           $this->setJson('MoreRow', $this->Pager->toString('more'));
           $this->View = 'discussions';
       }

In your view, a simple PagerModule::write() should be sufficient.