Best Of

Hi all, I wanted to give you an update on the state of security practices at Vanilla, especially as it relates to HackerOne and our open source patch releases. I'll start with some background info, update you on the latest news, and then do a brief question & answer.

HackerOne background

HackerOne is a security bug bounty platform, which means it's a place for hackers to report discovered security issues in exchange for money, rather than using them maliciously. We launched our campaign there over a year ago and overall it's been very successful. Our rationale for doing so was that our open source users overlap very little with security researchers and therefore we receive little security scrutiny naturally. We decided it was in the best interest of both the company and the community to invest in more experienced folks investigating our software for problems.

In short, it's a very tangible way to prioritize security and invest in it, so we did so.

We've had an uptick in security releases since the program started, and moreso in the last few months. We expect to do more security releases before the end of the year. Basically: the program is working and our code base is being scrutinized more than it's ever been, so we're shipping patches as fast as we're able.

Some current metrics:

• We've awarded over $34,000 in bounties.
• In total, we've now evaluated 600 reports.
• Among them, we identified 139 legitimate reports from 73 different hackers.
• We average 1 day to ticket response, 6 days to triage, and 1 month to resolution.

"Resolution" for us means shipping an approved patch to both cloud and open source (all supported versions).

In the month of September we saw an abnormal surge in activity (more than 3x our normal level of legitimate reports) which caused some minor delays in delivering all patches.

Latest news

Yesterday, our company was blackmailed on Twitter regarding the open source release of a security patch. (Cloud customers were not affected by this threat; they were previously patched.) This was done by one of the HackerOne security researchers ("hackers") participating in our program (thus breaking the terms of service on HackerOne) who also made several accusations about our security practices. I will indirectly address some of those via question and answer below. We had awarded the hacker $3,000 in bounties, including $900 for the security patch in question prior to the blackmail.

The issue was submitted on Sept 18. 20 days ago, the hacker asked for an additional status update on the open ticket. We had previously explained the cause of delay (trying to bundle multiple security patches into 1 release to reduce the need for OSS updates) so we did not reply this time. 5 days ago, they asked again for an update and minutes later publicly messaged me a passive aggressive inquiry to my personal Twitter account (not the company's). We have escalation points listed in our campaign, we have a public company Twitter account, and my personal account is not tied to HackerOne in any way. I viewed this as an implicit threat and immediately blocked the user and ceased engaging over HackerOne. In retrospect, I should have passed on the HackerOne ticket to a colleague.

Yesterday, they included our company account on threats to 0-day our open source community (release details on an exploit for which no patch has been released) on Monday morning if the situation wasn't rectified to their liking. I am still unclear whether that happened. I later unblocked and engaged with them on Twitter to address some of the accusations made.

The direct consequence was a rushed late Sunday night release of Vanilla 2.6.4. In our haste, we accidentally included a config.php in the initial release that caused problems for some users. This was fixed by 8:20am ET this morning (thank you to Softaculous, @pioc34, and Twitter user console for quickly pointing out my mistake). We apologize for the oversight. We have fixed our tools to prevent a repeat.

You may recall the 2.6.1 release was similarly rushed. This was a process error on our part, not malicious activity. We accidentally agreed to disclosure of a security vulnerability that had not been patched in the 2.6 release. We've altered our process to prevent a repeat of that incident.

There's a lot of communication overhead to dealing with a HackerOne campaign in a growing company. We're learning quickly and adapting, and we apologize for the missteps we make along the way.

Question & Answer

How does Vanilla organize security releases?

We have a 5-day SLA on patching Critical exploits (usually we patch in 1 day). We backport all 'Critical' and 'High' severity patches to the latest stable releases of Vanilla. We patch cloud before open source. For Critical issues, this is usually a delay of minutes. For High issues, it may be delayed to open source by days or weeks to roll together multiple patches if they will soon be available. (We optionally backport 'Medium' severity patches at our discretion and do not backport 'Low'.)

Why does Vanilla sometimes hold security patches before releasing to open source?

The terms of our HackerOne campaign forbid disclosure until the report is resolved. We may hold a patch to group with additional patches in a single release if we have other similar severity reports open. We do this to avoid "update fatigue", where users start skipping updates if they are issued too frequently. We do not hold 'Critical' severity reports; they are always released immediately.

Why doesn't Vanilla issue CVE identifiers for reports?

Neither our customers nor open source community have requested this. It would require more time and steps to our workflow to accommodate this. In general, the only folks who have requested this are hackers seeking recognition. We previously provided public recognition in our release notes on this forum. We now provide it via HackerOne report disclosure. We do not contest the disclosure of resolved issues on HackerOne.

Why doesn't Vanilla make more explicit commit messages when patching security issues?

Sometimes we make commit messages that are either vague or technically accurate but not entirely forthcoming. We do this in the interest of not raising attention to an issue before we can finish the open source release process. Because our core software is entirely open source, a careful observer could gain awareness of a vulnerability and figure out how to abuse it before anyone has the option to update. Basically, we're buying you as much time as we can by not detailing every exploit and drawing attention to them. We understand some hackers take issue with this. We are more interested in protecting our users.

Why doesn't Vanilla have an automatic updater?

This is a huge feature fraught with its own security implications. We would support an open source effort to add this functionality to Vanilla, but we're not able to dedicate the resources necessary to it at this time so it seems very unlikely at this time. It remains the responsibility of each forum administrator to monitor our accounts for updates and do them in a timely manner. We use all available channels to communicate new releases when they are available (social media, email, and in-Dashboard feed). Before anyone says "but WordPress did it," kindly recall they are 1000x our size.

Thanks for reading and let us know if you have questions or suggestions in the comments.

Hi All,

Woke up to the notification of the new 2023.001 Open Source release this morning.

Built it from source and am happy to report that most of the issues encountered in previous builds now all appear to be fixed from what I can see so far.

PHP 8 now properly works without any of my previous workarounds - Tested on PHP 8.1.12 and the absolute latest PHP 8.2.1! All works as far as I can see! 😁

The Admin Dashboard CSS display error that's plagued pretty much every release since 2021.012 is also sorted out now.

The blank "User Profile" page from the previous 2022.025 release is now working.

Have attached the built release for you all to enjoy!

Merry Christmas all! :)

Here is the latest available Vanilla release (2022.024) built from source code, with the broken / blank Dashboard issue from some of the most current releases also fixed up. ☺️

Note: While I messed about and managed to get 2021.009 working on PHP 8, I have not had success with that yet for 2022.024, so this build will require PHP 7.4 still - but hey I figure that this is still an improvement on the last 2021.009 release!

TIP: Head over to the "Labs" page in Settings (/settings/labs) and enable the new Layout Editor, then go to the Appearance tab to customize the site layout :D