Has Vanilla development gone private?

Bleistivt · July 2020

I usually like to read up on all commits in vanilla/vanilla daily. Having built several applications on top of the framework I find it highly interesting and useful to know where the core development is going and what I may have to change to adapt my plugins to future versions.

For some time now, it seems all core development has taken place in a private repository, which was synced back in one large commit with the public repository once.

I understand that there has been a move from vanilla/addons to vanilla/vanilla (core) and some previously proprietary parts have been moved into the core recently, so I figured, this was a temporary measure to prevent leaking internals. However it has been three weeks since the last regular commit, so I'm wondering if private development is the new norm?

Linc · July 2020

Seems reasonable to infer that, given this PR description.

Bleistivt · July 2020

@Linc Thanks for linking the PR. However my question towards the current development team still stands: Is the core development taking place in a private repository from now on?

Linc · July 2020

Since I’m unclear if that was implicitly including me, I’ll add I parted ways with Vanilla in February, and had been on leave since October, so I am far out of date on any plans. I haven’t seen anyone on staff comment here since I left, so I will provide what little I know anyway.

We never discussed going private during my time there, but I cannot think of any other rationale for a new private repo named “vanilla-cloud” and the public actions noted. My guess is they went monorepo (since there was a public poll about that, too, and being familiar with the pressures the React ecosystem added to the mix) and are doing a sync to the original repos. We were previously doing the inverse of that workflow for our private security patch repo (minority of patches on private; periodic sync via remotes). That’s all I know.

phreak · July 2020

@Adrian, @Todd and @charrondev: Can you share any information on what is planned with Vanilla OS?

charrondev · July 2020

I can shed a bit of light here. @Linc is on the right track here. We've essentially done the following.

Consolidated all repositories into a mono-repo.
This repository currently is being synced on an ad-hoc basis. I've been working on a CI integration to sync it daily, but have had some difficulties. The intention is for things to be synced daily though.
This is an inversion of how we worked with our security patches repo before.
Vanilla OSS releases will continue to happen.
Vanilla 4.0 is still coming this year with a few major new features for OSS.
- New theming system and new default theme.
- Numerous improvements to our APIv2 endpoints.
- SearchAPI and a new version of our advanced search are now open source (MySQL search is open source), Sphinx search will still be kept in cloud, but it is a pluggable system that anyone from the OSS community can write their own driver for (Sphinx, Elastic, Solr, whatever you want).
- User cards.
- Approximately a bajillion bug fixes.

I've had what's likely a bit of burnout since the start of the year which is why I've been a lot less active here. @initvector will be the one to push forward that release. The main blockers right now are still defined here.

https://github.com/vanilla/vanilla/issues/9888

Bleistivt · August 2020

Thanks or the answer @charrondev

I really appreciate you shedding some light on this. I am sorry that a lot of the community management burden of seems to fall on you as you are the only core member still actively responding here.

With the danger of going a little off topic let me explain how I feel about this recent change and how I think it could be leveraged to improve the Vanilla OS environment for everyone.

For the last years I experienced somewhat of a Vanilla fatigue ("burnout" being a too strong word for me). This is also why I am not as active in this forum as I used to be. It can be boiled down to two reasons: The pull request process and the inefficiencies of this forum.

The Pull Request Process

It may come to no suprise that I am (as most people here) myself a frequent user of Vanilla, both on the user and admin side. I have encouraged my users over the last 7 years to report any bugs or glitches they encounter.

Pretty much all of my open source contributions are a direct result of my users reporting bugs to me. I usually patch those for my production environment and then submit a pull request. In the last years it has become increasingly hard not having to maintain a fork of the software.

Just upgrading to the latest open source version before these are merged would introduce regressions my users would immediately notice ("haven't you fixed that some time ago?"). I had to write countless throwaway plugins containing method overrides or dirty hacks to be able to upgrade the forums.

This is not a critique of the team! I understand there are priorities and reviewing and testing code changes takes a lot of time.

I can totally understand the reasons for using a monorepo as well. Daily syncs will probably be a good substitute for regular commits in the vanilla/vanilla repo. To be clear however, there is still a loss for OS contributors. Not having the info about why something was changed, cripples the ability to track down bugs.

I am currently in the process of tracking down a bug with announcements and user-specific discussion data. I don't know if I'll be successful, but without the background information provided by pull requests, issues and git blame data, I don't think it could be done at all.

However, with the new process, I think there are also great opportunities on the horizon. Since commits in vanilla/vanilla no longer directly affects Vanillas production environment, they could all go to a "os-patches" branch that could be merged faster. Faster merges would be more encouraging for new contributors and people who need certain fixes could pull from that branch.

The Inefficiencies of This Forum

The Addon Application

The addon application has been aging very badly and I don't think it is needed very much anymore.

With a large part of the plugins being incompatible with recent versions, sorting by "most downloads" and the confidence rating become meaningless. Trying out plugins is very frustrating for new users since the majority does not work out of the box.

With most of the Vanilla created plugins soon being part of the core, the main reason, people still visit the addon repository on this site is most likely locales. But all locales should probably be included in the OS disctribution anyway. This is how most sofware seems to do it and it would eliminate the problem of having an outdated locale after updating. vanilla/locales zipped size is currently 4 MB which I don't think would be a problem to add to the standard download package.

For all user created plugins I propose a simple alternative: discussions

In my opinion, discussions are actually more suited for plugin discovery:

New plugins will get more attention as they appear on the recent discussions page
The discussion can be directly used for feedback: Every plugin having a discussion from the start removes the friction of creating one to ask a support question.
With a dedicated category for plugins, they can be found using the regular search, which will likely bring up more relevant results anyway.
This would also mean that the standard moderation and curation tools apply to plugins: New version can receive reactions and plugin threads could be closed if the plugin is unmaintained.

The only caveat I see here is the EditContentTimeout setting, which prevents a discussion starter from adding a new version to the first post. But this is also a good thing as attachment uploads can't be silently switched out that way, which will be just as secure as the current solution, if not more. However, it would require moderators to add a new version to the first post or link the comment with the attachment.

Something has gone wrong.

This is the most asked question on this site. It could be eliminated with a simple message at the top that reads something like this:

Something has gone wrong.

If you encounter this message, please edit the following value in your configuration file /conf/config.php before you ask for help:

$Configuration['Debug'] = true;

Reloading the page will show the actual error which you can add to your question.

Conclusion

I love Vanilla. I have built highly customized communities fully on Vanilla because it is a joy to develop against the framework and I believe in its future. There are many modern community solutions out there, but the value of software like Vanilla that scales and is battle-tested can't be overstated.

Since this was a little long, here is a TL;DR about the relatively cheap changes which, in my opinion, would enhance Vanillas open source environment drastically:

A message at the top containing debug instructions
Forum discussions for plugins in place of the addons app
A branch "OS-fixes" on vanilla/vanilla containing community patches
Ship the core with all locales

Linc · September 2020

A message at the top containing debug instructions

Forum discussions for plugins in place of the addons app

A branch "OS-fixes" on vanilla/vanilla containing community patches

Ship the core with all locales

Solid suggestions. I'd quibble the branch for patches is solving the problem in a way likely to cause other problems, but I understand the sentiment and agree the status quo has long been frustrating, as someone managing indie Vanilla-based sites myself.

If someone has some suggested copy for debug instructions, that's still within the purview of something I could assist with, since I still have admin permissions here.