It's still a ways off from reaching an open source release, but I wanted to followup with a bit of information about how we create our indexes for the discussion/comment table.
Unfortunately we hadn't fully evaluated our use of these indexes in production, and ran into some production issues (extra load, and locking tables while creating indexes). As a result we removed these index from comments and discussions.
In the end for our own production usage we do not want these indexes due to the extra DB load they create, but recognize that the OSS community relies on them.
@TwistedTwigleg would it be possible for you to add these indexes for your plugin specifically? If that's not possible we could explore a flag in Vanilla, that your plugin could opt-in to and enable these indexes.
I have not looked through the PR in detail, but from what I can gather, these changes remove the full text from the SQL database? That would limit this plugin a little bit, as then searches would be limited to discussion title only (which is currently bugged, I still need to fix it...) and QnA filters.
I need to take a closer look at the PR, but with my current understanding of the PR and the changes, I do not think it will be a major issue, it just might make things interesting. Worse case I might be able to make a tool that scans discussions and creates a table with the necessary data or something, at least in theory. My lack of PHP knowledge might be an issue, but nothing some learning and experimentation cannot fix.
Do you know which indexes will be effected?
Right now I’m really busy with school work, and I have not had time to work on the plugin, so it may take awhile before I can make any big changes. I expect things to lighten up in the a few weeks into the new year, but it will be difficult to make big changes before then.
Thanks for asking, I appreciate it. I’ll try to take a closer look at the PR soon (and I’ll try to fix the discussion title not appearing in search results)
Search results will not include duplicates of the same topic. Right now this is enabled by default with no way to turn it off, but in the future I plan to include this as an option. This was required because if you search a topic's title you will get a result for every comment within that topic.
Cleaned up code. Everything should be just a tad faster.
Updated the plugin description to mention the use of the Bootstrap theme.
Huge thanks to everyone for their patience! My apologizes on the delay in getting this update out.
@charrondev I looked through the changes and I am still uncertain how the changes will impact my plugin, and what changes will be needed if the plugin is affected. If possible, can you export a flag in Vanilla for my plugin to use, and then if my plugin ends up not needing the features, I can let you know? That way if it does cause breaking changes to the plugin then there is no loss of functionality while I try to find an alternative. Due to the limited amount of time I have due to college work, I am hesitant to take on a larger project than I have the time for and I do not want to leave anyone relying on the plugin without a solution if I can help it.
Comments
No problem. It will be easy to fix
If you want the german translation, let me know and I add a pull request.
It's still a ways off from reaching an open source release, but I wanted to followup with a bit of information about how we create our indexes for the discussion/comment table.
After evaluating a change to how our SQL driver handles FULLTEXT indexes proposed by @R_J (important for searches without a dedicated search engine like sphinx) we rebased the PR and pressed forwards with https://github.com/vanilla/vanilla/pull/9470.
Unfortunately we hadn't fully evaluated our use of these indexes in production, and ran into some production issues (extra load, and locking tables while creating indexes). As a result we removed these index from comments and discussions.
In the end for our own production usage we do not want these indexes due to the extra DB load they create, but recognize that the OSS community relies on them.
@TwistedTwigleg would it be possible for you to add these indexes for your plugin specifically? If that's not possible we could explore a flag in Vanilla, that your plugin could opt-in to and enable these indexes.
I have not looked through the PR in detail, but from what I can gather, these changes remove the full text from the SQL database? That would limit this plugin a little bit, as then searches would be limited to discussion title only (which is currently bugged, I still need to fix it...) and QnA filters.
I need to take a closer look at the PR, but with my current understanding of the PR and the changes, I do not think it will be a major issue, it just might make things interesting. Worse case I might be able to make a tool that scans discussions and creates a table with the necessary data or something, at least in theory. My lack of PHP knowledge might be an issue, but nothing some learning and experimentation cannot fix.
Do you know which indexes will be effected?
Right now I’m really busy with school work, and I have not had time to work on the plugin, so it may take awhile before I can make any big changes. I expect things to lighten up in the a few weeks into the new year, but it will be difficult to make big changes before then.
Thanks for asking, I appreciate it. I’ll try to take a closer look at the PR soon (and I’ll try to fix the discussion title not appearing in search results)
Awesome, and thank you for the offer! If you want to add a German translation, then I am all for it. 🙂
Version 2.4.0:
Huge thanks to everyone for their patience! My apologizes on the delay in getting this update out.
@charrondev I looked through the changes and I am still uncertain how the changes will impact my plugin, and what changes will be needed if the plugin is affected. If possible, can you export a flag in Vanilla for my plugin to use, and then if my plugin ends up not needing the features, I can let you know? That way if it does cause breaking changes to the plugin then there is no loss of functionality while I try to find an alternative. Due to the limited amount of time I have due to college work, I am hesitant to take on a larger project than I have the time for and I do not want to leave anyone relying on the plugin without a solution if I can help it.
Thanks!
@TwistedTwigleg well done! Merry Christmas to you too :)
Thanks, and happy holidays to you as well! 🙂