FilterForumSearch plugin

TwistedTwigleg · June 2019

Hello everyone,

For the forum I host, I needed to create a plugin that would allow for filtering searches, without the use of additional software (like Sphinx), as my server host does not support it. This lead to the creation of the FilterForumSearch plugin:

You can find the plugin here: https://open.vanillaforums.com/addon/filteredforumsearch-plugin-2.0

The plugin supports the following filters:

Category
Q&A status
Comment/Reply count
Username

The plugin has no dependencies outside of Vanilla and the Q&A plugin. This allows it to work with Shared hosting plans and other server plans that otherwise wouldn't allow for filtered searching through additional software. The plugin works by using an SQL query to retrieve data based on the parameters passed in, using Vanilla's SQL interface to (hopefully) minimize SQL injection potential.

Hopefully this plugin will be helpful for others who are looking for search filters but cannot install additional software.

Motivation behind creation

One of the members of the forum I host mentioned that they would prefer additional search parameters/filters to make it easier to find if similar questions have already been answered by using the search page in Vanilla. To me, this was a reasonable request, so I set about finding a solution.

One of the admins on the forum suggested the SphinxSearch plugin, which would have worked great and provided everything we need. Unfortunately, I discovered that my server provider does not have Spinx support with the hosting plan I have bought, making this plugin unusable for the forum.

I found the CategorySearch plugin, which uses SQL queries to filter the search, which looked like exactly what I was looking for. However, the plugin didn't work with the version of Vanilla the forums is using, 2.8.3 currently. After trying to get it to work for a few hours with no result, which is probably due to my lack of PHP experience, I decided to rewrite the non-working parts of the plugin entirely, using a slightly different method than the original.

Having made this decision, I set about rewriting a good chunk of the plugin, using the CategorySearch plugin as a reference. After a couple days of work, testing, and debugging, I finally had a working plugin that filters searches using direct SQL queries.

After this, it was pointed out to me that using SQL queries directly could introduce security vulnerabilities, so I rewrote the plugin to use Vanilla's SQL interface instead and removed all direct SQL queries, hopefully removing and vulnerabilities I introduced through the plugin.

Once I had that working, I cleaned a few things up, changed the plugin info, and uploaded it both here and to the GitHub repository holding all of the modified plugins for the forum I'm hosting.

Special Thanks

Huge thanks to @R_J, the creator of the CategorySearch plugin! Using the plugin as a base was extremely helpful and greatly sped up the time it would have taken to write the plugin if I wrote it entirely from scratch.

Note

One major thing I am not sure about, is the PHP code I wrote for the plugin.

While I have SQL experience, I only have used PHP minimally, so the code may be poorly written. If someone with PHP/SQL experience does not mind looking at the plugin and it's code, I would highly appreciate it! I think the code is probably stable and secure enough for most cases, but I'm concerned that I might have missed something obvious.

donshakespeare · June 2019

You sir, are a complete winner. I will check it out. You may also want to look at this other rather out-of-this-world plugin. I am about to give a small review on its latest version

https://github.com/rbrahmson/VanillaPlugins/tree/master/FilterDiscussion

We thank thee, gentle [TwistedTwigleg], for thy pains;

And to thy worth will add right worthy gains.

Thy pains, [TwistedTwigleg], shall not be forgot;

Right noble is thy merit, well I wot.

-King Richard II

TwistedTwigleg · June 2019

Thanks @donshakespeare!

I will look into the plugin, as it looks helpful.

Side note: Small error in the plugin with searching using a username. I have a fix on the way however that should solve the bug.

TwistedTwigleg · June 2019

Version 2.0.1:

Fixed issue where searching based on username would crash the plugin. This happened because the query to convert the username to user ID was messing up the main query. Moving the query before the main query fixed the problem and now it should be possible to search by username again.

R_J · June 2019

That looks really great! I haven't tested it but only looked at the code on GitHub. There is nothing much to say and I'm surprised you say you have no real experience with PHP. I only spotted two "errors" which are in fact "notes" and no real errors.

You've messed up the JS file. I guess you wanted to include a file that toggles the advanced options, but you have attached the wrong file.

The string "'Input text and/or a Username to search.'" is not translatable. You need to enclose it with t(), or even better Gdn::translate(). t() is just a shortcode to Gdn::translate() but someday in the future, t() will not be supported any more.

Another thing you might want to change is the getView() call. You should replace it with a fetchView()since getView already is deprecated.

You do not need to add parenthesis on each line in an array definition like you have done herehere:

	$AnswerDropdownOptions = array(   '' => ('All Discussions and Questions'),    'Answered' => ('Only Answered Questions'),    'Accepted' => ('Only Accepted Questions'),    'Unanswered' => ('Only Unanswered Questions'),    'NoQA' => ('Only Discussions (No Questions)'),    'OnlyQA' => ('Only Questions (No Discussions)'));

But beyond that: congrats to that piece of work!

TwistedTwigleg · June 2019

Thanks @R_J!

I took a class that briefly went over PHP or two several years ago, but that I haven't really touched PHP since. That said, normally I program games and have used a variety of programming languages over the years, so maybe that helped. Now that I'm hosting a forum, I'm trying to brush up a bit so I can write and support plugins.

As for the JS file, I will implement those changes soon!

Thanks you for the original plugin, for looking over everything, and for the feedback! I really appreciate it!

phreak · June 2019

Amazing @TwistedTwigleg. @R_J and I developed a similar "Advanced Search" plugin but never released it to the public. Great there is a new take on this! :)

TwistedTwigleg · June 2019

Thanks @phreak!

Version 2.0.2:

Made changes based on feedback by R_J:
- Converted all t() function calls to Gdn::translate()
- Replaced getView with fetchView
  - This required moving the PHP and CSS file to the view folder, as I couldn't get fetchView to find the file any other way. Probably something on my end, but I couldn't figure it out and concluded it is probably fine to move it.
- Reworked JS file, then realized the plugin was no longer using it and worked fine without it, so I removed the file.
Removed bloat .DS_Store files i accidentally included.

donshakespeare · June 2019

Some observations ... I found this one a bit more critical hence my haste in giving feedback.

In class.searchmodel.php, you have hard-coded a bit of the comment url. Vanilla allows plugins to customize this, and users like myself have this completely customized. I suggest utilizing whatever is the existing construct, like so ...

$comment = new CommentModel();
$comment = $comment->GetID($Value["CommentID"]);			
$Ret_Val["Url"] = commentUrl($comment, $withDomain=true) ;
// $Ret_Val["Url"] = "/discussion/comment/{$Value["CommentID"]}#Comment_{$Value["CommentID"]}"; //<-- this will break things for users like me

Cheers!

TwistedTwigleg · June 2019

Thanks @donshakespeare! I didn't realize there was a construct for this, though I should have looked into it instead of just hard-coding.

I just pushed a new version, 2.0.3, that fixes this issue. Thanks again!

pioc34 · July 2019

Hi TwistedTwigleg,

i made some commits to enable translate of your plugin

in index.php

$AnswerDropdownOptions = array(

'' => Gdn::translate('All Discussions and Questions'),

'Answered' => Gdn::translate('Only Answered Questions'),

'Accepted' => Gdn::translate('Only Accepted Questions'),

'Unanswered' => Gdn::translate('Only Unanswered Questions'),

'No_QA' => Gdn::translate('Only Discussions (No Questions)'),

'Only_QA' => Gdn::translate('Only Questions (No Discussions)'));

$AnswerDropdownFields = array('TextField' => 'Text', 'ValueField' => 'Code', 'Value' => $ADV_Filter_SearchedQNA);

$CommentCountDropdownOptions = array(

'' => Gdn::translate('All Discussions'),

'over_zero' => Gdn::translate('Only Over 0 comments/replies'),

'over_one' => Gdn::translate('Only Over 1 comments/replies'),

'over_two' => Gdn::translate('Only Over 2 comments/replies'),

'over_five' => Gdn::translate('Only Over 5 comments/replies'),

'over_ten' => Gdn::translate('Only Over 10 comments/replies'));

'<button type="button" class="sidebar-toggle" data-toggle="collapse" data-target=".advance-search-collapse">',

Gdn::translate('Additional Search Filters'),

'</button>',

My fr.php file

$Definition['Input text and/or a Username to search.'] = 'Entrer du texte et/ou un pseudo pour rechercher';

$Definition['No results for %s.'] = 'Aucun résultat pour %s. ';

$Definition['No results for <b>%s</b>.'] = 'Aucun résultat pour <b>%s</b>. ';

$Definition['Filter by Category'] = 'Filtrer par Catégorie';

$Definition['Filter by Q&A'] = 'Filtrer par Questions/Réponses';

$Definition['Filter by Comment Count'] = 'Filtrer par Nombre de Commentaires';

$Definition['Filter by Username (case sensitive)'] = 'Filtrer par Pseudo (Sensible à le casse)';

$Definition['All Discussions and Questions'] = 'Toutes les Discussions et Questions';

$Definition['Only Answered Questions'] = 'Seulement les Questions Répondues';

$Definition['Only Accepted Questions'] = 'Seulement les Questions Acceptées';

$Definition['Only Unanswered Questions'] = 'Seulement les Questions sans Réponses';

$Definition['Only Discussions (No Questions)'] = 'Seulement Discussions (Pas de Questions)';

$Definition['Only Questions (No Discussions)'] = 'Seulement Questions (Pas de Discussions';

$Definition['All Discussions'] = 'Toutes les Discussions';

$Definition['Only Over 0 comments/replies'] = 'Seulement plus de 0 Commentaires/Réponses';

$Definition['Only Over 1 comments/replies'] = 'Seulement plus de 1 Commentaires/Réponses';

$Definition['Only Over 2 comments/replies'] = 'Seulement plus de 2 Commentaires/Réponses';

$Definition['Only Over 5 comments/replies'] = 'Seulement plus de 5 Commentaires/Réponses';

$Definition['Only Over 10 comments/replies'] = 'Seulement plus de 10 Commentaires/Réponses';

$Definition['Search Text'] = 'Texte Recherché';

$Definition['Submit search to apply filter(s)'] = 'Soumettez la recherche pour appliquer le(s) filtre(s)';

$Definition['Additional Search Filters'] = 'Filtres de Recherche Additionnels';

pioc34 · July 2019

direct css injection is deprecated, you should

in class.filteredforumsearch.plugin

replace

$Sender->AddCssFile($this->GetResource('views/style.css', FALSE, FALSE));

by

$Sender->addCSSFile('style.css', 'plugins/FilteredForumSearch');

and put style.css file in a design folder

That's all thank you for this plugin!

pioc34 · July 2019

If i try to search with the title of a discussion, search returns 0 result...

TwistedTwigleg · July 2019

Thanks for the suggestions and the translation file @pioc34! I will make the changes once I have the ability/time to.

As for searching with the title of a discussion, currently the plugin only searches for text within discussions and doesn't account for the title of the discussion itself. I might be able to add support for searching by discussion name in the future, I'll add it as a feature request in the repository.

Thanks again!

TwistedTwigleg · July 2019

Version 2.1.0

Added more Gdn::Translate function calls for better localization support. Moved style.css to avoid direct CSS injection
- (Thanks @pioc34 for the suggestions!)
Changed how searching works by default
- Now by default searches will look for the search text both within the title and content/text of discussions.
- This behavior can be changed on a per search basis and is exposed through a new drop down menu.
  - Options include searching only within discussion content/text, title, or searching in both.

donshakespeare · July 2019

Good work.

Here is another semi major issue.

The plugin does not signify the search section name. Inspect the body element in the browser, this is missing 'Section-SearchResults'.

So, any custom smarty logic in the theme or another PHP plugin or even mere CSS rules that depend on Section-SearchResults, will ultimately fail.

Example of codes that will fail

.Section-SearchResults .SiteSearch{
 display:none;
}

or

if(inSection('SearchResults')){
  echo 'super duper js code';
}

Solution:

In class.filterforumsearch.plugin.php, insert the line within the function SearchController_Render_Before

Gdn_Theme::section('SearchResults');

Also, I suggest removing the label for the search field, vanilla does not come with one, and it is rather obvious what the search field is used for. A more common method is using a placeholder.

The above cleanup would prevent this glitch

donshakespeare · July 2019

Maybe you know already, but this your plugin comes as a very huge boon for those using categories as a kinda pseudo multi-site setup, where categories are used as major sections of a mother site.

And the attention you afforded the QnA plugin is superb.

Sphinx is awesome, but in many cases just not even possible.

TwistedTwigleg · July 2019

Thanks @donshakespeare! I will make the changes soon! I've been busy with non-online things recently and haven't had the time to see these posts or reply until now.

I totally agree that Sphinx is awesome, but as you said in many cases it is not possible, which ultimately lead me to working on this plugin. I actually didn't think about using categories as a pseudo multi-site setup, though I can totally see how it would be helpful there. I'm just glad the plugin is helpful to others, along with being what we needed for our forums, so ultimately everyone wins.

Thanks again! I will make the changes soon 🙂

TwistedTwigleg · July 2019

Version 2.2.0:

Added Gdn_Theme::Section to class.filteredforumsearch.plugin.php to make the plugin work with other plugins, Smarty code, and CSS.
- Thanks @donshakespeare for the feedback and suggestion!
Removed the search label and replaced it with a placeholder text to better match Vanilla's default search page.
- Thanks @donshakespeare for the feedback and suggestion!
Added placeholder text to the username input field.
Changed how searches are handled by default. Now inputted text is broken up into multiple search terms.
- This better mimics how most search engines work. For example, now when something like "cat dog" is searched, results containing "cat", "dog", "dog cat" and "cat dog" will be returned. Prior to this change the only results returned would have to contain "cat dog" exactly somewhere in the text.
- Results are ordered by date, not by occurrence match value/score. I don't know if this is doable using the LIKE query, I would need to investigate.
- Initially the plugin was going to use SQL WHERE MATCH (...) AGAINST (...) instead of SQL WHERE LIKE (...), however the results were not ideal so this was dropped. Perhaps in the future I'll try again, but I figure for now it is good enough to just use the LIKE query.
- Because this is still using the LIKE query, it isn't a perfect multiple term search. Changing the order of the search terms will give different results. If possible, I'd like to fix this in a future update.
Added a new filter: filter by occurrence.
- There are two modes: Return all occurrences and Return exact occurrences. Return all occurrences will break the text up into multiple search terms, while Return exact occurrences will only return occurrences containing the exact text inputted.
- Return exact occurences will give the same default behavior prior to this change (versions 2.1.0 and below).

Thanks to everyone that has downloaded and used the plugin! If anyone has any feedback, please let me know!

Kaspar · July 2019

@TwistedTwigleg Thumbs up. Really nice.

Might just be me being lazy but I am a little annoyed that I have to scroll/move the mouse up to the search field(above the filters), click the field and press enter in order to re-search/apply filters

In /views/index.php

replacing

	$Form->Label('Submit search to apply filter(s)', 'ADV_Search_Note'), ' ',

with

	'<button type="submit" class="sidebar-toggle" name="Search">',
		Gdn::translate('Apply filter(s) to search'),

gives a button below the filter to apply the filter.

Issue is it acts a little weird.

If there is nothing in "Username" then a search(initated by the new button) returns "Input text and/or a Username to search."

If I put a username it does search by clicking the new button - but the search field is emptied so the search only return results by username and not by original search word + username.

Anyone have inputs or wanna take this idea to the goal :-)

TwistedTwigleg · July 2019

Thanks @Kaspar!

@Kaspar said:

Might just be me being lazy but I am a little annoyed that I have to scroll/move the mouse up to the search field(above the filters), click the field and press enter in order to re-search/apply filters

...

gives a button below the filter to apply the filter.

That is not a bad idea. I'll add a search button at the bottom of the fields to the next update.