[3.3] (Special) Character Encoding in Rich editor not working (Umlaute)

egroeg · April 2020

hi there,

when using the rich editor the special characters are not encoded correctly.

for: äöü

this is what i get: Ã¤Ã¶Ã¼

when I use the Advanced Editor with WYSIWYG it wörks fine.

web searching away (german only, sorry) I found it has to do with UTF-8 characters getting encoded as ISO 8859-1? but I have no idea, where one would change that? 🤷‍♂️

thanks for any hints.

R_J · April 2020

Normally that is encodedin your web page. The first few lines of this web page look like this:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">

Could it be that you are using a different theme than KeyStone? The encoding could be set in the conf/config.php, too. Maybe there is something different set (but I really wouldn't know whay someone should have done so).

egroeg · April 2020

https://open.vanillaforums.com/discussion/comment/260599#Comment_260599

that's exactly what I see here.

noticed that I also can't see the tick icons on the backend? not sure, if that is related?

(disabling all plugins didn't help either. )

egroeg · April 2020

Hello all,

did a lot of playing around - that's all I can do as a interested hobbiest, I guess - but nothing worked. 🙈

Thought this article on character encoding would get me on the right track. But everything looks ok, up to the point "Bad Transfer Encoding" saying: "If you’ve reached this point, the problem is likely caused by incompatible encoding in transit from the server to the client." Unfortunately, i have no idea how and where to check all that. ;)

Looked at the database via phpmyadmin, and noticed that everything seems to be stored correctly:

And while typing in and also while editing e.g. a comment the encoding in the rich editor is correct. It's only when I post the comment that the encoding is wrong. (see your Merry Christmas below @R_J ;) )

Thanks for any more hints!

R_J · April 2020

Is your forum public and would you post the url?

egroeg · April 2020

unfortunately i can't. has to stay private. :/

R_J · April 2020

The problem has been the default charset not being UTF-8 in combination with the inconsistent use of htmlspechialchars/htmlentities in the code of the Rich Editor. I've created an issue. There need to be a PR for the changes, too.

If anybody likes to: every call to htmlspecialchars and htmlentities must include 'UTF-8' as a parameter. If I were to assign labels to issues I would give it "Quickie" and "Good first contribution" 😉

egroeg · April 2020

thanks a lot for your help @R_J

egroeg · April 2020

one question remains, though: I still see scrambled symbolds in the Vanilla admin panel and with reaction icons (YAGA) on the front end.

Admin panel

YAGA user front end:

althouth they might be unrelated. idk 🤷

In the admin panel, there seem to be two issues:

The content-type seems wrong again?
the file location of vanillicon.woff seems wrong, as the forum is not in the root folder, but in a subfolder, which i specified in the .htaccess file via RRewriteBase /forum

R_J · April 2020

I have adviced to et the default charset in the bootstrap.before.php because setting it in the .htaccess didn't work. Therefore your server still has the default charset when serving non php files. That shouldn't be a problem, but nevertheless it would be the best to find out where you can configure your Apache so to set the charset there to utf-8.

The second one is a bug. /applications/dashboard/design/admin.css uses absolute paths for the fonts, while using relative paths would be needed. Open that file and search for this block:

@font-face {
 font-family: "vanillicon";
 src: url("/resources/fonts/vanillicon/vanillicon.eot");
 src: url("/resources/fonts/vanillicon/vanillicon.eot?#iefix") format("embedded-opentype"), url("/resources/fonts/vanillicon/vanillicon.woff") format("woff"), url("/resources/fonts/vanillicon/vanillicon.ttf") format("truetype"), url("/resources/fonts/vanillicon/vanillicon.svg?#vanillicon") format("svg");
 font-weight: normal;
 font-style: normal;
}

Change it so that it reads:

@font-face {
 font-family: "vanillicon";
 src: url("../../../resources/fonts/vanillicon/vanillicon.eot");
 src: url("../../../resources/fonts/vanillicon/vanillicon.eot?#iefix") format("embedded-opentype"), url("../../../resources/fonts/vanillicon/vanillicon.woff") format("woff"), url("../../../resources/fonts/vanillicon/vanillicon.ttf") format("truetype"), url("../../../resources/fonts/vanillicon/vanillicon.svg?#vanillicon") format("svg");
 font-weight: normal;
 font-style: normal;
}

R_J · April 2020

No new bug, by the way. It was just some kind of forgotten... 🤔🙄

egroeg · April 2020

great! thanks again @R_J the admin panel looks nice and shiny again. :)

regarding the issue with the YAGA ReactIcons:

seems to have to do with relative paths as well? somehow the CSS with the ::before part seems to call the vanillicon-variables from the wrong directory

myurl.com/forum/theme-boilerplate/src/scss/patches/_vanillicon_variables.scss:180

When it should be something like

myurl.com/forum/themes/theme-boilerplate/src/scss/patches/_vanillicon_variables.scss:180

...right? but there is no file anyway. and I can't find that line of code in any of the yaga files.

This happens only on the yaga /best page, not on any other discussion page.

sorry, to bug you again. I can more or less handle css, but all that scss, variables, callling stuff... that's all trickery to me. 🙈

R_J · April 2020

Your browser only deals with css. What you are seeing are helpers shown by the browser. There is a mapping between the css which the browser uses and the "source" scss which you might want to look at if you want to debug something. So you do not have to understand scss.

I'm not using YAGA myself so maybe @Bleistivt could give better insights

[3.3] (Special) Character Encoding in Rich editor not working (Umlaute)

Comments