Another charset/encoding problem. Plz help, vanillers

crash_D.

Hello,

I have a website in cp1251 charset (russian/cyrrilic) and the collation in db string fields for vanilla are also set to cp1251.. the definitions from the russian language pack i wrote myself are fine, but anything from the database turns to question marks..

edit: and when i input the values directly from the phpmyadmin interface, they become even more horrible...

any ideas?

thanks

Max_B

@crash_D.: did you read an understand the memo I wrote on this matter?
Even if you do not intend to switch your database to utf-8 (a move i'd recommend anyway), you have there all informations needed to track down your problem.

SkyBoy

Hi,
When you input something on the forum (post some comments), is it correct in phpmyadmin?
When you input the values directly form phpmyadmin interface, did you change your browser's encoding to cp1251?
Did you edit the configure file to set the database character encoding?
If you change your browser encode to ISO-8859-1, are you view the correct character on your forum or phpmyadmin?

Also, I think the utf8 encode will be better.

crash_D.

thanks guys, i know that utf8 is the best, but i told you, the site is on cp1251 and I have integrated the site's main menu and header into vanilla, so that it looks like the section of the site, rather than this site's forum... so, turning to utf8 is out of the question (although next time i will use it for sure)..

When you input something on the forum (post some comments), is it correct in phpmyadmin?
No! they turn to question marks..

When you input the values directly form phpmyadmin interface, did you change your browser's encoding to cp1251?
it remains ok in the phpmyadmin, but on the website it looks awful... the encoding in the meta tag is correct (shall i also declare it in in xml encoding?)

Did you edit the configure file to set the database character encoding?
Yes, but no result

If you change your browser encode to ISO-8859-1, are you view the correct character on your forum or phpmyadmin?
No,, it turns to entities like &1033;

DraganBabic

Here is how I just solved my encoding issue according to this guide.

First of all, I must say that I have very little knowledge of server-side stuff - if any at all.

OK, so you have a forum that won't display UTF characters - here's what you do. Most likely you have installed Vanilla 1.1.2 on a MySQL version 5.x. Your content is non-encoded according to the guide mentioned above, here's what you do:

1) Go to phpmyadmin and export your database as an .sql file, save it to your hard drive
2) Opet it up in a text editor (I use editPlus2), find the table that contains comments - surely you'll notice ?s or some weird characters in place where proper characters should be. Go ahead and edit some of those - really just type in some of those characters manually (replacing the screwed up ones) and try to save the file. Surely your txt editor will ask you to try saving the file with another encoding, since the characters you just typed in don't match the current encoding. Save it in UTF8 (without the BOM, or the "signature" as it is somewhere called). There is one more thing you need to do - in the .sql file find every instance of "CHARSET=latin1" and change it to "CHARSET=utf8". Save the file again.
3) Go to phpmyadmin and drop your old database. Create a completely new one (same name as the old one is ok of coarse).
4) Through phpmyadmin, select the newly created database from the select box on the left, and from the upper menu clisk the "Operations" link
5) When the new page loads find the "Collation" box and select utf8_unicode_ci. Submit the changes.
6) Import the database back from your computer.
7) Enjoy your utf compatible forums.

Hope this helps, this is how I have done it, maybe it is not the most efficient way, but hey, I'm just a designer, I'm suprised I managed to fix at all.

SkyBoy

@crash_D.
"it remains ok in the phpmyadmin, but on the website it looks awful... the encoding in the meta tag is correct (shall i also declare it in in xml encoding?)"
I think it doesn't helpful, but you can get a try. :P

It seems some database collation isn't correct (in my experience). Could you double check all collation in your vanilla database are cp1251?

edit: May you check the collation of the body field in the LUM_Comment table (in the LUM_Comment's structure)?

Max_B

Crash_D:
Your posts show that you did not fully understand this charset stuff.
You can have cp1251 on html/browser side and utf8 in database, or the reverse, or both cp1251 or both utf8, all combination can be sane if you correctly set up the connection charset.
The wiki page method can be used to switch from any to any.
To keep it simple:
if you want everything cp1251, change every occurrence of utf-8/utf8 in Vanilla source to cp1251, AND be sure to also have
$Configuration['DATABASE_CHARACTER_ENCODING'] = 'cp1251';
in your settings.php

The current database content will probably be difficult to clean up but at least new inputs should be ok.

crash_D.

MySQL charset: UTF-8 Unicode (utf8)
db collation: cp1251_general_ci
lum_comment db structure snippet: Body text cp1251_general_ci
lum_role structure snippet: Description varchar(200) cp1251_general_ci
source of the forum contains:
<?xml version="1.0" encoding="windows-1251"?> and <meta http-equiv="Content-Type" content="text/html; charset=windows-1251" />

what can it be? please help guys

alast

having struggled with it myself, all i can tell you: follow Max_B' instructions :) the point is the combination of collation and connection so, export your db, clean it up from any occurances of anything but 'cp1251', save it in teh right encoding and import it back this should work :)

Max_B

@crash_D.:
… have you set-up the CONNECTION charset to cp1251 as suggested in my previous post?
your database is cp1251, your forum also BUT if your connection default to latin1 you are in the "worst" syndrome!

past the 2 following php lines in Vanilla index.php file, just below the include("appg/init_vanilla.php"); line :
$res=mysql_query("SHOW VARIABLES LIKE 'character_set_%'"); while ($v=mysql_fetch_object($res)) echo "$v->Variable_name : $v->Value <br />\n";
display the index page, look at the top lines, past them here. They should all be cp1251.
remove the two debug lines

And, one more time, switching a whole site to utf8 in not such a big task, Golive or DW can do it for you and you'd be ready for the future…

crash_D.

Max_B, i hope you meant 'paste', rather than 'past',,, couse i pasted those lines there, and nothing changed... i substituted the "character_set_%" with "cp1251" but nothing changed... and i left these two lines in the index page,,,, they did not change anything or output anything.... feels like they changed nothing....

I am desparate...

i could turn to utf8 if it were my site,,, but this one is a complex project, which uses distant importing tools, which are all in 1251,,, well, in my case, it IS problematic to switch to utf8.... guys, what could i do? the funny thing is that i ahve already configured vanilla twice for 1251 encoding... and every time i have a problem (solved them until now)... but this one is a new problem i haven't had before.... please help guys.,,, any suggestions would be greatly appreciated... thanks
* Description: Display and manipulate discussions */ include("appg/settings.php"); $Configuration['SELF_URL'] = 'index.php'; include("appg/init_vanilla.php"); $res=mysql_query("SHOW VARIABLES LIKE 'cp1251'"); while ($v=mysql_fetch_object($res)) echo "$v->Variable_name : $v->Value <br />\n"; // 1. DEFINE VARIABLES AND PROPERTIES SPECIFIC TO THIS PAGE // Ensure the user is allowed to view this page $Context->Session->Check($Context);

Max_B

But, crash_D, you ask for help and do not even answer precisely to asked questions!
For the third and last time: did you add, in your conf/settings.php the following line ?$Configuration['DATABASE_CHARACTER_ENCODING'] = 'cp1251';Also, I certainly made the "paste" typo twice in the same post (it was very late here), but DO NOT change the code I posted. It should output 7 lines on top of the page. This is a debug hack and theses lines might be hidden by some of your theme elements so look for them in the generated html code of the forum index page.
If they do not show up, nor any error message, that's probably a clue that you MySQL version is < 4.1, but you posted about collation setting which is > 4.1. What is it?
Give precise technical info.

crash_D.

Max_B, I apologize for the panic I have made around here... I have pasted the $Configuration['DATABASE_CHARACTER_ENCODING'] = 'cp1251';line the first time you mentioned it.

But, I have asked my friend, a Microsoft junkie I mentioned in the text editor interrogation, and he found the problem and solved it... he simply changed cp1251 to utf8 in the line above and it worked. I don't know how he found the solution, but it is all over now. Input strings are fine in both db and website. Please have a look at that version of vanilla at this suburban estate website (β). BTW, I have integrated the dynamic site menu into the forum, looks cool. Don't know if this a normal thing to do, but decided to boast

Thanks guys,
Especially you, Max_B.

p.s. that's why I love Lussumo; because of the "very friendly" community members, as Mark said in one of his video tutorials (which I think we all have seen).. Thanks again, comrades.

Discussion closed.

Max_B

Discussion closed.
Not yet! The link you posted shows three obvious warnings at the page top! You didn't remove my debug lines, remove it!.
And I bet you are in a messy/"worst" configuration. I'm rather sure that a sql dump of this database will NOT show cp1251 strings, neither utf-8… Your friend kludged a kludged setup…

crash_D.

I haven't noticed any differences even after removing the debug lines. .. I have just removed them..

And the second part of your post I did not understand:
"And I bet you are in a messy/"worst" configuration. I'm rather sure that a sql dump of this database will NOT show cp1251 strings, neither utf-8… Your friend kludged a kludged setup…"

Could you please clarify

thanks

Cerebellum

I have same problem. Who help us? Please!

Max_B

@Cerebellum
Be more precise in your request. Also reading and understanding this thread and the wiki page it refers to should give you a good starting point.