Vanilla 1 is no longer supported or maintained. If you need a copy, you can get it here.
HackerOne users: Testing against this community violates our program's Terms of Service and will result in your bounty being denied.

Another charset/encoding problem. Plz help, vanillers

edited January 2008 in Vanilla 1.0 Help
Hello,

I have a website in cp1251 charset (russian/cyrrilic) and the collation in db string fields for vanilla are also set to cp1251.. the definitions from the russian language pack i wrote myself are fine, but anything from the database turns to question marks..

edit: and when i input the values directly from the phpmyadmin interface, they become even more horrible...

any ideas?

thanks

Comments

  • Max_BMax_B New
    edited April 2007
    @crash_D.: did you read an understand the memo I wrote on this matter?
    Even if you do not intend to switch your database to utf-8 (a move i'd recommend anyway), you have there all informations needed to track down your problem.
  • edited April 2007
    Hi,
    When you input something on the forum (post some comments), is it correct in phpmyadmin?
    When you input the values directly form phpmyadmin interface, did you change your browser's encoding to cp1251?
    Did you edit the configure file to set the database character encoding?
    If you change your browser encode to ISO-8859-1, are you view the correct character on your forum or phpmyadmin?

    Also, I think the utf8 encode will be better.
  • thanks guys, i know that utf8 is the best, but i told you, the site is on cp1251 and I have integrated the site's main menu and header into vanilla, so that it looks like the section of the site, rather than this site's forum... so, turning to utf8 is out of the question (although next time i will use it for sure)..

    When you input something on the forum (post some comments), is it correct in phpmyadmin?
    No! they turn to question marks..

    When you input the values directly form phpmyadmin interface, did you change your browser's encoding to cp1251?
    it remains ok in the phpmyadmin, but on the website it looks awful... the encoding in the meta tag is correct (shall i also declare it in in xml encoding?)

    Did you edit the configure file to set the database character encoding?
    Yes, but no result

    If you change your browser encode to ISO-8859-1, are you view the correct character on your forum or phpmyadmin?
    No,, it turns to entities like &1033;
  • Here is how I just solved my encoding issue according to this guide.

    First of all, I must say that I have very little knowledge of server-side stuff - if any at all.

    OK, so you have a forum that won't display UTF characters - here's what you do. Most likely you have installed Vanilla 1.1.2 on a MySQL version 5.x. Your content is non-encoded according to the guide mentioned above, here's what you do:

    1) Go to phpmyadmin and export your database as an .sql file, save it to your hard drive
    2) Opet it up in a text editor (I use editPlus2), find the table that contains comments - surely you'll notice ?s or some weird characters in place where proper characters should be. Go ahead and edit some of those - really just type in some of those characters manually (replacing the screwed up ones) and try to save the file. Surely your txt editor will ask you to try saving the file with another encoding, since the characters you just typed in don't match the current encoding. Save it in UTF8 (without the BOM, or the "signature" as it is somewhere called). There is one more thing you need to do - in the .sql file find every instance of "CHARSET=latin1" and change it to "CHARSET=utf8". Save the file again.
    3) Go to phpmyadmin and drop your old database. Create a completely new one (same name as the old one is ok of coarse).
    4) Through phpmyadmin, select the newly created database from the select box on the left, and from the upper menu clisk the "Operations" link
    5) When the new page loads find the "Collation" box and select utf8_unicode_ci. Submit the changes.
    6) Import the database back from your computer.
    7) Enjoy your utf compatible forums.

    Hope this helps, this is how I have done it, maybe it is not the most efficient way, but hey, I'm just a designer, I'm suprised I managed to fix at all. :)
  • edited April 2007
    @crash_D.
    "it remains ok in the phpmyadmin, but on the website it looks awful... the encoding in the meta tag is correct (shall i also declare it in in xml encoding?)"
    I think it doesn't helpful, but you can get a try. :P

    It seems some database collation isn't correct (in my experience). Could you double check all collation in your vanilla database are cp1251?

    edit: May you check the collation of the body field in the LUM_Comment table (in the LUM_Comment's structure)?
  • Crash_D:
    Your posts show that you did not fully understand this charset stuff.
    You can have cp1251 on html/browser side and utf8 in database, or the reverse, or both cp1251 or both utf8, all combination can be sane if you correctly set up the connection charset.
    The wiki page method can be used to switch from any to any.
    To keep it simple:
    if you want everything cp1251, change every occurrence of utf-8/utf8 in Vanilla source to cp1251, AND be sure to also have
    $Configuration['DATABASE_CHARACTER_ENCODING'] = 'cp1251';
    in your settings.php

    The current database content will probably be difficult to clean up but at least new inputs should be ok.
  • MySQL charset: UTF-8 Unicode (utf8)
    db collation: cp1251_general_ci
    lum_comment db structure snippet: Body text cp1251_general_ci
    lum_role structure snippet: Description varchar(200) cp1251_general_ci
    source of the forum contains:
    <?xml version="1.0" encoding="windows-1251"?> and <meta http-equiv="Content-Type" content="text/html; charset=windows-1251" />

    what can it be? please help guys
  • having struggled with it myself, all i can tell you: follow Max_B' instructions :) the point is the combination of collation and connection so, export your db, clean it up from any occurances of anything but 'cp1251', save it in teh right encoding and import it back this should work :)
  • Max_BMax_B New
    edited April 2007
    @crash_D.:
    … have you set-up the CONNECTION charset to cp1251 as suggested in my previous post?
    your database is cp1251, your forum also BUT if your connection default to latin1 you are in the "worst" syndrome!

    past the 2 following php lines in Vanilla index.php file, just below the include("appg/init_vanilla.php"); line :
    $res=mysql_query("SHOW VARIABLES LIKE 'character_set_%'"); while ($v=mysql_fetch_object($res)) echo "$v->Variable_name : $v->Value <br />\n";
    display the index page, look at the top lines, past them here. They should all be cp1251.
    remove the two debug lines

    And, one more time, switching a whole site to utf8 in not such a big task, Golive or DW can do it for you and you'd be ready for the future…
  • edited April 2007
    Max_B, i hope you meant 'paste', rather than 'past',,, couse i pasted those lines there, and nothing changed... i substituted the "character_set_%" with "cp1251" but nothing changed... and i left these two lines in the index page,,,, they did not change anything or output anything.... feels like they changed nothing....

    I am desparate...

    i could turn to utf8 if it were my site,,, but this one is a complex project, which uses distant importing tools, which are all in 1251,,, well, in my case, it IS problematic to switch to utf8.... guys, what could i do? the funny thing is that i ahve already configured vanilla twice for 1251 encoding... and every time i have a problem (solved them until now)... but this one is a new problem i haven't had before.... please help guys.,,, any suggestions would be greatly appreciated... thanks
    * Description: Display and manipulate discussions */ include("appg/settings.php"); $Configuration['SELF_URL'] = 'index.php'; include("appg/init_vanilla.php"); $res=mysql_query("SHOW VARIABLES LIKE 'cp1251'"); while ($v=mysql_fetch_object($res)) echo "$v->Variable_name : $v->Value <br />\n"; // 1. DEFINE VARIABLES AND PROPERTIES SPECIFIC TO THIS PAGE // Ensure the user is allowed to view this page $Context->Session->Check($Context);
  • But, crash_D, you ask for help and do not even answer precisely to asked questions!
    For the third and last time: did you add, in your conf/settings.php the following line ?$Configuration['DATABASE_CHARACTER_ENCODING'] = 'cp1251';Also, I certainly made the "paste" typo twice in the same post (it was very late here), but DO NOT change the code I posted. It should output 7 lines on top of the page. This is a debug hack and theses lines might be hidden by some of your theme elements so look for them in the generated html code of the forum index page.
    If they do not show up, nor any error message, that's probably a clue that you MySQL version is < 4.1, but you posted about collation setting which is > 4.1. What is it?
    Give precise technical info.
  • edited April 2007
    Max_B, I apologize for the panic I have made around here... I have pasted the $Configuration['DATABASE_CHARACTER_ENCODING'] = 'cp1251';line the first time you mentioned it.

    But, I have asked my friend, a Microsoft junkie I mentioned in the text editor interrogation, and he found the problem and solved it... he simply changed cp1251 to utf8 in the line above and it worked. I don't know how he found the solution, but it is all over now. Input strings are fine in both db and website. Please have a look at that version of vanilla at this suburban estate website (β). BTW, I have integrated the dynamic site menu into the forum, looks cool. Don't know if this a normal thing to do, but decided to boast :)

    Thanks guys,
    Especially you, Max_B.

    p.s. that's why I love Lussumo; because of the "very friendly" community members, as Mark said in one of his video tutorials (which I think we all have seen).. Thanks again, comrades.

    Discussion closed.
  • Discussion closed.
    Not yet! The link you posted shows three obvious warnings at the page top! You didn't remove my debug lines, remove it!.
    And I bet you are in a messy/"worst" configuration. I'm rather sure that a sql dump of this database will NOT show cp1251 strings, neither utf-8… Your friend kludged a kludged setup…
  • I haven't noticed any differences even after removing the debug lines. .. I have just removed them..

    And the second part of your post I did not understand:
    "And I bet you are in a messy/"worst" configuration. I'm rather sure that a sql dump of this database will NOT show cp1251 strings, neither utf-8… Your friend kludged a kludged setup…"

    Could you please clarify

    thanks
  • I have same problem. Who help us? Please!
  • @Cerebellum
    Be more precise in your request. Also reading and understanding this thread and the wiki page it refers to should give you a good starting point.
This discussion has been closed.