Import fail with special characters such as å, ä, ö.

Johda

Hi,

We had high hopes migrating from IPB 3 to Vanilla, who doesn't
The import went through with no errors but checking the result; titles, posts, replies... got truncated/cut.

Our forum is in Swedish, we have characters such as å, ä, ö. Some titles, posts and replies with those characters did go through but the majority did not.

Checking the export file, a phrase like this one: "så skulle jag göra det" looks like this in the export: "s skulle jag gˆra det"

If anyone knows a way around this, let us know!

Thanks.

Bleistivt

What's the character set and collation of your old database?

I would try to convert it to utf8 first.

Also see this older discussion:
http://vanillaforums.org/discussion/17990/character-encoding-problem-after-moving-from-phpbb-to-vanilla2

Johda

Thank you for your reply. I tried converting the database to UTF-8 but the result on IPB is the same. Everything got truncated.

Apparently IPB 4 is UTF-8 only, it will convert the database (I'm waiting for their reply on this take). May a future version of Vanilla Porter work for IPB 4.

Thanks.

Johda

One more thing. The character set of the IPB forum is iso-8859-1. Can I change the character set in the vanilla2export.php from utf8 to iso-8859-1. As well as the vanilla installation to iso-8859-1.

What do you guys suggest?

Bleistivt

I suggest you use UTF-8 everywhere.

I tried converting the database to UTF-8 but the result on IPB is the same. Everything got truncated.

Could you explain what happened a little more?
How did you convert it? Did the conversion run successfully? Where did it get truncated?

Johda

There is a setting inside IPB for the character set, it is currently iso-8859-1. If I change that one to UTF-8, characters such as å, ä and ö are changed to �.

I used the IPS UTF8 Database Converter:
https://community.invisionpower.com/files/file/6823-ips-utf8-database-converter/
The result I got on IPB was the exact same as the one I got after I imported the Vanilla Porter export to my vanilla installation.
For example, titles that do not include any special characters are fine. But a title like "This title ä not gut" would look like this after conversion: "This title". It seems to stop when a special character occurs.

Bleistivt

Then there might be a problem with this tool.

You could also try to do the conversion on the database directly (see the stackoverflow link I posted above). Just make sure to backup first.