如何将UTF-8 std :: string转换为UTF-16 std :: wstring？

If I have a UTF-8 std::string how do I convert it to a UTF-16 std::wstring? Actually, I want to compare two Persian words.

如果我有一个UTF-8 std :: string怎么把它转换成UTF-16 std :: wstring?实际上,我想比较两个波斯语单词。

4 个解决方案

#1

Here's some code. Only lightly tested and there's probably a few improvements. Call this function to convert a UTF-8 string to a UTF-16 wstring. If it thinks the input string is not UTF-8 then it will throw an exception, otherwise it returns the equivalent UTF-16 wstring.

这是一些代码。只有轻微的测试,可能会有一些改进。调用此函数将UTF-8字符串转换为UTF-16 wstring。如果它认为输入字符串不是UTF-8那么它将抛出异常,否则返回等效的UTF-16 wstring。

std::wstring utf8_to_utf16(const std::string& utf8)
{
    std::vector<unsigned long> unicode;
    size_t i = 0;
    while (i < utf8.size())
    {
        unsigned long uni;
        size_t todo;
        bool error = false;
        unsigned char ch = utf8[i++];
        if (ch <= 0x7F)
        {
            uni = ch;
            todo = 0;
        }
        else if (ch <= 0xBF)
        {
            throw std::logic_error("not a UTF-8 string");
        }
        else if (ch <= 0xDF)
        {
            uni = ch&0x1F;
            todo = 1;
        }
        else if (ch <= 0xEF)
        {
            uni = ch&0x0F;
            todo = 2;
        }
        else if (ch <= 0xF7)
        {
            uni = ch&0x07;
            todo = 3;
        }
        else
        {
            throw std::logic_error("not a UTF-8 string");
        }
        for (size_t j = 0; j < todo; ++j)
        {
            if (i == utf8.size())
                throw std::logic_error("not a UTF-8 string");
            unsigned char ch = utf8[i++];
            if (ch < 0x80 || ch > 0xBF)
                throw std::logic_error("not a UTF-8 string");
            uni <<= 6;
            uni += ch & 0x3F;
        }
        if (uni >= 0xD800 && uni <= 0xDFFF)
            throw std::logic_error("not a UTF-8 string");
        if (uni > 0x10FFFF)
            throw std::logic_error("not a UTF-8 string");
        unicode.push_back(uni);
    }
    std::wstring utf16;
    for (size_t i = 0; i < unicode.size(); ++i)
    {
        unsigned long uni = unicode[i];
        if (uni <= 0xFFFF)
        {
            utf16 += (wchar_t)uni;
        }
        else
        {
            uni -= 0x10000;
            utf16 += (wchar_t)((uni >> 10) + 0xD800);
            utf16 += (wchar_t)((uni & 0x3FF) + 0xDC00);
        }
    }
    return utf16;
}

#2

This is how you do it with C++11:

这就是你用C ++ 11做的方法:

std::string str = "your string in utf8";
std::wstring_convert<std::codecvt_utf8_utf16<char16_t>> converter;
std::wstring wstr = converter.from_bytes(str);

And these are the headers you need:

这些是您需要的标题:

#include <iostream>
#include <string>
#include <locale>
#include <codecvt>

A more complete example available here: http://en.cppreference.com/w/cpp/locale/wstring_convert/from_bytes

这里有一个更完整的例子:http://en.cppreference.com/w/cpp/locale/wstring_convert/from_bytes

#3

There are some relevant Q&A here and here which is worth a read.

这里和这里有一些相关的问答,值得一读。

Basically you need to convert the string to a common format -- my preference is always to convert to UTF-8, but your mileage may wary.

基本上你需要将字符串转换为通用格式 - 我的偏好总是转换为UTF-8,但你的里程可能会谨慎。

There have been lots of software written for doing the conversion -- the conversion is straigth forwards and can be written in a few hours -- however why not pick up something already done such as the UTF-8 CPP

已经有很多软件用于转换 - 转换是向前转换的,可以在几个小时内编写 - 但是为什么不拿起已经完成的东西,如UTF-8 CPP

#4

This page also seems useful: http://www.codeproject.com/KB/string/UtfConverter.aspx

这个页面似乎也很有用:http://www.codeproject.com/KB/string/UtfConverter.aspx

In the comment section of that page, there are also some interesting suggestions for this task like:

在该页面的评论部分,还有一些有趣的建议,如:

// Get en ASCII std::string from anywhere
std::string sLogLevelA = "Hello ASCII-world!";

std::wstringstream ws;
ws << sLogLevelA.c_str();
std::wstring sLogLevel = ws.str();

Or

// To std::string:
str.assign(ws.begin(), ws.end());
// To std::wstring
ws.assign(str.begin(), str.end());

Though I'm not sure the validity of these approaches...

虽然我不确定这些方法的有效性......