如何使用iconv（3）将宽字符串转换为UTF-8？

I'm trying to use iconv(3) to convert a wide-character string to UTF-8 using the code below. When I run the below, the iconv call returns E2BIG, as if there were not enough bytes of space available in the output buffer. This occurs despite the fact that (I think) I have sized the output buffer to admit the worst-case expansion for UTF-8. In fact, given that the input is a simple ASCII 'A' encoded as a wchar_t followed by a zero wchar_t terminator, the output should be exactly two bytes/chars: an 'A' followed by a '\0'.

我正在尝试使用iconv(3)将宽字符串转换为UTF-8,使用下面的代码。当我运行以下时,iconv调用返回E2BIG,就好像输出缓冲区中没有足够的可用空间字节一样。尽管事实上(我认为)我已经确定了输出缓冲区的大小以承认UTF-8的最坏情况扩展。实际上,假设输入是一个简单的ASCII“A”编码为wchar_t后跟一个零wchar_t终结符,输出应该恰好是两个字节/字符:一个'A'后跟一个'\ 0'。

'man utf-8' on my Linux system says that the maximum length of a UTF-8 byte sequence is 6 bytes, so I believe that for an input buffer of 2 wchar_ts (a character followed by the null terminator), making (on my system) 8 bytes total (since sizeof(wchar_t) == 4), a buffer of 12 bytes (2 * UTF8_SEQUENCE_MAXLEN) should be sufficient.

我的Linux系统上的'man utf-8'表示UTF-8字节序列的最大长度是6个字节,所以我相信对于2个wchar_ts的输入缓冲区(一个字符后跟空终止符),make(on我的系统)总共8个字节(因为sizeof(wchar_t)== 4),12个字节的缓冲区(2 * UTF8_SEQUENCE_MAXLEN)就足够了。

By experiment, if I increase UTF8_SEQUENCE_MAXLEN to 16, iconv's return value indicates success (15 still fails). But I cannot see any way that any wchar_t value would occupy so many bytes when encoded in UTF-8.

通过实验,如果我将UTF8_SEQUENCE_MAXLEN增加到16,则iconv的返回值表示成功(15仍然失败)。但是当我用UTF-8编码时,我无法看到任何wchar_t值占用如此多的字节。

Have I gone wrong in my calculations? Are 16-byte UTF-8 sequences possible? What have I done wrong?

我的计算错了吗? 16字节的UTF-8序列是否可能?我做错了什么?

#include <stdio.h>
#include <stdlib.h>
#include <iconv.h>
#include <wchar.h>

#define UTF8_SEQUENCE_MAXLEN 6
/* #define UTF8_SEQUENCE_MAXLEN 16 */

int
main(int argc, char **argv)
{
    wchar_t *wcs = L"A";
    signed char utf8[(1 /* wcslen(wcs) */ + 1 /* L'\0' */) * UTF8_SEQUENCE_MAXLEN];
    char *iconv_in = (char *) wcs;
    char *iconv_out = (char *) &utf8[0];
    size_t iconv_in_bytes = (wcslen(wcs) + 1 /* L'\0' */) * sizeof(wchar_t);
    size_t iconv_out_bytes = sizeof(utf8);
    size_t ret;
    iconv_t cd;

    cd = iconv_open("WCHAR_T", "UTF-8");
    if ((iconv_t) -1 == cd) {
        perror("iconv_open");
        return EXIT_FAILURE;
    }

    ret = iconv(cd, &iconv_in, &iconv_in_bytes, &iconv_out, &iconv_out_bytes);
    if ((size_t) -1 == ret) {
        perror("iconv");
        return EXIT_FAILURE;
    }

    return EXIT_SUCCESS;
}

1 个解决方案

#1

The arguments to iconv_open are the wrong way around. The order of arguments is (to, from), not (from, to), as is clearly stated in the manpage.

iconv_open的参数是错误的。参数的顺序是(to,from),而不是(from,to),如联机帮助页中明确说明的那样。

Consequently, changing

iconv_open("WCHAR_T", "UTF-8");

iconv_open("UTF-8", "WCHAR_T");

causes the (otherwise unchanged) code above to work as expected.

导致上面的代码(否则未更改)按预期工作。

D'oh. Need to read manpages more closely.

D'哦。需要更仔细地阅读联机帮助页。

#1