MirOS Manual: optu8to16(3), optu8to16vis(3)

OPTU8TO16(3)               BSD Programmer's Manual                OPTU8TO16(3)


     optu8to16, optu8to16vis - converts multibyte characters to wide charac-
     ters preserving octets


     #include <wchar.h>

     optu8to16(wchar_t *pwc, const char *s, size_t n, mbstate_t *ps);

     optu8to16vis(wchar_t *pwc, const char *s, size_t n, mbstate_t *ps);
     /* deprecated */


     The optu8to16() function usually converts the multibyte character pointed
     to by s to a wide character, and stores the wide character in the wchar_t
     object pointed to by pwc if pwc is non-null and s points to a valid char-
     acter in the CESU-8 multibyte encoding, similar to mbrtowc() in a UTF-8
     locale. If s does not point to a valid character, the first octet is
     transliterated to either an ISO_646.irv:1991 (ASCII) mapping into UCS-2
     (U+0000 .. U+007F), or to the OPTU-16 raw octet range (U+EF80 .. U+EFFF).
     The optu8to16vis() function behaves the same, except raw octets are
     mapped into the normal unicode range as if they had been encoded in the
     legacy 8-bit codepage. The conversion happens in accordance with the
     conversion state described in the mbstate_t object pointed to by ps; it
     should be noted that raw octet conversion is stateful. This function may
     examine at most n bytes of the array beginning from s. If n is set to 0,
     the function behaves as if end of input (not a null character) has been
     read and ignores s.

     If s points to a valid character and the character corresponds to a null
     wide character, then the function places the mbstate_t object pointed to
     by ps to an initial conversion state.

     These are the special cases:

     pwc == NULL    The conversion from a multibyte character to a wide char-
                    acter has taken place and the conversion state may be af-
                    fected, but the resultant wide character is discarded.

     s == NULL      optu8to16() sets the conversion state object pointed to by
                    ps to an initial state and always returns 0. In this case,
                    optu8to16() ignores pwc but not n, and is equivalent to
                    the following call:

                          optu8to16(NULL, "", 1, ps);

     n == 0         Read end of input (not a null character, but an epsilon as
                    known from computer science automaton modelling) and ig-
                    nore s. optu8to16() will still emit up to two wide charac-
                    ters and return 0, if the conversion state contains infor-
                    mation about these, and (size_t)-2 otherwise. Application
                    note: If the end of input has been reached, call
                    optu8to16() with n == 0 until it returns (size_t)-2, and
                    process the remaining wide characters emitted. This en-
                    sures no raw octets in the OPTU-8 encoded source are lost.

     ps == NULL     optu8to16() uses its own internal state object to keep the
                    conversion state, instead of ps mentioned in this manual

                    Calling any other functions in libc never change the
                    internal state of optu8to16(), which is initialised at
                    programme startup time.


     0 or positive  Number of bytes read from s. If 0, the state contained
                    enough information to emit a wide character; if positive,
                    the bytes form a valid multibyte character in the OPTU-8

     (size_t)-2     s points to the byte sequence which possibly contains part
                    of a valid multibyte character, but which is incomplete.
                    All n bytes of the input have been processed and stored in

     (size_t)-1     Generic error condition; should not happen in the current
                    implementation. errno is set to indicate the error.


     The optu8to16() function is designed to be as robust as possible and can,
     in contrast to mbrtowc(), not throw EILSEQ. While EINVAL to indicate an
     invalid or uninitialised mbstate_t object is theoretically possible, nei-
     ther this nor other processing errors should ever happen with the current


     iswoctet(3), mbrtowc(3), optu16to8(3)


     At present, MirOS is limited to the Unicode BMP (Basic Multilingual
     Plane), thus OPTU-8 is limited to the common subset of CESU-8 and UTF-8.

     The optu16to8() and optu8to16() functions are standardised by MirOS and
     have been designed to behave as close to their ISO/IEC 9899:1999 ("ISO
     C99") equivalents wcrtombs() and mbrtowcs() as possible, with the follow-
     ing intentional exceptions:

     If n is 0, s is ignored, even if it is NULL, not the other way round. The
     return value 0 does not indicate that a null character was processed, use
     pwc for that. It indicates that no byte of the input has been read.

     The optu8to16vis() function assumes codepage 1252 and maps holes into
     distinguishable codepoints.

     All these extended functions declare macros with the same name that can
     be used to check for their presence.


     The optu8to16 function first appeared in MirOS #11.


     Thorsten Glaser <tg@mirbsd.de> wrote the entire internationalisation im-
     plementation in MirOS. He is also the steward for the OPTU encoding.


     On a system whose wide character type is only 16 bits wide, as opposed to
     31 bits of ISO 10646, the OPTU encoder and decoder are permitted to not
     de- and recompose any surrogates encountered and pass them through as if
     they were regular wide characters with no special function. Since MirOS
     is such a system, the reference implementation does not care about UTF-16
     surrogates posing as OPTU-16 characters at all; a planes-aware Unicode
     application is required to handle surrogates by itself. For compatibility
     purposes, optu8to16 should always be assumed to not treat surrogates spe-
     cially; applications must ensure to not produce invalid surrogates unless
     limited to the BMP.

MirOS                          February 9, 2014                              1

Generated on 2017-04-03 16:26:17 by $MirOS: src/scripts/roff2htm,v 1.88 2017/01/29 00:51:06 tg Exp $

These manual pages and other documentation are copyrighted by their respective writers; their source is available at our CVSweb, AnonCVS, and other mirrors. The rest is Copyright © 2002–2017 The MirOS Project, Germany.
This product includes material provided by mirabilos.

This manual page’s HTML representation is supposed to be valid XHTML/1.1; if not, please send a bug report — diffs preferred.