Tim Rentsch <tr.17687@z991.linuxsc.com> writes:

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:

[...]

The <uchar.h> header was introduced in C99. In C99, C11, and C17,
that header defines char16_t and char32_t. C23 introduces char8_t.

There doesn't seem to be any way, other than checking the value of
__STDC_VERSION__ to determine whether char8_t is defined or not.
There are not *_MIN or *_MAX macros for these types, either in
<uchar.h> or in <limits.h>. A test program I just wrote would have
been a little simpler if I could have used `#ifdef CHAR8_MAX`.

[...]

Since C23 defines char8_t to be the same type as unsigned char,
it seems better to just define it when it isn't there:

#include <limits.h>

#if CHAR_BIT == 8 && __STDC_VERSION__ < 202311
typedef unsigned char char8_t;
#endif

Yes. And the test for CHAR_BIT may not be necessary, depending on the programmer's intent. char8_t is the same type as unsigned char even if CHAR_BIT > 8. Similarly, char16_t and char32_t are the same type as uint_least16_t and uint_least32_t, respectively.

But before C23, u8"a" is a syntax error.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
void Void(void) { Void(); } /* The recursive call of the void */

--- PyGate Linux v1.5.2
* Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)

On 12/15/2025 7:27 PM, Keith Thompson wrote:
...

But before C23, u8"a" is a syntax error.

u8"a" was introduced in C11.
u8'a' was introduced in C23.





--- PyGate Linux v1.5.2
* Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)

Thiago Adams <thiago.adams@gmail.com> writes:

On 12/15/2025 7:27 PM, Keith Thompson wrote:
...

But before C23, u8"a" is a syntax error.

u8"a" was introduced in C11.
u8'a' was introduced in C23.

Thank you, I stand corrected.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
void Void(void) { Void(); } /* The recursive call of the void */

--- PyGate Linux v1.5.2
* Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)

On 10/20/2025 1:35 PM, Thiago Adams wrote:

speaking on signed x unsigned,

u8"a"ÿ in C11 had the type char [N]. Normally char is signed

in C23 it is unsigned char8_tÿ [N].

when converting code from c11 to c23 we have a error here
const char* s = u8""

I generally "cast char* " to "unsigned char*" when handling something
with utf8. I am not u8"" , I use just " " with utf8 encoded source code
and I just assume const char*ÿ is utf8.

It may not be so simple, as source-code bytes don't necessarily map 1:1
with string literal bytes (and are more likely to be translated than
passed through as-is).

Implicitly, it may depend on the default locale and similar assumed by
the C compiler.

If the source-code is UTF-8, and the default locale is UTF-8, then OK.

More conservative though is to assume that the default locale's
character encoding is potentially something like 8859-1 or 1252, which
will not preserve UTF-8 codepoints if not mapped into an area supported
by the relevant encoding (so, things may get remapped).

So, you need a UTF-8 string literal or similar to specify that the
string does in-fact encode text as UTF-8.



In a compiler, one may need to try to detect and deal with text
encoding, say:
ASCII text:
No BOM, limited range of characters
(0x20..0xx7E, 0x09, 0x0D, 0x0A, etc).
UTF-8:
Also Includes 80..EF
Only allow valid codepoint sequences
May include a BOM
8859-1 or 1252:
Includes 80..FF, excludes text which is also valid as UTF-8.
No BOM.
Other encodings possible,
Like 437 / KOI-8 / JIS / etc,
but far less common than 1252.
No good way to distinguish them reliably.
UTF-16 (*1):
Even number of bytes
Strongly hinted if even or odd bytes are frequently NUL;
Frequent even NUL: UTF-16, likely big-endian;
Frequent odd NUL: UTF-16, likely little-endian;
Excluded if matching the pattern for one of the above;
If text is valid ASCII or UTF-8, assume these instead.
May include a BOM.

*1: More commonly produced by older versions of Visual Studio or
Notepad, if a non-ASCII codepoint was present. Newer versions tend to
default to UTF-8 instead.

Compiler may normalize on UTF-8 or similar internally, but this again
doesn't mean it can be assumed for string literals (which are more
likely to be mashed into 1252 or something, such as with a compiler like MSVC).


Though, that said, does seem that GCC defaults to assuming UTF-8 if
nothing else is specified. So, UTF-8 => UTF-8 with default string
literals may be workable if one also assumes that the code is always
compiled with GCC or similar.

Though, curiously, it seems newer MSVC will still use UTF-8 with a
default string literal if the character is given as "\uXXXX", but will
use a single-byte encoding in other cases.

Checking, newer versions of MSVC are also aware of u8 literals.

...



--- PyGate Linux v1.5.2
* Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:

Tim Rentsch <tr.17687@z991.linuxsc.com> writes:

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:

[...]

The <uchar.h> header was introduced in C99. In C99, C11, and C17,
that header defines char16_t and char32_t. C23 introduces char8_t.

There doesn't seem to be any way, other than checking the value of
__STDC_VERSION__ to determine whether char8_t is defined or not.
There are not *_MIN or *_MAX macros for these types, either in
<uchar.h> or in <limits.h>. A test program I just wrote would have
been a little simpler if I could have used `#ifdef CHAR8_MAX`.

[...]

Since C23 defines char8_t to be the same type as unsigned char,
it seems better to just define it when it isn't there:

#include <limits.h>

#if CHAR_BIT == 8 && __STDC_VERSION__ < 202311
typedef unsigned char char8_t;
#endif

Yes. And the test for CHAR_BIT may not be necessary, depending on
the programmer's intent. char8_t is the same type as unsigned char
even if CHAR_BIT > 8.

That's humorous. It's like a name designed to be confusing or
misleading. But thank you for the information, I wouldn't have
guessed it.

Similarly, char16_t and char32_t are the same type as
uint_least16_t and uint_least32_t, respectively.

Kind of weird, but at least it's consistent, and it explains why
char8_t is the same as unsigned char. Then again, why not
uint_least8_t? Has C23 changed to the point where unsigned char
and uint_least8_t have to be the same type? My recollection is
that in earlier editions of the C standard it is possible, even
if unlikely, for these types to be distinct.

--- PyGate Linux v1.5.2
* Origin: Dragon's Lair, PyGate NNTP<>Fido Gate (3:633/10)