https://gitlab.synchro.net/main/sbbs/-/merge_requests/569#note_7579

So if the text contains any UTF-8 encoded characters, but the terminal does not support UTF-8, this change will cause SBBS to send the raw UTF-8 bytes to the terminal (e.g. appearing as garbage to any non-UTF-8 terminal). I'm not seeing how this is an improvement.

---
� Synchronet � Vertrauen � Home of Synchronet � [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/merge_requests/569#note_7580

Yeah, actually this probably isn't what I want to do. The problem I'm seeing seems to stem from not being able to get an accurate string length for some UTF-8 strings. I've been using console.strlen() with P_AUTO_UTF8 to get printed string lengths, and DDLightbarMenu uses these string lengths to ensure that text fields of specific widths within a menu item are filled in with the specified attributes. The attributes might have a background color, and what I'm seeing is that sometimes, console.strlen() reports a UTF-8 string length as more than what's actually printed on the screen, and the result is that the menu item text isn't always fully filled in with the set attributes. For example, in the following screenshot, there is some text that has 5 upside-down question marks (I believe where UTF-8 text can't be displayed); the blue background color should stretch the full width of the text, but there's a block missing where it's just black:

I have some JS code that demonstrates the string length issue:

`var str = "リそな銀行";
console.print(str, P_AUTO_UTF8);
console.crlf();
console.print("Str length: " + console.strlen(str, P_AUTO_UTF8) + "\r\n");`

In SyncTerm (not supporting UTF-8), the text string appears as "¿¿¿¿¿", which is 5 characters long, but the string length reported by console.strlen() here is 10. So my script would think more characters were printed than there actually are.

I had made a workaround that involves using console.getxy() to get the current cursor position and calculating the remaining width to fill in, but I don't feel like that's an ideal solution, as it is noticeably slow.

---
￭ Synchronet ￭ Vertrauen ￭ Home of Synchronet ￭ [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/merge_requests/569#note_7582

Just for additional info, there's debug-level log output that might be helpful in cases like this, e.g.

`Unsupported UTF-8 sequence: EF BF BF (U+FFFF) width=1`

---
� Synchronet � Vertrauen � Home of Synchronet � [vert/cvs/bbs].synchro.net

https://gitlab.synchro.net/main/sbbs/-/merge_requests/569#note_7584

I've updated with the change you made for con_out.cpp. That helps, as things look good with that. After that though, I got another spam email and I see another case where console.strlen() is reporting more characters than what are displayed, with the string "スマートＥ".

```
var str = "スマートＥ";
//console.print(str, P_AUTO_UTF8);
console.print(":" + str + ":", P_AUTO_UTF8);
console.crlf();
console.print("Str length: " + console.strlen(str, P_AUTO_UTF8) + "\r\n"); console.pause();
```

In a non-UTF8 terminal, the string gets printed as "¿¿¿¿¿¿¿¿E", which is 9 characters, but console.strlen(str, P_AUTO_UTF8) returns 10. When displayed, there is a gap of 1 space in the background color:


---
￭ Synchronet ￭ Vertrauen ￭ Home of Synchronet ￭ [vert/cvs/bbs].synchro.net