In-depth Unicode implementation for Windows and Linux

In-depth research report on cross-platform character encoding architecture and engineering practices for Windows/Linux 

Source : In-depth Research Report on Cross-Platform Character Encoding Architecture and Engineering Practice for Windows/Linux - CSDN Blog (blog-csdn-net.translate.goog)

Summary

The heterogeneity of character encoding has always been one of the deepest and most challenging technical issues in cross-platform software engineering. With the surge in global software delivery demands, developers have had to navigate between two vastly different architectures: Microsoft Windows and Linux (and Unix-like systems). This difference is not simply a disagreement over technology choices, but rather rooted in the historical mapping of the two operating system kernels' early evolution of the Unicode standard: Windows chose UCS-2 (later evolving into UTF-16) as its native kernel encoding in the early stages before Unicode matured, while Linux later embraced the more flexible and backward-compatible UTF-8 through glibc's locale mechanism.

This report aims to provide a comprehensive, expert-level programming guide, deeply analyzing the underlying differences in character encoding architecture between Windows and Linux. It covers kernel interfaces, system calls, compiler behavior (MSVC vs GCC/Clang), runtime evolution of mainstream programming languages ​​(C++, Java, Python), and coding pitfalls in modern development toolchains (Git, Docker, PowerShell). The report will pay particular attention to the architectural changes in UTF-8 support in Windows 10/11 in recent years (such as the Manifest mechanism) and provide targeted engineering practice strategies to assist architects and senior engineers in building robust "UTF-8 Everywhere" cross-platform systems.

—

1. Historical Origins and Architectural Divergence: The Tower of Babel of Computing Environments

To understand the current vast gap between Windows and Linux in character processing, we must go back to the decision-making moments of the early 1990s. This was not only a competition of technical standards, but also a game of predictions regarding the future scalability of character sets.

1.1 Windows' Unicode Gamble: UCS-2 and the Legacy of "Wide Characters"

In the early days of Windows NT kernel design (approximately 1990-1993), the Unicode standard (version 1.0) had just been released. The Unicode Consortium promised that all characters in the world could be accommodated within a 16-bit space (65,536 code points). For Microsoft's architects, this was an extremely tempting technical promise: a fixed-width character set meant that string processing algorithms could be extremely simplified, eliminating the complexity of variable-length encoding and providing a permanent solution to multilingual support issues. Therefore, Windows NT decided to make wchar_t (wide characters) a first-class citizen of the kernel, using UCS-2 encoding.

However, history did not unfold as expected. With the expansion of Unicode, the number of characters quickly exceeded the limitations of the 16-bit plane (BMP, Basic Multilingual Plane). To support supplementary plane characters (such as Emojis, rare Chinese characters, and historical characters), Unicode introduced the surrogate pair mechanism, evolving into UTF-16. Windows had to follow this change, migrating from UCS-2 to UTF-16LE (Little Endian). This shift led to an awkward architectural legacy issue: wchar_t remained 16 bits in size, but the actual character encoding became variable-length (one or two wchar_ts represent one character), completely breaking the initial assumption of "fixed-length encoding" while retaining the complexity of the fixed-length encoding API.

To maintain backward compatibility with older versions of Windows (such as Windows 95/98, based on ANSI code pages), Windows introduced a dual API architecture:

• W-suffix APIs (such as CreateFileW): directly accept UTF-16 strings, interact directly with the kernel, and have the highest performance.
• A suffix APIs (such as CreateFileA): Accept single-byte or multi-byte strings (i.e., so-called "ANSI" strings) based on the current system code page. The system allocates memory at runtime, converts the string to UTF-16, and then calls the W version API.

This design led to the long-standing "ANSI vs Unicode" binary opposition in Windows development. Developers had to be constantly wary of the performance overhead and potential data loss risks caused by character set conversion (when the character was not in the current ANSI code page).

1.2 Linux's Byte Stream Philosophy and the Triumph of UTF-8

Unlike Windows' tight kernel integration, Linux and its predecessor Unix followed the philosophy of "everything is a file" and "byte streams." At the Linux kernel level, filenames, paths, and system call arguments are simply sequences of non-empty (NUL, \0) bytes; the kernel doesn't care what characters these bytes represent (except for the path separator /). This design gives user space a great deal of flexibility.

When UTF-8 was invented by Ken Thompson and Rob Pike in 1993 and standardized in the following years, it quickly became the preferred choice in the Linux world. The advantages of UTF-8 perfectly align with the Unix philosophy:

• ASCII compatibility: Any standard ASCII file is also a valid UTF-8 file, which means that countless existing system tools (such as cat, grep, awk) can process UTF-8 text without modification.
• No byte order issues: As a byte-based encoding, UTF-8 does not have the big-endian (BE) or little-endian (LE) debate, avoiding the byte swapping overhead when transmitting data across architectures.
• Robustness: UTF-8's self-synchronization feature makes it easy to find the starting position of the next character even if the data stream is interrupted during transmission.

Unlike Windows, which enforces a specific encoding in the kernel, Linux achieves agreements in user space through glibc and locale environment variables. This loosely coupled architecture allows Linux to smoothly transition from single-byte encodings like ISO-8859-1 to UTF-8 without breaking the kernel ABI.

—

2. In-depth analysis of kernel interfaces and system-level programming

For system programmers (C/C++, Rust, Go), understanding the string processing mechanisms at the kernel boundary is a prerequisite for writing bug-free cross-platform code.

2.1 Windows Kernel Object and String Conversion Mechanism

On the Windows platform, almost all kernel objects (files, mutexes, events, registry keys) are named using UTF-16 at the underlying level. When a traditional C++ program calls `std::fstream::open("config.ini")`, Microsoft's C Runtime Library (CRT) calls `CreateFileA`. If the system locale is English (CP1252), the filename will be interpreted as Windows-1252; if it is Chinese (CP936), it will be interpreted as GBK.

The fatal flaw of this mechanism lies in its globality and lossy nature . The system locale is a global configuration that requires a reboot to take effect. If a program running on Chinese Windows attempts to read a file with a Greek name (such as αβγ.txt) that is not in the GBK character set, the CreateFileA conversion process will fail or use a substitute character (such as ?), resulting in the file being unable to be opened.

2.1.1 Cross-platform pitfalls of wchar_t

The wchar_t type was originally designed to provide a "wide character" type large enough to hold all characters. However, the C++ standard did not specify its size, leading to significant platform fragmentation.

• Windows (MSVC): wchar_t is 16 bits (2 bytes), corresponding to UTF-16 code units. This means that when processing supplementary plane characters (such as Emoji 🌍), programmers must handle surrogate logic, and the string length wcslen returns the number of code units rather than the number of characters.
• Linux (GCC/Clang): wchar_t is typically 32 bits (4 bytes), corresponding to UTF-32. This means that each wchar_t directly corresponds to one Unicode code point.

This size difference (2 bytes vs. 4 bytes) makes std::wstring completely unusable in cross-platform serialization or network transmission. Developers attempting to write std::wstring from Windows memory directly to a file and read it on Linux will face multiple problems, not only in terms of encoding format (UTF-16 vs. UTF-32) but also in byte order and data width.

2.2 Linux System Call Arbitration with Glibc's Locale

On Linux, when executing `open("filename.txt", O_RDONLY)`, only the memory address of the string is passed to the kernel. The kernel stores these bytes on the disk through the file system driver (such as ext4, xfs). If the file system itself does not enforce encoding (most modern Linux file systems do not), then theoretically, UTF-8 encoded filenames and GBK encoded filenames can exist in the same directory.

This confusion is managed by the user-space glibc library through the Locale mechanism. The Locale defines the language environment in which a program runs, controlled by a set of environment variables. Its priority logic is as follows:

1. LC_ALL : Highest priority. Once set, it forcibly overrides all other LC_* variables. It is often used in scripts to force deterministic behavior (e.g., export LC_ALL=C).
2. LC_CTYPE : Determines character classification (what constitutes letters and numbers) and encoding. This is one of the most critical variables in programming. If LC_CTYPE is set to C or POSIX, the system will fall back to 7-bit ASCII mode, and any multibyte characters (UTF-8) may be treated as gibberish or cause the program to crash.
3. LANG : Default value. Takes effect when the above variable is not set.

Potential Engineering Issues: In Docker containers or lightweight Linux distributions, the lack of a generated locale package often causes programs (especially Python or Perl scripts) to throw a LocaleError or fall back to ASCII by default, leading to crashes when processing UTF-8 filenames. It is essential to ensure that the locale (e.g., en_US.UTF-8) is explicitly generated and set in the Dockerfile.

—

3. C/C++ Development Ecosystem: The Growing Pains of Moving from Chaos to Standardization

C/C++ is at the forefront of the operating system API, and therefore is most directly affected by coding differences.

3.1 Compiler Behavior Differences: MSVC's /utf-8 Revolution

For a long time, the MSVC compiler has assumed by default that source code files are encoded according to the ANSI code page of the current system. If a developer writes code `const char* s = "你好";` under Chinese Windows and saves it as UTF-8 (without BOM), MSVC may incorrectly parse the source file according to GBK encoding, resulting in garbled characters in the compiled string literals at runtime.

To address this issue, modern MSVC (Visual Studio 2015 Update 2 and later) introduced the /utf-8 compilation option. This option serves a dual purpose:

1. /source-charset:utf-8 : tells the compiler that the source file is UTF-8 encoded.
2. /execution-charset:utf-8 : Tells the compiler to encode string literals (char*) as UTF-8 byte sequences when generating the executable file.

CMake Configuration Recommendation:
For cross-platform projects, it is strongly recommended to check MSVC in CMakeLists.txt and enable this option to ensure that std::string literal behavior on Windows is consistent with Linux (default UTF-8):

if (MSVC)
    add_compile_options(/utf-8)
endif()

This setting is the cornerstone of implementing the "UTF-8 Everywhere" strategy.

3.2 C++17 std::filesystem and path handling

Prior to C++17, handling file paths across platforms was a nightmare, with developers typically relying on Boost.Filesystem. C++17 standardized it to std::filesystem, attempting to shield developers from these underlying differences.

• std::filesystem::path : This is a smart container. On Windows, it typically maintains a std::wstring (UTF-16); on Linux, it maintains a std::string (UTF-8).
• The rise and fall of u8path : C++17 introduced the std::filesystem::u8path(u8_string) factory function, specifically designed to convert UTF-8 encoded strings to the system's native path format (automatically converted to wide characters on Windows). This was once the recommended standard practice.
• The dramatic changes in C++20 : C++20 introduced the previously missing `char8_t` type and marked `u8path` as deprecated. Now, the constructor of `std::filesystem::path` can directly accept `u8string` (i.e., `std::basic_string<char8_t>`). However, this introduced new complexities: `std::string` (usually UTF-8) and `std::u8string` are different types and cannot be implicitly converted, forcing developers to perform `reinterpret_cast` or data copying, sparking widespread controversy in the community.

Best practice: Although u8path is deprecated, it remains the safest and most portable way to handle UTF-8 paths during transition periods (or for C++17 compatibility). For C++20, it is recommended to build a unified string view adapter layer.

3.3 String type selection strategy

Based on research from various sources, the best strategy for cross-platform C++ development is "UTF-8 Everywhere" :

1. Internal storage: All internal logic of the application uses std::string to store UTF-8 data.
2. Avoid wchar_t: Never use wchar_t or std::wstring unless you are calling the Windows API.
3. Boundary conversion: Use lightweight conversion functions (such as MultiByteToWideChar or std::wstring_convert in modern C++, although the latter is also deprecated and alternatives such as ICU or simdutf need to be found) to convert UTF-8 to UTF-16 only at the moment the Windows API is called.
4. Linux passthrough: On Linux, std::string is passed directly to the system API with zero overhead.

—

4. Evolution of Managed Languages ​​and Runtime Environments

High-level languages ​​such as Java and Python attempt to shield OS differences, but this abstraction often leaks when handling file I/O.

4.1 Java: Full Standardization of JEP 400 and UTF-8

Prior to JDK 18, the behavior of Java's `Charset.defaultCharset()` depended on the operating system. On Windows, it was typically windows-1252 or GBK; on Linux, it was typically UTF-8. This meant that the code `new FileReader("file.txt")` would run correctly on a development machine (Mac/Linux), but might crash on a Windows production environment due to garbled Chinese characters.

The JEP 400 Revolution (JDK 18+):
Java 18 introduced JEP 400, which mandates that the default character set for the standard Java API is UTF-8, regardless of the underlying operating system configuration. This was a huge and disruptive change, but in the long run, it greatly improved portability.

• Compatibility switch: If legacy systems must rely on old behavior, a fallback is required via the startup parameter -Dfile.encoding=COMPAT.
• native.encoding: Introduces a new system property native.encoding, allowing developers to use it in very rare scenarios where they need to know the actual OS encoding.

4.2 Python: From mbcs to UTF-8 Mode

Python 3's strict distinction between Unicode (str vs bytes), while causing initial migration difficulties, laid the foundation for handling Windows encoding issues.

• PEP 529 (Windows File System Encoding): Starting with Python 3.6, Python no longer uses the legacy mbcs (ANSI) encoding by default when handling file paths on Windows. Instead, it internally converts to UTF-16 to call the Windows API, but displays it as UTF-8 externally. This solves the classic problem of "filenames containing characters that cannot be represented in the current code page".
• PEP 540 (UTF-8 Mode): Introduces the -X utf8 command-line option or the PYTHONUTF8=1 environment variable. When enabled, Python ignores POSIX locale settings, forcing stdin/stdout/stderr and file system interfaces to be treated as UTF-8. This is especially important for containerized deployments (where the locale is missing).
• Console output: Python has also undergone a long evolution on the Windows console. Modern versions (3.8+) use io.TextIOWrapper in conjunction with Windows' Unicode API, which basically solves the garbled character problem of print(“你好”) in cmd.exe, provided that the font is supported.

—

5. The Modernization Revolution of Windows: Manifest Mechanism and UTF-8 Code Pages

With the release of Windows 10 Version 1903 (May 2019 Update), Microsoft finally provided a system-level mechanism to break down the ANSI/Unicode binary opposition, which is considered a milestone in the Windows character encoding architecture.

5.1 ActiveCodePage Manifest

Modern Windows applications can now declare the ActiveCodePage property as UTF-8 in their Manifest file.

<assembly manifestVersion="1.0" xmlns="urn:schemas-microsoft-com:asm.v1">
  <application>
    <windowsSettings>
      <activeCodePage xmlns="http://schemas.microsoft.com/SMI/2019/WindowsSettings">UTF-8</activeCodePage>
    </windowsSettings>
  </application>
</assembly>

Technical Impact:
Once declared, all APIs with the "A" suffix called by this process (such as CreateFileA, SetWindowTextA) will no longer interpret char* as CP1252 or GBK, but directly as UTF-8. This means that developers can interact with the kernel on Windows using std::string (UTF-8) just like on Linux, without needing to perform MultiByteToWideChar conversion or rewrite code to use wchar_t. This greatly reduces the cost of porting legacy code to Windows.
Limitations: This only affects the current process. If the program loads a third-party DLL that internally assumes that the "A" API is necessarily single-byte encoded (e.g., using pointer arithmetic s++ to skip characters), it may cause crashes or logical errors.

5.2 System-level Beta Settings: A Double-Edged Sword

Windows provides an administrative setting: "Beta: Provide global language support using Unicode UTF-8". When enabled, the ACP (Active Code Page) for the entire system becomes 65001 (UTF-8).

Risk Warning: While this may seem like the ultimate solution, there's a reason this setting is marked as Beta. It will break a large number of legacy software programs not adapted for UTF-8. For example, some older CAD software, printer drivers, and even some game mods (like Skyrim's Nemesis) will crash or display garbled characters because they cannot handle multi-byte ANSI strings. Therefore, this option should never be enabled by default on general production environments or end-user machines unless it is fully controlled. The recommended approach is always to use application-level Manifests.

—

6. Coding pitfalls and configuration issues in engineering toolchains

Coding problems not only exist in the code itself, but also lurk in the configuration of the development toolchain.

6.1 Git's Cross-Platform Line Ending and Filename Strategy

Git's design is based on Linux, making it most friendly to case-sensitive file systems and LF newlines. On Windows, this can cause various problems.

Newline character (CRLF vs LF):

• Windows uses \r\n, Linux uses \n.
• Optimal configuration: To ensure repository consistency, it is recommended to set core.autocrlf true on Windows (convert to CRLF on checkout and LF on commit), and core.autocrlf input on Mac/Linux (do not convert on checkout, convert to LF on commit).
• Forced approach: A more robust approach is to add a `.gitattributes` file to the repository root directory:
  * text=auto eol=lf
  *.bat text eol=crlf

  This forces most text files to exist in LF format on all platforms (modern editors like VS Code handle LF without any problems on Windows), while batch files are retained as CRLF.

The core configuration `core.precomposeunicode`:
macOS's file system (HFS+/APFS) uses NFD (Non-Non-Decomposition) to store Unicode filenames (e.g., é is stored as e + ´), while Linux and Windows typically use NFC (Non-Non-Non-Combination). This causes Git to perceive filenames as changed. Setting `git config --global core.precomposeunicode true` allows Git to automatically handle this normalization difference on macOS.

6.2 Docker Mounting and File System Mapping

When a volume is mounted from a Windows host to a Linux container via Docker Desktop, an NTFS to ext4 mapping actually occurs.

• Mojibake Issue: If filenames contain non-ASCII characters (such as German diacritics or Chinese characters), you might see garbled characters or question marks (???) when running `ls` inside the container. This is because when Docker transmits filenames via SMB or gRPC protocols, mismatched Locale configurations at both ends can lead to encoding truncation or incorrect parsing.
• Solution: Use Docker's Named Volumes instead of Bind Mounts (host path mounting) to store databases or persistent data whenever possible. If you must mount host code, ensure that the LANG environment variable on both the Windows host and the container is explicitly set to a UTF-8 supported value (e.g., C.UTF-8).

6.3 PowerShell Coding Quirks

PowerShell (especially v5.1) is extremely prone to encoding corruption when processing external command output.

• Pipe corruption: When executing cmd.exe /c output_utf8.exe | Select-String “pattern”, PowerShell may use the default ANSI encoding to decode the EXE output, causing UTF-8 characters to become garbled.
• BOM Issue: PowerShell 5.1 uses UTF-16LE with a BOM by default when redirecting output to a file (> file.txt). This causes Linux tools (such as grep and bash) to be unable to read the file.
• Solution: Explicitly set the output encoding in the script header:

[Console]::OutputEncoding =::UTF8
$PSDefaultParameterValues['Out-File:Encoding'] = 'utf8'

To resolve garbled characters in the console, you can use `chcp 65001`, but be aware that this may cause some older command-line tools to malfunction.

—

7. Data Exchange and Persistence: The Curse of BOM and Solutions

In the persistence of text files, the BOM (Byte Order Mark) is a technical detail that has sparked numerous controversies.

7.1 The Game Between Excel CSV and BOM

In the UTF-8 specification, the BOM (0xEF 0xBB 0xBF) is optional and not recommended because UTF-8 does not have byte order issues. However, Microsoft Excel is an exception.

• Problem: If you double-click to open a UTF-8 CSV file without a BOM, Excel often defaults to using the system ANSI code page (such as CP1252), resulting in garbled Chinese characters or special symbols.
• trade off:
  • With BOM (UTF-8-BOM): Excel can automatically recognize and display it correctly, but in a Linux environment, the script will contain invisible BOM characters when reading the first row, causing the header parsing to fail (for example, ID becomes \ufeffID).
  • Without BOM (UTF-8): Linux/Web friendly, but the user experience in Excel is extremely poor (requires using "Data -> Get Data -> From Text" and manually selecting UTF-8).
• Visual Studio and EditorConfig: VS and VS Code have different default behaviors in this regard. The .editorconfig file can be used to standardize files within a project.

[*.{cs,json,xml}]
charset = utf-8

Note: There used to be a bug in VS that would add a BOM even if utf-8 was set, but modern versions have fixed it and clearly distinguish between utf-8 and utf-8-bom.

7.2 Databases and Web Stack

• MySQL: Never use utf8 (it's an older alias for MySQL that only supports a maximum of 3 bytes and cannot store emojis). You must use utf8mb4 .
• HTTP: charset=utf-8 must be explicitly specified in the Content-Type header. Although modern browsers have detection mechanisms, explicit declaration is the last line of defense against XSS attacks and garbled characters.

—

8. Strategic Recommendations and Best Practices Summary

Based on the above in-depth analysis, the following tiered best practices are proposed for cross-platform development teams:

8.1 Architectural Decision: UTF-8 Everywhere

In any new C++ project, the UTF-8 Everywhere manifesto should be adopted without fail.

• Internal data : All data uses std::string (UTF-8).
• Windows adaptation : Use the /utf-8 compilation option; declare ActiveCodePage as UTF-8 in the Manifest; only perform the conversion when you have to interact with older, unadapted APIs.
• Linux compatibility : No additional steps required, enjoy native performance.

8.2 Engineering Configuration List

1. Editor specifications : The project root directory must contain .editorconfig, and charset = utf-8 (no BOM) and end_of_line = lf must be enforced.
2. Git specification : Configure .gitattributes to unify the line ending strategy.
3. Containerization specifications : The Dockerfile must include the locale. Generation steps:

RUN apt-get update && apt-get install -y locales \
    && locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

4. Scripting guidelines : All PowerShell scripts should explicitly set OutputEncoding to UTF-8; all Bash scripts should avoid using non-ASCII characters as variable names.

8.3 Legacy System Migration Strategy

For the Windows C++ codebase, which carries a heavy historical baggage (heavily using CString and TCHAR):

• Phase 1 : Without modifying the code, introduce the /utf-8 flag in the build stream and observe whether the literal character encoding problem is resolved.
• Phase Two : Gradually replace the file I/O module and introduce std::filesystem to replace the Win32 API.
• Phase 3 : Enable the Manifest UTF-8 option and gradually remove the A to W conversion layer, but extensive regression testing is required, especially for compatibility with third-party DLLs.

By understanding the baggage of history and the capabilities of modern toolchains, developers can build a solid bridge between Windows and Linux, allowing code to continue to communicate freely in a world after the collapse of the "Tower of Babel".

—

9. Appendix: Reference Table of Core Technology Data

Table 1: Comparison of native string processing methods in operating systems

characteristic	Windows (Traditional/Standard)	Linux (standard)	Windows (Modern/Manifest)
Native API encoding	UTF-16LE	UTF-8 (User Space Convention)	UTF-8 (via “A” API)
wchar_t size	16-bit (2 bytes)	32-bit (4 bytes)	16-bit (2 bytes)
std::filesystem backend	std::wstring	std::string	std::string (conversion required)
Filename restrictions	Case insensitive (usually), UTF-16	Case sensitive, byte stream	Case insensitive, UTF-16
Console default encoding	OEM (e.g., CP437, GBK)	UTF-8 (via LANG)	UTF-8 (if chcp 65001)

Table 2: Analysis of Common Garbled Characters (Mojibake)

Form of expression	Cause diagnosis	Technical Explanation
é	UTF-8 was misread as Latin-1/CP1252	The UTF-8 bytes for the character é are C3 A9, which correspond to à and © in Latin-1, respectively.
``	Latin-1/GBK was misread as UTF-8	Many single-byte or double-byte encodings (such as GBK) have their high-order bytes as illegal start bytes in the UTF-8 specification and are displayed as replacement characters.
\u00E9	JSON/Java escape sequence not parsed	The source code or logs output escape sequences directly, rather than the characters themselves.
	UTF-8 BOM misread as Latin-1	The BOM (EF BB BF) in the file header is displayed as these three characters in editors that do not support BOM.

Cited works

1. The UTF-8 encoding was invented in 1992 and standardized by January 1993. | Hacker News, accessed January 1, 2026, https://news.ycombinator.com/item?id=29748679
2. wchar_t Is a Historical Accident - moria.us, accessed January 1, 2026, https://www.moria.us/articles/wchar-is-a-historical-accident/
3. Character encoding - Wikipedia, accessed January 1, 2026, https://en.wikipedia.org/wiki/Character_encoding
4. Working with Strings - Win32 apps | Microsoft Learn, accessed January 1, 2026, https://learn.microsoft.com/en-us/windows/win32/learnwin32/working-with-strings
5. Use UTF-8 code pages in Windows apps - Microsoft Learn, accessed January 1, 2026, https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page
6. Explaining the ANSI Charset - Chilkat Software, 访问时间为 一月 1, 2026， https://www.chilkatsoft.com/ansi_charset_explained.asp
7. Windows code page - Wikipedia, 访问时间为 一月 1, 2026， https://en.wikipedia.org/wiki/Windows_code_page
8. Difference between ANSI and UTF-8 - Vovsoft, 访问时间为 一月 1, 2026， https://vovsoft.com/blog/difference-between-ansi-and-utf-8/
9. Is “UTF-8 with BOM” the old UTF? What is UTF-8 now? - Super User, 访问时间为 一月 1, 2026， https://superuser.com/questions/1553666/is-utf-8-with-bom-the-old-utf-what-is-utf-8-now
10. Understanding locale environment variables - IBM, 访问时间为 一月 1, 2026， https://www.ibm.com/docs/ssw_aix_71/globalization/understand_locale_environ_var.html
11. Glibc and the kernel user-space API - LWN.net, 访问时间为 一月 1, 2026， https://lwn.net/Articles/534682/
12. What are the differences between char, char16_t, char32_t, string and wchar_t? - Reddit, 访问时间为 一月 1, 2026， https://www.reddit.com/r/C_Programming/comments/184tnyw/what_are_the_differences_between_char_char16_t/
13. wchar_t for UTF-16 on Linux? - Stack Overflow, 访问时间为 一月 1, 2026， https://stackoverflow.com/questions/12865564/wchar-t-for-utf-16-on-linux
14. Locale Environment Variables in Linux - Baeldung, 访问时间为 一月 1, 2026， https://www.baeldung.com/linux/locale-environment-variables
15. PEP 538 – Coercing the legacy C locale to a UTF-8 based locale | peps.python.org, 访问时间为 一月 1, 2026， https://peps.python.org/pep-0538/
16. How do I fix my locale issue? - Ask Ubuntu, 访问时间为 一月 1, 2026， https://askubuntu.com/questions/162391/how-do-i-fix-my-locale-issue
17. Setting the LANG Environment Variables and Locales (UNIX) - Informatica Documentation, 访问时间为 一月 1, 2026， https://docs.informatica.com/master-data-management/multidomain-mdm/10-3-hotfix-3/configuration-guide/introduction/configuring-international-data-support/configuring-language-in-oracle-environments/setting-the-lang-environment-variables-and-locales–unix-.html
18. /utf-8 (Set source and execution character sets to UTF-8) | Microsoft Learn, 访问时间为 一月 1, 2026， https://learn.microsoft.com/en-us/cpp/build/reference/utf-8-set-source-and-executable-character-sets-to-utf-8?view=msvc-170
19. /execution-charset (Set execution character set) | Microsoft Learn, 访问时间为 一月 1, 2026， https://learn.microsoft.com/en-us/cpp/build/reference/execution-charset-set-execution-character-set?view=msvc-170
20. Visual Studio /utf-8 source files - Scientific Computing, 访问时间为 一月 1, 2026， https://www.scivision.dev/msvc-utf8-source-files/
21. std::filesystem::u8path - cppreference.com - C++ Reference, 访问时间为 一月 1, 2026， https://en.cppreference.com/w/cpp/filesystem/path/u8path.html
22. Understanding std::u8string: Does It Have to Be UTF-8? - YouTube, 访问时间为 一月 1, 2026， https://www.youtube.com/watch?v=j47LAvnMk4U
23. How does std::u8string vary from the more common std::string? - Stack Overflow, 访问时间为 一月 1, 2026， https://stackoverflow.com/questions/56420790/how-does-stdu8string-vary-from-the-more-common-stdstring
24. Mastering Unicode in C++: A Comprehensive Guide to UTF-8 and UTF-16 in Windows | by Akash Pandit | Medium, 访问时间为 一月 1, 2026， https://medium.com/@panditakash38/mastering-unicode-in-c-a-comprehensive-guide-to-utf-8-and-utf-16-in-windows-355687c35641
25. JEP 400: UTF-8 by Default - OpenJDK, 访问时间为 一月 1, 2026， https://openjdk.org/jeps/400
26. JDK 18 and the UTF-8 as default charset | by Andrea Binello | Medium, 访问时间为 一月 1, 2026， https://medium.com/@andbin/jdk-18-and-the-utf-8-as-default-charset-8451df737f90
27. JDK 18 Release Notes, Important Changes, and Information - Oracle, 访问时间为 一月 1, 2026， https://www.oracle.com/java/technologies/javase/18-relnote-issues.html
28. File encoding and UTF-8 as default charset - IBM, 访问时间为 一月 1, 2026， https://www.ibm.com/docs/en/semeru-runtime-ce-z/21.0.0?topic=guide-file-encoding-utf-8-as-default-charset
29. PEP 529 – Change Windows filesystem encoding to UTF-8 | peps.python.org, 访问时间为 一月 1, 2026， https://peps.python.org/pep-0529/
30. PEP 540 – Add a new UTF-8 Mode - Python Enhancement Proposals, 访问时间为 一月 1, 2026， https://peps.python.org/pep-0540/
31. PEP 540: Add a new UTF-8 mode (v2) - LWN.net, 访问时间为 一月 1, 2026， https://lwn.net/Articles/741365/
32. UTF-8 in manifest in non-Unicode build - Codejock Forums, 访问时间为 一月 1, 2026， https://forum.codejock.com/topic24483&OB=ASC&MobileView=off.html
33. Inventor 2020 crashes when creating a category in Content Center Editor - Autodesk, 访问时间为 一月 1, 2026， https://www.autodesk.com/support/technical/article/caas/sfdcarticles/sfdcarticles/Inventor-2020-crashes-when-creating-a-category-in-Content-Center-Editor.html
34. Help with accents and regional characters : r/nanDECK - Reddit, 访问时间为 一月 1, 2026， https://www.reddit.com/r/nanDECK/comments/1j4w42k/help_with_accents_and_regional_characters/
35. Errors caused by Windows 10 Unicode UTF-8 encoding - nShift Help Center, 访问时间为 一月 1, 2026， https://helpcenter.nshift.com/hc/en-us/articles/360016886479-Errors-caused-by-Windows-10-Unicode-UTF-8-encoding
36. Configuring Git to handle line endings - GitHub Docs, 访问时间为 一月 1, 2026， https://docs.github.com/articles/dealing-with-line-endings
37. Complete Guide to Git for Multi-Platform Development: From Mac to Windows Without Conflicts | by Antonio Demarcus | Medium, 访问时间为 一月 1, 2026， https://medium.com/@antoniodemarcus/complete-guide-to-git-for-multi-platform-development-from-mac-to-windows-without-conflicts-01ca50f9e13d
38. Special characters in filenames of shared volumes are replaced with ?? (e.g. German umlauts) · Issue #36616 - GitHub, 访问时间为 一月 1, 2026， https://github.com/moby/moby/issues/36616
39. Files in host volume that have special characters filenames, have the filename translated to ?'s in Linux container · Issue #986 · docker/for-win - GitHub, 访问时间为 一月 1, 2026， https://github.com/docker/for-win/issues/986
40. UTF8 Script in PowerShell outputs incorrect characters - Stack Overflow, 访问时间为 一月 1, 2026， https://stackoverflow.com/questions/14482253/utf8-script-in-powershell-outputs-incorrect-characters
41. Extremely prone to utf8-related errors when executing commands using Windows PowerShell - Atlassian Community, 访问时间为 一月 1, 2026， https://community.atlassian.com/forums/Rovo-questions/Extremely-prone-to-utf8-related-errors-when-executing-commands/qaq-p/3060395
42. PowerShell pipe default OutputEncoding should match [System.Console]::OutputEncoding to avoid encoding issues with native console tools like more #25698 - GitHub, 访问时间为 一月 1, 2026， https://github.com/PowerShell/PowerShell/issues/25698
43. What’s the difference between UTF-8 and UTF-8 with BOM? - Codemia, 访问时间为 一月 1, 2026， https://codemia.io/knowledge-hub/path/whats_the_difference_between_utf-8_and_utf-8_with_bom
44. Should UTF-8 CSV files contain a BOM (byte order mark)?, 访问时间为 一月 1, 2026， https://softwareengineering.stackexchange.com/questions/372692/should-utf-8-csv-files-contain-a-bom-byte-order-mark
45. Opening CSV with UTF-8 BOM via Excel - java - Stack Overflow, 访问时间为 一月 1, 2026， https://stackoverflow.com/questions/20275470/opening-csv-with-utf-8-bom-via-excel
46. Specifying charset=utf-8 in .editorconfig incorrectly adds a BOM - Developer Community, 访问时间为 一月 1, 2026， https://developercommunity.visualstudio.com/content/problem/36385/specifying-charsetutf-8-in-editorconfig-incorrectl.html
47. Why is utf-8-bom discouraged? · Issue #297 · editorconfig/editorconfig - GitHub, 访问时间为 一月 1, 2026， https://github.com/editorconfig/editorconfig/issues/297
48. How to Handle Unicode and Emoji Encoding in Production Systems - Strapi, 访问时间为 一月 1, 2026， https://strapi.io/blog/unicode-and-emoji-encoding

This content is licensed under CC 4.0 BY-SA

Copyright Notice: This article is the author's original work and is licensed under CC 4.0 BY-SA. Please include the original source link and this statement when reprinting.