Telugu ‘క’ takes 3 bytes in UTF-8 but only 2 bytes in UTF-16. English ‘A’ takes 1 byte in UTF-8 but 2 bytes in UTF-16. Non-Unicode systems like Anu Script use only 1 byte per character but break on the web. Here is everything you need to know to choose the right encoding.
🔍 Quick Answer (30 seconds): UTF-8 uses 1-4 bytes per character and powers the web. UTF-16 uses 2-4 bytes and powers Windows and Java. UTF-32 uses fixed 4 bytes but is rare. Non-Unicode (legacy systems like Anu Script and Kruti Dev) uses 1 byte per character but only supports one language. Scroll down for the full comparison table and Indian script examples.
What Is UTF-8, UTF-16, and UTF-32? (Unicode Encoding Formats)
Before you can understand the differences, you need to know what each encoding format actually is. For a complete plain-language introduction to Unicode, read our guide: What is Unicode and Non-Unicode? A Complete Plain Language Guide.
UTF-8 – The Web’s Encoding
UTF-8 stands for Unicode Transformation Format – 8-bit. It is a variable-width encoding, meaning different characters use different numbers of bytes. English letters use 1 byte. Indian script characters use 3 bytes. Emoji use 4 bytes.
Here is how UTF-8 works:
- English letters and basic symbols (U+0000 to U+007F) = 1 byte
- Latin, Greek, Arabic, Hebrew (U+0080 to U+07FF) = 2 bytes
- Indian scripts like Telugu, Hindi, Tamil (U+0800 to U+FFFF) = 3 bytes
- Emoji and rare characters (U+10000 to U+10FFFF) = 4 bytes
UTF-8 is fully compatible with ASCII. The letter ‘A’ in UTF-8 is exactly the same byte (41) as in ASCII from the 1960s. This is why UTF-8 became the standard for the web.
Fact 1 – Source: W3Techs
*”UTF-8 encoding is used by over 98% of all websites on the internet as of 2025.”*
Source: W3Techs UTF-8 Usage Statistics
UTF-8 has no endianness issues. Endianness means the order of bytes. Because UTF-8 uses single bytes as its basic unit, it works exactly the same on every computer, whether Windows, Mac, Linux, or Android.
💡 Tip 1: Always use UTF-8 for websites, APIs, JSON files, and email. It is the safest choice and works everywhere.
UTF-16 – Windows and Java’s Encoding
UTF-16 stands for Unicode Transformation Format – 16-bit. It is also variable-width, but its basic unit is 2 bytes (16 bits) instead of 1 byte.
Most common characters (those in the Basic Multilingual Plane, or BMP) use 2 bytes in UTF-16. This includes all Indian script characters like Telugu ‘క’ and Hindi ‘अ’. Characters above U+FFFF, like emoji, use 4 bytes through something called a surrogate pair.
UTF-16 has two variants:
- UTF-16LE (little-endian) – Used by Windows. Stores the smallest byte first.
- UTF-16BE (big-endian) – Used by some Unix systems. Stores the largest byte first.
To tell these apart, files often start with a Byte Order Mark (BOM). A BOM is a special character (U+FEFF) at the beginning of the file. FF FE means UTF-16LE. FE FF means UTF-16BE.
Fact 2 – Source: Unicode Consortium
“The Basic Multilingual Plane (BMP) contains the most common characters for all modern scripts, including Devanagari (Hindi), Telugu, Tamil, and CJK languages.”
Source: Unicode Consortium BMP Chart
UTF-16 is used internally by Windows operating system, Java programming language, .NET framework, and JavaScript. SQL Server also uses UTF-16 for its nvarchar and nchar data types. For a deeper look at database implications, see our guide: Unicode vs Non-Unicode in SQL Server for Developers.
💡 Tip 2: Use UTF-16 if you are writing a Windows-only desktop application or working with Java strings. For everything else, UTF-8 is usually better.
UTF-32 – Fixed-Width (Rare)
UTF-32 stands for Unicode Transformation Format – 32-bit. Unlike the others, UTF-32 is fixed-width. Every character uses exactly 4 bytes (32 bits).
This simplicity is both its strength and weakness. UTF-32 is very easy to work with because each character is exactly 4 bytes. You can jump to any position instantly. But it wastes enormous amounts of memory. A simple English text file would be 4 times larger than UTF-8.
Fact 3 – Source: Unicode Consortium
*”UTF-32 is a fixed-width encoding that uses exactly 4 bytes (32 bits) per Unicode code point, making it simple but memory-inefficient.”*
Source: Unicode Consortium UTF FAQ
UTF-32 is rarely used in practice. You might find it in some compilers, text editors, or internal processing systems where random access is critical. For almost all real-world applications, UTF-8 or UTF-16 is the better choice.
What Is Non-Unicode? (Legacy Code Page Systems)
Now we come to something most guides completely ignore – Non-Unicode systems. This is your competitive advantage because no other article on this topic covers Non-Unicode properly.
Non-Unicode refers to all the encoding systems that existed before Unicode was created in the 1990s. Instead of one universal standard, each Non-Unicode system uses something called a code page.
A code page is a small lookup table. It maps byte values to characters for exactly one language or region. The problem is that a single-byte code page can only hold 256 characters. That is fine for English (which needs only 128 characters), but Telugu needs over 200 characters. Hindi needs even more.
Here are the most common Non-Unicode systems:
- ASCII – The original standard. 7-bit, 128 characters. English only.
- ANSI / Windows-1252 – Default Windows code page for Western European languages. 256 characters.
- ISCII – Indian Script Code for Information Interchange. A government standard before Unicode.
- Anu Script – Custom encoding for Telugu fonts. Very common in Andhra Pradesh and Telangana.
- Kruti Dev – Custom encoding for Hindi Devanagari fonts. Used in government typing exams.
- Nudi – Custom encoding for Kannada fonts. Used in Karnataka government.
Fact 4 – Source: Microsoft Documentation
*”A single-byte non-Unicode code page can represent only 256 characters, while Unicode supports 1,114,112 possible code points – a capacity difference of more than 4,000 times.”*
Source: Microsoft Code Pages Documentation
The real problem with Non-Unicode is incompatibility. The same byte value means completely different characters in different code pages. Byte value 0xC0 means ‘À’ in Windows-1252 (French), ‘А’ in Windows-1251 (Russian), and ‘ـ’ in Windows-1256 (Arabic). If you open a French document on a Russian computer, you see garbage. This is called mojibake.
💡 Tip 3: Non-Unicode is not one system. It is hundreds of different systems that do not talk to each other. Anu Telugu text opened on a Windows-1252 system shows completely different, meaningless characters. This is exactly why Unicode was created.
In India, thousands of newspapers, printing presses, and government offices still use Non-Unicode fonts like Anu Script and Kruti Dev because their software (PageMaker, CorelDraw, older QuarkXPress) was built before Unicode became standard. If you need to convert between Unicode and these legacy fonts, use our Unicode to Non Unicode converter.
Difference Between UTF-8, UTF-16, UTF-32 and Non-Unicode
Here is a complete comparison table showing all four systems side by side. No other website provides this level of detail. For a simpler side-by-side comparison of Unicode and Non-Unicode only, visit our guide: Unicode vs Non-Unicode Key Differences Explained Simply.
| Feature | UTF-8 | UTF-16 | UTF-32 | Non-Unicode (Legacy) |
| Code unit size | 8 bits (1 byte) | 16 bits (2 bytes) | 32 bits (4 bytes) | 8 bits (1 byte) |
| Bytes per character | 1, 2, 3, or 4 | 2 or 4 (surrogate pairs) | Always 4 | Always 1 (SBCS) |
| Variable or fixed? | Variable | Variable (with surrogates) | Fixed | Fixed |
| ASCII compatible? | Yes | No | No | No |
| Total characters supported | All 1.1M+ code points | All 1.1M+ code points | All 1.1M+ code points | 256 per code page |
| Endianness issues? | No | Yes (BE/LE) | Yes (BE/LE) | No (single byte) |
| BOM needed? | Optional (not recommended) | Required for unmarked | Required | No |
| Web usage | 98%+ of websites | Rare | None | None |
| Used internally by | Linux, Python 3, Go, Rust | Windows, Java, .NET, JavaScript | Rare (compilers) | Legacy DTP (PageMaker, CorelDraw) |
| Storage for English | 1 byte per char | 2 bytes per char | 4 bytes per char | 1 byte per char |
| Storage for Indian scripts | 3 bytes per char | 2 bytes per char | 4 bytes per char | 1 byte per char |
| Example fonts | Noto Sans, Gautami | Arial Unicode MS | Any Unicode font | Anu Script, Kruti Dev, Nudi |
| Best for | Web, files, network | Windows apps, in-memory | Internal processing | Legacy DTP only |
💡 Tip 4: For Indian scripts like Telugu, Hindi, and Tamil, UTF-16 uses 33% less storage space than UTF-8 (2 bytes vs 3 bytes). However, UTF-8 is still better for the web because of ASCII compatibility and no endianness issues. For pure English text, UTF-8 is much smaller (1 byte vs 2 bytes).
Byte-Level Examples: Same Character in All Encodings
This is where you see the real difference. Below are actual bytes for the same character in each encoding system. To understand how computers process these bytes at the lowest level, read our guide: How Unicode Works in Computers (Simple Explanation).
Example 1: English Character ‘A’ (U+0041)
| Encoding | Bytes (Hex) | Why |
| UTF-8 | 41 | ASCII range – 1 byte |
| UTF-16 | 41 00 (LE) or 00 41 (BE) | BMP character – 2 bytes |
| UTF-32 | 41 00 00 00 (LE) | Fixed-width – 4 bytes |
| Non-Unicode (ASCII) | 41 | Same as UTF-8 for this character |
The letter ‘A’ is simple. But watch what happens with Indian scripts.
Example 2: Telugu Character ‘క’ (U+0C15) – Indian Script Example
This is the Telugu consonant ‘ka’. Notice how different the byte counts are.
| Encoding | Bytes (Hex) | Why |
| UTF-8 | E0 B0 95 | U+0800–U+FFFF – 3 bytes |
| UTF-16 | 15 0C (LE) | BMP character – 2 bytes |
| UTF-32 | 15 0C 00 00 (LE) | Fixed-width – 4 bytes |
| Non-Unicode (Anu Script) | F0 A3 (example mapping) | Private Use Area – 1 byte (but non-standard) |
For Telugu text, UTF-8 uses 3 bytes per character. UTF-16 uses only 2 bytes. That is a 33% difference. For a full page of Telugu, UTF-16 files will be noticeably smaller.
Example 3: Hindi Character ‘अ’ (U+0905) – Another Indian Script Example
The Hindi vowel ‘a’ (अ) follows the same pattern.
| Encoding | Bytes (Hex) | Why |
| UTF-8 | E0 A4 85 | U+0800–U+FFFF – 3 bytes |
| UTF-16 | 05 09 (LE) | BMP character – 2 bytes |
| UTF-32 | 05 09 00 00 (LE) | Fixed-width – 4 bytes |
| Non-Unicode (Kruti Dev) | Custom mapping | Depends on code page |
Example 4: Emoji ‘😀’ (U+1F600)
Emoji are supplementary characters (above U+FFFF). They require special handling in UTF-16.
| Encoding | Bytes (Hex) | Why |
| UTF-8 | F0 9F 98 80 | U+10000–U+10FFFF – 4 bytes |
| UTF-16 | D8 3D DE 00 (LE) | Surrogate pair – 4 bytes |
| UTF-32 | 00 F6 01 00 (LE) | Fixed-width – 4 bytes |
| Non-Unicode | ❌ Not supported | Emoji didn’t exist in legacy systems |
Fact 5 – Source: Unicode Consortium
*”Supplementary characters (above U+FFFF) require a surrogate pair in UTF-16. The high surrogate range is U+D800–U+DBFF, and the low surrogate range is U+DC00–U+DFFF.”*
Source: Unicode Consortium Surrogate Pairs FAQ
💡 Tip 5: If you see D8 3D DE 00 in a file, that is not two separate characters. It is one emoji character stored as a surrogate pair in UTF-16. Never split these pairs apart or your data will be corrupted.
Which Encoding Should You Use? (Decision Framework for Indian Developers)
Now that you understand the differences, here is how to choose the right encoding for your project. For enterprise systems like SAP, see our guide: Unicode vs Non-Unicode in SAP Systems.
When to Use UTF-8
| Scenario | Why |
| Websites and web apps | HTML, CSS, JavaScript – all require UTF-8 |
| APIs and data exchange | JSON, XML, HTTP headers default to UTF-8 |
| Files stored on disk | Compact for English and mixed content |
| Cross-platform data | No endianness issues – works everywhere |
| Linux or Mac systems | Native encoding for Unix-like systems |
| SMTPUTF8 standard |
When to Use UTF-16
| Scenario | Why |
| Windows applications | Native Windows encoding (UTF-16LE) |
| Java or .NET internal strings | Runtime uses UTF-16 |
| SQL Server nvarchar columns | Database Unicode type for multilingual data |
| In-memory text processing | Faster indexing for BMP characters |
| Pure Indian language text | 2 bytes per character vs 3 in UTF-8 (33% smaller) |
When to Use UTF-32 (Rare)
| Scenario | Why |
| Internal processing | Fixed-width simplifies random access |
| Compilers and text editors | Fast character indexing |
| Generally not recommended | Memory waste – 4 bytes per character |
When to Use Non-Unicode (Only for Legacy)
| Scenario | Why |
| Editing old PageMaker files | PageMaker doesn’t support Unicode |
| Government forms requiring Anu 7.0 | Some systems still require legacy fonts |
| Legacy DTP workflows | Printing presses using CorelDraw 7 or older |
| ⚠️ Avoid for anything new | No web support, no cross-platform compatibility |
💡 Tip 6: For any new project – website, app, database, or document – choose UTF-8. It is the modern standard. Only use UTF-16 if you are writing a Windows-only desktop app. Only use Non-Unicode if you are forced to by legacy software. And if you need to convert between Unicode and Non-Unicode (like Anu Script), use our Unicode to Non Unicode converter.
Common Problems and How to Fix Them
Even when you know the theory, problems happen. Here are the most common encoding issues and how to solve them. If you are facing issues with Windows legacy software, see our detailed guide: Language for Non-Unicode Programs in Windows.
Problem 1: Mojibake (Garbled Text)
What it looks like: தமிழ௠instead of Tamil, or F087 F080 F05C instead of Telugu.
Why it happens: Your computer is reading bytes with the wrong encoding. For example, UTF-8 bytes being interpreted as Windows-1252.
How to fix: Ensure the encoding declared in your HTML (<meta charset=”utf-8″>) matches the actual file encoding. Convert files using a tool like our converter.
Problem 2: UTF-8 BOM Breaking Software
What it looks like:  at the beginning of files, or scripts failing to parse.
Why it happens: Microsoft Notepad adds a UTF-8 BOM (EF BB BF) by default. Some software (Unix scripts, PHP, older parsers) doesn’t expect it.
How to fix: Save files as “UTF-8 without BOM” in Notepad++ or VS Code. Or remove the BOM using a hex editor.
Fact 6 – Source: Unicode Consortium
*”Microsoft software (like Notepad) adds a UTF-8 BOM by default, which can break software that expects no BOM. The BOM for UTF-8 is EF BB BF in hexadecimal.”*
Source: Unicode Consortium BOM FAQ
Problem 3: Boxes (□□□) Instead of Indian Text
What it looks like: Empty boxes where Telugu or Hindi characters should be.
Why it happens: The font does not have glyphs for those Unicode code points.
How to fix: Install a Unicode font that supports Indian scripts. Good options include Noto Sans Devanagari, Noto Sans Telugu, Gautami, Mangal, and Nirmala UI.
Problem 4: SSIS “Cannot Convert Between Unicode and Non-Unicode”
What it looks like: SQL Server Integration Services package fails with this exact error.
Why it happens: Source outputs DT_WSTR (Unicode), destination expects DT_STR (non-Unicode). SSIS does not convert automatically.
How to fix: Add a Data Conversion Transformation between source and destination. For full steps, see our guide: Unicode vs Non-Unicode in SQL Server for Developers.
Need to Convert Between Encodings? Use Our Free Tool
If you have Telugu text in Unicode that needs to work in PageMaker or CorelDraw, you need to convert it to Anu Script (non-Unicode). Our free tool does this instantly.
How to use the Unicode to Non Unicode converter:
- Go to our Unicode to Non Unicode converter
- Paste your Unicode text into the input field
- Select the target non-Unicode font (Anu Script for Telugu, Nudi for Kannada, Kruti Dev for Hindi)
- Click “Convert”
- Copy the output and paste into your legacy DTP software
For Telugu-specific conversion to Anu fonts, use our dedicated Unicode to Anu Converter.
⚠️ Important warning: Unicode to non-Unicode conversion can cause character loss if the target code page doesn’t support every character in your source text. Diacritics, special punctuation, and uncommon conjuncts are most vulnerable. Always verify your output before production use.
Best Practices and Next Steps
Here is what you should do going forward.
Five Action Steps for Indian Developers
- Use UTF-8 for all new projects – Websites, APIs, databases, documents. It is the modern standard and works everywhere.
- Convert legacy non-Unicode documents to Unicode once – If you have old Anu Script or Kruti Dev files, convert them using our Unicode to Non Unicode converter. Store Unicode versions as your master copies. Keep originals only if needed for legacy editing.
- Install proper Unicode fonts for Indian scripts – Noto Sans (Google’s free font family) supports all Indian scripts. Also available: Gautami for Telugu, Mangal for Hindi, Nirmala UI as Windows default.
- Set Windows system locale correctly for legacy software – If you still use old Telugu or Hindi software that shows boxes or question marks, follow our guide: Language for Non-Unicode Programs in Windows.
- Understand the storage trade-off – For pure Indian script text, UTF-16 uses 2 bytes per character vs 3 in UTF-8. But for mixed English-Indian text, UTF-8 often wins because English stays at 1 byte.
Learn More from These Guides
- How Unicode Works in Computers (Simple Explanation)
- What Are Encoding Systems in Unicode? (UTF-8 vs UTF-16 Explained)
- What is Unicode and Non-Unicode? A Complete Plain Language Guide
- Unicode vs Non-Unicode Key Differences Explained Simply
- Unicode vs Non-Unicode in SQL Server for Developers
- Unicode vs Non-Unicode in SAP Systems
- Language for Non-Unicode Programs in Windows
Frequently Asked Questions (FAQ)
What is the difference between UTF-8 and UTF-16?
UTF-8 uses 1 to 4 bytes per character and is ASCII-compatible, making it the standard for the web. UTF-16 uses 2 or 4 bytes per character (via surrogate pairs) and is used internally by Windows, Java, and .NET. Both can encode all Unicode characters.
What is Non-Unicode?
Non-Unicode refers to legacy encoding systems that predate Unicode. These systems use code pages – small lookup tables that map bytes to characters for one specific language. A single-byte code page can only hold 256 characters. Examples include ASCII, ANSI/Windows-1252, ISCII, Anu Script (Telugu), and Kruti Dev (Hindi). For a deeper explanation, read What is Unicode and Non-Unicode? A Complete Plain Language Guide.
Which encoding should I use for a website?
Always use UTF-8. Over 98% of websites use UTF-8. It is ASCII-compatible, has no endianness issues, and works on all browsers and devices.
Does UTF-8 support Telugu and Hindi?
Yes. UTF-8 supports all Unicode characters, including all Indian scripts (Devanagari, Telugu, Tamil, Kannada, Malayalam, etc.). Telugu and Hindi characters require 3 bytes in UTF-8.
Does UTF-16 support Telugu and Hindi?
Yes. UTF-16 supports all Unicode characters. Telugu and Hindi characters in the BMP (U+0900–U+0CFF) require only 2 bytes in UTF-16 – more efficient than UTF-8 for pure Indian language text.
What is a surrogate pair in UTF-16?
A surrogate pair is two 16-bit code units used in UTF-16 to represent characters above U+FFFF (supplementary planes). The high surrogate is in the range U+D800–U+DBFF, and the low surrogate is in U+DC00–U+DFFF. Emoji characters like 😀 (U+1F600) require surrogate pairs.
What is a Byte Order Mark (BOM)?
A BOM is a special character (U+FEFF) placed at the beginning of a file to indicate which Unicode encoding and byte order was used. UTF-8 BOM is EF BB BF. UTF-16LE BOM is FF FE. UTF-16BE BOM is FE FF.
Can I convert Unicode to Non-Unicode without losing data?
Only if every character in your Unicode text has a matching equivalent in the target non-Unicode code page. Characters outside the code page’s range (256 max) will be lost or replaced with question marks. Always verify output before production use.
What is the difference between Unicode and UTF-8?
Unicode is the character set standard that assigns unique code points to every character. UTF-8 is one encoding format that stores those code points as bytes. Think of Unicode as the dictionary and UTF-8 as the printing format.
Why does my Telugu text show as boxes (□□□) in Windows?
Your font does not have glyphs for the Telugu Unicode code points. Install a Unicode font like Noto Sans Telugu, Gautami, or Nirmala UI. Also check your Windows system locale setting – see our Language for Non-Unicode Programs in Windows guide.
Is UTF-8 or UTF-16 better for Indian languages?
For pure Indian language text (no English), UTF-16 is more storage-efficient (2 bytes per character vs 3 in UTF-8). However, for web content (which includes HTML tags, CSS, JavaScript, and URLs), UTF-8 is still better because the ASCII parts stay at 1 byte. Choose based on your use case.
What is mojibake?
Mojibake is a Japanese word (文字化け) that means “character transformation.” It describes the garbage text you see when a computer reads a file using the wrong encoding. For example, seeing F087 F080 F05C instead of అమ్మ.