Video: How Text is Represented in Computers

• There are different standards for assigning a number to each character and different character encodings used to represent them
  • ISO 646
    • ASCII (1 byte, 256 characters)
  • ISO 8859
    • ISO 8859-1 Western Europe (1 byte, 256 characters)
    • ISO 8859-5 Cyrillic alphabet (1 byte, 256 characters)
  • MS-Windows character sets
    • Windows-1250 for Central European languages (1 byte, 256 characters)
    • Windows-1251 for Cyrillic alphabets (1 byte, 256 characters)
    • Windows-1252 for Western languages (1 byte, 256 characters)
  • Unicode
    • UTF-8 (1-4 bytes, 2 million characters)
    • UTF-16 (2 bytes, 65536 characters) - used by C# for example
    • UTF-32 (4 bytes, 4 billion characters)

• Example ASCII character encoding as hex file

00000000  45 45 45 2E 42 47 20 69 73 20 68 65 6C 70 69 6E  |EEE.BG is helpin|
00000010  67 20 79 6F 75 20 75 6E 64 65 72 73 74 61 6E 64  |g you understand|
00000020  20 68 6F 77 20 63 6F 6D 70 75 74 65 72 73 20 72  | how computers r|
00000030  65 70 72 65 73 65 6E 74 20 74 65 78 74           |epresent text|

• Control characters (0-31 and 127) were designed to control hardware devices
  • null character (0), carriage return (13), line feed (10), escape (27), delete (127)

• Unicode can be implemented by different character encodings
  • Unicode Transformation Formats (UTF) UTF-8, UTF-16, and UTF-32, and others

• 17 planes of characters (a plane is a continuous group of 65,536 ($2^{16}$) code points)
  • Unassigned planes 4-13 (40000–DFFFF)

• Example: Cyrillic Capital Letter A has code 0x0410

• The first 128 characters (US-ASCII) need one byte
  • Example: Latin Capital Letter F in UTF-8 is 0x46

• The next 1,920 characters need two bytes to encode
  • Latin-script alphabets and Greek, Cyrillic, Armenian, Hebrew, Arabic, etc.
  • Example: Cyrillic Capital Letter A in UTF-8 is 0xD0 0x90

• Three bytes are needed for characters in the rest of the Basic Multilingual Plane
  • Most Chinese, Japanese and Korean characters

• Four bytes are needed for characters in the other planes of Unicode
  • Various historic scripts, mathematical symbols, and emoji

• Designed to carry binary data across channels that only support text content
  • Send e-mail attachments (SMTP) or save image (binary) into HTML or CSS files

• Each Base64 digit represents exactly 6 bits ($2^6 = 64$) of data

• Example