Following up on the Python demo we did during the character representation lecture: when we finally got to the correct encoding scheme for the characters to make sense, why did our three characters turn into two characters?

For starters, this demo was drawn from Sebastian Mathot’s blog post, “A simple explanation of character encoding in Python.”

Sebastian addresses the 3 to 2 issue: “According to utf-8, 195 and 167 together make up a ‘ç’, and 97 makes up ‘a’. ”

But wait, isn’t the whole point of unicode that every symbol has one single code?

Yes, but sometimes it’s more efficient in practice to break certain things up. utf-8 is one implementation of unicode in which one of these options is to “decompose” the ‘ç’ into a c and a cedilla, separately. You can see this specified on this table of utf-8 encodings, with some fiddling with the settings to show the decimal encoding and going to the second page. You can see in the full unicode specification of the ‘ç’ that it does have it’s on unicode designation, but when it comes to bit storage, it combines two other symbols.