Mojibake Patterns Chart: Identify the Encoding Mix-Up from 縺, ã, ‚, ƒ, and $B

Mojibake is a mismatch between the encoding saved and the encoding read

All text on a computer is stored as a sequence of numbers (bytes). A character encoding is the set of rules for which number means which character, and four are common for Japanese: UTF-8, Shift_JIS, EUC-JP, and JIS (ISO-2022-JP). The hiragana “a”, for example, is the 3 bytes E3 81 82 in UTF-8 and the 2 bytes 82 A0 in Shift_JIS.

Mojibake happens when bytes saved in one encoding are read under the rules of another. The bytes themselves aren’t damaged; they’ve only been read the wrong way. So in most cases, turning the text back into bytes with the encoding it was wrongly read as, then reading those bytes with the correct encoding, restores the original.

Chart of the seven most common patterns

This table shows results actually garbled with the Mojibake Fixer & Decoder. If you see a lot of the marker characters on the left, suspect that row’s mix-up. The example word is the Japanese greeting “konnichiwa”.

With other words: 日本語 (“Japanese”) becomes “日本語” in UTF-8 → Latin-1 and ““ú–{Œê” in Shift_JIS → Latin-1; the katakana word for “test” becomes “ƒeƒXƒg” in Shift_JIS → Latin-1; 東京都 (“Tokyo”) becomes “譚ｱ莠ｬ驛ｽ” in UTF-8 → Shift_JIS; and the Japanese word for “mojibake” becomes “ʸ»ú²½¤±” in EUC-JP → Latin-1.

“縺”, “繧”, “譁”: UTF-8 opened as Shift_JIS

This is the pattern you’ll see most often. It happens when a CSV saved in UTF-8 is opened directly in Excel, when a file is opened in older Windows software or Notepad, or when UTF-8 data is imported into a business system that assumes Shift_JIS.

• Text heavy in hiragana fills up with “縺” (most hiragana begin with E3 81 or E3 82 in UTF-8, which Shift_JIS reads as “縺” and “繧”)
• Text heavy in katakana stands out with “繝” (most katakana begin with E3 83)
• Kanji garble into unusual kanji such as “譁”, “莨”, and “螻”
• UTF-8 uses 3 bytes per character and Shift_JIS divides by 2, so where the boundaries fall out of step you get “�” or “・”

Mixed-in “�”, as in “譁�ｭ怜喧縺�” (the Japanese word for “mojibake”), is the hallmark of this pattern. One original byte is lost at each “�”, but UTF-8’s rules reveal how many bytes are missing and where, so the tool infers characters that make natural Japanese. See Can “�” and “?” Mojibake Be Recovered? for details.

“ã”, “æ”, “‚”, “ƒ”: opened in software made for other markets

When everything turns into accented letters and symbols, Japanese has been read in a Western European encoding (Latin-1, or strictly speaking Windows-1252). It tends to happen in foreign email software and CMSs, WordPress with mismatched database encoding settings, and English versions of software.

• Many characters starting with “ã” → UTF-8 hiragana and katakana (first byte E3). “æ”, “å”, “ä”, “è”, “é”, and “ç” are UTF-8 kanji (first byte E4–E9)
• Prominent “— → UTF-8 symbols with first byte E2, such as “…”, ““”, and “–”
• Many “‚” → Shift_JIS hiragana (first byte 82). “ƒ” is katakana (first byte 83)
• Many “¤” or “¥” → EUC-JP hiragana (first byte A4) and katakana (first byte A5)

Latin-1 assigns a character to nearly all of the 256 possible bytes, so the original bytes survive intact and this pattern is easy to reverse. However, a few bytes with no character in Windows-1252 (81, 8D, 8F, 90, 9D) become invisible control characters and can drop out when copied. Even “ã“ã‚“ã«ã¡ã¯” actually has an invisible character between “ã” and ““”. The tool works out the positions of dropped bytes and restores them.

“､” and “$B”: encodings from old servers and email

Half-width “､”, “ｳ”, and “ﾋ” appearing in alternation means EUC-JP text was read as Shift_JIS. EUC-JP was used on older Linux and UNIX servers, bulletin boards, and old Perl CGI scripts. You’ll run into it when opening logs from old websites or email archives.

ASCII text wrapped in “$B” and “(B”, like “$BJ8;z2=$1(B”, means the ESC control characters that signal switching have dropped out of JIS (ISO-2022-JP) text. Japanese email has been sent in this encoding, so it happens when old messages are moved to other software or copied as text. Reading what’s between “$B” and “(B” two characters at a time restores it. Garbled email subjects are covered in Garbled Email Subjects.

Once you know the pattern, how do you reverse it?

1. 1

   If you have the original file, reopening it is the sure way

   If only the display is garbled, reopening the file in an editor or browser with the encoding specified restores it. Saving over the file while it’s garbled destroys the information at each “�”, so it’s important to notice before you save.

2. 2

   If all you have is the garbled text, paste it in

   When all you have is an email body or a copy from a screen, paste it into the Mojibake Fixer & Decoder. It brute-forces the seven mix-ups and double garbling (for example, text garbled by UTF-8 → Shift_JIS and then read as Latin-1), and lists results starting with those that read as Japanese.

3. 3

   Check the characters filled in by inference

   Characters inferred from “�” are shown in yellow. Guesses can miss on short text, so tap any character that doesn’t fit the context and swap it for another candidate.

If a CSV garbled when opened in Excel, How to Fix CSV Garbled in Excel shows how to resave it so it won’t happen again. Strings such as “%E3%81%82” or “\u3042” aren’t garbled; they’re encoded text shown as is. For those, see How to Read %E3, \u3042, and \343.

FAQ

What causes mojibake full of “縺” and “繧”?

Japanese saved in UTF-8 was opened as Shift_JIS. Most UTF-8 hiragana begin with E3 81 or E3 82, which Shift_JIS reads as “縺” and “繧”. Reopen it in UTF-8, or reread the garbled text as UTF-8, to restore it.

What is mojibake with rows of “ã” or “—?

It’s what UTF-8 Japanese or symbols look like when displayed as Latin-1 (Windows-1252). It comes from foreign software or mismatched database encoding settings. The original bytes are almost all intact, so most of it can be restored.

Can all mojibake be reversed?

It can be reversed if the garbled text still holds the original bytes. UTF-8 → Shift_JIS, UTF-8 → Latin-1, Shift_JIS → Latin-1, EUC-JP → Shift_JIS, and JIS without its ESC characters can all be reversed. Text read as EUC-JP that became all “�”, or text saved with “?” in place of characters, can’t.

I saved a file while it was garbled. Is it too late?

Even when saved in its garbled state, as long as the characters themselves remain, tracing the garbling backward may restore it. Characters already replaced with “�” or “?” at the time of saving have lost their information at those spots.