Fix CSV Encoding Issues (UTF-8 Guide)

You open a CSV file and Café appears as Café, or a name loses its accented characters. Those symptoms usually mean the program decoded the file using a different character encoding from the one that created it.

The most common conflict is between UTF-8 and a legacy single-byte encoding such as Windows-1252. Guessing and resaving can make the damage permanent, so keep the original file while you test the encoding.

Here is how to fix it and make your text readable again.

What is Encoding?

Computers store text as numbers (bytes). An "encoding" is the map that tells the computer which number equals which letter.

• UTF-8: The standard. Supports every character (English, Chinese, Emojis).
• Windows-1252 / ANSI: Older Windows standard. Supports English and Western European well, breaks on everything else.
• ASCII: Ancient. Only supports basic English letters.

The Problem: If you save a file as UTF-8 (using 2 bytes for "é") but Excel reads it as ANSI (expecting 1 byte), it misinterprets the data. Result: Garbage text ("Mojibake").

How to Fix It

Method 1: The "Import" Trick in Excel

Don't double-click the CSV. That forces Excel to guess (and it often guesses wrong).

1. Open a blank Excel workbook.
2. Go to Data > Get Data > From Text/CSV.
3. Select your file.
4. In the preview window, look for "File Origin".
5. Change it to "65001: Unicode (UTF-8)".
6. Check the preview. Does "Café" look right?
7. Click Load.

Method 2: Notepad (Windows)

You can use Notepad to convert the file.

1. Right-click your CSV > Open with > Notepad.
2. Does the text look correct here?
   • Yes: The file is good, Excel just read it wrong. Go to step 3.
   • No: The file itself is corrupted or saved in a very weird format.
3. Click File > Save As.
4. At the bottom, look for Encoding.
5. Select UTF-8 (or "UTF-8 with BOM").
6. Save.

Note: "UTF-8 with BOM" adds a hidden character at the start that tells Excel "Hey, I am UTF-8!". This often makes double-clicking work.

Method 3: TextEdit (Mac)

1. Open TextEdit.
2. Cmd+O to open.
3. Select your file.
4. Before clicking Open, click Options at the bottom.
5. Change Plain Text Encoding to Unicode (UTF-8) (or try others if the preview looks wrong).
6. Open, then Cmd+S to save (ensure UTF-8 is selected).

Method 4: Online Tools

If you don't want to fiddle with system settings, use a tool designed to fix this.

-> Fix CSV Encoding Tool

Preventing Encoding Issues

1. Always Export as UTF-8

When saving from any software, look for settings like "Export charset" or "Encoding". Always choose UTF-8.

2. Use the BOM (Byte Order Mark)

If you are generating CSVs for Excel users, add a BOM (\ufeff) at the very beginning of the file. This forces Excel to switch to UTF-8 mode.

Python Example:

import csv

with open('output.csv', 'w', newline='', encoding='utf-8-sig') as f:
    writer = csv.writer(f)
    writer.writerow(['Name', 'Café'])

utf-8-sig adds the BOM automatically.

3. Avoid "CSV (Macintosh)" or "CSV (MS-DOS)"

In Excel's Save As menu, these are traps. Just use CSV (Comma delimited) or CSV UTF-8 (Comma delimited).

Confirm the source encoding before converting

An encoding detector returns a probability, not proof. Check names with accents, currency symbols, curly quotation marks and any non-Latin scripts you expect to see. If several candidate encodings look plausible, ask how the source system exported the file or inspect a known value.

Convert from the untouched original. Opening a damaged file and saving it again can replace undecodable bytes with �, at which point the missing character may be impossible to recover. After conversion, compare a sample from the beginning, middle and end of the file and confirm the record count has not changed.

A UTF-8 byte-order mark can help some versions of Excel recognize UTF-8, but not every receiving system wants one. Follow the destination specification rather than adding a BOM automatically to every file.

Summary

• Weird characters = Encoding mismatch.
• UTF-8 is the solution.
• Don't double-click to open in Excel; use the Data Import tab.
• Convert legacy files using Notepad or a dedicated tool.

---

Encoding still broken? HappyCSV lets you choose a likely source encoding, preview a sample, and convert the text to UTF-8. Verify accented and non-Latin characters before using the output.