Decode pyproject.toml as UTF-8 regardless of system locale

[staticdev]

• Just passes binary to tomlkit. The only way to work on Windows to parse .toml files with special characters.

Closes #213

[cjolowicz]

Thank you for the report and for contributing a fix! 🙇‍♂️

Per the TOML spec, input documents must be valid UTF-8. IIUC the problem is that Path.read_text() ends up using the preferred locale encoding instead. Judging by the traceback in your issue description, this happens to be CP-1252 on your system.

As a fix, I would prefer to be explicit about the encoding:

-        text = path.read_text()
+        text = path.read_text(encoding="utf-8")

Could you try this and adapt the PR?

[staticdev]

Thank you for the report and for contributing a fix!

Per the TOML spec, input documents must be valid UTF-8. IIUC the problem is that Path.read_text() ends up using the preferred locale encoding instead. Judging by the traceback in your issue description, this happens to be CP-1252 on your system.

As a fix, I would prefer to be explicit about the encoding:

-        text = path.read_text()
+        text = path.read_text(encoding="utf-8")

Could you try this and adapt the PR?

Done @cjolowicz.

[cjolowicz]

Released in 0.7.1