Working with JSON in Python
The four functions, the type table, and where each job is covered in detail.
Python handles JSON with the json module, which is part of the standard library. It has four functions you will use all the time, and this page is the map: what each one does, how the types convert, and which recipe covers each job in depth.
The short answer
import json
text = '{"name": "Nitro", "stars": 120, "tags": ["python", "static"]}'
data = json.loads(text)
data["stars"] += 1
print(json.dumps(data))
json.loads turns JSON text into Python objects, and json.dumps turns Python objects back into JSON text. Everything else is a variation on those two.
The four functions
| Function | Direction | Works with | Recipe |
|---|---|---|---|
json.loads(text) | JSON to Python | a str or bytes | Parse JSON |
json.load(file) | JSON to Python | an open file | Read a JSON file |
json.dumps(obj) | Python to JSON | returns a str | json.dumps |
json.dump(obj, file) | Python to JSON | writes to an open file | Write a JSON file |
The s stands for "string". The versions without it work on files.
How JSON types map to Python types
| JSON | Python |
|---|---|
| object | dict |
| array | list |
| string | str |
| number without a fraction | int |
| number with a fraction or exponent | float |
true / false | True / False |
null | None |
Going the other way, a tuple is also written as an array, and dict keys that are numbers, booleans or None are written as strings. Anything else, such as a datetime, a set or one of your own classes, needs converting first. The json.dumps recipe shows how.
A round trip, and what does not survive it
Because JSON has fewer types than Python, dumping and loading does not always give back exactly what you started with:
original = {"point": (3, 4), 1: "one"}
copy = json.loads(json.dumps(original))
print(copy) # {'point': [3, 4], '1': 'one'}
print(copy == original) # False
The tuple came back as a list and the integer key came back as a string. If your data holds only dicts with string keys, lists, strings, numbers, booleans and None, the round trip is exact.
Working with a JSON file
Use json.dump to write a file and json.load to read it. Open the file with encoding="utf-8" both times:
settings = {"theme": "dark", "font_size": 14}
with open("settings.json", "w", encoding="utf-8") as f:
json.dump(settings, f, indent=2)
with open("settings.json", encoding="utf-8") as f:
loaded = json.load(f)
print(loaded == settings) # True
indent=2 makes the file readable, and pretty printing covers the other formatting options. Missing files, broken files and missing keys are handled in reading a JSON file. Updating or appending to a file that already exists is in writing a JSON file.
Common errors
json.JSONDecodeError means the text is not valid JSON. The usual causes are a trailing comma, single quotes, or an empty string:
try:
json.loads("{'name': 'Nitro'}")
except json.JSONDecodeError as e:
print(e) # Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
Parsing JSON goes through the messages one by one.
TypeError: Object of type ... is not JSON serializable means json.dumps met a value JSON has no type for:
try:
json.dumps({"tags": {"python", "static"}})
except TypeError as e:
print(e) # Object of type set is not JSON serializable
TypeError: the JSON object must be str, bytes or bytearray, not dict means you called json.loads on something that is already parsed.
AttributeError: module 'json' has no attribute 'loads' nearly always means there is a file called json.py in your project, and Python is importing that. Rename it.
KeyError after loading is not a JSON problem. The data parsed fine and the key is not there.
Querying what you loaded
Parsed JSON is ordinary dicts and lists, so indexing, loops and comprehensions are the query language. For anything more involved, see JSONPath and querying JSON.
The shorter way with Nitro DataStore
The json module stops once the data is loaded. Nitro DataStore picks up from there: it wraps the dict, or the file, and adds dotted paths with defaults, nested set, and a save that writes the file back.
from nitro_datastore import NitroDataStore
store = NitroDataStore.from_file("settings.json")
store.get("editor.tab_size", 4) # 4, no KeyError
store.set("editor.tab_size", 2)
store.save("settings.json")
store.to_dict()
# {'theme': 'dark', 'font_size': 14, 'editor': {'tab_size': 2}}
It uses the json module underneath and gives you a plain dict back whenever you ask. For a script that loads one file and reads two keys, it is more than you need. It is useful once several parts of a program read and edit the same nested data.
Common questions
Do I need to install anything to use JSON in Python? No. import json works on every Python installation.
Is a JSON object the same as a Python dict? No. JSON is text, and a dict is an object in memory. They look alike on the page, but JSON insists on double quotes, writes true, false and null, and only allows strings as keys.
Is there a faster JSON library? Yes. orjson and ujson are third-party packages that parse and write faster. The standard module is quick enough for most programs, so measure before you switch.
Related
Parse JSON in Python
Turn JSON text into dicts and lists, and deal with text that is not JSON after all.
Read ->json.dumps in Python
Turn a dict into JSON text, and teach it about dates, decimals and sets.
Read ->Read a JSON file in Python
Open a JSON file, get a dict back, and handle the three ways it goes wrong.
Read ->This page uses Nitro DataStore.
Read JSON with dot notation and a chainable query builder. No schema to define, no ORM to learn.
$ pip install nitro-datastore