Checking JSON for Repeated Keys in Python
A JSON object can contain the same key twice, and Python's usual reader keeps only the last value. Build a checker that stops on repeated keys and non-standard number constants before that information disappears.
You'll need Python 3.10 or later, a plain-text editor and a command window. No extra packages are required.
1. Make a small JSON file
JSON is a text format for values, lists and named fields. An object uses braces and key/value pairs; an array uses square brackets. Save this as settings.json in UTF-8, the text encoding used here:
{
"Name": "Ada",
"Count": 2,
"Enabled": true
}
Now make a second file, repeated.json:
{
"Name": "Ada",
"Name": "Ben"
}
The second object has two Name keys. Python's default JSON decoder accepts this and returns only the last value. Our checker will stop instead.
2. Save the checker
Save this as CheckJson.py beside the files:
import json as Json
import sys as Sys
from pathlib import Path
def UniqueObject(Pairs):
Result = {}
for Key, Value in Pairs:
if Key in Result:
raise ValueError("repeated object key")
Result[Key] = Value
return Result
def RejectConstant(Value):
raise ValueError("non-standard number constant")
def CheckJson(FileName):
with Path(FileName).open("rb") as Input:
Data = Input.read(65537)
if len(Data) > 65536:
raise ValueError("input exceeds 64 KiB")
Text = Data.decode("utf-8-sig")
Value = Json.loads(Text, object_pairs_hook=UniqueObject,
parse_constant=RejectConstant)
Kind = "object" if isinstance(Value, dict) else "array" if isinstance(Value, list) else "scalar"
print(f"JSON check passed; top-level value: {Kind}")
return 0
def Main():
if len(Sys.argv) != 2:
print("Usage: python3 CheckJson.py input.json", file=Sys.stderr)
return 2
try:
return CheckJson(Sys.argv[1])
except (OSError, UnicodeError, ValueError, RecursionError) as Error:
print(f"Check stopped: {Error}", file=Sys.stderr)
return 2
if __name__ == "__main__":
raise SystemExit(Main())
object_pairs_hook gives UniqueObject the object's pairs before they become a dictionary, a collection that looks up values by keys. The function checks each key before adding it. The hook runs for nested objects too, not only the outer one.
Matching happens after JSON escapes are decoded. Name and \u004eame are therefore the same key; Name and name are different. The checker reports a repeated key without echoing its text.
parse_constant calls RejectConstant for NaN, Infinity and -Infinity. These are number spellings Python accepts by default, but they are not part of standard JSON. Rejecting them is separate from checking repeated keys.
The input limit is 64 KiB (65,536 bytes). The program reads one extra byte to detect larger files before decoding. It accepts UTF-8 with or without an initial byte-order mark, an optional encoding marker. A deeply nested file can still reach Python's recursion limit and stop; the size cap does not make this a hostile-input sandbox.
3. Run both examples
Open a command window in the folder:
python3 CheckJson.py settings.json
Use your installation's Python 3.10-or-later command if it is not called python3. The output is:
JSON check passed; top-level value: object
Now run:
python3 CheckJson.py repeated.json
It stops with:
Check stopped: repeated object key
An exit code is the small result number returned when a program ends. This checker returns 0 when parsing and these checks finish, or 2 when they stop. It does not print a partial success result.
4. Check nested keys and number constants
Try this in a new file:
{"Outer": {"Count": 1, "Count": 2}}
The nested repeated key is rejected. Keys are checked within each object, not across the whole file. Try this array in a new file:
[{"Name":"Ada"},{"Name":"Ben"}]
It passes with top-level value: array: each object has its own Name key. Now put two Name keys inside either object and it stops. This matters when a file contains many records with the same field names.
Replace the contents with:
{"Count": NaN}
That stops with non-standard number constant. A trailing comma or two separate top-level values also stops parsing.
Objects are not the only valid top-level values. An array such as [1, 2], or a scalar (one value rather than a container) such as 42, true, null or a quoted string, can pass. The output names the kind; it does not require your file to be an object.
5. Add checks for what the values mean
Passing means the file parsed without repeated object keys or those non-standard constants. It does not mean a field exists, has the right type, or contains a sensible value. Save this as wrong-type.json:
{"Count": "two"}
Run python3 CheckJson.py wrong-type.json. It prints JSON check passed; top-level value: object and returns 0: "two" is a valid JSON string. If the receiving program requires Count to be a whole number, this document still fails that separate rule. Save {} as missing-field.json and run the checker on it too; it passes despite having no Count field. Decide the required fields and types before adding those checks, rather than treating parse success as permission to use the data.
This program also does not check numeric ranges. The valid number spelling 1e400 passes, although Python's default floating-point conversion produces an infinite value for that magnitude. If you need exact decimal amounts or bounded numbers, choose and test that conversion separately before using the data.
The checker passed 23 command-line checks on Python 3.10.12, covering nested and escaped repeated keys, constants, syntax errors, top-level types, UTF-8, the byte boundary, deep nesting, numeric conversion, missing input and usage errors. Every file test confirmed unchanged input bytes.
Use a saved file that will not change while it is read. Save output under a new filename: redirecting over the input can empty it before Python opens it. JSON parse errors may include line/column positions, while this repeated-key check does not identify the object's path. This is a small local format check, not a complete data validator.