Comparing Folder File Lists in Python

Compare two folders for missing paths and changed file sizes. Then try different contents with the same size to see what this check misses.

You'll need Python 3, a plain-text editor and a command window. No extra packages are required.

1. Prepare the folders

Start in a practice folder without Before or After. Save this as MakeFolders.py:

from pathlib import Path


Before = Path("Before")
After = Path("After")
if Before.exists() or Before.is_symlink() or After.exists() or After.is_symlink():
    raise SystemExit("Use a fresh practice folder without Before or After.")
Before.mkdir()
After.mkdir()
(Before / "Note.txt").write_bytes(b"abc")
(Before / "Old.txt").write_bytes(b"old")
(After / "Note.txt").write_bytes(b"xy")
(After / "New.txt").write_bytes(b"new")

Run:

python3 MakeFolders.py

Use your installation's Python 3 command if it is not named python3. It creates:

Before/
    Note.txt    abc
    Old.txt     old
After/
    Note.txt    xy
    New.txt     new

The setup writes exact bytes with no line endings, so editor settings do not change the sizes. A byte is a small unit of stored data. If either name already exists, including as a symbolic link, setup stops before creating or writing anything. Use another practice folder rather than removing files you need. This check is not protection against another process changing the paths during setup.

The folder names set the comparison direction, not proof of which copy is older. Keep both programs outside the inputs.

2. Add the folder reader

Save this as ListFolder.py:

import json as Json
import sys as Sys
from pathlib import Path


def ListFolder(FolderPath):
    Folder = Path(FolderPath)
    if Folder.is_symlink():
        raise ValueError("folder must not be a symbolic link")
    if not Folder.is_dir():
        raise ValueError("input must be a folder")
    Records = []
    SkippedLinks = []
    Pending = [Folder]
    Seen = 0
    while Pending:
        Current = Pending.pop()
        for Item in Current.iterdir():
            Seen += 1
            if Seen > 10000:
                raise ValueError("folder exceeds the 10000-entry limit")
            Relative = Item.relative_to(Folder).as_posix()
            if Item.is_symlink():
                SkippedLinks.append(Relative)
            elif Item.is_dir():
                Pending.append(Item)
            elif Item.is_file():
                Records.append({"Path": Relative, "Bytes": Item.stat().st_size})
            else:
                raise ValueError("folder contains an unsupported file type")
    Records.sort(key=lambda Record: Record["Path"])
    SkippedLinks.sort()
    return {"Files": Records, "SkippedLinks": SkippedLinks}


def Main():
    if len(Sys.argv) != 2:
        print("Usage: python3 ListFolder.py folder", file=Sys.stderr)
        return 2
    try:
        Report = ListFolder(Sys.argv[1])
    except (OSError, ValueError) as Problem:
        print(f"Listing stopped: {Problem}", file=Sys.stderr)
        return 2
    print(Json.dumps(Report, ensure_ascii=True, indent=2))
    return 1 if Report["SkippedLinks"] else 0


if __name__ == "__main__":
    Sys.exit(Main())

The reader builds an inventory, a list of files and sizes. Paths are relative: the starting folder is removed, so Before/Note.txt and After/Note.txt both become Note.txt.

It visits subfolders, skips symbolic links (paths pointing to other files or folders), and omits empty folders. Its 10,000-entry limit per input counts folders and links as well as files. A failed walk or unsupported special entry stops the comparison.

3. Add the comparison

Save this as CompareFolders.py beside the reader:

import json as Json
import sys as Sys
from ListFolder import ListFolder


def CompareFolders(BeforePath, AfterPath):
    BeforeReport = ListFolder(BeforePath)
    AfterReport = ListFolder(AfterPath)
    Before = {Record["Path"]: Record["Bytes"] for Record in BeforeReport["Files"]}
    After = {Record["Path"]: Record["Bytes"] for Record in AfterReport["Files"]}
    OnlyBefore = sorted(Before.keys() - After.keys())
    OnlyAfter = sorted(After.keys() - Before.keys())
    Changed = []
    SameSize = 0
    for Name in sorted(Before.keys() & After.keys()):
        if Before[Name] != After[Name]:
            Changed.append({"Path": Name, "BeforeBytes": Before[Name], "AfterBytes": After[Name]})
        else:
            SameSize += 1
    return {"OnlyBefore": OnlyBefore, "OnlyAfter": OnlyAfter, "SizeChanged": Changed,
            "SameSizeFiles": SameSize, "BeforeSkippedLinks": BeforeReport["SkippedLinks"],
            "AfterSkippedLinks": AfterReport["SkippedLinks"]}


def Main():
    if len(Sys.argv) != 3:
        print("Usage: python3 CompareFolders.py before-folder after-folder", file=Sys.stderr)
        return 2
    try:
        Report = CompareFolders(Sys.argv[1], Sys.argv[2])
    except (OSError, ValueError) as Problem:
        print(f"Comparison stopped: {Problem}", file=Sys.stderr)
        return 2
    print(Json.dumps(Report, ensure_ascii=True, indent=2))
    ReviewNeeded = any(Report[Name] for Name in
                       ["OnlyBefore", "OnlyAfter", "SizeChanged", "BeforeSkippedLinks", "AfterSkippedLinks"])
    return 1 if ReviewNeeded else 0


if __name__ == "__main__":
    Sys.exit(Main())

Before and After are dictionaries, collections for looking up values by keys. Each path points to a size. Subtracting their keys finds paths only on one side; & finds paths on both sides to compare sizes.

The output uses JSON, a text format for named values and lists. Paths are sorted, and unusual characters are escaped: a newline in a name becomes \n rather than breaking the report into another line.

4. Compare the folders

Open a command window beside your programs and inputs:

python3 CompareFolders.py Before After

Use your installation's Python 3 command if it is not named python3. The output is:

{
  "OnlyBefore": [
    "Old.txt"
  ],
  "OnlyAfter": [
    "New.txt"
  ],
  "SizeChanged": [
    {
      "Path": "Note.txt",
      "BeforeBytes": 3,
      "AfterBytes": 2
    }
  ],
  "SameSizeFiles": 0,
  "BeforeSkippedLinks": [],
  "AfterSkippedLinks": []
}

Old.txt exists only in Before, and New.txt only in After. Note.txt is on both sides but changed from three bytes to two. These labels describe the chosen folders, not proof that a file was created or deleted.

The exit code, a small result number another script can check, is 0 for matching paths/sizes with no skipped links, 1 for differences or links to review, and 2 for a command/comparison error. Even identical skipped links give 1, because their targets were not compared.

5. Try different content with the same size

Save this as ChangeNote.py beside your programs, after creating the practice files:

from pathlib import Path


Path("After/Note.txt").write_bytes(b"xyz")

This replaces the practice note, not a real recovered file. Run:

python3 ChangeNote.py
python3 CompareFolders.py Before After

SizeChanged is now empty and SameSizeFiles is 1, even though abc and xyz differ. The exit code stays 1 because Old.txt and New.txt are still present on only one side. Same size does not mean identical, and one changed result does not remove the other differences.

Next, compare their bytes directly or compare a fingerprint, a calculated content-reference value, against a trustworthy saved value. This inventory program copies, overwrites and deletes nothing.

The comparison passed 16 checks, including reversed direction, nested files, omitted empty folders, links and same-size different content. The reader passed 17 separate checks. The sample output was reproduced.

Use stable folders: the walks happen one after the other, not at one shared instant. The program does not freeze them, and replacing paths during a walk can change what gets visited. It omits ownership, permissions, empty folders and contents; a matching inventory is not proof that a backup is usable.

Save output outside both inputs, under a new filename. A report inside one side can list itself, and redirecting over a file you need can destroy it. Keep inventories private when their names reveal personal or work information.