Comparing Connection-Log Summaries in Python

Compare two connection logs to find source, destination and port groups that appeared, disappeared or changed count. Two small files will show how the direction of comparison changes the result.

You'll need Python 3, a plain-text editor and a command window. No extra packages are required.

1. Create two input files

Use a tab-separated text file with four columns: time, source address, destination address and destination port. The header must be Time, Source, Destination, Port, with actual tabs between the names. A .tsv file is a plain-text table that uses tabs to separate columns.

Write times as YYYY-MM-DDThh:mm:ssZ. The Z means Coordinated Universal Time (UTC), the shared reference time rather than a local clock. This reader accepts whole seconds in that exact form, not fractional seconds, local offsets such as +01:00 or leap-second values. Addresses use Internet Protocol version 4 (IPv4), four dot-separated numbers such as 192.0.2.10, without extra leading zeros. Ports are whole numbers from 1 to 65535.

The sample addresses are reserved for documentation in RFC 5737, a published document describing addresses to use in examples. Preserve the tabs when copying. This is the reader's own input format, not a direct export from a network-monitoring product.

Save this as before.tsv:

Time	Source	Destination	Port
2026-09-30T09:00:00Z	192.0.2.10	198.51.100.20	443
2026-09-30T09:00:01Z	192.0.2.10	198.51.100.20	443
2026-09-30T09:00:02Z	192.0.2.11	203.0.113.30	53
2026-09-30T09:00:03Z	192.0.2.13	203.0.113.40	25

Then save after.tsv:

Time	Source	Destination	Port
2026-09-30T10:00:00Z	192.0.2.10	198.51.100.20	443
2026-09-30T10:00:01Z	192.0.2.10	198.51.100.20	443
2026-09-30T10:00:02Z	192.0.2.10	198.51.100.20	443
2026-09-30T10:00:03Z	192.0.2.14	203.0.113.50	3389
2026-09-30T10:00:04Z	192.0.2.11	203.0.113.30	53

These files do not declare their collection start and end. We can compare row counts, but cannot calculate rates without knowing the periods they cover.

2. Add the reader

The complete reader is included below so you can run this guide on its own.

Save this as LogReader.py. This reusable source file checks each row and keeps its physical line number, either with accepted values or a reason it could not be read.

from dataclasses import dataclass as DataClass
from datetime import datetime as DateTime
from pathlib import Path


@DataClass
class LogEntry:
    Number: int
    Original: str
    EventTime: str = ""
    Source: str = ""
    Destination: str = ""
    Port: int = 0
    Problem: str = ""
    Accepted: bool = False


def Digits(Value, Maximum):
    if not Value:
        raise ValueError("empty number")
    if any(Character not in "0123456789" for Character in Value):
        raise ValueError("number needs ordinary digits")
    Number = int(Value)
    if Number > Maximum:
        raise ValueError("number outside supported range")
    return Number


def CheckAddress(Value):
    Parts = Value.split(".")
    if len(Parts) != 4:
        raise ValueError("expected four IPv4 address parts")
    for Part in Parts:
        if len(Part) > 3 or (len(Part) > 1 and Part.startswith("0")):
            raise ValueError("use canonical dotted IPv4 addresses")
        Digits(Part, 255)


def CheckTime(Value):
    if len(Value) != 20:
        raise ValueError("expected time YYYY-MM-DDThh:mm:ssZ")
    if any(Value[Index] != Mark for Index, Mark in
           ((4, "-"), (7, "-"), (10, "T"), (13, ":"), (16, ":"), (19, "Z"))):
        raise ValueError("expected time YYYY-MM-DDThh:mm:ssZ")
    NumberText = Value[0:4] + Value[5:7] + Value[8:10] + Value[11:13] + Value[14:16] + Value[17:19]
    if any(Character not in "0123456789" for Character in NumberText):
        raise ValueError("time needs ordinary digits")
    Year = Digits(Value[0:4], 9999)
    Month = Digits(Value[5:7], 12)
    Day = Digits(Value[8:10], 31)
    Hour = Digits(Value[11:13], 23)
    Minute = Digits(Value[14:16], 59)
    Second = Digits(Value[17:19], 59)
    try:
        DateTime(Year, Month, Day)
    except ValueError:
        raise ValueError("date does not exist") from None
    try:
        DateTime(Year, Month, Day, Hour, Minute, Second)
    except ValueError:
        raise ValueError("time does not exist") from None


def ParseEntry(Entry):
    try:
        if len(Entry.Original) > 1024:
            raise ValueError("line exceeds 1024 bytes")
        if any(Character != "\t" and not 32 <= ord(Character) <= 126
               for Character in Entry.Original):
            raise ValueError("unsupported byte in practice format")
        Fields = Entry.Original.split("\t")
        if len(Fields) != 4:
            raise ValueError("expected exactly four tab-separated fields")
        CheckTime(Fields[0])
        CheckAddress(Fields[1])
        CheckAddress(Fields[2])
        Entry.Port = Digits(Fields[3], 65535)
        if Entry.Port == 0:
            raise ValueError("port must be 1 to 65535 in this format")
        Entry.EventTime, Entry.Source, Entry.Destination = Fields[:3]
        Entry.Accepted = True
    except ValueError as Error:
        Entry.Problem = str(Error)
    return Entry


def ReadLog(FileName):
    with Path(FileName).open("rb") as Input:
        Data = Input.read(1048577)
    if len(Data) > 1048576:
        raise ValueError("input exceeds 1 MiB teaching limit")
    if not Data:
        raise ValueError("empty input")
    Lines = Data.split(b"\n")
    if Lines[-1] == b"":
        Lines.pop()
    Lines = [Line[:-1] if Line.endswith(b"\r") else Line for Line in Lines]
    if Lines[0] != b"Time\tSource\tDestination\tPort":
        raise ValueError("unsupported header")
    if len(Lines) - 1 > 1000:
        raise ValueError("more than 1000 data lines")
    return [ParseEntry(LogEntry(Number, Line.decode("latin-1")))
            for Number, Line in enumerate(Lines[1:], start=2)]

The reader handles up to 1,000 data lines in a file of at most 1 MiB, roughly one million bytes, and rejects data lines longer than 1,024 bytes. Its fields use ordinary printable English-character bytes and tabs. Use a saved file that will not change while it is read.

3. Add the comparison

Save this as CompareSummaries.py beside the reader:

import sys as Sys
from LogReader import ReadLog


def CountGroups(FileName):
    Groups = {}
    Bad = 0
    for Entry in ReadLog(FileName):
        if Entry.Accepted:
            Key = (Entry.Source, Entry.Destination, Entry.Port)
            Groups[Key] = Groups.get(Key, 0) + 1
        else:
            Bad += 1
    return Groups, Bad


def Describe(Key):
    Source, Destination, Port = Key
    return f"{Source} {Destination} {Port}"


def Main():
    if len(Sys.argv) != 3:
        print("Usage: python3 CompareSummaries.py before.tsv after.tsv", file=Sys.stderr)
        return 2
    try:
        Before, BeforeBad = CountGroups(Sys.argv[1])
        After, AfterBad = CountGroups(Sys.argv[2])
    except (OSError, ValueError) as Error:
        print(f"Input stopped: {Error}", file=Sys.stderr)
        return 2
    Differences = 0
    Same = 0
    for Key in sorted(set(Before) | set(After)):
        Old = Before.get(Key, 0)
        New = After.get(Key, 0)
        if Old == 0:
            print(f"New: {Describe(Key)} now {New}")
            Differences += 1
        elif New == 0:
            print(f"Disappeared: {Describe(Key)} was {Old}")
            Differences += 1
        elif Old != New:
            print(f"Changed: {Describe(Key)} {Old} -> {New}")
            Differences += 1
        else:
            Same += 1
    print(f"Differences: {Differences}; unchanged groups: {Same}.")
    print(f"Rejected lines: before {BeforeBad}, after {AfterBad}. A group missing from a file may sit on a rejected line.")
    return 1 if Differences or BeforeBad or AfterBad else 0


if __name__ == "__main__":
    raise SystemExit(Main())

CountGroups counts accepted entries for each source, destination and port. Main visits groups found in either file, using zero for an absent group, then labels differences as changed, disappeared or new. Equal counts contribute to the unchanged total.

Rejected rows are counted separately. A missing group may have rejected rows. If either file has rejections, inspect those rows with the reader before interpreting the group's disappearance. Repeated accepted rows count again on both sides.

4. Compare the files

Open a command window in the folder and run:

python3 CompareSummaries.py before.tsv after.tsv

Use your installation's Python 3 command if it is not named python3. The output is:

Changed: 192.0.2.10 198.51.100.20 443 2 -> 3
Disappeared: 192.0.2.13 203.0.113.40 25 was 1
New: 192.0.2.14 203.0.113.50 3389 now 1
Differences: 3; unchanged groups: 1.
Rejected lines: before 0, after 0. A group missing from a file may sit on a rejected line.

The port 443 group went from two entries to three. The port 25 group appears only in before.tsv, while port 3389 appears only in after.tsv. The port 53 group has one entry on each side.

The exit code, a small result number another script can check, is 0 for equal groups/counts without rejected rows, 1 for differences or rejections, and 2 when the command or comparison fails. This sample returns 1.

5. Check the direction

Compare the first file with itself:

python3 CompareSummaries.py before.tsv before.tsv

Differences: 0; unchanged groups: 3.
Rejected lines: before 0, after 0. A group missing from a file may sit on a rejected line.

Now reverse the original command's file order. New and disappeared groups swap sides, and changed counts run in the other direction. The labels depend on the starting point, not on why something changed.

Choose comparable collection periods and the same logging setup. A different schedule, missing collection or changed file scope can alter counts without indicating a threat.

The comparison and window programs passed 22 command-line checks, including reversed direction, equal files, rejected rows and load errors. Tested inputs remained unchanged.

Save reports under a new filename: redirecting output over an input can empty it before Python opens it. Keep reports private when they reveal personal or work activity.