Finding Time Gaps in a Log with Python

Sort a log by time and find gaps between neighbouring accepted entries. Choose a minimum gap, then review the timestamps and original line numbers.

You'll need Python 3, a plain-text editor and a command window. No extra packages are required.

1. Create the log file

Use a tab-separated text file with four columns: time, source address, destination address and destination port. The header must be Time, Source, Destination, Port, with actual tabs between the names. A .tsv file is a plain-text table that uses tabs to separate columns.

Write times as YYYY-MM-DDThh:mm:ssZ. The Z means Coordinated Universal Time (UTC), the shared reference time rather than a local clock. This reader accepts whole seconds in that exact form, not fractional seconds, local offsets such as +01:00 or leap-second values. Addresses use Internet Protocol version 4 (IPv4), four dot-separated numbers such as 192.0.2.10, without extra leading zeros. Ports are whole numbers from 1 to 65535.

The sample addresses are reserved for documentation in RFC 5737, a published document describing addresses to use in examples. Preserve the tabs when copying. This is the reader's own input format, not a direct export from a network-monitoring product.

Save this as Gaps.tsv:

Time	Source	Destination	Port
2026-09-30T09:05:00Z	192.0.2.10	198.51.100.20	443
2026-09-30T09:00:00Z	192.0.2.10	198.51.100.20	443
2026-09-30T09:00:30Z	192.0.2.10	198.51.100.20	443
2026-09-30T09:01:00Z	192.0.2.10	198.51.100.20	70000

The valid entries are out of time order. The last has an unsupported port, so it will be rejected rather than used to break a gap into shorter pieces.

2. Add the reader

The complete reader is included below so you can run this guide on its own.

Save this as LogReader.py. This reusable source file checks each row and keeps its physical line number, either with accepted values or a reason it could not be read.

from dataclasses import dataclass as DataClass
from datetime import datetime as DateTime
from pathlib import Path


@DataClass
class LogEntry:
    Number: int
    Original: str
    EventTime: str = ""
    Source: str = ""
    Destination: str = ""
    Port: int = 0
    Problem: str = ""
    Accepted: bool = False


def Digits(Value, Maximum):
    if not Value:
        raise ValueError("empty number")
    if any(Character not in "0123456789" for Character in Value):
        raise ValueError("number needs ordinary digits")
    Number = int(Value)
    if Number > Maximum:
        raise ValueError("number outside supported range")
    return Number


def CheckAddress(Value):
    Parts = Value.split(".")
    if len(Parts) != 4:
        raise ValueError("expected four IPv4 address parts")
    for Part in Parts:
        if len(Part) > 3 or (len(Part) > 1 and Part.startswith("0")):
            raise ValueError("use canonical dotted IPv4 addresses")
        Digits(Part, 255)


def CheckTime(Value):
    if len(Value) != 20:
        raise ValueError("expected time YYYY-MM-DDThh:mm:ssZ")
    if any(Value[Index] != Mark for Index, Mark in
           ((4, "-"), (7, "-"), (10, "T"), (13, ":"), (16, ":"), (19, "Z"))):
        raise ValueError("expected time YYYY-MM-DDThh:mm:ssZ")
    NumberText = Value[0:4] + Value[5:7] + Value[8:10] + Value[11:13] + Value[14:16] + Value[17:19]
    if any(Character not in "0123456789" for Character in NumberText):
        raise ValueError("time needs ordinary digits")
    Year = Digits(Value[0:4], 9999)
    Month = Digits(Value[5:7], 12)
    Day = Digits(Value[8:10], 31)
    Hour = Digits(Value[11:13], 23)
    Minute = Digits(Value[14:16], 59)
    Second = Digits(Value[17:19], 59)
    try:
        DateTime(Year, Month, Day)
    except ValueError:
        raise ValueError("date does not exist") from None
    try:
        DateTime(Year, Month, Day, Hour, Minute, Second)
    except ValueError:
        raise ValueError("time does not exist") from None


def ParseEntry(Entry):
    try:
        if len(Entry.Original) > 1024:
            raise ValueError("line exceeds 1024 bytes")
        if any(Character != "\t" and not 32 <= ord(Character) <= 126
               for Character in Entry.Original):
            raise ValueError("unsupported byte in practice format")
        Fields = Entry.Original.split("\t")
        if len(Fields) != 4:
            raise ValueError("expected exactly four tab-separated fields")
        CheckTime(Fields[0])
        CheckAddress(Fields[1])
        CheckAddress(Fields[2])
        Entry.Port = Digits(Fields[3], 65535)
        if Entry.Port == 0:
            raise ValueError("port must be 1 to 65535 in this format")
        Entry.EventTime, Entry.Source, Entry.Destination = Fields[:3]
        Entry.Accepted = True
    except ValueError as Error:
        Entry.Problem = str(Error)
    return Entry


def ReadLog(FileName):
    with Path(FileName).open("rb") as Input:
        Data = Input.read(1048577)
    if len(Data) > 1048576:
        raise ValueError("input exceeds 1 MiB teaching limit")
    if not Data:
        raise ValueError("empty input")
    Lines = Data.split(b"\n")
    if Lines[-1] == b"":
        Lines.pop()
    Lines = [Line[:-1] if Line.endswith(b"\r") else Line for Line in Lines]
    if Lines[0] != b"Time\tSource\tDestination\tPort":
        raise ValueError("unsupported header")
    if len(Lines) - 1 > 1000:
        raise ValueError("more than 1000 data lines")
    return [ParseEntry(LogEntry(Number, Line.decode("latin-1")))
            for Number, Line in enumerate(Lines[1:], start=2)]

The reader handles up to 1,000 data lines in a file of at most 1 MiB, roughly one million bytes, and rejects data lines longer than 1,024 bytes. Its fields use ordinary printable English-character bytes and tabs. Use a saved file that will not change while it is read.

3. Add the gap reviewer

Save this as FindLogGaps.py:

import json as Json
import sys as Sys
from datetime import datetime as DateTime
from LogReader import ReadLog


def FindLogGaps(FilePath, MinimumSeconds):
    Entries = []
    Rejected = []
    for Entry in ReadLog(FilePath):
        if Entry.Accepted:
            Entries.append(Entry)
        else:
            Rejected.append({"Line": Entry.Number, "Reason": Entry.Problem})
    Entries.sort(key=lambda Entry: (Entry.EventTime, Entry.Number))
    Gaps = []
    for Before, After in zip(Entries, Entries[1:]):
        BeforeTime = DateTime.strptime(Before.EventTime, "%Y-%m-%dT%H:%M:%SZ")
        AfterTime = DateTime.strptime(After.EventTime, "%Y-%m-%dT%H:%M:%SZ")
        Seconds = int((AfterTime - BeforeTime).total_seconds())
        if Seconds >= MinimumSeconds:
            Gaps.append({"From": Before.EventTime, "To": After.EventTime, "Seconds": Seconds,
                         "BeforeLine": Before.Number, "AfterLine": After.Number})
    return {"AcceptedEntries": len(Entries), "Gaps": Gaps, "RejectedLines": Rejected}


def Main():
    if len(Sys.argv) != 3:
        print("Usage: python3 FindLogGaps.py input.tsv minimum-seconds", file=Sys.stderr)
        return 2
    try:
        Value = Sys.argv[2]
        if not Value or len(Value) > 8 or any(Character not in "0123456789" for Character in Value):
            raise ValueError("minimum must be 1 to 31536000 whole seconds")
        MinimumSeconds = int(Value)
        if not 1 <= MinimumSeconds <= 31536000:
            raise ValueError("minimum must be 1 to 31536000 whole seconds")
        Report = FindLogGaps(Sys.argv[1], MinimumSeconds)
    except (OSError, ValueError) as Problem:
        print(f"Review stopped: {Problem}", file=Sys.stderr)
        return 2
    print(Json.dumps(Report, ensure_ascii=True, indent=2))
    return 1 if Report["Gaps"] or Report["RejectedLines"] else 0


if __name__ == "__main__":
    Sys.exit(Main())

Accepted entries sort by timestamp, then line number. zip(Entries, Entries[1:]) pairs each with the next. DateTime.strptime converts checked strings into date-and-time values; subtracting them gives a duration, and total_seconds includes days as well as seconds within a day.

All accepted times use UTC, so there is no time-zone conversion. Equal timestamps give a zero-second gap and do not qualify: the minimum starts at one second.

The setting accepts whole seconds from 1 to 31,536,000, equivalent to 365 days. This limits the setting, not how long a reported gap can be. All sources and ports are combined, rather than checked per device.

The output uses JSON, a text format for named values and lists. Rejected rows have a separate list of reasons.

4. Find gaps of at least a minute

Open a command window in the folder and run:

python3 FindLogGaps.py Gaps.tsv 60

Use your installation's Python 3 command if it is not named python3. The output is:

{
  "AcceptedEntries": 3,
  "Gaps": [
    {
      "From": "2026-09-30T09:00:30Z",
      "To": "2026-09-30T09:05:00Z",
      "Seconds": 270,
      "BeforeLine": 4,
      "AfterLine": 2
    }
  ],
  "RejectedLines": [
    {
      "Line": 5,
      "Reason": "number outside supported range"
    }
  ]
}

The 270-second gap runs from 09:00:30 to 09:05:00. Its line numbers run from 4 to 2 because timestamps were sorted without rewriting the file. Line 5 remains a rejection.

The exit code, a small result number another script can check, is 0 for no qualifying gaps or rejections, 1 when either needs review, and 2 for a command, setting or load error. A header-only file or one accepted entry has no pair to compare, so a 0 does not establish complete coverage.

5. Change the minimum

Run the same file with 271 instead of 60. The gap no longer qualifies, but the rejected row still gives exit code 1.

A gap between accepted entries does not prove nothing happened. It may reflect a quiet period, rejected rows, missing collection or an inaccurate clock. There is no check before the first or after the last accepted entry because the file does not declare collection boundaries. Combining sources can hide a gap from one device if another fills the interval.

The reviewer passed 24 checks, including the exact minimum, ordering, midnight, a leap day, rejections and invalid settings. The sample was reproduced and tested inputs remained unchanged.

Save reports under a new filename: redirecting output over an input can empty it before Python opens it. Keep reports private when they reveal personal or work activity.