Available at https://github.com/thecivvie/PKT-Reporter-for-Fidget-FTN-Systems

# pkt_report / pkt_indexer

A small, practical toolkit for indexing, analysing, and reporting
FidoNet-style packet and echomail traffic using SQLite.

Built for sysops who like clear stats, tidy reports, and scripts that
don’t get in the way.

Developed and used in real-world BBS / FTN environments.

-----------------------------------------------------------------------

## 📦 What’s Included

This repository contains three core tools that work together:

### pkt_indexer.py
Indexes FTN packets and messages into a SQLite database.

What it does:
- Parses incoming packet/message data
- Stores per-message metadata
- Tracks areas, posters, dates, sizes, and message characteristics

Data captured (depending on configuration):
- Area name
- Origin / poster
- Message date
- Message size (bytes)
- Line count (excluding kludges)
- Quoted percentage

This script is normally run periodically or via cron as packets arrive.

-----------------------------------------------------------------------

### pkt_db_check.py
Validates and prepares the SQLite database schema.

What it does:
- Checks the database exists and is accessible
- Verifies required tables and columns
- Creates missing tables/columns if needed
- Ensures schema compatibility with newer report features

This allows the reporter to evolve without breaking existing databases.

-----------------------------------------------------------------------

### pkt_report.py
Generates human-readable traffic reports from the indexed data.

Supported reports:
- Daily traffic tables by area
- Totals and per-day breakdowns
- TOP reports (posters, areas, message sizes)

Notable features:
- Clean day-based headers (robust across month changes)
- Optional filtering by area or zone
- TOP reports including:
- Total messages
- Total bytes posted
- Average message size
- Largest individual messages

Output is designed for terminal, email, or BBS posting.

-----------------------------------------------------------------------

## 🛠 Requirements

- Python 3.8+
- SQLite3 (standard library)
- No third-party dependencies

Tested on:
- Linux (including Raspberry Pi)
- macOS

-----------------------------------------------------------------------

## 🚀 Typical Workflow

1. Index packets as they arrive:

python3 pkt_indexer.py --folder /path/to/pkts --db pkt_index.db --delete

2. Verify / inspect the database (safe to re-run):

python3 pkt_db_check.py --db pkt_index.db

3. Generate reports:

python3 pkt_report.py --db pkt_index.db --date WEEK

-----------------------------------------------------------------------

## 🧰 Usage / CLI

### pkt_indexer.py
Indexes *.pkt files into SQLite.

Common options:
- --folder, -f Folder containing .pkt files (default: current directory)
- --db, -d SQLite database path (default: pkt_index.db)
- --recursive Scan subfolders for .pkt
- --delete Delete .pkt files after successful import
- --test Parse and display data without writing to DB

Examples:

Index a spool directory recursively:
python3 pkt_indexer.py -f /home/fmail/ftn/inbound --recursive -d pkt_index.db

Test parsing without DB writes:
python3 pkt_indexer.py -f /home/fmail/ftn/inbound --test

Import and delete packets after success:
python3 pkt_indexer.py -f /home/fmail/ftn/inbound --recursive --delete

-----------------------------------------------------------------------

### pkt_db_check.py
Database inspector / sanity checker.

Common options:
- --db, -d SQLite database path (default: pkt_index.db)
- --limit, -n Number of sample rows to display (default: 20)

Example:

python3 pkt_db_check.py -d pkt_index.db -n 50

-----------------------------------------------------------------------

### pkt_report.py
Generates traffic summary tables and TOP reports.

Date range options (pick one):
- --date WEEK | MONTH | CMONTH
WEEK = last 7 days
MONTH = previous calendar month
CMONTH = current calendar month so far
- --days N
- --from YYYY-MM-DD --to YYYY-MM-DD

Report shaping:
- --period auto | month | day
- --area-width N

Area inclusion / exclusion:
- --known-areas FILE
- --only-areas FILE
- --exclude-areas FILE

Files may be .txt (one area per line, # comments allowed) or .json.

TOP mode:
- --top ECHO
Shows top posters and message-size statistics (if size data exists)

Examples:

Last 7 days:
python3 pkt_report.py -d pkt_index.db --date WEEK

Last 30 days:
python3 pkt_report.py -d pkt_index.db --days 30

Explicit date range:
python3 pkt_report.py -d pkt_index.db --from 2026-01-26 --to 2026-02-01

Include known areas:
python3 pkt_report.py -d pkt_index.db --date WEEK --known-areas known.txt

Only selected areas:
python3 pkt_report.py -d pkt_index.db --date WEEK --only-areas areas.json

Exclude areas:
python3 pkt_report.py -d pkt_index.db --date WEEK --exclude-areas exclude.txt

TOP report for one echo:
python3 pkt_report.py -d pkt_index.db --date WEEK --top MIN_CHAT

-----------------------------------------------------------------------

## 📊 Example Output

Statistics from 26-Jan-26 to 01-Feb-26

Area 26 27 28 29 30 31 01 Total ============================================================================ MIN_CHAT 4 6 3 9 2 1 5 : 30 MIN_WEATHER 1 0 2 1 0 3 2 : 9 ============================================================================ TOTALS 5 6 5 10 2 4 7 : 39

TOP message size example:

=== TOP POSTERS BY TOTAL MESSAGE SIZE ===
Poster Messages Total KB Avg KB Max KB ------------------------------------------------------
Sean 42 418.2 9.9 22.1
MurphyBot 18 146.5 8.1 18.6

-----------------------------------------------------------------------

## 🧠 Design Notes

- Internal date keys are used for correctness
- Display formatting prioritises readability
- Schema changes are handled defensively
- Scripts are intentionally simple and hackable

-----------------------------------------------------------------------

## 📜 License

MIT License

Copyright (c) 2026 Sean Rima and Murphy

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal in the Software without restriction.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND.

-----------------------------------------------------------------------

## ☕ Final Words

If you’re running an FTN-style system in 2026 and still care about good stats,
this is for you.

Pull requests welcome. Hacks encouraged. Coffee optional but recommended.

--- FMail-lnx 2.3.2.6-B20251227
* Origin: TCOB1 A Mail Only System (86:553/20)

Available at https://github.com/thecivvie/PKT-Reporter-for-Fidget-FTN-Systems

# pkt_report / pkt_indexer

A small, practical toolkit for indexing, analysing, and reporting FidoNet-style packet and echomail traffic using SQLite.

Built for sysops who like clear stats, tidy reports, and scripts that
don't get in the way.

Developed and used in real-world BBS / FTN environments.

Thanks! I will pull that down later today and give it a look.

Mike

* SLMR 2.1a * "­Beavis! ¨Donde esta su hall pass?"
--- SBBSecho 3.28-Linux
* Origin: Capitol City Online (86:100/18)

Hello Mike!

01 Feb 26 09:53, you wrote to me:

Available at
https://github.com/thecivvie/PKT-Reporter-for-Fidget-FTN-Systems

# pkt_report / pkt_indexer

A small, practical toolkit for indexing, analysing, and reporting
FidoNet-style packet and echomail traffic using SQLite.

Built for sysops who like clear stats, tidy reports, and scripts
that don't get in the way.

Developed and used in real-world BBS / FTN environments.

Thanks! I will pull that down later today and give it a look.

I used the packet format as it is easier than trying to add all mail processors. I save my packets for it in a filebox, as it is easier for multi network systems

Sean


... TCOB1: binkd only binkd.rima.ie
--- FMail-lnx 2.3.2.6-B20251227
* Origin: TCOB1 A Mail Only System (86:553/20)

Sean Rima wrote to Mike Powell <=-

I used the packet format as it is easier than trying to add all mail processors. I save my packets for it in a filebox, as it is easier for multi network systems

I know of several utilities that do that. It's the easiest way to do it in my opinion.

-- Sean

... A sysop and his money are soon parted.
--- MultiMail/Linux
* Origin: Outpost BBS * Johnson City, TN (86:100/1)

Hello Sean!

01 Feb 26 11:19, you wrote to me:

Sean Rima wrote to Mike Powell <=-

I used the packet format as it is easier than trying to add all
mail processors. I save my packets for it in a filebox, as it is
easier for multi network systems

I know of several utilities that do that. It's the easiest way to do
it in my opinion.

Yeah it is a lot easier and also makes debugging easier as well

Sean


... TCOB1: binkd only binkd.rima.ie
--- FMail-lnx 2.3.2.6-B20251227
* Origin: TCOB1 A Mail Only System (86:553/20)

Hello Sean,

01 Feb 26 16:32, you wrote to me:

Yeah it is a lot easier and also makes debugging easier as well

Better than having to learn several message base formats.

-- Sean

... Wolfgang's Third Law: It can't work.
--- GoldED+/LNX 1.1.5-b20240209
* Origin: Outpost BBS * Johnson City, TN (86:10/1)

Hello Sean!

01 Feb 26 14:09, you wrote to me:

Yeah it is a lot easier and also makes debugging easier as well

Better than having to learn several message base formats.

That's a fact :)

Sean


... TCOB1: binkd only binkd.rima.ie
--- FMail-lnx 2.3.2.6-B20251227
* Origin: TCOB1 A Mail Only System (86:553/20)

Re: PKT Reporter released
By: Sean Rima to All on Sun Feb 01 2026 01:34 pm

Available at https://github.com/thecivvie/PKT-Reporter-for-Fidget-FTN-Systems
# pkt_report / pkt_indexer
A small, practical toolkit for indexing, analysing, and reporting FidoNet-style packet and echomail traffic using SQLite.
Built for sysops who like clear stats, tidy reports, and scripts that don't get in the way.

Very cool! I have grabbed it and am going to look at how best to integrate it in to the mail processing here.

Hello The!

01 Feb 26 13:01, you wrote to me:

Re: PKT Reporter released
By: Sean Rima to All on Sun Feb 01 2026 01:34 pm

Very cool! I have grabbed it and am going to look at how best to
integrate it in to the mail processing here.

Excellent
Sean


... TCOB1: binkd only binkd.rima.ie
--- FMail-lnx 2.3.2.6-B20251227
* Origin: TCOB1 A Mail Only System (86:553/20)

Thanks! I will pull that down later today and give it a look.

I used the packet format as it is easier than trying to add all mail processors. I save my packets for it in a filebox, as it is easier for multi network systems

Does that mean it won't work on compressed packets, i.e. *.MO0, etc.?

Mike

* SLMR 2.1a * Life after death - is that Terminate and Stay Resident?
--- SBBSecho 3.28-Linux
* Origin: Capitol City Online (86:100/18)

Hello Mike!

02 Feb 26 10:03, you wrote to me:

Thanks! I will pull that down later today and give it a look.

I used the packet format as it is easier than trying to add all mail
processors. I save my packets for it in a filebox, as it is easier
for multi network systems

Does that mean it won't work on compressed packets, i.e. *.MO0, etc.?

No, not sure I would add that. I tend to put the packets for it into a filebox

Sean


... TCOB1: binkd only binkd.rima.ie
--- FMail-lnx 2.3.2.6-B20251227
* Origin: TCOB1 A Mail Only System (86:553/20)

Thanks! I will pull that down later today and give it a look.

I used the packet format as it is easier than trying to add all mail
processors. I save my packets for it in a filebox, as it is easier
for multi network systems

Does that mean it won't work on compressed packets, i.e. *.MO0, etc.?

No, not sure I would add that. I tend to put the packets for it into a filebox

Thanks for the reply. I got it mixed up with the BSO reporter. That is
the one I am planning to try out this week. ;)

Mike


* SLMR 2.1a * Keep your stick on the ice
--- SBBSecho 3.28-Linux
* Origin: Capitol City Online (86:100/18)

Hello Mike!

03 Feb 26 08:58, you wrote to me:

Thanks! I will pull that down later today and give it a

look.

I used the packet format as it is easier than trying to add all

mail

processors. I save my packets for it in a filebox, as it is

easier

for multi network systems

Does that mean it won't work on compressed packets, i.e. *.MO0,

etc.?

No, not sure I would add that. I tend to put the packets for it into
a filebox

Thanks for the reply. I got it mixed up with the BSO reporter. That
is the one I am planning to try out this week. ;)

Ah ok. I have to fix a small bug in the bso script. it is not reading tic attached files properly

Sean


... TCOB1: binkd only binkd.rima.ie
--- FMail-lnx 2.3.2.6-B20251227
* Origin: TCOB1 A Mail Only System (86:553/20)