Automated Torrent Detection & Progressive Enforcement on ocserv VPN Infrastructure

Acceptable-Use Enforcement at the Tunnel Layer — Real-time P2P detection on a multi-tenant ocserv fleet, with progressive enforcement: immediate egress block, then session termination on a three-strike threshold.

Project Overview

Project Summary

Shared VPN infrastructure has a specific and expensive failure mode: one user running BitTorrent degrades the service for everyone on that egress IP. Bandwidth saturation is the visible symptom. The real cost is reputational and legal — copyright notices, IP blacklisting, and upstream provider action against the egress address that every other tenant on that server is also using.

I built an automated enforcement system that detects peer-to-peer traffic on live ocserv tunnels, attributes it to a specific authenticated session, and applies progressive enforcement: an immediate timed egress block on first detection, escalating to session termination after three consecutive violations. The entire system — detection engine, enforcement logic, systemd units, log rotation, and state cleanup — deploys from a single idempotent installer that is safe to re-run on a live server.

Confidentiality note: The implementation is proprietary to Kolpolok Limited and is not published. This write-up covers the detection strategy, enforcement architecture, and engineering trade-offs. Code blocks below are logic descriptions, not source.

The Problem

Manual response was the actual bottleneck. By the time a bandwidth spike was noticed on a graph, the traffic had been running for hours, and there was no reliable way to tie a spike on the node’s uplink back to a specific authenticated VPN user. Detection without attribution is not enforcement.

Architecture

The system runs as a persistent supervisor process that discovers active tunnel interfaces and spawns an independent monitor per interface — so a busy tunnel never starves a quiet one, and one crashed monitor never takes down enforcement fleet-wide.

          ocserv TORRENT ENFORCEMENT — CONTROL FLOW

  ┌────────────────────────────────────────────────────────────┐
  │  SUPERVISOR LOOP  (systemd service, Restart=on-failure)    │
  │  · enumerate live per-user tunnel interfaces               │
  │  · spawn one monitor per interface, track by PID           │
  │  · reap and respawn dead monitors                          │
  └───────────────────────────┬────────────────────────────────┘
                              │  one child per tunnel
          ┌───────────────────┴───────────────────┐
          ▼                                       ▼
  ┌───────────────────┐                   ┌───────────────────┐
  │ MONITOR: tunnel A │                   │ MONITOR: tunnel B │
  │ 1 packet capture  │                   │ 1 packet capture  │
  │ 2 signature match │                   │ 2 signature match │
  │ 3 attribute IP    │                   │ 3 attribute IP    │
  │ 4 map → session   │                   │ 4 map → session   │
  │ 5 enforce         │                   │ 5 enforce         │
  └─────────┬─────────┘                   └─────────┬─────────┘
            │                                       │
            └──────────────┬────────────────────────┘
                           ▼
              ┌────────────────────────────┐
              │  ENFORCEMENT LAYER         │
              │  ipset (timed membership)  │
              │  iptables FORWARD drop     │
              │  conntrack flush           │
              │  occtl session disconnect  │
              └────────────────────────────┘
                           │
                           ▼
              ┌────────────────────────────┐
              │  RECONCILIATION (timer)    │
              │  hourly idempotent cleanup │
              │  of rules and set state    │
              └────────────────────────────┘

Detection Strategy — Layered, Not Single-Signal

Port-based detection alone is trivially defeated; payload inspection alone misses obfuscated clients. The detector uses a layered signature approach with fallback, evaluated inside a short bounded capture window:

Layer 1 — Protocol signature matching. Bounded packet capture on the tunnel interface, matched against BitTorrent wire-protocol and DHT indicators: the protocol handshake string, tracker/peer exchange fields, DHT query verbs, and metadata-extension markers.

Layer 2 — Port heuristics fallback. If no payload signature matches, the capture falls back to well-known P2P port ranges — the classic BitTorrent range, common alternate client ports, and legacy P2P network ports.

Bounded capture, not continuous. Each cycle captures for a fixed short interval with a hard packet cap, then releases. This is a deliberate resource decision: continuous full capture on every tunnel of a busy node is unaffordable, and a bounded rolling sample catches sustained P2P activity reliably while a short-lived probe would not.

# LOGIC ONLY — implementation withheld (proprietary, Kolpolok Limited)

FOR each live tunnel interface, in its own process:
    LOOP:
        sample ← bounded_capture(interface, duration=short, packet_cap=fixed)

        IF protocol_signature_match(sample) OR p2p_port_match(sample):
            offender_ip ← extract_tunnel_client_address(sample)
            IF offender_ip is not attributable: CONTINUE

            session_id ← map_tunnel_ip_to_authenticated_session(offender_ip)
            IF session_id is null: CONTINUE          # stale or already gone

            strikes[offender_ip] += 1

            IF strikes[offender_ip] >= STRIKE_LIMIT:
                terminate_session(session_id)
                clear strikes[offender_ip]
                EXIT this monitor          # supervisor respawns on reconnect
            ELSE:
                apply_timed_egress_block(offender_ip)
                sleep(BLOCK_DURATION)
        sleep(short_interval)

Attribution — The Part That Makes It Enforcement

Detecting P2P traffic is straightforward. Tying it to a person you can act on is the engineering problem.

The detector extracts the tunnel-side client address from the captured traffic, then queries the ocserv control interface to resolve that address to an authenticated session ID, filtering for sessions in a connected state. Only a session that is currently connected and positively attributed is eligible for enforcement.

This ordering matters for a reason that only shows up in production: session IDs and tunnel addresses are recycled. Enforcing against a stale mapping means disconnecting an innocent user who happened to inherit the address of a previous offender. The detector validates connection state at the moment of enforcement, and abandons the cycle rather than acting on an unresolved mapping. A false negative costs bandwidth; a false positive costs a customer.

Progressive Enforcement — Three Layers

Enforcement escalates rather than jumping straight to disconnection, because most P2P activity on a corporate VPN is a user who does not realise their client is running in the background.

Level 1 — Immediate timed egress block

The offending tunnel address is added to a kernel ipset with a built-in membership timeout, and forwarding rules drop both source-matched and destination-matched traffic for members of that set.

Using an ipset with a timeout rather than per-IP iptables rules is a specific decision: the kernel expires the entry itself, so the block self-heals with no scheduled job required to lift it, and rule count stays constant regardless of how many offenders are blocked concurrently. Adding an IP is an O(1) set operation, not a rule-table insertion.

Level 2 — Existing connection teardown

A drop rule stops new flows. Established BitTorrent connections continue happily inside the kernel connection-tracking table. So enforcement also:

• drops traffic matching established and related connection states for set members
• issues explicit rejects for the offending address in both directions
• flushes the connection-tracking entries for that address

Without the conntrack flush, the “block” is cosmetic for several minutes — the single most important detail in the whole enforcement path, and the one that took production traffic to discover.

Level 3 — Session termination at threshold

On the third consecutive detection, the authenticated session is terminated through the ocserv control interface, with the violation recorded. The strike counter is per-address and resets on clean cycles, so a single accidental detection never escalates to disconnection.

Deployment Automation

The whole system ships as one idempotent installer that is safe to re-run against a live production server — a hard requirement when the fleet spans multiple regions and configuration drift is the enemy.

The installer:

• validates it is running with the required privileges before touching anything
• installs its own dependency set (packet capture, ipset, conntrack, iptables tooling)
• writes the detection engine and the reconciliation script to standard system paths
• generates the systemd service unit for the detector, with automatic restart on failure
• generates a systemd timer plus companion unit for scheduled state reconciliation
• provisions log files and the lock file used for write serialisation, with correct permissions
• reloads the systemd daemon and enables both units
• prints the operator runbook — status, start, stop, restart, log tail — as its final output

Re-running it converges the server to the intended state rather than duplicating anything. Idempotency is what makes fleet-wide deployment a loop instead of a project.

Engineering Details Worth Naming

Testing & Verification

• Controlled P2P client run inside a test tunnel — verified detection, attribution, block application, and connection teardown within a single detection cycle
• Verified established-flow teardown by confirming transfer stops, not just that new connections fail
• Strike escalation validated end to end: two blocks, then session termination on the third detection
• Installer re-run repeatedly against a live node — confirmed convergence with no duplicated units, rules, or log entries
• Reconciliation timer validated: rules and set state fully removed, script safe to run when nothing exists to clean
• Concurrency test with multiple simultaneous tunnels — confirmed independent monitors and clean, serialised log output
• Negative testing: legitimate high-bandwidth traffic (large downloads, video streaming, backup sync) confirmed not to trigger enforcement

Outcomes

Honest Limitations & Next Iteration

Publishing a system without its limitations is marketing, not engineering. Known gaps and the planned path:

• Encrypted / obfuscated P2P. Clients using Message Stream Encryption defeat payload-signature matching, leaving only the port heuristic. The correct fix is behavioural flow analysis — peer fan-out, connection churn rate, and upload/download symmetry are extremely distinctive for P2P and survive payload encryption entirely. Flow-based detection (nDPI, IPFIX/NetFlow, or eBPF-based flow accounting) is the next iteration.
• Sampling gaps. Bounded rolling capture trades completeness for cost. Sustained P2P is caught reliably; a very short burst can fall between windows. Flow accounting closes this too, since it observes every flow rather than sampling packets.
• Centralised state. Strike counters live in monitor process memory, so a supervisor restart resets them. Moving strike state to a shared store would give fleet-wide, restart-surviving reputation per user rather than per node.
• No user feedback loop. Currently a blocked user experiences an unexplained outage. A captive-portal notification or dashboard message explaining why access was restricted would convert most first-strike users into compliant ones without any further enforcement.