SFTP Monitoring and Daily Digest

Summary

Two shell scripts work together to monitor SFTP activity. sftp-monitor.sh runs persistently, tailing /var/log/auth.log for events from the sftp-feeds group, and posts real-time Slack alerts to #sftp-monitoring (channel ID C0APAF806F8). It maintains a daily JSON counter at /var/lib/sftp-monitor/daily.json. sftp-daily-digest.sh runs via cron at end-of-day, posts a summary table of the day’s activity, and resets the daily log.

State Diagram

stateDiagram-v2
    state "Surveillance active" as Watching
    state "Connexion réussie détectée" as SuccessDetected
    state "Échec de connexion détecté" as FailureDetected
    state "Upload détecté" as UploadDetected
    state "Vérification partenaire (succès)" as IsPartner
    state "Alerte Slack envoyée" as SlackAlert
    state "Événement ignoré" as Ignored
    state "Vérification partenaire (échec)" as IsPartnerFail
    state "Comptage silencieux" as SilentCount
    state "Digest quotidien envoyé" as DigestPosted
    state "Journal réinitialisé" as DailyLogReset
    [*] --> Watching: tail -n 0 -F /var/log/auth.log
    Watching --> SuccessDetected: auth.log shows accepted password
    Watching --> FailureDetected: auth.log shows failed password
    Watching --> UploadDetected: auth.log shows SFTP write close
    SuccessDetected --> IsPartner: check if user is in sftp-feeds group
    IsPartner --> SlackAlert: post connected alert and update daily.json
    IsPartner --> Ignored: not in sftp-feeds group
    FailureDetected --> IsPartnerFail: check sftp-feeds group
    IsPartnerFail --> SlackAlert: post "Connection failure" warning + update daily.json
    IsPartnerFail --> SilentCount: unknown user — count only in daily.json
    UploadDetected --> SlackAlert: post uploaded filename alert
    [*] --> DigestPosted: cron runs sftp-daily-digest.sh
    DigestPosted --> DailyLogReset: rm daily.json

Steps

1. Real-Time Monitor Setup (Actor: system)

sftp-monitor.sh starts by:

1. Checking that internal-sftp -l INFO is configured in sshd_config (upload detection requires this). Posts a Slack warning if missing.
2. Initializing the daily log JSON: { date, partners: {}, unknown_attempts: 0 }.
3. Starting tail -n 0 -F /var/log/auth.log — processes only new lines, follows log rotation.

Environment: Requires SLACK_BOT_TOKEN env var. Channel hardcoded to C0APAF806F8.

2. Successful Connection Event (Actor: system)

Pattern matched: Accepted password for {user} from {ip}.

is_sftp_user() uses id -nG to check if the user belongs to sftp-feeds group.

If a partner user:

• Posts Slack: "✅ *{Partner}* s'est connecté depuis {ip} — {date}"
• Increments daily.json partners[user].success

3. Failed Connection Event (Actor: system)

Pattern matched: Failed password for {user}.

If a partner user:

• Posts Slack: "🔴 Échec de connexion de *{Partner}* — check if password changed"
• Increments daily.json partners[user].failed

If unknown user:

• Increments daily.json unknown_attempts silently (no Slack noise)

4. File Upload Detection (Actor: system)

Pattern matched: internal-sftp.*close.*WRITE.

Extracts filename from log line. The recent user is heuristically derived from the last session opened for sftp line in auth.log (best-effort attribution).

Posts Slack: "📄 *{user}* a uploadé {filename}"

Increments daily.json partners[user].upload.

Note: Requires internal-sftp -l INFO in sshd_config. Without it, upload events are not logged.

5. Timestamp Formatting (Actor: system)

extract_ts() handles both systemd ISO format (2026-03-30T06:07:55...) and traditional syslog format (Mar 30 06:07:55). format_time() formats to French "30/03 à 06h07".

6. Daily Digest (Actor: system, tools: slack)

sftp-daily-digest.sh — intended to run via cron (e.g. 59 23 * * *):

1. Reads /var/lib/sftp-monitor/daily.json.
2. If no file or no activity: posts "Aucune activité aujourd'hui." and exits.
3. If only unknown attempts: posts count of blocked connections.
4. Otherwise: posts a Markdown table with columns Partenaire | Connexions | Échecs | Uploads | Downloads per partner, plus totals row and unknown attempts footer.
5. Deletes daily.json to reset for the next day.

Environment: Requires SLACK_BOT_TOKEN env var.

Error States

• SLACK_BOT_TOKEN not set → script exits with error
• internal-sftp -l INFO missing → upload events not detected, Slack warning posted on startup
• jq not installed → daily log operations will fail silently
• Log rotation → tail -F automatically re-follows the new log file

Related Processes

• partner-onboarding — partners must be in the sftp-feeds group for events to be monitored
• sftp-xml-ingestion — SFTP upload triggers the file-watcher pipeline after this monitoring detects the connection