Incident Response for Web Application Breaches: A Practitioner's Guide

A web application breach unfolds faster than most teams expect. By the time the alert fires, the attacker may have been in the environment for days. What you do in the first hour — and the first 24 hours — determines whether this is a contained incident or a full-scale data breach.

This guide covers web application incident response from the moment of detection through root cause analysis and hardening. It's written for security engineers and DevSecOps teams who are the first technical responders, not for executives who need the IR playbook summary.

Detection: Recognizing a Web Application Compromise

Most web application breaches are discovered through one of four signals:

  1. Security tool alert — WAF, SIEM, or IDS detects anomalous activity (SQLi attempts, unusual data exfiltration volumes, authenticated requests from unexpected geolocations)
  2. External notification — HackerOne disclosure, law enforcement notification, or data appearing in breach forums
  3. Operational anomaly — unusual server load, database query performance degradation, storage consumption spike, or unexpected process behavior
  4. User report — customers report unauthorized account access, or internal users notice unexpected behavior

The discovery-dwell time gap. The median time between initial compromise and discovery for web application breaches is measured in days to weeks, not hours. By the time you detect the breach, assume the attacker has had time to establish persistence, move laterally, and exfiltrate data. Your investigation scope should be broader than the initial alert suggests.

Phase 1: Triage (0–1 Hour)

Immediate Preservation Steps

Before doing anything else, preserve evidence. Forensic artifacts are volatile — logs rotate, memory is overwritten, container instances are replaced. The first responder's job is to stop the evidence clock:

  • Do not reboot or restart affected systems — memory forensics becomes impossible
  • Capture live memory if possible — especially valuable if malware or a webshell is suspected
  • Enable verbose logging immediately — if access logs, error logs, or database query logs are not being captured at sufficient detail, increase verbosity now
  • Snapshot affected systems/containers — take volume snapshots of web servers and database instances before any remediation
  • Preserve WAF and CDN logs — these are often not part of standard log retention and may have shorter TTLs
# Capture running processes on a Linux web server (before any changes)
ps auxf > /tmp/ir_processes_$(date +%Y%m%d_%H%M%S).txt
netstat -tulnp > /tmp/ir_netstat_$(date +%Y%m%d_%H%M%S).txt
ss -tulnp > /tmp/ir_ss_$(date +%Y%m%d_%H%M%S).txt
lsof -i > /tmp/ir_lsof_$(date +%Y%m%d_%H%M%S).txt

# Capture cron jobs and persistence mechanisms
crontab -l > /tmp/ir_cron_$(date +%Y%m%d_%H%M%S).txt
for user in $(cut -f1 -d: /etc/passwd); do
    echo "=== $user ===" >> /tmp/ir_allcron_$(date +%Y%m%d_%H%M%S).txt
    crontab -u $user -l 2>/dev/null >> /tmp/ir_allcron_$(date +%Y%m%d_%H%M%S).txt
done

# Check for recently modified files in web root (last 7 days)
find /var/www/ /srv/ /opt/app/ -type f -newer /tmp/reference_date \
     -not -path "*/logs/*" \
     -not -path "*/__pycache__/*" \
     2>/dev/null > /tmp/ir_modified_files_$(date +%Y%m%d_%H%M%S).txt

Initial Scope Assessment

Answer these questions in the first 30 minutes:

  • What is the suspected initial access vector? (SQLi, deserialization, upload, credential stuffing, supply chain?)
  • Which applications and systems are confirmed or suspected to be affected?
  • Is the attacker believed to be active in the environment right now?
  • What data could have been accessed or exfiltrated? (user PII, payment data, credentials, IP)
  • What are the regulatory notification timelines? (GDPR: 72 hours, PCI DSS: varies by card scheme)

Phase 2: Containment (1–4 Hours)

Active vs. Passive Containment

Before containing, decide: do you want the attacker to know they've been detected? In some cases, keeping the attacker active (in a monitored environment) provides intelligence on their TTPs and objective. In most cases — especially when data exfiltration is ongoing — immediate containment is the right call.

Web Application Containment Options

Containment Action Speed Impact When to Use
WAF block rule for attacker IP/ASN Minutes Low Active attacker with known IP; first response while investigating
Disable compromised user accounts Minutes Low-medium Account compromise confirmed; may not stop session-based access
Invalidate all sessions (force re-login) Minutes Medium (user disruption) Session hijacking suspected; attacker using stolen session tokens
Put application in maintenance mode Minutes High (outage) Active exploitation in progress; data loss risk outweighs downtime cost
Isolate affected containers/VMs Minutes High Post-exploitation confirmed; lateral movement risk
Rotate all secrets and credentials Hours High Credential exfiltration suspected; required before any recovery step

Webshell and Backdoor Identification

Attackers frequently drop webshells for persistent access. Look for them before attempting recovery:

# Search for common webshell indicators in web root
# PHP webshells
grep -r --include="*.php" -l "eval(base64_decode\|system(\$_GET\|exec(\$_POST\|passthru\|shell_exec" /var/www/ 2>/dev/null

# Look for recently created/modified PHP files
find /var/www/ -name "*.php" -newer /tmp/reference_7days_ago -type f 2>/dev/null

# Check for hidden files or files with suspicious names
find /var/www/ -name ".*" -o -name "*.php5" -o -name "*.phtml" 2>/dev/null

# Look for files with executable permissions in upload directories
find /var/www/uploads/ /var/www/media/ -type f -executable 2>/dev/null

# Check for PHP files masquerading as images
file /var/www/uploads/*.jpg 2>/dev/null | grep -i "PHP\|script\|text"

# Python/Node.js webshells
grep -r --include="*.py" -l "subprocess.call\|os.system\|exec(" /opt/app/ 2>/dev/null | \
    xargs grep -l "request\|POST\|GET" 2>/dev/null

Session and Credential Containment

# Force all sessions to expire (varies by framework)

# Django: flush all sessions from DB
python manage.py shell -c "from django.contrib.sessions.models import Session; Session.objects.all().delete()"

# Rails: if using ActiveRecord session store
rails runner "ActiveRecord::SessionStore::Session.delete_all"

# Express.js with Redis session store
redis-cli KEYS "sess:*" | xargs redis-cli DEL

# Check for active database connections from unexpected hosts
# MySQL
SELECT user, host, db, command, time, info
FROM information_schema.processlist
WHERE user != 'system user'
ORDER BY time DESC;

# PostgreSQL
SELECT pid, usename, application_name, client_addr, state, query, query_start
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY query_start;

Phase 3: Forensic Investigation

Web Server Log Analysis

Web server access logs are your primary evidence source. Know what you're looking for:

# Find the initial exploitation request
# Look for the first time a specific payload or path appeared

# Search for common SQLi indicators
grep -E "(?i)(union.*select|select.*from|insert.*into|drop.*table|xp_cmdshell|information_schema)" \
     /var/log/nginx/access.log | sort -t'"' -k2 | head -50

# Search for path traversal and LFI
grep -E "\.\./|%2e%2e%2f|%252e%252e|etc/passwd|etc/shadow" \
     /var/log/nginx/access.log

# Look for webshell access patterns
grep -E "\.(php|aspx|jsp|cfm)\?" /var/log/nginx/access.log | \
    grep -E "cmd=|exec=|system=|command=|c=" | sort | uniq

# Find large data transfers (potential exfiltration)
awk '{print $10, $7}' /var/log/nginx/access.log | \
    sort -rn | head -50

# Reconstruct attacker session from IP
attacker_ip="198.51.100.42"
grep "$attacker_ip" /var/log/nginx/access.log | \
    awk '{print $4, $6, $7, $9, $10}' | sort

Database Query Log Analysis

# MySQL: enable general query log (if not already enabled) for real-time monitoring
SET GLOBAL general_log = 'ON';
SET GLOBAL general_log_file = '/var/log/mysql/general.log';

# Review slow query log for anomalous queries
tail -n 500 /var/log/mysql/slow.log | grep -E "SELECT|INSERT|DELETE|UPDATE" | \
    grep -v "WHERE id = \|WHERE user_id = " | head -50

# PostgreSQL: query pg_log for suspicious patterns
grep -E "UNION|information_schema|pg_catalog|COPY|lo_import" \
     /var/log/postgresql/postgresql-*.log | tail -100

# Check for data exfiltration via SELECT INTO OUTFILE or COPY TO
grep -E "INTO OUTFILE|COPY.*TO|load_file|lo_export" \
     /var/log/postgresql/postgresql-*.log

Establishing the Attack Timeline

Build a chronological timeline of attacker activity. Key events to include:

  1. First observed reconnaissance — scanning, path enumeration, error triggering
  2. Initial exploitation — the specific request that triggered the vulnerability
  3. Privilege escalation or lateral movement — if the attacker moved beyond the initial foothold
  4. Data access events — which records were queried, when, and how much data was accessed
  5. Exfiltration — unusual outbound transfers, DNS queries, or API calls to external services
  6. Persistence mechanisms installed — webshells, cron jobs, new user accounts
  7. Last observed activity — when the attacker was last active before detection

Timeline accuracy matters legally. If this incident results in regulatory notification or litigation, your timeline becomes evidence. Correlate log timestamps across multiple systems (web server, database, OS) and account for timezone differences. Many systems log in UTC while others use local time — document this explicitly.

Phase 4: Root Cause Analysis

Vulnerability Identification

Identify the specific vulnerability that was exploited. This is not optional — if you don't know how they got in, you can't be confident they're out. Common root causes:

Attack Vector Evidence to Look For Validation Step
SQL injection SQLi payloads in access logs, slow queries, DB errors Reproduce in isolated environment; verify affected parameters
Credential stuffing High login failure rate from distributed IPs; then successful login Check if compromised account credentials appear in known breach datasets
File upload vulnerability POST to upload endpoint followed by GET to uploaded file executing code Review file type validation code; check MIME type handling
Deserialization Unusual Java/PHP error traces; encoded object payloads in requests Review deserialization points; check gadget chain applicability
Supply chain / dependency Malicious code in npm/pip/maven package; execution during build or startup Review package lockfile changes; compare installed vs. expected versions
SSRF leading to credential access Requests to internal metadata endpoints (169.254.169.254) Trace which application parameter triggered the outbound request

Phase 5: Eradication and Recovery

Eradication Checklist

Do not proceed to recovery until every item on this list is confirmed:

  • All webshells and malicious files identified and removed
  • All backdoor user accounts removed
  • All malicious cron jobs and startup scripts removed
  • All secrets and credentials rotated (database passwords, API keys, session secrets, OAuth client secrets)
  • The exploited vulnerability is patched or mitigated
  • Attacker-controlled C2 endpoints blocked at the network perimeter
  • All sessions invalidated; users re-authenticated
  • Logs preserved in tamper-evident storage before any system changes
# Rotate secrets systematically
# Generate new application secrets
python3 -c "import secrets; print(secrets.token_hex(32))"

# Rotate database user passwords
# MySQL
ALTER USER 'app_user'@'%' IDENTIFIED BY 'new_strong_password_here';
FLUSH PRIVILEGES;

# Revoke and regenerate all OAuth client secrets
# This varies by IdP — check your identity provider's API

# Check for hardcoded secrets in codebase (before deploying patched version)
git log --all --full-history -- "*.env"
grep -r "password\s*=" --include="*.py" --include="*.js" --include="*.rb" ./ | \
    grep -v "test\|spec\|mock" | grep -v ".env.example"

Clean Rebuild vs. In-Place Remediation

For containerized applications: rebuild from a known-good base image rather than attempting to clean an infected container. This eliminates the risk of missed artifacts. Only in-place remediate when:

  • The blast radius was limited to a single known file
  • You have high confidence in your webshell/malware detection
  • The rebuild would take longer than the business can tolerate

For VMs and bare-metal servers: the bar for confidence in in-place remediation is even higher. When in doubt, rebuild from a known-good snapshot that predates the estimated compromise date.

Phase 6: Post-Incident Actions

Notification Decisions

Regulatory notification timelines begin from when you knew (or should have known) about the breach, not from when your investigation is complete:

  • GDPR: 72 hours to notify the supervisory authority if the breach is likely to result in a risk to individuals. Notify affected individuals without undue delay if high risk.
  • PCI DSS: Contact your acquiring bank and card brands immediately upon confirmation of card data compromise.
  • US state breach laws: Most require notification to affected individuals within 30-90 days; some require notification to the state AG.
  • SEC disclosure (public companies): Material cybersecurity incidents must be disclosed on Form 8-K within 4 business days of determining materiality.

Hardening Priorities

After the incident, prioritize hardening based on what the attacker's path revealed about your control gaps. The most valuable insight from any incident is not "here's what broke" but "here's what would have stopped it earlier."

  • Which detection would have caught this earlier? (Instrument and alert on it)
  • Which preventive control was missing or misconfigured?
  • What would have limited the blast radius if the initial access had been prevented?
  • Where did log coverage fail the investigation?

The Post-Incident Review

Run a blameless post-incident review within 2 weeks while memories are fresh. The output should be:

  1. A finalized incident timeline
  2. Root cause(s) identified — both the technical vulnerability and the process failure that allowed it
  3. A prioritized remediation list with owners and due dates
  4. Lessons learned that improve the IR playbook itself

Document the findings in a format that can be shared with legal counsel under privilege if there is any risk of regulatory action or litigation.

Find Vulnerabilities Before Attackers Do

The best incident response is the one you never need. Ironimo runs continuous security scans using the same tools attackers use — finding SQL injection, authentication bypasses, and configuration exposures before they become breaches. Run your first scan before your next IR call.

Start free scan
← Back to blog