Incident Response for Web Application Breaches: A Practitioner's Guide
A web application breach unfolds faster than most teams expect. By the time the alert fires, the attacker may have been in the environment for days. What you do in the first hour — and the first 24 hours — determines whether this is a contained incident or a full-scale data breach.
This guide covers web application incident response from the moment of detection through root cause analysis and hardening. It's written for security engineers and DevSecOps teams who are the first technical responders, not for executives who need the IR playbook summary.
Detection: Recognizing a Web Application Compromise
Most web application breaches are discovered through one of four signals:
- Security tool alert — WAF, SIEM, or IDS detects anomalous activity (SQLi attempts, unusual data exfiltration volumes, authenticated requests from unexpected geolocations)
- External notification — HackerOne disclosure, law enforcement notification, or data appearing in breach forums
- Operational anomaly — unusual server load, database query performance degradation, storage consumption spike, or unexpected process behavior
- User report — customers report unauthorized account access, or internal users notice unexpected behavior
The discovery-dwell time gap. The median time between initial compromise and discovery for web application breaches is measured in days to weeks, not hours. By the time you detect the breach, assume the attacker has had time to establish persistence, move laterally, and exfiltrate data. Your investigation scope should be broader than the initial alert suggests.
Phase 1: Triage (0–1 Hour)
Immediate Preservation Steps
Before doing anything else, preserve evidence. Forensic artifacts are volatile — logs rotate, memory is overwritten, container instances are replaced. The first responder's job is to stop the evidence clock:
- Do not reboot or restart affected systems — memory forensics becomes impossible
- Capture live memory if possible — especially valuable if malware or a webshell is suspected
- Enable verbose logging immediately — if access logs, error logs, or database query logs are not being captured at sufficient detail, increase verbosity now
- Snapshot affected systems/containers — take volume snapshots of web servers and database instances before any remediation
- Preserve WAF and CDN logs — these are often not part of standard log retention and may have shorter TTLs
# Capture running processes on a Linux web server (before any changes)
ps auxf > /tmp/ir_processes_$(date +%Y%m%d_%H%M%S).txt
netstat -tulnp > /tmp/ir_netstat_$(date +%Y%m%d_%H%M%S).txt
ss -tulnp > /tmp/ir_ss_$(date +%Y%m%d_%H%M%S).txt
lsof -i > /tmp/ir_lsof_$(date +%Y%m%d_%H%M%S).txt
# Capture cron jobs and persistence mechanisms
crontab -l > /tmp/ir_cron_$(date +%Y%m%d_%H%M%S).txt
for user in $(cut -f1 -d: /etc/passwd); do
echo "=== $user ===" >> /tmp/ir_allcron_$(date +%Y%m%d_%H%M%S).txt
crontab -u $user -l 2>/dev/null >> /tmp/ir_allcron_$(date +%Y%m%d_%H%M%S).txt
done
# Check for recently modified files in web root (last 7 days)
find /var/www/ /srv/ /opt/app/ -type f -newer /tmp/reference_date \
-not -path "*/logs/*" \
-not -path "*/__pycache__/*" \
2>/dev/null > /tmp/ir_modified_files_$(date +%Y%m%d_%H%M%S).txt
Initial Scope Assessment
Answer these questions in the first 30 minutes:
- What is the suspected initial access vector? (SQLi, deserialization, upload, credential stuffing, supply chain?)
- Which applications and systems are confirmed or suspected to be affected?
- Is the attacker believed to be active in the environment right now?
- What data could have been accessed or exfiltrated? (user PII, payment data, credentials, IP)
- What are the regulatory notification timelines? (GDPR: 72 hours, PCI DSS: varies by card scheme)
Phase 2: Containment (1–4 Hours)
Active vs. Passive Containment
Before containing, decide: do you want the attacker to know they've been detected? In some cases, keeping the attacker active (in a monitored environment) provides intelligence on their TTPs and objective. In most cases — especially when data exfiltration is ongoing — immediate containment is the right call.
Web Application Containment Options
| Containment Action | Speed | Impact | When to Use |
|---|---|---|---|
| WAF block rule for attacker IP/ASN | Minutes | Low | Active attacker with known IP; first response while investigating |
| Disable compromised user accounts | Minutes | Low-medium | Account compromise confirmed; may not stop session-based access |
| Invalidate all sessions (force re-login) | Minutes | Medium (user disruption) | Session hijacking suspected; attacker using stolen session tokens |
| Put application in maintenance mode | Minutes | High (outage) | Active exploitation in progress; data loss risk outweighs downtime cost |
| Isolate affected containers/VMs | Minutes | High | Post-exploitation confirmed; lateral movement risk |
| Rotate all secrets and credentials | Hours | High | Credential exfiltration suspected; required before any recovery step |
Webshell and Backdoor Identification
Attackers frequently drop webshells for persistent access. Look for them before attempting recovery:
# Search for common webshell indicators in web root
# PHP webshells
grep -r --include="*.php" -l "eval(base64_decode\|system(\$_GET\|exec(\$_POST\|passthru\|shell_exec" /var/www/ 2>/dev/null
# Look for recently created/modified PHP files
find /var/www/ -name "*.php" -newer /tmp/reference_7days_ago -type f 2>/dev/null
# Check for hidden files or files with suspicious names
find /var/www/ -name ".*" -o -name "*.php5" -o -name "*.phtml" 2>/dev/null
# Look for files with executable permissions in upload directories
find /var/www/uploads/ /var/www/media/ -type f -executable 2>/dev/null
# Check for PHP files masquerading as images
file /var/www/uploads/*.jpg 2>/dev/null | grep -i "PHP\|script\|text"
# Python/Node.js webshells
grep -r --include="*.py" -l "subprocess.call\|os.system\|exec(" /opt/app/ 2>/dev/null | \
xargs grep -l "request\|POST\|GET" 2>/dev/null
Session and Credential Containment
# Force all sessions to expire (varies by framework)
# Django: flush all sessions from DB
python manage.py shell -c "from django.contrib.sessions.models import Session; Session.objects.all().delete()"
# Rails: if using ActiveRecord session store
rails runner "ActiveRecord::SessionStore::Session.delete_all"
# Express.js with Redis session store
redis-cli KEYS "sess:*" | xargs redis-cli DEL
# Check for active database connections from unexpected hosts
# MySQL
SELECT user, host, db, command, time, info
FROM information_schema.processlist
WHERE user != 'system user'
ORDER BY time DESC;
# PostgreSQL
SELECT pid, usename, application_name, client_addr, state, query, query_start
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY query_start;
Phase 3: Forensic Investigation
Web Server Log Analysis
Web server access logs are your primary evidence source. Know what you're looking for:
# Find the initial exploitation request
# Look for the first time a specific payload or path appeared
# Search for common SQLi indicators
grep -E "(?i)(union.*select|select.*from|insert.*into|drop.*table|xp_cmdshell|information_schema)" \
/var/log/nginx/access.log | sort -t'"' -k2 | head -50
# Search for path traversal and LFI
grep -E "\.\./|%2e%2e%2f|%252e%252e|etc/passwd|etc/shadow" \
/var/log/nginx/access.log
# Look for webshell access patterns
grep -E "\.(php|aspx|jsp|cfm)\?" /var/log/nginx/access.log | \
grep -E "cmd=|exec=|system=|command=|c=" | sort | uniq
# Find large data transfers (potential exfiltration)
awk '{print $10, $7}' /var/log/nginx/access.log | \
sort -rn | head -50
# Reconstruct attacker session from IP
attacker_ip="198.51.100.42"
grep "$attacker_ip" /var/log/nginx/access.log | \
awk '{print $4, $6, $7, $9, $10}' | sort
Database Query Log Analysis
# MySQL: enable general query log (if not already enabled) for real-time monitoring
SET GLOBAL general_log = 'ON';
SET GLOBAL general_log_file = '/var/log/mysql/general.log';
# Review slow query log for anomalous queries
tail -n 500 /var/log/mysql/slow.log | grep -E "SELECT|INSERT|DELETE|UPDATE" | \
grep -v "WHERE id = \|WHERE user_id = " | head -50
# PostgreSQL: query pg_log for suspicious patterns
grep -E "UNION|information_schema|pg_catalog|COPY|lo_import" \
/var/log/postgresql/postgresql-*.log | tail -100
# Check for data exfiltration via SELECT INTO OUTFILE or COPY TO
grep -E "INTO OUTFILE|COPY.*TO|load_file|lo_export" \
/var/log/postgresql/postgresql-*.log
Establishing the Attack Timeline
Build a chronological timeline of attacker activity. Key events to include:
- First observed reconnaissance — scanning, path enumeration, error triggering
- Initial exploitation — the specific request that triggered the vulnerability
- Privilege escalation or lateral movement — if the attacker moved beyond the initial foothold
- Data access events — which records were queried, when, and how much data was accessed
- Exfiltration — unusual outbound transfers, DNS queries, or API calls to external services
- Persistence mechanisms installed — webshells, cron jobs, new user accounts
- Last observed activity — when the attacker was last active before detection
Timeline accuracy matters legally. If this incident results in regulatory notification or litigation, your timeline becomes evidence. Correlate log timestamps across multiple systems (web server, database, OS) and account for timezone differences. Many systems log in UTC while others use local time — document this explicitly.
Phase 4: Root Cause Analysis
Vulnerability Identification
Identify the specific vulnerability that was exploited. This is not optional — if you don't know how they got in, you can't be confident they're out. Common root causes:
| Attack Vector | Evidence to Look For | Validation Step |
|---|---|---|
| SQL injection | SQLi payloads in access logs, slow queries, DB errors | Reproduce in isolated environment; verify affected parameters |
| Credential stuffing | High login failure rate from distributed IPs; then successful login | Check if compromised account credentials appear in known breach datasets |
| File upload vulnerability | POST to upload endpoint followed by GET to uploaded file executing code | Review file type validation code; check MIME type handling |
| Deserialization | Unusual Java/PHP error traces; encoded object payloads in requests | Review deserialization points; check gadget chain applicability |
| Supply chain / dependency | Malicious code in npm/pip/maven package; execution during build or startup | Review package lockfile changes; compare installed vs. expected versions |
| SSRF leading to credential access | Requests to internal metadata endpoints (169.254.169.254) | Trace which application parameter triggered the outbound request |
Phase 5: Eradication and Recovery
Eradication Checklist
Do not proceed to recovery until every item on this list is confirmed:
- All webshells and malicious files identified and removed
- All backdoor user accounts removed
- All malicious cron jobs and startup scripts removed
- All secrets and credentials rotated (database passwords, API keys, session secrets, OAuth client secrets)
- The exploited vulnerability is patched or mitigated
- Attacker-controlled C2 endpoints blocked at the network perimeter
- All sessions invalidated; users re-authenticated
- Logs preserved in tamper-evident storage before any system changes
# Rotate secrets systematically
# Generate new application secrets
python3 -c "import secrets; print(secrets.token_hex(32))"
# Rotate database user passwords
# MySQL
ALTER USER 'app_user'@'%' IDENTIFIED BY 'new_strong_password_here';
FLUSH PRIVILEGES;
# Revoke and regenerate all OAuth client secrets
# This varies by IdP — check your identity provider's API
# Check for hardcoded secrets in codebase (before deploying patched version)
git log --all --full-history -- "*.env"
grep -r "password\s*=" --include="*.py" --include="*.js" --include="*.rb" ./ | \
grep -v "test\|spec\|mock" | grep -v ".env.example"
Clean Rebuild vs. In-Place Remediation
For containerized applications: rebuild from a known-good base image rather than attempting to clean an infected container. This eliminates the risk of missed artifacts. Only in-place remediate when:
- The blast radius was limited to a single known file
- You have high confidence in your webshell/malware detection
- The rebuild would take longer than the business can tolerate
For VMs and bare-metal servers: the bar for confidence in in-place remediation is even higher. When in doubt, rebuild from a known-good snapshot that predates the estimated compromise date.
Phase 6: Post-Incident Actions
Notification Decisions
Regulatory notification timelines begin from when you knew (or should have known) about the breach, not from when your investigation is complete:
- GDPR: 72 hours to notify the supervisory authority if the breach is likely to result in a risk to individuals. Notify affected individuals without undue delay if high risk.
- PCI DSS: Contact your acquiring bank and card brands immediately upon confirmation of card data compromise.
- US state breach laws: Most require notification to affected individuals within 30-90 days; some require notification to the state AG.
- SEC disclosure (public companies): Material cybersecurity incidents must be disclosed on Form 8-K within 4 business days of determining materiality.
Hardening Priorities
After the incident, prioritize hardening based on what the attacker's path revealed about your control gaps. The most valuable insight from any incident is not "here's what broke" but "here's what would have stopped it earlier."
- Which detection would have caught this earlier? (Instrument and alert on it)
- Which preventive control was missing or misconfigured?
- What would have limited the blast radius if the initial access had been prevented?
- Where did log coverage fail the investigation?
The Post-Incident Review
Run a blameless post-incident review within 2 weeks while memories are fresh. The output should be:
- A finalized incident timeline
- Root cause(s) identified — both the technical vulnerability and the process failure that allowed it
- A prioritized remediation list with owners and due dates
- Lessons learned that improve the IR playbook itself
Document the findings in a format that can be shared with legal counsel under privilege if there is any risk of regulatory action or litigation.
Find Vulnerabilities Before Attackers Do
The best incident response is the one you never need. Ironimo runs continuous security scans using the same tools attackers use — finding SQL injection, authentication bypasses, and configuration exposures before they become breaches. Run your first scan before your next IR call.
Start free scan