Lab notesLog pipelines1 min read

ELK Security Lab

Local web security monitoring lab with OWASP Juice Shop behind Nginx. Filebeat forwards logs to Elasticsearch ingest pipelines that parse requests and flag attack patterns for exploration in Kibana.

elasticsearchkibanafilebeatowasp
Focus
Log pipelines
Scope
web security
Source
Public repository

The purpose

Understand how web requests become searchable security events, using OWASP Juice Shop as a target application and Elasticsearch ingest pipelines to enrich its proxy logs.

How it fits together

  1. 01

    Requests to OWASP Juice Shop through Nginx

  2. 02

    Filebeat reads Nginx logs

  3. 03

    Elasticsearch ingest pipelines parse and enrich events

  4. 04

    Kibana event exploration

Engineering choices

Observe traffic at the proxy

Nginx sits in front of Juice Shop, providing a common source of request and error logs for the monitoring pipeline.

Enrich events during ingestion

Elasticsearch ingest pipelines parse requests and add security fields. Filebeat handles collection and forwarding.

Test with recognisable requests

The README supplies SQL injection, XSS, and path traversal requests, with a query for finding events marked as threats.

What the project documents

The repository documents Nginx and Filebeat configuration, request and error ingest pipelines, and a pipeline setup script.

Documented detection examples include SQL injection, XSS, path traversal, command injection, failed logins, and scanner traffic.

Find flagged events in Kibana

Plain text
security.threat_detected: true

Use this filter in Discover after installing the pipelines and generating traffic.

Current scope & limitations

A local learning environment for parsing logs and recognising common web attack patterns. Pattern matches are signals to inspect, not proof that an attack succeeded.

Explore the implementation

Code, configuration & setup

The repository contains the source files and instructions for this project.

View repository on GitHub
End of lab noteBack to all labs