Projects
ProjectsDevOps FoundationsbeginnerCLI

Nginx Log Analyzer

CLI, parsing, aggregation and operational reporting.

~8 hours 5 phases
LinuxBashawkgrep

Overview

Build a CLI that parses Nginx access logs and aggregates requests, status codes, top clients and error reports.

The problem

Analyse Nginx access logs to surface traffic and error insights.

Why it matters

Log analysis is foundational to operations and observability.

What you'll build

  • A log parser
  • Aggregation reports
  • Error surfacing

What you'll learn

  • Text processing
  • Log analysis
  • Reporting

Business problem

Operators need insight from access logs to detect issues and trends.

Technical problem

Parse and aggregate Nginx logs into actionable reports.

When you're done: A CLI that turns raw logs into operational insight.

Requirements

User requirements

  • See top clients
  • See status distribution
  • Find errors

System requirements

  • Nginx access log
  • Shell tools

Functional requirements

  • Parse log lines
  • Aggregate by field
  • Report generation

Non-functional requirements

  • Fast on large logs

Technical requirements

  • awk/grep/sort
  • Report output

Security requirements

  • Sanitise display

Operational requirements

  • Works across log paths

Constraints

  • text-processing tools

Success criteria

  • Reports match the logs
  • Errors surfaced

Flows

Request Flow

  1. 1

    Browser

    User requests the application over HTTPS.

  2. 2

    CDN / DNS

    Edge routes the request to the origin.

  3. 3

    Load Balancer

    Distributes traffic across healthy app instances.

  4. 4

    Application

    Handles the request, applies business logic.

  5. 5

    Cache

    Serves fast reads for hot data.

  6. 6

    Database

    Persists data and returns results.

Project Structure

Root of the client application source.

Setup

  1. 1

    Prerequisites

    You need the tools required by Linux, Bash, awk installed and available on your PATH.

  2. 2

    Clone project

    Clone the repository locally.

    1. Clone the repository
    git clone <your-repo-url> <project-folder>
  3. 3

    Install dependencies

    Install the project dependencies.

    1. Check the OS
    cat /etc/os-release

    Expected

    Shows the OS name and version.

    If it fails

    File not found

    Fix

    Use a standard Linux distribution.

  4. 4

    Configure environment

    Set the environment variables documented for this project. Never commit secrets.

    1. Create the environment file
    cp .env.example .env   # then edit values
  5. 5

    Start project

    Run the development command and confirm it starts.

    1. List users
    cat /etc/passwd

    Expected

    Lists user accounts.

    If it fails

    Permission denied

    Fix

    Read the world-readable passwd file.

  6. 6

    Verify

    Open the printed local URL and confirm the primary flow works.

Build Phases

0 of 5 phases complete

  1. Objective: Establish the next milestone of the operational environment.

    What changes: The project advances to the next coherent state, keeping the previous milestones working.

    Files touched

    Relevant source and configuration files for this milestone.

    Run

    1. Check the OS
    cat /etc/os-release

    Expected

    Shows the OS name and version.

    If it fails

    File not found

    Fix

    Use a standard Linux distribution.

    Tasks

    • Implement the specific behaviour required for this milestone.
    • Verify the previous milestone still works.

    Validate: The milestone behaves as expected and does not break existing functionality.

    Common mistakes

    • • Skipping validation and moving on before confirming the milestone works.

  2. Objective: Establish the next milestone of the operational environment.

    What changes: The project advances to the next coherent state, keeping the previous milestones working.

    Files touched

    Relevant source and configuration files for this milestone.

    Run

    1. List users
    cat /etc/passwd

    Expected

    Lists user accounts.

    If it fails

    Permission denied

    Fix

    Read the world-readable passwd file.

    Tasks

    • Implement the specific behaviour required for this milestone.
    • Verify the previous milestone still works.

    Validate: The milestone behaves as expected and does not break existing functionality.

    Common mistakes

    • • Skipping validation and moving on before confirming the milestone works.

  3. Objective: Establish the next milestone of the operational environment.

    What changes: The project advances to the next coherent state, keeping the previous milestones working.

    Files touched

    Relevant source and configuration files for this milestone.

    Tasks

    • Implement the specific behaviour required for this milestone.
    • Verify the previous milestone still works.

    Validate: The milestone behaves as expected and does not break existing functionality.

    Common mistakes

    • • Skipping validation and moving on before confirming the milestone works.

  4. Objective: Establish the next milestone of the operational environment.

    What changes: The project advances to the next coherent state, keeping the previous milestones working.

    Files touched

    Relevant source and configuration files for this milestone.

    Tasks

    • Implement the specific behaviour required for this milestone.
    • Verify the previous milestone still works.

    Validate: The milestone behaves as expected and does not break existing functionality.

    Common mistakes

    • • Skipping validation and moving on before confirming the milestone works.

  5. Objective: Establish the next milestone of the operational environment.

    What changes: The project advances to the next coherent state, keeping the previous milestones working.

    Files touched

    Relevant source and configuration files for this milestone.

    Tasks

    • Implement the specific behaviour required for this milestone.
    • Verify the previous milestone still works.

    Validate: The milestone behaves as expected and does not break existing functionality.

    Common mistakes

    • • Skipping validation and moving on before confirming the milestone works.

Commands

1. Check the OS
cat /etc/os-release

Expected

Shows the OS name and version.

If it fails

File not found

Fix

Use a standard Linux distribution.

2. List users
cat /etc/passwd

Expected

Lists user accounts.

If it fails

Permission denied

Fix

Read the world-readable passwd file.

Testing

Unit tests

  • Core logic and pure functions behave correctly.

    cat /etc/os-release

    Success: All unit tests pass without errors.

Integration tests

  • Components integrate correctly with data sources and services.

    npm test

    Success: Integration tests pass for the primary flows.

API tests

  • Endpoints respond to valid and invalid input correctly.

    npm test

    Success: API tests cover happy and failure paths.

End-to-end tests

  • A user can complete the primary journey end to end.

    npm run test:e2e

    Success: The main user journey completes successfully.

Security tests

  • No obvious security regressions are introduced.

    npm audit

    Success: No high or critical vulnerabilities.

Debugging & Troubleshooting

Port already in use at startup

Why: Another process is binding the same port your app wants.

Check: List what is using the port.

lsof -i :3000   # Linux/macOS  |  netstat -ano | findstr :3000   # Windows

Fix: Free the port or start your app on a different port.

Prevent: Choose uncommon ports and document them in the README.

Environment variable missing or undefined

Why: A required value was not loaded into the runtime.

Check: Print the variable or inspect the config loader.

echo $MY_VAR   # Linux/macOS  |  echo %MY_VAR%   # Windows

Fix: Add the variable to your .env file and restart the process.

Prevent: Fail fast at startup when required env vars are absent.

API request times out

Why: An upstream call is slow or the network is blocked.

Check: Time a direct request against the upstream.

curl -w 'time_total: %{time_total}\n' http://upstream/health

Fix: Add timeouts and retries to the client; verify the upstream is up.

Prevent: Set explicit timeouts and circuit breakers on outbound calls.

Dependency fails to install or run

Why: Version conflicts or an incompatible runtime.

Check: Inspect the error and the installed versions.

npm ls <pkg>

Fix: Align versions or clear the lockfile and reinstall.

Prevent: Commit a lockfile and use the same Node version everywhere.

Security

Security is treated as a core requirement: input is validated, secrets are protected, and dependencies are kept up to date.

  • Input validation

    Validate and sanitize all user input before use.

  • Secrets management

    Store secrets in environment variables, never in source.

  • Dependency hygiene

    Regularly audit dependencies for known vulnerabilities.

Deployment

  1. 1

    Prepare for deployment

    Build the production artifact.

    1. Build for production
    npm run build
  2. 2

    Deploy

    Deploy to the project target platform following its documented steps.

    1. Deploy
    npm run deploy   # or the platform-specific command
  3. 3

    Verify

    Confirm the deployed application responds and its health check passes.

    1. Check health
    curl -I https://<your-deployed-url>

Documentation

Project Overview

Build a CLI that parses Nginx access logs and aggregates requests, status codes, top clients and error reports. This is a beginner cli project in the operational environment space, built to demonstrate real engineering, not just copied code.

What You Will Build

- A log parser - Aggregation reports - Error surfacing

Problem Statement

Operators need insight from access logs to detect issues and trends. Parse and aggregate Nginx logs into actionable reports.

Technology Stack

Linux, Bash, awk, grep

Installation

Clone the repository and install dependencies with the documented commands.

Local Development

Run the development command and open the printed local URL.

Testing

Run the test commands documented in the Testing section and confirm they pass.

Deployment

Follow the documented deployment steps for this project's target platform.

Interview Explanation

Use the Interview section to explain the architecture, technology choices and trade-offs.

Build Process

Phases guide the build in 5 checkpoints. Complete each and validate before moving on.

Interview Prep

30-second pitch

I built Nginx Log Analyzer — analyse nginx access logs to surface traffic and error insights. It uses Linux, Bash, awk and solves operators need insight from access logs to detect issues and trends.

2-minute walkthrough

Build a CLI that parses Nginx access logs and aggregates requests, status codes, top clients and error reports. The core requirements were Parse log lines, Aggregate by field, Report generation. I structured the work around Parse log formats, Aggregate with awk/sort, Generate reports and validated the result with the documented tests.

Architecture

Because this is a devops-foundations project, the architecture is concentrated in operational environment rather than a distributed system.

Why these technologies

I chose Linux, Bash, awk, grep because they are the pragmatic, well-supported tools best suited to building a cli of this kind.

Why this architecture

The structure exists to keep concerns separated, make each piece testable, and scale the project cleanly as requirements grow.

At scale

At scale I would introduce caching, move to a service boundary structure, add proper observability, and enforce stricter contracts between components.

What can fail

The data source becomes unavailable, A dependency introduces a breaking change, Performance degrades under load, Configuration drift between environments

Monitoring

Monitoring is covered in the Observability section, focusing on basic health and error signals.

Security

Security was handled by validating input, protecting secrets, and following the practices listed in the Security section.

Deployment strategy

Deployment follows the steps in the Deployment section for the target platform, with a health check to confirm success.

What I'd do differently

With more time I would add deeper test coverage, stronger observability, and automated deployment.

Practice Q&A

Why did you build Nginx Log Analyzer?

To demonstrate Parse log formats and Aggregate with awk/sort on a real, portfolio-worthy project.

What was the hardest part?

Coordinating the pieces so each stays correct as the project grows, and validating end to end.

How do you handle failure and scale?

By designing for correctness first and adding resilience once the core works.

Extensions

  • Add deeper test coverage
  • Add observability and alerting
  • Harden security
  • Automate deployment

Completion Checklist

Prerequisite Projects