Case Study: Autonomous Lane Detection Pipeline

01 Executive Overview

The Autonomous Lane Detection Pipeline is a modular computer-vision perception system developed to identify road-lane boundaries from onboard monocular camera feeds. Built primarily using classical image processing principles in Python and OpenCV, the project decouples raw visual feature extraction from virtual vehicle dynamics by isolating the CARLA simulator connection behind a dedicated bridge adapter.

✓ Author Contribution & Authorship Boundary

Sole author and developer. Designed and implemented the core image processing algorithms in lane_detector.py, built the synthetic test image generator and automated unit tests in tests/test_lane_detector.py, and developed the CARLA sensor-streaming interface in carla_adapter.py.

02 Problem & System Objectives

The Perception Challenge

Autonomous and semi-autonomous driving systems depend on precise, low-latency identification of drivable corridor boundaries. While deep neural networks (e.g. segmentation CNNs) provide rich representations, they present major engineering challenges:

High Compute Footprint: Deep learning models require substantial GPU memory and draw significant electrical power, making them costly on resource-constrained embedded automotive ECUs.
Lack of Determinism: Black-box neural models can fail unpredictably when confronted with out-of-distribution visual artifacts, making debugging and formal safety verification difficult.
Tight Coupling with Simulators: Many academic research projects hardcode simulator dependencies into the core perception logic, preventing the code from running on standard video streams or alternative simulators.

System Objectives

Deterministic Classical Pipeline: Formulate an interpretable pipeline using mathematical edge detection and Hough line clustering that runs at high frame rates on standard CPU architectures.
Temporal Boundary Smoothing: Prevent frame-to-frame jitter through exponential or moving-average smoothing of estimated left and right lane coordinates.
Decoupled Architecture: Maintain a strict boundary between image analysis and simulator interaction, allowing identical detection code to process prerecorded MP4 dashcam footage or live CARLA RGB sensor streams.

03 Solution Architecture

The pipeline processes images sequentially through filtering, gradient calculation, spatial masking, geometric feature extraction, and temporal filtering:

Perception & Simulation Pipeline

[CARLA Simulator / Dashcam Video]
               │
               ▼
   [Raw RGB Frame Acquisition]
               │
               ▼
[Grayscale Conversion & Gaussian Blur] (5x5 kernel, σ=1.5)
               │
               ▼
     [Canny Edge Extraction] (Low=50, High=150 hysteresis)
               │
               ▼
   [Dynamic Trapezoidal ROI Mask] (Clamping to roadway perspective)
               │
               ▼
[Probabilistic Hough Line Transform] (HoughLinesP: ρ=2, θ=π/180)
               │
               ▼
  [Slope & Intercept Clustering]
  ├── Left Lane:  Negative Slope (m < -0.5)
  └── Right Lane: Positive Slope (m > 0.5)
               │
               ▼
   [Extrapolation to Horizon & Bottom]
               │
               ▼
  [Temporal Moving-Average Smoothing] (Buffer N=5 frames)
               │
               ▼
[Visual Alpha Overlay & Metric Output] ──► [CARLA Visualizer]

Component Rationale

Gaussian Blur (5x5 Kernel): Suppresses high-frequency asphalt texture and sensor noise that causes spurious gradient spikes.
Dynamic Polygonal ROI: Zeroes out image regions containing sky, horizon, and roadside obstacles, confining line detection strictly to the projected driving corridor.
Slope Separation: Filters out horizontal and near-zero slope lines (guardrails, crosswalks, shadows) by enforcing threshold boundaries on candidate line gradients.

04 Implementation Details

Core Modules in Repository

lane_detector.py: Encapsulates all OpenCV perception logic. Provides methods for grayscale preprocessing, edge calculation, polygonal masking, line fitting, and weighted overlay rendering.
carla_adapter.py: Handles CARLA simulator client connection, world actor attachment, synchronous camera sensor listening, and format conversion from raw CARLA BGRA image buffers to NumPy BGR frames.
tests/test_lane_detector.py: Automated unit tests utilizing programmatic synthetic lane images (black canvases with programmatic high-contrast diagonal white stripes) to verify slope calculation and boundary extrapolation without external media files.

💡 Key Engineering Decision: Decoupled Simulation Adapter

By isolating CARLA code inside carla_adapter.py, the core lane_detector.py has zero CARLA Python API dependencies. This ensures the perception pipeline can be executed, benchmarked, and unit-tested on lightweight CI runners without installing the heavy 20GB Unreal Engine CARLA simulator package.

05 Evaluation & Verification Evidence

Verification was performed using unit testing suites and video playback inspection:

Automated Unit Tests: Programmatic synthetic lane tests pass consistently in CI via pytest, verifying that left and right lines are accurately separated and extrapolated to predefined vertical bounds.
Synthetic Video Playback: The pipeline successfully tracks lane markings on clean asphalt test sequences with smooth corridor visual overlays.

⚠️ Evaluation Scope Note

In strict compliance with portfolio standards: End-to-end steering angle prediction and closed-loop autonomous vehicle lateral control are explicitly not claimed. This project is a visual perception and road feature extraction pipeline, not a certified full autonomous driving stack.

06 Challenges & Engineering Trade-offs

Design Aspect	Trade-off Accepted	Mitigation Strategy
Classical CV vs. Deep Learning	Classical edge detection struggles with severe tree shadows and worn-out lane paint markings.	Applied adaptive thresholding and temporal frame averaging to carry previous valid lane trajectories across obscured frames.
Straight-Line Hough Fit	Hough transform fits straight lines (y = mx + b), which loses fidelity in sharp hairpin curves.	Restricted detection horizon to the immediate 30-meter vehicle front zone; planned higher-order polynomial fitting.

07 Future Roadmap

2nd-Order Polynomial Curve Fitting: Upgrade from straight-line Hough clustering to sliding-window polynomial fitting (x = Ay² + By + C) to accurately track curved highway ramps.
Hybrid Deep Learning Integration: Integrate a lightweight segmentation network (e.g. ENet or Ultra-Fast-Lane-Detection) for robust mark classification under challenging adverse weather conditions.
Closed-Loop Lateral Control: Implement a PID or Pure Pursuit steering controller inside the CARLA adapter to evaluate vehicle lane-keeping behavior in simulation.

Autonomous Lane Detection Pipeline & CARLA Bridge