ESP32-CAM Face Detection: Basics of Edge AI

"Face detection" and "face recognition" get used interchangeably, and that mix-up sets the wrong expectations before a single line of code is written. Detection answers "is there a face in this frame, and roughly where?" -- recognition answers "whose face is this?" An ESP32-CAM, running entirely on-device with no cloud service, handles the first one comfortably. The second one is a meaningfully bigger job that usually needs more memory and compute than this board carries alone. This guide covers what the ESP32-CAM can actually do on its own: real-time face detection, with a bounding box, running locally -- plus what happens once you go past a single face in a well-lit room, since that's where most people's first attempt starts to wobble.

Hardware Requirements

• An ESP32-CAM (AI-Thinker) board -- this guide assumes that specific variant, since it's what the standard Arduino example targets by default
• An FTDI/USB-to-serial programmer (the ESP32-CAM has no onboard USB port)
• A genuinely stable 5V supply -- brownouts during camera initialization are the single most common "it just won't work" report with this board, and a marginal USB port or thin cable is very often the actual cause

Crucially, face detection specifically requires PSRAM (extra external RAM the AI-Thinker board includes) -- a clone board without PSRAM will compile and flash without error, but will either fail to initialize the camera or crash the moment detection is enabled.

Enabling Face Detection in the Standard Example

Espressif's own camera web server example, bundled with the ESP32 Arduino core, already includes face detection -- it just needs the right settings enabled:

1. File > Examples > ESP32 > Camera > CameraWebServer
2. Uncomment the line matching your board -- for AI-Thinker: #define CAMERA_MODEL_AI_THINKER
3. Fill in your WiFi ssid and password
4. Tools > Partition Scheme > Huge APP (3MB No OTA/1MB SPIFFS) -- the sketch is large enough that the default partition scheme won't fit it
5. Tools > PSRAM > Enabled -- this is the setting most often missed, and skipping it causes a crash the moment the sketch tries to allocate a detection buffer

// Inside the example's config struct -- verify these values match your board
config.frame_size = FRAMESIZE_QVGA;   // smaller frame = detection actually keeps up
config.pixel_format = PIXFORMAT_JPEG; // streaming format; detection runs on a separate grayscale copy internally

Once flashed, opening the board's IP address in a browser shows the standard camera web UI, with a Face Detection toggle in the sidebar -- enabling it draws a live bounding box around any detected face directly in the video stream.

How Detection Actually Works, Briefly

Underneath, the example uses a lightweight detection model (fd_forward, part of Espressif's esp-face library) tuned specifically to run on a chip this small. It works on a downscaled, grayscale copy of each frame -- not the full-resolution JPEG being streamed to your browser -- using a sliding-window style scan tuned for speed over accuracy.

That trade-off means resolution has a direct, and fairly steep, effect on frame rate:

QVGA is the practical default for anything that needs to feel responsive -- pushing resolution higher for a sharper stream comes directly at the expense of detection speed on this hardware.

Multiple Faces and Group Scenes

The stock example isn't limited to one face -- net_boxes->len returns however many the scan found in a given frame, and each gets its own bounding box. In practice, though, reliability drops off faster than the FPS table above suggests once more than two or three faces are in frame at once.

Two things tend to cause missed detections in a group: faces smaller than roughly 40x40 pixels at QVGA get skipped by the sliding window entirely, and overlapping or partially occluded faces (someone standing behind another person) confuse the scan's edge-based scoring. Neither is a bug to fix -- it's the model doing exactly what a lightweight, speed-tuned detector is built to do. If reliable multi-face counting at a distance is the actual goal, that's a sign the project has outgrown this particular model and is better served by a heavier one running off-device.

Triggering an Action on Detection

The stock example only draws a box in the browser -- a real project usually wants to do something when a face appears, like lighting an LED or sounding a buzzer:

// Simplified concept -- inserted where the example's detection callback runs
#define ALERT_PIN 4

void onFaceDetected(int faceCount) {
  if (faceCount > 0) {
    digitalWrite(ALERT_PIN, HIGH);
  } else {
    digitalWrite(ALERT_PIN, LOW);
  }
}

The exact hook point depends on the example version, but the pattern is consistent: the detection routine already returns how many faces (and their bounding boxes) it found each frame -- that count is exactly what a simple presence-alert project needs, without touching any of the actual detection math.

For anything beyond a single LED, route ALERT_PIN through a transistor or relay module rather than driving a buzzer or servo directly off a GPIO pin -- the ESP32's pins aren't rated for much current, and a camera that's already fighting for power budget doesn't need a motor or coil drawing off the same 3.3V rail.

What This Setup Can't Do

No recognition without more. Telling faces apart requires an embedding model and a stored reference set, which is a meaningfully heavier workload than detection -- possible on an ESP32 in constrained forms, but well beyond flipping a setting in the stock example.

Lighting sensitivity. The detector works on grayscale brightness patterns, so strong backlighting or very dim rooms measurably hurt detection rate.

Group photos underperform single faces. The lightweight model is tuned for speed, and multiple overlapping or small faces in one frame reduce reliability compared to a single face reasonably close to the lens.

On-Device Detection vs. Cloud-Based Recognition

Once a project needs more than "a face is present," the choice is between pushing more model onto the ESP32 itself or offloading the harder work to something with real compute behind it. Both are legitimate paths, and the right one depends mostly on how much latency and internet dependency the project can tolerate.

A common middle ground: use the ESP32-CAM's on-device detection purely as a trigger -- only bother capturing and uploading a frame to a cloud recognition service when a face is actually present, instead of streaming continuously. That keeps bandwidth and API costs down while still getting recognition when it's genuinely needed.

Real Project Ideas Built on Basic Detection

Detection alone, without recognition, is still the core of several genuinely useful builds:

• Presence-triggered recording -- only save video or photos to microSD when a face enters frame, instead of continuously, which saves card space and makes later review far faster
• Doorbell wake-up -- pair with a PIR sensor as a coarse first-stage trigger, then confirm with face detection before sending a phone notification, cutting down false alerts from passing cars or blowing leaves
• Occupancy-aware lighting or displays -- turn on a screen, light, or fan only when someone is actually looking at the device, rather than on a fixed timer
• Simple attendance or headcount logging -- log a timestamp each time a new face appears in frame at a fixed station, useful for rough foot-traffic counting where exact identity doesn't matter
• Photobooth-style triggers -- fire a countdown and capture the instant a face is detected and centered, without a physical button

None of these need recognition -- they need to know a person is there, and that's exactly what this setup already provides out of the box.

Common Mistakes

Skipping the PSRAM setting. By far the most common cause of a crash or a camera that won't initialize once face detection is turned on -- verify Tools > PSRAM > Enabled before anything else.

Underpowering the board. A thin USB cable or a weak power source causes brownouts specifically during camera init or when the WiFi radio and camera are both active -- symptoms that look like a software bug but are almost always a power problem.

Picking the wrong CAMERA_MODEL_ define. Each ESP32-CAM variant wires its camera pins differently; the wrong define compiles fine but produces a garbled or blank stream.

Expecting recognition, not detection. "It found a face but doesn't know it's me" isn't a bug -- it's exactly what this feature is built to do, and no more.

Running at a high resolution and expecting real-time detection. The frame-rate table above is the direct trade-off; QVGA is the right starting point for anything that needs to feel responsive.

Power Draw and Thermal Behavior During Continuous Detection

Running detection continuously keeps the CPU noticeably busier than plain streaming, and that shows up in two places worth planning for. Current draw sits around 250-320mA during active detection versus roughly 180-220mA for streaming alone -- not dramatic, but enough to matter on a marginal power supply or a battery-powered build where every milliamp affects runtime.

The second effect is heat. The ESP32-S module and the camera sensor both warm up measurably after 20-30 minutes of continuous detection in an enclosure without airflow, and on some boards this shows up as a gradual increase in detection false negatives before anything actually fails outright -- the sensor's noise floor rises slightly as it warms. For anything running detection 24/7, a vented enclosure or a board mounted with some airspace around it avoids the issue entirely; it's not a fragile setup, it just wasn't designed with a sealed box in mind.

Improving Detection Accuracy Without Changing Hardware

Before reaching for a bigger model or a second board, a handful of software-only adjustments usually close most of the gap between "works in good conditions" and "works reliably":

Fix the exposure, don't let it hunt. The camera's auto-exposure routine actively fights detection in mixed lighting -- it keeps adjusting brightness as faces enter and leave frame, and each adjustment briefly degrades the grayscale image the detector scans. Setting a fixed exposure value (s->set_exposure_ctrl(s, 0) followed by a manual s->set_aeg_value(s, ...)) for a fixed-lighting installation, like a hallway or desk, removes that source of jitter entirely.

Mount at face height, not above it. A camera angled downward from a high mount -- common in doorbell and security placements -- captures faces at an angle the detector wasn't tuned for. Straight-on, roughly eye-level placement measurably outperforms a steep downward angle, even at identical distance and lighting.

Give the detector a consistent background. A cluttered or high-contrast background behind the subject increases false triggers, since the sliding-window scan is scoring edge patterns, not understanding scene content. A plain wall behind the expected face position, where that's controllable, noticeably reduces false positives.

Don't skip the warm-up frame. The very first frame or two after the camera initializes is sometimes underexposed while auto-exposure settles -- discarding the first 2-3 frames before starting to log or act on detections avoids a false "no face" reading on startup.

esp-face vs. Newer On-Device Options

fd_forward and the esp-face library are what ship in the stock example, and for a reason -- they're small, fast, and already integrated. They're not, however, the only on-device option anymore, and it's worth knowing what the alternatives trade off:

ESP-WHO in particular is worth a look once a project outgrows plain detection, since it's built by the same team, targets the same hardware, and adds a genuine (if basic) recognition path without leaving the ESP32 ecosystem entirely. It's a meaningfully bigger install than flipping the toggle covered in this guide, which is exactly why it isn't the default in the stock example -- most people trying face detection for the first time don't need that weight yet.

Frequently Asked Questions

Can the ESP32-CAM run face detection and video streaming at the same time? Yes -- the standard example does exactly that by default. The detection overlay draws on top of the same stream everyone else sees, rather than running as a separate process.

Does detection work through a window or at an angle? Reasonably well through clean glass with no glare, and up to roughly 30-40 degrees off-center before reliability drops noticeably. Sharp angles or reflective glass are where it struggles most.

Will a non-AI-Thinker ESP32-CAM variant work? Most will, provided PSRAM is present and the correct CAMERA_MODEL_ define is selected -- but pin mappings differ enough between variants that this guide's specific instructions assume AI-Thinker unless noted otherwise.

How far away can it detect a face? At QVGA resolution, reliably out to roughly 2-2.5 meters for a single, well-lit, forward-facing subject. Past that distance, the face occupies too few pixels for the sliding-window scan to register it consistently.

Does it work in complete darkness with the onboard LED flash? Partially -- the flash is a single white LED positioned for close-range illumination, not a proper IR night-vision setup, so detection range and reliability both drop substantially in full darkness even with it on. A dedicated IR illuminator paired with an IR-sensitive camera module is the more reliable path for genuine low-light or nighttime detection.

Where to Go Next

Once detection is running reliably, the natural next steps are the ESP32-CAM Pinout Guide for wiring anything beyond the basic setup, and the ESP32-CAM Web Server guide, which covers the streaming side of this same example in more depth for anyone building a standalone monitoring project rather than starting from detection specifically.