Perception and computer vision

What an image is to a computer

To a computer an image is just a grid of numbers. A colour image is a 3D array — height by width by three channels (blue, green and red in OpenCV's convention) — where each pixel is three values from 0 to 255 giving the intensity of each colour. A 640×480 colour image is nearly a million numbers. Computer vision is the craft of turning that wall of numbers into something meaningful: 'there is a red ball at the centre-left', 'the line curves right', 'a person is ahead'. Everything starts from understanding that the camera hands you an array, and your job is to reduce it to a decision the robot can act on.

The classic vision pipeline

Most simple robot-vision tasks follow the same pipeline, and knowing the stages lets you build almost any basic detector. Capture a frame from the camera. Pre-process it — resize smaller for speed, blur slightly to reduce noise, convert to a useful colour space. Segment — isolate the pixels you care about, for example everything within a colour range, producing a black-and-white mask. Find features — group the white pixels into shapes (contours), and measure them (area, centre, bounding box). Decide — pick the shape that matches what you want (the largest blob of the right colour) and turn its position into an action (steer toward its centre). Each stage is a few lines with OpenCV, and together they take you from raw pixels to a motor command.

• Capture → grab a frame from the camera.
• Pre-process → resize, blur, convert colour space.
• Segment → threshold to a mask of the pixels you want.
• Find features → contours, areas, centres, bounding boxes.
• Decide → choose the target and turn its position into an action.

Finding a coloured object with OpenCV

Here is the pipeline in code: convert to HSV, threshold to a mask for a colour range, find contours, pick the biggest, and compute its centre. The centre's horizontal position tells a robot whether the object is to its left or right — everything a colour-following robot needs.

import cv2, numpy as np

cap = cv2.VideoCapture(0)
lower = np.array([100, 120, 70])    # HSV range for a blue object
upper = np.array([130, 255, 255])

while True:
    ok, frame = cap.read()
    if not ok:
        break
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    mask = cv2.inRange(hsv, lower, upper)          # white where blue
    contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL,
                                   cv2.CHAIN_APPROX_SIMPLE)
    if contours:
        biggest = max(contours, key=cv2.contourArea)
        if cv2.contourArea(biggest) > 500:         # ignore tiny specks
            x, y, w, h = cv2.boundingRect(biggest)
            cx = x + w // 2                        # object's x centre
            print('object at x =', cx)             # steer toward cx
    if cv2.waitKey(1) == 27:
        break

When classic vision is enough — and when it is not

Colour and contour tricks are fast, run on a small computer, and are perfectly good for well-defined tasks: follow a bright ball, track a coloured line, find a marker of known colour. Their weakness is that they break when the world is messy — varied lighting, cluttered backgrounds, or objects defined by *what they are* rather than a single colour. 'Find any person' or 'is this a cat or a dog' cannot be done by thresholding a colour. That is the job of deep learning — neural networks trained on many labelled images that recognise objects by learned features. On robots this often runs as a pre-trained object detector (like a YOLO model) that returns boxes and labels. The practical guidance: reach for classic vision first because it is simple and cheap, and move to a learned detector only when the task genuinely needs to recognise *categories* of things rather than a distinct colour or shape.