From Image Detection to Safe Control Commands

Learning objectives

Use simulated inputs to examine how sign errors and oscillation arise when horizontal and vertical image errors are converted into movement commands.

Execution conditions

The basic activities use NumPy and Matplotlib. Use OpenCV when connecting to the previous unit.

Convert detected positions to errors

Starting conditions

The center of a 160×120 image is (80,60), and the observed center is (110,50). x points right and y points down.

Procedure

1. Calculate $e_x=x-80$ and $e_y=y-60$ using the same sign convention as command(center) in the example. Plot the image center and observed point, and label both errors on an arrow from the center to the observed point.
2. For each axis, set the command to 0 if $|e|\le10$ pixels; otherwise calculate it using $0.02e$, then limit it to $[-0.5,0.5]$. Calculate both axis commands for $(110,50)$ and record them beside the function outputs.

Results to retain

• A center/error arrow diagram and example commands.

import numpy as np
import matplotlib.pyplot as plt
def command(center):
    if center is None:return np.zeros(2)
    error=np.asarray(center)-[80,60]
    return np.clip(np.where(np.abs(error)<=10,0,.02*error),-.5,.5)
print(command((110,50)),command(None))
plt.scatter([80,110],[60,50]);plt.arrow(80,60,30,-10,head_width=3);plt.xlim(0,160);plt.ylim(120,0);plt.gca().set_aspect('equal')

Record results and self-check

Observed center:

x/y errors:

Validity:

Command:

Points to check

• Have you distinguished the image y-axis direction from upward in world coordinates?
• Did you distinguish pixel error and normalized commands from the aircraft's actual velocity [m/s]?

Handle fluctuations and missed detections

Starting conditions

x observations are [80,82,79,110,None,108], with $y=60$. None denotes a frame where the object was not found.

Procedure

1. For each valid observation, calculate the command by applying command((x,60)) directly, and separately by applying it to the mean x of up to the 3 most recent valid observations. At startup, if fewer than 3 samples are available, average only the collected samples.
2. At None, set the command to 0 and clear the averaging history. Treat the later 108 as the first sample in a new history.
3. Record frame order, raw observations, mean x, and both commands in a table and plot them against the same frame axis. Explain the differences by identifying both commands at the frame where x changes to 110 and the command immediately after a missed detection.

Results to retain

• Observation/command time series and missed detection rules.

from collections import deque
history=deque(maxlen=3)
for x in [80,82,79,110,None,108]:
    if x is None:history.clear();filtered=None
    else:history.append(x);filtered=(np.mean(history),60)
    print(x,filtered,command(filtered))
# TODO: Plot raw and filtered commands on the same axes.

Record results and self-check

Frame:

Observed x:

Filtered x:

Command:

Handling missed detections:

Points to check

• Did you also check that a moving average causes response delay?
• Have you avoided including None as numeric 0 in the average?

Connect detection to commands

Starting conditions

Connect the previous unit's mask center calculation to command. The basic activities use only saved or generated images.

Procedure

1. Separate the function returning center, area, and detection time from the command function.
2. Test an empty mask, a valid region, and a frame more than 0.2 seconds past detection, in that order. Identify stale frames by the difference between the current time and frame detection time, and set commands to 0 in that case.
3. Describe additional coordinate transformations, velocity limits, and stop conditions needed for a real device.

Results to retain

• An input→detection→validation→command flowchart and an error case table.

# TODO: Connect detect_center(frame) and command(center), and
# Set the command to 0 if the Frame timestamp is more than .2 s old.

Record results and self-check

Input frame:

Detection validity:

Command:

Stop reason:

Points to check

• Does a failed detection clear the previous velocity command?
• Can you test error paths without actual flight?

Official examples and data

• OpenCV · Color spaces and region extraction
• OpenCV · Circle detection