case study

yolo ornithopter tracker

senior capstone 2025 - 2026 programming team python · pytorch · yolo · tensorrt

my senior capstone. our student-run engineering company, arcturus industries, built corvus, which is a robot raven that flies, walks, and grabs stuff on its own for the 2026 carpa challenge. i was on programming, working on navigation and vision.

our team documentary. the bird, the mission, the lab.

cad render of the full corvus assembly
full corvus assembly (cad)
yolo object detection running on a live camera frame
yolo detecting objects on a live camera frame

the challenge

build a robot raven

brief: build an ornithopter (it flaps to fly) that looks like a raven and runs the whole mission by itself, with a kill switch just in case.

  • take off, fly, and land on a tower
  • walk on two legs
  • find items and carry them back to build a nest
  • use a stick as a tool
  • say "nevermore" in its handler's voice at the end
  • stay under 400 ft so the faa stays happy

the part i worked on

how corvus finds its way

01

know where it is

rtk gps gets position down to centimeters. it lags a bit, so we fuse it with imu data in a kalman filter.

02

see what's around

yolo looks at each camera frame and says what's there, where, and how sure it is. weak guesses get tossed, and distance comes from box size.

03

remember it

every find gets real coordinates and goes on a map with its point value.

04

plan a route

a* picks the shortest path to the next item: distance so far + distance left.

diagram of rtk gps base station correcting the drone's position
rtk gps: the base station corrects the drone's position
a* pathfinding on a grid around obstacles
a* pathfinding around obstacles

hardware

arducam ar0234 camera um960 rtk gnss bno055 9-dof imu esp32

in progress

testing the hardware

esp32 on a breadboard wired to an rtk gps module and antenna outdoors
the esp32 and rtk gps module, testing outside
arduino ide showing control code and serial monitor output
checking code and serial output

what i did

  • built a two-stage pipeline: a light yolo model runs on the bird, and a heavier one checks its work.
  • used box corrections as the fix signal, same idea as rtk gps.
  • sped it up with tensorrt so it could keep up mid-flight.
  • worked with the beak, legs, and wings teams so everything actually fit together.

what happened

  • survived five design reviews with industry experts.
  • ran the mission challenge on may 20, then showed off corvus at the expo on may 27.
  • detection ran at 30 fps on the bird.
  • mission was a half success with my section of the mission working properly.