case study
Human Anomaly Detection System
My final-year engineering project, restored and rebuilt. It watches live video and detects falls, sitting, squatting and standing, using a pretrained YOLOv8 pose model plus a posture classifier I built from keypoint geometry and calibrated against 3,106 detections drawn from 4,924 images.
The model I trained is the one I threw away
The version I submitted for my degree was a yolov8n fine-tuned on 8,340 Roboflow images to predict posture directly, in three classes. Replacing it wasn't a preference, it was a measurement. The eval harness replays every labelled fixture through six perturbations (horizontal flip, greyscale, darken 40%, 3px blur, downscale to 320px wide, and an 80% centre crop), and both the old detector and its replacement score perfectly on the clean images. Only the sweep separates them: my trained detector fell to 23 of 30, while the pose-plus-geometry pipeline held 30 of 30. The per-class breakdown said why. Under perturbation the sit class had precision 0.545 against recall 1.000. It never missed, and it was wrong nearly half the time it fired. Blur or shrink a standing person and the answer became sit at 0.63 to 0.88 confidence. That's a fallback attractor, the signature of a model keyed on scene appearance (bench, floor, indoor room) instead of on the body. More data or a bigger backbone wouldn't have fixed it, because the shortcut was built into the way the task was framed.
What replaced it is 17 keypoints and ordinary code
What ships is COCO-pretrained YOLOv8n-pose, unmodified. It's 12.9 MB, one class (person) and 17 keypoints, no fine-tuning, exported straight from the official release asset. It answers a far better conditioned question than three-way posture classification from 8,340 images: find the joints, on a problem trained across COCO's roughly 200,000 person instances with heavy augmentation. Posture is then decided in ordinary code from the geometry of those joints: torso angle off vertical, knee drop relative to torso length, knee interior angle, projected thigh-to-shin ratio, a signed hip-to-ankle drop, stance offset, and box aspect. Keypoint geometry has no equivalent shortcut to learn, because a shoulder-to-hip angle is the same blurred, greyscaled or 320px wide. What can still fail is finding the joints, and that's somebody else's well-solved problem. A nice side effect fell out of it, too. The pose model finds everyone in frame, so the street-fall fixture now returns the faller plus four background pedestrians where the old model saw one object.
A gate, not a vote
The thigh-to-shin rule is the clearest example of why I wrote this as an ordered chain instead of a scoring function. The femur and tibia are within about 10% of each other in real length, so their projected ratio sits near 1 whenever both lie in the image plane, which is what standing with vertical legs means. A thigh much shorter than its own shin can only mean the thigh points toward or away from the lens. The rule exists because of a real miss: a woman seated facing the camera with her legs stretched forward came back as stand at 63% confidence, because both other leg features genuinely read as standing, with knees well below hips at 0.64 and an almost straight leg at 172 degrees. Only the foreshortened thigh, 0.58 of its own shin, disagreed. Left as one signal among three, it would lose two to one in precisely the case it exists to catch. Overriding the other two is justified because this is a statement about projection geometry rather than a correlation: no standing pose puts a thigh at 0.58 of its shin. Measured separation backs the 0.75 threshold: standing subjects at 1.00 to 1.11, front-on seated at 0.47 and 0.58.
A man flat on his back, returned as sit
A 51-second aikido clip produced a frame of a man flat on his back returning sit at 0.77. Working out why it escaped taught me more than anything else in this project. His torso measured 25 degrees, under the 50-degree fall gate. His box aspect was 1.96, but the wide-box escape hatch also required 30 degrees of torso angle, so he missed it by 4.6. Falling through to the seated branch, his knee drop was minus 1.11: knees a full torso-length above his hips, which is a body on its back and never a seated posture. I'd already added a minimum on that feature for exactly this shape, on the reasoning that a fall relabelled as something calm is a missed alarm, but I'd applied it to the squat gate and nowhere else, so the leak didn't close, it just moved to sit. The wide-box condition was self-defeating in the same way: it existed for bodies foreshortened along the view axis, which have a low torso angle by construction, and then demanded a high one. I fixed both against the corpus instead of by argument. Among detections predicted sit, a knee drop below minus 1.00 splits 36 falls to 0 sits; the change recovers 254 falls against 9 sits, about 28 to 1. The floor sits at minus 0.25 instead of 0, because a genuine bench-sit with legs drawn up measures minus 0.16, and a floor at zero would have called that real sit a fall.
Every accuracy figure was measured on people who had already fallen
The fixture set reports 89.5% clean and 89.5% perturbed, macro-F1 0.901, and the repo says in as many words that this isn't a deployment accuracy, because the fixtures were picked to be hard. But the deeper problem was what nothing measured at all. The 2023 corpus is entirely accident scenes, so there was no population of confirmed not-fallen people to count false alarms against, and that's the one number that decides whether an anomaly detector is usable in a room where nothing is wrong. The POLAR posture dataset supplies it: 5,784 matched people across sit, squat and stand, with no falls, so every fall the system emits on it is wrong by construction. It calls 666 of them a fall (11.5%), and 470 of those, 8.1%, at a confidence the tracker would act on. Two-thirds of those are crouching, which is the deliberate kneeling trade-off finally carrying a number instead of an acknowledgement.
The tracker absorbs a crouch and cannot absorb a bed
A single frame can't tell a fall from someone bending over, so a tracker votes on each person's posture over a rolling window and only confirms a fall once it persists. Whether that actually absorbs the 11.5% was the last claim in the project resting on reasoning instead of measurement, so I measured it: the real detector run over the UR Fall dataset's 40 activity-of-daily-living clips (1,502 frames, five minutes, domestic interiors, no falls), feeding the real tracker in order. The per-frame rate came back at 11.5%, reproducing POLAR's figure exactly on a different dataset, in a different country, in a different decade, labelled by different people. Confirmed alarms: zero at 1 fps, zero at 2 fps, five at 5 fps. Frame rate changes the answer in the counter-intuitive direction, because the confirmation threshold is in milliseconds while the vote window is a fixed number of results, so the window spans 1.4 seconds at 5 fps and 7 seconds at 1 fps, and a brief crouch dominates a short window while getting diluted in a long one. The slower, worse-sounding rates are the robust ones.
The finding that spoils the good number
Three of those five alarms were transient: two men bending to pick something up and one kneeling. Two were a man lying on a bed. Reading zero-per-hour at 1 and 2 fps as proof the system doesn't alarm on someone in bed would have been exactly wrong. The only reason those two didn't confirm at the slower rates is that the clips are about seven and a half seconds long, so no posture in them lasts long enough for a slow sample rate to build up. That's a property of the footage and not of the system. Looping the settled-in-bed tail into 40 continuous seconds settles it: a confirmed fall at every rate, within 1.2 to 2.0 seconds, held for the full duration and never cleared. The tracker's only mechanism is whether a posture persisted, which separates a fall from a transient crouch and can't separate one from any sustained horizontal posture, because persistence is exactly what those two share. On the floor is a fall, on a bed isn't, and nothing in this system tells them apart. Fixing it needs a signal geometry doesn't carry: a floor and furniture zone mask, a ground-plane homography from a fixed camera, or the transition into the pose instead of the pose itself. I've written it down as the most deployment-relevant gap instead of leaving it for a demo to discover.
One learned component ships, at the narrowest scope that worked
A trained keypoint classifier was the obvious next step, so I built one and then rejected it. Testing it by holding out whole domains instead of shuffled images showed it didn't transfer as a unit: stand precision fell from 0.876 to 0.744, which was the criterion I'd set in advance. There's also a structural limit worth stating. The two corpora are nearly class-disjoint, with 1,970 falls in one and none at all in the other, so fall can't be cross-domain tested with local data however the experiment is arranged. But the measurement split the model cleanly: the squat half transferred where the fall half collapsed. So the squat half ships, as a binary probe consulted in exactly one situation, where the geometric gates have already decided sit, at full-visibility tier, on a body whose feet are under it. Three invariants make that safe structurally and not just behaviourally: the fall gates run earlier and return before it, so it can't suppress an alarm at any threshold or in any unseen domain; it never runs on a stand; and its worst case is a relabel between two non-alarming classes. Tests assert all three and exit non-zero if either of the first two breaks. It moved exactly one fixture, from 84.2% to 89.5%, and that's the honest size of it.