A hazard is different when you cannot see it
Percepta began as a three-day hackathon project built around one accessibility question: how could a person with a visual impairment get useful information about hazards in their path? The idea used a small camera mounted to glasses, a machine-learning model that classified the scene, and audio feedback that could warn them about a possible obstacle.
The project also had a community side. People who could see hazards could add them to a shared map. This gave users another source of information about places that might be difficult or dangerous to navigate.
Percepta's goal was to make hazards audible without making users rely on another visual interface.
The dataset was usable, but not ready for a safety-critical claim
I used a Kaggle hazard dataset with roughly 13 classes, including potholes, cones, and stairs. Most classes contained a few hundred images, but at least one had only around 24 examples. I checked the class distribution but did not yet use class weighting, oversampling, or augmentation to address the imbalance.
An overall score can hide the classes a system recognises poorly. At the time, I was still learning how to evaluate machine-learning models. The dataset and model gave me a fast route to a demonstration, but they were not evidence for a deployable accessibility tool.
A small CNN made the idea tangible
I built a lightweight custom CNN from scratch rather than using transfer learning. I had only three days and limited experience in deep learning, so I was learning what the model pipeline did while producing the model.
The model performed image classification: it assigned one hazard label to an image rather than locating objects with bounding boxes. This made the first implementation simpler, but it also exposed an important product constraint. A real scene can contain several hazards at once. This model could describe only one image-level class.
The model was one part of the accessibility idea
The intended experience connected a glasses-mounted camera to the classifier and an audio layer. ElevenLabs supplied the voice-alert direction, while OpenStreetMap was the basis for a community map of hazards. In the demo, the map was mainly a visual overview. Its hard-coded points represented reports that community members could contribute in a fuller version.
We did not build the complete camera hardware during the hackathon. The model notebook used example images or video frames to show how camera input could be processed. A teammate designed the glasses enclosure in Autodesk Fusion. I handled the AI and software work, most of the product concept, the map experience, and part of the presentation.
The demo worked, but the evaluation was only a first step
The model reached approximately 65% training accuracy and 67% test accuracy. I plotted training and validation loss and accuracy curves, but did not yet use a confusion matrix or per-class precision, recall, or F1 scores. The numbers show that the classifier learned something from the dataset. They do not show that it could reliably identify every hazard a user might encounter.
Percepta won first place in the beginner category at the 2025 BuildingBloCS June Conference hackathon. That result showed that the concept was clear and the demonstration was strong. It did not validate the model's safety or deployment readiness.
I would start with the data now
Percepta is a hackathon proof of concept. The camera hardware, real-time capture path, and reliable proximity behavior remained future work. The model was not evaluated with visually impaired users, so this was not a finished assistive device.
If I rebuilt it today, I would follow the full machine-learning workflow: exploratory data analysis, cleaning, visualisation, preprocessing, augmentation, experiments across different seeds, model comparison, and class-level evaluation. Getting a model to run was only the beginning. The evaluation has to explain where it works, where it fails, and whether the result is trustworthy for the people who depend on it.