I participated in Hack the North 2017 this week and built a really cool AI wearable assistant for blind people with my fabulous teammates! We won the IBM sponsor prize with this hackathon project! Many thanks to my teammates and Hack the North, it is really an excellent experience! Have a look at the Devpost page for our project here: https://devpost.com/software/third-eye-1jgh52
With 1 million Canadians who are visually impaired and and 100,000 who are blind, the number is only predicted to increase in the coming years. However, the technology has not been following the demand for devices to assist those who cannot see. To make a product to change their lives for the better has been the inspiration. Bring back one of the six senses.
What it does
Third Eye is the computer vision assistant for the visually impaired or the blind with the goal to create an affordable wearable device that uses live video feed from the glasses’ camera to identify and warn users of obstacles, facial recognition to tell them who it is that is in front of them, and lets them know what the object is using computer vision. Using cutting edge technology of depth perception, cloud processing, computer vision, and speech to text machine learning software, Third Eye provides another level of freedom for those who have lost a sense and giving them a new one, the Third Eye.
How we built it
We use the Raspberry Pi Suite of the Pi Camera, distance and depth Pi sensor, and wireless microphone for the purpose of feeding live video feed for computer vision analysis, object classification, distance/potential obstacle analysis, and facial recognition. The hardware side was mostly programmed in Python while the servers and the cloud based processing were programmed in PHP and working with tools such as IBM Watson and Google video intelligence API.
Challenges we ran into
The first problem we encountered was the lack of hardware such as a raspberry pi camera and a SIM card so we could not boot up the Raspberry Pi for the first day throwing our plans and time schedule to ciaos. But with multitasking, giving up some features in the race of time, and flexibly adapting to the change allowed us to end with a working product although not to our highest satisfaction.
Accomplishments that we are proud of
Even with many hardware drawbacks, we still accomplished most of what we wanted through all the late nights. We are so happy we could bring a product that could instantly improve the lives of so many people around the world.
What we learned
We learned how to use the Raspberry Pi Suite for the live video feed for computer vision analysis, object classification, distance/potential obstacle analysis, and facial recognition. We also learned how to work with Python on the servers and the cloud based processing in PHP as well as tools such as IBM Watson and Google video intelligence API. Last but now least, teamwork!
What’s next for Third Eye
Incorporating further learning algorithms and training to continue to improve recognition of objects and facial features of people. We will continue to also improve our speech recognition AI and location direction instructions to be more descriptive.