map pose and exit detection image

A Computer Vision-Based Indoor Wayfinding Tool

The ability to navigate safely and confidently is a fundamental requirement for independent travel and access to many settings such as work, school, shopping, transit and healthcare. Navigation is particularly challenging for people with visual impairments, who have limited ability to see signs, landmarks or maps posted in the environment. While a variety of GPS-based wayfinding aids are available for this population, the lack of GPS access indoors has meant that very few alternative sources of wayfinding information are available in this setting.

We are developing a smartphone-based indoor navigation assistance system that uses computer vision to estimate the user's location in an indoor environment and guide him/her to a desired destination. The advantages of our approach are (a) it runs on a standard smartphone and requires no new physical infrastructure, just a digital 2D map of the indoor environment that includes the locations of signs in it; and (b) it allows the user to walk freely without having to actively search for signs with the smartphone (which is challenging for people with severe visual impairments).

New: see a video demonstration of our navigation app in action, which issues turn-by-turn directions to guide the user to their desired destination.

Primary funding for this project is from a grant from NIH/NEI (R01EY029033).


Conference Papers
Fusco, G., Cheraghi, S. Ali, Neat, L., & Coughlan, J.. (2020). An Indoor Navigation App using Computer Vision and Sign Recognition. In 17th International Conference on Computers Helping People with Special Needs.
Fusco, G., & Coughlan, J.. (2020). Indoor Localization for Visually Impaired Travelers Using Computer Vision on a Smartphone. In 17th International Web for All Conference: Automation for Accessibility.
Biggs, B., Coughlan, J., & Coppin, P.. (2019). Design and Evaluation of an Audio Game-Inspired Auditory Map Interface. In The 25th International Conference on Auditory Display (ICAD 2019). Northumbria University, Newcastle-upon-Tyne, UK: Northumbria University, Newcastle-upon-Tyne, UK.
Fusco, G., & Coughlan, J.. (2018). Indoor Localization using Computer Vision and Visual-Inertial Odometry. In International Conference on Computers Helping People with Special Needs (ICCHP '18). Linz, Austria: Linz, Austria.
Rituerto, A., Fusco, G., & Coughlan, J.. (2016). Towards a Sign-Based Indoor Navigation System for People with Visual Impairments. In 18th International ACM SIGACCESS Conference on Computers and Accessibility. ACM: Reno, NV.
  • Collage of RERC staff mebers and RERC projects

    Rehabilitation Engineering Research Center

    The Center's research goal is to develop and apply new scientific knowledge and practical, cost-effective devices to better understand and address the real-world problems of blind, visually impaired, and deaf-blind consumers

    Read More