Coughlan Lab

Principal Investigator:

James Coughlan

Contact Information:

Email: coughlan@ski.org

Mobile Phone: (415) 345-2146

2318 Fillmore St.
San Francisco, CA 94115

The goal of our laboratory is to develop and test assistive technology for blind and visually impaired persons that is enabled by computer vision and other sensor technologies.

L to R: Huiying Shen, Ali Cheraghi, Brandon Biggs, James Coughlan, Charity Pitcher-Cooper, Giovanni Fusco

Publications

Navigation graph illustrating blind travelers' navigation paths and current target indicator, featuring nodes (n1 to n5) and associated navigation aids (NA(e1) to NA(e5)).

S. Mascetti, Dr. Ahmetovic, G. Galimberti and J. Coughlan. “NavGraph: Enhancing Blind Travelers’ Navigation Experience and Safety.” Proc. ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Volume 9, Issue 3, Article No.: 117, Pages 1 – 27. 2025.

Hand pointing at tactile map of 20 W 34th St. with Braille labels and an audio device for interactive navigation.

Manzoni, M. ., Mascetti, S. ., Ahmetovic, D. ., Crabb, R. ., & Coughlan, J. . (2025). MapIO: a Gestural and Conversational Interface for Tactile Maps. IEEE Access.

Bar chart displaying accessibility attainment percentages for various web map tools, including Google Maps, Bing Maps, and Audiom, relevant to assistive technology research for visually impaired users.

Biggs, B. ., Coughlan, J. ., & Walker, B. (2025). Systematically Evaluating Digital Map Tools Based on the WCAG. Journal on Technology and Persons With Disabilities, 13.

Map of confirmed COVID-19 cases in the U.S. by state and territory as of June 11, 2020, displaying varying shades of red to indicate case numbers, with specific figures for each state.

Biggs, B. ., Agbaroji, H. ., Toth, C. ., Stockman, T. ., Coughlan, J. ., & Walker, B. . (2024). Co-Designing Auditory Navigation Solutions for Traveling as a Blind Individual During The COVID-19 Pandemic. Journal of Blindness Innovation and Research, 14(1). (Original work published 2024)

Crosswalk scene featuring a pedestrian signal indicating "walk," highlighting urban navigation and accessibility in the context of eye research and visual perception studies.

Hong, J. ., & Coughlan, J. . (2023). VR Training to Facilitate Blind Photography for Navigation. Journal On Technology And Persons With Disabilities. (Original work published 2023)

Map illustrating a navigation route with colored arrows indicating direction and waypoints, relevant to Seyed Cheraghi's research on aiding blind and visually impaired travelers.

Crabb, R. ., Cheraghi, S. A., & Coughlan, J. . (2023). A Lightweight Approach to Localization for Blind and Visually Impaired Travelers. Sensors, 23(5). https://doi.org/10.3390/s23052701 (Original work published 2023)

Biggs, B. ., Pitcher-Cooper, C. ., & Coughlan, J. . (2022). Getting in Touch with Tactile Map Automated Production: Evaluating Impact and Areas for Improvement.Journal on Technology and Persons With Disabilities, 10. (Original work published 2022)

Logo of the MacKeben Lab, featuring a stylized eye design in blue, representing vision research related to eye movement training and gaze strategies.

Tekin, E. ., Vásquez, D. ., & Coughlan, J. . (2013). S-K Smartphone Barcode Reader for the Blind. Journal on Technology and Persons With Disabilities, 1.

Logo of the MacKeben Lab, featuring a stylized eye design in blue, representing vision research related to eye movement training and gaze strategies.

Coughlan, J. ., & Shen, H. . (2013). Crosswatch: a system for providing guidance to visually impaired travelers at traffic intersections. Journal of Assistive Technologies, 7.

Logo of the MacKeben Lab, featuring a stylized eye design in blue, representing vision research related to eye movement training and gaze strategies.

Manduchi, R. ., & Coughlan, J. . (2012). (Computer) vision without sight. Communications of the ACM, 55, 96–104.

Logo of the MacKeben Lab, featuring a stylized eye design in blue, representing vision research related to eye movement training and gaze strategies.

Shen, H. ., Coughlan, J. ., & Ivanchenko, V. . (2009). Figure-ground segmentation using factor graphs. Image and Vision Computing, 27, 854–863.

Logo of the MacKeben Lab, featuring a stylized eye design in blue, representing vision research related to eye movement training and gaze strategies.

Coughlan, J. ., & Manduchi, R. . (2009). Functional assessment of a camera phone-based wayfinding system operated by blind and visually impaired users. International Journal on Artificial Intelligence Tools, 18, 379–397.

Hand pointing at a drawing of an airplane wing, with highlighted features in green, illustrating tactile graphics for educational purposes related to Khadija Ezrouri's research on accessibility in visual materials.

Ahmetovic, D., Coughlan, J., Dal Santo, G., Ezrouri, K., Manzoni, M., & Mascetti, S. (2025). CamIO in the Browser: A Cross-Platform Audio Label Tool for Tactile Graphics. MobileHCI 2025

Interactive 3D tactile map of a playground with raised structures and QR code markers, hand pointing at a feature, illustrating non-visual access for spatial learning.

Coughlan, J. ., Biggs, B. ., & Shen, H. . (2022). Non-Visual Access to an Interactive 3D Map. Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE ’22). Presented at the Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE ’22). (Original work published 2022)

White headphones resting next to a Samsung smartphone displaying a forest wallpaper, illustrating auditory technology in the context of publications on auditory navigation for blind individuals.

Biggs, B. ., Toth, C. ., Stockman, T. ., Coughlan, J. ., & Walker, B. N. (2022). Evaluation of a Non-Visual Auditory Choropleth and Travel Map Viewer. International Conference on Auditory Display (ICAD) 2022. Presented at the International Conference on Auditory Display (ICAD) 2022. Virtual. Retrieved from PMCID: PMC10010675 (Original work published 2022)

3D tactile model of a landscape with various structures and trees, used for research on vision and spatial awareness, featuring QR code markers for interactive elements.

Coughlan, J. ., Biggs, B. ., & Shen, H. . (2021). Point and Listen: Bringing a 3D Map to Life with Audio-Based AR. 6th Annual Frameless XR Symposium. Presented at the 6th Annual Frameless XR Symposium. (Original work published 2021)

Biggs, B. ., Coughlan, J. ., & Coppin, P. . (2021). Design and evaluation of an interactive 3D map. RESNA 2021 Conference. Presented at the RESNA 2021 Conference. (Original work published 2021)

Indoor navigation diagram illustrating a wayfinding system with marked paths, elevator location, and coordinate points (x,y) and (x',y').

Fusco, G. ., Cheraghi, S. A., Neat, L. ., & Coughlan, J. . (2020). An Indoor Navigation App using Computer Vision and Sign Recognition. 17th International Conference on Computers Helping People With Special Needs. Presented at the 17th International Conference on Computers Helping People with Special Needs. (Original work published 2020)

Hand holding a measuring tool over a toy fire truck labeled "RESCUE," positioned on a patterned sheet with QR codes, illustrating a potential application in research related to navigation or assistive technologies.

Coughlan, J. ., Biggs, B. ., Rivière, M.-A. ., & Shen, H. . (2020). An Audio-Based 3D Spatial Guidance AR System for Blind Users. 17th International Conference on Computers Helping People With Special Needs. Presented at the 17th International Conference on Computers Helping People with Special Needs. (Original work published 2020)

Biggs, B. ., Coughlan, J. ., & Coppin, P. . (2019). Design and Evaluation of an Audio Game-Inspired Auditory Map Interface. The 25th International Conference on Auditory Display (ICAD 2019). Presented at the The 25th International Conference on Auditory Display (ICAD 2019). Northumbria University, Newcastle-upon-Tyne, UK. (Original work published 2019)

Fusco, G. ., & Coughlan, J. . (2018). Indoor Localization using Computer Vision and Visual-Inertial Odometry. International Conference on Computers Helping People With Special Needs (ICCHP ’18). Presented at the International Conference on Computers Helping People with Special Needs (ICCHP ’18). Linz, Austria. (Original work published 2018)

Fusco, G. ., & Morash, V. S. (2015). The Tactile Graphics Helper: Providing Audio Clarification for Tactile Graphics Using Machine Vision. ASSETS. Presented at the ASSETS. (Original work published 2015)

Manduchi, R. ., & Coughlan, J. . (2014). The last meter: blind visual guidance to a target. Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, 3113–3122. ACM.

Murali, V. N., & Coughlan, J. . (2013). Smartphone-based crosswalk detection and localization for visually impaired pedestrians. Multimedia and Expo Workshops (ICMEW), 2013 IEEE International Conference on, 1–7. IEEE. (Original work published 2013)

Projects

  • Other
  • Active
  • Completed
Data Science logo featuring AI elements, sound wave, and upward trend graph, representing the Empowering Data Vision course for blind individuals at Smith-Kettlewell Eye Research Institute.
Active

Empowering Data Vision: A Self-Paced Introduction to the Convergence of AI and Data Science

Read more
Audiom map of Smith-Kettlewell building layout, highlighting key areas for navigation, designed for accessibility for visually impaired individuals.
Active

Audiom Map of Smith-Kettlewell

An Audiom map of the main Smith-Kettlewell building at 2318 Fillmore St., San Francisco has been created by Brandon Biggs. This is an audio-visual map that allows users to explore a detailed map of the building with or without vision. The map runs in any browser and is available to anyone who visits the building.
Read more
Hand touching a tactile map with Braille labels and raised lines, alongside an interactive audio device, illustrating assistive technology for navigation and spatial awareness for visually impaired users.
Active

MapIO: a Gestural and Conversational Interface for Tactile Maps

For individuals who are blind or have low vision, tactile maps provide essential spatial information but are limited in the amount of data they can convey. Digitally augmented tactile maps enhance these capabilities with audio feedback, thereby combining the tactile feedback provided by the map with an audio description of the touched elements. In this context, we explore an embodied interaction paradigm to augment tactile maps with conversational interaction based on Large Language Models, thus enabling users to obtain answers to arbitrary questions regarding the map. We analyze the type of…
Read more
Person with visual impairment using a walk light detector app at a traffic intersection. Left photo shows user holding smartphone, right photo shows image acquired by app of traffic intersection including walk light.
Active

Using VR to Help Train Visually Impaired Users to Aim a Camera

People with visual impairments increasingly rely on camera-enabled smartphone apps for tasks like photography, navigation, and text recognition. Despite the growing use of these applications, precise camera aiming remains a significant challenge. This project explores the impact of virtual reality (VR) exploration in the context of learning to use a camera-based app. So far we have studied this approach in the context of training a visually impaired person to use walk-light detector app at traffic intersections.
Read more
Close-up photo showing person pointing right index finger to Ireland on a map of the British Isles with other fingers closed.
Active

CamIO Hands

This project builds on the CamIO project to provide point-and-tap interactions allowing a user to acquire detailed information about tactile graphics and 3D models. The interface uses an iPhone’s depth and color cameras to track the user’s hands while they interact with a model. When the user points to a feature of interest on the model with their index finger, the system reads aloud basic information about that feature. For additional information, the user lifts their index finger and taps the feature again. This process can be repeated multiple times to access additional levels of…
Read more
3D bronze model of the Magical Bridge Playground, featuring play structures and pathways, with a colorful informational sign highlighting accessibility for all abilities, including visually impaired visitors.
Active

Magic Map

The Magic Map is an interactive 3D map installed at the Magical Bridge Playground in Palo Alto, California. It consists of a 1/100 scale 3D bronze representation of the playground, which includes over seventy play structures organized into multiple play zones and paths. When the user's index fingertip touches a specific feature on the map, the name and description of the feature are read aloud in audio. This interactivity allows visitors with visual impairments to navigate the map without requiring them to read braille.
Read more
Smith-Kettlewell Eye Research Institute representatives engaging with attendees at the FVA Conference, showcasing assistive technology and resources for visual impairments, with informational materials and a "BLINDSPOT" display.
Active

Outreach at Smith-Kettlewell

Smith-Kettelwell is deeply committed to supporting our community. Please see the news tab for our upcoming and ongoing events. In the News tab, you will find information about our involvement in past outreach activities.
Read more
Person using a Lenovo laptop to interact with an audio-based mapping tool, demonstrating commands for navigation, relevant to accessibility for visually impaired users.
Active

Audiom

Audiom is a tool that allows blind and visually impaired individuals to view maps completely in audio. It is a web component and can be embedded into any webpage, similar to Google Maps. It allows non-visual use of route, landmark, and survey knowledge, which is the critical information needed for navigation.
Read more
Magnifying glass focusing on "Low Vision" text in a newspaper, highlighting topics relevant to low vision support and community outreach.
Active

Blindness and Low Vision Support Group

Join Dr. Don Fletcher, one of the world’s leading authorities on Low Vision Rehabilitation, to share experiences and learn about the things that help you maintain a full and happy life while living with low vision.
Read more
map pose and exit detection image
Active

A Computer Vision-Based Indoor Wayfinding Tool

The ability to navigate safely and confidently is a fundamental requirement for independent travel and access to many settings such as work, school, shopping, transit and healthcare. Navigation is particularly challenging for people with visual impairments, who have limited ability to see signs, landmarks or maps posted in the environment.
Read more
Virtual aerial view of intersection area near a pedestrian's feet, reconstructed by Crosswatch algorithms
Completed

Crosswatch

Crosswatch is a smartphone-based system developed for providing real-time guidance to blind and visually impaired travelers at traffic intersections. Using the smartphone's built-in camera and other sensors, Crosswatch is designed to tell blind and visually impaired travelers what kind of traffic intersection they are near, how to align themselves properly to the crosswalk, and when the walk light or other traffic light indicates it is time to cross. Click here  for link to zip file containing Crosswatch code (open source) and documentation.
Read more

Centers

People

Current People

Past People

Collaborators

Internal Collaborators

External Collaborators

News

Events

Event Category

Event Type

Get Involved

If you are interested in vision science or want to learn more about low vision and blindness, there are many opportunities to get involved at The Smith-Kettlewell Eye Research Institute.

Donate
Participate in a study
Patient undergoing eye examination with Optos OCT SLO device, assisted by researchers in a laboratory setting focused on vision science and eye-tracking accuracy.