James Coughlan

Photo of James Coughlan wearing glasses and blue shirt
Director - Rehabilitation Engineering Research Center
Ph.D.

The goal of our laboratory is to develop and test access technology for blind and visually impaired persons that is enabled by computer vision and other sensor technologies.

Download CV

Publications

Journal Articles

Crabb, R., Cheraghi, S. A., & Coughlan, J. (2023). A Lightweight Approach to Localization for Blind and Visually Impaired Travelers. Sensors, 23(5). https://doi.org/10.3390/s23052701 (Original work published 03/2023 C.E.)
Hong, J., & Coughlan, J. (2023). VR Training to Facilitate Blind Photography for Navigation. Journal On Technology And Persons With Disabilities. (Original work published 06/2023 C.E.)
Pitcher-Cooper, C., Seth, M., Kao, B., Coughlan, J., & Yoon, I. (2023). You Described, We Archived: A Rich Audio Description Dataset. Journal On Technology And Persons With Disabilities, 11. (Original work published 03/2023 C.E.)
Biggs, B., Pitcher-Cooper, C., & Coughlan, J. (2022). Getting in Touch with Tactile Map Automated Production: Evaluating Impact and Areas for Improvement. Journal On Technology And Persons With Disabilities, 10. (Original work published 05/2022 C.E.)
Cheraghi, S. A., Fusco, G., & Coughlan, J. (2021). Real-Time Sign Detection for Accessible Indoor Navigation. Journal On Technology And Persons With Disabilities, 9. (Original work published 06/2021 C.E.)
Coughlan, J., Shen, H., & Biggs, B. (2020). Towards Accessible Audio Labeling of 3D Objects. Journal On Technology And Persons With Disabilities, 8. (Original work published 04/2020 C.E.)
Ahmetovic, D., Manduchi, R., Coughlan, J., & Mascetti, S. (2017). Mind your crossings: Mining GIS imagery for crosswalk localization. Acm Transactions On Accessible Computing (Taccess), 9(4). 10.1145/3046790
Coughlan, J., & Shen, H. (2013). Crosswatch: a system for providing guidance to visually impaired travelers at traffic intersections. Journal Of Assistive Technologies, 7.
Tekin, E., Vásquez, D., & Coughlan, J. (2013). S-K Smartphone Barcode Reader for the Blind. Journal On Technology And Persons With Disabilities, 1.
Tekin, E., Coughlan, J., & Simon, H. J. (2013). Improving speech enhancement algorithms by incorporating visual information. The Journal Of The Acoustical Society Of America, 134, 4237–4237.

Conference Papers

Biggs, B., Toth, C., Stockman, T., Coughlan, J., & Walker, B. N. (2022). Evaluation of a Non-Visual Auditory Choropleth and Travel Map Viewer. International Conference on Auditory Display (ICAD) 2022. Presented at the. Virtual. Retrieved de PMCID: PMC10010675 (Original work published 07/2022 C.E.)
Coughlan, J., Biggs, B., & Shen, H. (2022). Non-Visual Access to an Interactive 3D Map. Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE '22). Presented at the. (Original work published 07/2022 C.E.)
Coughlan, J., Biggs, B., & Shen, H. (2021). Point and Listen: Bringing a 3D Map to Life with Audio-Based AR. 6th Annual Frameless XR Symposium. Presented at the. (Original work published 11/2021 C.E.)
Biggs, B., Coughlan, J., & Coppin, P. (2021). Design and evaluation of an interactive 3D map. RESNA 2021 Conference. Presented at the. (Original work published 07/2021 C.E.)
Coughlan, J., Biggs, B., Rivière, M. -A., & Shen, H. (2020). An Audio-Based 3D Spatial Guidance AR System for Blind Users. 17th International Conference on Computers Helping People with Special Needs. Presented at the. (Original work published 09/2020 C.E.)
Fusco, G., Cheraghi, S. A., Neat, L., & Coughlan, J. (2020). An Indoor Navigation App using Computer Vision and Sign Recognition. 17th International Conference on Computers Helping People with Special Needs. Presented at the. (Original work published 09/2020 C.E.)
Fusco, G., & Coughlan, J. (2020). Indoor Localization for Visually Impaired Travelers Using Computer Vision on a Smartphone. 17th International Web for All Conference: Automation for Accessibility. Presented at the. (Original work published 04/2020 C.E.)
Biggs, B., Coughlan, J., & Coppin, P. (2019). Design and Evaluation of an Audio Game-Inspired Auditory Map Interface. The 25th International Conference on Auditory Display (ICAD 2019). Presented at the. Northumbria University, Newcastle-upon-Tyne, UK. (Original work published 06/2019 C.E.)
Fusco, G., & Coughlan, J. (2018). Indoor Localization using Computer Vision and Visual-Inertial Odometry. International Conference on Computers Helping People with Special Needs (ICCHP '18). Presented at the. Linz, Austria. (Original work published 07/2018 C.E.)
Coughlan, J., & Miele, J. (2017). AR4VI: AR as an Accessibility Tool for People with Visual Impairments. AR for Good, part of ISMAR 2017. Presented at the. Nantes, France: IEEE. (Original work published 10/2017 C.E.)

Projects

CamIO

CamIO (short for “Camera Input-Output”) is a system to make physical objects (such as documents, maps, devices and 3D models) accessible to blind and visually impaired persons, by providing real-time audio feedback in response to the location on an object that the user is touching. CamIO currently works on iOS using the built-in camera and an inexpensive hand-held stylus, made out of materials such as 3D-printed plastic, paper or wood.

See a video demonstration of CamIO here, showing how the user can trigger audio labels by pointing a stylus at "hotspots" on a 3D map of a playground. See the Ma…

Magic Map

The Magic Map is an interactive 3D map installed at the Magical Bridge Playground in Palo Alto, California. It consists of a 1/100 scale 3D bronze representation of the playground, which includes over seventy play structures organized into multiple play zones and paths. When the tip of the "Magic Wand" tethered to the map is pointed to a specific feature on the map, the name and description of the feature are read aloud in audio. This interactivity makes the map accessible to visitors with visual impairments, and without requiring them to read braille.

Sign Finder

This project seeks to develop a computer vision-based system that allows a visually impaired traveler to find and read informational signs, such as signs labeling office doors, streets, restrooms and Exit signs.

Link to open source code

Tactile Graphics Helper (TGH)

Tactile graphics use raised lines, textures, and elevations to provide individuals with visual impairments access to graphical materials through touch. Tactile graphics are particularly important for students in science, technology, engineering, and mathematics (STEM) fields, where educational content is often conveyed using diagrams and charts. However, providing a student who has a visual impairment with a tactile graphic does not automatically provide the student access to the graphic's educational content. Instead, the student may struggle to decipher subtle differences between textures or…

Talking Signs

Created by William Loughborough in 1979, Talking Lights was a system of infrared transmitters and receivers allowing blind and visually impaired travelers to quickly and easily "read signs" at a distance.

Video-Based Speech Enhancement for Persons with Hearing and Vision Loss

Observing the visual cues from a speaker such as the shape of the lips and facial expression can greatly improve the speech comprehension capabilities of a person with hearing loss. However, concurrent vision loss can lead to a significant loss in speech perception. We propose developing a prototype device that utilizes a video camera in addition to audio input to enhance the speech signal from a target speaker in everyday situations.

ZoomBoard: an Affordable, Portable System to Improve Access to Presentations and Lecture Notes for Low Vision Viewers

The goal of the project is to develop a “ZoomBoard” system that students with low vision can use to better access visual material on a whiteboard or blackboard. The prototype version of the system that we plan to develop in this grant will consist of a dedicated camera system placed by the teacher to capture a view of the board, which wirelessly transmits a video stream that will be displayed on a student’s iPad. The student will use the ZoomBoard app to view this video stream, zoom in on any region of interest using a pinch gesture on the iPad, and apply image enhancements such as contrast…

Centers

Video Description Research and Development Center

The Smith-Kettlewell Video Description Research and Development Center (VDRDC) investigates innovative technologies and techniques for making online video more accessible to blind and visually-impaired students and consumers. Through collaboration with a broad array of partners and stakeholders in the Description Leadership Network, we are developing advanced video annotation methods for use in a wide variety of educational settings, as well as helping educators and other description providers make better use of the tools already available.

Collaborators

Internal

Joshua Miele photo

Joshua Miele

Principal Accessibility Researcher, Amazon Lab126

External

Trainees

Principal Investigators

Current

Past

Ender Tekin

Ender Tekin

Associate Scientist at The University of Wisconsin-Madison
Photograph of Laura Walker

Laura Walker

Senior Visual Experience Engineer, Apple, Cupertino CA

Research Staff

Current

Interns

Current

Research

Current

Past

Photo of Vidya Murali

Vidya Murali

Technical Expert in Applied Machine Learning at Ford Research
Photo of Pannag Sanketi

Pannag Sanketi

Tech Lead Manager/Senior Staff Software Engineer at Google
Ender Tekin

Ender Tekin

Associate Scientist at The University of Wisconsin-Madison
Photograph of Laura Walker

Laura Walker

Senior Visual Experience Engineer, Apple, Cupertino CA

SKERI Staff

Current