Coughlan Lab
James Coughlan
Senior Scientist, Director: Rehabilitation Engineering Research Center
Degrees: Ph.D.
The goal of our laboratory is to develop and test access technology for blind and visually impaired persons that is enabled by computer vision and other sensor technologies.
Tabs
Journal Articles
. (2023). VR Training to Facilitate Blind Photography for Navigation. Journal On Technology And Persons With Disabilities.
. (2023). A Lightweight Approach to Localization for Blind and Visually Impaired Travelers. Sensors, 23(5). http://doi.org/https://doi.org/10.3390/s23052701
. (2023). You Described, We Archived: A Rich Audio Description Dataset. Journal On Technology And Persons With Disabilities, 11.
. (2022). Getting in Touch with Tactile Map Automated Production: Evaluating Impact and Areas for Improvement. Journal On Technology And Persons With Disabilities, 10.
. (2021). Real-Time Sign Detection for Accessible Indoor Navigation. Journal On Technology And Persons With Disabilities, 9.
. (2020). Towards Accessible Audio Labeling of 3D Objects. Journal On Technology And Persons With Disabilities, 8.
. (2017). Mind your crossings: Mining GIS imagery for crosswalk localization. Acm Transactions On Accessible Computing (Taccess), 9(4). http://doi.org/10.1145/3046790
. (2013). Crosswatch: a system for providing guidance to visually impaired travelers at traffic intersections. Journal Of Assistive Technologies, 7.
. (2013). S-K Smartphone Barcode Reader for the Blind. Journal On Technology And Persons With Disabilities, 1.
. (2013). Improving speech enhancement algorithms by incorporating visual information. The Journal Of The Acoustical Society Of America, 134, 4237–4237.
. (2010). An Embarrassingly Simple Speed-Up of Belief Propagation with Robust Potentials. Arxiv Preprint Arxiv:1010.0012.
. (2009). A mobile phone wayfinding system for visually impaired users. Assistive Technology Research Series, 25, 849.
. (2009). Functional assessment of a camera phone-based wayfinding system operated by blind and visually impaired users. International Journal On Artificial Intelligence Tools, 18, 379–397.
. (2009). Staying in the crosswalk: A system for guiding visually impaired pedestrians at traffic intersections. Assistive Technology Research Series, 25, 69.
. (2008). A mobile phone system to find crosswalks for visually impaired pedestrians. Technology And Disability, 20, 217–224.
. (2008). Portable and Mobile Systems in Assistive Technology-Portable and Mobile Systems in Assistive Technology: Introduction to the Special Thematic Session. Lecture Notes In Computer Science, 5105, 1078.
. (2007). Color targets: Fiducials to help visually impaired people find their way by camera phone. Eurasip Journal On Image And Video Processing, 2007.
. (2007). Dynamic quantization for belief propagation in sparse spaces. Computer Vision And Image Understanding, 106, 47–58.
. (2007). Functional assessment of a camera phone-based wayfinding system operated by blind users. Ieee Computer Society And The Biological And Artificial Intelligence Society.
. (2006). A fast algorithm for finding crosswalks using figure-ground segmentation. 2Nd Workshop On Applications Of Computer Vision, In Conjunction With Eccv, 5.
. (2005). An information maximization model of eye movements. Advances In Neural Information Processing Systems, 17, 1121-8.
. (2005). Rapid and robust algorithms for detecting colour targets. 10Th Congress Of The International Colour Association, Aic Colour, 5, 959–962.
. (2003). The KGBR viewpoint-lighting ambiguity. Journal Of The Optical Society Of America (Josa) A, 20(1).
. (2003). A large deviation theory analysis of Bayesian tree search. Ima Volumes In Mathematics And Its Applications, 133, 1–18.
. (2003). A statistical approach to multi-scale edge detection. Image And Vision Computing, 21, 37–48.
. (2003). Algorithms from statistical physics for generative models of images. Image And Vision Computing, 21, 29–36.
. (2003). Manhattan world: Orientation and outlier detection by bayesian inference. Neural Computation, 15, 1063–1088.
. (2003). Statistical edge detection: Learning and evaluating edge cues. Pattern Analysis And Machine Intelligence, Ieee Transactions On, 25, 57–74.
. (2003). The generic viewpoint assumption and planar bias. Ieee Transactions On Pattern Analysis And Machine Intelligence, 25, 775–778.
. (2002). Bayesian A* tree search with expected O (N) node expansions: applications to road tracking. Neural Computation, 14, 1929–1958.
. (2002). Fundamental bounds on edge detection: learning and evaluating edge cues. Pattern Anal. Machine Intell.
. (2001). Order Parameters for Detecting Target Curves in Images: When does high level knowledge help?. International Journal Of Computer Vision, 41, 9–33.
. (2001). The KGBR viewpoint-lighting ambiguity and its resolution by generic constraints. Computer Vision, 2001. Iccv 2001. Proceedings. Eighth Ieee International Conference On, 2, 376–382.
. (2000). An A∗ perspective on deterministic optimization for deformable templates. Pattern Recognition, 33, 603–616.
. (2000). Efficient deformable template detection and localization without user initialization. Computer Vision And Image Understanding, 78, 303–319.
. (2000). Fundamental limits of Bayesian inference: order parameters and phase transitions for road tracking. Pattern Analysis And Machine Intelligence, Ieee Transactions On, 22, 160–173.
. (2000). The generic viewpoint constraint resolves the generalized bas relief ambiguity. Proc. Of Conference On Information Scienes And Systems (Ciss 2000), 15–17.
. (1999). Visual search: Fundamental bounds, order parameters, and phase transitions. Proc Ieee Workshop On Statistical And Computational Theories Of Vision. Cvpr 1999. Fort Collins, Co. June 1999.
. (1999). From Generic to Specific: An Information Theoretic Perspective on the Value of High-Level Information. Probabilistic Models Of The Brain, 135.
Conference Papers
. (2022). Evaluation of a Non-Visual Auditory Choropleth and Travel Map Viewer. In International Conference on Auditory Display (ICAD) 2022. Virtual: Virtual. Retrieved from PMCID: PMC10010675
. (2022). Non-Visual Access to an Interactive 3D Map. In Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE '22).
. (2021). Point and Listen: Bringing a 3D Map to Life with Audio-Based AR. In 6th Annual Frameless XR Symposium.
. (2021). Design and evaluation of an interactive 3D map. In RESNA 2021 Conference.
. (2020). An Audio-Based 3D Spatial Guidance AR System for Blind Users. In 17th International Conference on Computers Helping People with Special Needs.
. (2020). An Indoor Navigation App using Computer Vision and Sign Recognition. In 17th International Conference on Computers Helping People with Special Needs.
. (2020). Indoor Localization for Visually Impaired Travelers Using Computer Vision on a Smartphone. In 17th International Web for All Conference: Automation for Accessibility.
. (2019). Design and Evaluation of an Audio Game-Inspired Auditory Map Interface. In The 25th International Conference on Auditory Display (ICAD 2019). Northumbria University, Newcastle-upon-Tyne, UK: Northumbria University, Newcastle-upon-Tyne, UK.
. (2018). Indoor Localization using Computer Vision and Visual-Inertial Odometry. In International Conference on Computers Helping People with Special Needs (ICCHP '18). Linz, Austria: Linz, Austria.
. (2017). AR4VI: AR as an Accessibility Tool for People with Visual Impairments. In AR for Good, part of ISMAR 2017. IEEE: Nantes, France.
. (2017). Evaluating Author and User Experience for an Audio-Haptic System for Annotation of Physical Models. In 19th Int’l ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2017). ACM: Baltimore, MD.
. (2017). JustPoint: Identifying Colors with a Natural User Interface. In 19th Int’l ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2017). ACM: Baltimore, MD.
. (2016). Towards a Sign-Based Indoor Navigation System for People with Visual Impairments. In 18th International ACM SIGACCESS Conference on Computers and Accessibility. ACM: Reno, NV.
. (2015). Appliance Displays: Accessibility Challenges and Proposed Solutions. In 17th International ACM SIGACCESS Conference on Computers and Accessibility. ACM: Lisbon, Portugal. http://doi.org/http://dx.doi.org/10.1145/2700648.2811392
. (2015). Zebra Crossing Spotter: Automatic Population of Spatial Databases for Increased Safety of Blind Travelers. In ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2015). http://doi.org/http://dx.doi.org/10.1145/2700648.2811392
. (2014). Using Computer Vision to Access Appliance Displays. In ASSETS.
. (2014). An Investigation into Incorporating Visual Information in Audio Processing. In Computers Helping People with Special Needs (pp. 437–440). Springer International Publishing.
. (2014). Determining a Blind Pedestrian’s Location and Orientation at Traffic Intersections. In Computers Helping People with Special Needs (pp. 427–432). Springer International Publishing.
. (2014). Self-Localization at Street Intersections. In Computer and Robot Vision (CRV), 2014 Canadian Conference on (pp. 40–47). IEEE.
. (2014). The last meter: blind visual guidance to a target. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 3113–3122). ACM.
. (2013). Smartphone-based crosswalk detection and localization for visually impaired pedestrians. In Multimedia and Expo Workshops (ICMEW), 2013 IEEE International Conference on (pp. 1–7). IEEE.
. (2013). CamIO: a 3D computer vision system enabling audio/haptic interaction with physical objects by blind users. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (p. 41). ACM.
. (2012). The crosswatch traffic intersection analyzer: a roadmap for the future. In Computers Helping People with Special Needs (pp. 25–28). Springer Berlin Heidelberg.
. (2011). Localizing blurry and low-resolution text in natural images. In Applications of Computer Vision (WACV), 2011 IEEE Workshop on (pp. 503–510). IEEE.
. (2011). Real-time detection and reading of LED/LCD displays for visually impaired persons. In Applications of Computer Vision (WACV), 2011 IEEE Workshop on (pp. 491–496). IEEE.
. (2010). Real-time walk light detection with a mobile phone. In Computers Helping People with Special Needs (pp. 229–234). Springer Berlin Heidelberg.
. (2010). Anti-blur feedback for visually impaired users of smartphone cameras. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility (pp. 233–234). ACM.
. (2009). A Bayesian Algorithm for Reading 1D Barcodes. In 2009 Canadian Conference on Computer and Robot Vision (CRV 2009).
. (2009). An algorithm enabling blind users to find and read barcodes. In Applications of Computer Vision (WACV), 2009 Workshop on (pp. 1–8). IEEE.
. (2009). Elevation-based MRF stereo implemented in real-time on a GPU. In Applications of Computer Vision (WACV), 2009 Workshop on (pp. 1–8). IEEE.
. (2008). Detecting and locating crosswalks using a camera phone. In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW'08. IEEE Computer Society Conference on (pp. 1–8). IEEE.
. (2008). Computer vision-based clear path guidance for blind wheelchair users. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility (pp. 291–292). ACM.
. (2007). Accessible spaces: navigating through a marked environment with a camera phone. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility (pp. 229–230). ACM.
. (2006). Cell phone-based wayfinding for the visually impaired. In 1st International Workshop on Mobile Vision.
. (2006). Finding text in natural scenes by figure-ground segmentation. In Pattern Recognition, 2006. ICPR 2006. 18th International Conference on (Vol. 4, pp. 113–118). IEEE.
. (2006). Reading LCD/LED Displays with a Camera Cell Phone. In Computer Vision and Pattern Recognition Workshop, 2006. CVPRW'06. Conference on (pp. 119–119). IEEE.
. (2004). Shape matching with belief propagation: Using dynamic quantization to accomodate occlusion and clutter. In Computer Vision and Pattern Recognition Workshop, 2004. CVPRW'04. Generative Model-Based Vision. (pp. 180–180). IEEE.
. (2003). A bayesian network framework for relational shape matching. In Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on (pp. 671–678). IEEE.
. (2001). The g Factor: Relating Distributions on Features to Distributions on Images. In NIPS (pp. 1231–1238).
. (2000). Order Parameters for Minimax Entropy Distributions: When does high level knowledge help?. In Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on (Vol. 1, pp. 558–565). IEEE.
. (2000). The Manhattan world assumption: Regularities in scene statistics which enable Bayesian inference. In NIPS (pp. 845–851).
. (2000). Unified framework for performance analysis of Bayesian inference. In AeroSense 2000 (pp. 333–346). International Society for Optics and Photonics.
. (1999). Fundamental bounds on edge detection: An information theoretic evaluation of different edge cues. In Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on. (Vol. 1). IEEE.
. (1999). High-Level and Generic Models for Visual Search: When does high level knowledge help?. In Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on. (Vol. 2). IEEE.
. (1999). Manhattan world: Compass direction from a single image by bayesian inference. In Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on (Vol. 2, pp. 941–947). IEEE.
. (1998). A phase space approach to minimax entropy learning and the minutemax approximations. In NIPS (pp. 761–767).
. (1998). Convergence rates of algorithms for visual search: detecting visual contours. In NIPS (pp. 641–647).
. (1998). Efficient optimization of a deformable template using dynamic programming. In 2013 IEEE Conference on Computer Vision and Pattern Recognition (pp. 747–747). IEEE Computer Society.
. (1994). Hierarchical spline-based image registration. In Computer Vision and Pattern Recognition, 1994. Proceedings CVPR'94., 1994 IEEE Computer Society Conference on (pp. 194–201). IEEE.
Presentations/Posters
. (2022). Audiom: an Auditory Web-Based Geographic Map Viewer Showing COVID-19 State Data and a Travel Map. International Conference on Auditory Display (ICAD) 2022.
Other Publications
. (2013). Camera-based access to visual information. In Assistive technology for blindness and low vision (pp. 219–246).
. (2012). Towards a real-time system for finding and reading signs for visually impaired users. In Computers Helping People with Special Needs (pp. 41–47). Springer Berlin Heidelberg.
. (2011). Mechanisms for propagating surface information in 3-D reconstruction. In Computer Vision: From Surfaces to 3D Objects. Chapman and Hall/CRC Press.
. (2008). Portable and mobile systems in assistive technology. In Computers helping people with special needs (pp. 1078–1080). Springer Berlin Heidelberg.
. (2008). Search strategies of visually impaired persons using a camera phone wayfinding system. In Computers Helping People with Special Needs (pp. 1135–1140). Springer Berlin Heidelberg.
. (2007). Grouping using factor graphs: an approach for finding text with a camera phone. In Graph-Based Representations in Pattern Recognition (pp. 394–403). Springer Berlin Heidelberg.
. (2006). Computer vision-based terrain sensors for blind wheelchair users. In Computers Helping People with Special Needs (pp. 1294–1297). Springer Berlin Heidelberg.
. (2002). Finding deformable shapes using loopy belief propagation. In Computer Vision—ECCV 2002 (pp. 453–468). Springer Berlin Heidelberg.
. (1999). Bayesian A* tree search with expected O (N) convergence rates for road tracking. In Energy Minimization Methods in Computer Vision and Pattern Recognition (pp. 189–204). Springer Berlin Heidelberg.
. (1997). Twenty Questions, Focus of Attention, and A*: A theoretical comparison of optimization strategies. In Energy Minimization Methods in Computer Vision and Pattern Recognition (pp. 195–212). Springer Berlin Heidelberg.
Video Description Research and Development Center
The Smith-Kettlewell Video Description Research and Development Center (VDRDC) investigates innovative technologies and techniques for making online video more accessible to blind and visually-impaired students and consumers. Through collaboration with a broad array of partners and stakeholders in the Description Leadership Network, we are developing advanced video annotation methods for use in a wide variety of educational settings, as well as helping educators and other description providers make better use of the tools already available.
Rehabilitation Engineering Research Center
The Center's research goal is to develop and apply new scientific knowledge and practical, cost-effective devices to better understand and address the real-world problems of blind, visually impaired, and deaf-blind consumers
Active
Active
Summer Research Institute
The goal of the Summer Research Institute program is to help people with visual impairments acquire advanced skills related to STEM subjects and careers. Apply to participate in the inaugural session by May 1, 2023 using the link below.
Active
Magic Map
The Magic Map is an interactive 3D map installed at the Magical Bridge Playground in Palo Alto, California. It consists of a 1/100 scale 3D bronze representation of the playground, which includes over seventy play structures organized into multiple play zones and paths. When the tip of the "Magic Wand" tethered to the map is pointed to a specific feature on the map, the name and description of the feature are read aloud in audio. This interactivity makes the map accessible to visitors with visual impairments, and without requiring them to read braille.
Active
Audiom
Audiom: audiom.net
Active
ZoomBoard: an Affordable, Portable System to Improve Access to Presentations and Lecture Notes for Low Vision Viewers
The goal of the project is to develop a “ZoomBoard” system that students with low vision can use to better access visual material on a whiteboard or blackboard. The prototype version of the system that we plan to develop in this grant will consist of a dedicated camera...
Active
Sign Finder
This project seeks to develop a computer vision-based system that allows a visually impaired traveler to find and read informational signs, such as signs labeling office doors, streets, restrooms and Exit signs.
Link to...
Active
A Computer Vision-Based Indoor Wayfinding Tool
The ability to navigate safely and confidently is a fundamental requirement for independent travel and access to many settings such as work, school, shopping, transit and healthcare. Navigation is particularly challenging for people with visual impairments, who have limited ability to see signs, landmarks or maps posted in the environment.
Active
Tactile Graphics Helper (TGH)
Tactile graphics use raised lines, textures, and elevations to provide individuals with visual impairments access to graphical materials through touch. Tactile graphics are particularly important for students in science, technology, engineering, and mathematics (STEM) fields, where educational...
Active
Computer Vision Journal Club
The Computer Vision Journal Club meets periodically to discuss papers on topics in computer vision, machine learning and other topics of interest such as assistive technologies for persons who are blind or visually impaired, dual sensory loss (hearing and vision loss), neuroscience and...
Active
CamIO
CamIO (short for “Camera Input-Output”) is a system to make physical objects (such as documents, maps, devices and 3D models) accessible to blind and visually impaired persons, by providing real-time audio feedback in response to the location on an object that the user is touching. CamIO...
Completed
Completed
The Smith-Kettlewell Haptics Symposium
The Smith-Kettlewell Haptics Symposium was held on March 29, 2018 to honor and remember Dr. Val Morash and her research.
Completed
Regressions in Braille Reading
This project explores regressions (movements to re-read text) in braille reading. The image on the right plots the braille reading finger movements in blue and regressions in black.
Completed
Tutorials and Reference
These are tutorials and reference materials I have written on various topics in probability and geometry over the years.
Completed
Workshop Series on Computer Vision and Sensor-Enabled Assistive Technology for Visual Impairment
Recent workshop:
Workshop on Environmental Sensing Technologies for Visual Impairment (ESTVI '13 in San Francisco)
ESTVI '13 focused on emerging technologies capable of sensing environmental features for...
Completed
Display Reader
The goal of the Display Reader project is to develop a computer vision system that runs on smartphones and tablets to enable blind and visually impaired persons to read appliance displays. Such displays are found on an increasing array of appliances such as microwave ovens, thermostats and home...
Completed
BLaDE
BLaDE (Barcode Localization and Decoding Engine) is an Android smartphone app designed to enable a blind or visually impaired user find and read product barcodes. The primary innovation of BLaDE, relative to most commercially available smartphone apps for reading barcodes, is that it provides...
Completed
Video-Based Speech Enhancement for Persons with Hearing and Vision Loss
Observing the visual cues from a speaker such as the shape of the lips and facial expression can greatly improve the speech comprehension capabilities of a person with hearing loss. However, concurrent vision loss can lead to a significant loss in speech perception. We propose developing a prototype device that utilizes a video camera in addition to audio input to enhance the speech signal from a target speaker in everyday situations.
Completed
Talking Signs
Created by William Loughborough in 1979, Talking Lights was a system of infrared transmitters and receivers allowing blind and visually impaired travelers to quickly and easily "read signs" at a distance.
Contact Information
Email:
coughlan@ski.org
Office Phone:
(415) 345-2146
Links