The goal of our laboratory is to develop and test assistive technology for blind and visually impaired persons that is enabled by computer vision and other sensor technologies.
Tabs
Journal Articles
. (2017). Mind your crossings: Mining GIS imagery for crosswalk localization. Acm Transactions On Accessible Computing (Taccess), 9(4). http://doi.org/10.1145/3046790
. (2013). S-K Smartphone Barcode Reader for the Blind. Journal On Technology And Persons With Disabilities, 1.
. (2013). Crosswatch: a system for providing guidance to visually impaired travelers at traffic intersections. Journal Of Assistive Technologies, 7.
. (2013). Improving speech enhancement algorithms by incorporating visual information. The Journal Of The Acoustical Society Of America, 134, 4237–4237.
. (2010). An Embarrassingly Simple Speed-Up of Belief Propagation with Robust Potentials. Arxiv Preprint Arxiv:1010.0012.
. (2009). Functional assessment of a camera phone-based wayfinding system operated by blind and visually impaired users. International Journal On Artificial Intelligence Tools, 18, 379–397.
. (2009). A mobile phone wayfinding system for visually impaired users. Assistive Technology Research Series, 25, 849.
. (2009). Staying in the crosswalk: A system for guiding visually impaired pedestrians at traffic intersections. Assistive Technology Research Series, 25, 69.
. (2008). A mobile phone system to find crosswalks for visually impaired pedestrians. Technology And Disability, 20, 217–224.
. (2008). Portable and Mobile Systems in Assistive Technology-Portable and Mobile Systems in Assistive Technology: Introduction to the Special Thematic Session. Lecture Notes In Computer Science, 5105, 1078.
. (2007). Dynamic quantization for belief propagation in sparse spaces. Computer Vision And Image Understanding, 106, 47–58.
. (2007). Color targets: Fiducials to help visually impaired people find their way by camera phone. Eurasip Journal On Image And Video Processing, 2007.
. (2007). Functional assessment of a camera phone-based wayfinding system operated by blind users. Ieee Computer Society And The Biological And Artificial Intelligence Society.
. (2006). A fast algorithm for finding crosswalks using figure-ground segmentation. 2Nd Workshop On Applications Of Computer Vision, In Conjunction With Eccv, 5.
. (2005). An information maximization model of eye movements. Advances In Neural Information Processing Systems, 17, 1121-8.
. (2005). Rapid and robust algorithms for detecting colour targets. 10Th Congress Of The International Colour Association, Aic Colour, 5, 959–962.
. (2003). The KGBR viewpoint-lighting ambiguity. Journal Of The Optical Society Of America (Josa) A, 20(1). (Original work published 2003)
. (2003). A statistical approach to multi-scale edge detection. Image And Vision Computing, 21, 37–48.
. (2003). Algorithms from statistical physics for generative models of images. Image And Vision Computing, 21, 29–36.
. (2003). The generic viewpoint assumption and planar bias. Ieee Transactions On Pattern Analysis And Machine Intelligence, 25, 775–778.
. (2003). A large deviation theory analysis of Bayesian tree search. Ima Volumes In Mathematics And Its Applications, 133, 1–18.
. (2003). Manhattan world: Orientation and outlier detection by bayesian inference. Neural Computation, 15, 1063–1088.
. (2003). Statistical edge detection: Learning and evaluating edge cues. Pattern Analysis And Machine Intelligence, Ieee Transactions On, 25, 57–74.
. (2002). Bayesian A* tree search with expected O (N) node expansions: applications to road tracking. Neural Computation, 14, 1929–1958.
. (2002). Fundamental bounds on edge detection: learning and evaluating edge cues. Pattern Anal. Machine Intell.
. (2001). Order Parameters for Detecting Target Curves in Images: When does high level knowledge help?. International Journal Of Computer Vision, 41, 9–33.
. (2001). The KGBR viewpoint-lighting ambiguity and its resolution by generic constraints. Computer Vision, 2001. Iccv 2001. Proceedings. Eighth Ieee International Conference On, 2, 376–382.
. (2000). The generic viewpoint constraint resolves the generalized bas relief ambiguity. Proc. Of Conference On Information Scienes And Systems (Ciss 2000), 15–17.
. (2000). Fundamental limits of Bayesian inference: order parameters and phase transitions for road tracking. Pattern Analysis And Machine Intelligence, Ieee Transactions On, 22, 160–173.
. (2000). Efficient deformable template detection and localization without user initialization. Computer Vision And Image Understanding, 78, 303–319.
. (2000). An A∗ perspective on deterministic optimization for deformable templates. Pattern Recognition, 33, 603–616.
. (1999). Visual search: Fundamental bounds, order parameters, and phase transitions. Proc Ieee Workshop On Statistical And Computational Theories Of Vision. Cvpr 1999. Fort Collins, Co. June 1999. (Original work published 1999)
. (1999). From Generic to Specific: An Information Theoretic Perspective on the Value of High-Level Information. Probabilistic Models Of The Brain, 135.
Conference Papers
. (2019). Design and Evaluation of an Audio Game-Inspired Auditory Map Interface. In The 25th International Conference on Auditory Display (ICAD 2019). Northumbria University, Newcastle-upon-Tyne, UK. (Original work published 2019)
. (2018). Indoor Localization using Computer Vision and Visual-Inertial Odometry. In International Conference on Computers Helping People with Special Needs (ICCHP '18). Linz, Austria. (Original work published 2018)
. (2017). Evaluating Author and User Experience for an Audio-Haptic System for Annotation of Physical Models. In 19th Int’l ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2017). Baltimore, MD: ACM. (Original work published 2017)
. (2017). JustPoint: Identifying Colors with a Natural User Interface. In 19th Int’l ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2017). Baltimore, MD: ACM. (Original work published 2017)
. (2017). AR4VI: AR as an Accessibility Tool for People with Visual Impairments. In AR for Good, part of ISMAR 2017. Nantes, France: IEEE. (Original work published 2017)
. (2016). Towards a Sign-Based Indoor Navigation System for People with Visual Impairments. In 18th International ACM SIGACCESS Conference on Computers and Accessibility. Reno, NV: ACM.
. (2015). Appliance Displays: Accessibility Challenges and Proposed Solutions. In 17th International ACM SIGACCESS Conference on Computers and Accessibility. Lisbon, Portugal: ACM. http://doi.org/http://dx.doi.org/10.1145/2700648.2811392 (Original work published 2015)
. (2015). Zebra Crossing Spotter: Automatic Population of Spatial Databases for Increased Safety of Blind Travelers. In ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2015). http://doi.org/http://dx.doi.org/10.1145/2700648.2811392
. (2014). Using Computer Vision to Access Appliance Displays. In ASSETS. (Original work published 2014)
. (2014). The last meter: blind visual guidance to a target. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 3113–3122). ACM.
. (2014). Determining a Blind Pedestrian’s Location and Orientation at Traffic Intersections. In Computers Helping People with Special Needs (pp. 427–432). Springer International Publishing.
. (2014). An Investigation into Incorporating Visual Information in Audio Processing. In Computers Helping People with Special Needs (pp. 437–440). Springer International Publishing.
. (2014). Self-Localization at Street Intersections. In Computer and Robot Vision (CRV), 2014 Canadian Conference on (pp. 40–47). IEEE.
. (2013). Smartphone-based crosswalk detection and localization for visually impaired pedestrians. In Multimedia and Expo Workshops (ICMEW), 2013 IEEE International Conference on (pp. 1–7). IEEE.
. (2013). CamIO: a 3D computer vision system enabling audio/haptic interaction with physical objects by blind users. In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (p. 41). ACM.
. (2012). The crosswatch traffic intersection analyzer: a roadmap for the future. In Computers Helping People with Special Needs (pp. 25–28). Springer Berlin Heidelberg.
. (2011). Localizing blurry and low-resolution text in natural images. In Applications of Computer Vision (WACV), 2011 IEEE Workshop on (pp. 503–510). IEEE.
. (2011). Real-time detection and reading of LED/LCD displays for visually impaired persons. In Applications of Computer Vision (WACV), 2011 IEEE Workshop on (pp. 491–496). IEEE.
. (2010). Anti-blur feedback for visually impaired users of smartphone cameras. In Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility (pp. 233–234). ACM.
. (2010). Real-time walk light detection with a mobile phone. In Computers Helping People with Special Needs (pp. 229–234). Springer Berlin Heidelberg.
. (2009). Elevation-based MRF stereo implemented in real-time on a GPU. In Applications of Computer Vision (WACV), 2009 Workshop on (pp. 1–8). IEEE.
. (2009). An algorithm enabling blind users to find and read barcodes. In Applications of Computer Vision (WACV), 2009 Workshop on (pp. 1–8). IEEE.
. (2009). A Bayesian Algorithm for Reading 1D Barcodes. In 2009 Canadian Conference on Computer and Robot Vision (CRV 2009).
. (2008). Computer vision-based clear path guidance for blind wheelchair users. In Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility (pp. 291–292). ACM.
. (2008). Detecting and locating crosswalks using a camera phone. In Computer Vision and Pattern Recognition Workshops, 2008. CVPRW'08. IEEE Computer Society Conference on (pp. 1–8). IEEE.
. (2007). Accessible spaces: navigating through a marked environment with a camera phone. In Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility (pp. 229–230). ACM.
. (2006). Cell phone-based wayfinding for the visually impaired. In 1st International Workshop on Mobile Vision.
. (2006). Reading LCD/LED Displays with a Camera Cell Phone. In Computer Vision and Pattern Recognition Workshop, 2006. CVPRW'06. Conference on (pp. 119–119). IEEE.
. (2006). Finding text in natural scenes by figure-ground segmentation. In Pattern Recognition, 2006. ICPR 2006. 18th International Conference on (Vol. 4, pp. 113–118). IEEE.
. (2004). Shape matching with belief propagation: Using dynamic quantization to accomodate occlusion and clutter. In Computer Vision and Pattern Recognition Workshop, 2004. CVPRW'04. Generative Model-Based Vision. (pp. 180–180). IEEE.
. (2003). A bayesian network framework for relational shape matching. In Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on (pp. 671–678). IEEE.
. (2001). The g Factor: Relating Distributions on Features to Distributions on Images. In NIPS (pp. 1231–1238).
. (2000). The Manhattan world assumption: Regularities in scene statistics which enable Bayesian inference. In NIPS (pp. 845–851).
. (2000). Order Parameters for Minimax Entropy Distributions: When does high level knowledge help?. In Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on (Vol. 1, pp. 558–565). IEEE.
. (2000). Unified framework for performance analysis of Bayesian inference. In AeroSense 2000 (pp. 333–346). International Society for Optics and Photonics.
. (1999). High-Level and Generic Models for Visual Search: When does high level knowledge help?. In Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on. (Vol. 2). IEEE.
. (1999). Fundamental bounds on edge detection: An information theoretic evaluation of different edge cues. In Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on. (Vol. 1). IEEE.
. (1999). Manhattan world: Compass direction from a single image by bayesian inference. In Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on (Vol. 2, pp. 941–947). IEEE.
. (1998). A phase space approach to minimax entropy learning and the minutemax approximations. In NIPS (pp. 761–767).
. (1998). Convergence rates of algorithms for visual search: detecting visual contours. In NIPS (pp. 641–647).
. (1998). Efficient optimization of a deformable template using dynamic programming. In 2013 IEEE Conference on Computer Vision and Pattern Recognition (pp. 747–747). IEEE Computer Society.
. (1994). Hierarchical spline-based image registration. In Computer Vision and Pattern Recognition, 1994. Proceedings CVPR'94., 1994 IEEE Computer Society Conference on (pp. 194–201). IEEE.
Other Publications
. (2013). Camera-based access to visual information. In Assistive technology for blindness and low vision (pp. 219–246).
. (2012). Towards a real-time system for finding and reading signs for visually impaired users. In Computers Helping People with Special Needs (pp. 41–47). Springer Berlin Heidelberg.
. (2011). Mechanisms for propagating surface information in 3-D reconstruction. In Computer Vision: From Surfaces to 3D Objects. Chapman and Hall/CRC Press.
. (2008). Search strategies of visually impaired persons using a camera phone wayfinding system. In Computers Helping People with Special Needs (pp. 1135–1140). Springer Berlin Heidelberg.
. (2008). Portable and mobile systems in assistive technology. In Computers helping people with special needs (pp. 1078–1080). Springer Berlin Heidelberg.
. (2007). Grouping using factor graphs: an approach for finding text with a camera phone. In Graph-Based Representations in Pattern Recognition (pp. 394–403). Springer Berlin Heidelberg.
. (2006). Computer vision-based terrain sensors for blind wheelchair users. In Computers Helping People with Special Needs (pp. 1294–1297). Springer Berlin Heidelberg.
. (2002). Finding deformable shapes using loopy belief propagation. In Computer Vision—ECCV 2002 (pp. 453–468). Springer Berlin Heidelberg.
. (1999). Bayesian A* tree search with expected O (N) convergence rates for road tracking. In Energy Minimization Methods in Computer Vision and Pattern Recognition (pp. 189–204). Springer Berlin Heidelberg.
. (1997). Twenty Questions, Focus of Attention, and A*: A theoretical comparison of optimization strategies. In Energy Minimization Methods in Computer Vision and Pattern Recognition (pp. 195–212). Springer Berlin Heidelberg.
Video Description Research and Development Center
The Smith-Kettlewell Video Description Research and Development Center (VDRDC) investigates innovative technologies and techniques for making online video more accessible to blind and visually-impaired students and consumers. Through collaboration with a broad array of partners and stakeholders in...
Rehabilitation Engineering Research Center
The Center's research goal is to develop and apply new scientific knowledge and practical, cost-effective devices to understand and address better the real-world problems of blind, visually impaired, and deaf-blind consumers. The RERC has many ongoing R...
CamIO
CamIO (short for “Camera Input-Output”) is a system to make physical objects (such as documents, maps, devices and 3D models) accessible to blind and visually impaired persons, by providing real-time audio feedback in response to the location on an object that the user is touching. CamIO...
Display Reader
The goal of the Display Reader project is to develop a computer vision system that runs on smartphones and tablets to enable blind and visually impaired persons to read appliance displays. Such displays are found on an increasing array of appliances such as microwave ovens, thermostats and home...
ZoomBoard: an Affordable, Portable System to Improve Access to Presentations and Lecture Notes for Low Vision Viewers
The goal of the project is to develop a “ZoomBoard” system that students with low vision can use to better access visual material on a whiteboard or blackboard. The prototype version of the system that we plan to develop in this grant will consist of a dedicated camera...
Sign Finder
This project seeks to develop a computer vision-based system that allows a visually impaired traveler to find and read informational signs, such as signs labeling office doors, streets, restrooms and Exit signs.
Link to...
Tutorials and Reference
These are tutorials and reference materials I have written on various topics in probability and geometry over the years.
A Computer Vision-Based Indoor Wayfinding Tool
We are creating a smartphone-based indoor navigation app to estimate the user's location in an indoor environment and guide him/her to a desired destination.
Workshop Series on Computer Vision and Sensor-Enabled Assistive Technology for Visual Impairment
Recent workshop:
Workshop on Environmental Sensing Technologies for Visual Impairment (ESTVI '13 in San Francisco)
ESTVI '13 focused on emerging technologies capable of sensing environmental features for...
Computer Vision Journal Club
The Computer Vision Journal Club meets periodically to discuss papers on topics in computer vision, machine learning and other topics of interest such as assistive technologies for persons who are blind or visually impaired, dual sensory loss (hearing and vision loss), neuroscience and...
BLaDE
BLaDE (Barcode Localization and Decoding Engine) is an Android smartphone app designed to enable a blind or visually impaired user find and read product barcodes. The primary innovation of BLaDE, relative to most commercially available smartphone apps for reading barcodes, is that it provides...
Crosswatch
Crosswatch is a smartphone-based system developed for providing real-time guidance to blind and visually impaired travelers at traffic intersections. Using the smartphone's built-in camera and other sensors, Crosswatch is designed to tell blind and visually impaired travelers what kind of...
Video-Based Speech Enhancement for Persons with Hearing and Vision Loss
Observing the visual cues from a speaker such as the shape of the lips and facial expression can greatly improve the speech comprehension capabilities of a person with hearing loss. However, concurrent vision loss can lead to a significant loss in speech perception. We propose developing a prototype device that utilizes a video camera in addition to audio input to enhance the speech signal from a target speaker in everyday situations.
The Smith-Kettlewell Haptics Symposium
Date: Thursday, March 29, 2018
Location: The Smith-Kettlewell Eye Research Institute, 2318 Fillmore Street, San Francisco, CA 94115
The goal of this event...
Tactile Graphics Helper (TGH)
Tactile graphics use raised lines, textures, and elevations to provide individuals with visual impairments access to graphical materials through touch. Tactile graphics are particularly important for students in science, technology, engineering, and mathematics (STEM) fields, where educational...
Contact Information
2318 Fillmore Street
San Francisco, CA 94115
Room #
417
Email:
coughlan@ski.org
Office Phone:
(415) 345-2146
Links