Contact Information:
Email: coughlan@ski.org
Office Phone: (415) 345-2146
2318 Fillmore Street
San Francisco, CA 94115
The Center’s research goal is to develop and apply new scientific knowledge and practical, cost-effective devices to better understand and address the real-world problems of blind, visually impaired, and deaf-blind consumers. The RERC has many ongoing R&D projects and collaborative relationships, both internal and external to Smith-Kettlewell. Primary funding for the RERC comes from the National Institute on Disability and Rehabilitation Research, with other important sources of support, including the National Eye Institute, and The Smith-Kettlewell Eye Research Institute.
Please check out the latest news about the RERC on Twitter.
Publications
Projects
SKribbleEyes
Scribble Eyes is an Apple iPad/Pencil-based app that automates the data collection, analysis and interpretation process for paper-based tests traditionally used to screen patients with central visual field loss in the clinic.
Audiom Map of Smith-Kettlewell
An Audiom map of the main Smith-Kettlewell building at 2318 Fillmore St., San Francisco has been created by Brandon Biggs. This is an audio-visual map that allows users to explore a detailed map of the building with or without vision. The map runs in any browser and is available to anyone who visits the building.
MapIO: a Gestural and Conversational Interface for Tactile Maps
For individuals who are blind or have low vision, tactile maps provide essential spatial information but are limited in the amount of data they can convey. Digitally augmented tactile maps enhance these capabilities with audio feedback, thereby combining the tactile feedback provided by the map with an audio description of the touched elements. In this context, we explore an embodied interaction paradigm to augment tactile maps with conversational interaction based on Large Language Models, thus enabling users to obtain answers to arbitrary questions regarding the map. We analyze the type of…
Using VR to Help Train Visually Impaired Users to Aim a Camera
People with visual impairments increasingly rely on camera-enabled smartphone apps for tasks like photography, navigation, and text recognition. Despite the growing use of these applications, precise camera aiming remains a significant challenge. This project explores the impact of virtual reality (VR) exploration in the context of learning to use a camera-based app. So far we have studied this approach in the context of training a visually impaired person to use walk-light detector app at traffic intersections.
Mobility and Fall Risk in Central Visual Field Loss
Age-related macular degeneration (AMD) is the most common cause of vision loss in the developed world. Central visual field loss due to diseases such as AMD is a large and growing problem. It is also associated with higher risk of falls and, therefore injury. Although much has been done to understand visual limitations associated with this condition, one of its most dangerous and poorly understood outcomes is the increase in the risk of falls, which can be debilitating and even deadly, especially in the age group most affected by AMD. The exact reasons for the increased fall risk are unknown…
CamIO Hands
This project builds on the CamIO project to provide point-and-tap interactions allowing a user to acquire detailed information about tactile graphics and 3D models.
The interface uses an iPhone’s depth and color cameras to track the user’s hands while they interact with a model. When the user points to a feature of interest on the model with their index finger, the system reads aloud basic information about that feature. For additional information, the user lifts their index finger and taps the feature again. This process can be repeated multiple times to access additional levels of…
The 3 Ps of 3D: Perception, Pedagogy, and Practice for 3D Printing
3D printing is increasingly used in STEM education. How can it be made usable by students who are blind or visually impaired?
Audiom
Audiom is a tool that allows blind and visually impaired individuals to view maps completely in audio. It is a web component and can be embedded into any webpage, similar to Google Maps. It allows non-visual use of route, landmark, and survey knowledge, which is the critical information needed for navigation.
Tracking a target in depth with central field loss
Bilateral field loss due to maculopathy creates a scotoma that extends in depth — a volume scotoma. Morevoer the size of the scotoma depends on whether observers turn their eyes to track a target as it comes closer. This project investigates how the volume scotoma affects the ability to track oncoming targets in these individuals, and in controls with a simulated volume scotoma.
Interactive Tactile Map at Google ADC Milan
This is an installation of the CamIO project that adds interactivity to a tactile map of the Google Accessibility Discovery Center in Milan, Italy.
Smith-Kettlewell Summer Institute
The Smith-Kettlewell Summer Institute program is a multi-year, learning opportunity designed to provide training for blind and visually impaired students, recent graduates, and early career researchers interested in data skills that will enhance Science, Technology, Engineering and Math (STEM) jobs within industry, government, non-profit organizations or academia.
In today’s research climate, individuals are expected to have many skills at their fingertips. Often researchers are required to build software and hardware environments to run studies, to gather and analyze data and to prepare…
Hearing Aid for Blind Travelers with Hearing Loss
Blind travelers rely on subtle acoustic cues for independent mobility. When hearing becomes impaired, as in the normal aging process, a special type of hearing aid may be needed to enhance these cues.
Tactile Map Automated Production (TMAP)
Smith-Kettlewell’s Tactile Map Automated Production (TMAP) Project was initiated with the goal of developing the first web-based software tool for rapid production of highly specific, tactile street maps of any location in the USA.
Audio/Tactile Graphics Using LiveScribe Smartpen
Active from 2007 to 2010, this project was the first exploration of using a digital smartpen as a platform for creating and presenting audio/tactile graphics – a system of using touch-based audio to annotate tactile figures to improve access to graphical information for the blind. The project, a collaboration between Smith-Kettlewell and Vanderbilt University, used the Livescribe Pulse Pen – a commercial smart pen device that includes a video camera in its tip, a microprocessor that can run Java programs, memory, an audio speaker, a microphone, and a rechargeable battery. When the tip of the…
Display Reader
The goal of the Display Reader project is to develop a computer vision system that runs on smartphones and tablets to enable blind and visually impaired persons to read appliance displays. Such displays are found on an increasing array of appliances such as microwave ovens, thermostats and home medical devices.
for Display Reader software download.
BLaDE
BLaDE (Barcode Localization and Decoding Engine) is an Android smartphone app designed to enable a blind or visually impaired user find and read product barcodes. The primary innovation of BLaDE, relative to most commercially available smartphone apps for reading barcodes, is that it provides real-time audio feedback to help visually impaired users find a barcode, which is a prerequisite to being able to read it.
Link to BLaDE software download: http://legacy.ski.org/Rehab/Coughlan_lab/BLaDE/
Click here  for YouTube video demo of BLaDE in action.
Crosswatch
Crosswatch is a smartphone-based system developed for providing real-time guidance to blind and visually impaired travelers at traffic intersections. Using the smartphone’s built-in camera and other sensors, Crosswatch is designed to tell blind and visually impaired travelers what kind of traffic intersection they are near, how to align themselves properly to the crosswalk, and when the walk light or other traffic light indicates it is time to cross.
Click here  for link to zip file containing Crosswatch code (open source) and documentation.
Video-Based Speech Enhancement for Persons with Hearing and Vision Loss
Observing the visual cues from a speaker such as the shape of the lips and facial expression can greatly improve the speech comprehension capabilities of a person with hearing loss. However, concurrent vision loss can lead to a significant loss in speech perception. We propose developing a prototype device that utilizes a video camera in addition to audio input to enhance the speech signal from a target speaker in everyday situations.
Acoustic Cues for Wayfinding
This project aims to do a detailed analysis of the environmental acoustic cues that help some blind navigate successfully.
The Kinematics of Braille Reading
[Under construction]
When blind persons read braille, a system of raised dots for tactile reading and writing, how is the information processed? How do a few indentations on the fingerpads translate to linguistic information, and how does the text, in turn, influence the motions of the hands reading it? Our work on braille addresses these processes on several levels.
Braille, Assistive Technology, and Reading Comprehension
This project explores the impact of presentation mode (braille vs. audio) and using different assistive technology (i.e., refreshable braille displays and screen reader) on reading comprehension ability. The findings will have direct implications for how to present educational material to enhance comprehension among fluent braille readers.
The Smith-Kettlewell Technical File
The Smith-Kettlewell Technical File, edited by William Gerrey (WA6NPC), was a publication by and for blind and visually-impaired electronics professionals and enthusiasts. It was published by the Smith-Kettlewell Eye Research Institute from Fall 1980, through Spring 1998 and was at various times available in Braille, large print, audio cassette, and 5-1/4″ floppy diskette.
Although The Smith-Kettlewell Technical File is no longer in active publication, we offer this online archive as a resource of both current and historical value.
While many of the articles about devices and techniques remain…
Describeathon 2019 Announced!
overTHERE
Blind Arduino Project
The Blind Arduino Project grew out of a community effort led by Dr. Miele in late 2015 to better understand barriers faced by blind people wanting to participate in the vibrant global culture of DIY hardware prototyping.
Arduino is an inexpensive, open-source electronics platform used by everyone from young hobbyists to high-tech developers to build computerized devices integrating sensors, motors, displays, wireless communications, and a host of other tools. Arduino is used in schools to teach children basic principles of design and computer science, in industry to quickly and inexpensively…
Functional Assessment – Visual Fields
Most patients are unaware of their visual field defects and scotomata, be it from AMD, from glaucoma, from RP, or from other causes. Dr. Colenbrander developed an improved functional scoring system for visual fields, as part of the Functional Vision Score system (see Guidelines)
Functional Assessment – Reading
Difficulty reading is the most common complaint of low vision patients. It is the Activity of Daily Living that is most closely related to visual acuity, but involves many additional factors beyond.
Print SIZE provides an assessment (visual acuity) of “How the EYE functions”.
Reading SPEED provides a convenient assessment of “How the PERSON functions.”
PDF Reading acuity (2005)
The IVA system (see below) contains a module to facilitate reading speed measurements.
Functional Assessment – Contrast Sensitivity
For eye doctors measuring contrast sensitivity loss during routine eye exams is not a priority, since contrast losses are not disease specific and do not guide treatment decisions.
For the patient, on the other hand, knowledge about contrast losses, of which they are usually not consciously aware, is very important, since it may affect the performance of Activities of Daily Living and affect safety in mobility tasks.
When visual parameters are measured, the measurements are usually limited to determining peak contrast sensitivity and threshold visual acuity as separate, seamingly independent…
Functional Assessment – Book Chapters
Categorization of visual functioning requires appropriate measurement methods. Dr. Colenbrander has discussed the principles of functional assessment in several book chapters.
Measuring Vision and Vision Loss, Vol. 5, chapter 51, in Duane’s Clinical Ophthalmology, (2001, updated and expanded for 2010 edition). Comprehensive overview, 96 pages, 10 MB. PDF
Low Vision Rehabilitation, Special issue of: Ophthalmology Clinics of North America, Colenbrander, Fletcher eds., 7, 2, 1994.
The Visual System. Chapter 12 in Master the AMA Guides (L. Cocchiarella, S.J…
Labs
Teng LabParticipate in our studies! Visit Teng Lab Experiment Participation for details. Welcome to the Cognition, Action, and Neural Dynamics Laboratory We aim to better understand how people perceive, interact with, and...
Eye-Head Movement LaboratoryOur laboratory is interested in how changes in visual and/or vestibular function affect eye/head coordination, balance, and mobility, particularly in aging. We are currently pursuing two main lines of research:...
Coughlan LabThe goal of our laboratory is to develop and test assistive technology for blind and visually impaired persons that is enabled by computer vision and other sensor technologies.
Verghese LabOur laboratory studies the mechanisms of healthy vision and action, as well as the basis of attention and visual adaptation in clinical populations.
People
Current People
John BrabynIt has been my honor to serve as The Smith-Kettlewell Eye Research Institute’s Executive Director since 2011. Previously, I have […]
Charity Pitcher-CooperPronouns: she/her I joined The Smith-Kettlewell Rehabilitation Engineering Research Center (RERC) team in 2017 as an assistant to scientist Dr. […]
Fundings
Collaborators
News
Oakland High School’s Students Visit Smith-KettlewellOn Friday, February 6th, students from Oakland High School's Innovative Design and Engineering Academy visited Smith-Kettlewell. Students learned about different kinds of accessibility research directly from some of our scientists, giving...
Empowering Data Vision: A Self-Paced Introduction to the Convergence of AI and Data ScienceJuly 25, 2025 — San Francisco, CA
Events
State of the Science 2026: Rehabilitation Engineering Research Center on Blindness and Low Vision
Thursday, June 11th, 2026 – 9:00 AM to 1:00 PM
The goal of this conference is to identify the most pressing needs in the blind and low vision community that...
Blindness and Low Vision Support Group, Year Recap (Hybrid)
Wednesday, May 20th, 2026 – 3:30 PM to 5:00 PM
Get Involved
If you are interested in vision science or want to learn more about low vision and blindness, there are many opportunities to get involved at The Smith-Kettlewell Eye Research Institute.
