CamIO Receives Supplement to Enhance Software Tools for Open Science

September 5th, 2023

Dr. James Coughlan has been awarded funds to increase access to his CamIO tool for making objects accessible to blind and visually impaired persons. The funds were part of NIH’s Notice of Special Interest  for Administrative Supplements to Enhance Software Tools for Open Science and will allow Dr. Coughlan and his team to enhance the CamIO software to lower the barriers to researchers who wish to incorporate it in their own projects or test it out for their own applications.

Briefly, CamIO is a tool intended to make physical objects, including documents, tactile maps and 3D objects such as architectural models and appliances, fully accessible to people who are blind or visually impaired. In the system design, the user provides input to the system by pointing to a target on a 3D or 2D object and hears audio feedback about the target. CamIO has previously been successfully implemented as part of the Magic Map installation at the Magical Bridge Playground in Palo Alto.

With the additional funds, Dr. Coughlan and his team plan two major enhancements to the CamIO software: a restructure the software with the goal of making it easier for others to incorporate in their own accessibility projects, and the creation of SKIstream, a rapid prototyping and debugging tool for use with CamIO and similar accessibility projects.

Publications

A cartoon of two individuals walking along a street with thought bubbles imagining a fall and an ambulance for one and consternation about foot placement in the other.

Shanidze, N.M., Agathos, C.P., Ellmers, T.J., & Young, W.R., (2026). Links Between Central Visual Field Loss and Movement Processing during Walking. Gait & Posture, 110095.

(2026). The best stereoacuity is rarely at the fovea. Vision Research, 240, 108748. https://doi.org/https://doi.org/10.1016/j.visres.2025.108748

Guénot, J. & Verghese, P. (2026). Cue combination for depth perception in macular degeneration: Motion parallax augments disparity. Journal of Vision, 26(1), 11-11. https://doi.org/10.1167/jov.26.1.11

diagram of a head rotation that does not produce an eye counterrotation and instead the eyes are rotated to a new target by the head

Shanidze, N.M. & Velisar, A. (2025). Cancellation of the Vestibulo-Ocular Reflex during Smooth Pursuit in Patients with Maculopathy. Frontiers in Neurology, 16, 1632527.

Zhou, Z., Hou, C.; Attentional Eye Selection Affects Perception During Binocular Rivalry. Trans. Vis. Sci. Tech. 2025;14(10):26.

Verghese P, Chopin A, Gomes-Tomaz Â, Alcalde NG, Levi DM. (2025). Vergence anomalies are associated with impaired stereopsis in amblyopia. Vision Res. 237:108696. doi: 10.1016/j.visres.2025.108696.

Bonneh, Y. ., & Tyler, C. W. (2025). The Canonical Deep Neural Network as a Model for Human Symmetry Processing. IScience. (Original work published 2025)

Good, W. V., Wong, R. J., Norcia, A. ., Hou, C. ., Cellucci, J. ., McGovern, M. ., … Bhutani, V. K. (2025). Effect of bilirubin on visuocortical development in preterm infants. Journal of Perinatology. 10.1038/s41372-025-02213-4 (Original work published 2025)

Rubinstein, J. F., Alcalde, N. G., Chopin, A. ., & Verghese, P. . (2025). Oculomotor challenges in macular degeneration impact motion extrapolation. Journal of Vision, 25(1). 10.1167/jov.25.1.17

García-Lázaro, H. G., & Teng, S. . (2025). Novel Stimuli to Benchmark and Train Echolocation Skills. The Journal on Technology & Persons With Disabilities, 13. 20.500.12680/b2774511d (Original work published 2025)

Hartelius, G., Likova, T. L., Haldeman, D. D., Sester, M., Elfers, J., & Tyler, W. C. (2025). Evidence that Attentional Stance Shift is Learned Rapidly, Reliably Induces Mindfulness, and Constitutes a Distinctive Marker of Cognitive Process. Mindfulness, 16(5), 1258–1272.

Tyler, W. C., Mineff, N. K., Liang, M., & Likova, T. L. (2025). Spatiomotor dynamics of hand movements during the drawing of memory-guided trajectories without visual feedback. Journal of neurophysiology, 133(6), 1665–1674.

Overview of Robin, a wearable echolocation device designed to aid navigation and perception for people who are blind or have low vision. A mannequin wears the headset, which emits ultrasound pulses toward a chair, records the reflected echoes, and slows them down for playback in audible frequencies. A web app lets users customize the signal parameters.

Bhattacharyya, P., Tam, R., Reynolds, I., Orsmond, P., & Teng, S. (2025). Optimizing Signal Parameters to Enhance Echoacoustic Perception in Humans. Durham, UK.

García-Lázaro, H. G., & Teng, S. . (2025). The role of motor control in echolocation performance: self-initiated vs. passive listening for object spatial localization. Durham, UK. Retrieved from https://imrf2025.sciencesconf.org/resource/page/id/30

Agathos, C. P., Velisar, A. ., & Shanidze, N. . (2025). Examining gaze-driven postural adaptations in older adults with and without central visual field loss. Maastricht, Netherlands.

Krasovskaya, S. ., Coughlan, J. ., Patel, A. ., & Teng, S. . (2025). Modeling target search in blind echolocators using a Kalman Filter with realistic exploratory behavior simulations. Boston, MA.

Bhattacharyya, P. ., Tam, R. ., Reynolds, I. ., Orsmond, P. ., & Teng, S. . (2025). Optimizing Signal Parameters to Enhance Echoacoustic Perception of Objects and Scenes in Humans. Durham, UK.

García-Lázaro, H. G., & Teng, S. . (2025). Auditory masking release indexes human click-based echolocation performance. Boston, MA.

Shanidze, N. ., Velisar, A. ., & Agathos, C. P. (2025). Lifetime Noise Exposure is Associated with Greater Postural Sway during Quiet Stance.

Krasovskaya, S. ., Coughlan, J. ., & Teng, S. . (2025). A Kalman Filter model of echo-guided head movement. Durham, UK.

Agathos, C. P., Velisar, A. ., & Shanidze, N. . (2025). Postural Adaptations to Eccentric Viewing Strategies in Age-Related Central Visual Field Loss. Durham, UK.

Rosengen, B. ., Agathos, C. P., & Shanidze, N. . (2025). Fall-Related Ruminations are Linked to Spatial Orientation and Cognitive Flexibility. San Francisco, CA, USA.

Bhattacharyya, P. ., Tam, R. ., Orsmond, P. ., Hicks, S. ., & Teng, S. . (2025). Time-resolved EEG decoding of neural text representations during naturalistic braille reading. Boston, MA.

Velisar, A. ., Agathos, C. P., Telaprolu, D. S. L., & Shanidze, N. . (2025). Lifetime Noise Exposure Affects Age-Related Gait Changes. Maastricht, Netherlands.

Hand pointing to the wing of an airplane shown in a tactile graphic. The hand skeleton detected by computer vision is shown in green.

Ahmetovic, D., Coughlan, J., Dal Santo, G., Ezrouri, K., Manzoni, M., & Mascetti, S. (2025). CamIO in the Browser: A Cross-Platform Audio Label Tool for Tactile Graphics. MobileHCI 2025

Pitcher-Cooper, C. ., Ihorn, S. ., & Yoon, I. . (2024). Access granted: Unlocking accessibility in computer science through collaboration. ASSETS 2024 Workshop. Presented at the ASSETS 2024 Workshop. Association for Computing Machinery. (Original work published 2024)

Narcisi, A. ., Shen, H. ., Ahmetovic, D. ., Mascetti, S. ., & Coughlan, J. . (2024). Accessible Point-and-Tap Interaction for Acquiring Detailed Information about Tactile Graphics and 3D Models. International Conference on Computers Helping People With Special Needs. Presented at the International Conference on Computers Helping People with Special Needs. Linz, Austria. Retrieved from https://link.springer.com/chapter/10.1007/978-3-031-62846-7_30 (Original work published 2024)

Seo, J. ., O’Modhrain, S. ., Xia, Y. ., Kamath, S. ., Lee, B. ., & Coughlan, J. . (2024). Born Accessible Data Science and Visualization Courses: Challenges of Developing Curriculum to be Taught by Blind Instructors to Blind Students. EUROVIS 2024. Presented at the EUROVIS 2024. https://doi.org/10.2312/eved.20241053 (Original work published 2024)

Hong, J. ., & Coughlan, J. . (2024). Enhancing Walk-Light Detector Usage for the Visually Impaired: A Comparison of VR Exploration and Verbal Instructions. Web4All 2024. Presented at the Web4All 2024. Singapore. (Original work published 2024)

Scott, A. ., Narins, L. ., Kulkarni, A. ., Castanon, M. ., Kao, B. ., Ihorn, S. ., … Yoon, I. . (2023). Improved Image Caption Rating – Datasets, Game, and Model. 2023 CHI Conference on Human Factors in Computing Systems. Presented at the 2023 CHI Conference on Human Factors in Computing Systems. Hamburg, Germany: Association for Computing Machinery. 10.1145/3544549.3585632 (Original work published 2023)

Biggs, B. ., Toth, C. ., Stockman, T. ., Coughlan, J. ., & Walker, B. N. (2022). Evaluation of a Non-Visual Auditory Choropleth and Travel Map Viewer. International Conference on Auditory Display (ICAD) 2022. Presented at the International Conference on Auditory Display (ICAD) 2022. Virtual. Retrieved from PMCID: PMC10010675 (Original work published 2022)

Coughlan, J. ., Biggs, B. ., & Shen, H. . (2022). Non-Visual Access to an Interactive 3D Map. Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE ’22). Presented at the Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE ’22). (Original work published 2022)

Sarker, P. ., Zaman, N. ., & Tavakkoli, A. . (2022). Vr-sft: Reproducing swinging flashlight test in virtual reality to detect relative afferent pupillary defect. 193-204. Springer Nature Switzerland Cham.

Coughlan, J. ., Biggs, B. ., & Shen, H. . (2021). Point and Listen: Bringing a 3D Map to Life with Audio-Based AR. 6th Annual Frameless XR Symposium. Presented at the 6th Annual Frameless XR Symposium. (Original work published 2021)

Biggs, B. ., Coughlan, J. ., & Coppin, P. . (2021). Design and evaluation of an interactive 3D map. RESNA 2021 Conference. Presented at the RESNA 2021 Conference. (Original work published 2021)

Binaee, K. ., Sinnott, C. B., Capurro, K. ., MacNeilage, P. R., & Lescroart, M. . (2021). Pupil tracking under direct sunlight. 2021 ACM Syposium on Eye Tracking Research and Applications ActivEye Workshop. Presented at the 2021 ACM Syposium on Eye Tracking Research and Applications ActivEye Workshop. Germany: Association for Computing Machinery. (Original work published 2021)

a picture describing an experimental setup where a robot bird is shown on a platform waiting for a live bird to land on a perch to start a birdsong

Araguas, A., Valencia, D., Chopin, A., Thomas, A., Gauthier, P., Richer, F., Guellai, B., & Deregnaucourt, S. (2026). Influence of behavioural contingency on developmental song learning in young zebra finches (Taeniopygia guttata) tutored by a robot bird. Proceedings of the Royal Society B: Biological Sciences, 293(2066), 20252834. https://doi.org/10.1098/rspb.2025.2834

Zaman, N., Potluri, V., Biggs, B., & Coughlan, M. J. (2025). WhatsAI: Transforming Meta Ray-Bans into an Extensible Generative AI Platform for Accessibility. arXiv preprint arXiv:2505.09823.

diagram if the cameras directed at robotic eyes with labels of individual components and placement in 3D space

Shanidze, M. N., Velisar, A., Lotze, A. W., & Love, K. K. (2025). Oculomotor simulator for assessing eye-tracking accuracy. US Patent 12,474,771

Likova, T. L., Mineff, N. K., Zhou, Z., Liang, M., & Tyler, W. C. (2025). Training-induced Reorganization of the Cross-modal Connectivity of the Human Motion Area in Severe Low Vision. Electronic Imaging, 37, 1–8.

Zaman, N. . (2024). Extending Vision with Extended Reality. University of Nevada, Reno.

Mukhtar, A. ., Henderson, K. ., Bourquin, S. ., Teng, S. ., & Puri, A. . (2024). Exploring Object Properties that Support the Transfer of Information Across Active Echolocation and Touch. New York, NY.

Child-robot interactions in different body and emotions oriented tasks: comparison with a human partner

Waisberg, E. ., Ong, J. ., Zaman, N. ., Tavakkoli, A. ., & Lee, A. . (2023). The Impact of COVID-19 on Ophthalmology Clinic and Surgical Volume. Ir Med J, 116, P884.

Waisberg, E. ., Ong, J. ., Masalkhi, M. ., Zaman, N. ., Sarker, P. ., Lee, A. G., & Tavakkoli, A. . (2023). GPT-4 and medical image analysis: Strengths, weaknesses and future directions. Journal of Medical Artificial Intelligence, 6.

Waisberg, E. ., Ong, J. ., Masalkhi, M. ., Zaman, N. ., Sarker, P. ., Lee, A. G., & Tavakkoli, A. . (2023). Chat Generative Pretrained Transformer to optimize accessibility for cataract surgery postoperative management. The Pan-American Journal of Ophthalmology, 5, 46.

Zaman, N. ., Sarker, P. ., & Tavakkoli, A. . (2023). Calibration of head mounted displays for vision research with virtual reality. Journal of Vision, 23, 7-7.

Waisberg, E. ., Ong, J. ., Masalkhi, M. ., Zaman, N. ., Kamran, S. A., Sarker, P. ., … Lee, A. G. (2023). The case for expanding visual assessments during spaceflight. Prehospital and Disaster Medicine, 38, 518-521.

Projects

  • Active
Photo of Magic Map, a bronze scale map of the Magical Bridge Playground in Palo Alto. The bronze map is bordered by panels on both sides with descriptions of the map.
Active

Magic Map

The Magic Map is an interactive 3D map installed at the Magical Bridge Playground in Palo Alto, California. It consists of a 1/100 scale 3D bronze representation of the playground, which includes over seventy play structures organized into multiple play zones and paths. When the user’s index fingertip touches a specific feature on the map, the name and description of the feature are read aloud in audio. This interactivity allows visitors with visual impairments to navigate the map without requiring them to read braille.

Read more
Animation of CamIO in action shows biological cell model. When person points with their index finger to a feature on the model, its name is read aloud.
Active

CamIO

CamIO (short for “Camera Input-Output”) is a system to make physical objects (such as documents, maps, devices and 3D models) accessible to blind and visually impaired persons, by providing real-time audio feedback in response to the location on an object that the user is touching. CamIO currently works on iOS using the built-in camera and an inexpensive hand-held stylus, made out of materials such as 3D-printed plastic, paper or wood.

See a short video demonstration of CamIO here , showing how the user can trigger audio labels by pointing a stylus at “hotspots” on a 3D map of a playground. See…

Read more

Labs

  • Coughlan Lab
    Coughlan Lab
    Principal Investigator:
    The goal of our laboratory is to develop and test assistive technology for blind and visually impaired persons that is enabled by computer vision and other sensor technologies.
    Read More

Centers

  • Rehabilitation Engineering Research Center
    Rehabilitation Engineering Research Center
    The Center's research goal is to develop and apply new scientific knowledge and practical, cost-effective devices to better understand and address the real-world problems of blind, visually impaired, and deaf-blind...
    Read More