CamIO

Picture shows person pointing stylus to plastic model of a biological cell; webcam (not pictured) views model and stylus; computer, connected to webcam, announces "Nucleolus"

CamIO (short for “Camera Input-Output”) is a system to make physical objects (such as documents, maps, devices and 3D models) accessible to blind and visually impaired persons, by providing real-time audio feedback in response to the location on an object that the user is touching. CamIO currently works on iOS using the built-in camera and an inexpensive hand-held stylus, made out of materials such as 3D-printed plastic, paper or wood.

See a video demonstration of CamIO here, showing how the user can trigger audio labels by pointing a stylus at "hotspots" on a 3D map of a playground. See the Magic Map project page for more information on this specific application of CamIO.

The CamIO project received the 2020 Dr. Arthur I. Karshmer Award for Assistive Technology Research. It was supported by a four-year grant from NIH/NEI (R01EY025332), which was renewed in 2022 for another four years (2R01EY025332-05A1). It has also received support from the NIDILRR RERC (grant numbers 90RE5024 and 90REGE0018).

Publications

Conference Papers

Coughlan, J., Biggs, B., & Shen, H. (2022). Non-Visual Access to an Interactive 3D Map. Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE '22). Presented at the. (Original work published 2022)
Coughlan, J., Biggs, B., & Shen, H. (2021). Point and Listen: Bringing a 3D Map to Life with Audio-Based AR. 6th Annual Frameless XR Symposium. Presented at the. (Original work published 2021)
Biggs, B., Coughlan, J., & Coppin, P. (2021). Design and evaluation of an interactive 3D map. RESNA 2021 Conference. Presented at the. (Original work published 2021)
Coughlan, J., Biggs, B., Rivière, M. -A., & Shen, H. (2020). An Audio-Based 3D Spatial Guidance AR System for Blind Users. 17th International Conference on Computers Helping People with Special Needs. Presented at the. (Original work published 2020)
Coughlan, J., & Miele, J. (2017). AR4VI: AR as an Accessibility Tool for People with Visual Impairments. AR for Good, part of ISMAR 2017. Presented at the. Nantes, France: IEEE. (Original work published 2017)
Coughlan, J., & Miele, J. (2017). Evaluating Author and User Experience for an Audio-Haptic System for Annotation of Physical Models. 19th Int’l ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2017). Presented at the. Baltimore, MD: ACM. (Original work published 2017)
Mascetti, S., Gerino, A., Bernareggi, C., D’Acquisto, S., Ducci, M., & Coughlan, J. (2017). JustPoint: Identifying Colors with a Natural User Interface. 19th Int’l ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2017). Presented at the. Baltimore, MD: ACM. (Original work published 2017)
Shen, H., Edwards, O., Miele, J., & Coughlan, J. (2013). CamIO: a 3D computer vision system enabling audio/haptic interaction with physical objects by blind users. Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, 41. ACM.

Centers

People

Current

Photo of Giovanni Fusco

Giovanni Fusco

Engineering Manager and Lead Machine Learning Eng., Pixofarm