Huiying Shen

Photo of Huiying Shen
Staff
PhD. Civil and Environmental Engineering
MS, Civil and Environmental Engineering
BS, Civil Engineering

Publications

Journal Articles

Coughlan, J., Shen, H., & Biggs, B. (2020). Towards Accessible Audio Labeling of 3D Objects. Journal On Technology And Persons With Disabilities, 8. (Original work published 2020)
Coughlan, J., & Shen, H. (2013). Crosswatch: a system for providing guidance to visually impaired travelers at traffic intersections. Journal Of Assistive Technologies, 7.
Shen, H., Coughlan, J., & Ivanchenko, V. (2009). Figure-ground segmentation using factor graphs. Image And Vision Computing, 27, 854–863.
Coughlan, J., & Shen, H. (2010). An Embarrassingly Simple Speed-Up of Belief Propagation with Robust Potentials. Arxiv Preprint Arxiv:1010.0012.
Shen, H., Chan, K. -Y., Coughlan, J., & Brabyn, J. (2008). A mobile phone system to find crosswalks for visually impaired pedestrians. Technology And Disability, 20, 217–224.
Coughlan, J., Manduchi, R., Mutsuzaki, M., & Shen, H. (2005). Rapid and robust algorithms for detecting colour targets. 10Th Congress Of The International Colour Association, Aic Colour, 5, 959–962.
Ivanchenko, V., Coughlan, J., & Shen, H. (2009). Staying in the crosswalk: A system for guiding visually impaired pedestrians at traffic intersections. Assistive Technology Research Series, 25, 69.
Coughlan, J., & Shen, H. (2007). Dynamic quantization for belief propagation in sparse spaces. Computer Vision And Image Understanding, 106, 47–58.
Coughlan, J., & Shen, H. (2006). A fast algorithm for finding crosswalks using figure-ground segmentation. 2Nd Workshop On Applications Of Computer Vision, In Conjunction With Eccv, 5.

Conference Papers

Coughlan, J., Biggs, B., & Shen, H. (2022). Non-Visual Access to an Interactive 3D Map. Joint International Conference on Digital Inclusion, Assistive Technology & Accessibility (ICCHP-AAATE '22). Presented at the. (Original work published 2022)
Coughlan, J., Biggs, B., & Shen, H. (2021). Point and Listen: Bringing a 3D Map to Life with Audio-Based AR. 6th Annual Frameless XR Symposium. Presented at the. (Original work published 2021)
Coughlan, J., Biggs, B., Rivière, M. -A., & Shen, H. (2020). An Audio-Based 3D Spatial Guidance AR System for Blind Users. 17th International Conference on Computers Helping People with Special Needs. Presented at the. (Original work published 2020)
Fusco, G., Shen, H., & Coughlan, J. (2014). Self-Localization at Street Intersections. Computer and Robot Vision (CRV), 2014 Canadian Conference on, 40–47. IEEE.
Coughlan, J., & Shen, H. (2012). The crosswatch traffic intersection analyzer: a roadmap for the future. Computers Helping People with Special Needs, 25–28. Springer Berlin Heidelberg.
Shen, H., Edwards, O., Miele, J., & Coughlan, J. (2013). CamIO: a 3D computer vision system enabling audio/haptic interaction with physical objects by blind users. Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, 41. ACM.
Fusco, G., Shen, H., Murali, V., & Coughlan, J. (2014). Determining a Blind Pedestrian’s Location and Orientation at Traffic Intersections. Computers Helping People with Special Needs, 427–432. Springer International Publishing.
Sanketi, P., Shen, H., & Coughlan, J. (2011). Localizing blurry and low-resolution text in natural images. Applications of Computer Vision (WACV), 2011 IEEE Workshop on, 503–510. IEEE.
Ivanchenko, V., Shen, H., & Coughlan, J. (2009). Elevation-based MRF stereo implemented in real-time on a GPU. Applications of Computer Vision (WACV), 2009 Workshop on, 1–8. IEEE.
Tekin, E., Coughlan, J., & Shen, H. (2011). Real-time detection and reading of LED/LCD displays for visually impaired persons. Applications of Computer Vision (WACV), 2011 IEEE Workshop on, 491–496. IEEE.

Projects

CamIO

CamIO (short for “Camera Input-Output”) is a system to make physical objects (such as documents, maps, devices and 3D models) accessible to blind and visually impaired persons, by providing real-time audio feedback in response to the location on an object that the user is touching. CamIO currently works on iOS using the built-in camera and an inexpensive hand-held stylus, made out of materials such as 3D-printed plastic, paper or wood.

See a video demonstration of CamIO here, showing how the user can trigger audio labels by pointing a stylus at "hotspots" on a 3D map of a playground. See the Ma…

Crosswatch

Crosswatch is a smartphone-based system developed for providing real-time guidance to blind and visually impaired travelers at traffic intersections. Using the smartphone's built-in camera and other sensors, Crosswatch is designed to tell blind and visually impaired travelers what kind of traffic intersection they are near, how to align themselves properly to the crosswalk, and when the walk light or other traffic light indicates it is time to cross.

Click here for link to zip file containing Crosswatch code (open source) and documentation.

Display Reader

The goal of the Display Reader project is to develop a computer vision system that runs on smartphones and tablets to enable blind and visually impaired persons to read appliance displays. Such displays are found on an increasing array of appliances such as microwave ovens, thermostats and home medical devices.

Click here for Display Reader software download.

Magic Map

The Magic Map is an interactive 3D map installed at the Magical Bridge Playground in Palo Alto, California. It consists of a 1/100 scale 3D bronze representation of the playground, which includes over seventy play structures organized into multiple play zones and paths. When the tip of the "Magic Wand" tethered to the map is pointed to a specific feature on the map, the name and description of the feature are read aloud in audio. This interactivity makes the map accessible to visitors with visual impairments, and without requiring them to read braille.

Sign Finder

This project seeks to develop a computer vision-based system that allows a visually impaired traveler to find and read informational signs, such as signs labeling office doors, streets, restrooms and Exit signs.

Link to open source code

Labs

Centers

Collaborators

Internal

Joshua Miele photo

Joshua Miele

Principal Accessibility Researcher, Amazon Lab126