Remote Reading for Visually Impaired People:
A New Application for Telecommunications
Technology and Disability
Telecommunications
Volume 3 Number 3 August 1994
W. Crandall, W. Gerrey, J. Brabyn
Smith-Kettlewell Rehabilitation Engineering Research Center
of The Smith-Kettlewell Eye Research Institute
Blind people usually get their information about printed material by asking friends, family or office mates to read to them. It is also not unusual for printed material to stack up for several weeks before a blind person can find the help he needs; often from a professional "reader" who travels to the home of the blind person to provide that service for a fee.
The concept of a person who is print-impaired using a telephone and fax machine to transmit an image of a printed page to a sighted person so that it could be read aloud back to him or her came from a Smith-Kettlewell engineer, Mr. Bill Gerrey, who, himself is blind. Bill says:
"The 'fax project' turned out to be the silver lining to a cloud of frustration. As computers learned to talk and display large print, blind employees were looking forward to "The Paperless Society" in which all documents would be accessible. No more expensive readers and tiresome reading machines! Oopse, what's this? A "fax machine"? There are electronics suppliers I cannot order from unless I do it by fax. O.K., I can still type, but when they fax me the news as to which items are out of stock, I can't read it!" "It was in a fit of rage that I bellowed, 'I ought to fax this fish-wrap to someone and make 'm read it to me!' A rare occurrence for me, the light came on; ten persons like me could fax stuff to a sighted reader. And not just faxes either; I could have the sighted operator read me anything in print -- graphics or what not." Later, Bill quipped, "I'd always dreamed of having a person around whenever I needed something read. What I realized was that fax would allow me to have a reader at my beck and phone call." The idea was eminently do-able -- it was a routine task for modern telecommunication technology and the devices were "off the shelf."
Through the Rehabilitation Engineering Research Center, he was able to buy eight fax machines and test his idea of a "remote reader service" on a trial basis. The project grew rapidly during the next two years, aided by grants from the National Easter Seal Research Program, the National Institute on Disability and Rehabilitation Research, the Sidney Stern Memorial Trust, Allnet Communications Services, Pacific Bell Telephone and The Smith-Kettlewell Eye Research Institute. This funding provided for a project manager, two readers who are individuals with restricted mobility and who telecommute from within their homes, the telephone service and two "fax reading computers" which display the faxed image on their screens.
Statement of Problem:
An analysis of data from the National Center for Health Statistics estimated that 4.3 million people in the US had difficulty reading the newspaper with their corrected vision -- a functional definition of perceived limitations termed Severe Visual Impairment (Nelson and Dimitrova, 1993). Data currently available from the Bureau of the Census puts the figure for this same level of impairment at 9.7 million people (McNeil, 1993). Estimates of tested acuity define 1.1 million people as Legally Blind which is defined as corrected acuity of 20/200 or less and a visual field of 20 0 (Chiang, et al, 1992). Many other disabilities prevent persons from reading print. In addition to people who are blind or have low vision and may not be able to see the print, there are many stroke, head-injured, autistic and dyslexic (or even just educationally impaired) persons who may not be able to assimilate printed language even though they can see the page. Many people can accept this information through speech -- having print read aloud to them.
During a typical day, a sighted person routinely opens mail, reads instructions and recipes, reads TV guides, selects tapes, reads correspondences, invoices and bills, and looks at pictures and graphs. The blind individual has routine access to few, if any, of these.
The phenomenal growth of facsimile usage in offices is a new and unmet challenge to the print-disabled office worker. Facsimile technology is graphical in nature and is often used for the transmission of illustrations and handwriting.
Solution:
All of the above printed materials could be made accessible on demand by the remote human reader system of the type we are developing, using fax machines or other telecommunications technologies to transmit images of the material to a remotely located sighted reader. Our current real-world test has sampled over 800 reading transactions from our participants in order to evaluate the utility and cost-effectiveness of the system. Such information is needed for any future, permanent implementation by service agencies, private industry or regulatory agencies to realistically access technology transfer feasibility of this system. Strong emphasis has been given to designing a system which will exploit, rather than be limited by, the important real-world requirements of economy of scale and flexibility in the face of emerging technologies. In addition, telecommunications makes differences in distances transparent to its users. Blind people in Ohio and Tennessee can access one of the project's readers working in their homes in Oakland or Pittsburg, California as easily as can blind users located anywhere in the Bay Area. This demonstrates the power of telecommunications to overcome physical distances and disabilities.
Telecommuting and telecommuting are becoming significant forces in employment. Many mobility impaired people may, at least on a part time basis, choose to work from their homes. A demonstration of the successful use of telecommunications in augmentative communications may lead to a broader regulatory definition of "telephone" and enhance the opportunities of people who are blind or print-disabled to participate fully in the "Information Age."
The fax reader service has been designed to demonstrate and evaluate the role that telecomputing and telecommuting may play in enhancing the independence of and employment opportunities for significant numbers of people in the disabled community. This topic is of special importance at a time when the rules for the Americans with Disabilities Act (ADA) are being laid down. The Fax Reader Project may have high salience under the provisions on reasonable employment accommodation (which includes access to print by the visually impaired). In addition, the early exploration of the market and technical requirements for remote reader services may provide input into future standard-setting for emerging telecommunications technologies -- particularly those involving transmission of high-resolution, still images.
Figure 1. By using ordinary telephone lines, the print disabled person can have rapid, convenient and cost-effective access to printed information -- at home or on the job.
Participants:
In our experimental remote reader service, the two mobility impaired readers telecompute from their homes daily from 1:00 p.m. to 9:00 p.m. in two 4 hour shifts. Currently, forty-nine blind recipients of fax machines represent a broad spectrum of reading needs. They use the system for both personal and professional reasons and have extended periods in their day during which other types of reading assistance is not at hand. Because this is a telecommunications application, from the Bay Area we serve participants in Ohio as conveniently as we do local people.
Equipment:
Three major equipment components are involved in the task performance: 1. The fax machine is of the "flat-bed" type (Nitsuko FX E-500), which means that it can scan three-dimensional or odd-sized objects, if required. That is, common items such as business cards, magazine articles and food boxes can be identified without altering the form of the original document. The machine we selected also had simple, tactile controls and an audible alarm for operation errors. 2. The computer is a 386DX running at 33 mega Hertz. It contains 4 megabytes for random access memory and a 108 megabyte hard disk drive. The speed of the computer and size of the memory has much to do with the task of running several applications simultaneously; we are collecting data on the reading transaction, running an on-screen "stop-watch" and operating the on-screen controls of the fax card at the same time. A simpler, "reading only" application would not require this enhanced speed -- although this hardware configuration is now considered standard. The two unusual, but off-the-shelf components of our computer system are the monitor and facsimile card. The monitor (Portrait) is termed a "full page display" meaning that its proportions are the same as an upright piece of letter-sized paper. This allows the reader to quickly get an overview the entire page. The high resolution of the monitor is such that under most circumstances, the entire reading process takes place with the image for the whole page filling the screen. The second special component is the "fax card" (Zofax 96/24 or other 'class 2' fax modem) which is connected to the telephone line and stores the fax images on the computer disk (the image is stored before being displayed.) Because we want to have the reader's applications automatically run themselves on computer start-up and wanted automatic, periodic backup of the data files, we chose the Norton "Desktop for Windows" product as the main interface program (running under Windows ). The fax card is under software control provided by "Winfax" (Delrina Technologies). The stop-watch is a freeware application. 3. The telephone service consists of several consumer-grade features which allow great flexibility given the numerous "clients" serviced by a single reader at any one time. Call Forwarding allows one reader to move all client calls to the other reader while on extended break or out for the day. The Message Center allows the reader to receive call-back information from clients if the reader is busy with another client or on a short break. The Time of Day Routing moves the target number for our 1-800 phone service from one reader's house to the other reader's house on a predetermined, daily schedule. (This reduces our phone costs, as one of our readers lives 40 miles from San Francisco and routing the 1-800 phone service through San Francisco incurs toll charges.)
Data Collection:
Individual Transaction Records for each faxed document were entered directly by the reader into an on-screen computer form. This detailed data was recorded over six months of operation to determine the uses our "customers" made of the service.
The Monthly Interview, a brief telephone questionnaire to customers and readers, focused on early identification of problems and user needs. A telephone Exit Questionnaire is administered to each user at the end of his/her tenure. Feedback from users and readers also comes from a Problems and Suggestions Diary kept by the readers.
Data Analysis:
The data was analyzed to determine the breakdown of usage patterns, transmission times, types of documents read, formats of documents, and other user statistics. The data on information type, resolution, font size, etc., is utilized to determine the adequacy of system software and hardware. Cost-effectiveness of the system was based upon reader time, telephone cost, synthetic speech reading machine cost, and data from subjects on the cost and efficiency of visiting reader services they would normally use.
Results and Conclusions:
Of the 26 original blind participants in the study, 10 were considered to be active users where an active user is defined as one whose level of participation is equal to or greater than one document faxed for reading every two service days.
- The average level of participation for each active user was 1.5 documents faxed for reading each service day.
- The analysis of 800 Individual Transaction Records indicates:
- The total participation of all active users averaged 24 documents per 8 hour service day. That is, the readers received an average of 3 documents per service hour.
- The fax transmission time for each document averaged 68 seconds.
- The time to read each faxed document averaged 163 seconds.
- The non-reading time associated with the processing of each document (transmission, entry of statistics, miscellaneous comments, etc.) averaged 150 seconds.
Each participant brings to the project very different reading needs. However, on the average, we offer the following conclusions:
- In addition to typeset information, significant amounts of the information sent by the users to the readers involve handwriting (25%) and graphics (19%). Therefore, "reading machines" (optical character recognition or OCR) alone will not satisfy all reading needs. The fax reader system provides convenient and timely communication for all three forms of information (see Fig. 2).
- Only about 30% of the documents required transmission in high resolution (fine) mode of 200 X 200 pixels per inch. The low resolution (normal) mode is, in terms of the way the readers generally display fax information on their computer screens, quite superior to the quality available in most of the emerging consumer video conferencing and video phone technologies.
- About half of the documents (53%) were read completely. This may mean that OCR would be a helpful adjunct to the human reader in cases where the document is typeset and the text image is "clean" enough to be processed by such an automated system (see Fig. 2).
- About half of the print received for reading was smaller than the standard 12 point type font (48%). This means that people who are moderately visually impaired could benefit from the fax reader system. In the early 1980's, the National Center for Health Statistics estimated that 3 million people had difficulty reading the newspaper with their corrected vision. Current estimates are that 1.1 million people are classed as "legally blind" (acuity of 20/200 or less) (see Fig. 2).
- With some enhancements to the system, the amount of adjustment the reader performs on the document's image (size and position) could be further reduced so as to increase the accessibility of our fax reading computers by persons with more severe mobility impairments. The fax reading system is currently fully controlled by mouse and the statistics are mostly entered into the database by mouse.
- Most of the documents are the size of a standard letter (8.5 X 11 inches) or smaller (88%). This indicates that perhaps ordinary, "single sheet" fax machines would, in the majority of cases, be appropriate for blind people to use for sending faxes. These "single sheet" fax machines are common in the market and are the less expensive than the "flat bed" machines we currently use (see Fig. 2).
Figure 2. Four of the 14 parameters measured for each of the 800 documents read during the initial phase of the study.
Cost Effectiveness:
The data from our pilot shows that 10 of our 23 users were active users (ones who engaged the system with at least 1 document for every two reading days) and that the total average active users activity was 3 documents per service hour. The average transmission/read time of 231 seconds per document sets the maximum capacity of the system at about 15 documents per hour. Therefore, the pilot system could theoretically handle 5 times the number of documents we observed with our 10 active users.
For purposes of estimating the per user cost, we conservatively assumed that our present system will not reach capacity until we at least triple the number of active users. If the study's costs were spread over 30 active users, the per user cost (including manager salary) would be $88 per month. If the manager's salary is excluded, (as it effectively would be in a larger system serving many users) the per user monthly cost would be $47. This is comparable to (or less than) amounts typically paid by blind persons hiring readers for several hours each week. To get some idea of what level of support the users, themselves, were willing to provide, all said they would pay, at some level, for the service they were receiving (It is also, of course, likely that employers would be willing to defray these costs in lieu of hiring sighted readers). It must be noted that in the pilot phase of the R&D project, a significant amount of the user's transaction time was taken in recording survey information.
To compare the cost of the Fax Reader Service to the use of conventional, "visiting" readers, many factors must be addressed. We find in our informal poll that conventional readers are paid in a range from minimum wage to $9.00 per hour. The Disabled Students Program at San Francisco State University, for example, pays their readers $5.96 per hour with a limit of 20 reading hours per week. The University of California at Berkeley pays readers in the range of $5.00 to $7.50 per hour. The number of reading hours a blind individual is able or willing to support is determined by factors such as income, age, level of education, marital status, occupational status and household composition. However, assuming that it is usual for those non-academics who hire readers to do so for six hours each month (at $6.00 per hour), the monthly reading charge for a conventional reader would be $36.
Compare this to the volunteer reader program at a Bay Area agency for the blind where clients are limited to 2 hours of reading each week. According to the agency, it is difficult to find more volunteer readers in order to increase availability.
In another comparison, "Reading machines" have large start up costs and ongoing costs in the form of service contracts. Assuming that the mean purchase price of the three most popular OCR-based reading machines is $6,000 and the mean yearly service contract is $765, a three year amortization of the equipment would result in a monthly cost of $230 to the user.
Marketing Considerations:
The end point of this study is to provide both government and industry with the information they need to realistically determine technology transfer methods, costs, cost-to-benefit and feasibility. Accordingly, we are giving high priority to identifying those who can use and will use the service to benefit. We cannot expect to transfer this from a small scale study to a large and serious implementation without an analysis of the market. Therefore, we are engaged in constructing a survey instrument which will give us high reliability in identifying the profile of active users.
Most users of the system over the past year have been self-selected. That is, most learned about our study from a friend and contacted us with a request to be included. In the second phase of the study we will rely more heavily on referrals by professionals such as low vision specialists, case managers, speech and language pathologists, and vocational specialists. These professionals will have a good assessment of the type of aids practical for the individuals they consult. Therefore, we will be increasing the sample to include persons who are elderly and/or recently blind, persons located in rural settings where volunteer and other reader assistance programs are not readily available, persons who are recovering from traumatic head injury, and those who have a learning disability which would interfere with their reading.
Technological Considerations -- Rapid Image Transmission:
We have made significant progress toward implementing a non-fax medium for an interactive remote reading system. The goal here is to approach as closely as possible the "ideal" remote reader -- one which would function in the same way as having a live reader in the same room as the user. Such a system would, for all practical purposes, allow the remote reader to "look over the shoulder" of the blind user. We refer to this concept as "virtual presence" (Brabyn, et al, 1992).
The concept we are developing employs compressed video technology to allow very low frame-rate, high-resolution video transmission over ordinary telephone lines. The bi-directional transmission of data will allow the remote reader to pan and zoom over the document or object of interest to the user. These functions could be accomplished electromechanically or electronically by command from the reader's system to the user's system. From our initial engineering experiments, we are optimistic that employing these emerging technologies will avoid the delays inherent in the fax-based system and provide a quantum leap in user convenience. The increased efficiency would provide commensurate reduction in cost of operation for each document read. The goal is to design a system which provides the rapid real-time operation of a video system while retaining the proven effectiveness of the fax-based system.
A major limitation of most existing and planned consumer grade tele-video applications is that they are based upon the NTSC standard. This standard limits the image resolution to that of television broadcast. This provides for an image whose detail is minimally acceptable for the useful reading of small printed text, handwriting or detailed graphics (as derived from the video camera input). However, other non-consumer applications, such as transmission of medical images and remote security operations, may provide the commercial incentive for the development of the lower cost, higher resolution image transmission and visualization necessary for an effective visual "virtual presence."
Rapidly advancing technological enhancements being introduced by the telecommunications industry, itself, will add substantially to early realization of our goal. "Switch 56" and Integrated Services Digital Network (ISDN) are the new, high bandwidth telecommunications systems which are currently available in many US cities. In California, for example, Switch 56 service is an ideal next step for a video enhancement to our present system as it affords a 56 kilobit throughput. This represents a six fold increase in transmission speed over the current service, but uses the same "twisted pairs" which traditionally connect residences. In addition, Switch 56 is currently available at 100% of the state's telephone locations. This is especially significant because it is the non-urban environments where we see the remote reader service having the most impact.
The ISDN system has twice the bandwidth (56 kilobit + 56 kilobit) of Switch 56 service, but requires that the telephone be located within six miles of the central switching office. ISDN is projected to be installed in 100% of Pacific Bell's California system by 1997.
A leap forward in the ability of telecommunications to realize the type of virtual presence for blind people being discussed in this paper will be brought about by the implementation of fiber optic networks within the consumer market. A fiber optic/coaxial cable hybrid transmission system is currently being installed in San Diego and Silicon Valley. In this system, a fiber optic cable is brought into each neighborhood and then locally distributed to 500 residences or businesses by coaxial cable. This very high data rate system is scheduled to be ubiquitous in these two areas by 1997. In addition to the more conventional television "cable" functions, it will be the "backbone" for advanced consumer-grade information system.
System Components:
The primary component in the development system is a computer board which stores the incoming video image. The device we use is the AVer 2000 Window Card manufactured by ADDA Technologies.
The video image is formed by a high resolution (410,000 pixel), consumer grade "camcorder" (Yashica "Hi-8") which is mounted on a photographic copy stand over a desk. The camera is on an electromechanical Pan and tilt device which allows the camera to be positioned for best view of the material being read (Our development plans includes remote operation of the camera position by the reader).
The image data is then compressed by a software algorithm LEAD VIEW 32 by Lead Technology. The compressed image is then sent by modem (Satisfxtion by Intel) to the reader's computer for decompression and display. The capability of the compression software extends to a factor of 200. However, we begin to notice a degradation in image quality at compression factors of about 20.
We have tested the individual components of the system. The software to link the functions together and provide a number of user-friendly features is currently under development.
Technological Considerations -- Optical Character Recognition (OCR):
We are investigating ways of augmenting the fax reader service with the option of an automated system by determining the feasibility of integrating optical character recognition (OCR) and speech output into the Remote Reader System.
Clearly, human readers have a tremendous advantage over reading machines for such purposes as communicating random documents (e.g. mail). The human reader can visually scan, evaluate, and describe the document quickly so that the blind person can efficiently make a decision as to its disposition (e.g., read completely, read account balance or disregard document). However we feel that OCR might be useful in processing the subject document once its disposition -- as accessed by the sighted reader -- has been determined. This processing would free up the reader/operator for other customers, being done "off-line" -- storing the spoken text at the reader end for later retrieval at the customer's convenience. A prototype is being developed in which the remote reader selects material appropriate for OCR and suggests to the user that the text be converted to high-grade synthetic speech and stored in a voice-mail address supported by the system. The user could then access the document via voice mail at a convenient time, using built-in features such as scan and replay for transcribing notes.
The prototype we are developing has the following characteristics:
The remote reader selects material appropriate for OCR by considering the length of the text and the clarity of the printing. Documents which contain handwriting or graphics or type-set text which is blurred may be determined to be unsuitable for OCR conversion.
The OCR system identifies the sender by a printed number permanently affixed to the upper left corner of the scan space. This identification number is the voice mail address to which the converted text will be sent for storage.
The user then has access to the spoken text through a voice mail system on the reader's computer. The system is being designed to allow two methods of document retrieval: 1) at a convenient time, the sender calls the voice mail system to access their personal mailbox. 2) the computer will automatically call the sender (after processing the document) and announce that the text is in the sender's mailbox. In either case, the "playback" of the spoken text can be manipulated by features such as scan and replay (for transcribing notes, etc.) by pressing the touch tone keys on the sender's telephone.
We believe that although completely automated systems will not have the positive reading attributes of a remote human reader, users having documents which are easily processed by these systems (i.e.., no graphics or handwriting) may consider such systems to be good enough when human readers are not available (e.g., from midnight to 6 am).
System Components:
The conversion of the fax image into text (OCR) is performed by WinFax Pro 3.0 (Delrina), the fax management software we have used over the past two years for our conventional fax reader system. The actual text-to-speech conversion is provided by Vox Font software (Voice Information Systems). The interface between the computer and the telephone is a voicemail system based on the Dialogic's D/21B two line speech card.
Expanded User Base:
Through donations of new sponsors we have been able to expand the study's user base to include a broader socioeconomic cross section of the clients who are subjects for this experiment. These new clients include more retired persons, homemakers, self-employed persons working in their homes, and students.
Collaborative relationship with Easter Seal Society for the Redwood Coast.
We are fortunate to have been able to establish a close working relationship with our local Marin County Easter Seals chapter (Easter Seal Society for the Redwood Coast). Over the past year, we have donated computer equipment and a number of fax machines as well as consulted on technical and managerial issues in aiding our Easter Seals to establish and enhance their own volunteer fax reader program. In return, we have begun to rely on the skill of our Easter Seal's volunteer services program to select from their client base participants in our joint program.
Conclusion:
Remote reading services for blind persons represents an exciting new application for telecommunications technology. The results presented here demonstrate the usefulness of such systems using existing, low cost fax equipment. From the blind user's point of view, the fax-based reader service can combine the 24-hour immediate availability of a reading machine with the intelligence of a sighted reader, all at a cost significantly lower than purchasing an OCR-based reading machine alone. The future possibilities for even more user-friendly remote reader systems should be greatly enhanced by the emerging wide-bandwidth telecommunications infrastructure and image compression techniques, potentially allowing rapid personal transmission of video and other graphic information from blind persons to sighted readers.
References:
Brabyn, J, Crandall, W, Gerrey, W. Remote Reading Systems for the blind: a potential application of virtual presence. Proceedings, 14th Annual International Conference of the IEEE Engineering in Medicine and Biology Society 1992; 14:1538-1539.
Chiang, Y-P, Bassi, L, Javitt, J. Federal budgetary costs of blindness. The Milbank Quarterly 1992; 70(2):319-340.
Crandall, W, Gerrey, W, Brabyn, J. A fax-based reading system for the blind and print-handicapped. Proceedings, RESNA International '92 1992; 12:463-465.
Crandall, W. Using the fax machine in reading your mail. Paper presented at the American Council of the Blind Annual Convention, San Francisco, CA, July, 1993.
Crandall, W. Smith-Kettlewell's RERC Fax Reader Project and Talking Signs. Paper presented at the American Society of Public Administrators/Cooperative Administrative Support Program Annual Convention, San Francisco, CA, July, 1993.
Gerrey, W, Brabyn, J, Crandall, W. The use of fax technology to address the reading needs of blind and visually impaired persons. Journal of Visual Impairment & Blindness 1990; 84(10):509-513.
McNeil, J. Americans With Disabilities: 1991-92 U.S. Bureau of the Census. Current Population Reports, P70-33, U.S. Government Printing Office, Washington, DC. 1993.
Nelson, K. and Dimitrova, E. Severe visual impairments in the United States and in each state. Journal of Visual Impairment and Blindness 1993; 87(3):80-85.
Acknowledgments:
This research was supported by grants from The National Easter Seal Research Program, The Smith-Kettlewell Eye Research Institute, the National Institute on Disability and Rehabilitation Research, Allnet Communications Services Company, Pacific Bell Telephone, and the Sidney Stern Memorial Trust.
Biographical sketches:
Bill Crandall, Ph.D. came to The Smith-Kettlewell Eye Research Institute in 1973 from the University of Georgia during training for his Masters degree in Physiological Psychology. His Ph.D. is from the University of Pacific in oculomotor neurophysiology and vision research. He served as laboratory manager and co-investigator in Smith-Kettlewell's primate research laboratory until 1990 at which time he joined the Institute's Rehabilitation Engineering Research Center (RERC). In his present position as Scientist, his primary interests are in research, development and technology transfer of the Remote Reader Project and Talking Signs, an infrared remote signage system for people who are blind or have other print-reading disabilities.
William Gerrey, B.S.E.E. received his Electronics Engineering degree from California Polytechnic College in 1971. He began his career at Smith-Kettlewell as a prototyping technician in the Institute's RERC. At present, he is Senior Engineer, adapting instruments, publishing specialized techniques and verbally described plans for devices. He also oversees an experimental electronics-assembly training program for the blind. He describes his job as running a problem-solving laboratory.
John Brabyn, Ph.D. is Co-Director of the Rehabilitation Engineering Research Center at The Smith-Kettlewell Eye Research Institute, San Francisco. He received his bachelor's and doctoral degrees in electrical engineering from the University of Canterbury in New Zealand. His current interests include low vision research as well as development of vocational, reading and navigational aids for blind persons.
Figure Legends:
Figure 1. By using ordinary telephone lines, the print disabled person can have rapid, convenient and cost-effective access to printed information -- at home or on the job.
Figure 2. Four of the 14 parameters measured for each of the 800 documents read during the initial phase of the study.