Accessible Design for the Blind
P.O. Box 1212
Berlin, MA -1503
September 10,1993
Research partially supported by grants from the California Council of the Blind and the Association for the Advancement of the Blind.
With the passage of the Americans with Disabilities Act (ADA), the Architectural and Transportation Barriers Compliance Board (ATBCB) has raised questions as to how the environment should be labeled for blind, visually-impaired and other print-handicapped travelers.
"In areas where overhead signage is typically provided, such as conference centers and bus stations, how can information on these signs be made accessible to persons who use raised and Brailled characters?" (Federal Register, January 22, 1991)
The final Rules and Regulations stated that:
"Although technology is available for making overhead and remote signage accessible, the Board plans to further study this issue to determine where and in what type of buildings and facilities such technology may be necessary for future revisions of the guidelines." (Federal Register, July 26, 1991)
More recently (Federal Register, Dec. 21,1992), the ATBCB requested comments concerning new signage technologies for accessible street identification and pedestrian crosswalks. Specifically, the Access Board states that these emerging technologies "...suggest that tactile signage may not be the most effective way of providing such information..." and that the application of "...communications technology may present options preferable to tactile signage, such as the transmission of street names...."
Because of the desirability of a uniform signage standard, especially in cases where a separate receiver would be required to be carried by users, and because of the undesirability of making retroactive hardware changes in any large-scale signage installation, and finally because of the need for signage to be as effective as possible, it is clearly of the first importance to perform rigorous comparative evaluations of various technologies as early as possible in the development process. This paper reports an evaluation of the two most widely discussed technologies, conducted at the American Council of the Blind (ACB) Annual Convention in San Francisco, July 4-10, 1993.
The two technologies were represented by Verbal Landmarks (VL) and Talking Signs (TS). The VL technology is based on an inductive loop system; when a portable receiver is brought within range (5 feet) of a transmitter, a speech message can be heard. The TS technology is based on an infrared beam system; when a portable receiver is within range (10 to 60 feet) and aimed in the direction of a transmitter, a speech message can be heard.
There are several notable differences between these two technologies. Most generally, the inductive loop device is non-directional, the infrared beam device is directional. Consequently, the TS signal itself, independent of its message content, provides wayfinding information; a user typically navigates by following the direction in which the signal is strongest, leading to the actual sign location. The VL signal does not provide this type of wayfinding guidance. On the other hand, an infrared system is a so-called "line-of-sight" system, meaning that signal reception depends on a clear line-of-sight between receiver and transmitter; obstacles such as people or furniture can obstruct reception. A radio frequency device, such as the VL inductive loop, is not affected by such obstacles; a clear line-of-sight is not required between receiver and transmitter.
The other major difference between the two technologies is in the nature of wayfinding instructions contained in the speech messages; this difference is probably a consequence of the differences in directionality mentioned above, and can be summarized as the difference between "giving instructions" and "labeling locations." A typical VL broadcast contains precise wayfinding instructions to a number of destinations. A typical TS signal, by contrast, contains only a short message, such as "Men's Room", "Information Booth", etc., serving as a label for the location from which the signal is beamed. In a typical installation, a small subset of TS signals is used to provide layout or other wayfinding information that can be used to supplement location labels. In these cases, the messages may be longer, and the actual position of signal source is not used for wayfinding.
Conducting this study at the ACB Convention had a number of advantages and disadvantages. The convention represented an extraordinary opportunity to collect both performance data and subjective feedback from a large and diverse group of visually-impaired persons, most of whom have extensive experience with and interest in the problems addressed by these technologies. In addition, the "real-world" setting of a busy convention site in a major hotel provides unquestionable face validity to the study and undoubtedly helped to highlight strengths and weakness' of the technologies that might have been less apparent in the laboratory.
There were disadvantages to the site, as well. First, installation of the two technologies was performed by the vendors themselves; although this potentially permits evaluation of the vendors' competence in this respect, it creates a potential confound between the competence of the installation and the effectiveness of the technology itself. Because, as we discovered in the course of this study, effective installation is by no means a trivial task, this consideration will recur in the discussion section, below. Second, under the generally hectic and unpredictable circumstances of the convention, it was not possible to precisely equate subject variables across the two groups used in the experimental component of the evaluation. Consequently, a further potential for confound in the results arises, and additional statistical analyses are necessary to rule out such confounds, wherever possible. Further discussion of this matter is deferred to the results section.
All research was conducted in the context of the annual convention of the American Council of the Blind, a national organization of persons who are blind or visually impaired, July 4-10, 1993, at the San Francisco Airport Marriott Hotel. Vendors of the VL and TS technologies were invited to install transmitters providing wayfinding information for the first two floors of the hotel, where most convention activities were held. Information was available to the vendors regarding the use of various hotel facilities by the convention, but no further information was provided regarding placement or message content of transmitters. Both were simply instructed to apply their technologies in the way that they were intended to be applied. Both had the opportunity to survey the hotel well in advance of the convention. This was the most extensive demonstration either technology had received to date, as well as the first time human performance data had been obtained for each technology in a large, complex environment, by a large group of persons who are blind or visually-impaired.
"Verbal Landmarks" used five transmitters to label the hotel, while "Talking Signs" used 45. Each vendor also supplied a quantity of receivers to be used by persons having visual impairments.
Opportunity to participate in research on audible signs was widely announced to all convention attendees, and any who wished to participate were welcomed. There were two ways in which interested persons could participate; one was referred to as the "experiment," and the other as "guided exploration."
In the experiment, participants received brief instructions in the use of one technology and then traveled three predetermined routes in the hotel using the information provided by the audible signs plus their normal travel aids, techniques and skills. The experimenter (B.L. Bentzen) accompanied them at all times, and collected data on time to travel each route, distance actually traveled from each start point to each destination, number of times participants needed to return to the starting point and try again (because they did not think they would be able to reach the destination without assistance), and the number of times they mistakenly identified a destination. Following completion of the routes, participants completed a brief set of Summary Questions based on their experience with the particular technology they used to travel the routes.
In the guided exploration, participants were briefly familiarized with one of the technologies and then asked independently to travel two of six pre-determined routes in the hotel, following which they could continue to use the technology for a total of one and one-half hours for further exploration of the hotel. At the end of the allotted time, or when they desired to return their receiver, participants completed both the Summary Questions and an additional brief Survey based on their experience with the technology.
The convention venue was a challenging environment for any signage system and a challenging one in which to conduct research. Two of the three routes used in the experiment, and four of the six possible routes used in the guided exploration, traversed portions of an open area containing the main convention information desk, the sign-up table for this research, restrooms, telephones, a large buffet table which was in service about 75% of the time the research was in progress, and tables and chairs. At most times, it was also crowded with people, many of whom were needing various kinds of assistance. Thus, interruptions to the travel of participants were more the rule than the exception. Eager volunteer guides sometimes unwittingly provided participants with information that enabled them easily to reach destinations which would otherwise have been more difficult.
With the exception of principal investigator, B. L. Bentzen, all other project staff were volunteers. The primary role of the principal investigator in day-to-day running of the research was to collect human performance data and to complete the Summary Questions with each participant following the travel of the three pre-determined routes.
Volunteer E. Lozano, member of the Environmental Access Committee of the American Council of the Blind, coordinated all other staff activities. Familiarization with use of the two technologies was provided by volunteer orientation and mobility specialists or orientation and mobility specialists-in-training, or by representatives of the two vendors (each providing familiarization only in the use of the technology he or she represented). Activities carried out by other project volunteers included scheduling participants, checking receivers out and in, completing participant information forms, and completing guided explorations Surveys and Summary Questions.
There was great interest among convention-goers in participation in this research. All interested persons were given the opportunity to participate, without regard to amount of vision, travel aid, or other characteristics. Interested persons were encouraged first to participate in the experiment, using one technology, and then to participate in a guided exploration, using the other technology; however, there were other options for participation. If participants were unable to sign up for both the experiment and the guided exploration, they were encouraged to complete two guided explorations, one using each technology. Participant information, for both the human performance experiment and the guided exploration, is given below.
A total of 40 people participated in the experiment. Nineteen participants used the VL technology and 21 used TS. Data on travel aid are given in Table 1 below. An additional three participants who used residual vision as their principal means of wayfinding completed the experiment but were not included in the data analysis; these low-vision participants are discussed briefly in the Summary Questions results section below.
Table 1. Human performance experiment, participants by travel aid.
Device Total Participants Cane Dog VL 19 10 9 TS 21 17 4 TOTAL 40 28 12
Because of the "field" conditions under which this experiment was conducted, it was not possible to control the composition of the two experimental groups with respect to subject variables. Note that, in fact, nearly half of the VL group relied on a dog guide as primary travel aid, whereas about four-fifths of the TS group were long cane users. This difference introduces the possibility of confounds in interpreting the results. Additional analyses are included in the results section, below, to test possible alternative explanations of any significant results.
A total of 89 people participated in the guided exploration. Fifty-three participants used the VL technology and 71 used TS; included in these figures are 35 participants who used both technologies. In addition, 27 of the participants in the guided exploration (13 VL, 14 TS) had already participated in the human performance experiment, in which case they used which ever technology they had not used previously. Data on travel aid are given in the Table 2 below.
Table 2. Guided exploration, participants by travel aid.
Device # Participants Cane Dog No aid VL 53 30* 17 5 TS 71 43* 20 7 TOTAL 124 73 37 12
*Missing travel aid data for one participant
"Verbal Landmarks": `Each transmitter consisted of two parts: 1)a metal box (approx. 13"x13"x5") which contained the 110 volt power supply, message unit and transmitter driver electronics and 2) a copper wire coil on a wooden frame (approx. 13" x20") mounted above the ceiling where verbal directions (serial or hand-off instructions) were given. Each receiver consisted of a pickup coil and amplifier in a small handheld box. A separate speaker was attached by way of a coiled cord. Participants heard a digitally synthesized (DECtalk) message whenever the pickup coil in their receiver was anywhere in proximity to or within the perimeter of the inductive loop. In this implementation, the synthesized speech directly modulated (AM) the magnetic field at auditory frequencies.
There was one VL transmitter installed at each of the following five locations: (1) inside the hotel main entrance; (2) near the hotel registration desk; (3) at the top of the escalator to the mezzanine; (4) near the conference information desk on the mezzanine; and (5) at the top of stairs on the mezzanine going down to meeting rooms on the first floor. Each transmitter gave verbal directions from that location to a number of destinations. The transmitter at the conference information desk transmitted the following message:
"For information desk, walk fifteen steps south, desk on your right. For meeting room Marina Del Rey, walk five steps north, door on your left. For Monterey room, walk sixteen steps north, door on your left. For Santa Barbara room, walk thirty-two steps north, door on you left. For Santa Clara room, walk forty-four steps north, door on your left. For telephones, walk east twenty steps, turn left, walk three steps, then turn right and walk three more steps, telephones on the wall to your right."
All VL messages followed a similar format; that is, a destination, the direction of travel to that destination (typically in cardinal directions), and the number of steps in that direction.
"Talking Signs": Each infrared transmitter consisted of a rectangular plastic box (4"x2"x1") which contained the 9 volt battery, the message unit, transmitter driver electronics and three light emitting diodes (LED). The diodes had a dispersion angle (cone) of 56 degrees. The transmitters were placed either on the object (such as direct labeling of doors to rooms) for label-type messages (e.g., "Irvine Room," "Conference Information") or fastened to the ceiling in cases where verbal directions (serial or hand-off instructions, e.g., "Through restaurant to Junior Ballroom area, restrooms, and telephones.") were given. Each receiver consisted of a small handheld box containing photodetector, FM discriminator, amplifier, and internal speaker with grill. Participants heard the digitally recorded message whenever the front of the receiver was pointed in the direction of the infrared transmitter. The transmitters were adjusted to have a range of transmission from 10 feet to 60 feet, depending upon the function of each sign. The pre-recorded speech frequency modulated (FM) a 25 kilohertz carrier.
TS used 45 transmitters to label the first two floors of the convention site. Ten out of 45 were used for "hand-off" messages rather than labeling purposes, and were located at "funnel" points, such as the bottom of stairs where a transmitter might disclose a number of destinations reachable from the top of the stairs.
A hand-held digital stopwatch was used for measuring time of travel, and a measuring wheel was used for measuring the distance each participant traveled from "start" to "destination."
All participants were asked to travel three routes for the experiment, in the same order, regardless of which technology they used.
Given the differences in the two technologies and in the number of transmitters used for each, it was a challenge to create are search design which was not biased in favor of either technology. As the likelihood of successful use of VL was considered to be low if participants were placed outside the range of VL, participants always started routes within range of transmitters of both technologies relevant to those routes. For two of the three routes, participants were positioned directly under VL transmitters, at a point of clearest reception. For the third route, participants were positioned near the limit of the area of clear reception for the VL. (They could hear the message, but it was not very clear.) Participants using TS had to scan the environment from the starting location in order to find and identify a relevant sign. There were typically at least four signs detectable by scanning 360 deg. at the starting location. Because participants had only to activate their receivers to hear the relevant VL message, while they had to actively scan the environment to pick up and identify the relevant TS message, this design may have been biased in favor of VL. In actual use, travelers using VL would have to search for relevant transmissions by moving from one location to another, while travelers using TS would often (but not always) be able to pick up relevant transmissions by scanning the environment from an initial location. Thus, in the experimental design, the sub-task of finding a relevant VL message was probably easier than it would be in actual use of a VL system, while the sub-task of finding a relevant TS message was fairly comparable to the demands in actual use of a TS system.
Both systems, as implemented in this setting, required users to make some "reasoned guesses," as they would in the absence of audible signage, as to the location of relevant destinations (or information about those destinations). Independent blind travelers typically have a repertoire of "reasoned guesses" based on environmental redundancy, which lead them, for example, to anticipate that on the first floor of a hotel they will likely find a registration desk and a concierge nearby, and that there is likely to be a bank of elevators leading to rooms, and also stairs or escalators leading up or down to function areas (Foulke, 1973). Neither of the audible signage technologies compared in this research, as implemented in this setting, is intended to completely eliminate the spatial reasoning demands of independent travel in complex environments.
Route factors which were systematically varied included distance, number of turns, obstacles, and stairs. In addition, initial heading, and knowledge of initial heading, was varied across routes. In the case of VL, because instructions are given in terms of compass heading, knowledge of heading is potentially quite important.
Route 1: For this route, judged by the researcher to be the easiest, participants began under a "Verbal Landmarks" transmitter located near the end of the hotel registration desk, and traveled to the hotel gift shop, a straight line distance of about 14 feet in a southeasterly direction. The route typically required one turn, regardless of technology. All participants were positioned facing South. Participants using "Verbal Landmarks" were told that they were facing South; participants using TS were not given a heading.
Route 2: This route began under the "Verbal Landmarks" transmitter located near the convention information desk, and ended at a men's or women's room (whichever was appropriate) about 35 feet away, in a straight line. However, participants could not typically travel in a straight line because of the presence of tables, chairs, poles, people, and dog guides. The starting position was equidistant from the men's' and women's' room, and the room and furniture arrangement was normally symmetrical between the starting point and the two destinations. Participants were positioned at random headings; they were not told which direction they were facing.
Route 3: This route also began near the convention information desk on the mezzanine, but ended at a men's or women's room (whichever was appropriate) on the first floor, described by the experimenter as being at the "bottom of the stairs down to such rooms as the Anaheim Room and the Irvine Room.." The men's and women's rooms were equidistant from the starting point, and were symmetrically located in a small alcove off a large hallway. The route typically required at least three turns, as well as location and negotiation of stairs. Participants' right hands were placed on the information desk, and they were faced South; they were not told which way they were facing.
Regardless of whether a person was participating in the experiment or a guided exploration, the first step was completion of a participant information form with the assistance of a volunteer who read the questions to the participant, and recorded the answers. Next, the participant was familiarized with one of the technologies. The familiarization lasted a minimum of five minutes, involved travel to at least one destination using the information provided by the technology, and required the use of messages which were not required in the routes to be traveled in the experiment or the guided exploration. (As one of the five "Verbal Landmarks" transmitters was difficult to pick up and understand, and all four remaining transmitters were utilized in the experimental routes, initial familiarization with "Verbal Landmarks" had to include a transmitter which would subsequently be used in several routes; however, the participant traveled from this transmitter to a destination which was not selected for either the experiment or the guided exploration. Any bias introduced by using one of the same VL transmitters for training and experiment should have resulted in increased speed and decreased distance in travel on route 1. As will be seen in the results, below, this does not seem to have influenced the direction of results.)
A brief description of the experiment was read to participants, and they were asked to sign a consent form. They were then asked to rate the extent of their familiarity with each of the three routes to be traveled; "Yes", if they thought they could probably travel the route without assistance, "Maybe", if they thought they might be able to travel the route without assistance, and "No", if they thought they were unable to travel the route without assistance. Following this subjective rating, they traveled the four routes. For each route, participants were guided by the experimenter to the start, told their destination, and then they turned on their receivers when they were ready. The experimenter activated a stopwatch when the receiver w as activated. Participants traveled as quickly and directly to the destination as possible, using their usual aids, techniques and skills, plus the information provided by the audible signage. As participants walked, the experimenter followed behind with a measuring wheel. When participants reached the destination, they said "Here it is," and the experimenter stopped the stop watch and recorded the elapsed time and the distance traveled.
In the event that participants became frustrated and did not think they could reach the destination without other assistance, they notified the experimenter, and turned off the receiver. The clock was then stopped, and participants were taken back to the beginning of the route. When they turned the receiver on again, the clock was started again. Travel distance did not include distances back to the beginning of routes.
In the event that participants said "Here it is," but were not at the correct destination -- (for example, they might have been at the Marina del Rey Room instead of the Monterey Room) -- they were informed that they were not, in fact at the destination, and they continued to look for the destination.
During the time that participants were traveling the experimental routes, the experimenter provided encouragement and reminders specific to using the two technologies (i.e. "Be sure you are following the clearest signal," or "Remember to scan the entire area," in the case of "Talking Signs," or "Remember to get a good, clear signal," or "Try turning the receiver to get a clearer signal," in the case of "Verbal Landmarks"). No additional route information was provided to participants.
Following completion of the routes, the experimenter asked participants the Summary Questions, and recorded the answers.
After being familiarized with the use of one of the two technologies, participants were asked to rate their familiarity with six routes, using the same scale as for the experiment. They were then asked to travel two of these routes, preferably ones which they did not think they would be able to travel without soliciting assistance. They could then spend some additional time exploring the hotel using the technology if they wished to do so (up to a total of 1 1/2 hours). When participants returned the receivers, they completed a Survey and Summary Questions with the assistance of a volunteer. Some of these six routes began outside the range of relevant VL transmissions and also of relevant TS signals; thus they required both more spatial reasoning and more searching for transmissions than was required in the experiment. Furthermore, one route required two level changes as well as multiple turns.
Participants were not systematically observed as they traveled the routes, nor was assistance offered by project staff. It is probable, however, that volunteer convention guides and other persons in the hotel intervened with assistance or information from time to time, whether or not participants sought assistance. There was no external control to be certain that participants did, in fact, travel the routes that they had pre-selected.
TS (n=21) (feet) VL (n=19) (feet) TS (min.) VL (min.) Route 1 21.65**(n=20) 33.47**(n=19) 0.39* 0.86* Route 2 50.86* (n=21) 103.92* (n=13) 1.60* 3.05* Route 3 109.37* (n=19) 161.55* (n=11) 3.26* 4.67* Overall 59.90* (n=17) 89.58* (n=10) 1.68* 2.62** p<0.05
For both distances and times, Table I shows a consistent pattern of performance advantage for the Talking Signs technology. A mixed-design ANOVA, comparing VL and TS collapsed across the three routes, was significant for both distance F(1,27)=4.72, p<0.03) and time of travel F(1,27)=5.51,p<0.027). T-tests (two-tailed) were also performed on data for individual routes, allowing the inclusion of data from participants who did not complete all routes. As shown in the table, differences for routes 1 and 2 were significant, and for route 3, marginally significant.
Table 4. Requiring assistance or reaching an incorrect destination.
TS VL # times needing assistance 5 19 # times reaching incorrect destination 1 2 # participants needing assistance 1 8 # participants reaching incorrect destination 1 2 # participants who either needed assistance or reached an incorrect destination 2 8
A 2X2 chi-square, Figure 1, on the total number of participants either needing assistance or reaching an incorrect destination, with Yates correction for continuity, was 4.11, p<0.05, showing that significantly more users of VL had difficulty reaching the correct destination.
# S's not needing assistance or reaching correct destination. TS 19 VL 11 # S's needing assistance or reaching incorrect destination. TS 2 VL 8
As mentioned above, a number of participants "gave up" on one or more of the routes that they attempted. A chi-square analysis was performed to compare VL and TS with respect to this performance measure. The 2X2 chi-square, Figure 2, with Yates correction, was 8.09, p<0.01, showing that users of VL "gave up" significantly more than did users of TS.
Figure 2. Chi-square, # of times participants "gave up", human performance experiment.
# of completed routes # of routes on which participants "gave up" TS 60 3 VL 43 14
As mentioned in the introduction, conduct of this experiment in the context of a busy convention created some difficulties in control of subject variables. Although experimental groups are never precisely equivalent, the varying proportions of the two groups with respect to travel aid (cane vs. guide dog) raise the question of whether, or to what extent, the results of the human performance experiment can be explained on the basis of this difference in subject groups. To test this possibility, the original analyses were re-done for cane-users only, eliminating the potential confound. It was not possible to perform the analysis across users of guide dogs only, because too few users of TS fell into this category.
For cane users only, one finds again a consistent pattern of performance advantage for TS, except that the difference in the case of route 3 is quite small. Note, however, that four of the ten VL users failed to complete route 3 and are thus not represented in these data (2 of 17 TS users failed to complete route 3). A mixed-design ANOVA, for distance and time of travel, was not significant; however, for the one route completed by all participants, route 1, a t-test was significant. Similarly, 3 of 10 VL users either reached an incorrect destination or required assistance, while 2 of 17 TS users were in this category, indicating a possible performance advantage for TS; the chi-square test was not significant. Finally, VL users gave up on 8 of 30 attempted routes; TS users gave up on 2 of 51 attempted routes. Chi-square, with Yates correction, was 7.05, p<0.01.
The consistent performance advantage of TS over VL that appears in this analysis of the cane-users subgroup strongly suggests that the composition of the sample groups with regard to travel aid did not play a significant role in creating the effect. Although these performance differences are not always statistically significant, this fact must be balanced against the fact that the subgroups are smaller in size, and that (in the case of the ANOVA) the difference between technologies is systematically underestimated by the exclusion of participants who failed to complete routes.
Table 5. Guided Exploration, survey data on independent route travel.
TS VL # participants 54 44 # routes attempted: total 96 78 # successes 92 (96%) 65 (83%) # failures 4 (4%) 13 (17%) # routes needing assistance: total 15 (16%) 32 (41%) # routes participant needed one assist 11 (11%) 18 (23%) # routes participant needed two assists 2 (2%) 6 (8%) # routes participant needed >2 assists 2 (2%) 8 (10%) # participants needing assistance 13 (24%) 20 (45%)
A chi square analysis was performed to compare the number of users of each technology reporting need for assistance during guided exploration. A 2X2 chi-square, Figure 3, with Yates correction, was 4.96, p<0.05, showing that significantly fewer users of TS reported need for assistance.
Figure 3. Chi-square, # of participants needing assistance, Guided Exploration.
# of participants not needing assistance # of participants needing assistance TS 41 13 VL 24 20
A chi square analysis was performed to compare the proportions of successful and unsuccessful navigation attempts with the TS and VL devices. A 2X2 chi-square, Figure 4, with Yates correction, was 6.275, p<0.05, showing that users of TS reported significantly fewer failed attempts.
Figure 4. Chi-square, # of failed route attempts, guided exploration.
# of successful attempts # of failed attempts TS 92 4 VL 65 13
Ratings of ease of use and of speed, for each technology, were also collected in the Survey; ratings were made on a five-point scale, with "1" being much easier (much faster), "3" being about the same, and "5" being much more difficult (much slower), relative to the individual's usual wayfinding methods.
For VL users, the mean rating for ease of use was 3.24 and the mean for speed was 3.22; for TS users, the mean rating for ease of use was 1.69 and the mean for speed was 2.13. In other words, the addition of VL to participants' usual wayfinding techniques tended to be rated as making travel somewhat more difficult and slower, and the addition of TS to participants' usual wayfinding techniques tended to be rated as making travel somewhat easier and faster. Comparison of the means for ease, t(80)=6.19, p<0.01, and for speed, t(81)=4.34, p<0.01, shows that these differences between technologies are statistically significant.
Participants in both the experiment and the guided exploration rated the technologies on a variety of measures. A five-point scale was used, with "1" meaning "strongly agree" and "5" meaning "strongly disagree". Mean ratings are given in Table 6.
Table 6. Summary question ratings.
TS (n=77) VL (n=59) System is easy to use 1.79 3.17 Messages are easy to pick up 2.06 3.43 Messages are easy to understand 1.64 2.97 If signage were abundant, would want to have a receiver for trips in familiar areas 2.19 3.09 If signage were abundant, would want to have a receiver for trips in unfamiliar areas 1.35 2.18
T-tests comparing each pair of means showed highly significant differences (p<0.01) in each case. The number of participants ranged from 132 to 136, the small variation being due to the fact that some participants did not answer all questions; 59 participants used the VL technology and 77 used TS.
It is of anecdotal interest that four of the five legally blind participants whose only travel aid is residual vision indicated that they would use audible signage (VL and/or TS) for travel in unfamiliar areas. One indicated that it would be of use only in familiar areas. Also of interest is the case of one completely blind participant who travels only with the aid of a companion; this participant indicated that he would use audible signage (TS) for travel in unfamiliar areas.
A total of 43 participants had the opportunity to use both technologies; in this case, they were also asked to rate the two devices comparatively, on a five-point scale ranging from "1", meaning "much better than" to "5", meaning "not nearly as good as." Thirty-seven participants (86%) rated TS either "better" or "much better" than VL; four participants (9%) rated VL either "better" or "much better" than TS; two participants (5%) rated the two technologies "about the same." A 1X2 chi-square, equal to 25.42 (p<0.01), showed that significantly more participants preferred TS.
Finally, participants were asked to answer two open-ended questions:
"What did you especially like about this kind of signage? and "How could this kind of signage be improved?" The responses to these two questions are summarized below.
The total number of respondents was 100. Of these, 40 answered summary questions for both devices, 21 answered only for VL, and 39 answered only for TS; thus, a total of 61 respondents commented on VL, and a total of 79 respondents commented on TS. Note: the numbers in parentheses, below, indicate number of responses, not necessarily number of respondents.
What liked: The most frequent comment in this category, for both devices, was very general: users were thrilled with the basic possibility of access to directional information and freedom from assistance. Nine participants answered that they did NOT like, or there was nothing to like about, VL; one participant responded similarly to TS. Thirteen participants commented that TS were easy to use, only two made that comment about VL.
The most important feature of VL that participants liked seems to have been the precision of instructions (19), i.e., the use of compass directions, exact number of steps, etc..
The most important feature of TS that participants liked seems to have been the precision with which it identified locations (14). Other important features were the feedback provided by beam directionality and loudness (7) and the "perceptual" quality of the system (e.g. active scanning is analogous to "looking around") (5).
How improved: Comments in this category were of two kinds: those that concern the particular implementation at this site, and those that concern the basic design of the technology.
Both systems received the same two common implementation suggestions: (1) improve the ergonomics of the handheld unit, especially by making it NOT handheld, providing an earphone, strap, etc.; also by improving button position, volume control, etc.; (2) improve reception -- too much static. A few users, for both systems, also commented on the need for a positive signal, even when receiver is silent, to assure user of proper function.
The most frequent implementation criticism specific to VL was the poor voice quality ("screechy", "gurgly") of the synthesized speech (DECTalk) (16). Also commented upon for VL was the need for more transmitters (9), and that the messages were some times not straightforward, accurate, or precise enough (7).
The most frequent implementation criticism specific to TS was that descriptive information (especially at choice points) needed to be more precise, more complete, and worded to avoid confusion (15).
The most frequent design criticisms specific to VL concerned the system of wayfinding instructions used (42); many pointed out that compass headings were useless without some means of absolute orientation, and many noted that measuring distance in steps is not only inaccurate (strides vary) but impractical since counting steps requires the undivided attention of the user. Another frequent criticism was directed at the employment of a non-interactive, centralized message covering a large area (25); the list was too long and users did not want to have to wait through the entire list for the information they were seeking; some participants worried about conspicuousness and traffic obstruction associated with standing and listening to along message.
The most frequent design criticisms specific to TS concerned problems locating the signal (14), including echoes, interference from other signs, and obstructions. Two participants found that their dog guides became confused as the participants made continual adjustments in their line of travel in order to "home in" on the signal.
In every measure used, in both the human performance experiment and in the guided exploration, TS showed an advantage over VL; the most significant results have been reported above. The only question that remains is, to what may we attribute this dramatic performance difference? It seems clear that there are two distinct factors behind these results.
First, the two technologies are fundamentally different in terms of the cognitive demands made on the traveler. Users of VL have to determine their compass headings (from information not given by the technology), count steps (correcting for differences in stride length), remember directions, and make accurate turns in direction and degree implied by compass directions. Users of TS have no need for any of these mental operations. While occasionally confused as to the source of a sign (because of beam reflection, for instance), TS users have only to make scanning motions with the hand, analogous to scanning with the eyes, in order to find signs and to walk toward them, despite the detours which might be occasioned by intervening objects, pedestrians, etc. Maintaining a clear message always implies "you are traveling toward the sign," and an increase in message volume and signal clarity always implies "You are getting closer." In this respect, TS is a perceptual-motor device, not a cognitive one.
There is a related point concerning the system of wayfinding instructions used by VL. As indicated by many participants, the use of compass headings and step-counting in this installation seemed undesirable. Although it certainly seems conceivable to employ the VL technology with an alternative system of wayfinding instructions, it seems inevitable that it will remain a cognitively demanding technology. Thus it potentially may be usable by a smaller number of persons who are unable to use print signs than a system such as TS which is less cognitively demanding.
The second factor is a difference not between technologies, but between the completeness of their respective installations. Although VL regards as an advantage the ability to employ a small number of transmitters to cover a large number of routes ,their installation at the ACB convention site was clearly sub-optimal; coverage of the venue was not complete, and participants complained about the need to listen through long lists in order to receive the particular directions wanted.
In general, it is a fact about this kind of real-life evaluation that it is difficult to separate vendor competence from the worth of the technology itself. Because performance depends not only on hardware design, but also on installation specifications, including the choice of verbal instructions and of transmitter locations, it is certainly conceivable that either of these two technologies could have been employed to better effect. (It is also likely that both technologies would show performance benefits with additional training and/or practice available to users.) The reader should be aware that both VL and TS are developing technologies. It is to be expected that both vendors will incorporate suggestions and observations made during this research into refinement of their technologies and their implementation strategies, including message content.
In the human performance experiment, participants' travel distances and travel times over prescribed routes were measured and compared across two different audible signage technologies. A consistent, and statistically significant, performance advantage in both time and distance emerged for the TS technology, suggesting that TS provided a more efficient wayfinding aid for those routes than VL technology. Average times and distances for TS users ranged from 45% to 70% of the average times and distances for VL users.
Human performance data showed that participants who used TS were significantly less likely to require assistance, and also significantly less likely to become frustrated and be unable to independently complete the route than participants who used VL. In addition, self-report data from the guided exploration confirmed the finding that participants who used TS were less likely to require assistance or to give up, suggesting that TS provided a more reliable wayfinding aid for the given routes than VL.
Survey consistently showed significantly better scores for the TS technology than for the VL technology. Rated for ease of use, ease of comprehension of messages, and desirability of application in both familiar and unfamiliar areas, TS was generally rated better than VL, suggesting that the performance measures discussed above were reflected in participants' perceptions of the two technologies.
In addition, use of TS was generally considered to increase ease and speed of travel, while the use of VL was generally considered to decrease ease and speed of travel relative to travel without audible signage.
Overall, this comparison of VL and TS shows a clear performance advantage for TS.
Note #1. Installation and technical support for TS was provided by William Crandall, The Smith-Kettlewell Eye Research Institute (original developers of the TS technology), in association with Jerry Kunz, of Talking Signs, Inc. Installation and technical support for VL was provided by Chuck Hawkins, inventor of VL, and Mark Buell, Western Regional Representative for Verbal Landmarks, Inc.