Tactile discrimination of competing sounds

Perception & Psychophysics 1978,24 (6), 546-550 Copyright © 1978 by The Psychonomic Society, Inc. Tactile discrimination of competing sounds B. L. RICHARDSON, D. B. WUILLEMIN, and F. J. SAUNDERS Smith-Kettlewell Institute of Visual Sciences, San Francisco, California 94115 Discriminably different sounds, concurrently presented from the left and right of the medial plane, were reduced in angular separation until subjects could no longer detect which sound was "left" and which was "right." The procedure was repeated with hearing masked and judgments made on the basis of the tactile signals at two fingertip vibrators that received their inputs from two miniature microphones bilaterally located on the subject's head. Auditory and tactile performance were compared under active (head movements permitted) and passive (head held still) conditions. Active and passive performance were not significantly different. Auditory and tactile performance became no better than chance at angular separations of 2.7° and 4.4°, respectively. Touch compared sufficiently well with audition to support arguments for the inclusion of sound localization information in devices which use the skin as a substitute for the ear. Several "artificial ears" (Gault, 1924; Pickett & Pickett, 1963;' Binns, Note 1; Saunders, Hill, & Simpson, Note 2) have been designed to display frequency information on the skin of the deaf person so that various tactile patterns may eventually become associated with the sounds from which they originated. A serious practical problem for users of such devices is that simultaneously occurring sounds (coming from different places in the environment) are difficult, if not impossible, to unscramble (Binns, Note 1). Anyone who has listened to a tape recording of cocktail party conversations will appreciate the point. In audition, the process of "sorting out" competing sounds is called selective listening. Among the variables known to affect selective listening are frequency or voice quality (Spieth, Curtis, & Webster, 1954), sound intensity (Egan, Carterette, & Thwing, 1954), context cues (Treisman, 1960), and perceived location (Broadbent, 1958). Of these cues, perceived location appears to be the most important (Broadbent, 1958; Neisser, 1966). That the skin can be used for the purpose of sound localization has already been demonstrated (Gescheider, 1965). Moreover, Frost and Richardson (1976) showed that an analogue of auditory selective attention could be demonstrated when subjects used a tactile sound localization device. However, the "competing sounds" used in their experiment were This research was conducted at the Smith-Kettlewell Institute of Visual Sciences, San Francisco, California. This institute provided the apparatus for the experiment. B. L. Richardson was on Staff Development Leave from the Applied Psychology Department of the Caulfield Institute of Technology, Melbourne, Australia. B. L. Richardson and D. B. WuiJIemin are now at the Psychology Department of Scarborough College, University of Toronto, West Hill, Ontario, MIC lA4, Canada. F. J. Saunders is at the Smith-Kettlewell Institute of Visual Sciences, San Francisco, California. . 546 separated by 45° or more. Consequently, it was not possible to compare auditory and tactile performance when stimulus separation distances were very small and, as a result, discrimination of the stimuli most difficult. The concern of this experiment was to determine some limits of the skin's aptitude for selective attention and to compare tactile performance with that of normal audition. If tactile selective attention depends upon principles which are similar to those operative in normal audition, the inclusion of localization information in a tactile representation of the auditory signal might provide the benefits that such information apparently affords to normal audition. METHOD Subjects Three males and three females, whose ages ranged from 26 to. 43 years, performed in this experiment. Two subjects were experienced observers in experiments concerning tactile judgments. Apparatus Two Thermo Electron electric condenser microphones (Model 5553), measuring approximately 8 x 8 x 5 mm, were mounted in foam rubber and taped to the earpieces of a pair : of stereophonic headphones. The outputs of these microphones were independently amplified by a Teac Model A 1200 stereo tape deck preamplifier and a Realistic Model SA 10 stereo amplifier. The amplifier outputs activated two vibrators (two 4-ohm, 5-cmdiam speakers with paper cones removed), which were mounted, 10 cm apart, in a piece of foam rubber. This unit was positioned on a table in front of the subject who rested a left-hand fingertip on the left vibrator and a right-hand fingertip on the right vibrator. The subjects were free to use the fingertip of their choice, and optimal pressure was established on a trial-and-error basis. The stimulus sounds were presented from two lO-cm, 8-ohm Radio Shack speakers which were positioned on a shelf 40 em in front of the subject such that the speakers' centers were level with the subject's binaural axis. The combined outputs of the two channels (i.e., the monaural signal) consisted of a series of 10-V, O.5-msec pulses presented at a TACTILE DISCRIMINATION rate of 50/sec. In stereo, the signal was split between the two speakers such that seven consecutive pulses were presented through one speaker (Channel I) followed by two pulses in the other (Channel 2). From Channel 2, the phenomenal impression for both audition and touch was a steady stream of "clicks," and from Channel I, a buzzing sound interrupted at the same rate (see Figure I). Though the "clicks" in fact occurred during the gap between the "buzzes," this was not the phenomenal auditory (or tactile) impression created. At the rate of presentation used, the two signals sounded (and felt) like two sound signals occurring simultaneously and independent of each other's temporal pattern. When, however, the speakers were not directionally separated, the signals were perceived as a single steady 50-Hz train of pulses. After attenuation, the two-channel signal was fed dichotically to the subject's stereophonic headphones (Sharpe, Model HAS) where it produced sounds which totally masked normal auditory localization of the signals. The switching circuit was designed to permit independent alternation of channels from left to right in the speakers and headphones. The pulses were provided by a Model 865C Harrison Laboratories power supply, and pulse parameters were controlled by a KIM I minicomputer programmed to regulate pulse number, duration, and interval. The tactile stimulus intensity was measured at each vibrator for varying positions of the stimulus speaker. For this purpose, the headphones (with microphones attached) were placed on a styrofoam dummy head which faced directly towards the midpoint between the speakers. The voltage measures at each tact or for all experimental speaker positions were transformed to decibel measures relative to the mean detection threshold of four subjects (two subjects were unfortunately not available for these measures). Using the method of limits, six threshold values were measured for each subject and the mean of these values was calculated as 1.1 V. The formula used for transformation was 20 log., (stimulus voltage/mean threshold voltage). Figure 2 shows the decibel values at each tactor as a function of speaker separation. The differences in decibel values between the two tactors was calculated for the speaker positions to the right of the medial plane, and the same calculations were repeated for speaker positions to the left of the medial plane. The means of these differences were calculated and are displayed as a function.•of their associated speaker separations in Figure 3. Since the fingertips applied variable pressure to the speaker coils, their movement was damped and linearity of speaker response to voltage increase was probably destroyed. Consequently, the stimulus values shown in Figures 2 and 3 are not necessarily measures of the proximal stimulus. Procedure On each trial, the subjects were required to report, within the 15-sec period that the stimulus was presented, whether the "click" sound was left or right of the "buzz" signal. Errors and correct responses were recorded and feedback was provided before progressing to the next trial. A table of random numbers was used to determine which signal would be presented left, and which right, but an equal number of left and right presentations appeared within each set of 16 trials. The speakers' horizontal separation CH. .5 rns, 10 volt, square pulses Figure 1. Ten-volt, .S-msec pulses delivered at a rate of SO Hz. ' Channels 1 and 2 received trains of seven and two pulses, respectively. 547 :0- 20.0 "0 ~ V5 zw RR 15. LL I-- 10.0 :E 5.0 Z i= (J) o 0.3 0.6 0.9 1.2 1.5 1.8 2.1 LOG SEPARATION' Figure 2. The stimulus intensity (dB) of the two speakers as recorded at each of the two tactors at the seven separation angles of the speakers. RR: voltage at the right tactor as the speaker moves from center to right. RL: voltage at the right tactor as the , speaker moves from center to left. LL: voltage at the left tactor as the speaker moves from center to left. LR: voltage at the left tactor as the speaker moves from center to right. 10.0 w o zW 8.0 w 6.0 0:: u, u, 0 .0 "0 4.0 2.0 a 0.3 0.6 0.9 1.2 1.5 1.8 2.1 LOG SEPARATION Figure 3. Difference between left and right tactile stimulus intensity as a function of speaker separation. distance (center to center) was varied through seven intervals between 58 and 1.2 em. The angle which the speakers subtended at the midpoint of the subject's binaural axis was thus reduced from 116.0° to 43.0°, 18.(°, 11.4°,7.6°,3.8°, and finally 1.8°. By placing one speaker box on top of the other, it was possible to achieve the four smaller horizontal separation distances. To prevent any possible use of vertical discrimination of stimuli as a cue for horizontal discrimination, the lateral position of top and bottom speakers was varied randomly, as was the vertical placement of each speaker. Judgment accuracy was tested under four conditions: They were active auditory, passive auditory, active tactile, and passive tactile. In the auditory modality, the subjects were blindfolded and judgments were made on the basis of normal audition. In the tactile modality, the subjects were blindfolded, hearing was masked, and· judgments were made on the basis of the tactile stimulus alone.' Active judgments were those made where head rotation was permitted; in passive trials the head was held facing directly forward. Half the subjects began with the passive condition and half with the active. All subjects completed tactile responding before being tested auditorially. Sixteen trials were given at each of the seven RICHARDSON, WUILLEMIN, ANDSAUNDERS 548 separation angles, so that each subject made 112 forced-choice responses in each of the four conditions. It was clearly important to establish that tactile responses were not successful as a result of some auditory cue that remained in spite of the masking sound. Consequently, a control condition was included in which subjects were required to respond while blindfolded and wearing the masking headphones with their fingers removed from the tactors. In this condition, left-right location of the "click" and "buzz" was randomly varied at the speakers and at the stereophonic headphones (Lateral location of the stimuli at the speakers was independent of the lateral location of stimuli at the headphones.) Each of the six subjects was randomly allocated to one of the separation angles. The data for this condition appear in the results section (Figure 7). Each point in the control condition data consists of the total number of errors in 16 responses of one subject responding to the stimuli at one of the separation angles. 12.0 A 10.0 t/) c: Tactile passjve A Auditory active o Auditory passive 8.0 oc: 6.0 c: IJJ Iactlle active • 4.0 2.0 o 0.3 0.6 0.9 1.2 1.5 1.8 2.1 LOG SEPARATION Figure 6. Mean error scores of all subjects in tactile active, tactile passive, auditory active, and auditory passive conditions as a function of separation angle. 12.0 • Tactile ..Auditory 10.0 en 0:: 8.0 0:: 0:: 6.0 0 I.LJ RESULTS 4.0 2.0 o 0.3 0.6 0.9 1.2 1.5 1.8 2.1 LOG SEPARATION Figure 4. Mean error scores of all subjects in auditory and tactile modes as a function of separation angle. Tbe auditory scores are tbe means of auditory active and auditory passive results over tbe seven separations. Tbe tactile scores are tbe means of tactile active and tactile passive results over tbe seven separations. Tbe batcbed lines indicate tbe point at wbicb number of errors were at tbe .05 level of significance for tbe binomial distribution of probability (N = 16, p = 112, q = Yz). 12.0 en ~ 10.0 • Active • Passive 8.0 0:: 0:: I.LJ 6.0 4.0 2.0 o Figure 4 shows that overall auditory performance was superior to tactile (p < .01; dependent t test). However, the patterns of response errors as a function of the sound sources' separation are encouragingly similar for the two modalities. Figure 3 also shows that the separation angle below which performance was no better than chance (p < .05; binomial) was about 2.7 0 (.43 log? separation) for audition and 4.4 0 (.64 log? separation) for touch (according to the binomial probability distributions, withN = 16,p = Ih,q = Y2). A dependent t test analysis of passive vs. active performance revealed no significant differences in mean errors (p > .2); in fact, as can be seen in Figure 5, performance in these conditions was remarkably similar. Figure 6 shows active vs. passive performance within each modality. Figure 7 shows the data recorded during the control condition (see procedure section). The smaller number of data points contributing to the curve accounts for the variability around the "chance responding" line, but an analysis of the data using chi square and a binomial probability test revealed no evidence of any ability to discriminate one sound source from the other when the stimulus sounds were masked and the fingers removed from the tactors. DISCUSSION 0.3 0.6 0.9 1.2 1.5 1.8 2.1 LOG SEPARATION Figure S. Mean error scores of all subjects in tbe passive and active modes as a function of separation angle. Tbe passive scores are tbe means of auditory passive and tactile passive results over tbe seven separations. Tbe active scores are tbe means of auditory active and tactile active over tbe seven separations. The smallest angle at which the two signals could be discriminated auditorially was 2.7 0 , and tactually, 4.4 0 • As Figure 3 shows, at 4.4 0 (0.638 log" separation) the signal originating from the left speaker created a tactile stimulus that was just over 1 dB (relative to tactile threshold) stronger at the left tactor than at the right (similarly, intensity difference TACTILE DISCRIMINATION 12.0 10.0 ~ o ~ 8.0 l.LJ 6.0 4.0 2.0 o 0.3 0.6 0.9 1.2 1.5 1.8 2.1 LOG SEPARATION Figure 7. The control condition scores (total errors) at each of the seven separations. In this condition, subjects were blindfolded, they removed their fingers from the tactors, and wore the masking headphones. of 1 dB between the two tactors was also observed for signals originating from the right speaker). That discrimination judgments were possible on the basis of such small intensity differences seems most encouraging. This is particularly so because tactile judgments were likely made on the basis of intensity information alone (Gescheider, 1965, 1970) while auditory sound localization is achieved with the use of both intensity and temporal information (Mills, 1972). It is possible that, say, a female voice located to the left of the subject could be detected as such while a male voice, to the right, competes for attention. However, success or failure in experiments involving normal environmental sounds could be attributed to the existence of a wide variety of stimulus dimensions along which these sounds may differ. Consequently, we chose two stimuli which were, at the output end, matched in all respects except duration (i.e., number of pulses per train was seven for one channel and two for the other). Under these stimulus conditions, judgment of directional separation of the two sounds would presumably depend upon the intensity differences (for the skin) and intensityItemporal differences (for the ear) resulting from the sounds' angular separation from the subject's medial plane. 549 If, on the basis of such intensity difference information, the skin can be used to discriminate location of competing sounds in a manner which compares favorably with audition, it can be expected that such skills will assist in the localization of normal environmental sounds like voices, telephone rings, and so forth, since, no matter what extra cues these normal sounds mayor may not have, they are, for the most part, subject to the same relative intensity increment and decrement (as a function of azimuth), as were the stimuli used in this experiment. ,. Acuity of single sound source localization has been shown to be better when head movements are permitted than it is when they are not. This is true for normal auditory conditions (Freedman & Fisher, 1968; Wallach, 1939, 1940) and when the skin substitutes for the ear (Frost & Richardson, 1976). It has also been suggested that the ability to perceive the direction from which competing sounds are coming is an important determinant of selective attention. It would therefore appear to follow that, in this experiment, active performance should have been superior to passive. This was not the case for either audition or touch. The following is a possible explanation of this finding. As the head rotates, the difference between the intensities of tactile stimulation that each sound causes at the fingertips varies in a systematic way. At one point of head rotation, these differences may be optimal for the purpose of discriminating that the sounds are spatially separated. There is reason to believe that this point is zero azimuth or, in other words, the point at which the head is facing towards the midpoint between the speakers. In this position, the strength of the tactile stimulus originating from the left sound source will be stronger on the left finger than on the right and an equal intensity difference will also hold for the right sound source's representation at the fingertips. Thus, the superiority (in decibels) of the left sound source at the left fingertip will be equal to the superiority of the right signal at the right fingertip. Moving the head in either direction would tend to destroy this symmetry of intensity differences, though this would not necessarily impair perception, since the optimal intensity differences Table I Mean Error Scores for the Six Subjects in Each of the Four Conditions (Tactile Active, Tactile Passive, Auditory Active, and Auditory Passive) at Each of the Seven Separation Angles Separation (Deg) 116.0 43.0 18.8 11.4 7.6 3.8 1.8 I: X Tactile Active Tactile Passive Auditory Active 0 0.7 0 0 0.5 0 1.3 0.7 0 2.3 2.5 0 2.7 3.0 2.0 5.0 5.0 2.2 7.2 7.2 5.5 18.5 29.5 9.7 2.6 2.8 1.4 Auditory Passive 0 0.7 0.2 0 0.5 0.1 0.3 2.3 0.6 0.3 5.1 1.3 1.2 8.9 2.2 3.5 15.7 3.9 6.0 25.9 6.5 11.3 59.0 14.8 1.6 8.4 2.1 Condition I: X 550 RICHARDSON, WUILLEMIN, AND SAUNDERS would be approached and passed through with each head rotation. Thus the active condition, in effect, included the passive condition. (Consistent with this interpretation is the fact that active performance was not inferior to passive but closely paralleled it.) In summary, it is suggested that there was no difference between active and passive performance because, in the conditions of this experiment, no extra information was made available as a result of head movements. If the midpoint between the speakers had not always been coincident with the subjects' medial plane, head movements may have become important since they would facilitate finding the direction of this midpoint and the associated optimal intensity differences. This possibility can certainly be investigated. For the present, the results of this experiment provide support for the view that tactile "artificial ears" should include a localization unit, since, with such a unit, the skin can process complex patterns embedded in which are signals from more than one location. That tactile performance in this task compared favorably with auditory performance lends further support to this argument. REFERENCE NOTES 1. Binns, P. T. "The K amplex" tactile artificial ear and its use with the deaf-blind. Report from Perkins School, Watertown, New York, 1968. 2. Saunders, F. A., Hill, W. A., & Simpson, C. A. Hearing substitution: A wearable electrotactile vocoder for the deaf. Unpublished manuscript, Smith Kettlewell Institute for Visual Sciences, San Francisco, 1976. REFERENCES BROADBENT, D. E. Perception and communication. New York: Pergamon, 1958. EGAN, J. P., CARTERETTE, E. C., & THWING, E. J. Some factors affecting multi-channel listening. Journal of the Acoustical Society ofAmerica, 1954, 3, 774-782. FREEDMAN, S. J., & FISHER, H. G. The role of the pinna in auditory localization. In S. J. Freedman (Ed.), The neuropsy- chology of spatially oriented behavior. Homewood, 111: Dorey, 1968. FROST. B. L, & RICHARDSON, B. L. Tactile localization of sounds: Acuity, tracking moving sources, and selective attention. Journal of the Acoustical Society ofAmerica, 1976, 59,907-914. GAULT. R. H. Progress in experiments on tactual interpretation of oral speech. Journal ofAbnormal and Social Psychology, 1924, 19, 135-159. GESCHEIDER, G. A. Cutaneous sound localization. Journal of Experimental Psychology, 1965, 70,617-625. GESCHEIDER, G. A. Some comparisons between touch and hearing. IEEE Transactions on Man-Machine Systems, 1970, MMS-ll, 28-35. MILLS,A. W. Auditory localization. In J. V. Tobias (Ed.). Foundation of modem auditory theory, New York: Academic Press, 1972. NElSSER, U. Cognitive psychology. New York: Appleton-CenturyCrofts, 1966. PICKETT, J. M., & PICKETT, B. M. Communication of speech sounds by a tactual vocoder. Journal of Speech and Hearing Research. 1963.6, 207-222. SPIETH, W., CURTIS, J. F.,& WEBSTER, J. C. Responding to one of two simultaneous messages. Journal of the Acoustical Society ofAmerica, 1954, 26, 391-396. TRElSMAN, A. M. Contextual cues in selective listening. Journal of Experimental Psychology, 1960, 12, 242-248. WALLACH, H. On sound localization. Journal of the Acoustical Society ofAmerica, 1939, 10, 270-274. WALLACH, H. The role of head movements and vestibular and visual cues in sound localization. Journal of Experimental Psychology, 1940, 27, 339-368. NOTE 1. It has been pointed out to the authors that since the tactile stimulus went through a series of electronic components (microphone-amplifier-vibrotactors) before becoming "proximal," a better comparison of auditory and tactile performance would have been possible if the auditory stimulus was delivered through stereophonic headphones which received the output of the microphones. The point is acknowledged. However, it appears to the authors that audition was more likely to benefit from the difference than was touch, since the auditory signal was relatively free from distortions or noise to which the tactile system may have been exposed. (Received for publication October 27, 1977; revision accepted August 7, 1978.)