Test Development

A consistent finding among research studies on the vocational adjustment of mildly retarded workers is that many of these people have problems with social interactions in the employment setting (Foss & Bostwick, 1981; Greenspan & Shoultz, 1981). Given the obvious importance of social interaction skills to the vocational success of mentally retarded people, it is surprising that few assessment instruments have been developed in this area.

To address this need, the Test of Interpersonal Competence for Employment (TICE) was developed at the University of Oregon's Rehabilitation Research and Training Center. The development of the TICE was guided by the Behavioral-Analytic model of test construction (Goldfried & D'Zurilla, 1969). This four-stage model provides both a detailed analysis of the problems of the target population (i.e., mildly retarded work trainees) and criterion behaviors for these problem areas by persons in authority (i.e., community employers).

In the first stage of the model, situational analysis, problematic interpersonal situations were identified. In the second stage, response enumeration, a wide variety of responses to each problematic situation were solicited from mildly retarded persons. The third stage is response evaluation, and in this step, each response solicited earlier was rated for effectiveness by employers in community businesses. Finally, in stage four, the development and evaluation of the measure, the results of the first three stages were developed into an assessment instrument, and appropriate statistical procedures were applied for establishing the measure's psychometric properties. The remainder of this section describes each of these four steps used in the development of the Test of Interpersonal Competence for Employment.

Situational Analysis

Eight behavioral areas were initially identified as domains relevant to job tenure for mentally retarded adults (Foss & Peterson, 1981). These were: (a) following supervisor instructions, (b) responding to supervisor criticism or correction, (c) requesting assistance, (d) accepting a new supervisor, (e) disruptive behavior, (1) being distracted by other people, (g) aggressive behavior, and (h) bizarre or irritating behavior. To understand the content of problems in these eight areas, 18 production supervisors and 18 mildly retarded work trainees in vocational training facilities in Oregon reported problems in these areas through week-long observations (Cheney & Foss, 1984). Also, 18 competitive employers reported problems they had observed in these behavioral areas through structured interviews with project staff.

The above procedures produced nearly 250 problematic interpersonal situations that mentally retarded workers encountered in vocational settings. Table 1 shows the number of interpersonal problems in each of the six categories. These categories provided the content for TICE, and each is briefly described below.

Table 1: Content of Problematic Interpersonal Situations

Problems With Supervisor

As shown in Table 1, the problems reported between supervisors and workers consisted of three major types: (a) handling criticism and correction, (b) requesting assistance, and (c) following instructions. The problem most frequently reported by supervisors involved inappropriate worker responses to criticism or correction. These inappropriate responses usually occurred when the work trainee was corrected for doing the job wrong, or criticized for breaking a work rule.

Problematic situations concerning requesting assistance occurred because the work trainee didn't know when to ask for help, asked for help in an unacceptable manner, or didn't understand who to ask for help. For example, a number of assistance problems occurred because a work trainee's regular supervisor was unavailable, and the trainee didn't know who to ask for help. Finally, problematic situations concerning following instructions generally Occurred when the work trainee was given instructions that s/he didn't understand or an instruction with which s/he disagreed.

Problems Among Co-Workers

Three primary categories emerged from the reporting of these problematic situations: (a) cooperative work behavior, (b) handling teasing and provocation, and (c) resolving personal concerns. The first area, Cooperative Work Behavior, contains three types of problematic situations, including disagreements, conflicts, and distractions between co-workers. All of the interpersonal problems in this category occurred between co-workers assigned to a task that they were to complete cooperatively.

The second category, Handling Teasing and Provocation, includes situations wherein one worker was verbally or physically provoked by another. These situations generally were not related to the actual completion of a task, but instead seemed to occur between workers while they were working. An example here is one worker teasing another about the way s/he is dressed.

The final category in the co-worker area, Resolving Personal Concerns, contains problematic interpersonal situations which generally occurred during breaks or lunch, or at quitting time. These situations can be characterized as ones that require a worker to be able to effectively make or refuse personal requests. For example, a common problem in this area concerns borrowing and loaning things.

The final procedure utilized in this first step of the development of TICE was social validation. Brief descriptions of subsets of the problematic situations shown in Table 1 were sent to 111 production supervisors in vocational training facilities in 12 western states. Respondents were asked to note those situations they had observed in their facility over the past week. At the conclusion of the social validation procedure, 126 problematic interpersonal situations remained in the item pool for use in developing an assessment instrument.

Response Enumeration

The goal of response enumeration was to obtain a wide variety of responses (ranging from ineffective to effective) to the problematic situations identified in the above stage. Mentally retarded trainees in vocational training facilities in Oregon and Washington (N=64) were asked how they would respond to the situations (Peterson, 1982). In addition, 113 mentally retarded vocational rehabilitation clients in Oregon, California, and Indiana were interviewed in order to secure response options for each problematic situation. The results yielded a wide variety of responses to most of the problematic interpersonal situations. Those situations that failed to elicit a range of responses from the interviewees were eliminated, leaving 80 items to be used for final test development.

Response Evaluation

Community employers were asked to evaluate the effectiveness of response options gathered in the previous step. Questionnaires were developed and sent to approximately 500 employers in 12 western states. These employers represented five occupational categories which accounted for over 75% of those jobs in which mentally retarded clients of the Division of Vocational Rehabilitation were placed during the period of 1975-1980. These occupational categories are; (a) service, (b) processing, (c) machine trades, (d) benchwork, and (e) structural work. The employers were asked to evaluate the response options to each situation on a 4-point Likert scale of effectiveness. Guidelines to employers for evaluating effectiveness were that an effective response was one that would resolve the problem, and was appropriate in the work setting. Effectiveness ratings were completed by 117 employers and provided the basis for choosing effective correct answers and ineffective (incorrect answers or distractors) response options for the TICE.

Evaluation of the Measure

The 80 problematic situations remaining in the item pool constituted two major areas: interactions with supervisors and interactions with co-workers. These two areas form the two tests within the TICE. Further, each of these two areas consists of three distinct subareas. Specifically, the Interactions With The Supervisor test contains items concerned with (a) Following Instructions, (b) Requesting Assistance, and (c) Handling Criticism and Correction. The Interactions With Co-Workers test contains items about (a) Cooperative Work Behavior, (b) Handling Teasing and Provocation, and (c) Personal Concerns. These six areas correspond to the problem areas discovered in the first stage of test development, the Situational Analysis phase described earlier.

An orally administered multiple-choice test format was adopted for the test. This format is valid for use with mildly retarded individuals (Halpern, Irvin, & Landman, 1979; Irvin, Halpern, & Landman, 1980) and is highly congruent with the Behavioral-Analytic model of test development. Response alternatives identified by the competitive employers as being effective in resolving the problematic situations were designated as correct answers, while responses that were identified as ineffective were designated as incorrect answers and used as distractors for the test questions.

There were two distinct phases in the evaluation of the measure. In the first phase, three forms of the test were developed. Each form contained identical items and correct answers; however, the incorrect response alternatives varied on the three forms. The goal of this step was to identify items that exhibited adequate psychometric properties and to select those response alternatives that were the best distractors. Secondary-level mildly retarded students in Colorado public schools and mildly retarded workshop trainees in Oregon (N=191) were administered the three forms of the test. Analysis of this data resulted in the final form of the TICE.

In the second phase, TICE was standardized on 454 mildly retarded adolescents and adults in the United States and Canada, The particulars of the standardization process and the psychometric properties of TICE are presented in Section VI, Technical Report.

The data reported in Section VI establishes both the internal consistency and test-test reliability of the instrument. In addition, several studies relating to the validity of TICE are discussed. These studies are supportive of the content, construct and criterion validity of the instrument. In sum, TICE provides education and rehabilitation professionals with a useful measure for identifying mildly retarded adolescents and adults who may need training in the interpersonal skills needed for maintaining employment. TICE can also be used to measure the impact of interpersonal skills training.