Technical Report

Standardization Subjects

TICE was standardized with mildly retarded high school students and mildly retarded adults in vocational training workshops. The high school sample was comprised of 331 students from Oregon and Colorado. The average age of this group was 17.04 (sd = 1.45) with a range of 14-21. There were 195 males and 136 females. The average full-scale IQ for this group was 64.94 (sd = 7.45) with a range of 50-79. The sample included students in the following racial groups: Caucasian (n = 20),Black {n=60), Oriental (n=3). Native American (n=5), and Hispanic (n = 60).

The workshop sample was comprised of 123 persons from Oregon, Washington, and Canada. The average age of this group was 26.89 (sd = 7.57) with a range of 17-52. There were 61 males in the sample and 44 females (18 subjects did not supply information on gender). The average full-scale IQ for the group was 68.03 (sd = 8.22) with a range of 44-79, Information regarding race was not obtained for this sample.

TICE was administered to the students and work trainees by participating teachers and workshop staff. All participants were tested in small groups of 5­10 people. After testing was completed, group statistics, item analyses, and reliability estimates were calculated This information is presented in the following section.

Standardization Outcome

Mean, Standard Deviation, and Range

The Interactions with the Supervisor test consists of 31 items and the Interaction with the Co-Worker test of 30 items. The mean, standard deviation, and range of each of these tests for the two standardization samples are presented in Tables 2 and 3. The statistics shown in these two tables indicate that the scores of the two sample groups are quite similar.

Table 2: Mean, Standard Deviation, and Range

High School Sample

Table 3: Mean, Standard Deviation, and Range Workshop Sample

Item Correlations and Difficulty

Item to total correlations (point-bi-serial reliability coefficients) for the Interactions with the Supervisor and the Interactions with Co-Workers tests for the high school and workshop samples are reported in Tables 4 and 5, respectively. The coefficients shown range from .13 to .49 on the Supervisor test and from .15 to .46 on the Co-Worker test for the high school sample. The results are again quite similar to the workshop sample, as these coefficients range from .08 to .50 for the Supervisor test and from,07 to,55 for the Co¬-Worker test. Individual item difficulties for the high school sample are presented in Table 6 and for the workshop sample in Table 7. For the high school population, the item difficulties range from .45 to .89 for the Supervisor test and from .36 to .77 for the Co-Worker test. For the workshop sample, item difficulties range from .46 to .91 on the supervisor test, and from .24 to .77 on the Co-Worker test. These figures indicate that the item difficulties for both tests are, in general, in the moderate range for both standardization samples. It can be seen, however, that the indices from the workshop sample are more variable and also lower for some items as compared to the high school sample. The small size of the workshop sample is a factor here, as it contributes to fluctuations in item statistics. The differences in item statistics between the two standardization samples, however, do not appear to be functionally significant. Other data gathered {e.g., means, standard deviations, reliabilities) indicates that both groups generally performed in a reasonably similar fashion on TICE.

Table 4: Item to Total Correlations High School Sample

Table 5: Item to Total Correlations Workshop Sample

Table 6: Item Difficulties High School Sample

Table 7: Item Difficulties Workshop Sample

Reliability

Coefficient alpha internal consistency reliability estimates were computed for the Interactions with the Supervisor and Interactions with Co-Workers tests with both standardization samples. These reliability coefficients are presented in Table 8 for the high school sample and Table 9 for the workshop sample. For the high school group, the coefficient alpha reliability estimate for the total Supervisor test is .84 and .81 for the total Co-Worker test. For the workshop sample, the coefficient alpha index is .82 for the total Supervisor test and .81 for the total Co-Worker test. The reliability indices are very similar across groups and are within acceptable limits for adequate internal consistency reliability of a test.

Table 8: Internal Consistency Reliability Coefficients

High School Sample

Table 9: Internal Consistency Reliability Coefficients Workshop Sample

As mentioned in Section 1, Test Development, both the Supervisor and Co-Worker tests can be broken down into subareas. The Supervisor test consists of three areas: Following Instructions, Requesting Assistance, and Handling Criticism and Correction. The Co-Worker test also consists of three areas: Cooperative work Behavior, Handling Teasing and Provocation, and Personal Concerns. Coefficient alpha internal consistency indices for each of these subareas are also presented in Tables 8 and 9. Due to the small number of items constituting each subarea, these coefficients are not as strong as for the total test.

Test-retest reliability coefficients were obtained from 32 workshop subjects, using a two-week interval between testings. The Supervisor test yielded a tester-test reliability coefficient of .78, and the Co-Worker test a coefficient of .84. These indexes are well within accepted standards for test-retest reliability.

Validity

There are three major types of test validity: content, construct, and criterion. Efforts have been made to address each of these types of validity in the development of TICE. Brief descriptions of the research conducted in these areas are given in this section and references are provided for the reader interested in a more in-depth understanding of this material.

Content Validity. This type of validity is established by examining the procedures used to develop the content of the instrument. As noted in Section I, Test Development, and in Foss, Bullis, and Vilhauer 0984), the content of TICE was determined by an extensive analysis of the interpersonal problems of mildly retarded work trainees together with a study of the standards of the employment setting. Nearly 250 interpersonal problems in six major categories provided the content for the test items. Approximately 177 mildly retarded work ­trainees in four states were interviewed in order to develop response options to these test items. Finally, 117 employers in 12 states rated the options to the test items. These employer ratings were used to choose effective (correct answers) and ineffective (incorrect answers or distractors) response options for TICE. Thus, it is safe to conclude that the content of TICE is an accurate representation of the domain of interpersonal skills for employment.

Construct Validity, Construct validity refers to the relationship of a test to a theoretical framework. In an initial study of the construct validity of TICE (Bullis, 1983), a theory of interpersonal competence was constructed from a problem-solving perspective. It was found that TICE correlated positively and significantly to measures of role-taking ability, a construct that is believed to be an important component of interpersonal competence (Affleck, 1975).

Additional research has been conducted that relates to the construct validity of TICE (Halpern & Foss 1985). in that research, four different curricular formats were developed on the content assessed by TICE. Instruction was then provided to approximately 120 mildly retarded secondary students, using the four different formats. Students were tested with TICE prior to training and again after a six-week instructional period. The results of that research were that statistically significant gains were made by the students trained in the content of TICE, as compared to a control group. Furthermore, students instructed in the problem-solving curriculum made the greatest gains. These results provide further evidence for the construct validity of TICE.

Criterion validity. The authors of TICE, based on the procedures used to develop its content, believe that the competency areas it measures are related to job success. Put another way, it is reasonable to assume that more successful workers will score higher on TICE than less successful workers. A major criterion of success on the job is productivity. Therefore, its relationship to TICE scores was examined in a study recently completed (Auty, Goodman, & Foss, in press). In that research, positive and significant relationships were found between productivity rates and scores on the Supervisor (r =,38; p < .05)

and Co-Worker (r = ,66; p < .001) tests of TICE. The study was conducted with 29 work trainees in a vocational training facility in Seattle, Washington.

The above-described research supports the criterion validity of TICE, as it suggests that people who are more interpersonally competent in social interactions are more productive on the job. Such a proposition is reasonable in that unsuccessful social interactions take time away from work production. This research then lends further credence to the belief that TICE is a valid measure of interpersonal competence for the work setting.

In summary, TICE was developed to provide education and rehabilitation professionals with an instrument to measure interpersonal competencies necessary for success in community employment. Extensive procedures were used to ensure both the reliability and validity of the instrument. The Interpersonal Skills Training for Employment curriculum (Foss & Vilhauer, 1985) was recently developed to provide the educator/trainer with methods and materials for teaching the skills measured by TICE. It is hoped that these assessment and curriculum materials will be useful in assisting mildly retarded and other handicapped persons to be successful in finding and maintaining employment in the community.