Return to Commission IX Clearinghouse Index
Assessing the Development of Competence
Albert B. Hood and Lorraine M. Jackson
In Chickering's theory of student development that of achieving competency plays an important role as it affects the readiness to take on responsibilities, the readiness to take risks with one's self esteem, the professional and occupational alternatives that are considered as well as its influence on other vectors of development. Chickering believes that there are three types of competency important for undergraduate students. These are (l) intellectual competence, (2) physical and manual skills, and (3) interpersonal competence. A pool of items dealing with these competencies were developed, however, in attempting to operationlize the second competency area--physical and manual skills, problems were encountered. There was the problem of how to measure physical and manual growth in college students. Certainly much of this development exists before students enter college and for a large number of students, continued growth in this area does not take place in college. What physical ability may or may not be present, what desire to grow in this area and what opportunities there are for this growth may or may not be present for many students. There is also the problem of accounting for students with various physical handicaps both in the theory and in the assessment of these skills. For this reason assessment of that portion of the achieving competency vector was not attempted in the instrument which was developed.
In examining the intellectual competencies to be assessed, two of the most important appeared to be verbal and quantitative skills. Therefore two scales were constructed measuring writing scales--one dealing with perceived skill in writing and the second enjoyment of writing. Two measuring quantitative skills were also constructed--one perceived mathematical skills and the other enjoyment of mathematical concepts and problems. In attempting to assess interpersonal competence, five different scales were developed (l) perceived confidence in relating to supervisors, (2) perceived confidence in relating to peers, (3) postive regard for others, (4) perceived competence in understanding non-verbal communication, and (5) perceived competence in conducting smooth communications.
For each of these nine scales a large number of items were written and after a critical review of them, 20 were selected in such a way as to attempt to include clear and relevant items and to diminish the redundancy within each scale. Items were responded to on a five point scale:
l = Never characteristic of me
2 = Seldom characteristic of me
3 = Sometimes characteristic of me
4 = Often characteristic of me
5 = Always characteristic of me
For each of these experimental scales ten items were selected to be scored in the "positive" direction--that is--an answer "always characteristic of me" would give a person a higher score on a particular competency scale. Ten items were selected to be scored "negatively" in which a response "never characteristic of me" would add to the score on a particular competency. These equal numbers of positively and negatively scored items were included to control for one aspect of response set variance--that of acquiescence. Studies by Jackson and Messick (l96l), Edwards (l963), and Wiggins (l962) have demonstrated that stylistic consistencies of acquiescence and desirability account for much scale variance at the expense of validity and efficiency in assessment. Acquiescence refers to the tendency for persons to endorse many characteristics that are self-descriptive. Another aspect of acquiescence is the tendency to agree. By including equal numbers of positively and negatively scored items acquiescence is suppressed.
Social desirability was accounted for by including a 20 item Social Desirability Scale composed of items which Jackson (l974) has found to measure the tendency to endorse items which are cast in a desirable direction. Scores which students obtained on this Social Desirability subscale were then examined in relationship to each individual item in choosing items to be included in the final version of the inventory.
The resulting item pool thus contained 200 items--20 on each of the nine subscales designed to measure status on the achieving competency vector and 20 of Jackson's items yielding a measure of the tendency to respond items in a socially desirable direction. The 200 item inventory was administered to 102 students at The University of Iowa and 158 students at The Pennsylvania State University. These students represented heterogeneous groups on both campuses containing both undergraduates of all years and groups of graduate students. Although students from most fields were represented, at the both the graduate and undergraduate levels there was some loading of students in Education and also more women than men in the samples.
In the Penn State sample approximately 60% of the students were females and 40% males; in the University of Iowa sample 80% were females and 20% males. Approximately one-half of the students in the Penn State sample were freshmen and the remaining half was fairly evenly divided among the other four classes-- sophomores, juniors, seniors, and graduate students. The Iowa sample also included students from all five classes with somewhat larger numbers coming from the sophomore and the graduate student groups.
Reliability (internal consistency alpha) coefficients for both the total competency scale and the nine different sub-scales indicated substantial reliability for most of the subscales (Table A-l). Correlations between the two scales dealing with writing--both skill and enjoyment of writing--were highly intercorrelated suggesting that the two scales were measuring substantially the same factor. The same was true with the two scales dealing with mathematics. Three scales designed to assess interpersonal competence were also highly intercorrelated--Self- Confidence with Superiors, Confidence with Peers, and Competence in Smooth Communications. Correlations with scales representing other factors were much lower. The intercorrelations of the various scales are also shown in Table A-l. These results suggested that three separate factors dealing with competence were being assessed thus the decision was made to combine items on seven sub-scales to produce three separate sub-scales and a total score representing self-perceptions of competence in these three areas. Positive Regard for Others and Understanding of Non-Verbal Communication scales were neither highly reliable nor highly related to other scales and these scales were therefore not included in the final inventory.
Item correlations were computed for each item with, (l) its own subscale, (2) the total competency scale, (3) correlation with the 20-item social desirability scale score, (4) correlation with the student age, and (5) correlation with student class (freshmen through graduate or professional school).
To minimize the influence of tendency of students to answer items in the favorable direction, an index to assess the extent to which each item could be influenced by this tendency was computed. This Differential Reliability Index (DRI) is the result of computing the square root of the biserial correlation squared between an item and its own scale minus the biserial correlation squared between an item and the score on the desirability scale (Jackson, l974, p. l6). (Editors note - this formula will be re-typed soon into this document) This formula is as follows: DRI =
The final sub-scales were established by selecting 70 items from the l80 competency items in the original item pool. These items were selected by examining the correlation with the five factors listed above with particular emphasis on the social desirability sub-scale as assessed by the DRI. Some emphasis was placed on the correlations of the items with the total scale, and the relevant sub-scales. Correlations with age and class were required to be only in the positive direction and many of the items had correlations with these two factors which, although in the low positive direction, fell within the chance range.
The final Developing Competency Inventory is made up of three subscales. These subscales are entitled (l) Self Confidence, (2) Competency in Math, (3) Competency in Writing.
Self-Confidence Subscale
The Self-Confidence subscale is made up of 30 items. Approximately l0 of the items deal with self confidence in interacting with authority figures, l0 with peers (friends and classmates), and l0 deal with ease of communications with others. Two examples of each of the three types of items are as follows: "I can converse easily with people in positions of authority," "I find it difficult to ask for help from my teachers or professors (neg),"; "I am more self confident than most of the classmates," "I find it especially difficult to talk with students of the opposite sex" (neg); "I can readily introduce people at club functions," "I am able to disagree gracefully with my teachers or professors." The Self Confidence subscale has a reliability (alpha) coefficient of .92 (Table A-2). The relationship of the Self Confidence subscale with the total Developing Competency scale is .78. Its correlation with the Social Desirability subscale is .5l. The mean score obtained by the Iowa sample was 99.l6 with a standard deviation of l6.9l (Table A-3).
Competency in Math Subscale
The second subscale on the Developing Competency Inventory is composed of 20 items which deal with either self confidence in mathematics or the enjoyment of mathematics. Typical items include: "When working with fractions, I sometimes get confused" (neg), "I could tutor other students in mathematics"; "I enjoy working with numbers," "I like doing mathematical assignments."
The Competency in Math subscale has an alpha reliability of .96. It has a correlation of .66 with the total Developing Competency score and a correlation of .23 with the Social Desirability scale. The Iowa mean on this subscale was 53.24 and the standard deviation l8.34 (Table A-3).
Competency in Writing Subscale
The Competency in Writing subscale is also composed of 20 items. The items deal either with self confidence regarding writing or the enjoyment of writing. Sample items include: "I am able to write about most topics assigned to me," "I have little difficulty in revising my written papers," "I think I would like being an author," "I try to avoid classes that require a lot of writing" (neg). The Competency in Writing subscale has an alpha reliability coefficient of .92. It has a correlation with the total Developing Competency score of .60 and a correlation of .2l with the Social Desirability scale. The mean on this subscale was 64.75 and the standard deviation of l3.0l (Table A-3).
Total Developing Competency Inventory
The Developing Competency Inventory then is made up of 70 items. The alpha reliability of the total scale is .94. The Iowa mean was 2l7.l8 with a standard deviation of 32.90. The correlation with the Social Desirability subscale was .5l. Among the Iowa sample, the total scores were related to sex with a correlation of .25, indicating that as a group, males report greater self-confidence than females (Table A-4). The total score was positively related to age (.l8) and significantly related to student class (.3l). Among the sub-scales, the Self-Confidence sub-scale was slightly negatively related to sex (-.l3) and a stronger relationship to sex was found for the Competency in Mathematics (.26). The Competency in Writing sub-scale was unrelated to sex. The sub-scales showed slight but usually non-significant relationships to age and class (in the general vicinity of .l5). The exception to this was the Competency in Writing sub-scale which showed no relationship to age but a stroner relationship (.27) to class. As might be expected, seniors and graduate students appear more confident in regard to writing than do freshmen and sophomores. The correlation matrix of the sub-scale scores and the total score on the Developing Competency Inventory is shown in Table A-2.
Approximately half of the items on the scale are scored in a "positive" direction--with the higher score on the scale obtained from the response "almost always characteristic of me" and half in the negative" direction with the highest score from the response "never characteristic of me." Items have been placed on the scale to follow a pattern of scoring to make it easier if the inventory is to be hand scored. The first two items are scored in a "positive" direction, the next two in a "negative" direction, the next in a "positive" direction, and the next two in a "negative", and this pattern continues throughout the first 66 items on the inventory with the pattern breaking down only among the last four items (67-70). Similarly, a pattern among the scales on which the individual items fall has also been established for the inventory. This pattern is as follows: the first item is self-confidence, the second mathematics, the third self-confidence, the fourth writing, the fifth self-confidence, the sixth mathematics, and the seventh writing. That seven- sequence pattern of scales--self-confidence, math, self- confidence, writing, self-confidence, math, writing--continues l0 times throughout the entire seventy item inventory.
**************************************************
Table A-1
Correlation Matrix and Reliabilities of Pilot Competency Scale Scores
Self Pos. Conf. Enjoy Self Non- Enjoy Conf. smooth Social
Conf. Reg. Writ. Math Conf. verb. Writ. Math Comm. Desir.
Sup. Peers Comm.
1 2 3 4 5 6 7 8 9 10
Total .76 .32 .66 .52 .75 .34 .44 .61 .75 .58
1 .16 .49 .11 .79 .26 .30 .18 .83 .51
2 .17 -.03 .20 .36 .14 -.02 .26 .59
3 .01 .46 .10 .70 .14 .41 .30
4 .11 .03 -.17 .92 .11 .19
5 .26 .25 .19 .82 .52
6 -.05 .08 .32 .35
7 - .11 .27 .09
8 .18 .23
9 .55
Rel. .90 .75 .91 .94 .88 .56 .91 .95 .85 .77
N = 102
**************************************************
Table A-2
Correlation Matirx and Reliabilities of
Final Competency Inventory Scale Scores
Total Inter. Pers. Comp. Comp.
Score Self. Con. in in
Math Writing
Total Score .78 .66 .60
Interpersonal
Self Confidence .18 .41
Competency in
Math .62
Competence in
Writing
Reliability
(alpha) .94 .92 .96 .91
N = 95
**************************************************
Table A-3
Means and Standard Deviations of Final
Competency Inventory Scale Scores
Scale Mean Standard Deviation N
Total Score 217.18 32.90 91
Interpersonal
Self Confidence 99.16 16.91 95
Competency
in Math 53.24 18.34 95
Competency
in Writing 64.75 13.01 97
**************************************************
Table A-4
Correlations of Final Competency Inventory
Scale Scores with Demographics
Sex Age Class Social
(male = 1) Desirability
(female = 2)
Total Score -.25 .18 .31 .51
Interpersonal
Self Confidence -.13 .18 .17 .55
Competency
in Math -.26 .14 .15 .23
Competency
in Writing -.06 .07 .27 .21
N = 84
**************************************************
ReferencesChickering, A. W. (l972). Education and identity. San Francisco: Jossey-Bass.
Edwards, A. L. (l962). A factor analysis of experimental social desirability and response set scales. Journal of Applied Psychology, 47, 308-3l6.
Jackson, D. N. (l974). Personality research form manual. Goshen, NY: Research Psychologists Press.
Jackson, D. N., & Messick, S. J. (l96l). Acquiescence and desirability as response determinants on the MMPI. Educational and Psychological Measurement, 2l, 77l-792.
Wiggins, J. S. (l962). Strategy, method, and stylistic variance in the MMPI. Psychological Bulletin, 59, 224-242.