Validity and reliability of interactive Rating Scale (IRSA) as an indicator of social ability development
- (0) Download
https://www.eduzhai.net Public Health Research 2014, 4(1): 25-30 DOI: 10.5923/j.phr.20140401.05 Validity and Reliability of the Interaction Rating Scale Advanced (IRSA) as an Index of Social Competence Development Tokie Anme1,*, Kentaro Tokutake1, Emiko Tanaka1,2, Taejko Watanabe3, Etsuko Tomisaki1, Yukiko Mochizuki1, Bailiang Wu1, Ryoji Shinohara4, Yuka Sugisawa5, Shuntaro Okazaki6, Norihiro Sadato6 1Faculty of Medicine, University of Tsukuba, Ibaraki, Japan 2Research Fellow, Japan Society for the Promotion of Science, Tokyo, Japan 3Ashikaga University, Tochigi Japan 4University of Yamanashi, Yamanashi Japan 5Ushiku Health Centre, Ibaraki, Japan 6National Institute for Physiological Sciences, Aichi, Japan Abstract The purpose of the current study was to clarify the validity and reliability of the Interaction Rating Scale Advanced (IRSA) as a context-based practical index of social competence development. Participants were 50 adults who completed a five-minute interaction session, during which they were observed and assessed using the IRSA. Furthermore, health social professionals evaluated participants’ social competence using practical assessment. Results indicated a moderately high correlation (r = 0.58) between IRSA scores and professionals’ practical evaluation. The Cronbach’s alpha value was 0.89. Thus, the IRSA appears to measure social competence with high validity and reliability. Since the IRSA provides context-based evidence of social competence development, this measure should be useful for evaluating the various features of social interaction in a practical setting. Keywords Social Competence, Interaction, Evaluation, Scale, Adult 1. Introduction Social competence has been defined as a dimension of social intelligence[1, 2]. Social intelligence is the ability to engage in adaptive and positive social interactions, which is theoretically distinct from general academic intelligence. However, it is often difficult to discriminate social intelligence from general intelligence. Historically, social intelligence has somewhat overlapped with social competence, which is determined by the complex interaction between the individual, his/her home and school environments, peer relationships, and the larger sociocultural environment. Social competence is broadly defined as the ability to understand others in the context of a social interaction and engage in smooth communication. Thus, social competence should be evaluated by the interaction between the individual and his/her social environment. Dealing with social stress among adults who display impulsive and maladjusted behavior requires society to * Corresponding author: email@example.com (Tokie Anme) Published online at https://www.eduzhai.net Copyright © 2014 Scientific & Academic Publishing. All Rights Reserved prepare appropriate support systems and environments for those individuals. Researchers have been engaged in the study of social competence development for decades. However, methodologies that consider individuals in conjunction with their social environment across developmental stages are not yet well developed. We developed four social competence scales for different stages of lifespan development: (1) the Interaction Rating Scale (IRS), which is an observation method for child– caregiver interactions that assesses children under 8 years old[5,6]; (2) the Interaction Rating Scale between Children (IRSC), which is an observation method for child–child interactions that assesses children 3 to 18 years old; (3) the Interaction Rating Scale Advanced (IRSA), which is an observation method for adult–adult interactions that assess individuals over 15 years old; and (4) the Social Skill Scale (SSS), which uses an enumerator method to assess children under 7 years old. These scales, based on accumulated knowledge from the developmental sciences, have focused on measuring the quality of an environment where positive interactions with the environment are significantly related to healthy development. Two instruments assessing the home environment, namely the Home Observation for Measurement of the Environment (HOME) and the Index of Child Care Environment 26 Tokie Anme et al.: Validity and Reliability of the Interaction Rating Scale Advanced (IRSA) as an Index of Social Competence Development (ICCE) are often used in research related to the right to withdraw from the study at any time. To maintain context-based child social competence development. confidentiality, a personal ID system was used to protect The HOME and ICCE evaluate the environment within personal information. Furthermore, all video picture data natural settings, including one’s emotional and verbal were stored on a disk, which was password protected. Only responsiveness to another individual as well as one’s the researchers with necessary permission were given access acceptance of another’s behavior. The HOME suggests that to this data. The ethics committee of the National Institute context-based social competence is essential for lifespan for Physiological Sciences approved this study. development because it reflects one’s adaptability and convergence in the real world. The HOME is a very popular 2.2. Measures measure, having been used in more than 100 countries. The IRSA is a 92-item instrument designed as a brief but Several observation-based social competence measures comprehensive observation measure that assesses basic have been developed that focus on specific areas of social social competence for individuals over the age of 15. Social interactions; these measures include the Mediated Learning competence is examined through five-minute observations Experience Rating Scale (MLERS), which deals with of a social interaction. One advantage of the IRSA is that teacher–student interactions; the Social Skills Rating evaluations of interactions can be completed in a short System (SSRS), which is used to examine caregiver–child period within normal, daily situations. interactions; and the Autism Diagnostic Observation The IRSA includes a behavioral score and 6 subscales Schedule-Generic (ADOS-G) and Childhood Autism scores that combined provide an impression score: Rating-Scale (CARS), which are used to assess social “self-control,” “expressivity,” “sensitivity,” “assertiveness,” competence individuals suspected of having an autism “responsiveness,” and “regulation” (Appendix 1). The 92 spectrum disorder. items were extracted from several sources, including original Conversely, questionnaire-based social competence items from the study authors and items from the IRS, measures have focused on more generalized settings and SSRS, and the ENDCOREs. address multidirectional factors, as is the case with the Social Two different variables are scored: behavior and Skills Inventory (SSI) (containing six factors: impression. For the behavior variable, items are assessed in “emotional expressivity,” “emotional sensitivity,” terms of the presence of a behavior (0 = no, 1 = yes), and the “emotional control,” “social expressivity,” “social sum of all items in provides the overall score. The total score sensitivity,” and “social control”), the ENDCOREs (six can range from 0 to 92. As for the impression, each observed factors: “self-control,” “expressivity,” “sensitivity,” behavior is rated on a five-point scale where 1 is “not evident “assertiveness,” “responsiveness,” and “regulation”), the at all,” 2 is “not clearly evident,” 3 is “neutral,” 4 is “evident,” Adult Behavior Checklist for Ages 18-59 (ASEBA)[18,19] and 5 is “highly evident.” (six factors: “adaptive functioning,” “empirically based The rating procedure is as follows: the evaluator syndromes,” “substance use,” “internalizing,” completes the checklist, focusing on the participant’s “externalizing,” and “total problems”), and the Weinberger behaviors (e.g., expressing his/her own feelings to the Adjustment Inventory (WAI) (six factors: “distress,” partner). The health-social professionals provides separately “anxiety,” “depression,” “low self-esteem,” “low well-being,” an impression score for each observed behavior. “self-restraint,” “suppression of aggression,” “impulse The inter-observer reliability was 90%. control,” “responsibility,” and “consideration of others”). According to the above-mentioned measures, the current 2.3. Procedure study used the most common factors for assessing social The IRSA was evaluated during a five-minute video competence developing: “empathy/coordination,” “self-reg recording of an interaction (two participants play a game ulation,” and “assertion.” Our factors are referred to as called, “Keep it steady!” which consists of a wooden ring “sensitivity/responsiveness,” “self-control/regulation,” and and 6 inch long 27 sticks with varying widths. Players grab “assertiveness/expressivity.” all the sticks together, slide the wooden ring around the The purpose of the current study was to clarify the validity center of the bundle, give it a twist, and stand the sticks up on and reliability of the IRSA as a context-based practical index end. The game begins by pulling out a piece and taking turns of social competence. until the structure collapses). The recording was carried out in a room with four video cameras, which assessed the 2. Methods interaction from four angles (Figure 1). The participant dyads were escorted into a room furnished with a small table 2.1. Participants and two chairs. The instructor introduced the game to both participants. Participants were 50 adults, 25 males and 25 females, aged To score behaviors, two evaluators coded all participant’s 18 to 48 years old. In order to comply with ethical standards behaviors. Behaviors during the interaction were coded as before conducting the research, all participants signed follows. If the participant displayed the behavior described informed consent forms and were made aware that they had in the item, a score of 1 was given; conversely, if the Public Health Research 2014, 4(1): 25-30 27 participant failed to display the behavior described in the item, a score of 0 was given. The total score was the summed scores of all 6 subscales. A higher score indicated a higher level of social competence. Two health social professionals then evaluated social competence based on observations of the practical assessment using the IRSA impression items. The means of the six subscale impression scores were calculated. Figure 1. Video recording from four directions 3. Results The correlation analysis was used to validate the use of this measure to assess social competence (i.e., how well the IRSA assesses behaviors in conjunction with professional reports). Figure 2. Correlation between IRSA scores and professional evaluations Figure 2 shows the correlation between the total score on the IRSA and professionals’ practical evaluations of the interactions. Results suggest a moderately high correlation (r = 0.58, p < 0.001) between IRSA scores and professionals’ practical evaluations. No significant gender or age differences were found on the IRSA subscales. The internal consistency of the IRSA, as measured by Cronbach’s alpha, was 0.89. 4. Discussion This current study provides a measure (IRSA) that can assess social competence with moderately high validity and reliability. The social competence scale for child–caregiver interactions (IRS)[5,6] and child–child interactions scale (IRSC) have already shown adequate reliability, validity, feasibility, and practical utility for examining social interactions over time. It is meaningful that the IRSA can be used to assess social competence continuously along a lifespan developmental continuum. Several studies have indicated that observational outcomes differ from self-administrated outcomes because self-optimism produces biases that provide favorable evaluations for the self[21,22]. Observation-based scale is essential to understand the social competence development actually used in one’s life. Social skills play a role in how well an individual copes with stress. Thus, possessing adaptive social skills should be just as important to social adjustment and well-being as is psychological health. The IRSA may prove useful in research on psychological stress and coping by having the ability to examine individuals' abilities to cope with stress . Additionally, the IRSA could be used as part of communication skills training programs for health care professionals. The strength of the IRSA is the ability to assess objective social competence throughout the lifespan. Additionally, the IRSA is easy to use in practice because it is highly adaptable to various age groups. Furthermore, the framework of the IRSA is based on the most common measurement paradigms used around the world; this makes it easier to use this measure within international comparative studies. Additionally, the 6 subscales are based on several categories that are widely used in research on social competence indicators. Even though the IRSA has some valuable strengths, it is also important to acknowledge the IRSA’s limitations. First, the IRSA was assessed with only 50 participants in the current study. Thus, the generalizability of the present results should be taken with caution. Second, the IRSA subscales might not cover all the dimensions of social competence, even though we used the most common components of social competence addressed in previous studies. Despite these limitations, the IRSA appears to be a consistent, valid screening instrument that reflects attributes of social interactions. We have already started to employ the IRSA for assessing individuals with autism in order to characterize behavioral features and the evaluation of treatment outcomes. Further research should enhance the potential to reveal additional features of social interaction development that will be of use to both practitioners and researchers. 28 Tokie Anme et al.: Validity and Reliability of the Interaction Rating Scale Advanced (IRSA) as an Index of Social Competence Development 5. Conclusions ACKNOWLEDGEMENTS This study indicated a moderately high correlation (r = 0.58) between IRSA scores and professionals’ practical evaluation. The Cronbach’s alpha value was 0.89. Thus, the IRSA appears to measure social competence with high validity and reliability. Since the IRSA provides context-based evidence of social competence development, it should be useful for evaluating the various features of social interaction in a practical setting, continuously along a lifespan developmental continuum. This research was supported by a Grant-in-Aid for Scientific Research to N.S. (S21220005) from the Japan Society for the Promotion of Science. Part of this study is the result of a project called the "Development of biomarker candidates for social behavior," carried out under the Strategic Research Program for Brain Sciences by the Ministry of Education, Culture, Sports, Science and Technology (MEXT) and a Grant-in-Aid for Scientific Research (23330174, 24653134). Appendix 1. All items on the Interaction Rating Scale Advanced 1. Expressivity: Expresses his/her thoughts and feelings precisely 1) Vocalizes. 2) Expresses his/her own feelings to the partner. 3) Attempts to elicit help or consolation from the partner. 4) Shows self-assertiveness to the partner through a gesture. 5) Casts the partner a glance to seek sympathy. 6) Shows the change of his/her feelings through facial expressions 7) Smiles or laughs. 8) Attempts to make eye contact with the partner 9) Attempts to elicit a response from the partner. 10) Looks at the partner's face to get information/clarification. 11) Shows his/her feelings by words and actions together. 2. Assertiveness: States his/her opinion or position clearly to others. 12) Speaks up to the partner about what he/she thinks. 13) There are words and actions that indicate his/her decision. 14) Talks to, suggests, or lets the partner accomplish something while he/she pays attention. 15) Expresses his/her opinion to the partner. 16) Verbalizes a differing opinion or position. 17) Exhibits a differing opinion by his/her expression and gestures. 18) Uses both verbal descriptions and non-verbal instruction. 19) Provides guidance through explanation but not through order. 20) Explains his/her opinion according to the level of competence/ability of the partner. 21) Instructions and opinions are clear and unambiguous. 22) Explains his/her opinion logically. 23) Expresses his/her own idea after showing that he/she understands the partner's idea. 24) Expresses his/her ideas after indicating his/her understanding to the partner through expression and gesture. 25) Makes a decision after indicating that he/she understood the partner's idea/suggestion. 26) Makes a decision after showing through non-verbal expression that he/she understood the partner. 3. Sensitivity: Ability to read the partner's feelings and thoughts accurately. 27) Shows an appropriate reaction through a change in his/her expression and gestures. 28) Vocalizes or speaks in response to the partner's verbalization. 29) Vocalizes or adjusts own behavior in response to the partner's verbalization. 30) Looks at the partner or materials when he/she shows non-verbal behavior. 31) Vocalizes in response to the partner's behavior or nonverbal cues. 32) Vocalization, makes a facial expression, or moves in response to the partner's behavior or nonverbal cues. 33) Vocalizes after noticing changes in his/her partner’s facial expression. 34) Looks at his/her partner or materials after noticing the changes in the partner’s facial expression. 35) Vocalizes, expresses, or moves according to changes in partner's expression. 36) Smiles or frowns within five seconds after the partner's vocalization. 37) Looks at the partner's face or eyes when the partner attempts eye contact. 38) Behaves appropriately in response to the partner's gestures or changes in expression. 4. Acceptance: Understands and respects the partner's opinion or position Public Health Research 2014, 4(1): 25-30 29 39) Smiles in response to the partner's smile. 40) Praises the partner's efforts, success, and behavior. 41) Smiles, claps hands, or shows he/she is glad when the partner is feeling happy. 42) Shows empathy by verbal or non-verbal responses when the partner is in a bad mood. 43) Emits positive, sympathetic, or soothing verbalizations in response to the partner's feelings. 44) Responds to the partner's vocalizations with an affectionate verbal response. 45) Smiles at the partner's verbalization. 46) Nods in response to partner's verbalizations and/or actions 47) Emits a soothing non-verbal response (e.g., pat, touch, rock) at the partner's successes or failures. 48) Smiles and/or nods at the partner during the episode. 49) Does not vocalize or interrupt the partner while he/she is speaking. 50) Nods at the partner's comment. 51) Accepts the partner's opinion partially or totally by saying, "let's do it,” or by acting in a manner consistent with the partner's suggestion. 52) Accepts the partner's opinion even when his/her own opinion differs. 53) Pauses when the partner starts to verbalize. 54) Disturbs the partner. 55) Allows the partner to decide what he/she wants to do. 56) Praises the partner's skills during the assignment. 5. Regulation of the interpersonal relationship: Works with the partner to develop a good relationship. 57) Provides an environment free of distractions for the partner. 58) Does not make negative comments to the partner. 59) Does not behave negatively toward the partner. 60) Affirms the partner with nods or other gestures. 61) Laughs while they are looking at each other. 62) Laughs while they are looking at the same thing. 63) Moves in the same manner as the partner moves. 64) Does not turn away from the assignment and pays close attention to the partner. 65) Verbally praises the partner during the assignment. 66) Praises the partner with applause. 67) Talks to the partner positively or encouragingly during the assignment. 68) Says "Thank you" to the partner when he/she grants a concession. 69) Does not criticize the partner when they have differing opinions. 70) Tries to talk with the partner logically when they have differing opinions. 71) Tries to avoid emotional conflicts with the partner. 72) Tries to respond calmly when the partner becomes angry or agitated. 6. Self-control: Ability to control personal emotions and behaviors. 73) Waits for the partner's reaction or action for at least five seconds. 74) Emits appropriate movement of eyes. 75) Emits appropriate phonation. 76) Emits appropriate utterances. 77) Emits appropriate movements. 78) Makes clearly recognizable hand motions towards materials during the assignment. 79) Concentrates on the task and is gentle with the materials. 80) Does not interrupt the partner's activity 81) Is not destructive/rough with the materials. 82) Not tense. 83) Does not shout or raise his/her voice. 84) Does not display distress cues even when the task does not go well. 85) Is not rude to the partner. 86) Avoids displeasing the partner. 87) Does not speak negatively of others. 88) Does not curse at people or at things. 89) Follows the rules of the game. 90) Touches a task together. 91) Emits appropriate emotional expression. 92) Praises the partner when he/she succeeds or when the partner fails, he/she commiserates. 30 Tokie Anme et al.: Validity and Reliability of the Interaction Rating Scale Advanced (IRSA) as an Index of Social Competence Development REFERENCES  N. Cantor and J.F. Kihlstrom, Personality and social intelligence, Englewood Cliffs, NJ: Prentice-Hall, 1998.  H. Gardner. Frames of mind: The theory of multiple intelligences (10th Anniversary Edition). New York: Basic Books, 1993.  C. S. Lidz. Practitioner’s guide to dynamic assessment, New York: Guilford Press, 1991.  R. Feuersteing, Y. Rand, and M. B. Hoffman. Dynamic assessment of retarded performers: The Learning Potential Assessment Device: Theory, instruments and techniques, Baltimore: University Park Press, 1979.  Anme, T., 2010, Trajectories of social competence by using Interaction Rating Scale (IRS) as an evidence-based practical index of children’s social skills and parenting, Journal of Epidemiology, 20, 419-426.  Anme, T., 2010, Gender differences of children’s social skills and parenting using Interaction Rating Scale (IRS), Procedia Social and Behavioural Sciences, 2, 260-268.  Anme, T. et al., 2012, Validity and reliability of the Interaction Rating Scale between Children (IRSC) by using motion capture analysis of head movement, Public Health Research, 2(6).  Anme, T. et al., 2011, A pilot study of social competence assessment using International Rating Scale Advanced, IRSN Pediatrics, 2011, 1-6.  Anme, T. et al., 2013, Validity and reliability of the Social Skill Scale (SSS) as an index of social competence for preschool children, Journal of Health Science, 3(1).  B. M. Caldwell, and R. H. Bradley, Home observation for measurement of the environment, Little Rock, AR: Center for Research on Teaching and Learning, University of Arkansas, 1974.  Anme, T. et al., 2013, Validity and reliability of the Index of Child Care Environment (ICCE), Public Health Frontier, 2(6).  Burchnal, M. R., Campbell, F. A., Bryant, D. M., Warsik, B. H., and Ramey, C. T., 1997, Early intervention and mediating process in cognitive performance of children of low-income African American families, Child Development, 68(5), 935-954.  F. M. Gresham, and S. N., Elliot, Social skills rating system – Secondary, Circle Pines, MN: American Guidance Service, 1990.  Lord, C., et al., 2000, The Autism Diagnostic Observation Scale-Generic: A standard measure of social and communication deficits associated with the spectrum of autism, Journal of Autism and Developmental Disorders, 30(3), 205-223.  Schopler, E., Reichler, R. J., Devellis, R. F., and Daly, K., 1980, Toward objective classification of childhood autism – Childhood Autism Rating-Scale (CARS), Journal of Autism and Developmental Disorders, 10, 91-103.  Riggio, E. R., 1986, Social skills inventory manual 2nd ed. Assessment of basic social skills, Journal of Personality and Social Pychology, 51(3), 649-660.  Fujimoto, M., and Daibo, K., 2007, A hierarchical structure theory of communication skills, The Japanese Journal of Personality, 15, 347-361.  Rescorla, L. A., 2005, Assessment of young children using the Achenbach system of empirically based assessment (ASEBA), Mental Retardation and Development Disorder Research, 11, 226-237.  Achenbach, T. M., and Becker, A., 2005, Multicultural assessment of child and adolescent psychopathology with ASEBA and SDQ instruments: Research findings, applications, and future directions, Journal of Child Psychology and Psychiatry, 49(3), 251-275.  Weinberger, D. A., 1990, Distress and restraint as superordinate dimensions of self-reported adjustment: A typological perspective, Journal of Personality, 58(2), 381-417.  Abele, A. E., and Guido, H. E., 2007, Individual differences in optimism predict the recall of personality relevant information, Personality and Individual Differences, 43(5), 1125-1135.  Flyvberg, B., 2006, From Nobel Prize to project management, Project Management Institute Journal, 37(3), 5-15.  S. Cohen, and S. L., Syme, Issues in the Study and Application of Social Support, In Social Support and Health, pp. 3-22, S. Cohen and S. L. Syme, Eds. New York: Academic Press.  Riggio, R. E., Riggio, H. R., Salinas, C., and Cole, E. J., 2004, The role of social and emotional communication skills in leader emergence and effectiveness, Group Dynamics, 7, 83-103.  Harrigan, J. A., Oxman, T. F., and Rosenthal, R., 1985, Rapport expressed through nonverbal behavior, Journal of Nonverbal Behavior, 9, 95-110.
... pages left unread,continue reading
Free reading is over, click to pay to read the rest ... pages
0 dollars，0 people have bought.
Reading is over. You can download the document and read it offline
0people have downloaded it