Culturally appropriate assessment considerations

Ah: culture. This is the “customary beliefs, social forms, and material traits of a racial, religious, or social group” (Merriam-Webster Dictionary, 2021). Or "The way we do things around here" (Deal & Kennedy, 1982, p. 4). It is the trickiest thing to get right: and even more tricky when matching tests to client culture in career practice. As Betts said, “From a psychological perspective, culture is the milieu that provides a context for all of a person’s behaviour, thought, and other mediating variables” (2013, p. 98). Culture is all-encompassing and impacts on every aspect of our lives.

Understanding that none of us comes into a practice relationship free of our own cultural background and identity helps us to focus on understanding our own viewpoint, and to realise that understanding the viewpoint of the client is of equal importance. While reflection is a key tool for us learning our own cultural turangawaewae, we have to work harder to unlock the culture of our clients.

Career tools and instruments usually assess a client’s ability, aptitude, interests, personality, strengths, weaknesses, values & job preferences, and career decision making. These types of categorisation tests are largely normalised and standardised quantitative tools. They are used in different combinations with different clients, sometimes alongside non-standardised, qualitative assessments (Osborn & Zunker, 2016).

It’s important to always keep in mind the purpose of using career assessments and how they relate to the needs of the client, as the “underlying purpose of these assessments is to help clients gain a better understanding of who they are and to apply that knowledge to their career concerns and career decision-making process” (Osborn, 2012, p. 7).

Career professionals are expected to operate according to the ethical requirements of our professional body. Professional bodies have codes of ethics (COE) which guide our work. The CDANZ COE states that members will "Represent services, qualifications, and experience accurately and fully, and only undertake those practices for which they are qualified and in circumstances where they have appropriate experience" (2016). Further clarification can be gained from the US, where their national body states that career practitioners will “carefully consider the validity, reliability, psychometric limitations, and appropriateness of instruments when selecting assessments and, when possible, use multiple forms of assessment, data, and/or instruments in forming conclusions, diagnoses, or recommendations” and that “career professionals [must] use caution when selecting assessments for culturally diverse populations to avoid the use of instruments that lack appropriate psychometric properties for the client population” (NCDA, 2015, p. 14).

The psychometric properties that the NCDA refer to are validity, reliability and normalisation. Validity relates to whether or not the test is measuring what it claims to be measuring, especially when being used in a culture different from the one in which it was developed. Reliability relates to the consistency of the results of a test and the freedom from error of the resulting data. Normalisation refers to having norm data for different population groups for comparative purposes (Osborn & Zunker, 2016).

When working in a cross-cultural context it is essential to first determine whether any assessment under consideration can meaningfully be used with a specific cultural group: whether there is norm data, whether the test is valid and reliable for use with our client's cultural group. While it has been argued that no test is free of presuppositions about values, knowledge and communication, in considering test applicability, the idea of equivalency becomes important. Similar to reliability and validity, equivalency measures a test for generalisability between cultures (Greenfield, 1997). There are four equivalency questions (Betts, 2013; Marsella & Leong, 1995) for us to seek answers to so we can determine if an assessment is culturally appropriate:

1. Linguistic. Do the words used in an assessment tool have the same meaning either when translated or when used in a different cultural context?
2. Conceptual/construct. Does the test measure the same trait or construct in different cultures?
3. Functional. Do the behaviours being evaluated have the same meaning or purpose?
4. Metric. How are the variables quantified?

Lastly, we also need to be aware of context and experience. “Understanding a client’s [ethnic] and cultural experiences, as well as expectations from significant others and family, will help the career practitioner better understand the contextual issues impacting a career decision, as well as draw upon strengths within that client’s environment to help the client achieve their goals” (Osborn & Zunker, 2016, p. 21).

Like so many things, the more we look, the more complexity there is.

Kirstie

References:

Betts, D. (2013). A review of the principles for culturally appropriate art therapy assessment tools. Art Therapy, 30(3), 98-106. https://doi.org/10.1080/07421656.2013.819280

CDANZ (2016). Code of Ethics. Career Development Association of New Zealand. https://cdanz.org.nz/ModularPage?Action=View&ModularPage_id=26

Deal, T. E. & Kennedy, A. A. (1982). Corporate Cultures: The Rites and Rituals of Corporate Life. Penguin Books.

Greenfield, P. M. (1997). You can't take it with you: Why ability assessments don't cross cultures. American Psychologist, 52(10), 1115. https://doi.org/10.1037/0003-066X.52.10.1115

Marsella, A. J., & Leong, F. T. (1995). Cross-cultural issues in personality and career assessment. Journal of Career Assessment, 3(2), 202-218. https://doi.org/10.1177/106907279500300207

Merriam-Webster Dictionary (2021). Definition of culture. https://www.merriam-webster.com/dictionary/culture

NCDA (2015). Code of Ethics. National Career Development Association. https://www.ncda.org/aws/NCDA/asset_manager/get_file/3395

Osborn, D. S. (2012). An international discussion about cross-cultural career assessment. International Journal for Educational and Vocational Guidance, 12(1), 5-16.https://doi.org/10.1007/s10775-012-9220-0

Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

* Kirstie Smith has kindly prepared much of the materials used in this blog post