Making sense of testing

We use career assessments in order to help our clients in identifying their unique characteristics. Each assessment is designed to measure different components, thus - with appropriate interpretation - assisting our clients to find career options which match their particular attributes, values, and skills (Osborn & Zunker, 2016).

While tests can assist client's decision making processes (Whitfield et al., 2009), to be effective, those tests need to be reliable and valid (Walsh & Betz, 2000). If a test is valid, it means that it actually measures what it says it measures: it does what it says on the tin (Heale & Twycross, 2015). There are three key types of validity: content validity (test accuracy); construct validity (does what it says on the tin - e.g. testing for job search skills might inadvertently be evaluating problem-solving skills); and criterion-related validity (where the same factor - or variable - is measured each time, through 'convergent' validity which is strongly correlated with similar tests; 'divergent' validity with poor correlation to different tests; and 'predictive' validity where the test is highly correlated to related factors - e.g. being task-oriented should lead to being a completer/finisher) (Heale & Twycross, 2015). 

Tests also need to have been normalised for the population group our client affiliates (awhis) to. That means that, when assessments are created, researchers have run a number of sample tests (usually around 300; Steve Evans, personal communication, 13 September 2021) on each population group, seeking normal distribution in the test results via cultural, ethnic, gender, political and socio-economic group factors (Hansen, 2003; Osborn & Zunker, 2016). We can see that normalising tests is going to be an expensive business, in giving 300 tests to measure each norm group.

We also need to have consistent test-retest rates: the same result needs to be achieved each time the test is run (Heale & Twycross, 2015). If our client does a test in March, we don't want to see that they obtain a completely different result when they repeat the test in July (one of the main bug-bears of MBTI; Mastrangelo, 2001). While it’s not possible to perfectly assess each career instrument, we can estimate their replicability (Heale & Twycross, 2015) through “internal [...] and test-retest reliability” (Osborn & Zunker, 2016, p. 37).

And, while we might have all reliability, validity and representative norm groups, we might still find that our client does not suit the test we propose. The client may complete the test and end up with results which make no sense. For example, each time I complete a RIASEC test, I get a different score. Over the years, I think I have seen a pattern: that in those of us with very generalist skills, the RIASEC test may lose it's test-retest reliability. I offer RIASEC here as one example: it is not the only one I have noticed. I have had clients who achieve poor results from HBDI, from MBTI, and from DiSC. All tests do not necessarily suit all people.

We must take all quantitative tests with a pinch of salt :-)

Sam

References:

Hansen, S. S. (2003). Career counselors as advocates and change agents for equality. The Career Development Quarterly, 52(1), 43-53. https://doi.org/10.1002/j.2161-0045.2003.tb00626.x

Heale, R., & Twycross, A. (2015). Validity and reliability in quantitative studies. Evidence Based Nursing, 18(3), 66-67. https://doi.org/10.1136/eb-2015-102129

Herr, E. A. (2001). Chapter 2: Career Assessment: Perspectives on trends and issues. In J. T. Kapes, E. A. Whitfield (Eds.), A counselor's guide to career assessment instruments (4th ed., pp. 15-26). National Career Development Association.

Mastrangelo, P. M. (2001). [251] Myers-Briggs Type Indicator [Form M]. In B. S. Plake & J. C. Impara (Eds.), The fourteenth mental measurements yearbook (816-820). Buros Center for Testing.

Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

Walsh, W. B., & Betz, N. E. (2000). Tests and Assessment (4th ed.). Prentice Hall.

Whitfield, E. A., Feller, R. W., & Wood, C. (Eds.). (2009). A counselor’s guide to career assessment instruments (5th ed., pp. 13–25). National Career Development Association.

read more "Making sense of testing"

Using career assessments from other countries

It is quite a process to create, test and normalise career assessment instruments (Stuart, 2004), but living in the Antipodes, where we have such a small population - only 5m - it would also be a costly procedure. Pretty much the only quantitative tools we have in Aotearoa are tests which have been internationally-developed. So, if we career practitioners in New Zealand want to give our clients evidence-based assessments, we have to rely on those which have been developed elsewhere. But are those international assessments worth using, from a cultural appropriateness point of view, or should we avoid quantitative testing altogether?

Due to our geographic isolation, rural roots, and confluence of Pākehā, Māori and Pasifika ethnicities, New Zealand's multicultural society is unique. Māori and Pasifika cultures have tended to focus more on collective well-being, interdependence, and respect for the environment; as opposed to the Western individualism arising from the Pākehā settlers (Harmsworth, 2005; while noting that all three culture are moving closer together). Our social norms, leadership styles, and personal interactions of Aotearoa mean that we prize modesty, practicality, and resilience (Harmsworth, 2005). Due to Māori and Pasifika cultural influence, all New Zealanders may have more community-oriented career goals, on average, than other nations. In fact, the John Hopkins Institute collected and cross-tabulated UN volunteer data, which showed that New Zealand has the most volunteers by a third, even though our not-for-profit sector is smaller than some other nations (Belgium, Australia and Israel; GMVP, 2013). Volunteering in New Zealand appears more culturally endemic than in Australia; apparently 50% of Kiwis volunteer versus 5% of Aussies volunteer (SNZ, 2006; VNZ, 2024). 

Our differing values may mean that international test validity may not translate to test validity here in Aotearoa. But why should we use quantitative assessments anyway? Well, there are good reasons. It seems that clients who complete assessment instruments have a deeper understanding of their own interests, values and strengths (Heppner et al., 1994). In addition, clients tend to make more informed career decisions, and seem to experience less career indecision as a result of testing (Heppner et al., 1994). Even better, clients who took assessments as part of seeing a career practitioner experienced more positive career outcomes, including better career goal alignment, increased job satisfaction, and improved career advancement (Heppner et al., 1994).

It appears that knowing ourselves may assist our career decision making, how we further our careers, and make us happier in our work. So, as long as we don't put too much emphasis on the tests (don't treat them as gospel), then the tests give our clients some clarity.

Bonus.

Sam

References:

GVMP. (2011). The Global Volunteer Measurement Project. http://volunteermeasurement.org/

Heppner, M. J., O'Brien, K. M., Hinkelman, J. M., & Humphrey, C. F. (1994). Shifting the paradigm: The use of creativity in career counseling. Journal of Career Development, 21(2), 77-86. https://doi.org/10.1177/089484539402100202

SNZ. (2006). Finding and Keeping Volunteers [report]. Sport New Zealand [formerly SPARC]. http://www.sparc.org.nz/filedownload?id=850d18af-002f-40b7-b989-5a99e5b40f82

Stuart, B. (2004). Twelve Practical Suggestions for Achieving Multicultural Competence. Professional Psychology: Research and Practice 35(1) 3–9. https://doi.org/10.1037/0735-7028.35.1.3

VNZ. (2024). State of Volunteering Report 2024 [report]. Tuao Aotearoa | Volunteering New Zealand. https://www.volunteeringnz.org.nz/wp-content/uploads/f_SOV-report_2024_web.pdf

read more "Using career assessments from other countries"

Evaluating employee strengths

In my reading, last year, I encountered a meta-analysis (of sorts) by Miglianico et al. (2022) where the researchers evaluated 27 value or strength instrument studies published between 2010 and 2019, using a range of methods: cross-sectional; diary; experimental; and quasi-experimental.

Out of the review of this range of studies, and supported by the literature, the researchers proposed a five-step flow diagram for workplaces to identify and develop employee strengths. The five steps are shown in the image accompanying this post, and the steps themselves - as outlined by Miglianico et al. (2022) are as follows:

1. Work with the employee and "educate the[m...] about the strengths approach and the proposed [career] intervention". Employees must understand "and appreciate the value of the approach, to understand the steps involved in the process, and to be actively and genuinely involved in the intervention (Clifton and Harter 2003). The approach’s origins, advantages and limitations, as well as the overall process, must therefore be presented, and all questions must be answered (Dubreuil and Forest 2017). This step helps reduce negativity bias, the natural tendency of humans to give more attention to negative than positive information (Ito et al. 1998), and fully engage participants in the intervention from the start" (Miglianico et al., 2022, p. 757)
2. Next we "identify the person’s strengths" via "a psychometric instrument (e.g., StrengthsFinder, VIA-Survey, StrengthProfile), or in a less restrictive way by observing oneself (e.g., identifying activities that involve performance, energy, authenticity and flow; Biswas-Diener et al. 2011; Linley 2008; Linley and Burns 2010), or by collecting feedback from peers". Using a range "of different methods (e.g., psychometric instrument and feedback from peers) can yield a more accurate and complete picture of an individual’s strengths" (pp. 757-758).
3. Following that, we assist the employee to absorb the identified strengths and integrate them into their identity. Allowing time for employees "to fully grasp and assimilate this new information, better understand the reasons for [their] actions and observe [their] behavior in light of personal strengths. This new conceptualization of self can then be integrated into the identity before planning the next steps (Clifton and Harter 2003). It can be facilitated by appropriation exercises, such as specific questions linking strengths to previous successes (Dubreuil et al. 2016), feedback analysis (Roberts et al. 2005b), and self-portrayal exercises (Forest et al. 2012), in order to help the individual gain a deeper awareness of [their] strengths" (p. 758)
4. Once complete, we next put the ideas into action, in two parts. To begin, the employee "decides the specific changes [they want] to put in place to make better use of [their] personal strengths. The individual then implements the intended transformations. To help workers move from theory to action, strengths must be invested in specific individual, group, or organizational goals and initiatives (e.g., personal objectives, team projects, new tasks and responsibilities, complementary partnerships, etc.), and their application must be monitored or closely followed by managers, peers or coaches, who can provide support and encourage progress (Linley 2008). In the long term, it is important that the person always remain careful to avoid the overuse of strengths, and rather aims to use the right strength, to the right amount, and at the right time" (p. 758).
5. Lastly we review. "[R]esults can be evaluated subjectively through the individual’s appreciation of the progress made (in terms of strengths awareness and use, goal achievement, overall well-being, etc.), or objectively through changes in various variables that were measured prior to the intervention: wellbeing, job satisfaction, motivation, work engagement, or job performance. A measure of the impact of the intervention can then make it possible to ensure the effectiveness of the procedure and allow for readjustment if necessary" (p. 758).

This is a very handy process outline. Because it is research-based, we can probably rely on this meeting the needs of both individuals and the organisation. I think it would be relatively easy to implement; and I suspect it would also be easy to monitor, and to tweak. 

A simple tool for organisational growth, delivered through individual development.

Sam

Reference:

Miglianico, M., Dubreuil, P., Miquelon, P., Bakker, A. B., & Martin-Krumm, C. (2020). Strength use in the workplace: A literature review. Journal of Happiness Studies, 21, 737-764. https://doi.org/10.1007/s10902-019-00095-w

read more "Evaluating employee strengths"

Some assessment tools

While there are many assessment tools freely available online, some of them are of dubious quality. Hosted by Yissim, the Hebrew University of Jerusalem's technology transfer company, the Making Better Career Decisions (MBCD) is a not for profit site containing some sound assessment tools, here (Yissim, 2024b). 

As well as the Career Decision-making Difficulties Questionnaire (CDDQ) here *(Yissim, 2024a), this site includes a further eight free tools designed to assist individuals through the career decision process by clarifying difficulties (SCCI), providing career decision-making frameworks (CDMP), and a three-stage decision-making process (PIC). There is also some information about the MBCD planning system. 

The link for the CDDQ was originally at the National Career Development Association's website https://www.ncda.org/aws/NCDA/pt/fli/61764/false, but this has now been permanently moved to Yissim's site at https://kivunim.huji.ac.il/eng-quest/cddq_nse/cddq_nse_main.html (Yissim, 2024a). If we have the old link, it should continue to auto-route us onto the Yissim site.

Lastly, if we are seeking some homegrown Kiwi tools, we should head over to https://www.careers.govt.nz/tools/ for career checker, CareerQuest, Skill Matcher, Study Matcher, CV Builder and some planning tools (Careers New Zealand, 2024). 

All useful stuff!

Sam

References:

Careers New Zealand. (2024). Tools: Use our tools to get career ideas, explore study options and make your CV. https://www.careers.govt.nz/tools/

Yissim. (2024a). Career Decision-making Difficulties Questionnaire. https://kivunim.huji.ac.il/mobile/cddq/cddq_nse_2024/cddq_nse_main.html

Yissim. (2024b). Home. https://kivunim.huji.ac.il/cddq/

read more "Some assessment tools"

Evaluating career assessments

I found a website recently which provides an evaluation of a number of work and life instruments, such as the Career Decision-Making Model shown here (Emerge, 2023a). The research team - a consortium of Tulane University, the University of California San Diego, and the programme Evidence-based Measures of Empowerment for Research on Gender Equality, or "Emerge" -  uses a sensible framework of 14 attributes, against which each assessment is rated against, as shown on the image accompanying this post. The view taken is a feminist one, with a lot of psychological and sociological instruments measuring women's views being assessed. We need to search through the available instruments - I used the dropdown list of key dimensions on the home page (here; Emerge, 2023b) to find those which relate to career development work - and there are a LOT of instruments that have been rated. 

What is really simple is that the assessors have used an extended traffic light system. The green dots are for items with a full score, and orange for those which only partly apply. I presume that some assessments will have a red dot (i.e. they are unreliable in that category), which is not shown on this particular instrument (Emerge, 2023a). The extension to the traffic light system are that the grey dots, which appear to be for elements which the consortium have not evaluated; and black are for those items which do not relate to the particular construct. 

Quite a clever way of assessing an instrument at a glance :-)

Sam

References:

Emerge. (2023a). Career Decision-Making Self-Efficacy Scale. Tulane University; University of California San Diego; Evidence-based Measures of Empowerment for Research on Gender Equality (EMERGE). https://emerge.ucsd.edu/r_2y5nt5mpxmkhzen/#:~:text=The%20Career%20Decision%2DMaking%20Self,%2C%20Planning%20and%20Self%2DAppraisal

Emerge. (2023b). Home. Tulane University; University of California San Diego; Evidence-based Measures of Empowerment for Research on Gender Equality (EMERGE). https://emerge.ucsd.edu/

read more "Evaluating career assessments"

The Interpersonal circumplex assessment

The Interpersonal circumplex (IPC) theory is "is a two‐dimensional model in which various interpersonal qualities can be mapped in a circular space" (Mahar & Markey, 2020, p. 141), possibly based in Cartesian dualism with one axis indicating "Dominance-Submission and [the other axis being] Love-Hate" which helps us to define our interpersonal communication (Wiggins, 1996, p. 224).

This theory was originally named the Leary Circle, after an early Timothy Leary sketch of the idea which had springboarded from the work of Harry Sullivan, US psychiatry lecturer (Leary, 1957; Wiggins, 1996, p. 224; see diagram accompanying this post originally from LaForge, 1985, p. 615). The circle originally contained "16 interpersonal variables ordered around [two] primary dimensions of 'dominance‐submission' (i.e. agency) and 'love‐hate' (i.e. communion)" (Mahar & Markey, 2020, p. 143). These two continua are now considered to be a power axis, and a love axis.

What interests me is that the IPC model now seems to be thought of as a model of "extroversion, dominance, trust and warmth" (Williams, 2023), similar to the Big Five or OCEAN assessment (Mahar & Markey, 2020). However, as OCEAN "provides a framework with which to interpret the circumplex and the interpersonal circumplex provides an elaboration of two factors from the Five Factor Model" (Mahar & Markey, 2020, p. 143), it seems that the IPC still only measures the two continua of dominance (as extroversion) and love (as agreeableness), despite trust having crept into William's article (2023).

Regardless, these IPC elements of extroversion and agreeableness are useful when researching connection between humans and dogs. Professor and author Stanley Coren (GoogleScholar publications list here) researches how the human personality interacts with dogs, citing research showing that "couples who owned dogs were less likely to sue for divorce" (Williams, 2023). When we "come home, you’ve had a rotten day, all you want is a little TLC, but your partner has also had a rotten day and also needs a little TLC, that’s [likely] to end in an argument. But if you also have Lassie right here, you don’t put any extra pressure on your spouse" (Williams, 2023). It seems that dogs grow our love continuum. 

Aww!

Sam

References:

LaForge, R. (1985). The early development of the Freedman-Leary-Coffey interpersonal system. Journal of Personality Assessment, 49(6), 613-621. https://doi.org/10.1207/s15327752jpa4906_8

Leary, T. (1957). Interpersonal Diagnosis of Personality: a functional theory and methodology for personality evaluation. The Ronald Press Company.

Wiggins, J. S. (1996). An Informal History of the Interpersonal Circumplex Tradition. Journal of Personality Assessment, 66(2), 217–233. https://doi.org/10.1207/s15327752jpa6602_2

Williams, Z. (2023, October 4). Competitive, warm and conservative: what exactly makes someone a dog person?. The Guardian. https://www.theguardian.com/lifeandstyle/2023/oct/04/competitive-warm-and-conservative-what-exactly-makes-someone-a-dog-person

read more "The Interpersonal circumplex assessment"

Values and O*Net instruments evaluated by Buros

This is part 2 of an article published earlier this year (here), exploring commonly used tests which have been evaluated by the Buros Center for Testing, at the University of Nevada (Thorndike, 1999).

Following are all of the evaluations of O*Net and many of the values instruments completed by the Buros reviewers. As mentioned in the first half of this series, the number - 5th etc - stands for the Buros volume, and MMY stands for the title, which is the Mental Measurements Yearbook. The list of reviewed instruments is as follows (Buros Center for Testing, 2024):

• O*Net
  O*NET Career Interests Inventory, Third Edition: Based on the O*NET Interest Profiler, 20th MMY
  O*NET Career Interests Inventory: Based on the "O*NET Interest Profiler" developed by the U.S. Department of Labor, 16th MMY
  O*NET Career Values Inventory: Based on the "O*NET Work Importance Locator" developed by the U.S. Department of Labor, 16th MMY
  O*NET Interest Profiler, 16th MMY
  O*NET Work Importance Locator, 16th MMY
• Values Tests
  Values Inventory, 9th MMY
  Values Scale, Second Edition (The), 13th MMY
  Rokeach Value Survey, 12th MMY
  Career Values Card Sort, 13th MMY
  Career Beliefs Inventory, 12th MMY
  Work Values Inventory, 21st MMY
  Employment Values Inventory, 14th MMY
  Filipino Work Values Scale (The), 14th MMY
  Hall-Tonna Inventory of Values (The), 11th MMY
  Maferr Inventory of Feminine Values, 9th MMY
  Maferr Inventory of Masculine Values, 9th MMY
  Mirror Edition of the Personal Values Inventory, 16th MMY
  Motives, Values, Preferences Inventory, 14th MMY
  Personal Values Questionnaire, 13th MMY
  Rothwell Miller Values Blank, 14th MMY
  Selby MillSmith Values Indices, 14th MMY
  Survey of Personal Values, 10th MMY
  Survey of Work Values, Revised, Form U, 12th MMY
  Temperament and Values Inventory, 9th MMY
  Work Orientation and Values Survey, 17th MMY

I hope this is useful! 

Sam

References:

Buros Center for Testing (2024). Tests reviewed in The Mental Measurements Yearbook series. https://buros.org/tests-reviewed-mental-measurements-yearbook-series

Thorndike, R. M. (1999). Book Review: Conoley, JC, & Impara, JC (Eds.). (1995). The Twelfth Mental Measurements Yearbook. Lincoln, NB: The Buros Institute of Mental Measurements. Journal of Psychoeducational Assessment, 17(1), 50-55. https://doi.org/10.1177/073428299901700105

read more "Values and O*Net instruments evaluated by Buros"

Common tests evaluated by Buros

This is a topic I have covered before (here), but there is always more to say! I thought I would create a list of tests which are commonly used in Aotearoa, and have been evaluated by the Buros Center for Testing, at the University of Nevada (Thorndike, 1999).

The tests I have listed are those that my students have said they had previously used with clients when they began the "Assessment Tools" paper with me. The number - 5th etc - stands for the volume, and MMY stands for Mental Measurements Yearbook. The list is as follows (Buros Center for Testing, 2024):

• 16PF (Like MBTI)
  Sixteen Personality Factor Questionnaire, 9th MMY
  Sixteen Personality Factor Questionnaire, Fifth Edition, 12th MMY
• MBTI
  Myers-Briggs Type Indicator Step II (Form Q), 15th MMY
  Myers-Briggs Type Indicator, 10th MMY
  Myers-Briggs Type Indicator, 9th MMY
  Myers-Briggs Type Indicator, Form M, 14th MMY
  Myers-Briggs Type Indicator¨ Step III, 20th MMY
• Holland
  Self-Directed Search--Second Australian Edition, 15th MMY
  Self-Directed Search, 4th Edition [Forms R, E, and CP], 13th MMY
  Self-Directed Search, 4th Edition [Forms R, E, and CP], 14th MMY
  Self-Directed Search, 5th Edition [Form R], 20th MMY
• DiSC
  DiSC Classic, 16th MMY
  DiSC Indra, 16th MMY
• HBDI
  Herrmann Brain Dominance Instrument [Revised], 14th MMY
  Herrmann Brain Dominance Instrument, 11th MMY
• Schein's Career Anchors
  Career Anchors: Discovering Your Real Values, Revised Edition, 13th MMY
• Krumboltz's Career Beliefs
  Career Beliefs Inventory, 12th MMY
• Knowdell's Career Values Cards
  Career Values Card Sort, 13th MMY

If you already have a US Open Library account, you can review past Buros volumes here held at the Internet Archive for an hour at a time. The search in the left-hand sidebar allows us to find the tests we seek quite quickly: much faster than trawling through the book!

If you haven't yet opened an Open Library account, register here. This online service is a superb catalogue for finding texts long out of print, and this links seamlessly with the Internet Archive, which often stores the volumes.

This is part one of a two part article. In part 2, coming in a month or so, I will list inventories linked to O*Net and to other values tests. 

Sam

References:

Buros Center for Testing (2024). Tests reviewed in The Mental Measurements Yearbook series. https://buros.org/tests-reviewed-mental-measurements-yearbook-series

Thorndike, R. M. (1999). Book Review: Conoley, JC, & Impara, JC (Eds.). (1995). The Twelfth Mental Measurements Yearbook. Lincoln, NB: The Buros Institute of Mental Measurements. Journal of Psychoeducational Assessment, 17(1), 50-55. https://doi.org/10.1177/073428299901700105

read more "Common tests evaluated by Buros"

A range of personality tests

When we are casting about for our next logical steps in a career journey, sometimes it is worth doing a number of assessment inventories to see if they will show us something new about ourselves: or if we will see a new pattern in a broad range of results which was not visible to us previously. There are a range of assessments which we can take, which you can read more about here. 

Following are some free assessments which we can take to explore a number of facets of ourselves:

• The Big Five Inventory, or the OCEAN quiz, which is a personality traits test. This is the only test reputed to have scientific validity, but we need to remember that it was created in the USA, testing professional men. Download the page here (Daft, 2007)
• The personality sorting hat of MBTI, a personality test, via a workbook (here) and results analysis (here; McNeill, 2004). Read more on this assessment here
• A fast and dirty HBDI quiz (here), a thinking styles quiz. Learn more about this test here (Jamieson, 2011)
• What type of follower are you, here (Rohde & Ford, 2007). Learn more about Kelley's followership styles here
• One for fun, Smalley's personality test here (Smalley, 1997). The value in this test is considering the "alternative" way that our attributes may be seen
• An online values test here, based on Schein's Career Anchors (123 Test, 2024)
• An interest inventory, Holland Code or RIASEC test here (Open-Source Psychometrics Project, 2024a)
• A personality test, DISC inventory, here (Tripp, 2024)
• A communication style quiz here (LeadershipIQ, 2024)
• A range of other tests can be found here (Open-Source Psychometrics Project, 2024b)

As these are (a) self-reporting tests, and (b) free versions, some tests may not evaluate us well. Further, if the results of any tests seem off-track, we should get someone else to give us a double-check for sense. If any test results feel very insightful, we may decide to seek out a professional paid version with a consultant debrief to give us a better quality exploration.

I hope these are useful!

Sam

References:

123 Test. (2024). Work values test. https://www.123test.com/work-values-test/

Daft, R. L. (2007). The Leadership Experience (4th ed.). Thomson South-Western.

Jamieson, K. (2011). HBDI NMIT Students Presentation. Jamieson Harvey & Associates

Human Metrics. (2024). Discover your personality type. https://www.humanmetrics.com/personality

LeadershipIQ. (2024). Communication Styles Quiz And Assessment. https://www.leadershipiq.com/blogs/leadershipiq/39841409-quiz-whats-your-communication-style

McNeill, G. (2004). A Brief Introduction to Myers-Briggs. Careers New Zealand.

Open-Source Psychometrics Project. (2024a). Holland Code (RIASEC) Test. https://openpsychometrics.org/tests/RIASEC/

Open-Source Psychometrics Project. (2024b). Recommended tests. https://openpsychometrics.org/

Rohde, S., & Ford, D. (2007). Determining Your Followership Style. www.roosevelt.edu/hr/td/documents/DeterminingYourFollowershipStyle.ppt

Smalley, G. (1997). Chapter 10: Understanding Personality Types: A Key to Lovability in Making Love Last Forever. http://www3.dbu.edu/jeanhumphreys/SocialPsych/smalleytrentpersonality.htm

Tripp, R. E. (2024). What Color Is Your Personality? (Very in-dept Test!). https://www.proprofs.com/quiz-school/story.php?title=mtm5ntkznwk5rq

read more "A range of personality tests"

Exploring the NEO Personality Inventory-3 (UK Edition)

Our longer term characteristics - our personality - can be “defined as the relatively enduring patterns of thoughts, feelings, and behaviours that distinguish individuals from one another” (Roberts & Mroczek, 2008, p. 31). We can also consider personality as how our natural tendencies and inclinations differ from others within our own society, or as an “enduring set of Traits and Styles" where we exhibit certain "characteristics" (Bergner, 2019, p. 4).

Commonly used in the study and research of personality, the ‘Big Five’ or Five Factor Model (FFM) is made up of the five broad personality dimensions: extraversion; agreeableness; conscientiousness; neuroticism; and openness to experience. Developed from early research looking at how trait theory relates to individuals' temperament and behaviour (de Raad & Mlačič, 2015), research into the FFM has covered many populations and cultures and appears “to be the most widely accepted theory of personality today” (Lim, 2020). The FFM dimensions are structured into instruments to measure how individuals thinks, feels, and behaves, which collectively aids our understanding of personality difference (de Raad & Mlačič, 2015).

One of the instruments designed to test the FFM is the NEO Personality Inventory, or NEO PI. A number of research “studies in many different settings have verified the overall factor structure and construct validity of the Big Five [model...], based on many different demographic and cultural characteristics of individuals” as participants (Lounsbury, 2005, p. 709). Originally created in 1978 for use with adults, in the early stages it was clear that college students would also benefit from its use, but would require separate norms (McCrae et al., 2010). Later studies using samples as young as 10 years old showed that the revised version, NEO-PI-R, could be used but some items were difficult for younger respondents to understand. High school students - instructed to leave blank items not understood - found 30 of 240 test items difficult (McCrae et al., 2010). Using more current, colloquial language - although originally designed for adolescents - also improved the test for adult test-takers (McCrae et al., 2010). The latest version is the NEO-PI-3 (Lounsbury, 2005), and is a three-level self-report instrument, consisting of 240 items, a validity question, 30 facets, to test the five domains of “Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness” (Vassend & Skrondal, 2011, p. 1301). Individuals rate the 240 items on a 5-point Likert scales ranging from strongly disagree to strongly agree (Hattrup & Smith, 2021; Hey, 2022).

The UK version of the NEO-PI-3 consists of language and normative data more appropriate for use with British individuals aged 16 and up, although it should be noted that norm data is not available for those under 18 (Hattrup & Smith, 2021). The UK edition shows consistent reliability with the US version, showing retest reliability supporting the conclusion that it measures stable traits (Hattrup & Smith, 2021). Like the US version, the UK NEO-PI-3 can be administered and scored both online or in paper version, remotely or in-person. While the test itself may only take 30-40 minutes, it is suggested that an hour be allowed to brief the client, and for the client to make considered responses (Hattrup & Smith, 2021).

Research and testing from multiple sources indicate that the NEO-PI-3 is appropriate for many ages and stages in career development. But gender options are binary only (Kluck, 2014), which ignores - invalidates - those who don’t identify this way. In addition, like the US, the UK samples too are homogenous, with over 90% of participants identifying as Caucasian (Hattrup & Smith, 2021).

While it’s noted that UK norms resemble US data (Hattrup & Smith, 2021), it cannot be assumed that this necessarily translates to Aotearoa's super-diversity context (Chen, 2015). Practitioners must consider collective and individual culture (Laher, 2013), relevant when working with Māori and Pasifika kaimahi and ākonga. Further, considering Kiwis who like to get on with others, candidates may feel compelled to conform with societal 'expectations' when answering (Kumar, 2019). A step further on, some participants may fear negative ramifications if vulnerabilities or ‘flaws’ are exposed. However, there is a “Problems in Living Checklist” at the end of each report (Costa & McCrae, 2010) which is helpful in allaying client concerns.

Whether we decide to use the test or not, it is useful to explore the issues.

Alex

References:

Bergner, R. M. (2020). What is personality? Two myths and a definition. New Ideas in Psychology, 57, 100759, 1-7. https://doi.org/10.1016/j.newideapsych.2019.100759

Chen, M., (2015). Superdiversity Stocktake: Implications for business, government and New Zealand. Superdiversity Centre For Law, Policy And Business. https://www.superdiversity.org/wp-content/uploads/Superdiversity-Stocktake-Section1.pdf

Costa, P. T., & McCrae, R. R. (2010). NEO™ Personality Inventory-3: Interpretive Report. Australian Council for Education. https://www.acer.org/files/NEO_PI-3_Interp_Rpt_Sample_Report.pdf

de Raad, B., & Mlačič, B. (2015). Big Five Factor Model, Theory and Structure. In J. Wright, C. Fleck (Eds.), International Encyclopaedia of Social & Behavioural Sciences (2nd ed., Vol. 2, pp. 559-566). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.25066-6

Hattrup, K., & Smith, J. V. (2021). [101] NEO Personality lnventory-3 (UK Edition). In J. F. Carlson, K. F. Geisinger, & J. L. Jonson (Eds.) The Twenty First Mental Measurements Yearbook (pp. 450-455). The Buros Institute of Mental Measurements.

Hey, L. (2022). Presenting a new NEO-PI-3 International Senior Manager Norm for a post-covid-19 world. Hogrefe Ltd. https://www.hogrefe.com/uk/index.php?eID=dumpFile&t=f&f=10141&token=8296bcf0af59cbf9aa92753a20dce8f92057ad8f

Kluck, A. S. (2014). [116] NEO Personality lnventory-3. In J. F. Carlson, K. F. Geisinger, & J. L. Jonson (Eds.), The Nineteenth Mental Measurements Yearbook (pp. 477-483). The Buros Institute of Mental Measurements.

Laher, S. (2013). Understanding the Five-Factor Model and Five-Factor Theory through a South African cultural lens. South African Journal of Psychology, 43(2), 208–221. https://doi.org/10.1177/0081246313483522

Lim, A. G. Y. (2020). What Are the Big 5 Personality Traits? Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Simply Psychology. https://www.simplypsychology.org/big-five-personality.html

Lounsbury, J. W., Saudargas, R. A., Gibson, L. W., Leong, F. T. (2005). An Investigation of Broad and Narrow Personality Traits in Relation to General and Domain-Specific Life Satisfaction of College Students. Research in Higher Education, 46(6), 707-729. https://doi.org/10.1007/s11162-004-4140-6

McCrae, R. R., Costa, P. T., Martin, T. A. (2010). The NEO-PI-3: A More Readable Revised NEO Personality Inventory. Journal of Personality Assessment, 84(3),  261-270. https://doi.org/10.1207/s15327752jpa8403_05

Roberts, B. W., Mroczek, D. (2008). Personality Trait Change in Adulthood. Current Directions in Psychological Science, 17(1), 31-35. https://doi.org/10.1111/j.1467-8721.2008.00543.x

Vassend, O., & Skrondal, A. (2011). The NEO personality inventory revised (NEO-PI-R): Exploring the measurement structure and variants of the five-factor model. Personality and Individual Differences, 50(8), 1300-1304. https://doi.org/10.1016/j.paid.2011.03.002

* Alex Howe has kindly prepared much of the material for this post

read more "Exploring the NEO Personality Inventory-3 (UK Edition)"

Career challenge card sort

Anyone who wants to try a digital card sort might enjoy the SparkPath offering at McMaster University: via a set of digital challenge cards. I encountered this when attending the CANNEXUS conference earlier this year (2023), in a presentation from the authors of a new book, Mapping the Future of Undergraduate Career Education (Buford et al., 2022). One of the chapter authors detailed the use of the digital challenge cards, which was very interesting (Chapter 12; Stebleton & Franklin, 2022).

After watching the presentation I was quite keen to check out the cards, and found the digital resource online here (McMaster University, 2023). Students can go online, sign up so they get their results, then work through the cards. Their results will provide some ideas for discussion. The 'challenge' element of the cards comes in when users select their favourite cards in the card sort using emojis, as all the cards are focused on solving a problem, not on doing a job. This is really a form of interest inventory. 

The idea is that we go through the SparkPath card sort and select not interested, interested, or very interested for each card. Then we chose our first, second and third most loved cards at the end. We can download a personal report based on our choices. We can also download a workbook to continue our exploration.

However, having tried this myself, I found some negatives in the widespread use of this digital card sort: 

1. Age. Even though the card design is bright and cheerful, the language targets secondary school students; potentially younger secondary school students. I would not recommend this as a tool for adults as I think adults may get impatient and feel condescended to. 
2. Code. A paid access code is needed to even access the card sort. I used "macssccareers" to sign up, so you could try that. 
3. Personalisation. The resulting reports are extremely simplistic and the personalised report is not really personalised at all. It is a standard report containing three pages detailing our three main choices. For an adult there is simply not enough depth. The workbook might be helpful, but this contains absolutely no personalisation whatsoever.
4. Simplicity. I am not convinced that there is enough underlying complexity in the number and range of challenge cards, the resulting information, or the reporting to make paying for access to this card sort worthwhile.
5. Cultural alignment. Some of the cards I chose didn't fit my expectations due to language differences. Terms we use in New Zealand are not the same as those used in North America. Further, this was a very Western-centric card sort, being quite individualistic. There was little which focused on the collective, or the relational. 

Overall, while this is a great idea - for a monthly subscription model - I am unconvinced that there is enough value to warrant purchasing this card sort. If I was working with large numbers of 11-13 year olds I might. But otherwise, no.

I am glad I explored it though. 

Sam

References:

Buford, M. V., Sharp, M. J., & Stebleton, M. J. (Eds.). (2022). Mapping the Future of Undergraduate Career Education: Equitable Career Learning, Development, and Preparation in the New World of Work. Taylor & Francis.

CANNEXUS. (2023). Stebleton, M. J., Franklin, M., Michel, J. P., Mapping the Future of Career: A New Textbook [video]. CANNEXUS23 Conference 23-25 January 2023. https://cannexus23.gtr.pathable.com/meetings/virtual/jfn4JNM26uHoB8Xra

McMaster University. (2023). The Digital Challenge Cards. https://studentsuccess.mcmaster.ca/digital-challenge-cards/

Stebleton, M. J. & Franklin, M. (2022). Chapter 12: Applying Narrative Approaches to Support Undergraduate Career Decision-Making. In M. V. Buford, M. J. Sharp, M. J. Stebleton (Eds.), Mapping the Future of Undergraduate Career Education: Equitable Career Learning, Development, and Preparation in the New World of Work (pp. 183-199). Taylor & Francis.

read more "Career challenge card sort"

A critique of MBTI

Career assessments are part of a holistic career counselling process. Assessments have a role as a starting point for clients to “identify, understand and appreciate the unique aspects that make them up as an individual” (Osborn & Zunker, 2016, p. xii). Tests can catalogue client aptitudes, abilities, interests, values, personality and career decision-making skills, as an integral component in our career practitioner toolbox. Further, career assessment tools are often standardised quantitative instruments, which use norms - or “typical scores” - to enable comparisons of client results with a statistical representation of a similar population (Osborn & Zunker, 2016).

As part of our process of seeking appropriate instruments to use with a client, not only do we need to consider how reliable, and valid an assessment is; and what norms are used (Osborn & Zunker, 2016), but also how culture and personal characteristics may affect test norms. We should also factor in aspects of "age, gender, ability, race, ethnic group, national origin, religion, sexual orientation, linguistic background” (Flores et al., 2003, p. 45). During test construction, minorities are often not well catered for, meaning that we cannot necessarily assume accurate generalisations about client results (Flores et al., 2003).

We also must consider "equivalence" with assessment, as it relates to language, constructs, scales and norms within any test (Flores et al., 2003). The meaning of words and ideas - such as behaviour or values - change between different cultures. We need to ask ourselves if the questions/scales are relatable and understandable; and if the culture of clients is represented in norms. We need to be aware of the duality of those who walk in two value worlds - such as Māori and Pasifika people - of both individuality and the collective (Apulu, 2022). These differences must be noted and taken into account when interpreting results.

According to the Myers-Briggs website, the Myers-Briggs Type Indicator, or MBTI,  is “based on large representative norms that account for race, age and gender” (The Myers Briggs Foundation, 2022). They note that there have been “hundreds of studies over the past 40 years which have proven the instrument to be both valid and reliable" (The Myers Briggs Foundation, 2022). In the Buros Centre for Testing review, the Myers Briggs Foundation is quoted as stating that the MBTI is the "most popular personality type inventory" (Mastrangelo, 2001, p. 816).

MBTI is designed for ages 14 year and older, and is very accessible: there are a number of free tests which self-calculate online; if completing a full inventory with a licenced provider the test can be hand written and scored, or completed as an online version. MBTI is available in 29 languages, which should be useful for a range of cultures to complete the test in their first language (Mastrangelo, 2001; The Myers Briggs Foundation, 2022). 

While we may find this test personally valid, when working with clients it is essential to independently establish the validity of qualitative assessments. The Buros Centre for Testing found that the MBTI test/retest reliability after 4 weeks was only 65% (Mastrangelo, 2001). Sixteen years later in 2017, MBTI reliability estimates ranged from 38% to 97%, again averaging around two thirds (Harris, 2017). This means that over a third of clients will get a different result if they retook the test within 90 days. Our clients are likely paying $350 for a professional MBTI assessment from a licenced practitioner, and it is easy to see that many would be unhappy with that level of reliability in their results.

The founder of many of the concepts which MBTI is based upon, Carl Jung, warned that his personality types were useful primarily as tools for studying large numbers of people, and became all but meaningless when applied to individuals (Pittenger, 2005), throwing further doubt upon the validity of the test. It has also been noted that MBTI appears to have “no evidence to show a positive relation[ship] between MBTI tests and success within an occupation” (Pittenger, 1993, p. 52). Critiques have also been made about the binary scoring scales leading to less than valid results (Harris, 2017; Mastrangelo, 2001). Tests such as the Big Five (aka NEO) use scales, considered more valid and appropriate measure. 

MBTI can be a useful tool to gather general preference information and a good self reflection tool, but it should not be relied upon AT ALL for hiring decisions. It is a preference indicator not a personality measure (Mastrangelo, 2001).

Sam, Alexandra, Donna, Karen & Helen

References:

Apulu, M. (2022). How to grow a culturally responsive career practice. [Master's thesis: University of Otago]. https://www.researchbank.ac.nz/bitstream/handle/10652/5711/MPP_2022_Peter_Apulu.pdf

Flores, L. Y., Spanierman, L. B., & Obasi, E. M. (2003). Ethical and professional issues in career assessment with diverse racial and ethnic groups. Journal of Career Assessment, 11(1), 76-95. https://doi.org/10.1177/106907202237461

Harris, S. M. (2017). [127] Myers-Briggs Type Indicator® Step III. In J. F. Carlson, K. F. Geisinger, & J. L. Jonson (Eds.) The Twentieth Mental Measurements Yearbook (pp. 521-526). The Buros Institute of Mental Measurements.

Mastrangelo, P. M. (2001). [251] Myers-Briggs Type Indicator [Form M]. In B. S. Plake & J. C. Impara (Eds.), The Fourteenth Mental Measurements Yearbook (pp. 816-820). Buros Center for Testing.

Nord, C. (2017). Could fMRI be a viable biomarker in psychiatry? A test-retest reliability fMRI study. https://www.ucl.ac.uk/pals/research/experimental-psychology/blog/fmri-viable-biomarker-psychiatry-test-retest-reliability-fmri-study/

Pittenger, D. J. (1993). Measuring the MBTI...And Coming Up Short. Journal of Career Planning and Placement, 54(1), 48-53.

Pittenger, D. J. (2005). Cautionary comments regarding the Myers-Briggs type indicator. Consulting Psychology Journal: Practice and Research, 57(3), 210-221. https://doi.org/10.1037/1065-9293.57.3.210

Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

The Myers Briggs Foundation (2022). MBTI® Basics. https://www.myersbriggs.org/my-mbti-personality-type/mbti-basics/

* Karen Bennett, Alexandra Howe, Donna Manley & Helen Davie-Martin have kindly prepared much of the material used in this post. And I have mashed it up, and connected it :-)

read more "A critique of MBTI"

Qualitative versus quantitative career interventions

Assessment tools can be split into two broad categories, those of quantitative (number-, test- or survey-oriented, deductive, using mathematical modelling and numerical statistical patterns) or qualitative (relational, narrative, interpersonal, interview-, activity-, and discussion-oriented, using drawing) types. These tools are used to guide our work with clients, assisting us to measure client characteristics such as values, skills, abilities, interests and personality. These two categories also help both the client and ourselves understand how their personal characteristics connect with occupational selection (Swanson & Fouad, 2020). 

Quantitative tools tend to be represented by instruments such as standardised tests, measuring traits, counting and grouping interests; therefore, the psychometric properties of validity, reliability and norms hold high importance when considering the use of instruments that fall within this category (see here for more information). To be valid, tests must be normalised and standardised to be sure that the results are consistent over the population that is being assessed. Tests must be able to be taken once, then retaken and obtain close to the same result (test/retest validity; Osborn & Zunker, 2016).  

On the other hand, qualitative tools are non-standardised tools, such as Savickas’ Career Construction interview (CCI), narrative therapy, card sorts and career genograms. Qualitative tools assist when working with diverse clients as they “enliven the career counselling process” (Okocha, 1998, p. 5). For example, genograms - vocational family trees - capture a client’s heritage. This can be immensely useful for exploring family patterns, modelling, and dispelling outdated ideas (Osborn & Zunker, 2016). 

Rather than taking a trait-based, person-fit approach, some theories encourage a relational approach. Career construction theory, or CCT, addresses the needs of a workforce facing challenges  (Savickas, 2013). If we stop to think about how much the world has changed in recent years, it is easy to see how technology, rationalisation, redundancy, up-skilling, has resulted in 'new' roles: who would have thought of a "Work-from-home facilitator" pre-Covid? (Kelly, 2021). With CCT, the practitioner utilises 'life design' processes such as story-telling and self-construction techniques, taking either a group or individual approach (Maree, 2019). Career Construction enables clients to build their view of self “from the inside out,” rather than from the outside in, as trait theory prescribes (Savickas, 2013, p. 182). Research seems to indicate that a more relational approach builds greater adaptability and resilience (Savickas, 2013). Practitioners applying this approach may choose to use quantitative assessments - such as values inventories - or not, as best suits each client.

The main thing is that tools should not channel us or our clients: they should assist the client get to know themselves better, and to assist the client to make good quality choices. 

Sam

References:

Kelly, J. (9 May 2021). 10 Hot, Fast-Growing Jobs For The Future Post-Pandemic World. Forbes. https://www.forbes.com/sites/jackkelly/2021/05/19/10-hot-fast-growing-jobs-for-the-future-post-pandemic-world/?sh=f7d44ad5d064 

Maree, J. G. (Ed.). (2019). Handbook of Innovative Career Counselling. Springer.

Okocha, A. A. (1998). Using qualitative appraisal strategies in career counseling. Journal of Employment Counseling, 35(3), 151-159. https://doi.org/10.1002/j.2161-1920.1998.tb00996.x

Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

Savickas, M. L. (2013). Career Construction Theory and Counseling Model. In S. D. Brown & R. W. Lent (Eds.) Career Development and Counseling. Putting Theory and Research to Work (2nd ed.). John Wiley & Sons.

Swanson J. L., & Fouad, N. A. (2020). Career Theory and Practice: Learning through case studies. Sage Publications, Inc.

read more "Qualitative versus quantitative career interventions"

Culture-Infused Career Counselling

A person’s culture is determined by a combination of some or all of "ethnicity, gender, religion, ability, sexual orientation, age, and social class" (Arthur & Collins, 2011, p. 147). Developed in Canada, the Culture-Infused Career Counselling (CICC) model (Arthur & Collins, 2011, Arthur, 2019) is based on the premise that a person's culture and identity are relevant to career concerns and must be considered to provide a fair, just career intervention (Arthur, 2019). CICC provides a framework - model - to incorporate culture into our practice as "cultural influences are inextricably woven into a [client]’s career development" process (Arthur & Collins, 2011, p. 147). CICC "focuses on establishing an effective and culturally sensitive working alliance with clients" (p. 148).

The CICC model has four stages, as follows:

1. "Gaining awareness of personal cultural identities" (Arthur, 2019, p. 22)
2. "Gaining awareness of the cultural identities of other people" (p. 22)
3. "Understanding cultural influences on the working alliance" (p. 22)
4. "Implementing culturally responsive and socially just career interventions"  (p. 23).

As career practitioners, we need to be awake to our personal cultural approach in our client work, where those clients come "from nondominant populations" (Arthur & Collins, 2011, p. 148). Further, we must ensure that any interventions we choose will have an appropriate meaning and purpose "within the cultural contexts of [our] clients’ lives" (Arthur, 2019, p. 27). 

Finding information on what is 'appropriate' for our client norm group may be difficult to determine because of the lack of applied research in the Antipodes, but, following the CDANZ code of ethics (2016), considering the following elements will lead us to good practice:

• "Respect - the dignity and personal rights of the client involved and the client’s right to self-determination, and treat the client honestly, and with respect, empathy, and integrity at all times" 
• "Ensure – that any ethical and cultural dimensions relevant to the client are respected" 
• "Remain - fully aware of their social responsibility and the impact of their recommendations and actions" (CDANZ, 2016).

It is our role to ensure we "keep ethics and culture in dialogue with each other" (Agee et al., 2011, p. 29). It reminds us that if we are working with a client who is of a different culture to ourselves, whose shoes we have not walked in, we need to be very careful not to make ‘assumptions’.

Eleanor

References:

Arthur, N., & Collins, S. (2011). Infusing culture in career counseling. Journal of Employment Counseling, 48(4), 147-149. https://doi.org/10.1002/j.2161-1920.2011.tb01098.x

Arthur, N. (2019). Chapter 3: Culture-Infused Career Counselling: Connecting culture and social justice in career practices. In N. Arthur, R. Neault, McMahon, M (Eds.) Career Theories and Models at Work: Ideas for practice (pp. 21 – 30). CERIC.

Agee, M., Crocket, K., Fatialofa, C., Frater-Mathieson, K., Kim, H., Vong, C. & Woolf, V. (2011). Chapter 1.3 Culture is Always Present: A conversation about ethics. In K. Crocket, M. Agee, S. Conforth (Eds.) Ethics in Practice: A guide for counsellors (pp. 28-32). Dunsmore Press.

CDANZ. (2022). Code of Ethics. Career Development Association of New Zealand. https://cdanz.org.nz/ModularPage?Action=View&ModularPage_id=26

* Eleanor Blakey has kindly prepared most of the material for this post

read more "Culture-Infused Career Counselling "

A little look at Holland

Research and experience as a vocational counsellor in educational, military and clinical settings led to John Holland to conclude that people could be categorised by a mix of six main personality characteristics, or vocational types. The types begin with heredity, but are shaped by our activities, our interests, the competencies we develop, which then leads - via our home, school, Whanau and friends giving us opportunities and encouragement aligning with the dominant environmental types - to our "disposition". Our disposition is our self-concept, our individual view of ourselves and the world around us, our values, and how attuned we are to our environmental influences, and our personality traits (Holland, 1973).

The traits are: Realistic; Investigative; Artistic; Social; Enterprising; and Conventional - more commonly known as RIASEC (Holland, 1973). Early on in this process, Holland developed a hexagon model showing the relationship between types. Types closer to each other on the hexagon are more closely related than the types which are further away (Holland, 1973, p. 23). After completing an inventory, people can be classified by their vocational type by a three-letter code,  representing the individual’s dominant three vocational types (Holland, 1997). For example, someone who is highly Social, Realistic and Investigative is given the code SRI. 

The research, and the model has provided a useful person-job fit assessment for use in the US. Further, over time, Holland - in conjunction with Gary Gottfredson - developed a classification system, the Occupations Finder (Nauta, 2010), utilised to match individual  RIASEC types to current occupations. The occupations finder underpins O*Net, the US Department of Labour tool.

Researchers have tested Holland’s RIASEC model fairly rigorously during the past four decades. Much of Holland’s research findings - including via meta-analyses - are supported, including: support for type in secondary school settings (Sung et al., 2015); individuals in RIASEC matched jobs and environments tend to perform better and have higher job satisfaction (Nye et al., 2012); RIASEC types are stable over time (Low et al., 2005); and adjacent hexagon types are more strongly related than non-adjacent types (Armstrong et al., 2003).

Many of Holland’s assessments - including the self directed search, occupational finder, career key and strong interest inventory - have been altered and translated so that they can be understood and administered to different national and ethnic populations. Care is taken so that concepts are not ‘lost in translation’ and local perceptions of prestige and commonness of occupations are reflected in the altered version, hence cross-cultural equivalence is achieved. High scores for internal validity and consistency are evidence of valid Holland assessments for that particular population. Cross cultural studies on a range populations show construct validity, concurrent validity, internal validity and consistency of 'localised' Holland assessments, including: Singaporean Chinese (Soh & Leong, 2001); Croatian adolescents (Sverko & Babarovic, 2004, cited in Watson et al., 2005); Chinese, Japanese, Hong Kong, Spanish speakers and American Latinos/Latinas (Bullock et al., 2009; Hansen, 2013; Nauta, 2020; Tang, 2018).

However research about the relationships between positions on the hexagon shape/circular order theory has revealed inconsistencies across cultures. The research supports the structural validity for populations in Singapore, Korea and Hong Kong (Bullock et al., 2009) but it is not always supported in South African populations (Watson et al., 2005) and residents of Asian countries and Asian Americans (Nauta, 2020). In fact, it seems that the circular order theory fits well only in countries with higher economic development (Nauta, 2020).

Research also indicates that RIASEC scores appear to be gendered (Nauta, 2020). In general, women outscore men on the social, artistic, and conventional types, and men outscore women on the realistic and investigative types. This may reflect gender socialisation (i.e. women are societally 'expected' to have social, artistic and conventional role experiences and opportunities). Gender socialisation is supported by gender differences with some RIASEC types decreasing since the mid‐2000s (Nauta, 2020).

In summary, experts agree that the empirical research widely supports Holland's major propositions relating to the RIASEC typology and congruence theories, but that care should be taken when applying the interrelationship among the types across cultures (Bullock et al., 2009; Nauta, 2020; Tang, 2018).

Teresa

References:

Armstrong, P., Hubert, L., & Rounds, J. (2003). Circular unidimensional scaling: A new look at group differences in interest structure. Journal of Counseling Psychology, 50(3), 297–308. https://doi.org/10.1037/0022-0167.50.3.297

Bullock, E. E., Andrews, L., Braud, J., & Reardon, R. C. (2009). Holland’s Theory in an International Context: Applicability of RIASEC Structure and Assessments Technical (No. 50). Florida State University.

Holland, J. H. (1973). Making vocational choices: a theory of careers. Prentice-Hall.

Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Psychological Assessment Resources.

Holland, J. L., Shears, M., & Harvey-Beavis, A. (2001). Self-Directed Search (2nd Australian ed.).  Australian Council for Educational Research Ltd.

Low, K. S. D., Yoon, M., Roberts, B. W., & Rounds, J. (2005). The stability of vocational interests from early adolescence to middle adulthood: A quantitative review of longitudinal studies. Psychological Bulletin, 131(5), 713–737. https://doi.org/10.1037/0033-2909.131.5.713

Nauta, M. M. (2010). The Development, Evolution, and Status of Holland’s Theory of Vocational Personalities: Reflections and Future Directions for Counseling Psychology. Journal of Counseling Psychology, 57(1), 11–22. https://doi.org/10.1037/a0018213

Nye, C. D., Su, R., Rounds, J., & Drasgow, F. (2012). Vocational Interests and Performance: A Quantitative Summary of Over 60 Years of Research. Perspectives on Psychological Science, 7(4), 384–483. https://doi.org/10.1177/1745691612449021

Tang, M. (2019). Career Development and Counselling: Theory and practice in a multicultural world. SAGE Publications, Inc.

Watson, M., Duarte, M. E., & Glavin, K. (2005). Cross-cultural perspectives on career assessment. The Career Development Quarterly, 54(1), 29-35. https://doi.org/10.1002/j.2161-0045.2005.tb00138.x

* Teresa Winstone has kindly prepared most of the material for this post

read more "A little look at Holland"