Making sense of testing

We use career assessments in order to help our clients in identifying their unique characteristics. Each assessment is designed to measure different components, thus - with appropriate interpretation - assisting our clients to find career options which match their particular attributes, values, and skills (Osborn & Zunker, 2016).

While tests can assist client's decision making processes (Whitfield et al., 2009), to be effective, those tests need to be reliable and valid (Walsh & Betz, 2000). If a test is valid, it means that it actually measures what it says it measures: it does what it says on the tin (Heale & Twycross, 2015). There are three key types of validity: content validity (test accuracy); construct validity (does what it says on the tin - e.g. testing for job search skills might inadvertently be evaluating problem-solving skills); and criterion-related validity (where the same factor - or variable - is measured each time, through 'convergent' validity which is strongly correlated with similar tests; 'divergent' validity with poor correlation to different tests; and 'predictive' validity where the test is highly correlated to related factors - e.g. being task-oriented should lead to being a completer/finisher) (Heale & Twycross, 2015). 

Tests also need to have been normalised for the population group our client affiliates (awhis) to. That means that, when assessments are created, researchers have run a number of sample tests (usually around 300; Steve Evans, personal communication, 13 September 2021) on each population group, seeking normal distribution in the test results via cultural, ethnic, gender, political and socio-economic group factors (Hansen, 2003; Osborn & Zunker, 2016). We can see that normalising tests is going to be an expensive business, in giving 300 tests to measure each norm group.

We also need to have consistent test-retest rates: the same result needs to be achieved each time the test is run (Heale & Twycross, 2015). If our client does a test in March, we don't want to see that they obtain a completely different result when they repeat the test in July (one of the main bug-bears of MBTI; Mastrangelo, 2001). While it’s not possible to perfectly assess each career instrument, we can estimate their replicability (Heale & Twycross, 2015) through “internal [...] and test-retest reliability” (Osborn & Zunker, 2016, p. 37).

And, while we might have all reliability, validity and representative norm groups, we might still find that our client does not suit the test we propose. The client may complete the test and end up with results which make no sense. For example, each time I complete a RIASEC test, I get a different score. Over the years, I think I have seen a pattern: that in those of us with very generalist skills, the RIASEC test may lose it's test-retest reliability. I offer RIASEC here as one example: it is not the only one I have noticed. I have had clients who achieve poor results from HBDI, from MBTI, and from DiSC. All tests do not necessarily suit all people.

We must take all quantitative tests with a pinch of salt :-)

Sam

References:

Hansen, S. S. (2003). Career counselors as advocates and change agents for equality. The Career Development Quarterly, 52(1), 43-53. https://doi.org/10.1002/j.2161-0045.2003.tb00626.x

Heale, R., & Twycross, A. (2015). Validity and reliability in quantitative studies. Evidence Based Nursing, 18(3), 66-67. https://doi.org/10.1136/eb-2015-102129

Herr, E. A. (2001). Chapter 2: Career Assessment: Perspectives on trends and issues. In J. T. Kapes, E. A. Whitfield (Eds.), A counselor's guide to career assessment instruments (4th ed., pp. 15-26). National Career Development Association.

Mastrangelo, P. M. (2001). [251] Myers-Briggs Type Indicator [Form M]. In B. S. Plake & J. C. Impara (Eds.), The fourteenth mental measurements yearbook (816-820). Buros Center for Testing.

Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

Walsh, W. B., & Betz, N. E. (2000). Tests and Assessment (4th ed.). Prentice Hall.

Whitfield, E. A., Feller, R. W., & Wood, C. (Eds.). (2009). A counselor’s guide to career assessment instruments (5th ed., pp. 13–25). National Career Development Association.

read more "Making sense of testing"

Values and O*Net instruments evaluated by Buros

This is part 2 of an article published earlier this year (here), exploring commonly used tests which have been evaluated by the Buros Center for Testing, at the University of Nevada (Thorndike, 1999).

Following are all of the evaluations of O*Net and many of the values instruments completed by the Buros reviewers. As mentioned in the first half of this series, the number - 5th etc - stands for the Buros volume, and MMY stands for the title, which is the Mental Measurements Yearbook. The list of reviewed instruments is as follows (Buros Center for Testing, 2024):

• O*Net
  O*NET Career Interests Inventory, Third Edition: Based on the O*NET Interest Profiler, 20th MMY
  O*NET Career Interests Inventory: Based on the "O*NET Interest Profiler" developed by the U.S. Department of Labor, 16th MMY
  O*NET Career Values Inventory: Based on the "O*NET Work Importance Locator" developed by the U.S. Department of Labor, 16th MMY
  O*NET Interest Profiler, 16th MMY
  O*NET Work Importance Locator, 16th MMY
• Values Tests
  Values Inventory, 9th MMY
  Values Scale, Second Edition (The), 13th MMY
  Rokeach Value Survey, 12th MMY
  Career Values Card Sort, 13th MMY
  Career Beliefs Inventory, 12th MMY
  Work Values Inventory, 21st MMY
  Employment Values Inventory, 14th MMY
  Filipino Work Values Scale (The), 14th MMY
  Hall-Tonna Inventory of Values (The), 11th MMY
  Maferr Inventory of Feminine Values, 9th MMY
  Maferr Inventory of Masculine Values, 9th MMY
  Mirror Edition of the Personal Values Inventory, 16th MMY
  Motives, Values, Preferences Inventory, 14th MMY
  Personal Values Questionnaire, 13th MMY
  Rothwell Miller Values Blank, 14th MMY
  Selby MillSmith Values Indices, 14th MMY
  Survey of Personal Values, 10th MMY
  Survey of Work Values, Revised, Form U, 12th MMY
  Temperament and Values Inventory, 9th MMY
  Work Orientation and Values Survey, 17th MMY

I hope this is useful! 

Sam

References:

Buros Center for Testing (2024). Tests reviewed in The Mental Measurements Yearbook series. https://buros.org/tests-reviewed-mental-measurements-yearbook-series

Thorndike, R. M. (1999). Book Review: Conoley, JC, & Impara, JC (Eds.). (1995). The Twelfth Mental Measurements Yearbook. Lincoln, NB: The Buros Institute of Mental Measurements. Journal of Psychoeducational Assessment, 17(1), 50-55. https://doi.org/10.1177/073428299901700105

read more "Values and O*Net instruments evaluated by Buros"

Common tests evaluated by Buros

This is a topic I have covered before (here), but there is always more to say! I thought I would create a list of tests which are commonly used in Aotearoa, and have been evaluated by the Buros Center for Testing, at the University of Nevada (Thorndike, 1999).

The tests I have listed are those that my students have said they had previously used with clients when they began the "Assessment Tools" paper with me. The number - 5th etc - stands for the volume, and MMY stands for Mental Measurements Yearbook. The list is as follows (Buros Center for Testing, 2024):

• 16PF (Like MBTI)
  Sixteen Personality Factor Questionnaire, 9th MMY
  Sixteen Personality Factor Questionnaire, Fifth Edition, 12th MMY
• MBTI
  Myers-Briggs Type Indicator Step II (Form Q), 15th MMY
  Myers-Briggs Type Indicator, 10th MMY
  Myers-Briggs Type Indicator, 9th MMY
  Myers-Briggs Type Indicator, Form M, 14th MMY
  Myers-Briggs Type Indicator¨ Step III, 20th MMY
• Holland
  Self-Directed Search--Second Australian Edition, 15th MMY
  Self-Directed Search, 4th Edition [Forms R, E, and CP], 13th MMY
  Self-Directed Search, 4th Edition [Forms R, E, and CP], 14th MMY
  Self-Directed Search, 5th Edition [Form R], 20th MMY
• DiSC
  DiSC Classic, 16th MMY
  DiSC Indra, 16th MMY
• HBDI
  Herrmann Brain Dominance Instrument [Revised], 14th MMY
  Herrmann Brain Dominance Instrument, 11th MMY
• Schein's Career Anchors
  Career Anchors: Discovering Your Real Values, Revised Edition, 13th MMY
• Krumboltz's Career Beliefs
  Career Beliefs Inventory, 12th MMY
• Knowdell's Career Values Cards
  Career Values Card Sort, 13th MMY

If you already have a US Open Library account, you can review past Buros volumes here held at the Internet Archive for an hour at a time. The search in the left-hand sidebar allows us to find the tests we seek quite quickly: much faster than trawling through the book!

If you haven't yet opened an Open Library account, register here. This online service is a superb catalogue for finding texts long out of print, and this links seamlessly with the Internet Archive, which often stores the volumes.

This is part one of a two part article. In part 2, coming in a month or so, I will list inventories linked to O*Net and to other values tests. 

Sam

References:

Buros Center for Testing (2024). Tests reviewed in The Mental Measurements Yearbook series. https://buros.org/tests-reviewed-mental-measurements-yearbook-series

Thorndike, R. M. (1999). Book Review: Conoley, JC, & Impara, JC (Eds.). (1995). The Twelfth Mental Measurements Yearbook. Lincoln, NB: The Buros Institute of Mental Measurements. Journal of Psychoeducational Assessment, 17(1), 50-55. https://doi.org/10.1177/073428299901700105

read more "Common tests evaluated by Buros"

The Milgram studies

The Milgram studies - led by psychologist Stanley Milgram - were a series of post-WW2 experiments looking into how we obey those who have authority over us (Eldridge, 2023; Mcleod, 2023). The Milgram studies found that roughly two thirds of us - 65% - will obey an instruction from someone who looks as if they are in charge, if they accept responsibility for what is effectively our own action (Milgram, 1963).

To find this out, Milgram designed - and repeated - a number of experiments. In these studies, there were three men: an authority figure 'experimenter', a volunteer participant 'teacher', and a 'learner'. The 'experimenter' supervised 'teachers' in asking questions and then, when wrong answers were given by the 'learner', instructing 'teachers' to deliver electric shocks to the 'learner'. The aim stated to the 'teachers' was that the study was to find out if electric shocks could encourage learning (Eldridge, 2023; Mcleod, 2023). The volunteers consisted of "40 males, aged between 20 and 50, whose jobs ranged from unskilled to professional, from the New Haven area. They were paid $4.50 for just turning up" (Mcleod, 2023). 

Experiments took place in the Yale Interaction Laboratory (see image accompanying this post; Mcleod, 2023). The 'experimenter' was in a white lab coat, and the 'teacher' were on the other side of a clear window from the 'learner' (who was strapped into an electric chair), with an "electric shock generator and a row of switches marked from 15 volts (Slight Shock) to 375 volts (Danger: Severe Shock) to 450 volts (XXX)" (Mcleod, 2023). Each time the 'learner' gives an incorrect answer to a question, the 'teacher' is instructed by the 'experimenter' to deliver a shock. The shocks increase over time until they are into the danger zone.  

The 'teachers' believed they were actually giving shocks to the 'learners'. However, the 'learners' were in fact actors, and the shocks were faked (Eldridge, 2023; Mcleod, 2023). Today this type of experiment is considered unethical, as the data collection sessions lacked full disclosure, informed consent, involved deception, and lacked means for post-session debriefing to avoid the "trauma experienced by [some] teacher[ participants]" (Eldridge, 2023).

Why did some participants experience trauma? Because 65% had delivered what was a lethal electric shock - 450V - in the experiment (Milgram, 1963). They did not know at the time that the 'learner' was an actor. 100% of participants went to 300V (Mcleod, 2023; Milgram, 1963). 

The video below provides an overview of the Milgram experiment (BigHistoryNL, 2013):

It is strange how easy it can be for us to rationalise our behaviour into what works for us, instead of what is plainly inhumane. And any time we would like to think that we will always behave ethically, we can remember that two thirds of us will bow to the pressure of experts.

Sam

References:

BigHistoryNL (19 Mar 2013). Milgram Experiment - Big History NL, threshold 6 [video]. YouTube. https://youtu.be/xOYLCy5PVgM

Eldridge, S. (2023). Milgram Experiment. Britannica. https://www.britannica.com/science/Milgram-experiment

Mcleod, S. (2023, March 8). The Milgram Shock Experiment: Summary, Results, & Ethics. Simply Psychology. https://simplypsychology.org/milgram.html

Milgram, S. (1963). Behavioral Study of Obedience. The Journal of Abnormal and Social Psychology, 67(4), 371–378. https://doi.org/10.1037/h0040525

read more "The Milgram studies"

Exploring the NEO Personality Inventory-3 (UK Edition)

Our longer term characteristics - our personality - can be “defined as the relatively enduring patterns of thoughts, feelings, and behaviours that distinguish individuals from one another” (Roberts & Mroczek, 2008, p. 31). We can also consider personality as how our natural tendencies and inclinations differ from others within our own society, or as an “enduring set of Traits and Styles" where we exhibit certain "characteristics" (Bergner, 2019, p. 4).

Commonly used in the study and research of personality, the ‘Big Five’ or Five Factor Model (FFM) is made up of the five broad personality dimensions: extraversion; agreeableness; conscientiousness; neuroticism; and openness to experience. Developed from early research looking at how trait theory relates to individuals' temperament and behaviour (de Raad & Mlačič, 2015), research into the FFM has covered many populations and cultures and appears “to be the most widely accepted theory of personality today” (Lim, 2020). The FFM dimensions are structured into instruments to measure how individuals thinks, feels, and behaves, which collectively aids our understanding of personality difference (de Raad & Mlačič, 2015).

One of the instruments designed to test the FFM is the NEO Personality Inventory, or NEO PI. A number of research “studies in many different settings have verified the overall factor structure and construct validity of the Big Five [model...], based on many different demographic and cultural characteristics of individuals” as participants (Lounsbury, 2005, p. 709). Originally created in 1978 for use with adults, in the early stages it was clear that college students would also benefit from its use, but would require separate norms (McCrae et al., 2010). Later studies using samples as young as 10 years old showed that the revised version, NEO-PI-R, could be used but some items were difficult for younger respondents to understand. High school students - instructed to leave blank items not understood - found 30 of 240 test items difficult (McCrae et al., 2010). Using more current, colloquial language - although originally designed for adolescents - also improved the test for adult test-takers (McCrae et al., 2010). The latest version is the NEO-PI-3 (Lounsbury, 2005), and is a three-level self-report instrument, consisting of 240 items, a validity question, 30 facets, to test the five domains of “Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness” (Vassend & Skrondal, 2011, p. 1301). Individuals rate the 240 items on a 5-point Likert scales ranging from strongly disagree to strongly agree (Hattrup & Smith, 2021; Hey, 2022).

The UK version of the NEO-PI-3 consists of language and normative data more appropriate for use with British individuals aged 16 and up, although it should be noted that norm data is not available for those under 18 (Hattrup & Smith, 2021). The UK edition shows consistent reliability with the US version, showing retest reliability supporting the conclusion that it measures stable traits (Hattrup & Smith, 2021). Like the US version, the UK NEO-PI-3 can be administered and scored both online or in paper version, remotely or in-person. While the test itself may only take 30-40 minutes, it is suggested that an hour be allowed to brief the client, and for the client to make considered responses (Hattrup & Smith, 2021).

Research and testing from multiple sources indicate that the NEO-PI-3 is appropriate for many ages and stages in career development. But gender options are binary only (Kluck, 2014), which ignores - invalidates - those who don’t identify this way. In addition, like the US, the UK samples too are homogenous, with over 90% of participants identifying as Caucasian (Hattrup & Smith, 2021).

While it’s noted that UK norms resemble US data (Hattrup & Smith, 2021), it cannot be assumed that this necessarily translates to Aotearoa's super-diversity context (Chen, 2015). Practitioners must consider collective and individual culture (Laher, 2013), relevant when working with Māori and Pasifika kaimahi and ākonga. Further, considering Kiwis who like to get on with others, candidates may feel compelled to conform with societal 'expectations' when answering (Kumar, 2019). A step further on, some participants may fear negative ramifications if vulnerabilities or ‘flaws’ are exposed. However, there is a “Problems in Living Checklist” at the end of each report (Costa & McCrae, 2010) which is helpful in allaying client concerns.

Whether we decide to use the test or not, it is useful to explore the issues.

Alex

References:

Bergner, R. M. (2020). What is personality? Two myths and a definition. New Ideas in Psychology, 57, 100759, 1-7. https://doi.org/10.1016/j.newideapsych.2019.100759

Chen, M., (2015). Superdiversity Stocktake: Implications for business, government and New Zealand. Superdiversity Centre For Law, Policy And Business. https://www.superdiversity.org/wp-content/uploads/Superdiversity-Stocktake-Section1.pdf

Costa, P. T., & McCrae, R. R. (2010). NEO™ Personality Inventory-3: Interpretive Report. Australian Council for Education. https://www.acer.org/files/NEO_PI-3_Interp_Rpt_Sample_Report.pdf

de Raad, B., & Mlačič, B. (2015). Big Five Factor Model, Theory and Structure. In J. Wright, C. Fleck (Eds.), International Encyclopaedia of Social & Behavioural Sciences (2nd ed., Vol. 2, pp. 559-566). Elsevier. https://doi.org/10.1016/B978-0-08-097086-8.25066-6

Hattrup, K., & Smith, J. V. (2021). [101] NEO Personality lnventory-3 (UK Edition). In J. F. Carlson, K. F. Geisinger, & J. L. Jonson (Eds.) The Twenty First Mental Measurements Yearbook (pp. 450-455). The Buros Institute of Mental Measurements.

Hey, L. (2022). Presenting a new NEO-PI-3 International Senior Manager Norm for a post-covid-19 world. Hogrefe Ltd. https://www.hogrefe.com/uk/index.php?eID=dumpFile&t=f&f=10141&token=8296bcf0af59cbf9aa92753a20dce8f92057ad8f

Kluck, A. S. (2014). [116] NEO Personality lnventory-3. In J. F. Carlson, K. F. Geisinger, & J. L. Jonson (Eds.), The Nineteenth Mental Measurements Yearbook (pp. 477-483). The Buros Institute of Mental Measurements.

Laher, S. (2013). Understanding the Five-Factor Model and Five-Factor Theory through a South African cultural lens. South African Journal of Psychology, 43(2), 208–221. https://doi.org/10.1177/0081246313483522

Lim, A. G. Y. (2020). What Are the Big 5 Personality Traits? Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. Simply Psychology. https://www.simplypsychology.org/big-five-personality.html

Lounsbury, J. W., Saudargas, R. A., Gibson, L. W., Leong, F. T. (2005). An Investigation of Broad and Narrow Personality Traits in Relation to General and Domain-Specific Life Satisfaction of College Students. Research in Higher Education, 46(6), 707-729. https://doi.org/10.1007/s11162-004-4140-6

McCrae, R. R., Costa, P. T., Martin, T. A. (2010). The NEO-PI-3: A More Readable Revised NEO Personality Inventory. Journal of Personality Assessment, 84(3),  261-270. https://doi.org/10.1207/s15327752jpa8403_05

Roberts, B. W., Mroczek, D. (2008). Personality Trait Change in Adulthood. Current Directions in Psychological Science, 17(1), 31-35. https://doi.org/10.1111/j.1467-8721.2008.00543.x

Vassend, O., & Skrondal, A. (2011). The NEO personality inventory revised (NEO-PI-R): Exploring the measurement structure and variants of the five-factor model. Personality and Individual Differences, 50(8), 1300-1304. https://doi.org/10.1016/j.paid.2011.03.002

* Alex Howe has kindly prepared much of the material for this post

read more "Exploring the NEO Personality Inventory-3 (UK Edition)"

Personality and psychosocial maturity

A career development student, noting a number of test requirements for participants to be aged over 18 years, asked me when our clients become mature enough for testing. It is a really good question.

Firstly, what is personality? It is "the set of unseen characteristics and processes [which] underlie a relatively stable pattern of behaviour in response to ideas, objects, and people in the environment" (Daft, 2008, p. 98). So when our development seems to slow down and get more predictable. But how do we know when our personality is actually 'mature'? Well, we tend to use psychosocial maturity as a proxy for maturity, showing as three "capacities", or "general demands made by all societies on individuals. They are (1) the capacity to function effectively on one's own, or individual adequacy; (2) the capacity to interact adequately with others, or interpersonal adequacy; and (3) the capacity to contribute to social cohesion, or social adequacy (Greenberger et al., 1975, p. 128). So once we can be independent, conduct good interpersonal communication, and fit in, we are probably roughly mature. Roughly.

The field of psychology seemed to once 'assume' that our personality was relatively stable - so therefore mature - when we were aged around 25 years. But do our personalities roughly mature at 25? Well, maybe. 

The development and use of fMRI machines has finally allowed us a glimpse under the hood - so to speak - and we are now realising just how much we do not know about the brain. Some still put maturation as high as 30. For example, instead of being set like concrete our "personality is [...] like 'soft' plaster, [and] does change, albeit only marginally, beyond 30 and across the entire life cycle" (Boyce et al., 2013, p. 289). However, others put this earlier, as "young people may not reach levels of adult maturity until the age of 21 and are developing skills in this domain predominantly between the ages of 16 and 19" (Bryan-Hancock & Casey, 2010, p. 59). What is also quite interesting is that our levels of responsibility and "reasonably foreseeable"-ness changes: research has found that participants "in the 25-year-old group were significantly more inclined to take responsibility for their actions and view behaviours in the long term" (Bryan-Hancock & Casey, 2010, p. 66). I think we can give people tests, but must allow for this being an 'at the present time' test, rather like measuring height against a wall: that it is likely to change the next time we measure it. 

We still need to allow for some level of plasticity, of malleability, in the personalities of our younger career clients. This then will translate into what assessment instruments we may suggest our clients undertake, and how we interpret the results of any tests that are used. So, if we are working with young adults in secondary school or at university, we need to use not only our judgement of their maturity, but fully understand the test guidelines. 

We must ensure that we are professional in our recommendations, that we do not deliver any test which we are not specifically trained for, because we need to be open with our clients about our level of expertise. The CDANZ COE states that members will "Represent services, qualifications, and experience accurately and fully, and only undertake those practices for which they are qualified and in circumstances where they have appropriate experience" (2016).

The answer about being mature? It as you can see, it depends.

Sam

References:

Boyce, C. J., Wood, A. M., & Powdthavee, N. (2013). Is personality fixed? Personality changes as much as “variable” economic factors and more strongly predicts changes to life satisfaction. Social Indicators Research, 111(1), 287-305. https://doi.org/10.1007/s11205-012-0006-z

Bryan-Hancock, C., & Casey, S. (2010). Psychological Maturity of At-Risk Juveniles, Young Adults and Adults: Implications for the Justice System. Psychiatry, Psychology and Law, 17(1), 57-69. http://dx.doi.org/10.1080/13218710903268006

CDANZ (2016). Code of Ethics. Career Development Association of New Zealand. https://cdanz.org.nz/ModularPage?Action=View&ModularPage_id=26

Daft, R. L. (2008). The Leadership Experience (4th ed.). Thomson South-Western.

Greenberger, E., Josselson, R., Knerr, C., & Knerr, B. (1975). The measurement and structure of psychosocial maturity. Journal of Youth and Adolescence, 4(2), 127-143. https://doi.org/10.1007/BF01537437

read more "Personality and psychosocial maturity"

A and B Personality Types 2

In a recent post (here) I explored the antecedents for personality types A and B as based on Friedman and Rosenman's mid 20th century research (Benjamin, 2020). As was previously mentioned, there appears to be no apparent scientific grounding for the humourism theory, yet a number of personality assessments appear to use the humourism as their basis. I thought we would have a look at those assessments in this post.

Firstly, the diagram accompanying this post shows how humourist Choleric (Type A) intersects with the NF type in MBTI, the Keirsey Idealist (Berens, 2000, p. 2), the DISC D "Dominant", and the Enneagram Adventurer/Achiever (HireSuccess, 2022).

The humourist Sanguine (type B) aligns with SP types in MBTI, the Keirsey Artisan (Berens, 2000, p. 2), the DISC I "Inspiring", and the Enneagram Helper/Romantic (HireSuccess, 2022).

The humourist Phlegmatic (type C) fits with the NT MBTI types, the Keirsey Rational (Berens, 2000, p. 2), the DISC C "Conscientious", and the Enneagram Asserter/Perfectionist (HireSuccess, 2022).

The humourist Melancholic (type D) aligns with SJ types in MBTI, the Keirsey Guardian (Berens, 2000, p. 2), the DISC S "Supportive", and the Enneagram Peacemaker/Observer (HireSuccess, 2022). 

There is a further level of confusion in testing, with the language we use. A trait is "A replicable group of meaningfully labellable behaviours at approximately the same level of generality" (Howart, 1988, p. 175); and a state is "A transient labellable experience often of emotional connotation" (p. 175). But as Howart points out, "when is a trait a trait, and when is a state a state" in the inventories our clients take? Assessment results, and our clients, may be better served if test instruments deliberately asked "'how do you feel right now' for state, and 'how do you generally feel' for trait" (1988, p. 175). Not many questions - in my experience - are that overt. Personality - traits which are stable over time - instruments surely would be best meet intended outcomes if they asked about trait, not state.

Lastly, should we rely on such old structures in assessment work? It is useful to stop and consider just how reliable a 2,500 year old theory might be as an analysis tool; and how much validity it may have. 

And to look carefully at how questions are asked in those instruments.

Sam

References:

Benjamin, A. J. (2020). Chapter 59: Type A/B Personalities. In B. J. Carducci & C. S. Nave (Eds), The Wiley Encyclopedia of Personality and Individual Differences: Clinical, Applied, and Cross‐Cultural Research (Volume IV, 1st ed., pp. 383-386). John Wiley & Sons Ltd.

Berens, L. V. (2000). Understanding yourself and others: an introduction to temperament (2nd ed.). Telos Publications.

HireSuccess. (2022). Understanding the 4 Personality Types: A, B, C, and D. https://www.hiresuccess.com/help/understanding-the-4-personality-types

Howart, E. (1988). Mood differences between the four Galen personality types: Choleric, Sanguine, Phlegmatic, Melancholic. Personality and Individual Differences, 9(1), 173-175. https://doi.org/10.1016/0191-8869(88)90044-X

read more "A and B Personality Types 2"

A and B Personality Types 1

Lately I have been wondering about the basis for personality types, and I have been running down a few rabbit holes to try and track down the research underpinnings of "Type A" personalities. I am sure that most of us, when prompted, will remember that "Type A" people are more likely to have heart attacks, right?

Well, wrong, apparently (Benjamin, 2020; Shekelle et al., 1985). But before we look at that, let's define some terms. The "Type A/B Behavior Pattern construct [...] was first introduced by cardiologists Friedman and Rosenman" in 1959 (Benjamin, 2020, p. 383). Type A personalities had three main characteristics: those of "extreme competitiveness", "a strong sense of time urgency", and "hostility" (Benjamin, 2020, p. 383). Basically, driven, impatient and angry; or as Van Dihl phrases it, displaying "aggressivity" (1978, p. 101). Type B people are at the other end of these continua: "characterized by relative absence of drive, ambition, sense of urgency, desire to compete or be involved in deadlines" (Van Dihl, 1978, p. 101). Basically coasting, friendly, hands-off, laissez faire (Lewin et al., 1939) people.

There were a lot of papers written by Friedman and Rosenman into the apparent causality of personality type A and coronary heart disease (1959, 1960, 1974; Friedman, 1977; Rosenman et al, 1964), which turned out to be spurious (Benjamin, 2020; Shekelle et al., 1985). Even more interestingly, Friedman and Rosenman's research was extremely well-funded by the William Morris tobacco empire. Ouch. Type A personality as an alternative CHD factor, taking the heat from tobacco...?

Where it gets really interesting is that these "Type A" tendencies were apparently noted by a secretary in Friedman and Rosenman's cardiology practice. Rosenman himself was interviewed, explaining that:

"A discerning secretary in our office practice told us that in contrast to our other patients, those with coronary disease were rarely late for appointments and preferred to sit in hard-upholstered chairs rather than softer ones or sofas. These chairs also had to be reupholstered far more often than others because the front edges quickly became worn out. They looked at their watches frequently and acted impatient when they had to wait, usually sat on the edges of waiting room chairs and tended to leap up when called to be examined" (Rosch, 2004, section 4, para 3).

So... a receptionist noted that some people really were on the edge of their seats, and that led to the "Type A" classification? Well, not quite. Because this "Type A" personality type seem to have an earlier - but uncited in the research of Friedman and Rosenman - connection to some ancient Greeks, and some earlier 20th century researchers (Rosch, 2004, 2017).

Firstly, Hippocrates of Kos (c. 460 - c. 375 BCE). The credited founder of the field of modern medicine and the Koan school of Humourism. Hippocrates built on the work of an earlier Greek (Alcmaeon of Croton, c. 540–500 BCE) codifying and applying Alcmaeon's philosophy of humourism to the new field of medicine (Britannica, 2022). Humourism is most associated with the four 'humours' or the fluids needing balance for us to stay well: Flegmat/phlegm; Sanguin/blood; Coleric/yellow bile; and Melanc/black bile (Britannica, 2022; note the 'Coleric' as we will come back to this). So we have a 2500 year old theory which lasted until germ theory in the 1850s (Britannica, 2022).

Then enter Galen of Pergamon (129 – c. 216 CE). He is credited with applying Hippocrates work to the personality of patients, and interestingly being the founder of the dietetics field (Grant, 2000). Here is where the Phlegmatic, the Sanguine, the Choleric, and the Melancholic personalities, and their resulting moods, begin to translate into the "modern" personality types we know: C; B; A; and D (HireSuccess, 2022).

Alcmaeon, Hippocrates and Galen's work was used largely unchanged by a number of European physicians: Harvey in the 1600s, Corvisart in the 1700s, Von Duesch in the 1800s, and Osler into the early 1900s (Rosch, 2017). Similar to Rosenman's interview comments, Osler noted that "diagnosis of angina [could be] based on the appearance, demeanor and mannerisms of the patient in the waiting room and how he entered the consultation room" (Rosch, 2017, para 3). Rosch further outlines:

That "In the 1930s, the Menningers suggested that coronary heart patients tended to be very aggressive [..., and] Flanders Dunbar, who introduced the term “psychosomatic” into American medicine, characterized the coronary prone individual as being authoritarian with an intense drive to achieve unrealistic goals. [...] Kemple also emphasized fierce ambition and a compulsive striving to achieve power and prestige" (Rosch, 2017, para 3).

All this took place prior to Friedman and Rosenman's work. We always stand on the shoulders of giants in research (but good academic writing requires us to politely acknowledge who those giants are). 

Anyway: welcome to the "Type A" personality - the Choleric; Type B - the Sanguine; Type C - the Phlegmatic; and Type D - the Melancholic. These personality types are 2,500 year old. There is no apparent scientific grounding for the humourism theory, yet many, many personality assessments use the humourism as their basis. Exploring this further will be the subject of a future post.  

There is, interestingly, an alignment between depression, anxiety and Type D personalities (Melancholic) and heart disease (Rosch, 2017). The current thinking is that stress hormones may be a causal player for CHD and stroke.

Sam

References:

Benjamin, A. J. (2020). Chapter 59: Type A/B Personalities. In B. J. Carducci & C. S. Nave (Eds), The Wiley Encyclopedia of Personality and Individual Differences: Clinical, Applied, and Cross‐Cultural Research (Volume IV, 1st ed., pp. 383-386). John Wiley & Sons Ltd.

Bortner, R. W. (1969). A short rating scale as a potential measure of pattern A behaviour. Journal of Chronic Diseases, 22(2) 87-91. https://doi.org/10.1016/0021-9681(69)90061-7

Britannica. (2022). Hippocrates: Greek physician. https://www.britannica.com/biography/Hippocrates

De Raad, B. E., & Perugini, M. E. (2002). Big five factor assessment: Introduction. In Big Five Assessment (pp. 1-26). Hogrefe & Huber Publishers.

Edwards, J. R., Jr, A. B., & Cooper, C. L. (1990). The psychometric properties of the Bortner Type A Scale. British Journal of Psychology, 81(3), 315-333. https://doi.org/10.1111/j.2044-8295.1990.tb02364.x

Friedman, M. (1977). Type A behavior pattern: some of its pathophysiological components. Bulletin of the New York Academy of Medicine, 53(7), 593-604. https://doi.org/10.1111/j.1749-6632.1977.tb29381.x

Friedman, M., & Rosenman, R. H. (1959). Association of Specific Overt Behavior Pattern with Blood and Cardiovascular Findings. Journal of the American Medical Association JAMA, 169(12), 1286-1296. https://doi.org/10.1001/jama.1959.03000290012005

Friedman, M., & Rosenman, R. H. (1960). Overt Behaviour Pattern in Coronary Disease: Detection of Overt Behaviour Pattern A in Patients With Coronary Disease by New Psychophysiological Procedure. Journal of the American Medical Association JAMA, 173(12), 1320-1325. https://doi.org/10.1001/jama.1960.03020300032011

Friedman, M., & Rosenman, R. H. (1974). Type A behavior and your heart. Knopf.

Grant, M. (2000). Galen on Food and Diet. Routledge.

HireSuccess. (2022). Understanding the 4 Personality Types: A, B, C, and D. https://www.hiresuccess.com/help/understanding-the-4-personality-types

Jenkins, C. D., Zyanski, S. J., & Rosenman, R. H. (1965). Jenkins Activity Survey [Form C]. The Psychological Corporation.

Katahira, K., Kunisato, Y., Yamashita, Y., & Suzuki, S. (2020). Commentary: A robust data-driven approach identifies four personality types across four large data sets. Frontiers in Big Data, 3, 8. https://doi.org/10.3389/fdata.2020.00008

Kluck, A. S. (2014). [116] NEO Personality lnventory-3. In J. F. Carlson, K. F. Geisinger, & J. L. Jonson (Eds.). The Nineteenth Mental Measurements Yearbook (pp. 477-483). The Buros Institute of Mental Measurements.

Lewin, K., Lippitt, R., & White, R. K. (1939). Patterns of aggressive behavior in experimentally created “social climates”. The Journal of Social Psychology, 10(2), 269-299. https://doi.org/10.1080/00224545.1939.9713366

McCrae, R. R., & Costa Jr, P. T. (2007). Brief versions of the NEO-PI-3. Journal of Individual Differences, 28(3), 116-128. https://doi.org/10.1027/1614-0001.28.3.116

Paunonen, S. V., & Jackson, D. N. (2000). What is beyond the big five? Plenty!. Journal of Personality, 68(5), 821-835. https://doi.org/10.1111/1467-6494.00117

Piedmont, R. L. (1998). The Revised NEO Personality Inventory: Clinical and research applications. Springer Science+Business Media

Rosenman, R. H., Friedman, M., Straus, R., Wurm, M., Kositchek, R., Hahn, W., & Werthessen, N. T. (1964). A predictive study of coronary heart disease: The Western Collaborative Group Study. Journal of the American Medical Association JAMA, 189(1), 15-22. https://doi.org/10.1001/jama.1964.03070010021004

Rosch, P. J. (2004). Separating Fact From Fiction: An Interview with Ray H. Rosenman, M.D [interview transcript]. The American Institute of Stress. https://www.stress.org/type-a-and-coronary-disease-part-1

Rosch, P. J. (2017). Stress and Heart Disease. The American Institute of Stress. https://www.stress.org/stress-and-heart-disease

Shekelle, R. B., Gale, M., & Norusis, M. (1985). Type A score (Jenkins Activity Survey) and risk of recurrent coronary heart disease in the aspirin myocardial infarction study. The American Journal of Cardiology, 56(4), 221–225. https://doi.org/10.1016/0002-9149(85)90838-0

Smith, T. W., Houston, B. K., & Zurawski, R. M. (1983). The Framingham Type A Scale and anxiety, irrational beliefs, and self-control. Journal of Human Stress, 9(2), 32-37. https://doi.org/10.1080/0097840X.1983.9936122

Thomas, K. (1990). Chapter 8: Dimensions of personality. In I. Roth (Ed.) Introduction to Psychology (Vol 1, pp. 373-416). Psychology Press.

Van Dijl, H. (1978). The A/B typology according to Friedman and Roseman and an effort to test some of the characteristics by means of a psychological test (RSL or BUL). Journal of Psychosomatic Research, 22(2), 101-109. https://doi.org/10.1016/0022-3999(78)90035-1

Wilhelmsen, I. U. (1984). Assessment of type A behaviour by the Bortner scale and ischaemic heart disease. European Heart Journal, 5(6), 440–446. https://doi.org/10.1093/oxfordjournals.eurheartj.a061689

read more "A and B Personality Types 1"

Buros Center for Testing

I think most of us working in the Career Development field would know of the Buros Center for Testing, which - since the founder's death (Oscar Buros is Dead, 1978) - has been based at the University of Nevada (Thorndike, 1990). What is great is that we can find out if tests we are proposing to use have been reviewed by Buros (here; 2023a).

There is also a list of the triennial publication, the Mental Measurements Yearbook (often known as "MMY"), here (Buros, 2023c). At that link we can find out which test has been published in which edition. The latest issue is the 21st from 2021, presumably with the 22nd edition due out in 2024.

However, in Aotearoa New Zealand, the texts cost around NZD$350 each landed, so are quite pricey. I tend to buy these second hand via AbeBooks (here), once the cost has come down, and I can get a copy into the country for a more reasonable price. If there is a test I want to see in between time, I can pay USD$15 to purchase a copy of the review directly (here; Buros, 2023b). Additionally, if we still have library access at our Alumni Library, It can be worth contacting them to see if they can supply us a scan of a particular test. For very old test reviews, we can also go to the Internet Archive (here).

The process used by Buros is as follows:

"(1) identifying tests to be reviewed, (2) obtaining tests and preparing test descriptions, (3) determining whether tests meet review criteria, (4) identifying appropriate reviewers, (5) selecting reviewers, (6) sending instructions and materials to reviewers, (7) checking reviews for factual accuracy, (8) editing content of reviews, (9) copy-editing and updating reviews, (10) obtaining reviewer approval, and (11) seeking comments from publishers" (Plake & Conoley, 1995, p. 122).

Each test review averages between one and three pages, and usually supplies the "black" information following, with the "grey" being supplied where possible (Thorndike, 1999, p. 51):

1. The name of the test.
2. The author’s stated purpose for the test, usually taken from a statement in the test manual.
3. The population for which the author considers the test appropriate.
4. The date(s) of publication (first and most recent editions).
5. The common acronym for the test.
6. The scores yielded by the test.
7. Method of administration (group or individual).
8. Price data, including the date for which the figures apply.
9. Time required for administration.
10. Author of the test.
11. Publisher of the test. (Addresses for all publishers of tests included in this volume are listed in the Publisher Index.)
12. Cross references to reviews and entries in other MMY publications.
13. References to the test found by Buros Institute staff.
14. The name and affiliation of the first reviewer.
15. The first review.
16. References supplied by the first reviewer.
17. The name, affiliation, review, and references of the second reviewer.

Buros relies on two key constraints to test: that experienced reviewers will volunteer their time to do the mahi; and that the organisation producing the test will have the gumption to send it into the Buros Center when they call for it (Plake & Conoley, 1995), so it can be independently reviewed.

"Presenting their tests for review is an important part of a test publisher’s obligation to the professional community. Reputable test publishers should welcome critical professional reviews of their products and should send copies of all relevant materials to the Buros Institute as soon after publication as possible. A favorable review in the Mental Measurements Yearbook should be seen as something worth including in test publicity, something to be sought and highly prized. If a test is more than 2 or 3 years old, prospective test users should expect to see MMY reviews and view with extreme caution any test for which the publisher cannot produce them" (Thorndike, 1999, p. 52).

I too would think that the independent review would be a major selling point, but each triennial edition contains a significant list of 'tests called for', yet not included in the reviews.

Sam

References:

Buros Center for Testing. (2023a). Tests reviewed in The Mental Measurements Yearbook series. https://buros.org/tests-reviewed-mental-measurements-yearbook-series

Buros Center for Testing. (2023b). Test Reviews Online. https://marketplace.unl.edu/buros/

Buros Center for Testing. (2023c). The Mental Measurements Yearbooks. https://buros.org/mental-measurements-yearbook

Carlson, J. F., & Geisinger, K. F. (2012). Test reviewing at the Buros Center for Testing. International Journal of Testing, 12(2), 122-135. http://dx.doi.org/10.1080/15305058.2012.661003

Internet Library. (2023). Search "Mental Models Yearbook" . https://archive.org/search.php?query=%22Mental+Measurements+Yearbook%22&sin=

Oscar Buros is Dead, Rutgers Professor. (21 March 1978). The New York Times, p. 38. https://www.nytimes.com/1978/03/21/archives/oscar-buros-is-dead-rutgers-professor-expert-on-psychological.html

Plake, B. S., & Conoley, J. C. (1995). Using Buros Institute of Mental Measurements Materials in Counseling and Therapy [call number: ED391987 1995-00-00]. https://ia801306.us.archive.org/15/items/ERIC_ED391987/ERIC_ED391987.pdf

Thorndike, R. M. (1999). Book Review: Conoley, JC, & Impara, JC (Eds.). (1995). The Twelfth Mental Measurements Yearbook. Lincoln, NB: The Buros Institute of Mental Measurements. Journal of Psychoeducational Assessment, 17(1), 50-55. https://doi.org/10.1177/073428299901700105

read more "Buros Center for Testing"

Total labour cost revenue percent

Earlier this year I watched the documentary, Persona. There is a point in the movie where Ben Dattner, an organisational psychologist and executive coach, is interviewed. He justifies the recruitment testing industry with the rationale that "approximately 70 percent of an organisation's expenditures is on human capital" (Hawkins, 2021, 14:18), apparently implying that any strategy to reduce the organisation's exposure to such a significant cost as labour is a no-brainer. My immediate thought was: bollocks.

There was no way that the total cost of labour could be anywhere near 70%. Perhaps if you had a service industry it could be, but only if it was low tech (i.e. required no capital investment that the business would need to fund from earnings). So I went looking online. There is indeed a white paper stating that organisations tended to have "total human capital costs, or total cost of workforce, [at] nearly 70% of operating expenses" (HCMI, 2022). I found the 'technical' term for this, which is "total labour cost revenue percent".

Hmm. To calculate our cost of labour, we need to add all remuneration per staff member (car, insurance, retirement, housing etc), and to include all staff who are overhead (i.e. not directly responsible for production) such as managers, administrators and directors. Costs of hiring can also go into this bucket. But the cost of clothing, health and safety planning, scheduling etc should be part of infrastructure costs, not part of the cost of labour (Stone, 2019). Most of the firms I have had experience with would have had the cost of labour well down the expenses list. For example, a local manufacturing firm's main monthly cost is raw materials, followed by $1m in electricity. Total labour (including fringe benefits) is 11th down the list of costs at about 10% of the monthly total.

My 'instinctive' reaction to the 70% comment - also based on my initial accounting degree - was to have placed labour at around 20% of operational costs. However, I may be underestimating, as "[t]ypically, labor cost percentages average 20 to 35 percent of gross sales. Appropriate percentages vary by industry, A service business might have an employee percentage of 50 percent or more, but a manufacturer will usually need to keep the figure under 30 percent" (Johnson, 2011).

Organisations are keen to automate as much as possible to drive down the cost of labour. It is unlikely that labour costs will ever increase. And - while it will vary across sectors - it will not be 70% of costs.

However, aside from the extremely dubious claim made by interviewee Ben Drattner, the movie Persona (Hawkins, 2021) is a very interesting watch.

I firmly recommend it!

Sam

References:

Hawkins, T. T. (Director). (2021). Persona: The Dark Truth Behind Personality Tests [movie]. HBOMax.

HCMI. (2022). Total Cost of Workforce (TCOW) [whitepaper]. Human Capital Management Institute. https://www.hcmi.co/Resources/White-Papers/Total-Cost-of-Workforce#:~:text=Including%20the%20Fortune%20500%E2%84%A2,nearly%2070%25%20of%20operating%20expenses.

Johnson, K. (31 July 2011).>Employee Sales vs. Cost Ratio. Chron. https://smallbusiness.chron.com/employee-sales-vs-cost-ratio-38936.html

Stone, D. (2019). Estimating, Labor Burden, and Cost of Goods Sold. Construction Program & Results. https://www.markupandprofit.com/articles/estimating-labor-burden-and-cost-of-goods-sold/

read more "Total labour cost revenue percent"

Evidence and MBTI

Last year I read Merv Emre's book on the Myers-Briggs Type Indicator (2018), which was a fascinating read. I had expected a clear-eyed exploration of the type indicator, and I think the resulting book was fair in its approach to the story of the development of the MBTI tool. Ms Emre's book delivered more than a hint of admiration for the doggedness and drive of the tool's founders: Katharine Briggs, and her daughter, Isabel Briggs-Myers (2018).

This book enlightened me in a number of ways. For example, I had not realised the considerable amount of work undertaken between 1957 to 1975 from the Educational Testing Service (ETS) in attempting to validate the MBTI tool, which was pretty much an epic fail. It was only after ETS had given up on validation and returned the rights to Katharine Briggs - who by that time had dementia - and her daughter Isobel Myers-Briggs that the tool actually took off. The rise and rise of MBTI was via a deal with Consulting Psychologists Press (CPP) in California, which still markets the tool (Emre, 2018).

I found the chapters on the issues with validation to be very enlightening. There were clearly issues with replicability, construct and content validity, and generalisability. Isabel Myers-Briggs was reported as having closely held the process of scoring the returned, completed test scripts, appearing reluctant to allow computer scoring of data ...which was likely to have been more independent and objective (Emre, 2018). Other sources also mention issues of validity: MBTI's "ipsative construction of the questions, lack of criterion-related validity, and its tendency to be a 'feel good' instrument" are common complaints (Wood & Hay, 2013, p. 54). Test re-test replicability is as low as 50%: half those who have taken the test will fail to get the same result when retaken after three months (Emre, 2018; McCaully & Moody, 2007; Menand, 2018).

Despite my management and career background - or perhaps because of it! - I am a personality instrument skeptic. I am happy to use a personality instrument as a starting point for a career conversation: but not as an end point to categorise people; to put them in a box; to confine them.

Further, I like to have sound evidence of reliability, validity and generalisability, which many tests cannot provide; including MBTI (Mastrangelo, 2001). For me, my MBTI instrument type test re-test has always been consistent - unlike RIASEC, where my test-re-test is consistently inconsistent (I suspect that my interests are too broad to return a meaningful RIASEC score). What is interesting though is that - in light of Ms Emre's book (2018) - when I reread my MBTI ENTJ type profile, the actual type wording now seems to possess the vagueness of a horoscope:

"The ENTJ personality type is a competitive, highly motivated and focused person who sees just about everything by focusing on the bigger picture. ENTJs thrive by setting long-term goals and making highly analytical decisions, and they often do well in high-stress leadership roles. ENTJ types tend to see things in black and white, or by the numbers. In personal relationships they are fair, measured, and supportive" (MBTI Online, 2021).

So... hang on a minute. Competitive AND highly analytical AND fair AND supportive AND black and white AND big picture... right. Some of these things are not like the others.

What is particularly troubling is that MBTI has grown so popular that it is now blindly being used for making decisions in organisations; to determine who manages; who is in the team; who is hired; and who is let go (Macabasco, 2021). Additionally, MBTI also appears to be being used the classroom like the debunked learning styles (here) to categorise how different students learn (Emre, 2018, 2020; Harel, 2021; Macabasco, 2021).

We should not be making such decisions without good quality evidence. And - in my view - MBTI does not yet provide good enough quality evidence.

An HBO documentary documentary, "Persona: The Dark Truth Behind Personality Tests", was released on HBO last year (Macabasco, 2021). While I have not yet seen this film, it apparently explores the pervasiveness of personality testing use in organisational HR decisions. I can't wait to see it, but I am sure it will be somewhat depressing viewing. Ah well.

Sam

References:

Emre, M. (2018). The Personality Brokers: The Strange History of Myers-Briggs and the Birth of Personality Testing. Penguin Random House.

Emre, M. (23 December 2020). TED-Ed: Do personality tests work? [video]. https://youtu.be/lN7Fmt1i5TI

Harel, A. (29 April 2021). The Problems With Using Personality Tests For Hiring. Vervoe. https://vervoe.com/personality-tests-hiring/

HBO Max (23 February 2021). Persona | Official Trailer [video]. https://youtu.be/XWBXniurrA0

Macabasco, L. W. (4 March 2021). 'They become dangerous tools': the dark side of personality tests. The Guardian. https://www.theguardian.com/tv-and-radio/2021/mar/03/they-become-dangerous-tools-the-dark-side-of-personality-tests

Mastrangelo, P. M. (2001). Myers-Briggs Type Indicator [Form M]. In B. S. Plake & J. C. Impara (Eds.) The Fourteenth Mental Measurements Yearbook (pp. 818-820). Buros Center for Testing.

MBTI Online (2021). ENTJ. https://www.mbtionline.com/en-US/MBTI-Types/ENTJ

McCaully, M. H. & Moody, R. A. (2007) Multicultural applications of the Myers-Briggs Type Indicator, in L. A. Suzuki:, J. G. Ponterotto, & P. J. Meller (Eds.), Handbook of Multicultural Assessment: Clinical, psychological, and educational applications (3rd ed., pp. 402-424). Jossey-Bass.

Menand, L. (10 September 2018). What Personality Tests Really Deliver. The New Yorker. https://www.newyorker.com/magazine/2018/09/10/what-personality-tests-really-deliver

read more "Evidence and MBTI"

Some career assessment resources

If we take qualitative assessment instruments to mean those “methods [which] are flexible, open-ended, holistic, and nonstatistical” (Goldman, 1992, p. 616), then logically, quantitative methods should be those which are structured, close-ended, focused, and use statistical methods. Testing should be standardised, valitated, and replicable. However, testing is also largely normed for US populations (Osborn & Zunker, 2016). A review of Buros shows that there are very few tests which have actually been normed for other nations, let alone for the smaller population groups within those nations (Spies et al., 2010).

While I appreciate the cost in developing career assessment instruments, It can be difficult to find sound quantitative testing methods - and their base theories - which aren't hidden behind a pay wall. Fortunately, the following two pages of testing resources are kindly supplied and maintained by US academic Paul Spector, of the University of South Florida. They should be of interest to anyone in career practice:

• Assessment Archive List
• Personality Test Battery list

Paul says on his page: "Looking for measures for a study? I have created two free assessment archives with links to dozens of organizational and nonorganizational measures. Almost all are free to use for noncommercial (educational/research) purposes. Where possible, I linked to the article itself that contains the measure, mostly on ResearchGate". These resources provide "organizational measures of attitudes, behavior, environment, leadership, occupational health/safety", plus "general measures of mental and physical health, health behavior, positive well-being, personality".

Paul can be emailed at pspector@usf.edu.

Sam

References

• Goldman, L. (1992). Qualitative assessment: An approach for counselors. Journal of Counseling and Development, 70(5), 616–621. http://dx.doi.org/10.1002/j.1556-6676.1992.tb01671.x
• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.
• Spector, P. (2021). Assessment Archive. https://paulspector.com/assessments/assessment-archive/
• Spector, P. (2021). Personality. https://www.stevenericspector.com/mental-health-assessment-archive/personality/
• Spies, R. A., Carlson, J. F., & Geisinger, K. F. (Eds) (2010). The Eighteenth Mental Measurements Yearbook. The Buros Institute of Mental Measurements.

read more "Some career assessment resources"

Career assessment blind areas

While in career practice we get clients to undertake a number of career assessments, we sometimes forget to remind ourselves of the limitations of those assessments.

For example, a key limitation in personality testing is that most tests are self-reporting. We may see ourselves clearly - but it is most likely that we do not see 'all' of ourselves. If we consider the JoHari window - now there is an old theory! - we know that there are parts of ourselves that we are blind to (Luft, 1963; Luft & Ingham, 1955).

If we consider Joseph Luft and Harrison Ingham's model (1955), we can see that, of the four windows, only one is evident to ourselves. One area - the 'blind area' is only evident to others. There other two areas - the area we avoid, and the area of completely unknown activity - can be seen by no one. This can become a serious limitation in assessments.

As we age, we get to know ourselves better. We know what others say about us, so we make the yellow cell larger, making the blind area smaller. We may also, if we are reflective enough, push the avoided area back also. In doing those two things, we manage to reduce the area of unknown activity at the same time.

In addition, our training may well affect how we see ourselves over time, which will change our self-reported results. While we like to think that our work does not affect our personality, if we considered five different professions - perhaps a police detective, a doctor, an engineer, an accountant, and an career practitioner - we are likely to get five different focuses. They would all be valid, although biased due to the nature of our profession, our training and our experience.

What might those five professions look for? My biases suggest the following:

1. Police detective: alert to potential criminal activity. Looking for falsehood. Seeking gaps in stories.
2. Doctor: seeking symptoms of disease. Assuming the client will not/cannot be honest (alcohol reporting for example: doctors automatically double what they are told). May not seek symptoms of health, though some do
3. Engineer: seeking evidence of systems problems to take corrective action. Assumes the client knows too little to provide quality information.
4. Accountant: evaluating past evidence of financial health. Focusing on historic information. Not necessarily developing future financial health, though some now do
5. Career Practitioner: reflecting back to our client what they tell us to check the validity of what the client is seeking. Not telling, but asking. Mirroring client truth to assist clients to make their own decisions.

It is easy to see that assessment self-reporting may be skewed, based on our professional shaping. Accountants may be blind to the future. Engineers and doctors may focus on what is going wrong, not what is going well. Detectives may be blind to truth-telling. Career practitioners may be poor decision-makers. Although I know plenty of people in 3, 4 and 5, it would be good to talk to people who work in professions 1 and 2 to see if my take has much 'professional' accuracy.

Interesting.

Sam

References:

• Luft, J. (1963). Group processes: An introduction to group dynamics. National Press.
• Luft, J., & Ingham, H. (1955). The Johari Window: A Graphic Model for Interpersonal Relations. Western Training Laboratory in Group Development. University of California at Los Angeles, Extension Office.
• Luft, J., & Ingham, H. (1961). The Johari Window: a graphic model of awareness in interpersonal relations. Human Relations Training News, 5(1), 6-7.

read more "Career assessment blind areas"