The four uses of career assessment

Although those of us in career practice should know when assessment is useful, I don't think we often stop to remind ourselves of just when that is. However, Williamson (1939, 1949) provides us with four key uses, which Osborn & Zunker outline as follows (2016):

Diagnostic. Diagnostic tests define something, which can be helpful as it helps the practitioner to better understand the client's world. Providing we use diagnostic tests as a place to start from, to raise awareness, to explore, to brainstorm ideas, and not as a limiter, or a pigeon-hole, diagnostic tests can be very useful.

Predictive. These tests are used to predict performance: either academic, aptitude, or specific achievement levels or tasks. A personal assistant may be asked to complete a TOSA word assessment to ensure that they have the skills required for a particular role, or a pilot candidate may complete reaction tests ("assess the present levels of developed abilities", Osborn & Zunker, 2016, p. 17).

Comparative. This is where the client's results are compared to a norm for a particular group. This might be as simple as examining secondary school grades with the requirements to get into Medical School or into Engineering, or having a group of career students all complete an assessment together, and comparing their individual results with the class as a whole.

Developmental. This is where, according to Osborn and Zunker, "Meaningful assessment during all phases of career development [may involve] the diagnostic, predictive, and comparative use of assessment results" (2016, p. 17). "For the older adult, measured interests and leisure activities, skills needed in part-time or volunteer work, and assessment of established values are relevant developmental uses of assessment results" (Osborn & Zunker 2016, p. 17).

I hope these help to remind us that assessment has its place in career development.


Sam

References:

• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.
• Williamson, E. G. (1939). How to counsel students: A manual of techniques for clinical counselors. McGraw-Hill.
• Williamson, E. G. (1949). Counseling adolescents. McGraw-Hill.
  

read more "The four uses of career assessment"

A framework for CACGs

Osborn & Zunker (2016) propose a framework for use in when delivering online career guidance. Their framework is for use after the client has interacted with an inventory, and - while it is very simple - it asks good questions:

"Ask the client about their interaction with the system. What was their experience, their reactions to the way the system work, and their satisfaction with what they found?

"Have the client read through the results and highlight descriptors and/or occupations that are of particular interest.

"Ask the client to talk about how the descriptors relate to what [they] already knew about [themselves], what new knowledge was found, and how that relates to the career decision being made.

"Move on to the list of highlighted occupations, and have the client describe what is appealing about each of those occupations. The counselor may choose to take notes or write on the report beside each of the highlighted options.

"Compare the reasons given for liking specific occupations to determine themes and further clarify self-knowledge, e.g., 'It's really important to you that your career choice offers you opportunities for advancement and independence.'

"Determine next steps. Review the model for using assessments, and the purpose for using this particular assessment. Was the stated need met completely, partially or not at all? Were additional needs discovered (e.g., a need for information about training opportunities or scholarships) during the process? Is a different assessment or a follow-up activity on the system warranted? What will help the client move towards making the career decision?" (p.152).

Considering this type of framework can help us make more use of our sessions together.


Sam

• Reference: Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

read more "A framework for CACGs"

Ethical considerations for online career development

There are some ethical issues we need to consider with online career practice:

Quality. We need to ensure that all the materials we use are of good quality. We need to ensure the validity of what we use meets "the same set of standards used for other psychometric measures. For instance, the validity of scoring standardized instruments includes the weighting of items into scales and assurance of error-free scoring. However, errors in these two processes are difficult to identify in computer-based assessment. Thus, career service providers might not be aware of potential errors and subsequent misleading results" (Osborn & Zunker, 2016, p. 150).

Evaluation. We need to ensure that we have evaluated the validity by obtaining test validity evidence; or that we let the client know that the test cannot be evaluated for quality.

Reliability. As CACGs can be delivered globally, we need to consider whether items have been normalised for local populations: those of us outside the USA are unlikely to have "high internal consistency" (Osborn & Zunker, 2016, p. 150). If tests have not been normalised for our populations, we need to advise the client of this, and have a discussion about the results and the likely meaning as applied to our own context.

Unfamiliar tests. As practitioners we need to be able to discuss any test that our clients come to us with. If it is a test that we are unfamiliar with, we can use the structures introduced in Week 2 and 3 to help us learn about a new inventory, to attempt to establish validity, reliability and to find evidence of population normalisation. Only then can we discuss the test within the boundaries of our own practice: or refer the client to another practitioner who does have experience with that particular instrument.

Standards. We need to ensure that tests our clients do provide "Evidence of valid testing standards [which is] clearly delineated in promotional materials of CACGs and online assessments as well as in professional manuals" (Osborn & Zunker, 2016, p. 150).

Security. Advise clients of the potential for security breaches during online test use, and suggest more secure alternatives wherever possible. Remember that Cambridge Analytica harvested some 64,000 Facebook profiles by using quizzes which users had to agree to allow the test access to their Facebook profile (Iles, 2018). A more secure practice is to get students to create an additional email account to use only for test results, with minimal profile information.

Some things for us all to think about carefully.


Sam

References:

• Iles, J. (12 April 2018). Cambridge Analytica took my data - here's how you find out if they took yours. https://www.stuff.co.nz/business/103029596/cambridge-analytica-took-my-data--heres-how-you-find-out-if-they-took-yours
• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

read more "Ethical considerations for online career development"

Computer-assisted career guidance: key terms

I was putting together some definitions by Osborn and Zunker (2016) recently for my students on computer-assisted career guidance, and then thought that they might be useful for all of us.

CACGs: These are "computer-assisted career guidance systems" (Osborn & Zunker, 2016, p. 148). It is possible that CACGs may become app-driven: "having an app for a CACG or interactive website where you can toggle between your assessment results and occupational information, portfolio, educational opportunities, and so forth, might be an app that people would be interested in seeing" (p. 153).

CACG Outcomes: What the client gets as a result of participating in a CACG. Clients should reasonably have "certain expectations about the outcomes, including increased career options, enhanced self-knowledge and strengthened occupational knowledge" from CACGs (Osborn & Zunker, 2016, p. 150). As practitioners, we need to "check with clients at least once during their interaction with the CACG to address any frustrations, misperceptions, or to allow them to talk about their experience thus far" (p. 151).

Online inventory: Often the computerised version of a paper-based test, an "online inventory is somewhat static in nature. An individual completes the inventory by clicking a radial button or rating an item. A script in the program calculates scores for the total and for subscales, and a report is generated based on these results. The report may have hyperlinks in it that allow for further exploration of specific occupations" (Osborn & Zunker, 2016, p. 148).

Online Career System: An "online career system [is] more comprehensive and interactive [than a single online inventory]. Gati and Asulin-Peretz (2011) state that online systems should go beyond just having assessment results, and should provide an interpretation of these scores, as well as suggested interventions to help clients with difficulties. These systems usually include assessments of interests, values and skills, as well as an interactive component in which an individual can change their profile based on a combination of preferences" (Osborn & Zunker, 2016, p. 148).

Career Planning: This includes elements such as assessments, analysing opportunities, determining the client's goals, deciding what is in the current plan (and what is out), then keeping an eye on plan progress, and tweaking as things change. "Creates long-term congruence between individual goals and organizational career opportunities" (Schermerhorn et al., 2004, p. 372).

Career Gaming: "Do you want to play a career game? That doesn't sound very fun, does it? And yet, people are playing games that simulate living all the time. Three career games that are online currently include realgame.com, driveofyourlife.com, and Sims 3 Ambitions" (Osborn & Zunker, 2016, p. 153). Consider the video here: https://youtu.be/9vJRopau0g0 (Rober, 2018).

I hope you find these useful!


Sam

References:

• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.
• Rober, M. (1 June 2018). TEDxPenn: The Super Mario Effect - Tricking Your Brain into Learning More. https://youtu.be/9vJRopau0g0
• Schermerhorn, J. R., Hunt, J. G., & Osborn, R. N. (2004). Core Concepts of Organizational Behavior. John Wiley & Sons Ltd.

read more "Computer-assisted career guidance: key terms"

A continuum of decision-making models

Recently I have been thinking about decision-making models, and which ones are useful. Of course, usefulness depends on what our purpose is, so I will qualify that a little: which models are useful for an individual to use. We all need to be able to make decisions. Sometimes we have a lot of time to consider: at others we need to decide on the fly. It is almost inevitable that we will not have enough information to make our decision, so we have to satisfice (do the best we can with what we have got).

Additionally, I suspect that we think of decision-making models as us using this one OR this one, whereas the models I am going to talk about today are really on a continuum.

So let's run through a couple of frameworks from the management field.

The Rational Model (Robbins & Coulter, 2012; aka the classical approach): this is probably the most commonly use framework, and is good to use for process transparency, as each step is deliberate, we can document what we find, and we can compare and contrast. It seems really arms-length, but it is easy for us to manipulate the data we collect to put our 'chosen' option to the top of the pile. That is something we really have to guard against. In addition, it takes a long time, so if we are in a hurry, it can be difficult to use this approach.

The framework consists of eight steps:

• Identify the issue(s)
• Identify the decision criteria
• Weight the criteria
• Develop alternatives/options
• Evaluate and rank each alternative
• Chose the optimal alternative
• Implement the chosen option
• Evaluate the effectiveness of the chosen option (Robbins & Coulter, 2012, p. 195).

The Intuitive Model (Robbins & Coutler, 2012; aka experience): this model is about making "decisions on the basis of experience, feelings, and accumulated judgment. Researchers studying managers’ use of intuitive decision making have identified five different aspects of intuition" (p. 184), which includes making decisions on values or ethics; on experience; on feelings or emotions; on skills, knowledge or training; or on subconscious data (the latter is 'gut feel'). It is 'fast and dirty', but about 50% of managers use this model regularly. It allows "managers [to] eliminate counter-intuitive ideas in reaching their decision" (Gazprom Marketing & Trading, 2020). This is a subjective approach using observed patterns, similarity and 'salience' (how prominent, important or noticeable something is). However, what has gone right in the past does not necessarily mean it will go right in the future, so we can be blind to how the world has changed since the last time we encountered this problem.

The Recognition-Primed Model (Helander et al., 1997; or the case approach): this model is simply mixed methods. We use as much of the rational, and as much of the intuitive model as we need to suit our situation. For example, if we have time, we can carefully analyse alternatives (rational), while following our gut on our chosen option (intuition). We can replace the rational approach with "more intuitive or case-based approaches to navigating the problem [... and] As the person becomes more adept, the consolidated knowledge and problem solving approaches support the use of seemingly more 'intuitive,' 'holistic,' or case-based approaches. Klein (1989 [sic]) has termed this process 'recognition primed'. Whatever it is called, this approach allows the decision maker to capitalize on the more highly organized knowledge and goal structures that experience supplies" (pp. 1240-1241). This is really the best of both worlds, PROVIDING we are honest about our option evaluation, and measure our gut feel against the same criteria if others are hesitant about our choices. This model is often used by emergency services: take what we know, then rationalise what we don't know about the new situation. However, sometimes this can take more time, as it can become a "trial-and-error approach [which may make] it relatively time-consuming" (Gazprom Marketing & Trading, 2020).

I hope these are useful!


Sam

References:

• Gazprom Marketing & Trading. (2020). The different decision-making models you need to know: and their pros and cons. https://careers.gazprom-mt.com/blog/the-different-decision-making-models-you-need-to-know-and-their-pros-and-cons/
• Helander, M. G., Landauer, T. K., & Prabhu, P. V. (Eds). (1997). Handbook of Human-Computer Interaction (2nd ed). Elsevier Science BV.
• Kline, G. A. (1993).A recognition-primed decision (RPD) model of rapid decision making. In G.A. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and methods (pp. 138–147). Ablex.
  
• Robbins, S. P., & Coulter, M. (2012). Management (11th ed.). Pearson Education Ltd.

read more "A continuum of decision-making models"

Belief and decision-making: key terms

Taxonomy of career decision barriers
(Gati et al., 1996, p. 520)

I have really been enjoying putting together definitions of key terms lately for my career development students, so I thought I would share my latest crop with you all.

This time I have some definitions around career beliefs and career decision-making, as follows:

Career indecision: the "inability to specify an educational or occupational choice and can include focal problems, such as lack of information, or a combination of problems, such as choice anxiety and trait indecision" (Psychology, 2020b).

Beliefs: the "positive and negative thoughts or assumptions people hold about themselves, occupations, and the career development process" (Roll & Arthur, 2002, p. 2).

Correlates of Career Indecision: These are barriers to decision-making which may include "anxiety, lack of self-knowledge, lack of structure, personal conflict, perfectionism, fear of commitment, and perceived occupational" blocks (Osborn & Zunker, 2016, p. 132).

Negative Beliefs: these "affect clients’ perceptions of themselves and the world of work, increase clients’ level of negative emotions associated with making a career decision, and immobilize clients’ action toward their career goals", and "can influence clients’ actions at any stage of career counselling" (Roll & Arthur, 2002, p. 2).

Self-Talk: self-report inventories can "be a form of self-talk" as while they may be reporting "what they believe is true about their situations [, what they report] are perceptions and may not be reality". We can delude ourselves due to our internal narrative. However, perception becomes the individual's reality, as "regardless of how real the barrier is, people's self-talk can have a dramatic impact on how they view themselves, as well as on their options and how they make career decisions" (Osborn & Zunker, 2016, p. 133).

Career Decision-Making Difficulties: these "decision-theory-based systematic framework for characterizing an individual’s career decision-making difficulties. It comprises three major categories of difficulties, which are further divided into 10 specific difficulty categories. Lack of Readiness includes three categories that precede the actual engagement in making a specific career decision: lack of motivation to engage in the career decision-making process, general indecisiveness concerning all types of decisions, and dysfunctional beliefs about career decision making. The other two major categories focus on difficulties that may arise during the engagement in the process. Lack of Information includes lack of knowledge about the steps involved in the process of making a career decision, lack of information about the self, lack of information about the various occupations, and lack of information about the ways of obtaining additional information. Inconsistent Information includes three categories of difficulties in using the information: unreliable information, internal conflict within the individual (such as contradictory preferences), and external conflicts, involving disagreements with significant others" (Psychology, 2020a).

Dysfunctional Career Thinking: this type of thinking tends to be is absolutist, full of "all-or-nothing thinking, overgeneralizations or language peppered with “should’s, must’s and ought’s", and is "a primary influencer of the other elements essential to making an effective career decision (i.e., knowledge about self, knowledge about options and the decision-making process as a whole)". At some point, "the client has had an experience that suggested that belief. Over time and in different situations, that belief has been reinforced and may have been integrated into [the client's] core" understanding of themselves. (Osborn, 2019).

I hope you found these useful!


Sam

References:

• Gati, I., Krausz, M., & Osipow, S. H. (1996). A taxonomy of difficulties in career decision making. Journal of Counseling Psychology, 43(4), 510-526. https://doi.org/10.1037/0022-0167.43.4.510
  
• Osborn, D. S. (2019). Dysfunctional career thoughts: Are they really a big deal? https://careerwise.ceric.ca/2019/06/07/dysfunctional-career-thoughts-are-they-really-a-big-deal/
• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.
• Psychology (2020a). Definition of Career Decision Making Difficulties. http://psychology.iresearchnet.com/counseling-psychology/career-assessment/career-decision-making-difficulties-questionnaire/
• Psychology (2020b). Definition of Career Indecision. http://psychology.iresearchnet.com/counseling-psychology/career-counseling/career-indecision/
• Roll, T., & Arthur, N. (2002). Beliefs in Career Counselling. https://contactpoint.ca/wp-content/uploads/2013/01/pdf-02-03.pdf

read more "Belief and decision-making: key terms"

Personality testing

A phrenological diagram
(Combes, 1934, p. 20, citing Dolci, 1562)

We often conflate assessments into one pile: those of personality tests. So what are personality tests?

"Personality tests are are self-report questionnaires in which the respondent provides information about [their] feelings or behaviors". Personality plays "a significant role in helping people determine occupations that may or may not be a good match for them", with the matching self-knowledge growing from self-reflection (Greenhaus & Callanan, 2006, p. 633).

This understanding of ourselves brings "a better understanding of which occupations and career paths would better match [our] interests and personality. Being honest with ourselves about who we are and our strengths and weaknesses can help us choose situations that we will be comfortable in, as well as make us aware of situations that we might want to avoid" (Greenhaus & Callanan, 2006, p. 633).

Interestingly, psychologists agree that "the optimal personality test is one that measures the Big Five personality dimensions: extroversion, agreeableness, conscientiousness, emotional stability, and openness to experience. [...] Literally hundreds of personality tests have been used for selection; however, tests of the Big Five are among a very small group of such tests that have demonstrated value in selection" (Greenhaus & Callanan, 2006, p. 633).

It is surprising how few tests have been rigorously tested (Cripps, 2017; Osborn & Zunker, 2016). However, there are barriers to personality testing: many tests are proprietary, requiring the permission of the owners before any research can take place; we need a stable population to be able to test and re-test on; we need to be immensely careful about how questions are asked; we need to be very careful of participant bias, of researcher bias, of sample size, of methodological approach; and we need to be careful of vested interest. There are large career and human resource trucks which drive the economic engine of the testing sector (Cripps, 2017).

Like the study of phrenology in the 19th century, I think personality testing will be a hard train to derail. It has usefulness in places, but not in ALL the places we currently use it.
We just need to be careful to use testing appropriately, and not assume that we will get 'answers' from it. With good use, we should get better questions.

Let's be careful out there.


Sam

References:

• Combe, G. (1834). A System of Phrenology (3rd American ed.). Marsh, Capon and Lyon.
• Cripps, B. (Ed.) (2017). Psychometric Testing: Critical perspectives. Wiley Blackwell.
• Greenhaus, J. H., & Callanan, G. A. (2006). Encyclopedia of Career Development (Vols. 1 & 2). SAGE Publications Ltd.
• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

read more "Personality testing"

Personality test terms

I was putting together some definitions by Greenhaus and Callanan (2006) recently for my students, and then thought that they might be useful for all of us!

Personality: A "Personality refers to characteristics that make individuals unique, including their prototypical thoughts, emotions, interests, habits, and behaviors. Psychological in nature, personality is relatively stable over time. Almost every element of a person’s career is in part determined by [...] personality, [including] occupational choice, early career socialization, job performance, career satisfaction, and career changes are affected by personality" (Greenhaus & Callanan, 2006, p. 627).

Personality Test: "Personality tests are are self-report questionnaires in which the respondent provides information about [their] feelings or behaviors" (Greenhaus & Callanan, 2006, p. 633).

Broad Personality Factors: "Early work in the personality field identified nearly 18,000 words from the dictionary as being potentially useful in distinguishing people from one another", with later research narrowing this to "3,500 [...] stable personality traits", resulting in psychologists continually trying simplify and rationalise the taxonomies, "to reduce the thousands of personality traits into a smaller, more manageable set". Broad traits are at one continuum-end, demonstrating the multi-dimensionality of personality (Greenhaus & Callanan, 2006, p. 629).

Individual Personality Factors: "individual personality trait[s are] a narrow and precise approach to categorizing one of the unique features of a human being[, ...] focused on a single element of personality". Many "sets of narrow personality traits [are] related; for example, someone who is generally dependable might also tend to be very organized. Narrow traits are at the other continuum-end, demonstrating the uni-dimensionality of personality (Greenhaus & Callanan, 2006, p. 629).

Self-evaluation: Self-evaluation is a "broad dispositional concept referring to the extent to which people hold a positive self-concept, core self-evaluations are related to several career processes and outcomes" (Greenhaus & Callanan, 2006, p. 631). "Core self-evaluations are basic evaluations that individuals hold about themselves, and they are indicated by four specific traits: self-esteem, generalized self-efficacy, locus of control, and emotional stability (low neuroticism). Core self-evaluations are related to job satisfaction, task motivation, and job performance" (Greenhaus & Callanan, 2006, p. 631).

I hope these are useful!


Sam

• Reference: Greenhaus, J. H., & Callanan, G. A. (2006). Encyclopedia of Career Development (Vols. 1 & 2). SAGE Publications Ltd.

read more "Personality test terms"

What interest inventories (or assessments) are

A spider plot of an RIS score

Working as a career practitioner is a "process of understanding and helping people cope with problems", a "process [which] includes defining the problem, gathering information about the person and the environment, understanding and interpreting the information, and problem coping" Any assessment we use is part of the information gathering process, which "attempts to link person, environment, and behavior in an information-based framework to help people understand and cope with problems. Tests are frequently used as part of this process to collect meaningful information about people and the psychological instrument is part of their context" (Walsh & Betz, 2000, p. 19). There are many things we can test: cognition, "personality, interest, and value factors [, which] are essential determinants of behavior" (Walsh & Betz, 2000, p. 13).

Interest inventories have been around since the Strong Vocational Interest test from 1927 (Walsh & Betz, 2000), later popularised by Holland in the 19560s and 70s (Osborn & Zunker, 2016). Interest inventories are instruments which focus solely on interests - not aptitude, achievement, values or personality. If we know how someone is likely to behave and which environment they are likely to be comfortable in, then we can match their interests to a field of work.

Holland probably has the most "well known theory for examining interests [...] RIASEC" in the US (Osborn & Zunker, 2016, p. 54), but this theory is less well-known here in New Zealand. Further, in New Zealand we have our own interest inventory developed by CNZ, called Career Quest (here).

Many US tests are unavailable in New Zealand, and very, very few tests - if any - have been normalised for the New Zealand population, or for our local minority groups. As we have no comparative data, all test results must be taken as 'general' rather than specific.

Additionally, as practitioners working with 'minority' groups who may not "trust the use of tests, [perhaps] feeling that tests have been used to establish power and maintain discriminatory practices", we "may choose to use nonstandardized instruments instead of standardized tests [... to minimise] the influence of culture on [ourselves], [our] clients, and the counseling process" (Fouad, 1993, p. 6).

NB: it may be possible that norm testing has been done on Career Quest and/or Smithells, but if so, I am not aware of it. Please advise if you know!


Sam

References:

• Fouad, N. A. (1993). Cross‐cultural vocational assessment. The Career Development Quarterly, 42(1), 4-13. https://doi.org/10.1002/j.2161-0045.1993.tb00240.x
• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.
• Walsh, W. B., & Betz, N. E. (2000). Tests and Assessment (4th ed.). Prentice Hall.

read more "What interest inventories (or assessments) are"

Career minorities

While I have posted on this before (here), it is probably worth restating. New Zealand is made up of a lot of minority groups:

• 15% Maori; 12% Asian, 7% Pasifika and the remainder Pakeha (Statistics NZ, 2013)
• 51% female and 49% male (Statistics NZ, 2013)
• 18% with a bachelor's degree or higher from a range of disciplines; 17% with diplomas or certificates (this may include trades); 30% with a secondary qualification; 19% with no quals; and 17% that doesn't seem to be counted anywhere but may be tradespeople if they are not in the cert/dip group (Statistics NZ, 2013)
• a range of socio-economic backgrounds with 6% in Socio-economic status 1 (the 'upper' class); 6% in 2; 28% in 3; 21% in 4; 19% in 5 and 20% in 6 (Statistics NZ, 2006a)
• 25% born overseas, 75% born in New Zealand (Statistics NZ, 2013)
• Ages with 50% over 42 years of age and 50% under 42; broken down into 18% 15-24, 41% 25-44, 38% 45-64, 8% 65+ (EEO Trust, 2012)
• 16% with differing disabilities (Statistics NZ, 2006b).

When it comes to careers, women - who are a majority: a 'major minority' (Statistics NZ, 2013) - do not tend to identify with mathematics or science. The lack of interest - identification - with these subjects appear to be a key driver in the limited entry of women into the engineering field. In America, women make up "45% of the work force, yet only 16% of scientists and engineers" (Osborn & Zunker, 2016, p. 69; Betz, 1997). These statistics seem to be similar in New Zealand, along with Maori and Pasifika peoples also being very under-represented in science and engineering.

Where New Zealand is different to the US is that 60% of the law graduates in Aotearoa are women (New Zealand Law Society, 2020), and 45% of doctors (Medical Council of New Zealand, 2017). Doctors of Maori ethnicity are also increasing, with 16% of graduates identifying as Maori, up almost 4% on 2016. Further, women doctors are expected to outnumber men in 2015 (Medical Council of New Zealand, 2017).

There is some shift happening. Long may that last :-)


Sam

References:

• Betz, N. E., & Hackett, G. (1997). Applications of self-efficacy theory to the career assessment of women. Journal of Career Assessment, 5(4), 383-402. https://doi.org/10.1177/106907279700500402
• EEO Trust (2012). Workforce Demographic Trends. http://www.eeotrust.org.nz/content/docs/reports/FINAL%20Workforce%20Demographic%20Trends%202012.doc
• Medical Council of New Zealand (2017). The New Zealand Medical Workforce in 2017. https://www.mcnz.org.nz/assets/Publications/Workforce-Survey/c3f49fa2d2/Workforce-Survey-Report-2017.pdf
• New Zealand Law Society (2020). By the numbers. https://www.lawsociety.org.nz/law-society-services/women-in-the-legal-profession/by-the-numbers#:~:text=Over%2060%25%20of%20law%20graduates,or%20directors%20in%20those%20firms
• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.
• Statistics NZ (2013). New Zealand Census 2013. http://archive.stats.govt.nz/Census/2013-census/profile-and-summary-reports/quickstats-about-national-highlights/cultural-diversity.aspx
• Statistics NZ (2006a). New Zealand socio-economic index 2006. http://archive.stats.govt.nz/~/media/Statistics/surveys-and-methods/methods/research-papers/nz-socio-economic-index-2006/nz-socio-economic-index-2006.pdf
• Statistics NZ (2006b). Disability Status By Place of Residence, Age Group, Sex and Ethnic Group, 2006. http://nzdotstat.stats.govt.nz/wbos/modalexports.aspx?exporttype=excel&FisrtDataPointIndexPerPage=&SubSessionId=6b20dd2f-aa09-44f4-9566-8f580ab9cc35&Random=0.7374431257710761

read more "Career minorities"

Aptitude & achievement tests

We talk about assessments often in career practice, but we very rarely get down and dirty and define exactly what it is that we are talking about.

Aptitude is a "specified proficiency or the ability to acquire a certain proficiency" (Osborn & Zunker, 2016, p. 53), which might be a skill we have now OR one we have yet to develop. They can also be thought of as "a tendency, capacity, or inclination to do a certain tasks", and are "the result of both heredity and environment; an individual is born with certain capacities that might or might not be nurtured by the environment" (p. 53). So an aptitude test is a test which identifies "existing knowledge and skills", and uses that "to predict the ability to be successful in future training or education and to learn new skills within a certain aptitude" that we have. This then allows us to "reflect the interaction of heredity and environment and predict the capacity to learn" (Osborn & Zunker, 2016, p. 54).

Achievement is the development of a particular piece of knowledge or skill, or "the competence a person [may] have in a area of content" (Algarabel & Das, 2001, p. 44). This type of testing assesses competence levels in a range of specified areas of academic proficiency, such as "reading, language usage, [or] mathematics" with the roles or courses of study they are interested in. Their resulting performance "may be the key to rejection or consideration of certain educational and vocational plans" (Osborn & Zunker, 2016, p. 68).

"Aptitude [and achievement] tests have been associated with career counseling since the time of the early trait-and-factor approach", and help us to "identify existing knowledge and skills". These tests "are also used to predict [our] ability to be successful in future training or education and to learn new skills" (Osborn & Zunker, 2016, p. 54).

"Both aptitude and achievement tests measure learning experience". However, achievement tests measure a narrow range, and benchmarks the results (Osborn & Zunker, 2016, p. 68).

The authors explain that aptitude testing became popular after WW2, with the return of those who served in the armed services (Osborn & Zunker, 2016). The GI Bill allowed veterans a grant to go to university, which initially drove research into testing, which the US armed services also adopted to attract 'the right stuff' for forays into Korea, the Cold War, and Vietnam.

What is also interesting is how "Early vocational counseling programs advocated psychological testing in vocational counseling specifically to analyze an individual's potential in relation to requirements of training programs and occupations". This in turn "inspired the study of job descriptions and job requirements in an attempt to predict success on the job from the measurement of job-related traits" (Osborn & Zunker, 2016, p. 55). What are now common Human Resource Management tools came from career development.

Many US tests - both aptitude AND achievement - are not available in New Zealand, including the military test batteries and SkillTRAN database and test (though the latter would be VERY useful when working with ACC claimants!). Additionally, many US standardised tests - MCAT, GMAT & LSAT - are not used in New Zealand. Further, I think that culturally we would be disinclined to adopt them. Research evidence indicates that standardised testing is not that useful (Gunzelmann, 2005), and New Zealand's recent foray into it under the last National government was certainly not very successful.

An interesting topic!


Sam

References:

• Algarabel, S., & Dasi, C. (2001). The definition of achievement and the construction of tests for its measurement: A review of the main trends. Psicologica, 22(1), 43-66. https://www.redalyc.org/pdf/169/16917746001.pdf
• Gunzelmann, B. (2005). Toxic Testing: It's Time to Reflect upon Our Current Testing Practices. Educational Horizons, 83(3), 212-220.
• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.

read more "Aptitude & achievement tests"

Statistical definitions for career assessment

Wikimedia Creative Commons (2020)

Carrying on the theme of assessment, Osborn & Zunker provide some great definitions of the statistical terms (2016):

1. Norms: "Norms represent the level of performance obtained by the individuals (normative sample) used in developing score standards. Norms can thus be thought of as typical or normal scores. Norms for some tests and inventories are based on the general population. Other norms are based on specific groups such as all 12th-grade students, 12th-grade students who plan to attend college, left-handed individuals, former drug abusers, former alcoholics, or individuals with physical disabilities" (Osborn & Zunker, 2016, p. 29).
2. Norm Tables: These "provide specific definitions of normative groups. Such detailed descriptions of persons sampled in standardizing am inventory provide good data for comparing the norm samples with [specific] client groups. [It is useful to have...] score differences between age and ethnic groups and between individuals in different geographical locations. The more descriptive the norms are, the greater their utility and flexibility" (Osborn & Zunker, 2016, p. 29).
3. National Norms (Country Profile): National population norms "are usually controlled in the sampling process to be balanced in geographical area, ethnicity, educational level, sex, age, and other factors. National norms may be helpful in determining underlying individual characteristics and patterns" (Osborn & Zunker, 2016, p. 30), and for identifying cultural difference. What is normal in the US is not necessarily normal in New Zealand, and less likely to be 'normal' for Maori.
4. Local Norms: While we are aware of local differences within New Zealand, the idea of local norms is less applicable here. "requirements vary from one location to another, using local norms is recommended". For example, in the US, if a client wanted to know their chances of success at a local tertiary provider, the practitioner could tap into secondary graduate grade data from the client's high school, map it to the tertiary institute grades of previous graduates from the client's secondary school (and therefore where the client ranked), and develop an expectancy table showing where the client would be likely to perform at tertiary. This allows the client an opportunity examine their fit with the tertiary institute's requirements. "Think of this as acclimatisation, or preparation" (Osborn & Zunker, 2016, p. 30).
5. Score Profiles: These "provide a visual representation of the peaks and valleys in a person's test results" and "help to identify what falls within the 'normal' range, as well as indicators of where the individual scored higher or lower" (Osborn & Zunker, 2016, p. 30). However, we also need to be careful about generalising too much with score profiles: Osborn and Zunker note that we need to be careful when explaining variation; that we are cautious about comparing too closely to norms and not taking enough notice of cultural difference, and that we use ranges rather than specific scores to allow for bias and natural variation.
6. Bell Curve (Normal Distribution): Normal distribution of a sample or a population is usually shown as a "bell-shaped curve [where] M represents the mean, or midpoint (50th percentile), with 4 standard deviations on each side of the mean" (Osborn & Zunker, 2016, p. 32). "It is useful at times to compare an individual's score to where the majority of the scores lie. For example, + 1 or -1 standard deviation from the mean will capture approximately 68% of the variance" (Osborn & Zunker, 2016, p. 33). Two standard deviations from the mean captures 95% of the variance, and three, 99%.
7. Percentiles: "The most common definition of a percentile is a number where a certain percentage of scores fall below that number. You might know that you scored 67 out of 90 on a test. But that figure has no real meaning unless you know what percentile you fall into. If you know that your score is in the 90th percentile, that means you scored better than 90% of people who took the test" (Glen, 2020a). "Percentile equivalents are direct and relatively easy to understand, which is a primary reason for their popularity. However, it is important to identify the norm reference group from which the percentile equivalents have been derived", otherwise we may "attach labels to these percentile equivalents" which are inappropriate, such as assuming a high score is an 'A', or that a low score is a 'D' (Osborn & Zunker, 2016, p. 34).
8. Stanines: Divide the bell curve into 9, with the mean range at 5, and four standard deviations each way, and we get stanines. "A stanine ('standard nine') score is a way to scale scores on a nine-point scale. It can be used to convert any test score to a single-digit score" and "Stanines are also similar to normal distributions. You can think of these scores as a bell curve that has been sliced up into 9 pieces. These pieces are numbered 1 through 9, starting at the left hand section. However, where a standard normal distribution has a mean of 0 and a standard deviation of 1, stanines have a mean of 5 and a standard deviation of 2" (Glen, 2020b). "For example, stanine scores 1, 2, and 3 are considered below average; stanine scores 4, 5, and 6 are considered average; and stanines 7, 8, and 9 are above average" (Osborn & Zunker, 2016, p. 35-36).
9. Reliability: "Reliability looks at how consistently a test measures the construct under consistently, and the degree to which tests scores are free from error", while acknowledging that our "client's observed score on a test is actually their true score with some error added". To assess reliability we consider reliability coefficients (internal reliability), and test-retest reliability (testing again), both of which can be assessed using Cronbach's alpha (Osborn & Zunker, 2016, p. 37). We may have good internal reliability, but poor test-retest; poor internal, and good test-retest; both good; both poor. A number of test companies get internal reliability scores, where they are comparing their test against their own data.
10. Validity: "Validity answers the question, 'Does the test measure what it purports to answer?'". There are three types of validity: content validity, where what is being tested is accurate; criterion-related validity, how much the test results are related to a particular outcome; and construct-validity, where the test measures what it says it measures. Tests being evaluated for validity will have a lower coefficient score than reliability, as they are compared to other tests (Osborn & Zunker, 2016, p. 37), and this is what we practitioners must be alert to when relying on these tests. Few companies obtain external validity scores, possibly - note this is just supposition - because they would score poorly.

A very useful list of terms to understand.


Sam

References:

• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.
• Wikimedia Commons (2020). File:Standard deviation diagram.svg. https://upload.wikimedia.org/wikipedia/commons/thumb/8/8c/Standard_deviation_diagram.svg/1280px-Standard_deviation_diagram.svg.png

read more "Statistical definitions for career assessment "

Career Assessment in New Zealand

Career development grew in New Zealand from the "Employment and Vocational Guidance Service", established in the 1970s, to "the Career Development and Transition Education Service [, which was formed] under the Ministry of Education" in the 1990s (Furbish, 2012, p. 16). The service was renamed Careers New Zealand (CNZ) in June of 2011, and on the first of July 2017, was absorbed into the Tertiary Education Commission, commonly known as TEC (Careers New Zealand, 2020).

CNZ provided "resources and support to career activities in secondary schools", and "career services to clients of other government agencies and fee-paid career services to the public" (Furbish, 2012, p. 16-17). It was also considered internationally to be "a model government-funded career services provider", having, as well as "extensive career information available" online, invested in hotline and virtual career consultancy with the in-house career development practitioners (p. 17). Since the move to TEC, CNZ's offer has been significantly reduced. The website remains active, but there are no individual career counselling services offered, no training, and little secondary school support. The focus is now on the tertiary sector. No funding arrangements or services appear to be tagged for career development services in tertiary institutions from CNZ under TEC.

This short history relates to career assessment is because - unlike America - New Zealand now has no national, independent, co-ordinating organisation with the resources to gather test information, to evaulate and certify tests, or to normalise test results for our New Zealand population groups.

To date, other than Smithells (2020), no other career assessments appear to have been normalised for use in New Zealand (Smithells is only 'normalised' for New Zealand because it was developed here). This means that for New Zealanders as a whole, for New Zealand Maori, or for Pasifika peoples, no other test distributions or norms have been established. Although Extended DISC report MAO (Maori) as one of the cultural groups, succeeding biannual reports from the company have not gathered any data on this population group. There is no code for New Zealand Europeans (Extended DISC, 2013, Extended DISC, 2015).

This means that we do not have local population data to compare sample statistics to for any assessments done in New Zealand.

In the USA it is possible to get very good statistical and population data for comparisons (such as the Buros Center for Testing's Mental Measurements Yearbooks). In New Zealand, normalised data is not available, so the data we collect is not readily comparable, and therefore lacks validity, reliability and generalisability. Osborn and Zunker explain in the chapter why validity and reliability are important. Within the US cultural context they do not have to consider generalisability as norms and stanines already exist.

Something for us all to think about down in the South Pacific.


Sam

References:

• Buros Center for Testing (2020). The Mental Measurements Yearbooks. https://buros.org/mental-measurements-yearbook
• Careers New Zealand (CNZ) (2020). Our Publications: Annual Report. https://www.careers.govt.nz/about-careers-nz/our-publications/annual-report/
• Extended DISC (2013). Extended DISC Personal Analysis Validation Report 2013. https://docplayer.net/49674254-Extended-disc-personal-analysis-validation-report-2013.html
• Extended DISC (2015). Extended DISC Personal Analysis Validation Report 2015. https://www.one4.eu/blog/wp-content/uploads/2017/05/Extended-DISC-Validation-Report-2015_client.pdf
• Furbish, D. (2012). An Overview of New Zealand Career Development Services. Australian Journal of Career Development, 21(2), 14-24. https://doi.org/10.1177/103841621202100203
• Osborn, D. S., & Zunker, V. G. (2016). Using Assessment Results for Career Development (9th ed.). Cengage Learning.
• Smithells NZ (2020). Shaping Careers. https://www.smithellsnz.co.nz/

read more "Career Assessment in New Zealand"