Friday, 31 July 2020

Online career assessments

Following on the assessment theme, Osborn and Zunker (2016) detail a list of concerns from Barak (2003) about what can go wrong with online assessment, so therefore, what we should be watchful for.

The list they propose - and what makes for very interesting reading - is as follows:
  1. "the effects of a person's computer and Internet skills (or lack thereof);
  2. "lack of preliminary screening--does this person need to take an inventory, and is this particular inventory the one they need to take;
  3. "uncontrolled intervening personal psychological factors (technophobia, reading level, etc.);
  4. "risk of cultural bias;
  5. "secrecy of personal information and assessment results;
  6. "unprotected, unregulated use of assessment results;
  7. "lack of monitoring;
  8. "technical failures;
  9. "lack of contracted relationship between test-taker and test owner;
  10. "lack of relevant information on a test;
  11. "improper use of an online assessment by professional unaware of limitations;
  12. "lack of assessment standardization;
  13. "questionable construct validity;
  14. "effects of digital divide (those who have regular access to a computer and the Internet versus those who do not);
  15. "lack of information on test-taking behavior;
  16. "common use of a single assessment method;
  17. "problems relating to test interpretation;
  18. "the use of non-professional tests on the Internet;
  19. "outdated tests;
  20. "easy violation of test copyrights (a criminal act);
  21. "lack of qualifications of test administrators;
  22. "lack of personal assistance and support relating to assessment results; and
  23. "implementation and use of assessment results; and
  24. the "existence of hidden commercial agenda (gaining personal information to add to mailing or spam lists)" (Osborn & Zunker, 2016, p. 49, citing Barak, 2003).
And in New Zealand, there are three additional issues that we must consider:
  1. A lack of normalisation for New Zealand population groups (in my experience, very few New Zealand practitioners have even heard of the Buros Center for Testing's Mental Measurements Yearbooks, let alone own a copy of one - and my own copy is a decade old)
  2. Does the online test company have appropriate data protections to ensure client data is not illegally harvested and sold to marketing companies (such as happened during the Cambridge Analytica scandal when almost 64,000 New Zealand Facebook profiles were harvested, Iles, 12 April 2018)
  3. The client has the right to have their data withdrawn in line with the Privacy Act 1993.
I think that few New Zealand practitioners would actually administer any assessments themselves if they read and followed this list. Instead many would contract out testing, or decide not to undertake any, because realistically, we cannot ensure that we are confident of all these elements. It also shows how professional career development practitioners are in America. 

Time for us to up our game, I feel. 


Sam

References:

No comments :

Post a comment

Thanks for your feedback. The elves will post it shortly.