The Emperor has no clothes & AI

While I am sure that AI platforms will improve, I was struck by a Guardian long read article last year where a journalist reported that, "when I asked ChatGPT to write a bio for me, it told me I was born in India, went to Carleton University and had a degree in journalism – about which it was wrong on all three counts (it was the UK, York University and English). To ChatGPT, it was the shape of the answer, expressed confidently, that was more important than the content, the right pattern mattering more than the right response" (Alang, 2024).

I think that is the core of the AI problem. The confidence of the delivery from the AIs we consult (Alang, 2024). The large language models which AI is trained upon is logically North American. That is where the tech companies are. The USA has driven much of the research and IT work for the past half century. The US is probably the most WEIRD society (here; Henrich et al., 2010): a Western, educated, industrialised, rich and democratic society, which collectively make up 12% of the global population. Researchers have considered "how WEIRD [society populations] measure up relative to the available reference populations" (p. 62), finding that in most behavioural research studies, a full "68% of [research participants] came from the United States, and a full 96% of subjects were from Western industrialized countries, specifically those in North America and Europe, as well as Australia and Israel" (p. 63); and even more narrow, that "67% of the American [participants] (and 80% of the [participants] from other countries) were composed solely of undergraduates in psychology courses" (p. 63).

So not very representative then. And if we think of the 12% of global population in WEIRD societies, 50% will be female. Around 40% of Americans go to college (National Center for Education Statistics, 2020). So lets assume of the 6% of WEIRD societies which are male, that 40% have gone to college. While this is very rough maths, at the most, AI is based on 2.4% of the global population (and it will be a fraction of that number, because few will have completed an IT degree, let alone a behavioural science degree, as per Henrich et al., 2010). Yes, I know I am comparing apples with oranges, but I don’t think we can safely assume that the data being used to 'train' the AI models is unbiased. I think it is pretty clear that the training data is based on a tiny non-representative percentage of the global population. 

The software and hardware engineers working on AI are also likely to be male, with a good chunk from North America (Alang, 2024). While, 23% of workers in IT are women (Deloitte, 2021), it was noted at one large US company that there were "641 people working on 'machine intelligence,' of whom only 10 percent were women" (Simonite, 2018). So yes, while nearly a quarter of the IT sector has women in it, the gender distribution is uneven. And if we come back to Von Bertalanffy's system theory (1968), this shows that the input is definitely biased. Thus the transformation - no matter what we do elsewhere - will also be biased. This means that the output too will be biased. 

We are used to consulting the internet for factual answers. Yet there is a growing trend that what is on the internet is a mashup of fact and fiction. Since the early 1990s, we 'little people' have been able to create our voices without the peer review of publishers and others to filter what we say. And now, perhaps throwing a massive spanner in the works, generative AI creates blends of fiction and fact... and - unless we know our field - we have little idea which elements are fiction, and which are factual (Alang, 2024; Lingard, 2023). We consult the oracle and lack the understanding to be able to point out that the emperor has no clothes.

But the more I read, the more I think that the emperor is indeed naked. So far, anyway.

Sam

References:

Alang, N. (2024, August 8). No god in the machine: the pitfalls of AI worship. The Guardian. https://www.theguardian.com/news/article/2024/aug/08/no-god-in-the-machine-the-pitfalls-of-ai-worship

Deloitte. (2021, December 1). Women in the tech industry: Gaining ground, but facing new headwind. https://www2.deloitte.com/us/en/insights/industry/technology/technology-media-and-telecom-predictions/2022/statistics-show-women-in-technology-are-facing-new-headwinds.html

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?. Behavioral and Brain Sciences, 33(2-3), 61-83. https://doi.org/10.1017/S0140525X0999152X

Lingard, L. (2023). Writing with ChatGPT: An illustration of its capacity, limitations & implications for academic writers. Perspectives on Medical Education, 12(1), 261-270. https://doi.org/10.5334/pme.1072

National Center for Education Statistics. (2020). Chapter 2: College Enrollment Rates. In The Condition of Education. https://nces.ed.gov/programs/coe/pdf/coe_cpb.pdf

Simonite, T. (2018, August 17). AI Is the Future—But Where Are the Women?. WIRED. https://www.wired.com/story/artificial-intelligence-researchers-gender-imbalance/

Von Bertalanffy, L. (1968). General System Theory:  Foundations, Development, Applications. George Braziller.