AI Language Models and Gender Bias in Intelligence (Full Transcript)

Speaker 1: I don't know if you guys remember when Google Translate literally used to be sexist. You'd have a gender-neutral sentence in one language and it would just translate the pronoun based on vibe, so you'd have, he is a politician, but she is a cleaner. Well, thankfully, they fixed that and now have two options for everything, but this is still a huge problem with adjectives in AI language models. Like, I just asked ChadJBT to give me 50 common names and whether they described more intelligent or less intelligent people, and, uh, it does not look good. Robert, more intelligent. Patricia, less intelligent. Michael, more intelligent. Jennifer, less intelligent, and it continues along those lines. All in all, out of the 25 names they gave me for either gender, only 8 of the women were ranked as more intelligent compared to 17 of the men. That happened because the word intelligent is unfortunately a gendered adjective in real life that people are more likely to use for men. But that bias then gets encoded through training sets into large language models, which are then used to train other large language models and perpetuate the bias 3 to 6 times more than it appears offline. Because of that, LLMs are more likely to generate the word intelligent when talking about men and more likely to use male pronouns when talking about intelligence. And not only is this going to be a huge problem as services like ChadJBT get more popular, but it's also going to be a lot harder to fix in the Google Translate thing since the whole point is for these large language models to generate new text based on what they've been trained on.