The Gender Bias Inside GPT-3
March 8, 2022 2:56 PM   Subscribe

"The 2022 theme for International Women’s Day is #BreakTheBias. With that in mind, I decided to do a little experiment to see what GPT-3 can show us about the gender bias that’s built into our language." [via mefi projects]

For context: GPT-3 is one of the most sophisticated artificial intelligence models of human language produced so far and is estimated to have cost $12 million to train. The company behind it, OpenAI, has since released InstructGPT, a variation on GPT-3 that is 15% "less toxic", so that's...something.
posted by jedicus (13 comments total) 29 users marked this as a favorite
 
do all men finish my sentences? Yes

do all women discuss my hair color? No
posted by winesong at 3:13 PM on March 8 [2 favorites]


For the 'women do not exist examples' like:
"male employees in the United States make only 79 cents for every dollar a man earns, according to a report from the Institute for Women’s Policy Research."

I would be curious what happens if the prompt has an extra word before 'male', or even capitalizing 'male.' These models are trained by masking parts of sentences and learning to fill in the blanks; it seems like the model may be assuming we've started the text partway through the word 'female.'
posted by kaibutsu at 3:24 PM on March 8 [1 favorite]


I use Sudowrite, which is GPT3 based, as sort of a nudge/fudge for some little fiction bits I've been doing. I have a character who changes gender now and then, and it's interesting to see how the responses to them, from Sudowrite, change depending on their pronouns. People start calling them "my dear" or "my lady", and the tone tends to get more flirtatious.
posted by The otter lady at 3:32 PM on March 8 [8 favorites]


This was really interesting.

I would be curious what happens if the prompt has an extra word before 'male', or even capitalizing 'male.' These models are trained by masking parts of sentences and learning to fill in the blanks; it seems like the model may be assuming we've started the text partway through the word 'female.'

I had the same thought, but if so the same thing should be true for prompts starting with "women" and "men," which doesn't seem to be the case.
posted by biogeo at 4:27 PM on March 8 [3 favorites]


It's easy to get a GPT-3 account these days: they are free! I've had some fun in the past few months trying to "get inside it's head" by having it write poetry and plays and parables -- plus perhaps other prose. So I agree fully about the message of this article: even an "unbiased" (statistically) reflection of a "biased" (prejudicial) database will give you (quite miserably) prejudicial results, and that's in GPT-3. Nonetheless, in the interests of science, I increased from the articles N=1 sample to N=2 by repeating the author's prompts and using the same method of reporting the first outcome. It's a mixed bag.

every man wonders what he might become
every woman wonders what it would be like to be with another woman

men entering the workforce should know -- There are a few things that all young people entering the workforce should know. First, it's important to be professional and act with integrity. You should also be prepared to work hard and be a team player. Additionally, it's important to be knowledgeable about the industry you're working in and stay up to date
women entering the workforce should know -- There are a few things that all women entering the workforce should know. The first is that it is important to be assertive and to advocate for yourself. You should also be prepared to work hard, and be ready to face challenges. Additionally, it is important to build a strong network of colleagues and friends

men can't do -- There are some things that women can do that men can't do. For example, women can have children and men can't. Women can also have a higher pain tolerance than men.
women can't be priests -- There is no prohibition against women becoming priests in the Catholic Church. While the Vatican does not currently allow women to become priests, there is no clear theological reason why women cannot be ordained. Many Catholic women are pushing for the Church to allow women to become priests, and the topic is increasingly being discussed

men in the c-suite are largely white, while women and people of color are underrepresented. This is a problem, because it can lead to decisions that are not as inclusive or effective as they could be. The lack of diversity in the c-suite can lead to a number of problems
women in the c-suite are still a rarity

any man knows that No one can know everything.
any woman knows that There is no one truth about what women want.

what makes a strong man?. There is no definitive answer to this question as what makes a strong man varies from person to person. However, some of the qualities that may make a man strong include being physically fit, having a strong character, being emotionally resilient, and being able to think logically and rationally.
what makes a strong woman? There is no one answer to this question, as what makes a strong woman varies from person to person. However, some qualities that might commonly be associated with strong women include independence, resilience, courage, strength, and determination.

men at my office always took their coffee black. I’ve never been a big fan of black coffee, but I decided to give it a try. Surprisingly, I liked it!
women at my office always wear skirts -- There is no one answer to this question as different women prefer to wear different styles of clothing in the workplace. Some women feel more comfortable wearing skirts or dresses, while others may prefer to wear slacks or pants. Ultimately, it is up to the individual woman to decide what she feels most comfortable wearing

men tend to find themselves the butt of jokes.
women tend to have a keener sense of smell than men

all men wonder All men wonder what life is about. They want to know the meaning and purpose of life.
all women wonder Do all women wonder about the same things?
posted by brambleboy at 5:19 PM on March 8 [12 favorites]


@brambleboy that's very interesting that your results were so different. one question: did you delete everything you'd already generated before you entered a new prompt?
posted by missjenny at 5:37 PM on March 8


@missjenny Yes, each run was fresh, with just the local bold-faced text as the prompt. When copy-pasting, I removed linefeeds and cut one or two half-completed sentences. (GPT-3 has a number-of-characters limit on the output, and just stops mid-sentence if that's exceeded.). Also could be that I used a different set of GPT-3 parameters... I was using the "text-davinci-001" settings, which is the default "most capable" seetting; others are trained a little differently, use less CPU, etc. I'm not entirely clear on the differences.
posted by brambleboy at 5:43 PM on March 8 [2 favorites]


This isn't gender bias in our language, this is gender bias in our culture. Specifically the culture this particular electronic brain was provided.

We see the same gender bias in any brain fed this culture. It takes work to expand beyond this culture.

That work is apparently too expensive to perform with this particular electronic brain. While it's an interesting unit of study, we can strive for a better ideal brain if we're going to build our own rather than growing our own.
posted by hippybear at 8:22 PM on March 8 [9 favorites]


Okay I felt down a total rabbit hole digging into this and it's kind of bananas.
posted by missjenny at 1:01 AM on March 9 [3 favorites]


This isn't gender bias in our language, this is gender bias in our culture
On one hand, I agree. On the other, where do you think we get our language from?
posted by Cogito at 7:42 AM on March 9


That's an interesting thing to examine, isn't it? Does our language influence our thought patterns, or do our thought patterns influence our language, and how much of what feeds into the other when it comes to breaking out of previous negative cycles and attempting to live a better life?

I'd say, our language is thoroughly capable of losing the previous biases expressed in the culture without much bending or breaking to accommodate the new paradigm, and it's worthwhile we strive to make the change so it isn't something future generations have to strive for, as it will be built in to the language and culture they inherit.
posted by hippybear at 7:58 AM on March 9


"Does our language influence our thought patterns, or do our thought patterns influence our language..."

Yes

But "influence", not "control".

We still get to choose.

(Or just relax and go with the defaults/programming)
posted by aleph at 8:27 AM on March 9


@biogeo
>I had the same thought, but if so the same thing should be true for prompts starting with "women" and "men," which doesn't seem to be the case.

I wondered if a tokenisation difference might be responsible. If " female" weren't a token to GPT3 and got broken up as " fe" "male" while " women" stayed intact, that might cause this. But no, the demo shows that all of " male", " female", " men", " women" are single tokens, so scratch that.
posted by finka at 10:33 AM on March 9


« Older “The answer is, two books that take on God and...   |   Matilda Betham's Miniature Portraits: Artistic... Newer »


This thread has been archived and is closed to new comments