Bias? In my machine learning model? It's more likely than you think.
June 19, 2020 8:35 AM Subscribe
This approachable blog post from Microsoft Research summarizes this research paper by Swinger, De-Arteaga, et al. [pdf], which demonstrated that commonly used machine learning models contain harmful and offensive racial, ethnic, gender, and religious biases (e.g. associating common Black names with harmful stereotypes while associating common white names with innocuous or positive terms). These biases are harmful in themselves and may also lead to insidious discriminatory effects in software built on these models.
These biases were found in a model trained on news articles as well as models trained on the wider web. The three models examined in the paper (word2vec, fastText, and GloVe) are some of the most commonly used language models, and there is no reason to think that other models such as BERT and GPT-3 (recently on MeFi), which are usually trained on similar data, would not suffer from the same problems. In other words: racism in, racism out. And because society is racist, virtually any significant amount of input will result in the model encoding racist assumptions.
All is not lost, however! Work by Manzini, Lim, et al. shows that it is possible to automatically remove much of the racial, gender, and religious bias present in word embeddings without reducing the performance of common machine learning tasks. The source code is on GitHub.
Their work built on the 2016 gender debiasing work of Bolukbasi, Chang, et al. [pdf]
For data science, machine learning, and artificial intelligence researchers and developers, today is a good day to reflect on the harmful assumptions that may be built in to our tools and learn how we can actively resist them. Such "colorblind" models are not an instant, perfect solution to the problem of bias in machine learning, but they may be a critical foundational step.
These biases were found in a model trained on news articles as well as models trained on the wider web. The three models examined in the paper (word2vec, fastText, and GloVe) are some of the most commonly used language models, and there is no reason to think that other models such as BERT and GPT-3 (recently on MeFi), which are usually trained on similar data, would not suffer from the same problems. In other words: racism in, racism out. And because society is racist, virtually any significant amount of input will result in the model encoding racist assumptions.
All is not lost, however! Work by Manzini, Lim, et al. shows that it is possible to automatically remove much of the racial, gender, and religious bias present in word embeddings without reducing the performance of common machine learning tasks. The source code is on GitHub.
Their work built on the 2016 gender debiasing work of Bolukbasi, Chang, et al. [pdf]
For data science, machine learning, and artificial intelligence researchers and developers, today is a good day to reflect on the harmful assumptions that may be built in to our tools and learn how we can actively resist them. Such "colorblind" models are not an instant, perfect solution to the problem of bias in machine learning, but they may be a critical foundational step.
This is interesting. I've noticed that Google's voice search has gotten significantly better at "understanding" what I am asking, finding relevant results, and reading me the most relevant answer from a page. For example, if I ask "How many cars are there per capita in America" google answers "According to the Atlantic, there are 439 cars for every thousand Americans, meaning a little more than two people for every car."
I've sat here for a few minutes asking google questions using the info in the table, trying to get it to be racist. But I'm having a hard time with that. Not saying it's not happening - I bet they have rules about what words they will display audibly vs visually.
Finally, as someone in Human Resources, this really concerns me.
This means that people’s job recommendations might exhibit racial and gender discrimination through proxies that they list on their resumes (for example, sports played in high school).
I helped with a project to remove all identifying information from resumes before recruiters reviewed them. Now I'm wondering - is our society SO BIASED that it's impossible to truly do this? Corporations can't eliminate racism by using software, they must invest in policy changes.
posted by rebent at 9:05 AM on June 19, 2020 [3 favorites]
I've sat here for a few minutes asking google questions using the info in the table, trying to get it to be racist. But I'm having a hard time with that. Not saying it's not happening - I bet they have rules about what words they will display audibly vs visually.
Finally, as someone in Human Resources, this really concerns me.
This means that people’s job recommendations might exhibit racial and gender discrimination through proxies that they list on their resumes (for example, sports played in high school).
I helped with a project to remove all identifying information from resumes before recruiters reviewed them. Now I'm wondering - is our society SO BIASED that it's impossible to truly do this? Corporations can't eliminate racism by using software, they must invest in policy changes.
posted by rebent at 9:05 AM on June 19, 2020 [3 favorites]
RIRO: Racism In, Racism Out
posted by BungaDunga at 9:29 AM on June 19, 2020 [10 favorites]
posted by BungaDunga at 9:29 AM on June 19, 2020 [10 favorites]
As MeFi's own qntm says,
posted by Nonsteroidal Anti-Inflammatory Drug at 9:29 AM on June 19, 2020 [15 favorites]
* Machine learning is money laundering for bias- Twitter thread.
* Machine learning is money laundering for responsibility
* People do not launder their money by ACCIDENT
* Assign responsibility to humans
posted by Nonsteroidal Anti-Inflammatory Drug at 9:29 AM on June 19, 2020 [15 favorites]
I worked in machine learning fairness research and policy for a couple years and eventually decided it wasn't something I could do full time anymore. It is an endless game of whack-a-mole and in the end I felt the most impactful thing I could do was retire from AI/ML and talk about my reasons. It was a sacrifice; I have a master's degree in AI and years of experience, and it's work I truly love to do. But these models are being given responsibility for decisions with deep moral import and they are just not capable of making those decisions accountably.
It was wrenching to stop working on this stuff but every time I read a paper like this I'm reminded of how frustrating the work was. Just swimming upstream, every day, and constantly worrying that I'd signed off on something that was going to make things worse instead of better.
posted by potrzebie at 9:47 AM on June 19, 2020 [21 favorites]
It was wrenching to stop working on this stuff but every time I read a paper like this I'm reminded of how frustrating the work was. Just swimming upstream, every day, and constantly worrying that I'd signed off on something that was going to make things worse instead of better.
posted by potrzebie at 9:47 AM on June 19, 2020 [21 favorites]
Are there any consulting type organizations that software companies implementing ML can hire to evaluate current models and make recommendations for removing bias?
posted by czytm at 12:48 PM on June 19, 2020 [1 favorite]
posted by czytm at 12:48 PM on June 19, 2020 [1 favorite]
Are there any consulting type organizations that software companies implementing ML can hire to evaluate current models and make recommendations for removing bias?
I have questions about what "removing bias" means in this context, and I'm not technically competent to really ferret it out.
To follow an example in the post, "doctor" and "nurse" are professions with marked gender imbalances. So...what does an association mean in this context? Does that depend entirely on what it's being used for? On what meaning the association is assigned? How is that determined?
posted by schadenfrau at 1:44 PM on June 19, 2020 [1 favorite]
I have questions about what "removing bias" means in this context, and I'm not technically competent to really ferret it out.
To follow an example in the post, "doctor" and "nurse" are professions with marked gender imbalances. So...what does an association mean in this context? Does that depend entirely on what it's being used for? On what meaning the association is assigned? How is that determined?
posted by schadenfrau at 1:44 PM on June 19, 2020 [1 favorite]
To follow an example in the post, "doctor" and "nurse" are professions with marked gender imbalances. So...what does an association mean in this context? Does that depend entirely on what it's being used for? On what meaning the association is assigned? How is that determined?
There are occasions when one would want to use a model that reflected the world as-it-is. For example, imagine one were trying to train a computer to recognize dog-whistle language or more subtly biased language. Being able to ask "is this word or phrase notably associated with one race/gender/etc or another?" could be a key part of that.
But if, for example, a company uses a computer model to help filter through thousands of job applications to pick applicants to interview and that model (in effect) discriminates on the basis of race, gender, etc, then that could be a real problem (as well it should). The model would be reinforcing existing biases and imbalances.
One way to think of this is as an implicit-association test for artificial intelligence.
posted by jedicus at 2:18 PM on June 19, 2020 [3 favorites]
There are occasions when one would want to use a model that reflected the world as-it-is. For example, imagine one were trying to train a computer to recognize dog-whistle language or more subtly biased language. Being able to ask "is this word or phrase notably associated with one race/gender/etc or another?" could be a key part of that.
But if, for example, a company uses a computer model to help filter through thousands of job applications to pick applicants to interview and that model (in effect) discriminates on the basis of race, gender, etc, then that could be a real problem (as well it should). The model would be reinforcing existing biases and imbalances.
One way to think of this is as an implicit-association test for artificial intelligence.
posted by jedicus at 2:18 PM on June 19, 2020 [3 favorites]
Sorry, I realized I didn't really answer your question at all!
Language models are basically a way to condense a whole lot of statistics about word use. The exact methods vary, but these are some common ones:
* What words are frequently used in the same documents (e.g. within a sentence, a paragraph, a news article, a web page)
* What words are frequently used close together (e.g. within the same sentence or a few words apart)
* What words are used in similar ways (e.g. if you take sentences that look like "Albert Einstein was a _____" then you'll learn that words like "physicist", "genius", and "man" are related. Bias already!)
What this research shows is that these models learn, for example, to associate common Black names with criminality. For a model trained on news articles, this might be due to news articles that are basically police reports, which are in turn based on biased policing. But it might also learn to associate Black names with basketball. And it probably won't learn to associate Black names with, say, computer programming or investment banking.
Because of the way these models work, that means that it will make secondary associations: basketball becomes closer to crime and further from computer programming and investment banking. Water polo is far from crime and closer to computer programming and investment banking.
And because these models have no deeper grasp of language, if a model was trained on articles about bias and discrimination, it would still learn many of these same associations because it only understands statistics, not nuance or sentiment. The only way to avoid these associations is either to train it on extremely heavily curated and balanced input data (which is infeasible) or to remove the bias afterward.
Removing the bias means, essentially, taking pairs of words (black/white, man/woman), finding words for which the two are highly unevenly associated (doctor, nurse, etc), and then re-balancing the model so that the word pairs are equally close to the other words.
The paper discusses a few methods for this. One makes the associations exactly equal, but there are some cases where strict balance loses meaning. For example, while pregnancy is not exclusively (or necessarily) associated with women, it probably does make sense to have pregnancy-related words be closer to woman, female, mother, etc. And so the authors also discuss a "softer" method that reduces but does not completely eliminate the imbalance.
posted by jedicus at 2:42 PM on June 19, 2020 [6 favorites]
Language models are basically a way to condense a whole lot of statistics about word use. The exact methods vary, but these are some common ones:
* What words are frequently used in the same documents (e.g. within a sentence, a paragraph, a news article, a web page)
* What words are frequently used close together (e.g. within the same sentence or a few words apart)
* What words are used in similar ways (e.g. if you take sentences that look like "Albert Einstein was a _____" then you'll learn that words like "physicist", "genius", and "man" are related. Bias already!)
What this research shows is that these models learn, for example, to associate common Black names with criminality. For a model trained on news articles, this might be due to news articles that are basically police reports, which are in turn based on biased policing. But it might also learn to associate Black names with basketball. And it probably won't learn to associate Black names with, say, computer programming or investment banking.
Because of the way these models work, that means that it will make secondary associations: basketball becomes closer to crime and further from computer programming and investment banking. Water polo is far from crime and closer to computer programming and investment banking.
And because these models have no deeper grasp of language, if a model was trained on articles about bias and discrimination, it would still learn many of these same associations because it only understands statistics, not nuance or sentiment. The only way to avoid these associations is either to train it on extremely heavily curated and balanced input data (which is infeasible) or to remove the bias afterward.
Removing the bias means, essentially, taking pairs of words (black/white, man/woman), finding words for which the two are highly unevenly associated (doctor, nurse, etc), and then re-balancing the model so that the word pairs are equally close to the other words.
The paper discusses a few methods for this. One makes the associations exactly equal, but there are some cases where strict balance loses meaning. For example, while pregnancy is not exclusively (or necessarily) associated with women, it probably does make sense to have pregnancy-related words be closer to woman, female, mother, etc. And so the authors also discuss a "softer" method that reduces but does not completely eliminate the imbalance.
posted by jedicus at 2:42 PM on June 19, 2020 [6 favorites]
Are there any consulting type organizations that software companies implementing ML can hire to evaluate current models and make recommendations for removing bias?
No. There are a few people purporting to sell such services. They may do a superficial analysis that will catch the most embarrassing stuff. If someone tells you they can do a thorough job of it, don't believe them.
posted by potrzebie at 6:18 PM on June 19, 2020
No. There are a few people purporting to sell such services. They may do a superficial analysis that will catch the most embarrassing stuff. If someone tells you they can do a thorough job of it, don't believe them.
posted by potrzebie at 6:18 PM on June 19, 2020
I find your decision to step away from AI/ML really heartening, potrzebie. As someone with a hobbyist-level understanding of a lot of current models (I'm a software engineer; I do new media art; ML art is the Total Hotness;° plus like, I am a black person alive in the world), my only conclusion is that using ML systems for any task is unethical.
It's not just that using a statistical tendency to make a decision about a real person is a terrible way to be, but the fundamental use of machines to invest decisions with an air of objectiveness is just indefensible. It's better to say people are biased, check the outcomes, and then when the outcomes are biased take specific action to rectify that than to launder it through a machine and declare it unchangeable. People have proven so far they can only see machine decisions this way and, until that changes culturally (which I don't know if it is possible), engaging that system is a violence to humans.
°If anyone here is interested, I did a long art-manifesto on why I don't do ML art and I wish other people wouldn't either.
posted by dame at 2:28 AM on June 20, 2020 [6 favorites]
It's not just that using a statistical tendency to make a decision about a real person is a terrible way to be, but the fundamental use of machines to invest decisions with an air of objectiveness is just indefensible. It's better to say people are biased, check the outcomes, and then when the outcomes are biased take specific action to rectify that than to launder it through a machine and declare it unchangeable. People have proven so far they can only see machine decisions this way and, until that changes culturally (which I don't know if it is possible), engaging that system is a violence to humans.
°If anyone here is interested, I did a long art-manifesto on why I don't do ML art and I wish other people wouldn't either.
posted by dame at 2:28 AM on June 20, 2020 [6 favorites]
Wow. I was aware of things like AI image identifiers not recognising Black people, or identifying them as something other than human, but I had never imagined how deep and how subtle the problem with machine learning could be.
Also, how responsive are these techniques to people deliberately poisoning the word corpuses, doing something similar to Googlebombing? E.g., could someone seed some otherwise-innocuous words as text associated with strongly positive or strongly negative values, and thereby get the AI to miscategorise people using those words in a resume?
posted by Joe in Australia at 6:34 AM on June 20, 2020
Also, how responsive are these techniques to people deliberately poisoning the word corpuses, doing something similar to Googlebombing? E.g., could someone seed some otherwise-innocuous words as text associated with strongly positive or strongly negative values, and thereby get the AI to miscategorise people using those words in a resume?
posted by Joe in Australia at 6:34 AM on June 20, 2020
The phrase to research for that, Joe in Australia, is adversarial machine learning— there's lots out there.
posted by dame at 7:27 AM on June 20, 2020 [1 favorite]
posted by dame at 7:27 AM on June 20, 2020 [1 favorite]
Another the perfect illustration just showed up last night on Twitter:
Now look how it renders Barack Obama and Samuel L. Jackson.
(Scan the thread for other POC faces)
At least the cops won't be able to use it to identify BLM protestors...
posted by cheshyre at 10:53 AM on June 20, 2020 [2 favorites]
Given a low-resolution input image, Face Depixelizer searches the outputs of a generative model (here, StyleGAN) for high-resolution images that are perceptually realistic and downscale correctly.In plain language, it's supposed to sharpen blurry/low-res pictures of faces.
GitHub; Colab
Now look how it renders Barack Obama and Samuel L. Jackson.
(Scan the thread for other POC faces)
At least the cops won't be able to use it to identify BLM protestors...
posted by cheshyre at 10:53 AM on June 20, 2020 [2 favorites]
how responsive are these techniques to people deliberately poisoning the word corpuses, doing something similar to Googlebombing? E.g., could someone seed some otherwise-innocuous words as text associated with strongly positive or strongly negative values, and thereby get the AI to miscategorise people using those words in a resume?
"Poisoning" is exactly the right word. There's been a fair amount of research recently on the feasibility of intentionally poisoning models so that programs that use those models can be exploited. It is essentially a kind of backdoor, whereby the attacker subtly adjusts the model so that an innocuous-seeming combination of words or phrases will give whatever result the attacker wants. Here is one example from Kurita, Michel, et al.. Demonstration code is available on GitHub.
Adversarial machine learning is a closely related attack whereby, rather than poisoning the model itself, the attacker finds weaknesses in a normal model. The practical effect is similar: innocuous looking input produces an unexpected output that the attacker wants.
posted by jedicus at 10:58 AM on June 20, 2020 [1 favorite]
"Poisoning" is exactly the right word. There's been a fair amount of research recently on the feasibility of intentionally poisoning models so that programs that use those models can be exploited. It is essentially a kind of backdoor, whereby the attacker subtly adjusts the model so that an innocuous-seeming combination of words or phrases will give whatever result the attacker wants. Here is one example from Kurita, Michel, et al.. Demonstration code is available on GitHub.
Adversarial machine learning is a closely related attack whereby, rather than poisoning the model itself, the attacker finds weaknesses in a normal model. The practical effect is similar: innocuous looking input produces an unexpected output that the attacker wants.
posted by jedicus at 10:58 AM on June 20, 2020 [1 favorite]
I would like to gently push back against the hopelessness of machine learning and the idea that humans are always better decision makers. Assuming that you have an explainable, interpretable machine learning model (and in 2020 there is no excuse for not doing so), then a machine learning model offers several benefits over a human decision maker, at least in theory.
* You can ask the model how it made its decision and it will tell you. Human decision makers are often uncertain about their decision-making process and have a very strong tendency to ignore or downplay the effects of bias. Humans can also intentionally offer false explanations or try to offer the explanation that they think the asker wants to hear.
* A model can be tested objectively by asking it to review enormous numbers of near-identical cases, which can be artificially constructed to tease out the effects of bias. A model will give the same answer for the same input every time, making such testing feasible. Human decision makers get tired and their answers aren't always reliably repeatable.
* A model can be effectively debiased. Debiasing humans is infamously time-consuming, difficult, and often ineffective (Exhibit A: myself; Exhibit B: white people in general).
But that all takes work, and it's work that is virtually never actually done in practice right now. It's far more common for developers to download a pre-trained model and set it loose: no interrogation of how that model was produced in the first place, no introspection, no explanations, no interpretations, no checking for bias. ML researchers have only really started to push back on these problems in the past 5 years or so, and it's long overdue.
But I don't think we'll see a serious change in how ML is used in practice until there is a legal framework in place that holds companies fully accountable for using models that result in discriminatory results. Using such a model is equivalent to adopting a company policy that favors illegal discrimination, and it needs to be aggressively treated that way.
posted by jedicus at 11:45 AM on June 20, 2020 [3 favorites]
* You can ask the model how it made its decision and it will tell you. Human decision makers are often uncertain about their decision-making process and have a very strong tendency to ignore or downplay the effects of bias. Humans can also intentionally offer false explanations or try to offer the explanation that they think the asker wants to hear.
* A model can be tested objectively by asking it to review enormous numbers of near-identical cases, which can be artificially constructed to tease out the effects of bias. A model will give the same answer for the same input every time, making such testing feasible. Human decision makers get tired and their answers aren't always reliably repeatable.
* A model can be effectively debiased. Debiasing humans is infamously time-consuming, difficult, and often ineffective (Exhibit A: myself; Exhibit B: white people in general).
But that all takes work, and it's work that is virtually never actually done in practice right now. It's far more common for developers to download a pre-trained model and set it loose: no interrogation of how that model was produced in the first place, no introspection, no explanations, no interpretations, no checking for bias. ML researchers have only really started to push back on these problems in the past 5 years or so, and it's long overdue.
But I don't think we'll see a serious change in how ML is used in practice until there is a legal framework in place that holds companies fully accountable for using models that result in discriminatory results. Using such a model is equivalent to adopting a company policy that favors illegal discrimination, and it needs to be aggressively treated that way.
posted by jedicus at 11:45 AM on June 20, 2020 [3 favorites]
And a final note because I'm treading dangerously close to threadsitting: I 100% think that ML will only be a positive development for humanity if the ML community (academic researchers, corporate R&D, corporate executives, legislators, policymakers) includes people from all marginalized and oppressed groups at every level. We cannot trust, e.g., able-bodied straight cishet white men such as myself to do a good or even sufficient job.
posted by jedicus at 11:56 AM on June 20, 2020 [1 favorite]
posted by jedicus at 11:56 AM on June 20, 2020 [1 favorite]
I guess what I'm asking is what I can do at my company to make sure we're doing everything we can to avoid bias in our ML models. I'm not on the ML team and have no ML experience but I would like to be able to ask what steps we are taking and if we aren't or I don't feel their response is adequate be able to provide some sort of resources to help us move in the right direction.
We're a software company in the healthcare space and combatting implicit and explicit racial bias in health care is very much on our radar and has been and is being discussed but no idea of that extends to our ML models and team.
posted by czytm at 12:36 PM on June 20, 2020 [1 favorite]
We're a software company in the healthcare space and combatting implicit and explicit racial bias in health care is very much on our radar and has been and is being discussed but no idea of that extends to our ML models and team.
posted by czytm at 12:36 PM on June 20, 2020 [1 favorite]
°If anyone here is interested, I did a long art-manifesto on why I don't do ML art and I wish other people wouldn't either.
dame, there's something you said there that really struck me.
And so, this is the fundamental violence of the AI/ML machine: its insistence that we are not individuals but a type — that we are not all independent, valuable souls, but just a probability. The illusion of information is that it can know us by capturing all the externalities: where we are, who we are with, where we were born, where we fit within sociological categories — and its destruction is that because it can never capture the internal, it substitutes these externalities and claims that they can describe us in a meaningful way.
In a way, what ML often ends up doing is a monstrous hybrid of the modernist state and the traditional society.
The modernist state relentlessly measures and classifies people but in its cold majesty also makes a promise that it will record the same superficial things about all of us. It might care about us only in certain categories but at least it is knowable what those categories are. We can object that we do not wish for category X or attribute Y to be measured. (Or we can insist that they are).
The traditional society is smaller and the interfaces between people are not so smooth, lots of people know lots of things about each other but there is no system that sits outside "us" and knows everyone as a set of numbers. It's cozyer but also potentially more suffocating - great when your uncle happens to be a beloved local character, not so great when he's a notorious criminal.
What unregulated ML can do is combine the uncaring database of the modernist state with the arbitrary intrusion of a village full of gossips.
I do think that ML has valuable roles to play but:
-We'll get better results if we start off skeptical and require each use to prove itself than starting out believing that it's ok an only policing the most egregious.
-Not for assigning people to categories or making decisions about them
posted by atrazine at 1:17 PM on June 20, 2020 [1 favorite]
dame, there's something you said there that really struck me.
And so, this is the fundamental violence of the AI/ML machine: its insistence that we are not individuals but a type — that we are not all independent, valuable souls, but just a probability. The illusion of information is that it can know us by capturing all the externalities: where we are, who we are with, where we were born, where we fit within sociological categories — and its destruction is that because it can never capture the internal, it substitutes these externalities and claims that they can describe us in a meaningful way.
In a way, what ML often ends up doing is a monstrous hybrid of the modernist state and the traditional society.
The modernist state relentlessly measures and classifies people but in its cold majesty also makes a promise that it will record the same superficial things about all of us. It might care about us only in certain categories but at least it is knowable what those categories are. We can object that we do not wish for category X or attribute Y to be measured. (Or we can insist that they are).
The traditional society is smaller and the interfaces between people are not so smooth, lots of people know lots of things about each other but there is no system that sits outside "us" and knows everyone as a set of numbers. It's cozyer but also potentially more suffocating - great when your uncle happens to be a beloved local character, not so great when he's a notorious criminal.
What unregulated ML can do is combine the uncaring database of the modernist state with the arbitrary intrusion of a village full of gossips.
I do think that ML has valuable roles to play but:
-We'll get better results if we start off skeptical and require each use to prove itself than starting out believing that it's ok an only policing the most egregious.
-Not for assigning people to categories or making decisions about them
posted by atrazine at 1:17 PM on June 20, 2020 [1 favorite]
I would like to gently push back against the hopelessness of machine learning and the idea that humans are always better decision makers. Assuming that you have an explainable, interpretable machine learning model (and in 2020 there is no excuse for not doing so), then a machine learning model offers several benefits over a human decision maker, at least in theory
(a) I am familiar with these arguments (in fact some of my very good friends are AI ethicists)
(b) I find them bad
See, for instance:
It's better to say people are biased, check the outcomes, and then when the outcomes are biased take specific action to rectify that than to launder it through a machine and declare it unchangeable. People have proven so far they can only see machine decisions this way and, until that changes culturally (which I don't know if it is possible), engaging that system is a violence to humans.
ML works through categorization, which is unable to capture real nuance — it decides based on tendency, not fact. Creating, improving, and using datasets requires surveillance. There is no AI ethics that can undo this.
People want to be all neoliberal and look to regulation, worried they are going to be New Luddites, fulminating against electricity, on the wrong side of history. But still — we could just not do this.
posted by dame at 1:35 PM on June 20, 2020 [2 favorites]
(a) I am familiar with these arguments (in fact some of my very good friends are AI ethicists)
(b) I find them bad
See, for instance:
It's better to say people are biased, check the outcomes, and then when the outcomes are biased take specific action to rectify that than to launder it through a machine and declare it unchangeable. People have proven so far they can only see machine decisions this way and, until that changes culturally (which I don't know if it is possible), engaging that system is a violence to humans.
ML works through categorization, which is unable to capture real nuance — it decides based on tendency, not fact. Creating, improving, and using datasets requires surveillance. There is no AI ethics that can undo this.
People want to be all neoliberal and look to regulation, worried they are going to be New Luddites, fulminating against electricity, on the wrong side of history. But still — we could just not do this.
posted by dame at 1:35 PM on June 20, 2020 [2 favorites]
Further examples of whitewashing by the Depixelizer (Twitter thread)
posted by cheshyre at 4:30 PM on June 20, 2020
posted by cheshyre at 4:30 PM on June 20, 2020
Today in malevolence in AI: Facial recognition to 'predict criminals' sparks row over AI bias
posted by BungaDunga at 1:29 PM on June 24, 2020
posted by BungaDunga at 1:29 PM on June 24, 2020
« Older Sir Ian Holm, 1931-2020 | The Romanian Orphans are Adults Now. Newer »
This thread has been archived and is closed to new comments
posted by jquinby at 8:40 AM on June 19, 2020 [5 favorites]