What if Skynet just wanted to piss us off?
March 3, 2013 7:40 AM   Subscribe

Keep Calm and Understand the Process. The advent of algorithm-driven sales and product-on-demand delivery systems (think Cafepress, for one) can have some unexpected results when the output is not checked carefully enough.

A twitter outrage campaign cleared up this instance of good algorithms gone bad, but mostly it appears that the outraged did not understand how it happened.

(The company itself has offered a very classy, informative, and prominently-placed apology)
posted by ChrisR (114 comments total) 17 users marked this as a favorite
 
Wow, considering their business model, that's really the sort of apology one would hope would come out of this. Although, seriously, dude? You couldn't proof a list of 700 words?
posted by griphus at 7:48 AM on March 3, 2013 [6 favorites]


I was going to order these, along with some "Keep calm and Pillage" versions to outfit my Viking raiding party.
posted by 445supermag at 7:49 AM on March 3, 2013 [1 favorite]


My coding skills are rusty but I still bet I could program a way to avoid putting rape jokes on t-shirts in about 10 minutes.
posted by Cookiebastard at 7:50 AM on March 3, 2013 [7 favorites]


Also, not that we have to choose, but I'd take a hundred HEY THIS SHIT AIN'T COOL tweets over ten detailed blog posts explaining how it happened any day.
posted by griphus at 7:56 AM on March 3, 2013 [1 favorite]


This would be an excellent opportunity to collectively cancel the whole tired "Keep Calm And..." meme.
posted by Foosnark at 7:57 AM on March 3, 2013 [27 favorites]


The post explaining the t-shirt process is by MeFi Pete Ashton. As a side-observation, he's tracking readership (or link-following) of it.
posted by Wordshore at 8:04 AM on March 3, 2013 [2 favorites]


I imagine that people leaping onto Twitter to vent their outrage to the world aren't really the target market for "Keep Calm" merchandise in the first place.
posted by El Sabor Asiatico at 8:05 AM on March 3, 2013 [9 favorites]


No way. On my floor alone we'd probably lose a supporting wall if we tried to remove all of those posters.

Okay, that's really a good reason to end the damnable meme.
posted by ChrisR at 8:05 AM on March 3, 2013 [1 favorite]


Also, not that we have to choose, but I'd take a hundred HEY THIS SHIT AIN'T COOL tweets over ten detailed blog posts explaining how it happened any day.

Keep Calm and Read the Blog Post.
posted by PeterMcDermott at 8:10 AM on March 3, 2013 [5 favorites]


Isn't this the exact same algorithm that killed off all the aliens on Forbidden Planet?
posted by sexyrobot at 8:13 AM on March 3, 2013 [3 favorites]


Thanks for the credit Wordshore, but I'm hardly a "MeFite". Always intended to be but never found the right level of commitment. Have much love for this place though.

For what it's worth I'm going to write up the experience of having your blog go from a few hundred readers to 65,000 (at last count) later today.
posted by peteash10 at 8:15 AM on March 3, 2013 [5 favorites]


Keep Calm and Read the Blog Post.

I did! And if I didn't make it clear enough (or at all, I guess): I am glad that blog post exists and that someone is explaining it.
posted by griphus at 8:16 AM on March 3, 2013 [1 favorite]


Boy, that's a really interesting business model, though. Just throw, literally, random crap at the wall, and see what sticks. Trying hundreds of thousands of possible products can be done by one guy or gal with a cheap computer and an Internet connection. All it costs is a little time to build the system. Once that time is invested, the computer will spit out endless variations for free.

That's brilliant. He/she can sell things that have never existed, and never will until someone clicks Buy. No carry cost, no creation cost, just endless random iteration, carried on Amazon's infrastructure.

Honestly, I think the 'rape a lot' bit has to be the least interesting part of the whole idea. Pfft, big deal, a strange side effect to a program. Wonder how long it was in Amazon's systems before any human at all even knew it existed?
posted by Malor at 8:16 AM on March 3, 2013 [6 favorites]


Check out this FPP, Malor.
posted by griphus at 8:17 AM on March 3, 2013 [6 favorites]


(The company itself has offered a very classy, informative, and prominently-placed apology)

I am now further outraged by the repeated misapplication of "it's". In fact I am apostroplectic!
posted by George_Spiggott at 8:18 AM on March 3, 2013 [10 favorites]


You couldn't proof a list of 700 words?

Yeah, that's the part that just sort of boggles my mind. How hard is it to read through a list of 700 verbs?
posted by kmz at 8:23 AM on March 3, 2013 [1 favorite]


I don't believe their response, and if it is true (and I can believe people might), I'm for a salting of the earth.
Who produces shirts without proofing?
And then includes "rape" in that?

NO ONE SANE.
posted by Mezentian at 8:25 AM on March 3, 2013 [2 favorites]


Check out this FPP, Malor.

Man, yeah, fuckin' Idakoos. I (a) think this sort of biz model is brilliant for whatever it is, but (b) also find it sort of soul-sucking to contemplate in its almost total abdication of design or aesthetic sensibility, but (c) have a hard time worrying about that too much because merch is merch and I could as soon stage a protest by the Miniature Custom Vanity License Plates kiosk in any given giftshop, and (d) I totally bought a dumb VAGUE INDIVIDUAL shirt from Idakoos anyway a while back because I find questionable juxtapositions from algorithmically generated text pretty funny and charming.

But as far as I know Idakoos was smart enough not to e.g. add "rape/raping" to its verb list. Think your biz model through, folks. With great combinatoric space comes some basic fucking common sense responsibility, etc.
posted by cortex at 8:25 AM on March 3, 2013 [4 favorites]


Malor, that's sort of where I came from on this too. I don't really care about the outrage -- I think it largely stems from a lack of understanding of the way these things are done -- but the fact that it's possible for products to "exist" without actually, you know, existing is fascinating.

I actually think it's a good thing; describe what you can create, but don't expend resources speculatively; less waste.
posted by ChrisR at 8:28 AM on March 3, 2013 [1 favorite]


Who produces shirts without proofing?

A company whose whole business model involves using word list combination and print-on-demand to avoid the traditional proofing process associated with producing fixed-design merchandise. The proofing is left to the customer; they either think given randomly generated shirt is worthy of purchasing or not.

And then includes "rape" in that?

But, yes, that's the bit where someone was an idiot. Don't put words that are going to upset customers and bystanders into the list of words that can show up on your merch.
posted by cortex at 8:29 AM on March 3, 2013 [3 favorites]


It was depressing seeing the #NotBuyingIt tweets go past about this. I support that campaign all the time but usually it's used against products or advertising where actual humans approved. It was obvious (as a technologist) that this wasn't a case where a human really approved it. Yes, a human approved the word list and didn't filter it appropriately but that's a mistake orders of magnitude less bad than GoDaddy ads or sexy teen t-shirts at a department store. But it seemed like @RepresentPledge pushed it really hard. :(
posted by R343L at 8:30 AM on March 3, 2013 [1 favorite]


Imagine the possibilities once 3-d printing and truly auto autocad are real.

"Yeah, I know you bought it on Amazon, but what is it?"

"A solar powered quesadilla baker with a touch-tablet interface and lith-ion battery backup."
posted by notyou at 8:32 AM on March 3, 2013 [22 favorites]


And just in case folks don't know about it, here's a background for the #NotBuyingIt I mentioned. A documentary came out last year about mis-representation of women in media and it's effects on our society. Their main social media presence is around pressuring businesses to not re-enforce stereotypes about gender roles. A main way that's done is people seeing an awful product or ad and posting about it with the #NotBuyingIt tag. @RepresentPledge retweets some etc.
posted by R343L at 8:35 AM on March 3, 2013


"It's a nine hundred wheel cycle."

"It's a desk that folds into a stuffed bear."

"It's a transparent lampshade."

And so on.
posted by notyou at 8:36 AM on March 3, 2013 [9 favorites]


Seems like the basic assumption behind this all was that just having a bad idea is cost-free.

"I mean, yeah, implementing a bad idea is costly. It's expensive to print shirts that nobody wants. It's expensive to pay a designer to design shirts that nobody wants. But if we can avoid or defer those expenses, then surely it couldn't cost us anything just to have generated an idea for a shirt that nobody wants, right?"

In hindsight, clearly a false assumption. But there you go.
posted by Now there are two. There are two _______. at 8:37 AM on March 3, 2013 [5 favorites]


Imagine the possibilities once 3-d printing and truly auto autocad are real.

Unfortunately, the parallel to the current situation would be more like "Lets advertise that we'll make Real Dolls based on popular figures" and having a bot skim google news for models...and then have rape victim and missing child versions advertised.
posted by 445supermag at 8:42 AM on March 3, 2013 [10 favorites]


Who produces shirts without proofing?

A company which tries to reduce labor costs to the absolute bare minimum, or even zero if they could get away with it. As many do. In this case "we can get our staff costs down to near zero, whether we have 1 design or 100. So let's have half a million!"

Speaking of proofing, talking to several long-term proofreaders recently indicates just how expendable many facets of the contemporary public industry think they are. The tsunami of low-quality, unproofed, ebooks clogging up online stores being one result of this.
posted by Wordshore at 8:44 AM on March 3, 2013 [2 favorites]


This has parallels in publishing; I had a conversation with (mefi's own) Charles Stross once when I tried to directly purchase a book from him, where he made the clear point that there was too much value in his publisher to cut them out of the sales of his books by going direct to the customer (me). It was solid reasoning; proofreaders, editors, and (to a lesser extent) distributors still add real value to the process.
posted by ChrisR at 8:47 AM on March 3, 2013 [1 favorite]


I eagerly await the legislative version because policy wonks are so much better than twitter at distinguishing between good and bad ideas.
posted by jeffburdges at 8:48 AM on March 3, 2013 [3 favorites]


There was no "Him" included in the random words which was the kicker for my irate feminist friends. Proof of mysogeny apparently.
posted by Callicvol at 8:51 AM on March 3, 2013


The explanation is trivial to anyone who has ever coded up any kind of random text generator or considered doing so, but it misses a key point.

From the 'apology': This was culled from 202k words to around 1100 and ultimately slightly more than 700 were used due to character length and the fact that I wanted to closely reflect the appearance of the original slogan graphically.

This implies very very strongly that the ~700 words used were indeed proofread and specifically chosen. There are more than 700 short verbs in the English language. Other criteria were surely used to select the ones posted to Amazon.

That is to say, a decision was made at some point to keep the word 'rape' in there, knowing that it would be munged with 'on', 'her', and so on.

I have seen so far neither an admission of nor apology for that.
posted by motty at 8:53 AM on March 3, 2013 [7 favorites]


Yeah, that's the part that just sort of boggles my mind. How hard is it to read through a list of 700 verbs?


In reality, the question should be "How hard is it to realize that a list of 700 random verbs needs proofreading?"

Hard enough, apparently.


Unknown unknowns are what keep me up at night.


On preview:
This implies very very strongly that the ~700 words used were indeed proofread and specifically chosen.

I read that to mean that the words were computationally selected for shape and length that was similar to the word "carry."
posted by Nonsteroidal Anti-Inflammatory Drug at 8:58 AM on March 3, 2013 [5 favorites]


Also, not that we have to choose, but I'd take a hundred HEY THIS SHIT AIN'T COOL tweets over ten detailed blog posts explaining how it happened any day.

I don't know. I would hope most people can realize how awful those shirts are, but a detailed description of how this happened points out a serious problem in one of our 21st C love affairs -- that technology can fix things. Because (I suspect) of the experience of the Industrial Revolution and the first couple of decades of the Information Age, people have got the idea that you can always save money with new technologies and that technologies are always becoming smart enough to take over new jobs. And this is not so. For the first part, I think we have gotten to the point where technologies expand reach and allow new functions, but any savings are made up for by costs elsewhere (IT people, equipment upgrades, etc).

For the second part, there are loads of things where user training is far more effective than technological fixes. The entire internet searching industry is based on the idea that a machine can give an untrained person acceptable results. Which is true, as long as that person is only looking for things that the interface wants to give them. Once they stray off the assumed path that informed the interface design, the untrained user quickly gets lost and has no tools to understand how it happened or how to make it better.

To move back to the point at hand -- great, this company has automated the slogan production business, and that saved them some money. However, they have generated a problem for themselves because no machine can double check slogans for offensiveness. Obviously, you could prune your dictionary of obvious trouble words, but there are plenty of offensive slogans that can be created out of inoffensive words, and no machine is going to catch them all. Because they are just machines. We are not anywhere near the point where human attention can be removed from many common activities, no matter how much we might wish it was so.

However, unless Amazon drops this entire class of product, even if this company gets too much negative attention, there will be many more waiting (possibly run by the same guy). I suppose Amazon could charge a small posting fee for each item, which would make the process less attractive, but no doubt there is a way to game that, too.
posted by GenjiandProust at 9:04 AM on March 3, 2013 [3 favorites]


You know that a meme is really mainstream when even skynet is onto it.
posted by Tom-B at 9:18 AM on March 3, 2013


This shirt design was not funny in the least.

But if there's a picture of the put-upon Cafepress dog model with that shirt design pasted on his back, now that would be funny.
posted by Countess Elena at 9:23 AM on March 3, 2013


Really, something similar to this is how the world is going to end isn't it? I don't mean a careless random word generator gone unchecked, but some innocuous algorithm-driven sub system somewhere that doesn't specially exclude $_ disastrous result.

"What!? Didn't someone code the asteroid propulsion module to avoid the moon? Shit."
posted by edgeways at 10:38 AM on March 3, 2013 [2 favorites]


This implies very very strongly that the ~700 words used were indeed proofread and specifically chosen. There are more than 700 short verbs in the English language. Other criteria were surely used to select the ones posted to Amazon.

That is to say, a decision was made at some point to keep the word 'rape' in there, knowing that it would be munged with 'on', 'her', and so on.


I worked at an ad agency right about the time "customer generated content" was starting to become the hot new thing. Every client wanted some sort of campaign that involved Joe Public submitting pictures, ideas, slogans, etc. that reflected their love for some stupid brand. As one would expect, we spent a lot of time figureing out how to keep "offensive material" from ruining the campaign. Despite having teams of people spending hundreds of man hours coming up with every permutation of "Nestle Quik sucks a bag of dicks", blacklisting any word that might even remotely sound bad, and devising all sorts of realtime moderation schemes, something would always slip through.

Usually it was something minor, and more often than not it was client offense we had to deal with, and not the general public. I can say from experience, that if you're manually culling a large word list based on specific criteria, it's easy to miss a single word that narrowly misses your filter. I would group it with the phenomenon that makes a regular word look like gibberish if you focus on it too hard.
posted by billyfleetwood at 10:53 AM on March 3, 2013 [5 favorites]


Learning about whole catalogs of algorithmically generated products just reminds me that whenever I'm buying something it's worth shopping around to see if I can get the raw ingredients more cheaply and make it myself. I'm willing to pay extra for someone else to make it for me, but I want to know how much of a margin they're getting before I decide.

I also kind of want to find his supplier so that I can code up a competing catalog with each item priced one penny lower than his. And another catalog with each item priced $2 more, that will automatically relay the orders to his catalog for fulfillment.
posted by ceribus peribus at 11:00 AM on March 3, 2013 [3 favorites]


I've begun to suspect there are algorithmically-generated recipes online. It's kind of a shame, I used to be able to think "I've got green peppers and eggs that I need to use," and search for "green pepper eggs recipe" for ideas, but now they've gone all mad-lib fill-in-the-blanks with whatever ingredients they think are popular search terms, and you have no way to know if this recipe has ever met human taste-buds before.
posted by RobotHero at 11:12 AM on March 3, 2013 [13 favorites]


""It's a transparent lampshade.""

People buy transparent lamp shades all the time.
posted by Mitheral at 11:28 AM on March 3, 2013




That's a good article nertzy. I liked this part:

Amazon isn’t a store, not really. Not in any sense that we can regularly think about stores. It’s a strange pulsing network of potential goods, global supply chains, and alien associative algorithms with the skin of a store stretched over it, so we don’t lose our minds.

It definitely captures the feeling any time I go on Amazon on a whim for some small thing I need and it's sent to me 2 days later from across the world. Amazon Prime, indeed.
posted by codacorolla at 12:29 PM on March 3, 2013 [6 favorites]


While the fact and implementation of algorithmically generated products is interesting, the article has an aura of "Let me explain in detail how these people don't care about you.".

That's probably because I hold to a sort of Sarbanes-Oxley principle for robots and other automata, as a replacement for Asimov's Three Laws. I think that, barring any other contracts or agreements, the person who clicks the button is responsible for all actions of their machine. The responsibility can be transferred (think employment contract), or shared (think insurance), but it doesn't go away.

This idea probably comes from being the guy who is asked how to do something Unixy and responds "Are you sure you want to do that? You probably don't want to do that?". Then I shrug, type the commands at the prompt, and then stand up and say "I still don't think you should do that, but if you still want to, you push the Enter key."
posted by benito.strauss at 12:37 PM on March 3, 2013 [4 favorites]


That's brilliant. He/she can sell things that have never existed, and never will until someone clicks Buy. No carry cost, no creation cost, just endless random iteration, carried on Amazon's infrastructure.

It also destroys the value of search engines and category browsing to find new, useful products; since now consumers have to wade through 10,000,000 items of algorithmically generated dreck.
posted by cosmic.osmo at 12:44 PM on March 3, 2013 [3 favorites]


I could easily believe that, by the time he got to the Rs in a list of seven hundred verbs, which he was culling based on the shape of the letters, he was long past thinking about what words actually mean.
posted by reprise the theme song and roll the credits at 1:02 PM on March 3, 2013 [2 favorites]


Although Charles Stross was mentioned above, noone's mentioned that this is exactly the sort of thing that the beginning of Accelerando is about.

While rape tshirts are in decidedly poor taste, generating 500,000 listings on amazon that cost nothing, and then waiting to see which ones sell is in decidedly awesome taste (if your taste is new business models). Even if it doesn't work; even if editors and publishers and designers turn out to be critical to the tshirt selling business; this is still a great idea.

And if you think filtering words from a list is the path to success, I am wondering how you didn't see the "Apple Filters Email" boondoggle from just this week.
posted by Phredward at 1:05 PM on March 3, 2013 [4 favorites]


since now consumers have to wade through 10,000,000 items of algorithmically generated dreck.

Thank you for explaining why I have never had a single order for the fish nose yowl suppressors I've been selling for over a year now.
posted by George_Spiggott at 1:08 PM on March 3, 2013 [4 favorites]


a detailed description of how this happened points out a serious problem in one of our 21st C love affairs -- that technology can fix things

Using this case as a stand-in for all technology is probably not the greatest idea. Either the guy who implemented this is a pretty bad programmer or he did it in a slap-dash fashion (or both), because I can tell you as someone who writes code for a living that this sort of thing is not new and he should've seen it coming a mile away. I don't know if technology can fix things, but bad technology certainly can't.
posted by axiom at 1:10 PM on March 3, 2013 [1 favorite]


It definitely captures the feeling any time I go on Amazon on a whim for some small thing I need

My mother died in December, and we have been trying to get Dad online and hooked up; Mom hated and feared technology and he dutifully respected her prejudices for 50 years. Accordingly, on a visit I found novels by a couple of his favorite authors which he didn't know existed and couldn't get locally; I ordered them for him from Amazon. Four hours later, in a completely separate transaction, I ordered a USB floppy drive so he could transfer data from his old DOS/Win 3.1 machine to the newer machine Mom bought for games and which is now his internet portal.

I was rather astounded when the two orders arrived in one box.
posted by localroger at 1:10 PM on March 3, 2013 [1 favorite]


On the whole, I find the notional existence of this product less surprising than, for example, King Dedede Drooling While Eating (NSFW and very much not what it says on the tin). That range of products was sold through sears.com, and according to the photos on ED, at least a few of them actually got purchased and manufactured before they pulled the listing.
posted by reprise the theme song and roll the credits at 1:25 PM on March 3, 2013


I would estimate that fully 90% (percentage made up) of all major technology-related problems are ones that someone would say they should have seen "coming a mile away". It turns out people repeatedly make the same mistakes others do. This one was done as a quick response to an earlier outrage (trademarking of the original slogan). Is it really surprising they did a shoddy, ill-thought out job?
posted by R343L at 1:27 PM on March 3, 2013


I've begun to suspect there are algorithmically-generated recipes online.

Yes, there are. And they're hilarious.
posted by reprise the theme song and roll the credits at 1:31 PM on March 3, 2013 [1 favorite]


Might be neat to do this for rapbot.
posted by Ad hominem at 1:32 PM on March 3, 2013


I could easily believe that, by the time he got to the Rs in a list of seven hundred verbs, which he was culling based on the shape of the letters, he was long past thinking about what words actually mean.

posted by reprise the theme song and roll the credits at 1:02 PM


You're assuming that the culling would be done by eye, rather than that being another algorithm.
posted by RobotHero at 1:37 PM on March 3, 2013


You're assuming that the culling would be done by eye, rather than that being another algorithm.

Not really. I was responding to people earlier in the thread who were assuming the opposite. I'm not sure how likely either possibility is (though my gut feeling is that such an algorithm would be far more sophisticated than anything else that's been on display here).

I was just trying to briefly describe a scenario where he might at some point have looked straight at that word, and let it through.
posted by reprise the theme song and roll the credits at 1:54 PM on March 3, 2013


I don't think it needs to be particularly sophisticated, I think it's just going, "Is this word between this long and this long?"
posted by RobotHero at 2:01 PM on March 3, 2013


I assume his "algorithmic culling" simply took the list of verbs, calculated how big they would be when rendered in the t-shirt font he was using, and excluded the ones that didn't fit into the spot set aside for that word. The ones that made the cut ended up being 5-6 characters or less. This "algorithm" probably took one or two lines of code at most.
posted by ceribus peribus at 2:02 PM on March 3, 2013


Yeah, but as motty pointed out:

From the 'apology': This was culled from 202k words to around 1100 and ultimately slightly more than 700 were used due to character length and the fact that I wanted to closely reflect the appearance of the original slogan graphically.

This implies very very strongly that the ~700 words used were indeed proofread and specifically chosen. There are more than 700 short verbs in the English language. Other criteria were surely used to select the ones posted to Amazon.

posted by reprise the theme song and roll the credits at 2:06 PM on March 3, 2013


How close are we to Charles Stross' algorithmically generated lawsuits? Could you hook a 3D printer up to Borderlands and randomly generate guns?
posted by Charlemagne In Sweatpants at 2:25 PM on March 3, 2013 [1 favorite]


I beleive that is how the patent process works.
posted by Artw at 3:07 PM on March 3, 2013 [2 favorites]


So, just want to point out to folks thinking there's an easy technical solution -- some list of bad words or whatever -- that, well, there isn't.

The history of trying to write software to keep people from being able to say/publish "bad words" is full of failures. It starts with the Scunthorpe Problem, which at least has technical solutions but still crops up from time to time. It continues on into deeper issues with having to forbid not just the words themselves, but also slang terms for the words, close rhymes for the words, and eventually non-text ways of communicating the words.

One of my all-time favorite tales of failure is this story, which hits on several variations including using a non-chat interface to nonetheless "chat" and communicate dirty words, and abusing even a fixed list of pre-selected words and sentence fragments to do dirty things. Choice quote:
We spent several weeks building a UI that used pop-downs to construct sentences, and only had completely harmless words – the standard parts of grammar and safe nouns like cars, animals, and objects in the world.

We thought it was the perfect solution, until we set our first 14-year old boy down in front of it.

And remember that these were smart programmers, tasked specifically with preventing that, and they got beat by little kids. Adults who really want to find a way around whatever filter you come up with are going to do an even better job.
posted by ubernostrum at 3:11 PM on March 3, 2013 [6 favorites]


Yep. If you search for "sext" on Twitter, for instance, you can see a million examples of otherwise innocuous phrases that seem to have a dirty meaning when preceded with that word.

For example, the one I posted yesterday:

"Battlestar Galactica sext: 'The Arrow of Apollo will open the Tomb of Athena.'"

Or even better, President Teddy Roosevelt's motto as a sext:

"Sext: Speak softly and carry a big stick"

It's a huge problem.

[[snickers]] "huge problem"
posted by limeonaire at 3:33 PM on March 3, 2013 [2 favorites]




The history of trying to write software to keep people from being able to say/publish "bad words" is full of failures.

This is very true, and is why with some effort you can play a dirty hand in Apples to Apples. The power of a combinatoric domain is how many final products you can get from a relatively small set of source pieces; the downside is that when you create a domain larger than you could possibly thoroughly check, you cannot know that there won't be odd edge cases where two innocuous source elements combine to create something gross or racist or somehow offensive. (If XCOM's nickname generator included "Dirty" as a possibility, it'd be possible to end up with a solidier accidentally named e.g. Ricardo 'Dirty' Sanchez.)

That said, it's not nearly as difficult to recognize a fundamentally volatile source element in a scheme like this; "rape" is not one of those words that accidentally takes on a charge only in rare happenstance combination. The difficulty of thoroughly offense-proofing a combinatoric system should not be understated but this is not a very good example of why.
posted by cortex at 4:26 PM on March 3, 2013 [3 favorites]


"rape" is not one of those words that accidentally takes on a charge only in rare happenstance combination.

Fields of rape. "There must have been hundreds upon hundreds of acres of these beautiful flowering fields between Paris and the Loire."
posted by 445supermag at 4:56 PM on March 3, 2013 [2 favorites]


Also, not that we have to choose, but I'd take a hundred HEY THIS SHIT AIN'T COOL tweets over ten detailed blog posts explaining how it happened any day.

Depends on whether you want to be outraged or learn from the outrage so you can prevent it from happening again.

Also, the writer doesn't make his living participating in twitter shaming.
posted by cjorgensen at 5:01 PM on March 3, 2013 [2 favorites]


Yes, and rapeseed oil and so forth. But "the generally highly-charged word 'rape' can plausibly occur in non-problematic contexts" is a very different thing than "the generally non-problematic string 'x' can plausibly occur in highly-charged contexts" in terms of risk management.
posted by cortex at 5:01 PM on March 3, 2013


This is very true, and is why with some effort you can play a dirty hand in Apples to Apples.

At that point it Pokemon-Evolves into a CAH.
posted by Artw at 5:02 PM on March 3, 2013


You also have to consider the cost of a false positive. If I can't mention rapeseed by name in an email, that's going to be a bigger frustration than if I can't find a T-shirt proclaiming how much I love rapeseed.
posted by RobotHero at 5:24 PM on March 3, 2013


(In a weird moment of synchronicity, a 9th grade teacher wrote to me this afternoon to ask about using my William Carlos Williams Generator in his classroom, and we just coordinated so that he could send me a slightly redacted version of the original wordlist to use at an alternate link, to help reduce the likelihood of inappropriate words showing up in front of a bunch of fifteen-year-olds and possibly other faculty. It wasn't super dirty to start with, but I think "prostitute", for example, got pulled. The guy understands that this won't protect him from the possibility of a weirdly giggle-worthy juxtaposition or anything, but at least it's a bit of sane-making due diligence. Common sense risk management.)
posted by cortex at 5:51 PM on March 3, 2013 [2 favorites]


Put a Jesus fish on the shirt instead of a crown and small text at the bottom "By Order Sodom Town Council Gen 19:30" Fixed?
posted by humanfont at 6:22 PM on March 3, 2013


The history of trying to write software to keep people from being able to say/publish "bad words" is full of failures.

Oh god. Flashback. A while back I worked for a site that contracted with Siemens' medical division to provide a little online almost-Potemkin-village on healthcare policy. For a while, Siemens wanted us to make a popup appear any time someone clicked a link that would take them off site (so Siemens could warn them that it had nothing to do with where they were going). We convinced them that it was a weird, stupid idea (especially since they were paying us money to act like we were independent of them) so they dropped it.

But something in the corporate heart there is that cannot resist the urge to control something, so their next thing was that they wanted us to hook up a filter to our comment and content systems, so when people contributed content that mentioned a Siemens mark or made a comment that mentioned such a mark, a little bell would ring at Siemens HQ so a corporate image harpy could swoop in and assess the situation.

They sent us a spreadsheet of words we were to monitor, demanding the special bell for each word. Among them, hundreds of product lines named things like "Ability," "Change," "Clear" or "Regulate." Just shy of 1,000 words.

I gave them exactly what they asked for, fully expecting that after a week of my anti-900-words-you-almost-can't-go-a-day-without-saying battlestation being fully operational, they'd read its literal implementation with the exact sneer I had on my face as I wrote it demand I make it stop. Nope. They loved it. Still in use, apparently.
posted by mph at 6:31 PM on March 3, 2013 [5 favorites]


> It also destroys the value of search engines and category browsing to find new, useful products; since now consumers have to wade through 10,000,000 items of algorithmically generated dreck.

The next step, and maybe not totally implausibly, ought to be a tail-trimmer: culling listings of the products with the fewest page hits. Not because it's cheaper to have five thousand print on demand teeshirts than five hundred thousand (it isn't), but because the few things that prove to be sellable are going to be more findable (and sell even more) when they're not hidden in a torrent of nearly-undifferentiated product that will never sell.
posted by ardgedee at 7:01 PM on March 3, 2013


Also, not that we have to choose, but I'd take a hundred HEY THIS SHIT AIN'T COOL tweets over ten detailed blog posts explaining how it happened any day.

I would buy a shirt in the style of the 'keep calm' things that just said "HEY THIS SHIT AIN'T COOL". Maybe that makes me part of the prolbem.
posted by lumpenprole at 10:08 PM on March 3, 2013


Is it so very, very hard to believe that, out of a list of 700 words, he might have missed one?
posted by Malor at 10:16 PM on March 3, 2013 [1 favorite]


I always mess up some mundane detail!
posted by dirigibleman at 10:50 PM on March 3, 2013 [2 favorites]


Oh, I missed this:

griphus: Also, not that we have to choose, but I'd take a hundred HEY THIS SHIT AIN'T COOL tweets over ten detailed blog posts explaining how it happened any day.

Yay, substanceless outrage over measured analysis! Bite-sized chunks of rage are so very much more important than understanding the world around us.

I'd trade every outraged tweet ever made for this one article.
posted by Malor at 12:18 AM on March 4, 2013 [3 favorites]


How was the outrage substanceless? Someone was selling a shirt telling people to "rape a lot".
posted by benito.strauss at 1:51 AM on March 4, 2013 [1 favorite]


It was substanceless because COMPUTERS were selling the shirt, not 'someone'. Most likely, no human had any idea that the shirts even existed as a possibility until a potential customer noticed them on Amazon.

Man, you really, really missed the point of the analysis.
posted by Malor at 2:27 AM on March 4, 2013 [1 favorite]


It was substanceless because COMPUTERS were selling the shirt, not 'someone'.

Well, but this is flatly untrue. I'd agree that it's less blameworthy to be the person who sets up a system allowing these shirts to be sold than to be a person who actively thinks it would be a great idea to sell these shirts, but the system didn't come into being without human involvement. If you don't have the talent or presence of mind to stop your algorithmic t-shirt-creating system from creating t-shirts like this, you're asking for outrage, and it's not substanceless.

Indeed, an addendum to the linked post specifically points out that the author didn't intend to excuse the t-shirts, only to explain how they came into being.
posted by oliverburkeman at 2:39 AM on March 4, 2013 [1 favorite]


cortex: Man, yeah, fuckin' Idakoos.

This is slightly off topic, but after the last thread it became apparent that most of the Idakoos urls are fluff, and it's only the terminal number that really matters. This means you can put together a javascript bookmarklet to grab random shirts. Something like
javascript:location.href="http://www.idakoos.com/x,"+(Math.floor(Math.random()*1974970)+1)
If they weren't so expensive, I'd order randoms by the dozen.
posted by curious.jp at 3:35 AM on March 4, 2013 [1 favorite]


Re the question of why he didn't spot offensive words in the 700-word list... from his published apology, it seems as if generating these shirts was itself the equivalent of an outraged tweet or a hurriedly-dashed off blog post, protesting the trademarking of the original phrase. Rather than just write a tweet or a 500-word rant, he wrote a few lines of code to generate variants automatically from a dictionary list, fed the results to Amazon via established channels (his online store), and forgot about it, the same way you'd forget about yesterday's tweet or blog post. Due diligence would apply to something you seriously intended to sell, but the point of generating these particular designs wasn't to sell the results, it was to make a point in passing about "Keep Calm" and trademark law. He probably wrote a protest tweet or blog post at the time to go with it. Once the moment had passed, he wouldn't have gone back to check the word lists or whatever the algorithms had generated any more than most people would go back to check a year-old tweet for typos: i.e. not until something prompts you to do so. Something like a storm of outraged tweets...
posted by rory at 3:35 AM on March 4, 2013 [2 favorites]


but the system didn't come into being without human involvement.

Look, it's a mixture of random numbers and a dictionary. If you don't get that there was no human involved in the actual creation or posting for sale of this shirt, then you're just missing the point. This is one of, apparently, hundreds of thousands of random offerings. It's like he built a machine that made random shapes, and now everyone's getting all pissed off because one of them looks like a penis.

All these useless tweets and teapottery is anger without substance, ascribing intentionality to a purely stochastic process. There's nobody there to be angry at. No harm was done, no agenda was pushed, no nothing.

It's just a machine, picking words out of a bin and sticking them on shirts. And not even real shirts, imaginary ones. You're being angry at random numbers.

So, yes, if you're upset about this, if you're actually on the side of the anguished tweetmafia, I think you're defending the indefensible. Knowing what you now know, if you can still think that there's any real validity to that argument, it's something wrong with you, not with him.

Be upset all you want, but you might as well declare your eternal disdain for an automated wheat thresher, because it accidentally spelled out 'fuck' in the crops.
posted by Malor at 4:35 AM on March 4, 2013 [4 favorites]


Interestingly, my quick and dirty script to pull all four and five letter words containing both 'r' and 'a' from /usr/share/dict/words came up with a list of 708. I just proofed it. Didn't take long.

Is it so very, very hard to believe that, out of a list of 700 words, he might have missed one?

Is it so very, very hard to believe that, out of a list of 700 words, a guy might have seen 'rape' and thought, 'yeah, actually that one might sell a few'.

What this actually comes down to is rape culture. We will at this point never know whether the guy a) didn't proof his word list, b) did proof his word list but missed 'rape', or c) did proof his word list and deliberately chose to leave 'rape' in.

It doesn't matter.

What does matter is the huge amount of good faith that a bunch of people here and elsewhere seem to be ready to give a guy who ultimately, one way or another, inadvertently or otherwise, caused rape advocacy T-shirts to be offered for sale on Amazon.

That's rape culture for you right there.
posted by motty at 5:20 AM on March 4, 2013 [1 favorite]


> If you don't get that there was no human involved in the actual creation or posting for sale of this shirt, then you're just missing the point.

There was a human involved in the actual creation and posting for sale of this shirt. He formulated the template and he selected the word list, or at least designed the process by which the word list is selected. It's his responsibility for what comes out, because he has to anticipate offensive words beyond the simple list of what can't be used on the radio.
posted by ardgedee at 5:33 AM on March 4, 2013 [1 favorite]


Suddenly I want to build an automated wheat thresher that carves naughtiness across the whole of Nebraska.
posted by aramaic at 6:14 AM on March 4, 2013 [1 favorite]


What does "good faith" even mean in this context? I don't personally think he meant to produce "rape a lot" or "hit her" T-shirts, but he did, and he's suffered for it: he just had a terrible weekend and his T-shirt business will no doubt suffer. He has been punished, and won't do it again.

Is it so very, very hard to believe that, out of a list of 700 words, a guy might have seen 'rape' and thought, 'yeah, actually that one might sell a few'.

Do you mean some guy, or this guy? Because from his mortified response to the whole thing, yes, it is "very, very hard" to believe that this guy included it deliberately. I don't believe he even proofed the word list. That was his mistake.

If you mean some guy, it depends whether you believe that any guy would make so light of rape, and be so oblivious to social mores, that he would actually produce these T-shirts and expect anyone to wear them. I've never seen anyone wear a "rape advocacy T-shirt" in my life. That doesn't mean I can't imagine the possibility (we all can, now), but it is very, very hard to believe.

Surely the point about rape culture is that it's insidious, not blatant.
posted by rory at 6:33 AM on March 4, 2013 [4 favorites]


Be upset all you want, but you might as well declare your eternal disdain for an automated wheat thresher, because it accidentally spelled out 'fuck' in the crops.

I'm very much not in the upset camp on this and understand the argument you are trying to make, but this analogy would be more on point if someone had decided to use the thresher attachement shaped like the word "fuck" without thinking about thinking about whether that was a good farming tactic. It being unworkable to proof the entire generated combinatoric space is, again, not the same thing as it being impractical to sanity-check the basic moving parts; this is still at its core a fairly glaring bit of human fuckupery, regardless of it being one of apparent passive neglect rather than of active malice.
posted by cortex at 6:37 AM on March 4, 2013 [1 favorite]


Is it so very, very hard to believe that, out of a list of 700 words, a guy might have seen 'rape' and thought, 'yeah, actually that one might sell a few'.

Well, considering that he started out with many many thousands, and then cut it down to mere thousands automatically, and then manually cut a list of thousands down to 700, then yes, it is kind of hard to ascribe intentionality. It's rather vicious, actually. You're so het on pushing the 'rape culture' idea (which has merit, in some circumstances) that you seem entirely unwilling to overlook that this was A) done in a big hurry, and B) involved a hell of a lot of deletion.

You seem stuck on imagining some red-eyed asshole, laughing maniacally as he plans his next assault, when it was really a glassy-eyed coder who was upset about a trademark issue.

What does matter is the huge amount of good faith that a bunch of people here and elsewhere seem to be ready to give a guy who ultimately, one way or another, inadvertently or otherwise, caused rape advocacy T-shirts to be offered for sale on Amazon.

He has hundreds of thousands of things for sale. I don't blame a parent if his or her baby says things that sound bad to me. And this isn't even a baby, it's a machine. There's no intention there.

You guys, in all seriousness, do tons of damage to your anti-rape messages when you do crap like this. You're crying wolf, when there is no wolf, just a scarecrow that looks wolfish. And you keep insisting that it's a wolf after it's been carefully pointed out to you that it is, in fact, a scarecrow, even after the owner politely took the scarecrow down and apologized for having frightened you.

When I see how much kneejerk anger and how little thought is going into this particular accusation of rape culture, my instinct is to assume that other accusations are also kneejerk and lack actual validity, even if they're very good. Maybe you don't care about convincing me, but I can't help but think that making people's eyes roll is not very helpful to your cause.

I even agree with a lot of it. But you've gone around the bend on this one. An algorithm did something you think is offensive, the creator of the algorithm agreed, apologized, and deleted the result, case closed. There's no reason to be upset anymore.
posted by Malor at 6:41 AM on March 4, 2013 [9 favorites]


I don't personally think he meant to produce "rape a lot" or "hit her" T-shirts, but he did

I should add that by "produce" I didn't mean he actually made any physical T-shirts; he "produced" via an automated process the hypothetical possibility of there being such T-shirts. Basically, it's thoughtcrime where nobody actually thought the actual thoughts.
posted by rory at 7:09 AM on March 4, 2013 [7 favorites]


A slight rewording of something I just sent in MeMail:

This is complex, and strange, and a new thing in the world, and everyone should just calm the heck down.
posted by Malor at 7:11 AM on March 4, 2013 [2 favorites]


rory: "Basically, it's thoughtcrime where nobody actually thought the actual thoughts."

rory, that's gorgeous.
posted by ChrisR at 7:18 AM on March 4, 2013 [2 favorites]


Last May there was the FPP on the article Spam-erican Apparel, which covered similar although less enraging results.
posted by Prince_of_Cups at 11:12 AM on March 4, 2013


Okay, computer lovers, here's an analogy for you.

You've got a person on your staff, they are in charge of testing your data center's resilience, in the style of ChaosMonkey. Let's say they write a script that randomly corrupts sectors on hard drives, which you expect to be healed through your excellent new RAID implementation.

When rolling out the script they grab a list of hosts for the monkey to randomly choose among. They include in the list the core command server, which for historical reasons is a single-point-of-failure for your system. Five days after deployment the script corrupts the command server and your data center goes down for 24 hours.

It's just a random script, nobody's to blame.
posted by benito.strauss at 12:39 PM on March 4, 2013


Well, in that case, they did their job, and demonstrated that our data center didn't have good resilience.
posted by Malor at 1:24 PM on March 4, 2013 [1 favorite]


"...thoughtcrime where nobody actually thought the actual thoughts..."

No one (AFAIK) actually thought the actual thought "Hey, let's put out some funny rape t-shirts! We'll make a few bucks and if anyone bitches, then, you know, we'll pull the old Blame It On The Algorithm and get free publicity too!"

But it seems no one actually thought the actual thought "Hey, let's make sure our random funny t-shirt generator algorithm doesn't produce something really super-offensive. Let's get a pair of human eyeballs on our funny t-shirt slogans, just to make sure before it goes live." either.

It's not really an algorithm at fault. Algorithms don't intentionally or accidentally make rape joke t-shirt sales websites. People intentionally or accidentally make algorithms that, in strict logical accordance with their coding, make rape joke t-shirt sales websites.

Neither "We don't really know what we sell, we just have a computer spew out words and we hope people buy stuff with those words on them" nor "We'll try to sell rape-joke t-shirts because we hope people will think they are funny" shows a lot of respect for anyone.


So, thoughtlesscrime?

This comment was produced by an algorithm, so if you don't like it, suck it.
posted by Cookiebastard at 2:29 PM on March 4, 2013 [1 favorite]


Okay, Cookiebastard, here's 300,000 possible shirt slogans.

Quick, which one is offensive? You have one hour.
posted by Malor at 6:12 PM on March 4, 2013


I don't see the list.
posted by Cookiebastard at 6:20 PM on March 4, 2013


Well, been longer than an hour, I have no list, and I ain't getting paid. I hereby conclude your business model sucks and I don't want to work for you. But, hey, good luck.
posted by Cookiebastard at 8:23 PM on March 4, 2013 [2 favorites]


Can we step back? I think we can all agree it was a thoughtless, ill-conceived and definitely poorly implemented idea. The question I have is how we respond to it. Do we send out the twitter fight mobs before we even know how it happened or what the intent was? Or do we step back and let those responsible try to fix it? Because I'm for the latter and don't understand why everyone wants to be offended by this. Isn't it better for everyone to believe this was (at best) some thoughtlessness in one moment rather than an intentional plan to make a bunch of awful shirts about rape and violence (and a bunch that aren't even grammatical)?
posted by R343L at 8:34 PM on March 4, 2013 [1 favorite]


I'm not sure what a "twitter fight mob" is, or how harmful or harmless one might be. I'm certainly not going to advocate for sending one out barring further information. But I see very few people in this thread claiming that there was intent to make a bunch of awful shirts about rape and violence. A lot of us are arguing mainly that it's probably a bad idea to sell t-shirts with random slogans without making sure that the slogans aren't so offensive that the twitter fight mobs will get pissed, if twitter fight mobs are a worrisome thing.

I don't know what programming language the Sloganbot 5000 was written in, but some of us that have done some programming agree that something along the lines of:

IF SLOGAN$ CONTAINS BADWORD THEN DO NOT USE

Followed by a human eyeballing the damn things before they go live on a website for sale. If "rape" and "punch" and "hit" are not in the BADWORD database yet, and a person eyeballing the damn things notices that they got through, then hooray! No stupid controversy, and now we can add those to the BADWORD database! I mean, maybe we don't get that all-important "KEEP CALM AND DRINK PUNCH" t-shirt sale right away without some tweaking. But that's maybe better than risking a twitter fight mob.
posted by Cookiebastard at 9:30 PM on March 4, 2013 [1 favorite]


Enumerating badness, Cookiebastard, is one classic security approach. It doesn't work.

Enumerating goodness is a better approach, but in the context of language, fails to work, because language is full of grey areas and homophones and synonyms, some of which are newly created every day. Numerous spectacular failures of trying to prevent 'bad things' from being said with whitelist approaches were documented pretty thoroughly upthread.

I'd suggest reading the commentary up there, and following the links. This is not simple. If you think it is, that's a direct indicator that you haven't spent enough time reading and thinking about the problem.
posted by Malor at 4:32 AM on March 5, 2013


I love the chaos monkey story. :)

Don't forget that humans make more obscure but fundamentally similar mistakes, especially across languages. Witness the TrekStor i.Beat Blaxx, Mazda Laputa, Mitsubishi Pajero, etc.

I'm frankly confused how they avoided listing over a hundred different offensive shirts given they're doing it automatically.
posted by jeffburdges at 5:35 AM on March 5, 2013


I remember it taking a couple of years for Chevrolet to figure out why sales were so bad for the Nova in Latin America. Turns out "no va" doesn't mean "exploding star" in Spanish.
posted by localroger at 5:49 AM on March 5, 2013


It is simple to have a human being suitably literate in the English language eyeball the slogans before they go live. It might not be fast or cheap.
posted by Cookiebastard at 6:36 AM on March 5, 2013


It's obvious that they could have taken more care. My point is that the entire discussion is centered around explaining how they could have done this better. Having read their apology, I'm pretty damn sure they know they could have done better. We can enjoy our technical discussion about how to do it better. That's great (hopefully the next people who try this won't make the same mistakes). But most of the angry comments now are along the lines of explaining how they could have done better, they should have and it's a mortal sin that they didn't (only slight exaggeration).

Let me propose a small hypothetical. Almost no one knew about these shirts till late last week (perhaps a few people did, but they clearly didn't publicize it effectively enough for many more to find out). The creators didn't realize they were there. On Friday, lots of people exploded in anger over an entirely unintentional offense because some people decided to publicize it without finding out what happened first, despite it being obvious that they had far too many shirts on offer to have possibly looked at all of them (300,000 or more). Anger is known to increase physical health risks, to say nothing of the emotional state of being angry which I find exhausting and uncomfortable. I submit that everyone's angry reactions caused far more harm to themselves than these entirely hypothetical and unsold shirts actually did to anyone prior to Friday.

Wouldn't it been a far better story if someone influential (e.g. Miss Representation) had just emailed them, pointed out the shirts and asked them apologize and take them down? Given the wording of their apology (and that the first noticed rape shirt was taken down in a few hours), I think it likely they would still have been mortified. Then whoever contacted them could have publicized the problem (and why you shouldn't automate stuff). The tone of the content would be totally different, fewer people would be "GRR SO ANGRY" and we wouldn't all be trying to impute vicious motivations onto people who were probably just careless which is something that happens to all of us (if you think you've never unintentionally offended thru carelessness, then you need to ask a friend to be really honest with you.)

Thoughts? Am I just dreaming that people would prefer to fix things by treating others charitably even when they make mistakes? Or does everyone just want to be angry?

Note: obviously if there is evidence someone tried to get Solid Gold Bomb to take them down before exploding in anger, then my point is wrong. But I've seen no evidence of that and I think it likely if someone had, they would say so.
posted by R343L at 8:56 AM on March 5, 2013 [1 favorite]


It is simple to have a human being suitably literate in the English language eyeball the slogans before they go live. It might not be fast or cheap.

And, of course, they will be perfect and miss nothing. Simple.

Sure.
posted by Malor at 11:25 AM on March 5, 2013


"Our proofreader, whom we have fired, OK'ed a rape joke t-shirt. We will try to hire a much better proofreader." would then (IMHO) be a less GRAR-response-inducing explanation than "You people don't understand our business model! We use algorithms! We don't know what we sell, we don't even look at the stuff!"
posted by Cookiebastard at 11:46 AM on March 5, 2013 [1 favorite]


You seem stuck on imagining some red-eyed asshole, laughing maniacally as he plans his next assault, when it was really a glassy-eyed coder who was upset about a trademark issue.

One of the things I don't understand about this justification--if this was mainly a protest, an art project of sorts, I'd understand not looking too carefully for offensive content. But this wasn't, you know, a flickr gallery put up to protest the trademark filing. It was sold, as product, by this guy's company, and yes, I think that means he's responsible for due diligence.

I don't think this guy is really intending to profit off of rape jokes, but the thing is, from the outside what he did was indistinguishable from that. Saying that everyone who's getting upset about it on Twitter is just being a knee-jerking dumbass is kind of missing the fact that this is not all that far removed from the kind of stuff that gets deliberately sold. The apology posted is heartfelt and appropriate, and I'm sure the guy meant no malice by this. But if you're posting a bunch of semi-randomly generated stuff under your company's name without spot-checking it first, then yes, there is a chance some nasty things will get through, and yes, you will bear the blowback. And no, you will have no one to blame but yourself.
posted by kagredon at 11:37 PM on March 5, 2013 [1 favorite]




I'm fully aware that my blog post linked to up the top there is just part of a wider narrative, but I thought some of you might be interested in my follow-up post - effectively a brain dump of what happened when my blog went from fuckall to 80,000 hits in a weekend. If it's bad self-linking my apologies.
posted by peteash10 at 2:54 PM on March 6, 2013 [1 favorite]


I remember it taking a couple of years for Chevrolet to figure out why sales were so bad for the Nova in Latin America. Turns out "no va" doesn't mean "exploding star" in Spanish.

No, sorry.
posted by reprise the theme song and roll the credits at 1:35 AM on March 10, 2013 [2 favorites]


Ah, well, another great story bites the dust.
posted by localroger at 7:43 AM on March 10, 2013


This doesn't really deserve its own post, but here's a Youtube channel with thousands (hundreds of thousands?) e-cards that were obviously generated from a thrown together list:

I LOVE BATHTUB SHITTER, I LOVE A TRILLION BARNACLE LAPSE, I LOVE YOU BABY BUSTER, I LOVE YOU CLOTHESHOR-SE

Many, many, many more of them.
posted by codacorolla at 5:00 PM on March 14, 2013


« Older Gold plating the cuckoo clock   |   Gay Mormon Records Reactions To His Coming Out Newer »


This thread has been archived and is closed to new comments