A weird epic ramble about Etsy homepages from the mid 2000s.
July 22, 2022 1:15 PM Subscribe
Dan McKinley has a Twitter thread about working at Etsy in the mid 2000s, including:
It was scheduled for 3PM, but didn’t start until 3:30 because my interviewer was “really hung over.”
and
Steve had this intricate conspiracy theory he believed in centered around Bertrand Russell, which I never figured out.
and
There was an internal Flash endpoint you could use to exfiltrate all of our sales data, and in fact there was a whole other company that had figured this out and was selling our sales insights to Etsy sellers.
The complete thread is here (there is no /32 tweet, apparently).
I’ve been sitting on a weird epic ramble about Etsy homepages from the middle 2000’s for the better part of 15 years now so I figured I would write it up as a thread 1/
The story begins while I was still working in the financial industry in lower Manhattan, watching Visual Studio whitescreen for a half hour as it often did and questioning my life choices. 2/
One day while that was happening a woman I knew DM’d me a link and said “I want everything on this website.” 3/
I don’t know what she was trying to link me to because the site was broken, and it served me this error page. “Hey, sick error page,” I remember thinking. 4/
This kept happening though, and eventually I came to be dimly aware that there was a new site called Etsy with a reputation for being full of beautiful stuff. 5/
That reputation came from the homepage, which at the time was a grid of items that looked like this 6/
The homepage grids changed periodically, and they were always interesting. The items would be beautiful individually, or beautiful together, or it’d contain some oblique joke. They’d be a rainbow of color, or they’d be the same shade of eggshell white, whatever. 7/
It was this really wonderful handcrafted thing about handcrafted things, from the pre-personalization era. 8/
Anyway I became familiar enough with this via osmosis to recognize the name Etsy when I saw it on the http://python.org jobs page, so I jumped and applied. 9/
There were a few red flags in my Etsy interview which I was not conditioned to notice or emphasize at the time. It was scheduled for 3PM, but didn’t start until 3:30 because my interviewer was “really hung over.” 10/
The interview was also about Lisp. There was this Lisp moment in the middle 2000s. You had to be there. 11/
So after joining the company I eventually learned that the homepages were sourced from a feature called Treasury. Treasuries were a way for Etsy users to create collections, roughly the same as Pinterest these days. 12/
Treasuries were selected and promoted to homepages via human cron. There was one poor employee living the Leonardo Da Vinci lifestyle, waking up every hour around the clock to turn over the homepage. 13/
When I started she’d been at it for a while, and it took another six or eight months for that to end 14/
Obviously this is not the kind of thing a human being should be asked to endure for a startup. But you have to understand, there was a problem of approximately this magnitude everywhere you looked. 15/
The p95 of search in that era was several minutes. The site was down for a full hour each week or so, and not on purpose. The human tragedy of someone waking up every hour to change the homepage felt kind of contained I guess 16/
So like many of the frontend features of Etsy in those years, the Treasury was built in Flash by a visionary named Jared Tarbell, the “fourth founder.” He was more of a visual artist working in Flash as his medium than an engineer. 17/
He lived in New Mexico and was very difficult to get ahold of. Once I talked to him and he told me he was working on etching intricate patterns into rocks with a laser, so he could drop them in the desert for future raccoon civilizations to wonder about. 18/
Search ranking was a big problem for us, and we figured we could improve this if we knew things like “this or that listing is popular in popular treasuries.” 19/
Beyond that, the homepage was the same for everyone looking at it. We thought maybe we could iterate away from that to something less artificially scarce. Getting an item on the homepage was a big deal for sellers, but the math of the situation made it unattainable for most. 20/
The Flash architecture really held those ambitions back. There were only 250 Treasuries live on the site at any given time, which was a function of the RTMP server backend it was using. This was some kind of Flash chat server dingus. 21/
We didn’t even have access to it. There was a contractor managing a Windows box someplace. And there was no database backing it, so as far as we knew once Treasuries were off the site they were lost forever. 22/
In our heads this beautiful, wonderful thing feeding the public perception of Etsy was a small scale tragedy in terms of data loss and lost opportunity. 23/
Back in the constant chaos of daily goings on on the engineering team, we were trying to get our arms around things like security vulnerabilities. We hired some pen testers for the first time, and they found problems everywhere. 24/
SQL injection, cross site scripting, really just everywhere. If you name a category of security vulnerability, we definitely had it on every page. 25/
Flash was one of the systemic problems. Everywhere we had Flash, we had XSS. And there were worse issues with it, for example, you could impersonate the Etsy CEO in chat with users without authenticating if you wanted. 26/
There was an internal Flash endpoint you could use to exfiltrate all of our sales data, and in fact there was a whole other company that had figured this out and was selling our sales insights to Etsy sellers. 28/
Anyway we realized that, oh god, we are going to need to get our arms around this Flash stuff. And Jared had his hands full with the raccoons and the lasers, so we tried hiring somebody. 29/
We hired this guy that I guess I will call Steve. That’s not his real name. We let Jared interview Steve all by himself, which was a thing we never did again. 30/
Steve worked there for about a month, or a month and a half tops. A couple of vignettes I can remember about Steve: one day he freaked out at me and called me a pseudo-intellectual. Touché. Totally fair. 31/
Steve had this intricate conspiracy theory he believed in centered around Bertrand Russell, which I never figured out. 33/
There was one day when the site was down—the new CEO ran into the eng room yelling at everybody. Everyone was hair on fire freaking out about it. And then suddenly, we all just stopped as we gradually realized that Steve … 34/
… was at his desk watching an Infowars video at maximum volume. He’d been cranking the volume that whole time because he had his headphones on, but they were not plugged in. 35/
One day, I was walking back to the office after lunch and I see Steve coming out of the building. I hid, crouching between two parked cars, so he wouldn’t see me. 36/
When I got up to the office, the head of ops handed me Steve’s laptop. “Here you go, you’ve got to figure this Flash stuff out. We just kicked Steve out.” 37/
The first thing I did was make sure there were neither physical nor virtual booby traps on the laptop. After establishing that it wasn’t going to explode or send my passwords to Steve, I opened up Adobe tools and started fucking around 38/
I quickly surmised that he’d gotten nowhere, and was just watching Infowars videos that entire time. But luckily I got in touch with Jared who delivered a tarball of the Flash source (it wasn’t in source control anywhere) 39/
Somewhere in that tarball I found a set of credentials for an ftp server in a comment. After connecting to it, I do an ls and find a giant pile of binary files, format unknown. 40/
However the file timestamps are a clue: this is the schedule of when the treasuries turn over. It’s the treasury data! 41/
This happened on a Wednesday or a Thursday, and I barely slept through the weekend as I tried to reverse engineer and recover all of this data. Which remember, we all believed was going to be a gold mine that would change everything. 42/
I never figured out what the file format was, but I opened it up in hexl-mode in emacs and defocused my eyes and could figure out the basic size of the records and overall structure. 43/
By Sunday I had figured it out just enough to be able to write a Python script that downloaded all of the data and grepped through it to dump out the Etsy listing ID’s that were in them along with comment counts and other metadata. 44/
And I had reverse engineered Jared’s ranking formula, so I could emit the most popular (and therefore best) treasuries of all time. I hit enter on this script and let it crank for hours. 45/
When it was finally done, after days and days of effort I opened up MacOS finder and flipped through the results. It was almost entirely men with their shirts off, which surprised me at the time. 46/
I went back to the office and never mentioned it. I don’t think anyone even asked what happened with the whole treasury data thing, because again, the entire building was on fire. 47/
“Yeah, I don’t know, the treasury data thing didn’t pan out.” 48/
There apparently was this entire phenomenon of sellers putting up sexy pictures of themselves, and then people making treasuries out of them. But engineers and leadership never knew about this because the moderators were just filtering that stuff out. 49/
A few years later, we rebuilt the treasury and let people have as many treasuries as they wanted. And it turned out that what most of them wanted was to make treasuries full of penises and vote them to the top. 50/
This generated an entire protracted incident where I spent two weeks working on “class DickFilter” - and then pretty much the day that I was ready to deploy this the hivemind decided it wasn’t funny anymore and the meme just kind of dissolved. 51/
I guess it’s a testament to the power of compartmentalization that I rebuilt the entire treasury and didn’t forsee this crisis that was 100% predictable on the basis of my own experience. 52/
Evolution probably weeded out everyone that would have dwelled on the Shirtless Men Result too much and given up back on the Serengeti or something. 53/
There are probably several morals here I guess, but the main one for me was never to overlook the human in the loop. The magic in the system was the person yanking 24 beautiful treasuries a day out of the sea of shirtless dudes. 54/
I still think about this saga a lot, for example when someone suggests in a legal filing that you could find bots on Twitter using AI without worrying about the human labeling. 55/55
The complete thread is here (there is no /32 tweet, apparently).
I’ve been sitting on a weird epic ramble about Etsy homepages from the middle 2000’s for the better part of 15 years now so I figured I would write it up as a thread 1/
The story begins while I was still working in the financial industry in lower Manhattan, watching Visual Studio whitescreen for a half hour as it often did and questioning my life choices. 2/
One day while that was happening a woman I knew DM’d me a link and said “I want everything on this website.” 3/
I don’t know what she was trying to link me to because the site was broken, and it served me this error page. “Hey, sick error page,” I remember thinking. 4/
This kept happening though, and eventually I came to be dimly aware that there was a new site called Etsy with a reputation for being full of beautiful stuff. 5/
That reputation came from the homepage, which at the time was a grid of items that looked like this 6/
The homepage grids changed periodically, and they were always interesting. The items would be beautiful individually, or beautiful together, or it’d contain some oblique joke. They’d be a rainbow of color, or they’d be the same shade of eggshell white, whatever. 7/
It was this really wonderful handcrafted thing about handcrafted things, from the pre-personalization era. 8/
Anyway I became familiar enough with this via osmosis to recognize the name Etsy when I saw it on the http://python.org jobs page, so I jumped and applied. 9/
There were a few red flags in my Etsy interview which I was not conditioned to notice or emphasize at the time. It was scheduled for 3PM, but didn’t start until 3:30 because my interviewer was “really hung over.” 10/
The interview was also about Lisp. There was this Lisp moment in the middle 2000s. You had to be there. 11/
So after joining the company I eventually learned that the homepages were sourced from a feature called Treasury. Treasuries were a way for Etsy users to create collections, roughly the same as Pinterest these days. 12/
Treasuries were selected and promoted to homepages via human cron. There was one poor employee living the Leonardo Da Vinci lifestyle, waking up every hour around the clock to turn over the homepage. 13/
When I started she’d been at it for a while, and it took another six or eight months for that to end 14/
Obviously this is not the kind of thing a human being should be asked to endure for a startup. But you have to understand, there was a problem of approximately this magnitude everywhere you looked. 15/
The p95 of search in that era was several minutes. The site was down for a full hour each week or so, and not on purpose. The human tragedy of someone waking up every hour to change the homepage felt kind of contained I guess 16/
So like many of the frontend features of Etsy in those years, the Treasury was built in Flash by a visionary named Jared Tarbell, the “fourth founder.” He was more of a visual artist working in Flash as his medium than an engineer. 17/
He lived in New Mexico and was very difficult to get ahold of. Once I talked to him and he told me he was working on etching intricate patterns into rocks with a laser, so he could drop them in the desert for future raccoon civilizations to wonder about. 18/
Search ranking was a big problem for us, and we figured we could improve this if we knew things like “this or that listing is popular in popular treasuries.” 19/
Beyond that, the homepage was the same for everyone looking at it. We thought maybe we could iterate away from that to something less artificially scarce. Getting an item on the homepage was a big deal for sellers, but the math of the situation made it unattainable for most. 20/
The Flash architecture really held those ambitions back. There were only 250 Treasuries live on the site at any given time, which was a function of the RTMP server backend it was using. This was some kind of Flash chat server dingus. 21/
We didn’t even have access to it. There was a contractor managing a Windows box someplace. And there was no database backing it, so as far as we knew once Treasuries were off the site they were lost forever. 22/
In our heads this beautiful, wonderful thing feeding the public perception of Etsy was a small scale tragedy in terms of data loss and lost opportunity. 23/
Back in the constant chaos of daily goings on on the engineering team, we were trying to get our arms around things like security vulnerabilities. We hired some pen testers for the first time, and they found problems everywhere. 24/
SQL injection, cross site scripting, really just everywhere. If you name a category of security vulnerability, we definitely had it on every page. 25/
Flash was one of the systemic problems. Everywhere we had Flash, we had XSS. And there were worse issues with it, for example, you could impersonate the Etsy CEO in chat with users without authenticating if you wanted. 26/
There was an internal Flash endpoint you could use to exfiltrate all of our sales data, and in fact there was a whole other company that had figured this out and was selling our sales insights to Etsy sellers. 28/
Anyway we realized that, oh god, we are going to need to get our arms around this Flash stuff. And Jared had his hands full with the raccoons and the lasers, so we tried hiring somebody. 29/
We hired this guy that I guess I will call Steve. That’s not his real name. We let Jared interview Steve all by himself, which was a thing we never did again. 30/
Steve worked there for about a month, or a month and a half tops. A couple of vignettes I can remember about Steve: one day he freaked out at me and called me a pseudo-intellectual. Touché. Totally fair. 31/
Steve had this intricate conspiracy theory he believed in centered around Bertrand Russell, which I never figured out. 33/
There was one day when the site was down—the new CEO ran into the eng room yelling at everybody. Everyone was hair on fire freaking out about it. And then suddenly, we all just stopped as we gradually realized that Steve … 34/
… was at his desk watching an Infowars video at maximum volume. He’d been cranking the volume that whole time because he had his headphones on, but they were not plugged in. 35/
One day, I was walking back to the office after lunch and I see Steve coming out of the building. I hid, crouching between two parked cars, so he wouldn’t see me. 36/
When I got up to the office, the head of ops handed me Steve’s laptop. “Here you go, you’ve got to figure this Flash stuff out. We just kicked Steve out.” 37/
The first thing I did was make sure there were neither physical nor virtual booby traps on the laptop. After establishing that it wasn’t going to explode or send my passwords to Steve, I opened up Adobe tools and started fucking around 38/
I quickly surmised that he’d gotten nowhere, and was just watching Infowars videos that entire time. But luckily I got in touch with Jared who delivered a tarball of the Flash source (it wasn’t in source control anywhere) 39/
Somewhere in that tarball I found a set of credentials for an ftp server in a comment. After connecting to it, I do an ls and find a giant pile of binary files, format unknown. 40/
However the file timestamps are a clue: this is the schedule of when the treasuries turn over. It’s the treasury data! 41/
This happened on a Wednesday or a Thursday, and I barely slept through the weekend as I tried to reverse engineer and recover all of this data. Which remember, we all believed was going to be a gold mine that would change everything. 42/
I never figured out what the file format was, but I opened it up in hexl-mode in emacs and defocused my eyes and could figure out the basic size of the records and overall structure. 43/
By Sunday I had figured it out just enough to be able to write a Python script that downloaded all of the data and grepped through it to dump out the Etsy listing ID’s that were in them along with comment counts and other metadata. 44/
And I had reverse engineered Jared’s ranking formula, so I could emit the most popular (and therefore best) treasuries of all time. I hit enter on this script and let it crank for hours. 45/
When it was finally done, after days and days of effort I opened up MacOS finder and flipped through the results. It was almost entirely men with their shirts off, which surprised me at the time. 46/
I went back to the office and never mentioned it. I don’t think anyone even asked what happened with the whole treasury data thing, because again, the entire building was on fire. 47/
“Yeah, I don’t know, the treasury data thing didn’t pan out.” 48/
There apparently was this entire phenomenon of sellers putting up sexy pictures of themselves, and then people making treasuries out of them. But engineers and leadership never knew about this because the moderators were just filtering that stuff out. 49/
A few years later, we rebuilt the treasury and let people have as many treasuries as they wanted. And it turned out that what most of them wanted was to make treasuries full of penises and vote them to the top. 50/
This generated an entire protracted incident where I spent two weeks working on “class DickFilter” - and then pretty much the day that I was ready to deploy this the hivemind decided it wasn’t funny anymore and the meme just kind of dissolved. 51/
I guess it’s a testament to the power of compartmentalization that I rebuilt the entire treasury and didn’t forsee this crisis that was 100% predictable on the basis of my own experience. 52/
Evolution probably weeded out everyone that would have dwelled on the Shirtless Men Result too much and given up back on the Serengeti or something. 53/
There are probably several morals here I guess, but the main one for me was never to overlook the human in the loop. The magic in the system was the person yanking 24 beautiful treasuries a day out of the sea of shirtless dudes. 54/
I still think about this saga a lot, for example when someone suggests in a legal filing that you could find bots on Twitter using AI without worrying about the human labeling. 55/55
This guy is a good storyteller, because it is incredibly specific but also gave me vivid flashbacks to every time I get hired at a company and then look behind the curtain and say "Oh no..."
posted by muddgirl at 2:02 PM on July 22, 2022 [12 favorites]
posted by muddgirl at 2:02 PM on July 22, 2022 [12 favorites]
I know Dan and a bunch of former Etsy folks and some of the stories they tell are... wow. Another former Etsy employee, Moishe Lettvin, has his own story to share in response.
Funny to see Jared Tarbell feature in Dan's story; Jared is a very impressive computational artist.
posted by Nelson at 2:02 PM on July 22, 2022 [8 favorites]
Funny to see Jared Tarbell feature in Dan's story; Jared is a very impressive computational artist.
posted by Nelson at 2:02 PM on July 22, 2022 [8 favorites]
"The interview was also about Lisp. There was this Lisp moment in the middle 2000s. You had to be there."
It was a Paul Graham thing, people used to take that guy seriously.
posted by mhoye at 2:08 PM on July 22, 2022 [16 favorites]
It was a Paul Graham thing, people used to take that guy seriously.
posted by mhoye at 2:08 PM on July 22, 2022 [16 favorites]
Metafilter: Jared had his hands full with the raccoons and the lasers
posted by nathan_teske at 2:21 PM on July 22, 2022 [6 favorites]
posted by nathan_teske at 2:21 PM on July 22, 2022 [6 favorites]
This generated an entire protracted incident where I spent two weeks working on “class DickFilter”
I have a working theory that every tech company of sufficient size (or that decides early-on to allow user-generated content) has their own version of
posted by Mayor West at 2:58 PM on July 22, 2022 [5 favorites]
I have a working theory that every tech company of sufficient size (or that decides early-on to allow user-generated content) has their own version of
class DickFilter. My favorite from my own experiences is still the (big, well-known, you've definitely heard of them) company I worked for that kept a big ol' blacklist of cursewords in a CSV built into a common module. When I left, it had eclipsed 10MB, though it was kind of unclear how much of it was just in-jokes by that point. The git history for that thing was... interesting.posted by Mayor West at 2:58 PM on July 22, 2022 [5 favorites]
Wow. My most vivid memory of mid-2000s Etsy is that they allowed you to search items by color, which was a really cool feature. The visualizations for the search were unique and this wasn't just a traditional faceted search. I remember some kind of color wheel feature. The mid-2000s was a peak time for unique search engines that attempted to elicit some kind of serendipity and Etsy's was one of my favorite examples.
posted by twelve cent archie at 3:00 PM on July 22, 2022 [11 favorites]
posted by twelve cent archie at 3:00 PM on July 22, 2022 [11 favorites]
That's really interesting, thanks for sharing it. I was working a lot of terribly boring desk jobs in the era before they figured out how to eliminate traditional secretarial downtime. So anyway Etsy was one of my favorite time killers & especially because of their unique visualizations. I was super interested in learning how websites worked back then so I had a little mental catalog of cool sites & Etsy always made me say wow, how lovely. The interesting & unique things I could buy that people were producing was also quite compelling. Its interesting to find out now it was all produced by an artiste, it really had to be, because it's an artist's job to show us things we didn't know we wanted & have never seen before. It's fell a long way.
posted by bleep at 3:01 PM on July 22, 2022 [11 favorites]
posted by bleep at 3:01 PM on July 22, 2022 [11 favorites]
I do find it kinda maddening that it takes the engineering whiz 50 odd tweets and fifteen years to admit that an actual human with great taste choosing the best stuff might be the actual magic after spending all that time failing at enginering something to replace her. This Human curator who was almost certainly a woman on the marketing or community team who had an incredibly deep knowledge of the Etsy, its sellers, and the spectacular array of stuff for sale that made it so awesome in the first place. She probably had a liberal arts degree. The fact that he calls her the magic and then sums it all up as "yanking from a sea of shirtless photos" is frankly kinda insulting.
Let's be real-- this engineering effort was not at all to save her from the act of having to stay up all night, because they could have built her a damn queue for that. They thought they could replace her eye and her judgement with a some blind computer code! optimized by performance data! The Leadership and engineering teams didn't know about the shirtless pictures problem because they didn't spend enough time her and her team to understand how they did the job of surfacing so many beautiful things amidst the sea of shirtless pictures, and what they'd learned in the process. What ever happened to this poor woman who was curating the stuff? I believe he called her the magic in the system? I dunno he never tells us! She doesn't even get a pseudonym!
Engineering Hubris and resistance to embracing the leadership and collaboration of non-engineers with good taste (often women) has ruined SO MANY user generated content + community sites. One could make a whole series of arguments for it having ruined society in general.
Good on this guy for publicizing his great revelation about the magic of human taste 15 years later, but I wish part of his engineering effort would have been collaborating more closely with this woman and her team. I'm sure they could have come up with some great ideas together. Who knows where we'd be if Etsy celebrated their curators and community managers as much as they celebrated their coders.
If you're an engineer or product designer or a PM someplace, please be curious about what you might learn from collaborating with the non-coding people on your team that spend time with your customers and community constituents. I promise they will be full of insights you may not have considered, and you'll make better ideas together. Don't overlook the humans in the loop. (yes, i could go on and on and on about this topic)
posted by wowenthusiast at 3:17 PM on July 22, 2022 [105 favorites]
Let's be real-- this engineering effort was not at all to save her from the act of having to stay up all night, because they could have built her a damn queue for that. They thought they could replace her eye and her judgement with a some blind computer code! optimized by performance data! The Leadership and engineering teams didn't know about the shirtless pictures problem because they didn't spend enough time her and her team to understand how they did the job of surfacing so many beautiful things amidst the sea of shirtless pictures, and what they'd learned in the process. What ever happened to this poor woman who was curating the stuff? I believe he called her the magic in the system? I dunno he never tells us! She doesn't even get a pseudonym!
Engineering Hubris and resistance to embracing the leadership and collaboration of non-engineers with good taste (often women) has ruined SO MANY user generated content + community sites. One could make a whole series of arguments for it having ruined society in general.
Good on this guy for publicizing his great revelation about the magic of human taste 15 years later, but I wish part of his engineering effort would have been collaborating more closely with this woman and her team. I'm sure they could have come up with some great ideas together. Who knows where we'd be if Etsy celebrated their curators and community managers as much as they celebrated their coders.
If you're an engineer or product designer or a PM someplace, please be curious about what you might learn from collaborating with the non-coding people on your team that spend time with your customers and community constituents. I promise they will be full of insights you may not have considered, and you'll make better ideas together. Don't overlook the humans in the loop. (yes, i could go on and on and on about this topic)
posted by wowenthusiast at 3:17 PM on July 22, 2022 [105 favorites]
It's fell a long way.
It's still pretty good, imo, but no, you're not going to replicate a human being updating the page manually every hour. Can't beat that, lol.
That poor person. I can imagine. I haven't quite been there, but close.
they could have built her a damn queue
Amen. (Making 24 pages a day would still suck chunks.) I never really got past that part of the story. Every hour? Every day?!?!?
Lisp was invented in 1958, fwiw, and it was definitely a thing back then.
posted by mrgrimm at 3:21 PM on July 22, 2022 [1 favorite]
It's still pretty good, imo, but no, you're not going to replicate a human being updating the page manually every hour. Can't beat that, lol.
That poor person. I can imagine. I haven't quite been there, but close.
they could have built her a damn queue
Amen. (Making 24 pages a day would still suck chunks.) I never really got past that part of the story. Every hour? Every day?!?!?
Lisp was invented in 1958, fwiw, and it was definitely a thing back then.
posted by mrgrimm at 3:21 PM on July 22, 2022 [1 favorite]
I want to read an interview with the woman who did the curation work -- we never feature the people who actually do this labor. This reminds me why I hated engineers growing up in the 2000s!!!
posted by yueliang at 3:22 PM on July 22, 2022 [29 favorites]
posted by yueliang at 3:22 PM on July 22, 2022 [29 favorites]
I definitely did not get the impression that the person finding content-wheat in a sea of dick-chaff for the front page, was the same person as the woman who was changing out the front page content every hour. And my experience with sites that have experienced growth past their ability to scale (like Etsy, apparently) is that it's absolutely on-brand for them to ignore processes that "work", even for really loose definitions of "work" that include "23-year-old employee has to push a button once an hour to push out new stuff." It would take ten minutes to automate, yes, but that's ten minutes of engineer time that's probably much more in demand in one of the other 9000 dysfunctional corners of the business.
posted by Mayor West at 3:37 PM on July 22, 2022 [5 favorites]
posted by Mayor West at 3:37 PM on July 22, 2022 [5 favorites]
Looking at the current etsy homepage, they are doing a mix of stuff reccommended for me, stuff "handpicked by our editors and stuff that has been chosen by their editors into an array of specific aesthetics/occaisions/giftees.
This is good. Probably goes a long way towards restoring their reputation as a handmade marketplace for makers after a decade+ of dominance from alibaba resellers. I do wonder how many worse-perfoming technical recommendation powered iterations they went to to land back here. Search results are still hit or miss, but at least they are celebrating editors instead of pretending they are fully powered by machines (which is so silly for a place sellling handmade goods anyway??? lol)
posted by wowenthusiast at 3:38 PM on July 22, 2022 [2 favorites]
This is good. Probably goes a long way towards restoring their reputation as a handmade marketplace for makers after a decade+ of dominance from alibaba resellers. I do wonder how many worse-perfoming technical recommendation powered iterations they went to to land back here. Search results are still hit or miss, but at least they are celebrating editors instead of pretending they are fully powered by machines (which is so silly for a place sellling handmade goods anyway??? lol)
posted by wowenthusiast at 3:38 PM on July 22, 2022 [2 favorites]
So as not to abuse the edit window: I am in no way condoning the decision to make this poor woman do tasks every hour on the hour, all day and all night, and it's clearly a terrible decision not to devote some engineer time to fixing the problem. But the decision to keep doing the stupid thing, simply because the stupid thing is functional and lots of other things are not, is very on-brand.
posted by Mayor West at 3:40 PM on July 22, 2022 [1 favorite]
posted by Mayor West at 3:40 PM on July 22, 2022 [1 favorite]
It was a Paul Graham thing, people used to take that guy seriously.
I went looking for "that Paul Graham LISP essay I kinda remember" for nostalgia's sake, and instead I found this, which is on point for the Etsy story:
posted by clawsoon at 3:53 PM on July 22, 2022 [8 favorites]
I went looking for "that Paul Graham LISP essay I kinda remember" for nostalgia's sake, and instead I found this, which is on point for the Etsy story:
You should take extraordinary measures not just to acquire users, but also to make them happy. For as long as they could (which turned out to be surprisingly long), Wufoo sent each new user a hand-written thank you note. Your first users should feel that signing up with you was one of the best choices they ever made. And you in turn should be racking your brains to think of new ways to delight them.The only thing that seems off is his confidence that it's easy to scale customer delight.
Why do we have to teach startups this? Why is it counterintuitive for founders? Three reasons, I think.
One is that a lot of startup founders are trained as engineers, and customer service is not part of the training of engineers. You're supposed to build things that are robust and elegant, not be slavishly attentive to individual users like some kind of salesperson. Ironically, part of the reason engineering is traditionally averse to handholding is that its traditions date from a time when engineers were less powerful — when they were only in charge of their narrow domain of building things, rather than running the whole show. You can be ornery when you're Scotty, but not when you're Kirk.
Another reason founders don't focus enough on individual customers is that they worry it won't scale. But when founders of larval startups worry about this, I point out that in their current state they have nothing to lose. Maybe if they go out of their way to make existing users super happy, they'll one day have too many to do so much for. That would be a great problem to have. See if you can make it happen. And incidentally, when it does, you'll find that delighting customers scales better than you expected.
...
There's a more extreme variant where you don't just use your software, but are your software. When you only have a small number of users, you can sometimes get away with doing by hand things that you plan to automate later. This lets you launch faster, and when you do finally automate yourself out of the loop, you'll know exactly what to build because you'll have muscle memory from doing it yourself.
When manual components look to the user like software, this technique starts to have aspects of a practical joke. For example, the way Stripe delivered "instant" merchant accounts to its first users was that the founders manually signed them up for traditional merchant accounts behind the scenes.
Some startups could be entirely manual at first. If you can find someone with a problem that needs solving and you can solve it manually, go ahead and do that for as long as you can, and then gradually automate the bottlenecks. It would be a little frightening to be solving users' problems in a way that wasn't yet automatic, but less frightening than the far more common case of having something automatic that doesn't yet solve anyone's problems.
posted by clawsoon at 3:53 PM on July 22, 2022 [8 favorites]
oh i definitely got the impression it's the same person or small team finding the content and populating the front page, but mostly because I have been on a team doing exactly that at a non-etsy place. Plus, if they had someone from engineering updating the homepage every hour, that queue would have built a already. But, hard to say, she's much more depersonalized than the other characters in the thread.
You are definitely right that many startups are held together with duct tape, some dreams, and under-resourced liberal arts humans doing what engineering can't/won't/doesn't have time for. I just wish tech culture was generally more honest about that fact. <3
posted by wowenthusiast at 3:53 PM on July 22, 2022 [2 favorites]
You are definitely right that many startups are held together with duct tape, some dreams, and under-resourced liberal arts humans doing what engineering can't/won't/doesn't have time for. I just wish tech culture was generally more honest about that fact. <3
posted by wowenthusiast at 3:53 PM on July 22, 2022 [2 favorites]
Etsy in general is such a frustrating case study in the failure of a more humane, personal market concept losing out to mass produced junk.
I believe this linked story is from the heady Rob Kalin days when Etsy was genuinely focussed on this mission to bring hand-crafted personal arts and crafts to market. The resulting web shop was cool but not a great business. Kalin finally got pushed out and a few years Chad Dickerson took over in 2011 and some things improved. The company went public in 2015. And there the stock price languished for years, I think well below earlier private valuations.
Dickerson was pushed out in 2017 and some more profit-focussed folks took over and turned Etsy into another generic website store with a bunch of shlock. You can still find interesting handcrafted things there (and Cthulhu dildos) but mostly it's just a bunch of the same shit you can buy everywhere else. But it must have worked, business wise, at least the stock is way way up.
posted by Nelson at 3:55 PM on July 22, 2022 [10 favorites]
I believe this linked story is from the heady Rob Kalin days when Etsy was genuinely focussed on this mission to bring hand-crafted personal arts and crafts to market. The resulting web shop was cool but not a great business. Kalin finally got pushed out and a few years Chad Dickerson took over in 2011 and some things improved. The company went public in 2015. And there the stock price languished for years, I think well below earlier private valuations.
Dickerson was pushed out in 2017 and some more profit-focussed folks took over and turned Etsy into another generic website store with a bunch of shlock. You can still find interesting handcrafted things there (and Cthulhu dildos) but mostly it's just a bunch of the same shit you can buy everywhere else. But it must have worked, business wise, at least the stock is way way up.
posted by Nelson at 3:55 PM on July 22, 2022 [10 favorites]
I have a working theory that every tech company of sufficient size (or that decides early-on to allow user-generated content) has their own version of class DickFilter.
The reason Minecraft exists and a Lego-branded virtual-world-building thing does not is that Lego the company decided, right up front, that children were their core audience and their minimum standard for safety was, non-negotiably, nobody and nobody's children are going to look through our software and see a dick. "A dick" is not the comprehensive list here, you understand, but for the purposes of this discussion just assume that it is.
It is, perhaps unsurprisingly, computationally impossible to guarantee that. Algorithmic certainty that when seen from any angle, that this construction against this backdrop in this world does not look like a dick? It can't be done.
And to their enormous credit, Lego did something that I wish every company in tech had the backbone to do: they decided that if they could not do this safely, to a clear definition of safe, for their core audience, that they weren't going to do it at all. So they put their tools down and walked away from the idea.
It's hard to imagine how much better the world would be if Facebook or Twitter had the courage and integrity to do that.
posted by mhoye at 4:28 PM on July 22, 2022 [37 favorites]
The reason Minecraft exists and a Lego-branded virtual-world-building thing does not is that Lego the company decided, right up front, that children were their core audience and their minimum standard for safety was, non-negotiably, nobody and nobody's children are going to look through our software and see a dick. "A dick" is not the comprehensive list here, you understand, but for the purposes of this discussion just assume that it is.
It is, perhaps unsurprisingly, computationally impossible to guarantee that. Algorithmic certainty that when seen from any angle, that this construction against this backdrop in this world does not look like a dick? It can't be done.
And to their enormous credit, Lego did something that I wish every company in tech had the backbone to do: they decided that if they could not do this safely, to a clear definition of safe, for their core audience, that they weren't going to do it at all. So they put their tools down and walked away from the idea.
It's hard to imagine how much better the world would be if Facebook or Twitter had the courage and integrity to do that.
posted by mhoye at 4:28 PM on July 22, 2022 [37 favorites]
My favorite Etsy "something has gone awry" story is four-and-a-horse-stars. I think about it every time I see either of those as emoji.
posted by fifteen schnitzengruben is my limit at 4:31 PM on July 22, 2022 [17 favorites]
posted by fifteen schnitzengruben is my limit at 4:31 PM on July 22, 2022 [17 favorites]
You are definitely right that many startups are held together with duct tape, some dreams, and under-resourced liberal arts humans doing what engineering can't/won't/doesn't have time for. I just wish tech culture was generally more honest about that fact.
I have been this person multiple times where I was basically a semi-intelligent web scraping and archiving robot.
Sure, some of my tasks could be automated by someone with better scripting and scraping skills, but those people cost a lot more money and so do the frameworks or SaaS tools they often use, and with each new scraping or archiving project you'd have to build out and invent new sets of scripts and tools.
It was often just faster, easier, cheaper and more accurate to turn me loose on a few dozen or few hundred blog posts or other managed content from an outdated site and let me go to town on it with some basic scraper plugins and downloading tools or a few wget strings.
The benefit of doing it the way I liked to do it with a mix of manual scraping and automated archiving is that I'm going to have to get my hands and eyes on the content anyway to proof read and format it properly for a new CMS, so that actually can save some steps and busy work of trying to validate scraping/archiving results since I can just do it on the fly as I go instead of spinning my wheels trying to automate all of it with scripting.
There have also been other benefits like finding hidden directories or content thanks to wget path-traversal hacks and strings. I often found old archives of valuable or even sensitive data that both the clients and my lead/manager didn't even know were there and weren't included in the initial quotes or proposals, all because I was there as a human with eyeballs and I wasn't just a mindless script.
I saved a few projects in the discovery and proposal phase simply by doing things like preemptively archiving a clients entire site and sitemap to my local computer before we even signed a contract and then something bad happened like they lost their hosting, didn't have the right credentials or other loss of access to their content, because I would use tools like wget to get a very accurate overview, file count and total content size for the purposes of the proposal and price quotes.
Which, yeah, these are basically the some of the same exact skills and tools used by content pirates and hackers trying to scrape, say, a music hosting site and steal a bunch of content with brute force directory crawling.
And honestly, I didn't hate this kind of work. I'm sure a lot of people would have found it to be mind-numbingly dull but I liked it just fine. I had some clever, adaptive tricks and hacks to speed up my work flow and develop my own workflow algorithms to the point I could basically just put on my headphones and go full auto once I had the workflow patterns sorted out and my semi-automated tools in place.
But I also sure didn't get the same kind of cred or pay as someone who spends two weeks noodling around with a few kilobytes balky scripts and jellybean copy-paste code while I was doing the heavy lifting moving gigabytes of text, photos and media around.
posted by loquacious at 5:45 PM on July 22, 2022 [12 favorites]
I have been this person multiple times where I was basically a semi-intelligent web scraping and archiving robot.
Sure, some of my tasks could be automated by someone with better scripting and scraping skills, but those people cost a lot more money and so do the frameworks or SaaS tools they often use, and with each new scraping or archiving project you'd have to build out and invent new sets of scripts and tools.
It was often just faster, easier, cheaper and more accurate to turn me loose on a few dozen or few hundred blog posts or other managed content from an outdated site and let me go to town on it with some basic scraper plugins and downloading tools or a few wget strings.
The benefit of doing it the way I liked to do it with a mix of manual scraping and automated archiving is that I'm going to have to get my hands and eyes on the content anyway to proof read and format it properly for a new CMS, so that actually can save some steps and busy work of trying to validate scraping/archiving results since I can just do it on the fly as I go instead of spinning my wheels trying to automate all of it with scripting.
There have also been other benefits like finding hidden directories or content thanks to wget path-traversal hacks and strings. I often found old archives of valuable or even sensitive data that both the clients and my lead/manager didn't even know were there and weren't included in the initial quotes or proposals, all because I was there as a human with eyeballs and I wasn't just a mindless script.
I saved a few projects in the discovery and proposal phase simply by doing things like preemptively archiving a clients entire site and sitemap to my local computer before we even signed a contract and then something bad happened like they lost their hosting, didn't have the right credentials or other loss of access to their content, because I would use tools like wget to get a very accurate overview, file count and total content size for the purposes of the proposal and price quotes.
Which, yeah, these are basically the some of the same exact skills and tools used by content pirates and hackers trying to scrape, say, a music hosting site and steal a bunch of content with brute force directory crawling.
And honestly, I didn't hate this kind of work. I'm sure a lot of people would have found it to be mind-numbingly dull but I liked it just fine. I had some clever, adaptive tricks and hacks to speed up my work flow and develop my own workflow algorithms to the point I could basically just put on my headphones and go full auto once I had the workflow patterns sorted out and my semi-automated tools in place.
But I also sure didn't get the same kind of cred or pay as someone who spends two weeks noodling around with a few kilobytes balky scripts and jellybean copy-paste code while I was doing the heavy lifting moving gigabytes of text, photos and media around.
posted by loquacious at 5:45 PM on July 22, 2022 [12 favorites]
clawsoon: I remember getting a few of these types of emails from services I signed up for. Timeframe checks out too. I found them a little obnoxious and very robotic. Another thing to thank pg for I guess.
posted by dumbland at 6:52 PM on July 22, 2022
posted by dumbland at 6:52 PM on July 22, 2022
Paul Graham wrote a whole book On Lisp, and remains entirely up his own ass about it.
posted by snuffleupagus at 8:49 PM on July 22, 2022
posted by snuffleupagus at 8:49 PM on July 22, 2022
> And to their enormous credit, Lego did something that I wish every company in tech had the backbone to do: they decided that if they could not do this safely, to a clear definition of safe, for their core audience, that they weren't going to do it at all. So they put their tools down and walked away from the idea.
I heard a story about a decision like this, but it was before Minecraft was a thing and I don’t remember the company. The prototype software had a chat function, but the users could only write by selecting from a curated list of inoffensive words.
The testers put a 14-year-old boy in front of this system, and he promptly wrote “I’d like to put my tall giraffe in your fluffy bunny.”
The company abandoned the project.
posted by fantabulous timewaster at 12:56 AM on July 23, 2022 [14 favorites]
I heard a story about a decision like this, but it was before Minecraft was a thing and I don’t remember the company. The prototype software had a chat function, but the users could only write by selecting from a curated list of inoffensive words.
The testers put a 14-year-old boy in front of this system, and he promptly wrote “I’d like to put my tall giraffe in your fluffy bunny.”
The company abandoned the project.
posted by fantabulous timewaster at 12:56 AM on July 23, 2022 [14 favorites]
Wowenthusiast - I agree with the spirit of your rant but I think this storyteller (and any engineer i know) might think… no reason not to have the human curator make 24x7 selections on Monday and put them in the queue other than madness, this was like 1973, algorithmic tastemaking was probably not what they were driving at.
posted by thedaniel at 1:35 AM on July 23, 2022
posted by thedaniel at 1:35 AM on July 23, 2022
Bring back Etsy Alchemy!
posted by inflatablekiwi at 4:33 AM on July 23, 2022
posted by inflatablekiwi at 4:33 AM on July 23, 2022
Paul Graham wrote a whole book On Lisp, and remains entirely up his own ass about it.
So someone asked "what book made you think the most?", and he answered with two books he had written?
Heh.
I suppose that'd be technically true for any author - what book is going to force you to think more than a book you're writing? - but, still, heh.
posted by clawsoon at 4:35 AM on July 23, 2022 [3 favorites]
So someone asked "what book made you think the most?", and he answered with two books he had written?
Heh.
I suppose that'd be technically true for any author - what book is going to force you to think more than a book you're writing? - but, still, heh.
posted by clawsoon at 4:35 AM on July 23, 2022 [3 favorites]
The testers put a 14-year-old boy in front of this system,
This is an underrated testing technique in general.
posted by mhoye at 6:01 AM on July 23, 2022 [15 favorites]
This is an underrated testing technique in general.
posted by mhoye at 6:01 AM on July 23, 2022 [15 favorites]
I was at Etsy 2016–17 and got to experience what I now see as the twilight years of a truly glorious and weird culture that valued creativity and learning as much as good engineering. It felt like a tragedy when the leadership got pushed out in favor of folks with a track record of more predictably turning the capitalism crank, but it remains one of the few places I am genuinely proud to have worked at and miss to this day.
posted by Cogito at 6:39 AM on July 23, 2022 [6 favorites]
posted by Cogito at 6:39 AM on July 23, 2022 [6 favorites]
"Who handed you this tarball now ?"
"Jared, it's cool"
"Jared who? Nobody named Jared works here."
"Jared Tar…bell. Jared Tarbell."
"Jared. Tar. Bell. …?"
"…Yep."
posted by SaltySalticid at 6:39 AM on July 23, 2022 [6 favorites]
"Jared, it's cool"
"Jared who? Nobody named Jared works here."
"Jared Tar…bell. Jared Tarbell."
"Jared. Tar. Bell. …?"
"…Yep."
posted by SaltySalticid at 6:39 AM on July 23, 2022 [6 favorites]
I liked Dan Luu's comment
Dan McKinley has a lot of small sites capturing this shift in expertise to making stuff work like a calm professional. The most well known is Choose Boring Technology, see also his other sites. I just had the pleasure of using his PugSQL in a project, it's a very simple Python/Database adapter that is nicer than writing plain database code but far less magic than an ORM. (To the Lisp discussion above, PugSQL is inspired by a Clojure library.)
posted by Nelson at 7:00 AM on July 23, 2022 [7 favorites]
I find these stories about Etsy's early days interesting because they sound wild from the outside but are fairly typical. If you ask around about other major tech companies, these kinds of stories are commonDefinitely agree, the amount of engineering and cultural shenanigans in startups is remarkable. I appreciate that the tech industry building these things has gotten more professional over the years and there's now more consistency in how to do things like devops or release management. But I miss the days when we were all just making shit up.
Dan McKinley has a lot of small sites capturing this shift in expertise to making stuff work like a calm professional. The most well known is Choose Boring Technology, see also his other sites. I just had the pleasure of using his PugSQL in a project, it's a very simple Python/Database adapter that is nicer than writing plain database code but far less magic than an ORM. (To the Lisp discussion above, PugSQL is inspired by a Clojure library.)
posted by Nelson at 7:00 AM on July 23, 2022 [7 favorites]
fantabulous timewaster: The testers put a 14-year-old boy in front of this system...
mhoye: This is an underrated testing technique in general.
The video games industry agrees with you but it is (marginally) constrained by employment law.
From the testing side, I agree with much of the thread's commentary: the real engineering needs are about giving people ways to automate their work moving and transforming business information, something always rooted with the workflows of users. "Digital Transformation" got some buzz in the past few years but I've never seen it described as "not users working at computers but users managing computers doing the work."
posted by k3ninho at 7:41 AM on July 23, 2022 [2 favorites]
mhoye: This is an underrated testing technique in general.
The video games industry agrees with you but it is (marginally) constrained by employment law.
From the testing side, I agree with much of the thread's commentary: the real engineering needs are about giving people ways to automate their work moving and transforming business information, something always rooted with the workflows of users. "Digital Transformation" got some buzz in the past few years but I've never seen it described as "not users working at computers but users managing computers doing the work."
posted by k3ninho at 7:41 AM on July 23, 2022 [2 favorites]
Should have spent that weekend coding a queuing system for the poor moderator. Or any type of moderator human support tools.
posted by brendano at 12:13 PM on July 23, 2022 [3 favorites]
posted by brendano at 12:13 PM on July 23, 2022 [3 favorites]
« Older The United States of Amazon: Same day treatment... | Lepp allots herself precisely 49 days to write and... Newer »
This thread has been archived and is closed to new comments
Not Hotdog made real.
posted by pwnguin at 1:42 PM on July 22, 2022