intitle:"index of"

July 8, 2018 6:48 AM

This is the story of why I created https://buckets.grayhatwarfare.com/ a free tool that lists open s3 buckets and helps you search for interesting files.

It's like googling for open directories, but IN THE CLOUD.
posted by pompomtom (30 comments total)

Can someone explain this to me? I'm just a caveman. Your world frightens and confuses me.
posted by Don.Kinsayder at 7:29 AM on July 8, 2018 [5 favorites]

Can someone explain this to me?

It's a fun way to get malware
posted by thelonius at 7:35 AM on July 8, 2018 [12 favorites]

I searched “Lasgna recipes” because it was the first thing that popped up in my mind & all 8 results were low-fat.

Still, an interesting tool. I can remember backspacing in URL’s to get to the file directory on random websites just to see what I could find back in the 90’s & it occasionally yielded some fun stuff.
posted by Devils Rancher at 7:36 AM on July 8, 2018 [3 favorites]

Can someone explain this to me? I'm just a caveman.

ugg have big pile of tablets with stories
ugg need to store stories somewhere and be able to find story quickly to impress other uggs
pile of tablets too big for tent
too big for cave
too big for all caves
ugg say why not develop concepts of algorithms and programming and complex material societal and technical systems to allow ugg to store mindblowingly large number of tablets in decentralized manner
ugg say now here we are
posted by lalochezia at 7:36 AM on July 8, 2018 [39 favorites]

Thank you, thelonius and ugg.
posted by Don.Kinsayder at 7:40 AM on July 8, 2018 [2 favorites]

Can someone explain this to me?

S3 is a widely used file storage service, offered by Amazon.

An S3 "bucket" is (more or less) like an individual FTP site: if you're an S3 subscriber, your bucket is the container that you put your files into.

And, just like an FTP site, you probably want to password-protect your S3 bucket, so that any rando who stumbles across it can't access your files.

But sometimes people forget, or screw up, and leave their S3 bucket completely open and unprotected.

The project linked in the FPP finds these open buckets, and lists them for all the world to rifle through.

If that doesn't help, you might get better results with a question more specific than "don't understand plz explain" :)
posted by escape from the potato planet at 7:53 AM on July 8, 2018 [8 favorites]

Okay, I got it. 😝
posted by Don.Kinsayder at 7:58 AM on July 8, 2018 [3 favorites]

I created a tool that finds unlocked doors on houses in your neighborhood. It's cool!
posted by davebush at 8:06 AM on July 8, 2018 [9 favorites]

As far as physical metaphors that problematize the ethics of it I'd say it's closer to "I created a tool that finds storage units that have no doors", but I feel you.
posted by cortex at 8:18 AM on July 8, 2018 [5 favorites]

Hey is mine unlocked? Because I left the oven on.
posted by chavenet at 8:20 AM on July 8, 2018 [3 favorites]

but then you only make a copy of the stuff inside the storage locker and leave it as you found it
posted by Salvor Hardin at 8:24 AM on July 8, 2018 [2 favorites]

you wouldn't download a couch
posted by fantabulous timewaster at 8:39 AM on July 8, 2018 [12 favorites]

> fantabulous timewaster:
"you wouldn't download a couch"

Is it a comfy couch?
posted by Samizdata at 8:41 AM on July 8, 2018 [1 favorite]

does loonette come with it?
posted by pyramid termite at 8:48 AM on July 8, 2018

I have created a tool that finds storage units sealed with an impermeable glass wall but in which the contents have been carefully organized such that every square inch of surface area can be clearly read from outside said glass wall.
posted by cortex at 8:49 AM on July 8, 2018 [1 favorite]

These S3 leaks frequently make big news. For instance a Republican voter file on 198 million American voters was left unsecured, was fo und, and downloaded and exposed. Verizon exposed 14 million customer records via S3 awhile back too. There's a lot more.

I like the idea of this tool in that I'm sure the bad guys already have tools like this. But boy does it make things uncomfortable how easy it is to find data that no one knows is publicly accessible.

See also Shodan, a search tool for unsecured Internet connected devices. Lots of webcam voyeurism there.
posted by Nelson at 8:55 AM on July 8, 2018 [4 favorites]

I think the analogy has just about run its course by now - next someone's going to describe a virtual storage unit that only stores digital items that can be read and copied but not deleted, protected by a digital padlock, or "password" that many people forget to put on in the first place...
posted by dazed_one at 8:58 AM on July 8, 2018 [1 favorite]

well let's not be silly
posted by cortex at 9:03 AM on July 8, 2018 [1 favorite]

you wouldn't download a couch

Of course not! You'd download the 12 RARs first, then assemble the couch.
posted by Thorzdad at 9:11 AM on July 8, 2018 [19 favorites]

This is like that tool a guy I worked with once wrote in the late 90's: it used to scan the internet for open windows fileshares. Oh the things you would see. Seems what's old is new again.
posted by some loser at 9:25 AM on July 8, 2018

I think the analogy has just about run its course by now...

Two more points: there's no lights in the storage unit, so you can find any of the things in your storage unit as long as you remember exactly where you put them, but there may be stuff in there you forgot about. Or that someone else with a copy of the key put there.

And when you get behind on your bills, instead of an HGTV camera crew and some reality TV character trying to buy your stuff, it's all instantly destroyed.
posted by wenestvedt at 9:40 AM on July 8, 2018

Sooo… it's a searchable bucket list?
posted by farlukar at 9:41 AM on July 8, 2018 [2 favorites]

Cut to the chase. Where does my friend get the porn?
posted by cjorgensen at 9:45 AM on July 8, 2018 [2 favorites]

This kinda made me think of the old Veronica FTP search concept. Interesting to read the bucket described in comment above as like FTP...

Wonder if this person had inspiration from Veronica or what.
posted by symbioid at 9:57 AM on July 8, 2018

This is like that tool a guy I worked with once wrote in the late 90's: it used to scan the internet for open windows fileshares.

Oh you mean scour.net? Such crazily unethical software. "These folks don't know their personal files are being shared on the Internet; let's make them easier to find so we can make a buck off music piracy". Travis Kalanick worked there in 1998, I wonder what other ethical disasters he went on to be involved with?

This S3 bucket search is a little different ethically in its stated goal is to call attention to the security problem. It still makes me a little uncomfortable.

(On preview, yeah, this is a lot like Veronica. Only Veronica searched FTP servers that were deliberately publicly accessible.)
posted by Nelson at 9:58 AM on July 8, 2018 [1 favorite]

Hah, first random term I tried found me a site containing static precompiled HTML files belonging to a keyword spammer with the usual word salad blog content: "Finding nemo woman costume ideas,how to win a married ex back permanently,how to find a us marine - PDF Review Category: How To Get A Girl Back | 01.09.2015 Step 11: Under the eyes, below the horizontal construction line, draw Nemo's open mouth using two curved lines. Step 12: Inside Nemo's mouth, draw a long line at the top for the row of teeth and a small line at the bottom for the tongue. When checked, Shutterstock's safe search screens restricted content and excludes it from your search results."

A quick view of the page source shows it wants to pull in scripts from some pretty recognizably garbage sites, and I'd probably see several times as many Javascript files if PiHole and Ublock weren't doing their jobs.

So it's actually like storage units and all the boxes are legitimately labeled with their contents so you can tell what everybody's holding and you can pick through the boxes and look at anything you like but the moment you do it all the deer ticks and tapeworms and rabid bats also leap out at you.
posted by ardgedee at 9:58 AM on July 8, 2018 [1 favorite]

Ah, takes me back to the days of doing this on google when there was a whole lot more to be found there (after a while the amount of pirated content to be found decreased remarkably) back as a teenager. For the betterment of everyone, these exploits became less available.

So far, just found some bible readings, recorded sermons, and bird sounds.
posted by fizzix at 10:00 AM on July 8, 2018 [1 favorite]

The decision to filter out images, but keep svg's and fonts is a little odd. Overall, though, a nice little tool.

Public buckets are pretty good for intentionally sharing files, and aws does repeatedly warn you about it. Perhaps the actual sensitive stuff was slipped in through automated creation and uploads, were you don't get those warnings.
posted by cowcowgrasstree at 10:12 AM on July 8, 2018 [1 favorite]

So far, just found some bible readings, recorded sermons, and bird sounds.

S3 is just one big 21st century Chick tract.
posted by rhizome at 10:33 AM on July 8, 2018

I removed all uninteresting (in my opinion) files like images. Most images names are auto-generated.

O, my friend.
posted by Going To Maine at 10:56 AM on July 8, 2018