Figure / Ground
October 12, 2018 2:02 PM Subscribe
The New York Times has just created an online map showing the silhouettes of the 125,192,184 buildings in the United States of America.
The map was created using Microsoft data (available and documented on GitHub) which was created using machine learning techniques from satellite and aerial imagery. A version of the map is also available on Esri's site.
The map was created using Microsoft data (available and documented on GitHub) which was created using machine learning techniques from satellite and aerial imagery. A version of the map is also available on Esri's site.
From TFA: but this is the first comprehensive database covering the entire United States.
I'm a little puzzled because google maps has had this data for years? But maybe not for, like, every square mile of the US?
Anyway, the black & white rendering without any other geographical/topological features definitely reveals patterns in an interesting way.
posted by GuyZero at 2:36 PM on October 12, 2018 [1 favorite]
I'm a little puzzled because google maps has had this data for years? But maybe not for, like, every square mile of the US?
Anyway, the black & white rendering without any other geographical/topological features definitely reveals patterns in an interesting way.
posted by GuyZero at 2:36 PM on October 12, 2018 [1 favorite]
Google Maps hasn't had this data consistently for the entire US. More importantly even if Google had it, no one else did, because their data is closed. Microsoft generated this building data from aerial imagery and then made it free for everyone to use. Like the NYT here. It's good stuff.
posted by Nelson at 2:43 PM on October 12, 2018 [4 favorites]
posted by Nelson at 2:43 PM on October 12, 2018 [4 favorites]
The name of the small town where I live, obliterates my whole neighborhood, and I have reached the limits of the zoom, I cannot see what is underneath the name.
It is interesting to see the newer subs, how they make smiley faces and stick figures.
posted by elizilla at 2:46 PM on October 12, 2018 [1 favorite]
It is interesting to see the newer subs, how they make smiley faces and stick figures.
posted by elizilla at 2:46 PM on October 12, 2018 [1 favorite]
They omitted my (free-standing) garage. Who is covering up?
posted by hexatron at 2:46 PM on October 12, 2018 [5 favorites]
posted by hexatron at 2:46 PM on October 12, 2018 [5 favorites]
Um no, Google Maps API is wide open and freely available. Note that link is pretty wonky and for developers, but it's wide open.
posted by Abehammerb Lincoln at 2:53 PM on October 12, 2018 [1 favorite]
posted by Abehammerb Lincoln at 2:53 PM on October 12, 2018 [1 favorite]
Microsoft generated this building data from aerial imagery and then made it free for everyone to use.
Yeah, "first comprehensive open database covering the entire United States" is something I could get behind. Sad that the NYT has had to let articles go without editing. Great visualizations.
posted by GuyZero at 2:58 PM on October 12, 2018 [1 favorite]
Yeah, "first comprehensive open database covering the entire United States" is something I could get behind. Sad that the NYT has had to let articles go without editing. Great visualizations.
posted by GuyZero at 2:58 PM on October 12, 2018 [1 favorite]
that is some scifi shit right there boy howdy
posted by gwint at 3:03 PM on October 12, 2018 [1 favorite]
posted by gwint at 3:03 PM on October 12, 2018 [1 favorite]
Pretty good way to view population density too.
Also I'm honestly pretty stunned at the web performance.
posted by gwint at 3:06 PM on October 12, 2018
Also I'm honestly pretty stunned at the web performance.
posted by gwint at 3:06 PM on October 12, 2018
Have you priced what it'd cost to build this app and serve it on the NYT web site using Google Maps? Assuming it's even technically possible; last I checked the API libraries didn't have support for anything like this rendering and you weren't allowed to download a copy of the data from Google for your own rendering and use. Also Google doesn't have a complete set of buildings for the whole country.
FWIW the NYT map is being served via MapBox. It looks like it contains some OpenStreetMap data (the labels, the coastlines and boundaries). Not positive where the terrain shading comes from but presumably MapBox's terrain product which in turn comes from various sources. I believe it uses Tippecanoe to boil down all of Microsoft's building data into vector tiles for serving with the map. As gwint mentions, it is remarkably fast.
One unusual feature of the MapBox presentation; the projection they're using, where it feels like the map is stretching away from you at the top of the screen.
posted by Nelson at 3:10 PM on October 12, 2018 [2 favorites]
FWIW the NYT map is being served via MapBox. It looks like it contains some OpenStreetMap data (the labels, the coastlines and boundaries). Not positive where the terrain shading comes from but presumably MapBox's terrain product which in turn comes from various sources. I believe it uses Tippecanoe to boil down all of Microsoft's building data into vector tiles for serving with the map. As gwint mentions, it is remarkably fast.
One unusual feature of the MapBox presentation; the projection they're using, where it feels like the map is stretching away from you at the top of the screen.
posted by Nelson at 3:10 PM on October 12, 2018 [2 favorites]
Interesting, there's a big blank spot where Fenway Park should be. Is that not a building?
posted by bondcliff at 3:12 PM on October 12, 2018
posted by bondcliff at 3:12 PM on October 12, 2018
I can see my house from here!
Damn it, Corpse in the Library. Isn't there another thread where you'd be happier??
posted by Naberius at 3:13 PM on October 12, 2018 [2 favorites]
Damn it, Corpse in the Library. Isn't there another thread where you'd be happier??
posted by Naberius at 3:13 PM on October 12, 2018 [2 favorites]
I like the way it fades away into pure icy whiteness at the northern border, where Canada is sometimes rumoured to exist.
posted by sfenders at 3:20 PM on October 12, 2018 [1 favorite]
posted by sfenders at 3:20 PM on October 12, 2018 [1 favorite]
If you hold down control (or right-click with a mouse) you can rotate the projection.
posted by gwint at 3:29 PM on October 12, 2018 [2 favorites]
posted by gwint at 3:29 PM on October 12, 2018 [2 favorites]
The Google Maps API is not at all free or open in the same sense.
The Microsoft data is available for the everyone to download in full. It's licensed to the general public under the terms of an Open Data Commons license, which lets anyone redistribute, modify, or build upon the data, subject to only a few requirements (you have to give proper attribution, and maintain the same license terms for any modified version of the database).
The Google Maps data is only available through their API. You need a Google account and access token to access it, and your use is subject to Google's terms which are much more restrictive. For example:
posted by mbrubeck at 3:40 PM on October 12, 2018 [12 favorites]
The Microsoft data is available for the everyone to download in full. It's licensed to the general public under the terms of an Open Data Commons license, which lets anyone redistribute, modify, or build upon the data, subject to only a few requirements (you have to give proper attribution, and maintain the same license terms for any modified version of the database).
The Google Maps data is only available through their API. You need a Google account and access token to access it, and your use is subject to Google's terms which are much more restrictive. For example:
(a) No Scraping. Customer will not extract, export, scrape, or cache Google Maps Content for use outside the Services. For example, Customer will not:(i) pre-fetch, index, store, reshare, or rehost Google Maps Content outside the services; (ii) bulk download geocodes; (iii) copy business names, addresses, or user reviews; or (iv) use Google Maps Content with text-to-speech services…Fees? Yes, that's right. Google charges fees for access to the Maps Platform beyond certain limits.
(b) No Creating Content From Google Maps Content. Customer will not create content based on Google Maps Content, including tracing, digitizing, or creating other datasets based on Google Maps Content.
(c) No Re-Creating Google Products or Features. Customer will not use the Services to create a product or service with features that are substantially similar to or that re-create the features of another Google product or service…
(d) No Use With Non-Google Maps. Customer will not use the Google Maps Core Services in a Customer Application that contains a non-Google map…
(e) No Circumventing Fees. Customer will not circumvent the applicable Fees.
posted by mbrubeck at 3:40 PM on October 12, 2018 [12 favorites]
Thank you, gwint! That also helps zoom in even further. But boy, for a second I really confused myself. Somehow I rotated everything so south was at the top of the page.
posted by queensissy at 3:42 PM on October 12, 2018
posted by queensissy at 3:42 PM on October 12, 2018
Given how much construction/destruction there's been in Seattle, particularly around the South Lake Union area, we can infer that (at least for Seattle) the satellite data used to make this map is at least 5 years old.
posted by mhum at 4:00 PM on October 12, 2018 [2 favorites]
posted by mhum at 4:00 PM on October 12, 2018 [2 favorites]
stoneweaver: can you point out a couple of places where there should be buildings but you think there are none because there are no street addresses? Not doubting you at all! Just curious to learn more, both for what it means for Microsoft's data and for addresses.
(Folks on the rez with no street addresses are a hot topic in North Dakota right now, which figured out how to disenfranchise Native American's votes).
posted by Nelson at 4:30 PM on October 12, 2018 [1 favorite]
(Folks on the rez with no street addresses are a hot topic in North Dakota right now, which figured out how to disenfranchise Native American's votes).
posted by Nelson at 4:30 PM on October 12, 2018 [1 favorite]
Not only can you see my house from here, but you can see my guest house (former garage) and barn! It's kind of interesting, in that you can kind of tell the new developments from the old settlements both based on the size of the the buildings and the number of what appear to be outbuildings.
posted by drlith at 4:50 PM on October 12, 2018
posted by drlith at 4:50 PM on October 12, 2018
If you go look at International Falls, MN 56649, the main dense grid of the town should be virtually completely black with houses. Every single one of them does have its own full street address, and yet still aren't listed. The apartment complex I'm in right now as I type is missing the other building, even though it has a unique street address (not just fake "apartment complex" addresses, but this building is e.g. 1000 Major Town Road, and that building is 1002 Major Town Road).
Weird.
posted by tubedogg at 4:50 PM on October 12, 2018
Weird.
posted by tubedogg at 4:50 PM on October 12, 2018
Now I can really start working on that flight simulator I've been putting off since forever!
posted by RobotVoodooPower at 5:25 PM on October 12, 2018
posted by RobotVoodooPower at 5:25 PM on October 12, 2018
It seems to be missing some, but not all, fire lookouts. For example, in North Cascades National Park, Copper Mountain Lookout (48.909167, -121.461389) doesn't seem to be there, but Desolation Peak Lookout (48.911243, -121.016235) is. I also saw at least one purported building a mile or two south of where the Copper Mountain Lookout would be that I'm pretty sure isn't there in real life (way off trail, doesn't show up on the google maps satellite view). Perhaps it's not surprising that the building detection wouldn't work as well in places where there are very few buildings.
posted by surlyben at 6:00 PM on October 12, 2018 [1 favorite]
posted by surlyben at 6:00 PM on October 12, 2018 [1 favorite]
It's pretry far from accurare... in our town is took me less than a few seconds to find several major problems, including outlines of buildings that hadn't been there for years and some bizarre misshapen anomalies where big box-type buildings are.
posted by saladin at 7:06 PM on October 12, 2018 [1 favorite]
posted by saladin at 7:06 PM on October 12, 2018 [1 favorite]
Yeah my alma mater, University of Minnesota Duluth, is wildly wrong with many buildings missing. No discernible pattern for age or addressability either.
posted by traveler_ at 7:33 PM on October 12, 2018
posted by traveler_ at 7:33 PM on October 12, 2018
The patterns are really neat, though I agree with others that it is a bit hit and miss when you look at close details in a place you know well.
posted by Dip Flash at 8:46 PM on October 12, 2018
posted by Dip Flash at 8:46 PM on October 12, 2018
I don't think this was designed for people without a fast internet connection.
I'm sure the images will load one day.
[...]
posted by lollusc at 2:37 AM on October 13, 2018
I'm sure the images will load one day.
[...]
posted by lollusc at 2:37 AM on October 13, 2018
[...]
Okay, I think the images have loaded now, but the interactive map hasn't. Assuming it's not disguised as one of the images?
posted by lollusc at 2:40 AM on October 13, 2018
Okay, I think the images have loaded now, but the interactive map hasn't. Assuming it's not disguised as one of the images?
posted by lollusc at 2:40 AM on October 13, 2018
Customer will not extract, export, scrape, or cache Google Maps Content for use outside the Services.
Scraper, no scraping!
Scraper, no scraping!
Scraper, no scraping!
Awww, man...
posted by Naberius at 5:46 AM on October 13, 2018 [2 favorites]
Scraper, no scraping!
Scraper, no scraping!
Scraper, no scraping!
Awww, man...
posted by Naberius at 5:46 AM on October 13, 2018 [2 favorites]
Building with building footprints, a MapBox blog post about how to build a map similar to the NYT's. (I believe the NYT did a lot of work cleaning up the Microsoft data; this post is more about how to wrangle the Microsoft data into a format you can serve quickly using Tippecanoe.)
posted by Nelson at 3:11 PM on October 16, 2018
posted by Nelson at 3:11 PM on October 16, 2018
The building I'm living in in Columbia City (South Seattle), isn't showing on the map, though it began construction at least 3 years ago.
posted by QuakerMel at 5:00 PM on October 16, 2018
posted by QuakerMel at 5:00 PM on October 16, 2018
« Older Say not the struggle naught availeth | Resembled a budgie that had inhaled a lumberjack’s... Newer »
This thread has been archived and is closed to new comments
Be smarter than me: clean your laptop's screen before zooming over to the wilderness areas and trying to figure out what the unexpected buildings in them are.
posted by The corpse in the library at 2:36 PM on October 12, 2018 [5 favorites]