Not as subtle and intricate as AskMefi. But way faster.

November 25, 2007 4:17 PM

Ask 500 (or 100) people: Random participants answer each other's polls on prayer in school, the bible, philosophers, iraq, social habits, love & marriage, materialism, freedom of speech, or whatever topic of interest someone wants to open up for a very momentary spotlight, and reasonably accurate data.

I just found the questions people choose to ask kind of interesting - often you learn as much from the way they frame their issue and the choices they offer as from the numbers that come back.
Not sure if the current audience is skewed, but be warned results are occasionally depressing :)

posted by mdn (28 comments total)

Of course, one could markmany ways in which "random internet" users are not random. They skew younger, wealthier, single male, more liberal, more English-speaking (and so on.) The poor, and members of many minority groups, are less likely to have access to the internet. I'd imagine that the older one is, and the more gainfully employed / married / with kids one is, the less likely they are to participate.

It's not going to be very accurate at all. But possibly interesting anyhow.
posted by Dee Xtrovert at 4:27 PM on November 25, 2007

this is a great idea. When musing about psychology and and such, often imagine how cool it would be to have access to a large group of people t pole. I imagine, however, that data gleaned from this site would be skewed towards a rather narrow demographic of folks able and willing to answer.

still a cool idea and i'll check out.
thanks, mdn!
posted by es_de_bah at 4:28 PM on November 25, 2007

How accurate are the results?

Sometimes I want to throttle people who substitute a statistic for analysis of validity. Sometimes.
posted by a robot made out of meat at 4:44 PM on November 25, 2007

Not sure if the current audience is skewed...

Well, sure, it's skewed. It's the opposite of a random sample. How could it not be skewed? According to their FAQ:

Who are the respondents?

We reach a broad demographic of internet users, representing every walk of life.

And:

How does Ask500People gather votes?

* People put our widget on their sites and help us gather votes.
* We've made arrangements with several high-traffic sites to be able to display polls there when needed.

So their respondents are a broad range of internet users...who are self-selected, and motivated enough to choose to download the widget of a website encouraging people to ask hot-button questions?

Cool interface, though.
posted by rtha at 4:47 PM on November 25, 2007

Reading the questions and comments makes my faith in humanity go back to youtube levels :(
posted by a robot made out of meat at 4:49 PM on November 25, 2007

Interesting, but only representative if the 'average' you want is of 'Internet users' rather than the public at large.
posted by Kadin2048 at 4:49 PM on November 25, 2007

I can't wait till someone asks, "How is babby formed?"
posted by SassHat at 4:53 PM on November 25, 2007 [2 favorites]

The question I usually ask strangers: "Should a dame go all the way on a first date?"
posted by growabrain at 4:56 PM on November 25, 2007

These new sites where a map of the world sprouts pins in real time are all sort of mesmerizing to me, though I wait the killer-app one.

In any case, this may give you an idea of how skewed this "data" is: someone just asked "do you believe in God," and the "nos" were 38%. I don't believe for a second that we atheists exist in such large numbers worldwide.
posted by grumblebee at 5:00 PM on November 25, 2007

Did we break it? I'm just getting connection timeouts...
posted by normy at 5:32 PM on November 25, 2007

Same, normy.
posted by Stewriffic at 5:43 PM on November 25, 2007

Hey folks, thanks for checking out the site. We're working on bringing it back up and I'll be back in a bit to respond more in-depth.

We have a bit more of a ... talk radio/right-leaning crowd on the site right now as it was just featured as the Kim Komando cool site of the day. If you go back a week's time you'll see more moderate and often left-leaning comments. We're hoping as we scale up we'll achieve a happy medium with good discussion/debate happening.
posted by adragushan at 7:11 PM on November 25, 2007 [7 favorites]

A site gets borked and then the CEO turns up to apologize.

Only on MetaFilter, folks.
posted by Avenger at 7:47 PM on November 25, 2007 [3 favorites]

heh, didn't think we'd break it... sorry about that. Yeah, I expected "skewed" in the standard internet sense of not representing the poor or non-geeky very well, but I got a vague sense of maybe more skewed than that from the brief overview I saw - but as adragushan said above, that's apparently due to its recently having been featured on a right wing site.

Which to me is just another interesting aspect of these kinds of polls - will there be a wave of traffic from that posting resulting in biased data, that you can then see in the overall data of the site, so to speak? I like the idea that at a certain point the data would more or less even out and provide responses that were meaningful, if not perfect representatives of the entire culture. (but I guess plenty of the questions put up could be to locate the demographics of the audience, if we wanted to)
posted by mdn at 8:04 PM on November 25, 2007

This is great fun, thanks.
posted by zardoz at 8:26 PM on November 25, 2007

So where is the vote button? I signed up and logged in but the main box in the middle just has some sort of rocket ship icon and nothing else in it. There doesn't seem to be any way to actually vote.
posted by octothorpe at 8:52 PM on November 25, 2007

The ability to get your friends together to respond to a poll makes such polls so much more vulnerable to coordinated attack. There's got to be an online polling service somewhere that serves visitors random poll questions, without giving users any ability to navigate to a known question.
posted by a snickering nuthatch at 9:16 PM on November 25, 2007

(a serve-visitors-random-questions kind of site would also make it more difficult for a single person to vote multiple times)
posted by a snickering nuthatch at 9:18 PM on November 25, 2007

Ok, we're back up for now. The site is hosted across 5 dedicated web servers right now but it's pretty resource intensive.

rtha: Regarding skewed results, the people voting on the Ask500People site are of course self-selecting, which affects the *questions* asked and the comments on the site, but in terms of results and votes, the people on the site rarely represent more than 40% of votes cast and the two types of voters are broken down on the results tab so you can see how the community votes differently than people seeing the questions on other sites.

And of course asking 100 people isn't statistically significant by any margin, but we're in beta and learning like crazy how to improve things and scale the site. Once we're asking 500 people the results will be more meaningful.

mdn: About whether or not the recent influx of right-leaning folks is changing the results, it certainly has. I expect the effect will settle down in time. (The weekend is a low point for our usual crowd, but it's when so many new people just arrived..) That said, because many respondents aren't on (and have no idea about) ask500people, the results might not skew as far as you think.

24 days ago: http://www.ask500people.com/questions/do-you-believe-in-god-or-any-other-divine-entity
18 hours ago: http://www.ask500people.com/questions/do-you-believe-in-god-4819

There are many other factors in the mix as well such as time of day which determines in part the countries which are online/active at that time, which also affects results.

Long story short, the site is a proof of concept for now, and we'll see where we can take it. :)

octothorpe: My guess is you're using Linux? I'm afraid there's a bug we haven't gotten to yet which affects Linux users. Sorry about that. The vote button would be at the top-right corner of the map if you could see it.

Jpfed: Absolutely, the polls are "gameable" to some degree. As the vote counts get higher that will take more and more effort, and we also balance that out by having many votes come from people who aren't on the site itself. About serving people random questions - good point. It's not really part of our model right now, but at some point we'll be moving towards something that might accomodate that.

Thanks for the feedback and ideas everyone.
posted by adragushan at 9:54 PM on November 25, 2007 [2 favorites]

Octothorpe, you vote by clicking the green vote button in the upper-right corner of the map. Unfortunately, you can only vote on the question the site has chosen to highlight until that question reaches the required number of votes and is replaced by another one. (See the FAQ link at the bottom of the page.)
posted by saslett at 10:49 PM on November 25, 2007

Here was the question when I checked it out.

Everything is Gravity Light is everywhere Celestial bodies form from the movement that is existence

How do you answer that?
posted by Bonzai at 10:54 PM on November 25, 2007

I said undecided.
posted by cell divide at 10:58 PM on November 25, 2007

I'd go for

The whole is pervaded by a smell of paraffin...
posted by Phanx at 1:41 AM on November 26, 2007

See also Jyte. It's simpler.
posted by hoverboards don't work on water at 2:27 AM on November 26, 2007

I suppose it might be possible to 'de-skew' the results somewhat by asking users to complete a questionnaire as part of the registration process. One could for instance record age, gender, location, ethnicity and religious and political leanings, although of course it's down to individual honesty.

With a sufficiently large user-base one could then specify that a particular question should be answered by 100 atheists, or by individuals from a statistically balanced group of ages, locations or genders.

I can't seen any major technical hurdles there, although one would need a significant level of membership, and you'd get a few "sorry, too many right-leaning european bisexual catholics have already answered this" - type situations.... and of course there are issues of personal privacy.... but still, it would be pretty cool.
posted by le morte de bea arthur at 2:59 AM on November 26, 2007

No voting by Linux users might skew the results a bit but since it's skewed so far to the right right now, it probably doesn't matter. (Actually I have no idea where American Linux users might fall on the political spectrum).
posted by octothorpe at 4:36 AM on November 26, 2007

Neat, but a wee bit dizzying. Glad I don't have ADHD.
posted by Unicorn on the cob at 11:10 AM on November 26, 2007

>How do you answer that?

A suffusion of yellow.
posted by pompomtom at 4:51 PM on November 26, 2007