The crowd finds what no one room would.
Ask everyone. People compare two ideas at a time. The comparisons build the ranking. No one picks a winner. The group does.
- Ask everyone
- Compare two
- Ranking builds
- The group decides
Built for the people who want to give more
A traditional survey asks everyone the same questions and stops. Nobody can add what you did not think to ask. A vision lab project works the other way. Most people vote a few times and leave. A smaller group keeps going, and what they give is worth more than the rest combined.
- Everyone can stop after a handful of quick votes.
- Anyone who wants to can agree or disagree, suggest an improvement, or add a new idea.
- The ones who keep going add the ideas and fixes you would never have thought to ask for.
Why ask the crowd
One boss or one workshop hears a few voices. A crowdsourced question hears everyone, in days, in their own words. When many people judge independently, the combined view is usually more reliable than any one opinion. It also surfaces ideas nobody in the room would have thought to ask for.
Why compare two ideas at a time
Ticking favorites from a list goes wrong in familiar ways:
- Ideas posted first collect the most ticks.
- People tick their friends’, their boss’s, or their own.
- Nothing forces a trade-off, so everything looks good.
Comparing two ideas is quick and forces a choice. Fifty or so quick choices add up to a full ranking. You never see your own idea, or who wrote the ones in front of you.
How the ranking is built
Every idea starts on 1,000 points. It works like a chess rating. Beating a stronger idea earns more. Losing to a weaker one costs more.
Scores settle as ideas meet more opponents. The leaderboard and report show each score on a 0 to 100 scale, strongest idea at 100.
Do late ideas have a fair chance?
Yes. Early ideas have had longer to collect points, so late ideas get help:
- New ideas are shown more often at first.
- New ideas move faster. Their score changes about six times as much per comparison, easing off over their first 40.
- New ideas meet opponents near their level, then tougher ones as they win.
We simulated thousands of projects with a known best idea. How often a great idea finished in the top 5:
A simulation, not a guarantee. It assumes people usually, but not always, pick the better idea. A merely good idea isn’t lifted by the faster start.
How ideas are grouped into themes
Once people have voted, ideas are grouped into themes, so you can see what the group wants and not only which single ideas won. The system suggests the themes, and you can rename or move any of them.
- Themes are ranked by how often their ideas win, not by how many ideas there are. A topic people mention a lot can still lose when its ideas are compared with others.
- An idea has one main theme. That is the one counted in the rankings, so nothing is counted twice.
- People taking part never see your changes.
What keeps it honest
What it can and can’t tell you
The ranking shows what this group prefers, as a group. It is a strong starting point for decisions, not a verdict. It works best with a good question, plenty of people, and decision-makers who act on the top ideas and say so.
We test it openly, including where it falls short. See the test results and known limits.
Questions people ask
Why can’t I just vote for the ideas I like?
Comparing two ideas gives each one a fair share of attention. Early or popular ideas can’t run away. Every choice you make still moves ideas up or down.
How many should I compare?
Ten is plenty. Stop or keep going after a round. More comparisons make the ranking more accurate.
What do the skip options do?
“Say the same thing” counts as a tie and helps organizers spot duplicates. “Can’t decide”, “Neither is strong” and “Both are strong” also count as ties.
Can I add more than one idea?
Yes, one at a time, while the project accepts new ideas.