Hive Architecture

paradox of the crowd?

If I told you that a big group of people is better at doing things than one person alone, or even a smaller group of people, you’d probably agree with me. Science benefits from collaboration, as does carrying heavy objects. Kitchen wisdom says that ‘many hands make light work.’ And then there’s Wikipedia.

But if I told you that big groups of people are terrible at doing things, you’d probably still agree with me. Large organizations are inefficient, and mob mentality causes regular people to make stupid/appalling decisions. Our kitchen sage contradicts itself with the warning that ‘too many cooks spoil the broth.’ There was the recession.

Isn’t there a contradiction here? The web is brimming with discussions about the value or deficiencies of social production and over ‘the wisdom of the crowd’ versus ‘the stupidity of the crowd,’ as if it were one or the other, but the truth is that a group is neither wise nor stupid, powerful nor impotent. What’s important is just the observation that a group is not equal to the sum of its parts. Groups of smart people can do stupid things, a bunch of ignorant guessers can make better predictions than an expert, and, despite the fact that I live with three friends who keep their bedrooms fairly neat, our kitchen is more or less repulsive.

Part of the confusion surrounding ‘crowd wisdom’ comes from the fact that there are two principles at work here. First, Social Psych Rule #1: the variation in a person’s behavior across situations is much greater than the variation between people’s behavior in the same situation. In other words, we’re extremely susceptible to the influence of a situation, basing our behavior much more on the specifics of our environment than our snowflake-unique personalities. Every group has a particular social structure and environment, and that is what’s mostly going to determine whether we give it our all, shirk responsibility, or let our better judgment be swayed. Groups with different structures will do completely different things.

Second, there’s a bit of oft-overlooked math underlying crowdsourcing, that was part of the original formulation of the wisdom of the crowd: collective error = individual error – prediction diversity. Meaning, if you have a group of people with diverse opinions about something tangible (say, the number of jellybeans in a jar), their individual inaccuracy can be averaged out when you put together their answers, yielding one that is surprisingly accurate (better than 55 of 56 individual guesses). It all depends, though, on getting a random sampling of people with diverse opinions – which often isn’t the case with a self-selected group. Take a group of people with similar assumptions/biases and add the influence of a group’s environment on their behavior (the strong tendency to adjust opinions closer to the majority’s, for example) and you end up with a group that does astonishingly dumb things.

The point? Groups aren’t necessarily smart or powerful, but they could be. It isn’t enough to get a bunch of people together in a room or on a website with a great vision of what they could accomplish. The group’s potential hinges on its structure.

architecting the hive

The structure of an online social platform plays a pretty significant role, then. Its aim is to facilitate the kind of group that will lead to the most efficient collaboration, the wisest knowledge, the broadest possible imagination. It must prevent the flood of simultaneous contributions from descending into counterproductive anarchy.

The platform consists of a two-part structure: its architecture and its user community (see Wikipedia sociology for the most thorough obsessive mapping of an online community). The architecture is coded and therefore controllable; the social structure – while ultimately determined by users – is guided by the architecture, interface, and marketing of the site. Ideally, the designer of such a platform would be cognizant of the social repercussions of every element of the site. Ideally, he/she would make deliberate choices about each one of those elements to encourage specific group dynamics and cohesion. In practice, of course, we don’t know nearly enough about what goes on between a website and its users to make such fine-tuned predictions (especially any time a rebellious userbase decides to transform a site into something it wasn’t intended to be). Luckily, what’s unique to web services is that the code of a site can be changed in response to user behavior – a kind of symbiosis between technical and social structure. The immediate feedback in the form of changed behavior on the site provides the perfect opportunity to learn.

There’s a lot to learn. The Internet is still so young that things are exploding on top of it without us really knowing why. The slightest variation in group structure – say, introducing the one-way relationship of ‘follower’ in place of the two-way relationship of ‘friend’ – can have a huge impact on the nature of the group, and make or break a start-up. There are interesting experiments in group formation everywhere and new trends, like incorporating gaming techniques into social network platforms, are rife with implications about human motivation. Academically, all of this is fascinating, but ultimately I’m interested in learning how to build more deliberately-crafted platforms that will help us work together better.

This blog is about that.

Advertisement