Every year the Cartography and Geographic Information Society holds a competition – the equivalent of the Academy Awards for maps — for the best map of the United States. While it’s often won by one of the major players in the mapping world, like the US Census bureau, in 2010 it was won by a one-man shop run by David Imus of Eugene Oregon.
Imus’s map differed not just in the scale of operation, but in the very way he went about constructing it. Traditional map-makers make use of algorithms to position labels, size towns and arrange points of interest, and they farm out the rest of the work to teams in India to manually fill in. While Imus’ map was constructed on a computer it didn’t use algorithms, leading to Imus toiling 6,000 hours, 7 days a week, for two years, obsessing over font types, state boundary colors and things like what symbol to use for airports. The little touches made the difference – the map was beautiful.
Imus, his map and his technique for producing it, seem like an anachronistic throw back to a 1950s world of exacto knives. But his approach is both an example of the future of information and a reminder of its past. As the world is under going an explosion of big data, it’s the multi-dimensional map interface that will play the key role for displaying connected intelligence. As Imus’ showed, producing these maps will require art and algorithms to come together.
Finding intelligence in Big Data
Much has been written about the proliferation of data over the past few years. Big data has been compared to the new oil, and like with oil, the environmental footprint of data is being felt with data warehouses now consuming around 2 percent of the electricity in the US.
But data in and of itself is useless — just a pile of 1′s and 0′s stacked together in server farms dotted across the nation. For data to actually be useful it needs to have algorithms run on top of it, and these algorithms need to lead to decisions. These decisions could be small — like which ad should I insert at the start of a YouTube video — or large such as should I insert 30,000 new troops into Iraq.
For many smaller decisions we employ software agents (algorithms that maximize utility functions) and these work without human intervention. These algorithms are the backbone of the global online advertising industry and now dominate the global financial markets. We call them “Artificial Intelligence.”
But we should be aware of the limitations of artificial intelligence — it still has not been able to fully solve the game of chess (checkers yes, but chess no). There is a second class of more complex decisions that algorithms can’t make by themselves. You can’t type into Google “should I send troops into Iraq” and push “I feel lucky” and expect to get an answer that you would bet human lives on. We should remember that chess itself was once a training tool for war.
When Kasparov was beaten by IBM’s deep blue back in 1997, the narrative was that Artificial Intelligence had finally surpassed human thinking. But what most people don’t know is that following that loss Kasparov founded a competition known as freestyle chess, where human chess players could use any software they wanted to play games. The results were interesting: humans teamed up with machines could beat any of the autonomous machines, and it was the hackers – who knew intimately how software works – and not the Grand Masters that came out victorious.
This is the secret to augmented intelligence – the second AI. Systems that rely on humans and computers interacting are smarter than systems that use each individually. The world’s most difficult problems will be solved by a combination of humans and computers coming together.
Already augmented intelligence is having a significant impact on weather forecasting, and generating new scientific breakthroughs. The combination of human minds and the machine-based processing of Big Data is creating a new toolkit to solve the most difficult problems in the world.
How much heavy-lifting can humans do? The human brain can take in massive amounts of data — it has been calculated that the human brain can take in 10^7 bits/second through the combination of our senses. However, despite the large amount of data coming in, we are only conscious of a fraction of this data. Our brains perform an amazing filtering operation on the input data and choose only a select amount for us to be made conscious.
But experts brains, it turns out, work a little differently. The data coming into the brains of expert chess players passes outside the bounds of consciousness and straight into the pattern-matching region of the brain, allowing them to quickly determine the state of the chess pieces and enabling them to rapidly compute the next optimal move. The experts have a multi-dimensional view of the data — a kind of internal map that they call on.
It is this map interface — with the subtle relationships between the data points — that will allow humans to extract more information from the data then the simple one dimensional list view will allow. Like the expert’s brain, the map layout is a break away from the one-dimensional interface, adding a level of augmented intelligence on top of the artificial and collective layers. I think the map design will become the backbone of today’s web experience.
Maps take advantage of one of our highly developed evolutionary traits: spatial reasoning. Modern maps present you with information in a way that allows you to visually search, navigate, explore, get lost and see patterns.
In a connected environment, the map design is even more powerful. The beauty of an application like Google Maps is that it shows, or hides, information from you. As a user you are unaware of these subtle tricks, but if you deconstruct the algorithms you can see why it works. Zoom into somewhere like Austin, Texas on Google Maps and you will see the town just sort of ‘stand out.’ Google has created an information buffer zone around the important object — a sort of ‘white space’ that allows the user to see the important piece of data.
The connected map is enjoying a resurgence thanks to the rise of mobile phones, GPS and Wi-Fi positioning systems. Indeed Google Maps is now ingesting 20 petabytes of information every week from 1,300 different data sources into its mapping software. But mapping is not always about geography.
A map is a multi-dimensional rendering of any type of information that highlights the relationships and relative importance of objects. We are now starting to see maps of many different types of information, led primarily by the field of network theory.
Maps are also becoming the primary interface for how humans gain insights from big data. It is the place where they can do the things that humans are uniquely suited to: exploring, looking for patterns, telling stories and of course asking questions and getting lost. We can now render maps in three dimensions. If we add color, size and shape we have a pseudo 6-dimensional space. Time can slice the data into a 7th dimension.
By making these maps interactive we can allow the user to project different slices of the data into this 7-dimensional space and in doing so the user can analyze very complex data landscapes from multiple perspectives. If we make manipulating the data in the multi-dimensional space easy, the user can have a conversation with the data.
We are in essence trying to recreate with algorithms and maps the intuitive response that an expert has to the patterns they see on a chessboard. It’s a way for the computer to calculate the shapes, and to let the human do the strategic thinking. This is what we are building at my startup Quid
I predict the world of information technology will move from the 1-dimensional list view of interacting with information to the multi-dimensional map view of information. Here users will be able to do what humans do best and this is the interface that we will use to make very complex decisions. If you picture the war General of the movies, it’s a map he’s depicted strategizing over — not a list.
(*article reposted from Roadmap 2012 book that was published as part of the 2012 Gigaom conference)