New Zealand-born physicist Sean Gourley on the mathematical patterns that underly war, global intelligence, and his roles as a political advisor to the Iraqi government, the UN and the Pentagon.
{ Comments on this entry are closed }
Every year the Cartography and Geographic Information Society holds a competition – the equivalent of the Academy Awards for maps — for the best map of the United States. While it’s often won by one of the major players in the mapping world, like the US Census bureau, in 2010 it was won by a one-man shop run by David Imus of Eugene Oregon.
Imus’s map differed not just in the scale of operation, but in the very way he went about constructing it. Traditional map-makers make use of algorithms to position labels, size towns and arrange points of interest, and they farm out the rest of the work to teams in India to manually fill in. While Imus’ map was constructed on a computer it didn’t use algorithms, leading to Imus toiling 6,000 hours, 7 days a week, for two years, obsessing over font types, state boundary colors and things like what symbol to use for airports. The little touches made the difference – the map was beautiful.
Imus, his map and his technique for producing it, seem like an anachronistic throw back to a 1950s world of exacto knives. But his approach is both an example of the future of information and a reminder of its past. As the world is under going an explosion of big data, it’s the multi-dimensional map interface that will play the key role for displaying connected intelligence. As Imus’ showed, producing these maps will require art and algorithms to come together.
Finding intelligence in Big Data
Much has been written about the proliferation of data over the past few years. Big data has been compared to the new oil, and like with oil, the environmental footprint of data is being felt with data warehouses now consuming around 2 percent of the electricity in the US.
But data in and of itself is useless — just a pile of 1′s and 0′s stacked together in server farms dotted across the nation. For data to actually be useful it needs to have algorithms run on top of it, and these algorithms need to lead to decisions. These decisions could be small — like which ad should I insert at the start of a YouTube video — or large such as should I insert 30,000 new troops into Iraq.
{ Comments on this entry are closed }
“Killing drug lords gets headlines, but complexity analysis suggests they are the wrong people to target to bring down a cartel”.
New Scientist put together an interesting piece called ‘Destroying drug cartels, the mathematical way’. The article focussed on the recent death of Lazcano the leader of the Mexican cartel Los Zetas and on how mathematics, modeling and complexity theory are being used to fight drug wars like these.
Mexican drug ecosystem
My research looked at similar drug fueled conflicts and we found that the mathematical signatures that defined the violence in places like Colombia looked very similar to the signatures in more traditional type insurgencies like Iraq. The way that drug cartels organise themselves, evolve and compete with each other is very similar to the dynamics seen within insurgent groups in places like Iraq and Afghanistan. This is because there are only a few effective ways to organise a force against a conventional army/paramilitary unit – and either the groups evolve to find this solution or they die trying. The upside of this though is that strategies employed in Iraq to break apart insurgencies can be used to inflict damage to the Mexican drug ecosystem.
{ Comments on this entry are closed }
A few months back I got an email from Bruce Cahan who was putting together a TEDx event called TEDxNewWallStreet. The purpose of the event was to challenge some of the fundamental paradigms of Wall Street, and Bruce wanted to know if I would put together a lecture for it based on my work with algorithms and Big Data. The only requirement is that the content was somewhat experimental, with bonus points if it was a touch controversial. Experimental and controversial are two great motivators for me so I told him I was a ‘maybe’. I spent the next week bouncing around ideas with some of my favorite thinkers in Silicon Valley, reading journal articles, sketching notes, analyzing equations and drinking a lot of coffee down at cafe Trieste. At the end of the week I finally came up with something that I wanted to talk about. A talk at the intersection of High Frequency Trading, machine readable news, and predator-prey ecosystem stability. A talk challenging our reliance-of and our control-by algorithms.
That was the video, click through to read more of the story behind the ideas
[click to continue…]
{ Comments on this entry are closed }
While building Quid, in the beginning, one of the questions that was always hardest to answer was ‘what does your company do’. This was hard because there really isn’t a short soundbite answer to the question. How do you explain the collection of data, structuring of information and visualization of multi-dimensional space in 30 seconds?
Over time though we got better at telling the Quid story, and over time the rest of the world has started to understand more and more of the technical elements of the Big Data space. The basic idea behind Quid is this: we are living in an increasingly complex world, with millions of interacting components and feedback loops, a world that may be beyond our raw human ability to understand it. With this in mind we set out to build a new set of tools to enhance the human ability to understand this complexity. Think of Quid as the new AI — not Artificial Intelligence, but instead Augmented Intelligence. I could go into more details, but I think this video does a pretty good job of telling the Quid story…..enjoy.
{ Comments on this entry are closed }
It’s been a little while since I last got the chance to sit down and write here. But for the last 18 months or so I’ve been immersed in a new project that is called Quid. Quid is the company I started back in Dec 2009, it is now at 60 people and we’re doing some pretty amazing things with data, mathematics and visualization. At Quid we are building a global intelligence platform, a place where open source intelligence is collected, structured and visualized to help people understand and make better decisions about the complex world we live in. It’s been an interesting transition out of the academic world and into the world of startups, products and venture capital. But it has been a move that has allowed me to scale up an awesome team, raise a significant amount of money and engage directly with real world problems. Now that Quid is coming up to it’s 2 year mark I hope that I will get more time to sit and write here and share some of my thoughts, ideas and theories that are driving what we are doing here at Quid. Check out Quid.com for more information about what we are doing.
{ Comments on this entry are closed }