While building Quid, in the beginning, one of the questions that was always hardest to answer was ‘what does your company do’. This was hard because there really isn’t a short soundbite answer to the question. How do you explain the collection of data, structuring of information and visualization of multi-dimensional space in 30 seconds?
Over time though we got better at telling the Quid story, and over time the rest of the world has started to understand more and more of the technical elements of the Big Data space. The basic idea behind Quid is this: we are living in an increasingly complex world, with millions of interacting components and feedback loops, a world that may be beyond our raw human ability to understand it. With this in mind we set out to build a new set of tools to enhance the human ability to understand this complexity. Think of Quid as the new AI — not Artificial Intelligence, but instead Augmented Intelligence. I could go into more details, but I think this video does a pretty good job of telling the Quid story…..enjoy.
It’s been a little while since I last got the chance to sit down and write here. But for the last 18 months or so I’ve been immersed in a new project that is called Quid. Quid is the company I started back in Dec 2009, it is now at 60 people and we’re doing some pretty amazing things with data, mathematics and visualization. At Quid we are building a global intelligence platform, a place where open source intelligence is collected, structured and visualized to help people understand and make better decisions about the complex world we live in. It’s been an interesting transition out of the academic world and into the world of startups, products and venture capital. But it has been a move that has allowed me to scale up an awesome team, raise a significant amount of money and engage directly with real world problems. Now that Quid is coming up to it’s 2 year mark I hope that I will get more time to sit and write here and share some of my thoughts, ideas and theories that are driving what we are doing here at Quid. Check out Quid.com for more information about what we are doing.
My colleague and coauthor Neil Johnson makes an appearance on the Russian cable channel RT. In the 6 minute video he talks about the results of the Nature paper and looks at how the model can be used to inform strategic decisions.
Insurgencies are by their very nature difficult to understand. However each time an attack is launched and every time an IED explodes we start to know a little more about the structure of an insurgency. If we combine together enough of these attacks we start to build up a mosaic picture of the insurgency. Their actions can start to be defined mathematically and we can work backwards from these signatures to understand the fundamental forces that underlie the insurgency. This is exactly what we did in our latest research study “Common Ecology Quantifies Human Insurgency“.
With these models we can for the first time quantitatively understand more about what makes an insurgency successful. From our analysis and modeling we find that there are 14 key characteristics that define a successful insurgent ecosystem; these are listed below with a short name to describe the feature.
Many body: There are many more autonomous insurgent groups operating within conflicts than we had previously thought. For example there are 100+ autonomous groups operating in Iraq (as of 2006).
Fluidity: The insurgents are loosely grouped together to form fluid networks with short half-lives. This is very different from the rigid hierarchical networks that have been proposed for insurgent groups.
Redundancy: If we remove the strongest group from the system another group will rise to replace the previous strongest group
Splinter: When a group is broken it does not generally split in half but instead shatters into multiple pieces
Redistribute: When a group is broken the components are redistributed amongst the other groups in the system. The redistribution is biased towards the most successful remaining groups.
Snowball: The strongest groups grow fastest
Tall poppy: The strongest groups are the predominant targets for opposition forces
Internal competition: There is direct competition amongst insurgent groups for both resources and media exposure. They are competing with each other in addition to fighting the stronger counterinsurgent forces.
Independent co-ordination: Autonomous groups act in a coordinated fashion as a result of the competition that exists between them.
Emergent structure: Attacks in both Iraq and Colombia become ‘less random’ and more coordinated over time
Evolution: The strategies employed by the groups evolve over time where successful groups/strategies survive and unsuccessful strategies/groups are replaced.
High dimensional: Connection occurs over high dimensions (i.e. Internet, cell phone etc) and is not dominated by geographic connections.
Non-linear: It is approximately 316* times harder to kill 100 people in an attack than it is to kill 10 people. (*Results for a conflict with alpha=2.5).
Independent clones: the fundamental structure and dynamics of insurgent groups is largely independent of religious, political, ideological or geographic differences.
What can we learn from insurgents? Should the US military adopt more of these principles? Can we apply these organizational characteristics to other problems? You can read more about the research over at the TED blog, including the in depth interview I did with them.
I have been working over the last few days with my team to put together the new website mathematicsofwar.com. This website contains a set of resources to help people better understand the details of our research. It is more science/math focused site and includes copies of our latest research papers, background reading, working drafts and details of the modeling and analysis. We’ll be adding to this siteover the coming weeks and months and building it up as a resource for the quant conflict analysis space.
It’s quite a lot of literature to get through and covers a range of different topics including economics, statistics, physics and computer science. But then that’s what the holidays are for right :)
It looks like our research just got picked up on Slashdot. How long before quant analysis of conflict is also on the daily agenda of policy/strategy/military types.
To explain what was driving this common pattern, the researchers created a mathematical model which assumes that insurgent groups form and fragment when they sense danger, and strike in well-timed bursts to maximize their media exposure. Johnson is now working to predict how the insurgency in Afghanistan might respond to the influx of foreign troops recently announced by US President Barack Obama
A good post from John Robb over at Global Guerrillas about the “Ecology of war” Nature paper. Robb argues that the research put forward in the study has applications to what he calls Open Source Warfare. Robb believes that Open Source Warfare has become the dominant form of conflict for the 21st Century and it [...]
One of the important things to understand about our Nature paper on the ‘Ecology of war’ is that the results in this study go beyond first order statistical analysis and power-law distributions. In this paper we look at the deviations from power-law and we analyse the temporal distribution of attacks. This in itself represents a [...]
A new thought provoking piece by David Berreby on the Ecology of war paper is now up on bigthink.com. In the piece, Berreby does a good job of explaining the underlying forces that drive everyday human behavior. He gives examples of simple behaviors such as how we vote, or what we order that are governed [...]
If you’ve been following the Nature research and you want more detail about the latest work on the ‘Ecology of war’ head across to the TED blog where I’ve done an in depth Q&A with them. In this piece I talk about distributions, modeling, the role that media plays in covering conflict and the history [...]