Center for a New American Security

Any and all scholars of the contemporary Arabic-speaking world need to read Greg Gause's nostra culpa in the latest Foreign Affairs, "Why Middle Eastern Studies Missed the Arab Spring."

Scholars did not predict or appreciate the variable ways in which Arab armies would react to the massive, peaceful protests this year. This oversight occurred because, as a group, Middle East experts had largely lost interest in studying the role of the military in Arab politics.

I am proud to have completed my own studies under the supervision of one of the few scholars still both inclined and equipped to carefully study the role of the military in Arab politics. But as Harb and Leenders point out (.pdf) with regards to Hizballah, few contemporary area studies scholars have either the training or inclination to carefully study the role of military organizations and their activities.

Richard K. Betts, writing in the National Interest:

Ikenberry faults Bush for rejecting the International Criminal Court (ICC) and the “Germ Weapons Ban” (Ikenberry must mean the compliance protocol to the Biological Weapons Convention, not the treaty itself, which Washington ratified in the 1970s). Yet although Clinton signed the ICC treaty, he immediately thought worse of it and recommended against ratification. Clinton also refused to sign the Mine Ban Treaty, a favorite of global-governance enthusiasts—although not because, as Ikenberry suggests, “unipolarity leads to demands by the lead state to be treated differently.” Rather, it is because states with serious national-security policies keep the military capabilities they believe they need. The land-mine treaty is a perfect example of an institution that looks strong on the surface, but weaker in substance. It is a perfect example too of how governments pick and choose which rules they want to accept (and reject) in the vaunted rule-based international system. Indeed, the treaty includes more than 120 signatories. But most of this membership consists of countries without pressing military concerns. The smaller number of states that have not participated are ones that do have such concerns (various vulnerable actors like Pakistan, Iran, Israel, Vietnam, Georgia, Cuba and the two Koreas) and most major powers (China, Russia and India as well as the United States). The nonsignatories represent the most important countries, and more than half the population of the world.

 

Ikenberry compliments Obama for returning to norms of liberal order after the Bush defection, yet the difference for national-security policy is far from dramatic. Obama too rejected all the accords just mentioned.

Obviously enough, what Betts writes here pertains to the somewhat controversial post that kicked off the week. But you should read the entire review essay, because Betts makes a number of other good points and raises many other questions in what is a courteous if brutal review of John Ikenberry's Liberal Leviathan. Among the criticisms of the book advanced by Betts, I find it interesting how even some of the most internationally minded Americans more or less assume U.S. interests to be the same as the interests of the world at large. (This leads to all kinds of problems, you might have read, in third-party military interventions where we assume our interests match up with those of the host nation.)

***

Speaking of the National Interest, the staff there have really outdone themselves in putting together a marvelous May-June issue. As my followers on Twitter are aware, I spent last night and much of this morning reading and digesting some of the essays, including Jacob Heilbrunn's essay on Samantha Power and liberal internationalism. Heilbrunn chastises Power for "dramatizing history through people rather than considering broader forces," and ironically, I think he might be guilty of doing the same here in choosing to focus so exclusively on the words and texts of Power in the context of our military intervention in Libya. But he deserves much credit for taking exception to the content of an individual's arguments without ever resorting to argumentum ad hominem, and his broader criticism of humanitarian intervention is a good one.

Elsewhere in the National Interest is an essay by Eugene Rogan, author of The Arabs: A History, on the revolutions of 2011. Rogan focuses on the region-wide variables that have led to uprisings, so his essay should be read in tandem with Lisa Anderson's brief essay in the new Foreign Affairs parsing the differences between what has thus far transpired in Tunisia, Libya and Egypt. (In light of yesterday's discussion of quantitative methods in conflict analysis, by the way, Anderson remains a living, breathing advertisement for the continued value of area studies in political science: how many other scholars out there have such deep knowledge of not just Libya, Tunisia or Egypt but all three countries?)

Longtime contributor to the blog Erin "Charlie" Simpson is back with a guest post for the ages...

Instead of going line by line through MAJ Thiel’s SWJ paper (which I characterized on the Twitters as “horrible, terrible stats work”), I’d like to offer some general guidelines for policy-relevant, conflict research. As Ex will tell you, I am not an Iraq expert. But I know a little bit about COIN and another bit about quantitative research.

1) Big Claims require Big Methods. I’m not one to argue that sophisticated statistics can answer all of our research and policy questions. But if you want to wade in on one of the biggest (conflict) policy debates of the last 10 years, you best bring a lot of stats firepower. Correlations among yearly, national data won’t cut it. There are people who do this for a living: Ivy League professors, Army ORSAs, DIA analysts, DARPA geeks, think-tank types. And they do it with care and sophistication. Learn from them, understand the data and model choices they make, and realize the complexity and contingency of the problem at hand. We cannot adjudicate these complicated causal claims with descriptive statistics.

2) Avoid Sigacts. Sigacts suck. I’m sorry. But they do. They are a function of our presence. More troops (outside of more bases) leads to more sigacts? <sarcasm>You don’t say!</sarcasm> Sigacts are as much a measure of our presence as they are of violence.* (There are also a ton of non-violent sigacts reported. So make sure you knock out those key leader engagements and non-battle injuries before you run your analysis.)

*And as we know, COIN isn’t just about violence (if you’re a Kalyvas person, you know violence has a non-monotonic relationship with control such that low-violence doesn’t always mean good things). So, sigacts are a bad measure of violence and violence is an unreliable measure of stability or “progress” or whatever. But that’s a slightly different debate.

What I'm trying to say here is: Moneyball that shit and find the COIN version of on-base percentage or WHIP.

3) Correlation is not causation. We all know this. But did you also know that low correlation does not preclude findings of causation? Two variables may appear to have a low correlation – until you control for various background conditions. Sometimes this can be tested with jury-rigged chi-square analysis (stratifying one of the variables of interest into various segments -- for example, divvying up Iraqi provinces by #’s of battalions present in 2006 and seeing if there are statistically different levels of violence in 2007). But the only real way to determine which variable among many has a causal effect is with something like regression analysis – correlation won’t cut it.

4) Model specification matters. Ok, so now you want to run some regressions? Which kind? For most conflict data, you won’t want ordinary least squares (OLS). In the parlance of our time, you’ll need to consider the underlying “data generating process.” How do the data come to be observed, and which models’ statistical assumptions best match that process? In general conflict researchers should evaluate various time series, time series-cross sectional, and count models (ie, Poisson) for their work.

5) Level of analysis matters more. How do you plan to aggregate your data? In many instances conflict researchers will want to look at how violence changes across time and space. Global investigations of violence (think Correlates of War or Fearon-Laitin style research) will look at the country-year. That is, annual level national data. This data is usually pre-collected and easy to work with. But if you’re focusing on Iraq or Afghanistan, you need subnational data. And while these wars are long, 5-10 years doesn’t generate enough data points for a useful time series. The more dynamic the conflict, the more detailed you want the data. So you need to dig down to province-month or district-week. (In Afghanistan, sigacts are relatively stable at the district-week level. If you’ve got some data or computing horsepower, you can even carve up the whole country into 10kmX10km grid and go from there.) Unfortunately, that means your other variables need to be measured at the same level, which can be tricky. But them’s the rules.

6) Regression has limitations, too. If you’re doing some sort of “policy evaluation” chances are we didn’t randomly assign the policy “treatment.” What does that mean? That means we probably spent development money in the most violent areas. Or established joint-security stations in safe areas first. Or otherwise implemented a policy based on the very thing you’re trying to study. From a causal inference perspective, that’s a humdinger. One set of solutions is to “match” or pair districts based on their “propensity for treatment,” which can deal with some of the non-random assignment problems. (See Gary King’s paper on health policy evaluation in Mexico for a good example.) There is a lot of good work that needs to be done in the realm of conflict research. Let’s figure out how to do it well.

(Those interested on the academic side may want to get involved in the Minerva-grant funded Empirical Studies of Conflict project run by Jake Shapiro, Eli Berman, Joe Felter and Radha Iyengar. Otherwise, talk to me about cool kids at Caerus Associates.)

From Abu Muqawama: check out Mike Few in SWJ while you're at it. Also, there is a good conversation on Twitter between @drewconway, @charlie_simpson, @abumuqawama, @chrisalbon, @jay_ulfelder and others on this post.

This David Brooks op-ed on Moammar Ghadafi and Libya makes a really broad claim about regime survival based on a single case study. I like David Brooks in part because he takes social science seriously, so he should know better than this. If you look to the countries directly to Libya's left and right on the map, you see examples of recently deposed leaders who managed to survive for quite a long time, thank you very much, as the kind of bland autocrats that Brooks hints are at a disadvantage when compared to Gadhafi. In summary, Gadhafi is crazy, true, but he is still just one guy. You can't base broader conclusions about the durability of regimes on a single oil-rich country of just 6.5m people.

Issandr doesn't pull his punches:

Andrew Exum touches on an academic issue here worth mentioning: that the events in Egypt have been poorly predicted by North American academia, perhaps because political science departments largely focus on quantitative analysis. Andrew, as ever (and I blame living in Washington as well as his southern roots for this), is very polite about not bashing the "quants", as he calls them.

 

Personally, I would be more blunt. Quantitative analysis and the behaviouralist approach of most American PoliSci academics is a big steaming turd of horseshit when applied in the Middle East. Statistics are useful, yes, when you are in a country that has relevant statistics or where polling is allowed. But things like electoral statistics tell you very little about the political reality of dictatorships, because the data sets are inherently flawed, since they're either unavailable, fraudulent, or irrelevant.

This is not a new problem, right? Garbage in equals garbage out. If the data you are plugging into your analysis is unreliable, your conclusions are not going to pass muster -- not with the political scientists using "soak and poke" methods or, for that matter, any dude you happen to pass on the street. A buddy of mine commented this is less about the divide between quantitative methods and qualitative methods as it is an epistemogical debate. But any debate over methods is ultimately a debate over epistemology: how does the researcher "know" what he or she knows? If he or she is relying on laughably poor data harvested from a semi-closed police state, Issandr points out, he or she can't claim to know much at all. All of this has direct relevance to the study of conflict, of course. Conflict zones are really difficult places to gather reliable data. On the one hand, the U.S. military harvests all kinds of data from its wars. But on the other hand, studying the war in Afghanistan, I have come to trust the data less and less over time and the more I have asked questions about how the data was collected. The numbers look neat on a PowerPoint slide, sure, but when you start asking hard questions, they are less impressive.

(This all reminds me of that quote/warning about how all government statistics are ultimately generated by a civil servant somewhere writing down whatever the hell he pleases on a sheet of paper. Help me out with the exact quote, readers.)

I have been greatly entertained by the debate between Daniel Drezner and Arpoova Shah over the question of whether the situation in Egypt says anything about the strength of political science in the United States. I encourage you all to read what the two of them have written, but there is something going on here that neither Drezner nor Shah deal with. I was standing in line a few hours ago, waiting on a sandwich at Potbelly's, when I read this, from Greg Gause, in a volume of the International Journal of Middle Eastern Studies last year:

Over the past five years, from volume 37, number 1 (February 2005) to volume 41, number 3 (August 2009), IJMES published thirty-seven articles that deal with politics in the contemporary Middle East, broadly understood. This is my count, of course, and others might add or drop some articles. I define contemporary as post World War II and have a relatively expansive definition of politics. My count does not include short features, only full articles.

 

Eighteen of the authors of these articles are identified as having academic appointments in political science departments, fewer than 50 percent of the total (some of the articles are co-authored, so there are more than thirty-seven authors involved). The other authors are concentrated in the discipline of anthropology (with one sociologist and one historian) or have appointments in religious studies or Middle East studies departments. Of the eighteen political scientists who have published in IJMES during this period, only eight were employed in North American universities. The majority of the political scientists appearing in IJMES during this period have appointments in European or Israeli universities; one political scientist working at an Arab university appeared in the pages.

 

Although those North American political scientists who did publish in IJMES during this period did some very good work, and it was my pleasure to review many of their articles, these numbers lead me to the troubling conclusion that there is a growing gap between the professional requirements for disciplinary success in political science in North America and the standards and forms expected of the best Middle East studies work. Increasingly, particularly at the best research universities, advancement in political science requires work concentrated in formal and statistical methods. There are, of course, exceptions. Some political scientists working on the Middle East who use postpositivist methods have secured leading jobs at top research universities. There is a refreshing recent trend toward encouraging mixed-methods research in dissertations, with large-n statistical and/or rational-choice formal mathematical components supplemented by case studies based on field work and more classic discursive and qualitative approaches. However, professional advancement in the field is driven by publication in journals that are heavily weighted toward quantitative and formal methods. In the subfield of comparative politics, where most Middle East work is done in the discipline, there are also strong currents arguing that cross-regional work, not intense concentration on a single region, is preferred. In promotion and tenure decisions, publication in regional-studies journals, although not actively discouraged, is not credited as highly as publication in disciplinary journals. The sad fact is that, for ambitious political scientists looking to get the best North American jobs, publication in IJMES is not a great career move. ...

 

The professional situation of political scientists outside of North America is not as constrained. Good area-studies work that is informed by the epistemology of social science but relies on “old-fashioned” area-studies methods of qualitative analysis and considerable field work is more highly respected in the discipline in Europe and elsewhere. One can advance professionally at the best universities in Europe and the Middle East doing such work. Because of these different incentives, and different financial-support systems, graduate students at European universities who are interested in the Middle East tend to spend more time in the field and produce work that is more accessible to cross-disciplinary Middle East studies audiences. The significant representation of European-trained political scientists in the pages of IJMES over the last five years is testament to this different set of career structures and incentives.

I am not trying to demonize quantitative methods here. Although I tease "Quants" because I myself am an area studies geek, let's be honest: the more "tools" you can bring to bear on a question, the better. And I am not trying to say -- and neither is Gause -- that one cannot publish smart scholarly work on the Arabic-speaking world outside of IJMES (which is the flagship journal of Middle Eastern Studies). But I am trying to say that American political scientists are, by and large, rewarded for doing work that does not immediately lend itself to relevance in situations such as the one in which we currently find ourselves.

There are some excellent American political scientists working on the Arabic-speaking world. Greg Gause is one of them. So is Marc Lynch, whose writing during this most recent crisis has been excellent and necessary. So too is Josh Stacher, who published a great essay in Foreign Affairs over the weekend. So it is unfair, Drezner is correct to point out, to start bashing political science. I actually think American political scientists -- from Samer Shehata to Nathan Brown -- have been quite prominent in offering informed commentary during this crisis. But that's not a reason not to fret that political scientists trained in America might not be doing the kind of field work necessary for both top-flight area studies as well as providing policy-relevant insights onto events on the ground when crises arise. I spoke at THE Ohio State University last week, and one of the professors there similarly worried to me that students trained in the American academy would not be able to "keep up" with their European peers on regional expertise. That, to me, might be worth American political scientists thinking about. 

Work cited (emphasis mine):

Yesterday, I was interviewed by the great Jim Michaels of USA Today concerning news that a tribe in Helmand Province had more or less changed sides. Although the article quoted me faithfully, I feel the need to slightly correct and expand on what I said:

Andrew Exum, a military analyst at the Center for a New American Security, said there are key differences in Afghanistan, where tribal rivalries and drug trafficking complicate the enemy situation.

 

But Exum, who formerly served as an Army officer in Afghanistan, said the agreement reflects the military success that U.S. Marines and British forces have had over the past year in Helmand.

 

According to Exum, the progress on the battlefield has helped build security and convince locals that coalition forces will not suddenly depart. Those factors were critical in convincing Iraqis to join the Awakening revolt.

I do not think what I said does justice to what I was trying -- ineffectively -- to communicate. Now, first off, I know about as much about tribal dynamics in northern Helmand Province as I do Sanskrit. I have spent a grand total of one day of my life in Helmand Province. That having been said, anyone who has read Chapters Four and Five of The Logic of Violence in Civil War will be familiar with the idea that as an armed group's control of an area increases, so too does collaboration. That might be what is happening in Helmand, though I would not go so far as to say this is definitely what is happening or that this process is irreversable. Also, I am not saying that this is what happened in the Awakening, though there is certainly both annecdotal evidence that suggests this was the case and, via Kalyvas, a whole mess of historical data that suggests this kind of phenomenon is normal given the circumstances. 

"An economist is an expert who will know tomorrow why the things he predicted yesterday didn't happen today." -- Laurence J. Peter (Whose words, I think, might apply to the social sciences more broadly.)

Here's a fun project for the readership. This should keep you busy through the weekend. I was reading a book chapter by Stathis Kalyvas (.pdf) and came across his definition of civil war, which will be familiar to those of you who have read this book:

Civil war can be defined as armed combat taking place within the boundaries of a recognized sovereign entity between parties subject to a common authority at the outset of the hostilities.

This got me thinking about Afghanistan and whether or not we can define the conflict in Afghanistan as a civil war. Words like "authority" and "sovereign" seem to me to be in need of exploration (assuming we agree with the definition offered by Kalyvas). Even "outset" is tricky. That in turn got me thinking about Iraq as well. How would we describe the conflict there? Maybe we would say "conflict" is the wrong word and that "political violence" is more appropriate. I don't know myself, but I am interested in the thoughts of the readership.

Update: So I asked a serious social sciency question related to current wars for the readership to mull over the weekend, and I get ... a bunch of inane crap about the mosque they want to build in the old Burlington Coat Factory in Manhattan. Thanks, gang.

I start my week with diffusion on my mind: why do tactics, techniques, procedures and strategies migrate from conflict to conflict and from military organization to military organization? One of the reasons this subject is on my mind is the publication of my friend Michael Horowitz's new book by Princeton University Press. I just bought The Diffusion of Military Power: Causes and Consequences for International Politics for my Kindle. Don't let the fact that Mike is my friend and teaches at my alma mater fool you: he is widely considered one of the smartest young security studies scholars in the United States. (Check out this article of his on the Crusades in International Security.) He has also been writing about diffusion for some time, and I find his thesis persuasive. ("My theory, named adoption-capacity theory, argues for any given innovation, the financial resources and organizational changes required for adoption govern the system-level distribution of responses and influence the choices of individual states.")

I can't say the same for Laleh Khalili's article in the new International Journal of Middle East Studies, which I just read this morning. On the one hand, I really appreciate the time and attention Dr. Khalili has devoted to considering counterinsurgency from a left-of-center, Russell Square perspective, although a lot of what she has to say seems mired in a post-colonial narrative that makes it tough for her to consider counterinsurgency operations in another context. Most of us counterinsurgency scholars, granted, prefer to consider counterinsurgency theory and operations from a purely pragmatic perspective, examining operations and strategies without considering the colonial context in which many of these operations were carried out in history -- and there are rather obvious scholarly weaknesses to this approach.

On the other hand, I find Dr. Khalili's attempts to link counterinsurgency as practiced by the United States and its allies to "counterinsurgency" as practiced by the Israel Defense Forces in the Palestinian Territories only expose her lack of understanding of the U.S. military. I feel I have a pretty good understanding of the debates and movement that brought counterinsurgency to the fore in U.S. military doctrine, training and thought, and I can't recall the IDF having ever been used as a reference point in those debates. The lone exception to this would be when folks use the IDF's performance in 2006 in southern Lebanon as a warning for what can happen when military organizations allow their "conventional" skills to atrophy while engaged in long-term low-intensity combat operations that demand a different skill set, but that's pretty much it. If anything, the adoption of population-centric counterinsurgency by the U.S. military has caused U.S. military officers and analysts to cast new doubts on the efficacy of Israeli strategies and tactics in the Palestinian Territories. (And in southern Lebanon, as my buddy Dan Helmer points out.) I looked through Dr. Khalili's extensive endnotes and didn't see the U.S. military's counterinsurgency manual referenced once. Maybe that's because you can't look at the way the United States wages counterinsurgency warfare and the way Israel occupies the Palestinian Territories and determine shared paternity. The tactical and operational preferences of the two armies are just too different, and I suspect the political aims of the combatants -- the Israelis wish to stay; the Americans wish to train up local forces and leave -- determine some of that. I just moved offices here at CNAS, but as soon as the e-mail is back up and running, I plan to write to Dr. Khalili and see if she wants to expand on this for the readership, because it is, at the least, an interesting topic for discussion.