Evaluating Complex Social Interventions in a Complex World

Abstract

The social world is complex and emergent. Inquiry, directed towards establishing universal empirical regularities (i.e. nomothetic inquiry), cannot establish causality in such a world. We can never assign a causal effect to any intervention without assessing the whole context of that intervention. However, we can develop generalizable knowledge if we adopt research approaches that recognize both the implications of assigning causal powers to context (the essence of the realist take on evaluation) and the significance of human agency in relation to ‘the social type of causal nexus’. There are literatures that can contribute to developing such knowledge. These include macro-political science’s concern with the importance of temporal ordering in relation to outcomes; Ragin’s set theoretic understanding of causal relations and his development of systematic comparison as a basis for explicating those relations through Qualitative Comparative Analysis (QCA); and the presentation of causal narratives as foundation for process tracing. Every complex social intervention has to be considered as a ‘case’. Systematic comparison across cases allows us to generalize within limits – but this still means we can transfer knowledge beyond the unique ideographically described instance. We can never establish universal/nomothetic accounts of causality in complex systems by using variable-based methods such as Randomized Controlled Trials (RCTs). However, through careful comparison and exploration of complex contingent causation, we can begin to get a handle on what works where (in what context), when (in what temporal context), and in what order.