Discussing Systems Thinking Ahead of the American Evaluation Association Conference
This year’s AEA Conference theme is visionary evaluation – systems thinking, equity and sustainability. Which begs the question what is systems thinking?
There are dozens of definitions, but for me it is the combination of three things:
- Understanding inter-relationships
- Engaging with multiple-perspectives
- Reflecting on boundary choices
…. And how does that contribute to visionary evaluation?
Because it has the ability to change how we do evaluation and indeed what we think evaluation is about. Let’s take those three ideas in turn. Evaluators use the idea of inter-relationships a lot, but often in a relatively limited way. Take your classic program logic. Evaluators generally worry and argue about what’s in the boxes and tend to ignore the arrows between them. In contrast the systems and complexity field tend to focus more what the arrows mean and rather less on what’s in the boxes. Evaluators talk a lot about multiple perspectives, but do we really deeply engage with the consequences of those perspectives on the situations we evaluate? If we did then we’d never consider an intervention having a single purpose or single framing. Yet this is something which our program logics and theories of change nearly always do. Finally if we assume that boundaries distinguish between what is important and what is unimportant, then boundary choices are essentially about what is valued. If we commonly reflected on and critiqued boundary choices then we’d never allow the values by which an intervention is judged to be determined solely by the program implementer or the evaluation client. These are all big issues for evaluation to engage with and has the potential to change what we do quite fundamentally.
What’s the first thing an evaluator should do when trying to think more systemically?
Treat the systems and complexity field with the respect it deserves. It’s a big field and like the evaluation field has diverse methods and methodologies, big unresolved disputes and a history. Do your homework and avoid grabbing hold of simple clichés.
Simple clichés like?
Clichés like systems approaches are about including everything. That’s clearly impossible and will lead to worse evaluation practice not better evaluation practice. Every endeavour is limited in some way – hence the focus on boundaries. So ‘holism’ for me is about being very smart, very informed and very considered about what to leave out, rather than opening the floodgates to more stuff. Another cliché is that systems approaches are only about big things. I frequently hear people talk about ‘systems change’ only in terms of large entities. It’s a notion that comes primarily from the management field rather than the systems field. Something as small as a cell can be considered a system. The final cliché is that while systems approaches help us deal with ambiguous and uncertain situations, the way we understand situations and why they behave the way they do is not magic, it’s not ‘stuff happens’. Systems and complexity approaches are very disciplined approaches to making sense of how things happen the way they do.
You spoke earlier about the systems and complexity field being large and full of many methods. If evaluators want to think systemically how do they choose which ones to use?
That’s an important question. If you rephrase it in terms of evaluative thinking you can also see how difficult it is to answer. Yes of course there are some great systems and complexity methods out there and I know ones that could be particularly helpful to evaluators. But any method takes time to learn and apply well. So in the first instance I’d prefer to see evaluators start where they are now and use the methods they already know in more systemic ways. Once they’ve got the hang of that, then they can gain the full benefit from learning specific systems methods – and they are likely to learn faster and make fewer mistakes.
So how do evaluators make their current methods more systemic?
Easy. Improve those methods’ approaches to understanding inter-relationships, engaging with multiple perspectives and reflecting on boundary choices.
This article, by Bob Williams, was cross-posted from the BetterEvaluation.org blog, which was originally published on September 15, 2013. Visit the website to read other perspectives and reactions to Bob's post.