How to use Net-Map in Monitoring and Evaluation

Image result for images project evaluation international development cartoon

Are you planning or implementing complex projects, that aim at social change? Are you wondering how innovations spread through a given social system? Or maybe connecting people to opportunities or strengthening the networks of the people you work with is an explicit goal of your work?

In all of these cases, Net-Map may be a useful tool to establish a baseline and keep track of the social and network changes that happen, as you do your work. I have recently been asked by a number of colleagues about the use of Net-Map in M&E and my advice to them may be useful for you too.

  • Be intentional and get a baseline at the beginning. This might sound rather basic, but I have interacted with a lot of teams who were very excited to Net-Map the impact of their interventions but had nothing to compare it to. If you don’t know how the networks looked before you started, how much can you learn from a snapshot, just one point in time?
  • Put hard work into finding the right big question: “Who influences XY?” This is where most mediocre Net-Map studies start, with a question that is not quite on target, often too big (sometimes too small), two or three questions in one or using language that is easy to misunderstand. Pre-test and see if it works. Often at the beginning of a project you may need to start with a broad general landscape question, just to know where you are. Follow it up with a second session that asks a specific question closely linked to your project, so that it will be valid as a baseline.
  • Engage a broad enough range of participants, that they will show you your blind spots and give a balanced picture. Don’t just hang out with your friends and group-think. As you do more than one Net-Map over time, for M&E, will you be able to re-assemble the same group or at least a group of similar constellation?
  • Don’t just Net-Map. This is a tool that is great for telling you about the HOW and WHY, especially if you have a good interviewer who will dig deep into the qualitative discussion of the map and take good notes of it. However, for the most part Net-Map is not a tool to tell you a lot about the WHAT, to answer whether you have achieved your goals. Ensure to combine it with a solid methodology to evaluate the results achieved. Also, I rarely use Net-Map in a large, quantitative approach – but having it either inform a quantitative evaluation or be informed by it, can enrich both.
  • Use Net-Map as a tool for learning, not just for proving. As your project engages with local stakeholders, having a Net-Map landscape in mind can help you be more strategic and flexible and be less surprised by the way they react.

I know a lot of practitioners out there have used Net-Map in M&E context and would love to hear from your examples. Either in the comments or as a guest post.

The participant who drives you crazy is you!

Edgar Degas, Madame Jeantaud in the Mirror (1875)

I recently had the great learning opportunity to deal with a workshop participant who drove me crazy and rubbed me all the wrong way.

Why is this so great? Because, let’s face it: If I didn’t recognize parts of myself in her, I wouldn’t have reacted with strong emotions. I would just have thought (in my head, not in my whole body): “Oh, this participant does things that don’t work well. What can I do to help her?”

But when you see someone who does something you tend to do, and you see it from the outside, experience how your behavior must feel to others, that is a whole body experience of: “Why can’t you just STOP DOING THIS?!” And if I didn’t take a deep breath and looked in the mirror before reacting, I would be tempted to shout something like this. I think an important part of being a facilitator is actively feeling what is going on in your whole body (not just above the shoulders), observing it and then taking yourself beyond your immediate gut reaction. It may sound paradox but that’s what it is, you have to closely listen to your gut reactions, but then not just say whatever your gut tells you.

So what did I do when my participant just couldn’t stop doing it: Looking at everything from an evaluation or co-facilitation perspective, getting stuck in thinking and discussing about process instead of allowing herself to experience it? And keeping her group from the experience as well?

Well, the first thing I did was walk away and take a deep breath. Acknowledge that the participant who drives me crazy is me. And then I tried to think of her as if she was me: “What would be a kind thing that someone could say to me if I were stuck in the same way?” The “kind” part of it can be the most difficult one, because your gut may be far more ready to pick a fight, push, dragg and put pressure, than to open a door and get out of the way so the other person can walk through the door.

When I am the participant who is stuck in analyzing the process, it won’t help me if someone tells me to just stop thinking and get on with it. Because if I feel like something in the process is not going to work out the way I think it should, I can get pretty stuck and feel like this needs to be fixed. So what kind of door could I open for my participant that would be easier to walk through? Here is what I said: “Try to let your concern rest for half an hour, join your group in going through the process and let’s talk about your concern afterward.”

That is basically saying a number of things:

Your concern is valid.

You have permission to let go of it for a while.

You also have permission to pick it up again afterward.

Did it work for her? I’m not sure… but it definitely taught me a few things for my next group meetings – both as facilitator and participant.

I’d be interested to hear from you: Does this ever happen to you? How do participants show you your own face in the mirror? How do you deal with it?

We’re all just half-angels

Angle, Devil or what? (copyright by kcmckell on flickr)

Researching public health politics in Africa in the daytime and re-reading Tortilla Flats by Steinbeck at night. And both point from different directions toward my most difficult challenge in trying to understand the world: We are all just half-angels. Most of us want to be good people, or to at least think of ourselves as good people. And we have selfish needs and wants and evil impulses as well. The characters in Tortilla Flats are all charming drinkers who come up with the most twisted arguments for making their selfish behavior sound like they are doing the other person a favor: Everybody knows that money doesn’t make you happy and separates you from your poorer friends, so by not paying rent to their friend Danny they actually save him from this sad and lonely fate…

Now what do these guys have in common with health workers and advocates in Africa? Not much from the first look of it, because the outside observer can easily come to the conclusion that Danny’s friends are the bad people while people who dedicate their lives to health care in Africa must be angels. Steinbeck writes most of his novel from the perspective inside different people’s heads, so you can see how they negotiate their different impulses and how much thought they put into getting what they want AND feeling like good people at the same time. Obviously, it is written in a humorous and exaggerated way. But is it so far from what we all do every day?

When I look back at my first hand experience in and research about health systems in different African countries, I realize that people enter the health professions for a broad mix of reasons, ranging from “saving babies’ lives” to “income”, “power” and “status”. And while most professions carry mixed motivations, in a field like health they are especially obvious, because what you can achieve is so large. Imagine, you can save someone’s live! What a large and gloriously good thing to do. But also: How powerful it makes you, when everyone knows you are the one who can save lives – how tempting to use this power for your own benefit (e.g. by demanding excessive charges or favors). And where there are temptations (call them incentives, if you are an economist), people will give in to them. Not all will give in to the same extent, but very few will completely resist, especially if they know that their behavior will not be sanctioned. At the same time, they will try to keep the self-image of being an ultimately good person. And for many, the result will not be too far from what Danny’s friends do…

But why is this my biggest challenge in understanding the world? Because I love a clear and simple story. I want to be able to have clear feelings and unambiguous answers. My clients like them too, by the way. So I want to be able to say: This system or person is corrupt and not working. And this system or person is not corrupt and working very well. These are the good and these are the bad people, the angels and the devils. But if I delay putting things in boxes labeled “good” and “bad” and instead just allow them to tell me their story and observe what they do, I realize that we are all just half-angels. Yes, I have seen some people with a much larger leaning towards selfless or selfish behavior than others. But another typical character I have met a lot in my research is the powerful person who wears both wings and horns in XXL, the very charismatic, well connected guy (or lady) who achieves far more for “his people” than others in his position would, and, at the same time, lines his pockets with more bribes and favors than anyone else could extract from this position. How am I to think and write about him? What do I recommend? Do we want a smaller person in his position, who achieves less for his people and his own pockets? May we find a full angel, or let’s say a three-quarter one to replace this guy and tilt the scale a bit towards public benefit? Can we change the system, it’s incentives and opportunities in a way that reigns in the selfish behavior better? Or do I just decide, depending on whether I am a cynic or romantic, to close one eye and only see either the wings or the horns and praise or condemn wholeheartedly?

All you ever wanted to know about Agricultural Innovation Systems

Agricultural Innovation by connecting farmers to the world... (copyright by IICD on flickr, SEND Westafrica Program http://www.sendwestafrica.org/west/index.php)

It arrived on my desk yesterday and the paper version is heavy enough that you might use it as a weapon: 1.52 kg (or 657 pages) of looking at Agricultural Innovation Systems from all directions: Examples from the field (from Peru to India), methods for supporting, understanding and researching agricultural innovation from practical and academic perspectives. As the introduction states:

“Although the sourcebook discusses why investments in AIS are becoming more important, it gives its most attention to how specific approaches and practices can foster innovation in a range of contexts. Operationalizing an AIS approach requires a significant effort to collect and synthezise the diverse experiences with AISs.”

“For innovation to occur, interactions among these diverse stakeholders need to be open and to draw upon the most appropriate available knowledge. Aside from a strong capacity in R&D, the ability to innovate is often related to collective action, coordination, the exchange of knowledge among diverse actors, the incentives and resources available to form partnerships and develop businesses, and the conditions that make it possible for farmers or entrepreneurs to use the innovations.”

As you can see, a lot of agricultural innovation relies on the structure and content of multiplex, complex networks. This is why Net-Map is a natural fit for people who want to understand, monitor and support the development of viable agricultural innovation systems.

Agricultural Innovation Systems, an Investment Sourcebook, Part 1 and Part 2.

In case you are not intending to read it back to back… The use of Net-Map to understand Agricultural Innovation Systems is described on pages 593-597 in Part 2.

What is the best network structure?

It's easy: The one with the biggest trophy has to be the best - but how do you judge what's the best network structure? (Beauty contest winners, copyright by "What makes the Pie Shops Tick?" on Flickr)

That is THE question that my clients normally want me to answer. Tell us the best network structure and help us get there.

Today I exchanged emails with a colleague with whom I am involved in an evaluation project about knowledge flows and we found that in the same country and among very similar actors the information in one domain flows very much in a hierarchical, hub-and-spoke kind of fashion, the responsible Ministry sits in the middle and informs everyone else, mainly in one way communication. In the next domain the network is much more interconnected, while there are some more and some less connected actors, basically everyone has more than one source of information and there is much more exchange between actors on similar hierarchy levels.

Our intuitive initial reaction was: One structure has to be bad, the other one has to be good. And because of where we are comming from (our views of the world), we know that the hierarchical information distribution structure is bad and the interconnected web of information exchange is good. Now if we look beyond what we like or prefer for the sake of empowerment and just ask: How well suited are these structures to get the information where it should go, the picture is more mixed and I have to give an answer that you will hear from me whenever you ask me about what the best network structure is: That depends. Both structures have pros and cons.

The hierarchical structure where one central node controls all the information flow are great for distributing clear and undisputed information in an efficient and effective manner, everybody knows: I have to go to the Ministry to get the right information and the Ministry has total control over the content of the message. On the flip side however, this puts a large burden on the Ministry (or whoever the hub is), because if they don’t perform, no one can take their role and the system will collapse. And there are many ways in which they might fail: their capacity might be overwhelmed with the sheer number of requests, they might have outdated information, they might focus on another issue, etc. Also, innovative ideas, learning from field experience, experimentation and alternative solutions are not encouraged in a system that has pre-defined who owns and controls all relevant information.

A less centralized structure with stronger inter-connectivity and lateral flows is much less vulnerable to one actor’s lack of performance, allows for more cross-pollination and the integration of alternative approaches. On the other hand, a lot of the actors on the ground, who just want to quickly get the relevant information and then get the job done, are easily confused by multiple, contradictory messages and might not always be experts enough to judge which one is the most valid one for their work. Also, less centralized networks tend to take much more time to mobilize, there is no one actor who can take on the responsibility for training everyone. Also it is more difficult to monitor, evaluate and compare the effects of the different interventions that abound. So while there might be a lot of experimentation, that doesn’t automatically lead to learning and innovation – it might just be a lot of reinventions of the wheel. Often low centralization networks do not survive and thrive for a long time, as there is no central driving force (though, sometimes they do…).

So, as far as ideal network and recommendations go, I’d say, neither is ideal. They both have strengths and weaknesses. If you work in a highly centralized network you have the benefit of knowing who to interact with to get your message out. And your role as NGO could be to make sure they have cutting-edge information to start with, to encourage this central actor to allow for more two-way information flow and to expand the core of the network, inviting more actors to share their burden. However, this has to be done delicately, as the hub might fear loosing control and power. On the other hand, if working in a dispersed, low centralization network, you want to see if this is really better for the front line implementers in terms of enabeling them to get their job done. Especially if there is a lot of confusion around contradicting messages, your role as an NGO coming in might be to help the different actors coordinate and consolidate and develop more predictable ways of defining messages, delivering information and facilitating comparable monitoring and evaluation.

Net-Map Level 1 Certification Course (Washington, DC)

We offer Net-Map certification courses on 4 levels:

Level 1: Net-Map Facilitation

Level 2: Net-Map Qualitative and Visual Data Collection and Analysis

Level 3: Net-Map Quantitative Data Collection and Analysis

Level 4: Net-Map Mastery – Train the Trainer

Join us for a 2 day, Level 1 Net-Map class on the 4.-5. of August in Washington DC!

You will learn how to use this pen-and-paper method in meetings, individual interviews and to structure your own thinking process. It will improve your project planning, monitoring and evaluation, team work and strategic networking.

From years of Net-Mapping experience, I have distilled the most common prototypical influence network structures, which I will share with you. This will help you detect network problems, bottlenecks and opportunities while you are mapping the network so that you can immediately develop improved networking strategies. By mapping out your own case studies (challenges from your work experience), you will learn the method, develop a networking plan for a complex work related issue and improve your “network eyes”.

Because the most difficult questions normally come up once you are back to your own work, wanting to implement what you have learned, we have added a free 1 hour phone or skype consultation, redeemable within 6 months after the training, to the package.

No prior knowledge of social network analysis is needed. However, even SNA experts will learn a lot of new things in this training.

Sign up!

Is evaluation a cost or an investment?

Turn your evaluation into a catapult (picture by trebuchetstore.com)

The answer to this question basically depends on the attitudes of the people involved: you (as a project leader), your organization/leadership, the body requesting the evaluation and the external expert who helps you do it.

Your attitude: If your organization, project, unit evaluates just because somebody else (donors, board, supervisors) said so, you see evaluation as a necessary evil to get more funding in the future, you collect data to fill in required forms and you make sure to be seen in the best possible light, if the people who require the evaluation punish you for admiting mistakes and obsess about their forms being filled in properly… well, evaluation will not only be a pain and an annoying and scary exercise where people bend the truth as far as they can get away with, it will also be a cost. Or shall I go so far as to say: a waste of money? Well, you might get some benefit out of it, for example the future funding you were after, but you will loose out on the opportunity of learning and improving based on experience and analysis.

If, on the other hand, you can develop the common understanding that the struggles your project went through are learning opportunities, that everyone involved has something to offer to understand what happened in the past and how to do things even better in the future, if the people who request the evaluation require measureable outcomes, but also know that obstacles overcome make you stronger and that you sometimes need to change your direction in the middle of a project (because of changes in the world or learning processes within the project), then your evaluation will be an investment, it will be an amazing and empowering learning experience for everyone involved and catapult your work to the next level in the future.

If your organization sees evaluation as a cost, you might not want to call me just yet, because my tools and approaches won’t work for you. If, however, you want to turn your evaluation into an investment for your future and a transformative learning experience for everyone involved (while still collecting data about your performance), we should talk.