From TORs to proposal to evaluation

The humanitarian evaluation sector is of extreme importance in addressing gaps in humanitarian programs, ensuring accountability, and making sure that agencies with ample resources use them in the best possible way. This sector has the capacity to apply pressure onto these agencies in ways not generally known to the public. Civil society should have more exposure to these evaluations for them to be truly impactful. Otherwise, professed goals of learning and accountability might lean more towards lofty promises, rather than substantive objectives.

I wanted to briefly outline part of the process in the producing of evaluations. Dara is part of so-called frameworks composed of other similar evaluating organizations. This allows a limited number of actors to compete in proposing evaluation plans to big humanitarian agencies. The key here is that humanitarian agencies distribute TORs, or terms of reference, which outline the program they want evaluated and details about the kind of evaluation they want. Thus, a lot of the terms are set before proposals roll in from organizations like Dara. Proposals comprise things like detailed expertise, team leaders and team members that work alongside in a coherent manner, proposed desk support by the organization and a very detailed financial outline. Depending on the agencies involved, the geographic scope of the programs in question, or the temporal span of the evaluation, evaluations can cost between several tens of thousands of dollars to several hundred thousand. A lot of back and forth ensues in this process until the evaluating organization is finally chosen.

Whenever Dara is chosen to be the evaluating organization, team members are dispatched to the country where the program to be evaluated has taken place (though sometimes they meet in the headquarters of the organization beforehand). Our team in Madrid coordinates team members’ actions as they perform the necessary fieldwork and advises them though desk support that is not directly accessible on the field location. That work can take many months and leads to a set of deliverables, many of which are compiled into the final evaluation to be presented to the commissioning organization. I have worked on precisely these evaluations, since they are among the primary tasks of the Madrid office. My help has evidently constituted more basic elements of the process than those that occupy my colleagues, but those have nevertheless exposed me to the complexity behind the reports. I have also had the opportunity to review alternative deliverables unique to Dara’s standards, like narrative reports containing large amounts of photography to depict a particular situation more authentically and make it appealing to larger audience beyond the usual teams responsible for evaluations at humanitarian organizations.

Sample work

This is a version of a video I created on the RRI for a side meeting that took place during a larger conference. The video was aimed to be a background animation to help attendees visualize the ideas being presented. Creating the video and the visualizations it contains was the result of lots of discussion with RRI team members regarding the means and the goal of the index.

D.C. to Madrid

During the first half of my time at Dara, Madrid’s office got a visit from Dara’s executive director from the Washington, D.C. office. The purpose of his visit – he’s one of two men in management positions in an organization otherwise made up of and run by women – was to do an internal review of the way Dara does things, for reasons of quality and productivity assurance. I got to sit in on two full-day sessions in which all Dara employees discussed those questions of management and effectiveness.

The sessions were exemplary of two quite different approaches to a work environment. Dara’s director in the U.S. is an American-educated American, while the rest of Dara’s employees in Spain are from various Latin American and European countries, including Ecuador, Bolivia, Spain, and France. The environment at Dara has always been calm and the importance of employees wellbeing is institutionally and individually recognized. That is not to say our work environment is unproductive or that employees are not dedicated to the work we do. On the contrary, tight proposal and evaluation deadlines can leave my coworkers working weekends and I’ve been happy to join them for those café weekends on several occasions. (Of course, I’ve also had the pleasure of spending time with them outside work recreationally and hope to be able to do that again during a weekend trip to Madrid from Toulouse, where I’ll be studying abroad next semester, and which is a 15-euro Ryanair flight away from Madrid.)

These interactions showed me (or maybe confirmed my preconceptions) that people and their humanity are often far more prioritized in European working environments, while efficiency at the expense of, well, people is the priority in American working environments (the exclusionary European-US binary used only because of my limited professional experience). For NGOs tight on funding, working efficiently is obviously important. Dara used to be a 30+ employee organization with strong philanthropic support. Unfortunately, this is not a permanent and sustainable source of funding and some years ago it ended. Now, Dara is in a smaller office and with a smaller staff. It has to look for innovative ways to fund its operations and keep them sustainable. We have to take on an ever-larger amount of evaluations and present projects we are working on at conferences, academic, governmental, and nongovernmental gatherings. We don’t have much prominence in civil society, although our partner organization Dahlia, founded by the same Silvia Hidalgo that is in charge of Dara, specializes in communication in humanitarian aid – ensuring local populations are in conversation and in understanding with the agencies that work to provide support and aid to their communities.

I was lucky to be present to witness these internal restructuring and evaluation efforts. Prior to these two days, the Dara environment comprised a very communicative office where everyone works on everything and decisions are made collectively. Because of the small size of Dara’s workforce, this was not as disruptive as it might sound, and discussion was a good mechanism for ensuring confidence, quality, and meticulousness. Nevertheless, the involvement of everyone in everything did take away from the specialization of employees in the complex process of proposal and evaluation writing. The changes that were proposed during those two days encouraged my coworkers to divide work in a more defined manner. The efficiency our US director strives for would require less discussion between coworkers, more focus on one’s individual screen, and less interaction overall.  Although I’d wish to comment more on these two days and the conversations that ensued in the meeting rooms and behind the scenes, for the sake of confidentiality, I’ll  have stop at this.