From TORs to proposal to evaluation

The humanitarian evaluation sector is of extreme importance in addressing gaps in humanitarian programs, ensuring accountability, and making sure that agencies with ample resources use them in the best possible way. This sector has the capacity to apply pressure onto these agencies in ways not generally known to the public. Civil society should have more exposure to these evaluations for them to be truly impactful. Otherwise, professed goals of learning and accountability might lean more towards lofty promises, rather than substantive objectives.

I wanted to briefly outline part of the process in the producing of evaluations. Dara is part of so-called frameworks composed of other similar evaluating organizations. This allows a limited number of actors to compete in proposing evaluation plans to big humanitarian agencies. The key here is that humanitarian agencies distribute TORs, or terms of reference, which outline the program they want evaluated and details about the kind of evaluation they want. Thus, a lot of the terms are set before proposals roll in from organizations like Dara. Proposals comprise things like detailed expertise, team leaders and team members that work alongside in a coherent manner, proposed desk support by the organization and a very detailed financial outline. Depending on the agencies involved, the geographic scope of the programs in question, or the temporal span of the evaluation, evaluations can cost between several tens of thousands of dollars to several hundred thousand. A lot of back and forth ensues in this process until the evaluating organization is finally chosen.

Whenever Dara is chosen to be the evaluating organization, team members are dispatched to the country where the program to be evaluated has taken place (though sometimes they meet in the headquarters of the organization beforehand). Our team in Madrid coordinates team members’ actions as they perform the necessary fieldwork and advises them though desk support that is not directly accessible on the field location. That work can take many months and leads to a set of deliverables, many of which are compiled into the final evaluation to be presented to the commissioning organization. I have worked on precisely these evaluations, since they are among the primary tasks of the Madrid office. My help has evidently constituted more basic elements of the process than those that occupy my colleagues, but those have nevertheless exposed me to the complexity behind the reports. I have also had the opportunity to review alternative deliverables unique to Dara’s standards, like narrative reports containing large amounts of photography to depict a particular situation more authentically and make it appealing to larger audience beyond the usual teams responsible for evaluations at humanitarian organizations.

Sample work

This is a version of a video I created on the RRI for a side meeting that took place during a larger conference. The video was aimed to be a background animation to help attendees visualize the ideas being presented. Creating the video and the visualizations it contains was the result of lots of discussion with RRI team members regarding the means and the goal of the index.

D.C. to Madrid

During the first half of my time at Dara, Madrid’s office got a visit from Dara’s executive director from the Washington, D.C. office. The purpose of his visit – he’s one of two men in management positions in an organization otherwise made up of and run by women – was to do an internal review of the way Dara does things, for reasons of quality and productivity assurance. I got to sit in on two full-day sessions in which all Dara employees discussed those questions of management and effectiveness.

The sessions were exemplary of two quite different approaches to a work environment. Dara’s director in the U.S. is an American-educated American, while the rest of Dara’s employees in Spain are from various Latin American and European countries, including Ecuador, Bolivia, Spain, and France. The environment at Dara has always been calm and the importance of employees wellbeing is institutionally and individually recognized. That is not to say our work environment is unproductive or that employees are not dedicated to the work we do. On the contrary, tight proposal and evaluation deadlines can leave my coworkers working weekends and I’ve been happy to join them for those café weekends on several occasions. (Of course, I’ve also had the pleasure of spending time with them outside work recreationally and hope to be able to do that again during a weekend trip to Madrid from Toulouse, where I’ll be studying abroad next semester, and which is a 15-euro Ryanair flight away from Madrid.)

These interactions showed me (or maybe confirmed my preconceptions) that people and their humanity are often far more prioritized in European working environments, while efficiency at the expense of, well, people is the priority in American working environments (the exclusionary European-US binary used only because of my limited professional experience). For NGOs tight on funding, working efficiently is obviously important. Dara used to be a 30+ employee organization with strong philanthropic support. Unfortunately, this is not a permanent and sustainable source of funding and some years ago it ended. Now, Dara is in a smaller office and with a smaller staff. It has to look for innovative ways to fund its operations and keep them sustainable. We have to take on an ever-larger amount of evaluations and present projects we are working on at conferences, academic, governmental, and nongovernmental gatherings. We don’t have much prominence in civil society, although our partner organization Dahlia, founded by the same Silvia Hidalgo that is in charge of Dara, specializes in communication in humanitarian aid – ensuring local populations are in conversation and in understanding with the agencies that work to provide support and aid to their communities.

I was lucky to be present to witness these internal restructuring and evaluation efforts. Prior to these two days, the Dara environment comprised a very communicative office where everyone works on everything and decisions are made collectively. Because of the small size of Dara’s workforce, this was not as disruptive as it might sound, and discussion was a good mechanism for ensuring confidence, quality, and meticulousness. Nevertheless, the involvement of everyone in everything did take away from the specialization of employees in the complex process of proposal and evaluation writing. The changes that were proposed during those two days encouraged my coworkers to divide work in a more defined manner. The efficiency our US director strives for would require less discussion between coworkers, more focus on one’s individual screen, and less interaction overall.  Although I’d wish to comment more on these two days and the conversations that ensued in the meeting rooms and behind the scenes, for the sake of confidentiality, I’ll  have stop at this.

Evaluating humanitarianism

I came to Dara because I wanted to experience the humanitarian sector and see what it really represents in practical terms. Collectively, the classes I took at Dickinson over the past few months forced me to think about global justice as an opposition to humanitarianism. Humanitarianism is nice – it’s nice branding, the packaging includes good intentions and alleges to correct the world’s ills; the word appeals and the institutions exist to legitimize it. But in many ways it comprises blanket solutions through feel-good institutions that are part of the very establishment that causes some of the world’s gravest violations of human rights and human dignity. Global justice, on the other hand, is less palatable. It’s more abstract, it lacks the same amount of legitimacy through institutions and it can be dismissed far too easily as nothing more than a regulative ideal. Global justice, theoretically, aims for the root of the problems humanitarianism tries to fix after their happening. Global justice comprises a belief in the need to address the structural factors that perpetuate those inequalities, injustices, violations, disasters that humanitarianism looks at as a field for its work. Global justice is about stopping occurrences that give humanitarian organizations a reason for their existence in the first place – it’s sustainable, it’s long-term and it’s just.

I do not want to speak on this assuming too much authority. I have had a little bit of exposure to the theory, giving me just the right amount of confidence and privilege to criticize and preach from an academic point of view. That’s all I can do, however. I don’t have an elaborate solution in mind and do not have an implementable agenda for global justice. The primary role of these reflections is to provide context for my skepticism of the sector I am part of during my two months at Dara.

What Dara – and many other similar organizations all around the world – do is evaluate the humanitarian programs of UN agencies and other humanitarian agencies, whether national or non-governmental. Through evaluations reports, we strive to promote accountability and impact in those programs. A laudable goal, but to what effect? Again, I do not have an answer, but reflection-level skepticism. The evaluation sector is not well known outside the humanitarian sector, so the role of civil society in upholding accountability does not seem relevant. Moreover, the evaluations we conduct, albeit with independent experts, are commissioned by the very humanitarian agencies we evaluate. No need to expand on that, but, of course, no need for ungrounded assumptions either. As I’ve expressed in previous posts as well, I think it is worth questioning what sort of impact lengthy formal reports have on the actions of humanitarian agencies and what power for change they carry. Large amounts of money flow throughout the evaluation sector and while its existence is necessary, more public awareness about the specifics it might uncover is vital to its usefulness.

It is important to note here that this skepticism is not a reflection of the work Dara does. Rather, it comprises more general thoughts on the entirety of the sector we occupy. Without assuming a spokesperson role for Dara, I can say that I believe in the integrity of the work that we do as one organization within the confines of the establishment we belong to.

My agenda is not to bring humanitarian institutions down to the ground. The world is a better place for their work. Even with the tiny scope of its work, the world is a better place for Dara’s work. The peculiar reality of the UN, as one example, is that it exists if we believe in it; if we pretend we believe in it, it might progressively acquire the integrity, recognition, and validity through which it could achieve its mission. Many facets of the idea of state sovereignty are an intangible construction. It is years over years of leaders attempting to legitimize their higher authority. Time, tradition, and enduring legitimacy are as important as the material forces that maintain state sovereignty nowadays. The prospects for some kind of enduring UN legitimacy are based upon similar factors. Conviction to this agenda, however, does not exempt the UN and similar organizations from criticism. I do not want to blindly criticize the work of those institutions, but too little criticism is harmful. It’s tricky work to simultaneously denigrate and champion the sole institutional players of global significance who rely on a belief in the abstract and the prospective.

A day in the life

by Alexander Bossakov

Every morning, I press the buzzer on Calle Meson de Paredes as I sprint down the stairs of my building. I swoosh past Plaza Agustín Lara, in a hurry, and walk up and down and up and down the windy hilly streets of Madrid. I start work at 10, so I am not supposed to ever be in a rush, but somehow, every single morning I find myself lingering on my breakfast table for a tad bit too long.

I live in Lavapiés, a humble and breezy historic neighborhood in Madrid, that is made what it is thanks to its dense immigrant population. Shops and restaurants are owned mostly by individuals from Asia, Africa, and South America. Streets are steep and plazas stay populated whenever the sun is not at its peak. By now, I am well in the habit of falling asleep to loud conversations and music seeping in through my open window until well after midnight. In my close vicinity are a beautifully low-key Senegalese restaurant called Baobab, the unquenchably spirited plazas of Agustín Lara and Nelson Mandela, and a placid mercado hidden under the ruins of what used to be a religious school burned down almost a century ago by a radical leftist, anti-religious group.

Madrid is still cool in the morning, when the sun has just begun infiltrating  street corners, so my 20 minute walk to work is a very pleasant start of the day. Dara is on the third floor of an elegant building a few steps away from El Prado, one of the world’s finest collections of European art. We share the same address with the Madrid office of the European Council on Foreign Relations, FRIDE, a European think tank on foreign relations, a few other NGOs and multiple private apartments. Every morning, I avoid the classic elevator that extends through the middle of the building to climb up the red-carpeted stairs till the third floor. Ring the bell, buenos días, and seconds later find myself on my desk.

Dara’s office is a bright common space where everybody faces each other as we work and work-related conversation frequently interrupts our tasks. Tall, wide windows with wooden blinds occupy the southern wall of the office and stay open until we’ve had enough of the heat a little past midday.

Throughout the day, I work on proposals, evaluation reports, occasionally get pulled in for an update on the Refugee Response Index and often listen up during conversations on the difficulties of getting a particular evaluation team to access a particular part of Syria.

My commute back home presents the difficult reality of 33°C Madrid. That’s when I usually get a bike from Madrid’s bike share system and enjoy the privilege of its electric assist motor as I breeze past the most shadowy streets I find on my way back home or to a café.

This is it. A brief snapshot into (part of) my life here.

Cold croissant-induced reflections

by Alexander Bossakov

My internship has been flying by. Two months in Madrid seemed like a lot of time on the day I arrived, suitcase in hand, trying to load directions to where I would be staying on my phone. I am currently on an early morning flight back to Madrid, after having spent a three-day weekend just south of Paris for a large family gathering, arriving just in time to get to the office for work at 10am. I just realized I have not much more than 2 weeks left [less at the time of posting] at Dara and got down to writing this blog post, only to be interrupted by a flight attendant waking up les mesdames et les messieurs with a charming basket of croissants. With all due respect, that is what I prioritized over this blog post, but now that I’ve finished my breakfast, here are some reflections from me from over the clouds.

My last internship was not the greatest of experiences. In fact, it was incredibly helpful in showing me what I did not want to do with my life. My internship at Dara has been far more positive on many levels, not excluding the great people I work with, the pleasant work environment I inhabit every day, the nature of the work that we do and, last but not least, Madrid itself. Several weeks in, I feel part of the team in a way that leaves me committed to every step of the process of our proposals and evaluations, feeling the pressure of deadlines like those around me, and working to achieve what needs to be achieved in the best possible way. I work with everybody around me, get pulled in for meetings, lament our failures and celebrate our achievements.

At the same time, I still have certain reservations. I question how removed I sometimes feel from the ultimate purpose of the evaluations. I question the evaluation sector within the larger sphere of humanitarian work and how it sometimes seems like a self-fulfilling cycle that keeps itself alive for the wrong reasons. We evaluate the programs of, primarily, UN agencies, and by doing that we become part of the UN structures, for better or for worse. What I need to find more about, however, is how much of an impact the 100-page reports have in promoting accountability, effectiveness and impact. Are these evaluations a check on a checklist for UN agencies or true accountability mechanisms that have an impact on the quality of the programs designed to help those most in need? This is not something I necessarily have the perspective to find out from here in Madrid and that is why all we can resort to is making sure our proposals, teams and evaluations are the best that they can be. This, however, leads to an obliteration of our awareness of the human impact, cost and consequence that is purportedly at the center of the work of the UN agencies we evaluate. I find myself focusing on format, presentation, wording, design, team structure; my direct goal is to have our proposal accepted by the UN and turned into a commissioned evaluation on long-term refugee camps in Algeria. My direct goal is not to improve the conditions of Sahrawis living in camps in Algeria. That is the UN’s works and it is another part of the process that I cannot afford to be part of.

This is not at all Dara-specific. In so many circumstances, one has to reconcile with the reality of being part of a complex process made up of multiple agencies, groups of people, sub procedures and vast timelines. We are an essential component of the humanitarian aid process but there is no way I can attach myself to its every single component. Nevertheless, I believe there is a lot of value in tightly grasping the ultimate aim of the work we do instead of blindly and intensely focusing on the technicalities of proposal- and evaluation-writing from the capital of Spain. I still struggle with the extent to which I lack this awareness as I discover my position at this step of the process.

I cannot skip mentioning that despite my own thought processes, which happen from my own very specific perspective, I deeply appreciate the work done by my coworkers. They work hard every day committed to produce high-quality evaluations of humanitarian programs. I help around with all that I can, but my knowledge and experience is shadowed by theirs and my thoughts and reservations are young and undeveloped.


by Alexander Bossakov

Over the past week, there has been a scarcity of opportunities to free my mind from the current administration’s actions and statements on immigration. Ceaseless news reports and public statements persistently highlighted the gravity of the situation, yet it took way too long – more than 2,000 children – for the zero tolerance policy to be amended to halt child and family separation. It would be underwhelming at this point to say that this is an issue that belongs on the moral plateau, rather than the political one. The pragmatics behind the rule of law arguments propagated by those who unconditionally support the administration’s actions should raise alarm bells in a civil society that purports to constructively look back at history and throw around “never again” castigations.

Working at an organization that engages in humanitarian work does not inevitably face me with human problems and moral qualms. Even though humans are at the receiving end of the work that I do, there is little mention and thought of that reality. Most days, we engage in questions of bureaucratic practicalities, quantitative and qualitative analysis, report writing and the like. Reflecting on what position I occupy every day in the office – not professionally, but internally, personally – becomes a necessity for me, if I want to continue being intentional about my values. In order to avoid growing numb and reduce tasks to mere administrative routine, I want and need to remind myself that the work that I do – the proposals I edit, the expert databases I analyze, the presentations I prepare – will ultimately, indirectly, potentially impact the lives of those fleeing or stuck in conflict, of the vulnerable, of the unwanted, of the subhuman.

Ultimately, indirectly, potentially. Arguably, this kind of lack of certainty and the inability to see the relationship between work and product or impact is present in every sector. When dealing with such human issues, however, I struggle with the appearance of any trace of numbness to the humanity that is so defining of the work. As I work on tasks related to the RRI, I find myself thinking too little about the individuals whose lives we aim to improve by working on the index. They should be at the center of the work that I do, and that is especially so considering what I have advocated for in the past, having worked on The Refugee Show and Tell Project. And yet, I find myself more than I’d like to thinking analytics, presentation, data, methodology, with no perception of humanity on any horizon.

What I am touching on can easily be laughed off as a messy bundle of naive and unworkable suggestions. It is not possible to humanize every single step of the process and one cannot possibly be completely in touch with the ultimate product of one’s work. When developing an analytical framework for a refugee response measurement tool, surely one cannot possibly mentally retain images of refugee families in movement at all times. Might be true, but I cannot advocate for such abstraction. Being intentional and forceful about reminding oneself and those around oneself about the human factor, t he human cost, the human impact of one’s work should be a given. Avoiding that responsibility removes the human recipients from ongoing procedures that are purportedly developed for them – a saying that has become trite in the humanitarian sector. So trite, that it can easily be dismissed as another unworkable, progress-hindering suggestion.

Many will insist that emotion should not occupy any position of relevance in intellectual or technical processes. One of the main objections to the gun violence movement is that it feeds off emotion and not pragmatic legislative considerations. Most astonishingly, that same argument is being made in the ever-so-pertinent debates on the current administration’s “zero tolerance” policy and the procedures of child separation it entails.

Bureaucracy should not entail the abstraction of humanity. Bureaucracy is about rational procedure, but rationality does not exclude humanity. Emotion does not contradict rational intellect. From the extreme to the mundane, I’d like to be an advocate for the omnipresence of humanity. For those with good intentions, like humanitarian organizations such as DARA, it would be at the very least irresponsible to reduce my work on a refugee response monitoring tool to desk administrative procedure. For those with ill intentions, action and thought that abstracts humanity as a way of approach, it would be at the very least dangerous.

Calle de Felipe IV, 9

by Alexander Bossakov

At DARA, office conversations are linguistically diverse in the best of ways. Mid-sentence, my colleagues will blend English, Spanish, and French – whether we’re gathered for Tuesday’s team meeting, someone’s reaching out over the phone to DARA staff members in another country, or one person is talking to another one across the room. This is just one aspect of the work environment on the fourth floor of Calle de Felipe IV, 9.

The DARA team does not exceed more than a handful of people at a time. Many more back up the work we do and a vast network of experts perform work on the field – in places ranging from Pakistan to Bolivia and from Haiti to the DRC –  that is at the heart of our humanitarian evaluations. Management and the board are close-knit, but extend across continents and partner organizations. All this makes for a distinct work culture fueled by generous levels of cooperation, simultaneous interdependence and self-reliance.

DARA occupies a very specific place in the humanitarian arena, that generally has little public exposure.  The vast majority of our work is directly commissioned by agencies like UNHCR, UNICEF, WFP, DFID, ECHO and others, but we remain a small organization that does not parallel the prominence of the UN or government agencies. We have gained admission to a framework of similar non profit humanitarian organizations that conduct evaluations of the on-the-ground programs of the aforementioned agencies. At this very moment in time, we are finishing up our evaluation report of UNICEF’s response to Hurricane Matthew in Haiti, while drafting proposals for several new evaluations.

I work with colleagues who are stationed in Madrid and others who either travel regularly or are stationed elsewhere – like in our office in Washington, D.C. During the past two weeks and a half, I have worked heavily on preparing various presentation material for DARA’s Refugee Response Index. This has required collaborating with numerous people in trying to understand their vision for the RRI. On occasion, these visions do not overlap entirely and concessions must be made or discussions must be had to determine the best way to present a certain element of the project to a specific audience. I position myself as somewhat of an external weigh-in during such discussions, as a civil society member, as opposed to someone from within the NGO or humanitarian sector.

On our agenda for the next month are a UNHCR-convened NGO forum in Geneva, during which the RRI will be showcased, as well as a tentative meeting with experts in Thessaloniki. These events, and the sub-programming they will consist of, require a variety of approaches and methods of articulation, in a lot of which I happen to play a central role. At this time, I have reviewed proposals, drafts, correspondence, and have sat on meetings related to the RRI so as to be able to understand what the most comprehensive, precise, and concise presentation of the RRI could look like.

Primeros días

by Alexander Bossakov

I sit down to write this first post as my first half-week in Madrid comes to a close. I am devoting this summer to some serious exploration of my interests, following their uncontrollable flourishing during the past academic year. To put things into perspective, without digressing too much: I am a student of international studies at Dickinson, but I’ve always felt like I have just one foot in the sphere of international relations. The other part of me wanders (not entirely cluelessly, I can assure you) on cross-disciplinary paths which are the reason for my coming to the U.S. to pursue a liberal-arts college experience. More precisely, I find myself not wholly satisfied with the intellectual approach most prevalent in the field of international relations, which explains my complementary academic background in the humanities and my other pursuits at this point in my college career.

Before delving into the specifics of my internship at DARA in Madrid, it would be helpful to put it into context, as it falls between two experiences which complement it very nicely. I began my summer with a one-week institute on a curation of issues within the topic of human rights, based at an international law firm in New York City and led by Penn Law faculty. I was part of an inspiring group of graduate and undergraduate students with an extraordinarily genuine commitment to the human rights doctrine as it is and as we aspire it to become. It was also an opportunity to meet with human rights activists and leaders from organizations, such as Human Rights First, or UN agencies, such as UN Women, UNHCR, and more. Admittedly, most were from the human rights establishment, but the presence of Penn faculty and other students counterbalanced that in helpful ways.

Following DARA, I will spend three weeks in China doing fieldwork with Dickinson’s own Community Studies Center, related to internal labor migration and its impact on family structures, cultural dynamics, agricultural practices and more. Along with my internship, these three experiences offer me a rich and multifaceted look at the issues I care about. I am eternally grateful to have been able to receive grants and scholarships for all three of them, without which I would most probably have had to stay at home in Sofia, Bulgaria.

Without further ado, my internship at DARA.

DARA is a non profit that, as per the subtitle of my blog, “supports principled humanitarian action through impact evaluation.” There are myriad ways, in which I could describe all the work that DARA engages in with varying degrees of specificity, especially as I get more used to the vernacular used in the sector of humanitarian evaluation. On the one hand, DARA conducts evaluations and policy studies centered around conflicts or natural disasters that necessitate or have necessitated humanitarian action. These are evaluations commissioned by humanitarian organizations, such as UN or EU agencies, whose purpose is to ensure effective and quality aid and action. WFP, UNHCR, OCHA are among the many names one often hears around the office or sees labeled on folders laid on the shelves that encompass the offices of DARA.

On the other hand, DARA also has its very own independent projects, such as the Humanitarian Response Index, the Disaster Risk Reduction Index, or the Refugee Response Index (RRI). The latter is the project I will be working on heavily throughout my two months. These comprise research and policy studies that independently analyze and measure policies related to conflict, human rights issues, or natural disasters. The results help state governments and relevant institutions understand a particular humanitarian landscape, identify best practices, and collaborate in improving their approach to the issues in question.

During my first few days, I dove into the proposal documents and reports that served as the foundation for the Refugee Response Index. The RRI analyzes states’ response to refugee influx in a multi-dimensional manner: To avoid creating a disaggregated set of results, an analytical framework has been developed and is being fine-tuned by a team of experts, academics, and researchers. The framework comprises six pillars according to which a country’s response will be analyzed and will result in a baseline of country performance in line with goals set forth by international agreements on refugees. Over the next few stages of the RRI, data will be collected through a network of local experts, regional coordinators, and refugees themselves. The data will undergo a specialized process of review, calibration, and validation, to ensure it can be compared adequately across countries. Ultimately, the RRI will serve as a systematized, accessible, and real-time database on all dimensions of national, regional, and global refugee response.