Notes

^[1] The 2012 UNODC report “Corruption in Afghanistan: Recent Patterns and Trends” is available at https://www.unodc.org/documents/frontpage/Corruption_in_Afghanistan_FINAL.pdf, visited 22.02.2016.
Overview of development assistance 2011–2014
Before reviewing the Norwegian development assistance provided from 2011–2014 we will summarise the main trends in the period 2001–2011. The data is derived from the 2012 Norad evaluation and from official Norwegian aid statistics.^[2]
Norway reported a total spending of NOK 5,363 million from 2001–2011, and then NOK 3,008 million from 2011–2014. The annual budget allocation was approximately NOK 750 million.
Figure 2 from the 2012 report documents a fairly equal distribution of funding for NGOs, the United Nations (UN), the ARTF and a miscellaneous category.

^[2] Available at https://www.norad.no/en/front/toolspublications/norwegian-aid-statistics/, visited 25.01.2016.
Assessment of the follow up to evaluation recommendations
A key document for this review is the Norad 2012 report Evaluation of Norwegian Development Cooperation with Afghanistan 2001–2011 (Report 3/2012). In practical terms, the evaluation only covered activities in the period 2001–2009 (p. xv). The aim of the evaluation was to “…assess the contributions of the Norwegian development assistance to promote socio-economic conditions and sustainable peace through improvements in the capacity of the Afghan state and civil society to provide essential public services” (page iii), applying the OECD/DAC evaluation guidelines. We will here present the main evaluation findings before discussing, first, how they was addressed and followed up by the MFA, the Embassy and Norad, and afterwards, how this was done by the various implementers of Norwegian-funded assistance. 
A key finding from the evaluation is that Norway’s policy and interventions “match closely the international agenda for Afghanistan and within that framework its development agenda is certainly relevant”. Moreover, that “the focus on governance, gender equality, education and community development has been consistent over the years, just as consistent as the choice of channels and partners” (p.133). The evaluation found alignment with Afghan priorities consistently high on the Norwegian agenda, although there is a concern that it primarily was the international community that defined the Afghan priorities, which then limited Afghan participation and ownership. The report points out that as only NGO funds remain earmarked for the Faryab province, with ARTF and EQUIP no longer being preferences for the province, the provincial government feared about reduced ownership and about needed capacity strengthening of the provincial administration. 
Another finding is that Norwegian policy towards Afghanistan and the choice of aid channels (trust funds, UN organisations, NGOs and civil society organisations) has been remarkably consistent over the years (which was found to reflect a political consensus across two parliaments), The report noticed that “apart from increased funding for ARTF there is remarkably little change over the past decade” (p.134).
The evaluation made the following observation on predictability and relevance, and the underlying analysis (ibid.):   
On the one hand Norway is a good example of predictability of resources and clear commitment to internationally agreed goals and therefore Norwegian assistance is definitely relevant. On the other hand, the use of underlying analysis remains weak and does not seem to inform policy choices, which may weaken the relevance of Norwegian assistance.
The evaluators are, however, of the opinion that “limited administrative capacity (at the Embassy) is one clear reason why policies are weak on the operational side” and that “follow-up on identified risks is not always satisfactory”. While they argue that the reasons for such an unsatisfactory follow-up is not clear to them, they identified that “pressure to disburse large amounts of funds is a contributor, given the limited staff and effects of the security situation on working conditions” (ibid.). 
Turning to effectiveness of the assistance their finding is that “in output terms, real achievements can be reported to which Norway had contributed” (ibid.).  Referring here to a range of development and governance achievements, as school enrolment figures of 7 million children by 2010 (of which 37% were girls), which – to place it in context – is up from less than 1 million in 2001 (and very few girls). They, however, go on to conclude that “…there is still limited evidence of concrete outcomes”, except for “improved access to services (such as midwifery) and enhanced pedagogical skills of teachers” (p. 135). There is, according to their assessment, not sufficient evidence to outweigh that “the overall quality of newly constructed schools is poor, literacy remains low and school dropout rates are high, governance remains poor and gender equality is still far from reality.” 
Following a discussion on the elusive prospect for a sustainable peace, the evaluation found that “donors, including Norway, made attempts to reduce corruption, but despite all efforts corruption remains endemic and negatively affects the attainment of real outcomes” (ibid.). They go on to identify the “weakness of monitoring and evaluation systems” to be “the main reason why there is so little good quality information about outcomes.” Arguing that by the start of the century all agencies were so preoccupies with getting activities up and running “that M&E was one of many important design considerations that were sacrificed in favour of speed. Gender was another” (ibid). This, according to the evaluation team, “meant that virtually no baseline were done and, as M&E gradually improved, there was nothing to measure progress against.” As security declined after 2005, “M&E has become increasingly problematic logistically and insecure for staff to visit project areas”.    
The evaluation concluded that “the overwhelming reasons for the limited results is poor governance and corruption.” It goes on to state that “donors have known about, tolerated, and in some cases exacerbated these for many years in spite of simultaneous efforts to bring improvements” (ibid.). This could be a result of “lack of agreement among donors about how to go about state building and governance agendas” (p.136).  Turning specifically to the role of Norway (ibid.), the report says:
Although the MFA has systems in place to prevent corruption, and requires its partners to have anti-corruption policies and strategies, these may go some way to minimising, though not eliminating, corruption at the lower level but they have no effect on the far more damaging grand corruption which takes place in some of the ministries. ARTF has not proved able to manage these and the lack of monitoring is a contributory factor. All donors have taken enormous risks which have increased with the increase in budgets.
The evaluation found the assessment of efficiency problematic due to weak M&E and the lack of data, and therefore concluded that “no reliable assessment can be made to compare the efficiency of various aid channels or aid partners” (p. 137). Although they state that “ARTF as a multi-donor mechanism appears to be a relatively efficient undertaking when viewed from the perspective of fund management and administration”. They moreover draw attention to the Norwegian Embassy in Kabul’s increased management responsibility since 2005, in 2011 managing two thirds of the Norwegian aid budget, and observed that “this has created a heavy management burden for an Embassy that is chronically understaffed”. This leads the evaluators to conclude that “the management of such a complex portfolio in a very complex environment has received insufficient attention”, finally stating that “for a portfolio of this size, the human resources at the Embassy are wholly inadequate.” 
We can see two trends in this area. One trend, as documented in Figure 6 below, is that the number of agreements (and partners/projects) was gradually reduced from 2007 onwards. The second trend, in Figure 7, is that increasing responsibility for handling the development portfolio was placed at the Embassy in Kabul until 2010.  

^[3]   For more details see the 2009 Norad report “Strengthening Nordic Development Cooperation in and with Afghanistan”, available at http://www.cmi.no/publications/file/3323-strengthening-nordic-development-cooperation-in.pdf , visited on 15.02.2016

^[4] The Danish Afghanistan Strategy 2015-17, available at http://afghanistan.um.dk/en/~/media/Afghanistan/FACT%20SHEET%20The%20Danish%20Afghanistan%20Strategy%202015-17.pdf , visited 21.01.2016.

^[5] Results strategy for Sweden’s international development cooperation with Afghanistan 2014 – 2019, available at  http://www.regeringen.se/contentassets/c3f71737c5f84cebb8550f61b214ab78/results-strategy-for-swedens-international-development-cooperation-with-afghanistan-20142019 visited 21.01.2016.

^[6] Sida: Approaches and Methods, available at http://www.sida.se/English/how-we-work/approaches-and-methods/ visited 21.01.2016.

^[7] The 2011–2012 Helmand plan is available at http://www.fmn.dk/temaer/afghanistan/baggrundforindsatsen/Documents/Helmandplan2011_FINAL_web.pdf , visited 13.02.2016.

^[8] Based on T. Wimpelmann and A. Strand. 2014. Working with Gender in Rural Afghanistan: Experiences from Norwegian-funded NGO projects. (Norad Evaluations no. 10/2014). Oslo: Norad.
Assessment of NGO activities
Analyzing the above information, we see a diversity of orientations and approaches among the NGOs, as well as across the type of humanitarian and development assistance they provide and its geographical coverage. Activities are primarily within the three priority areas of Norwegian engagement, particularly for rural development but also for the education sector, capacity building and the advocacy part of good governance. The NGOs hold a distinct capacity for responding to and mitigating the outcomes of natural disasters, responding to internal displacements and providing humanitarian assistance. Gender issues are addressed by most NGOs, with some working specifically on human rights.
As expected, we find a concentration of NGOs in the Faryab province, but also presence in the poorest and least developed provinces (such as Daikondi, Ghor, Uruzgan, Badakshan and Nooristan) and in those areas with large influx of returnees or concentration of IDPs (such as Nangarhar, Herat and Kabul). While there is some variation depending on their type of activity, all NGOs undertake conflict analysis and have developed risk mitigation plans. Most of them prioritize capacity building, though the extent of inclusion of government staff varies, as does their perspective on whether government staff represents an opportunity for collaboration or a major obstacle for the NGO operations. This differences are then reflected in how each NGO aims to build national ownership. The practice of placing NGO technical staff in government offices and inviting government staff to take part in evaluations is among the clearest examples of a long-term capacity building strategy; merely informing them about ongoing activities is just the short-term option.
It is evident from interviews and documents reviewed that most of the development oriented NGOs have performed baseline surveys since 2011, and some of them have devised their own Theory of Change. This puts them in a better position to not only report outcomes and numbers, but to measure impact (at least over time) in accordance with OECD/DAC guidelines:
The positive and negative changes produced by a development intervention, directly or indirectly, intended or unintended. This involves the main impacts and effects resulting from the activity on the local social, economic, environmental and other development indicators.
Some NGOs report on their projects’ impact or state the expected impact of the assistance. However, these exercises consist mostly of isolated examples instead of the systematic assessment of impact that are required to comply with the guidelines.  
Although with variations, these NGOs are better prepared to meet and adapt to contextual challenges, document results and address corruption challenges than they were in 2011. They have M&E strategies and procedures in place, as well as anti-corruption policies and regulations, and dedicated staff to do the follow up. This is reassuring, when compared to challenges identified in other channels. Still, testing over time is the only way to determine how well functioning are the systems the various NGOs have in place, and what are the results and impact they can subsequently document.
Gender issues were a high priority for Norway over the period studied. We will refer to the main conclusion of a 2014 Norad study of rural Afghanistan that examined the performance of most of the NGOs under study here. One of its findings stands out as particularly relevant for this report (p. v):
The study zoomed in on women’s income generation projects in order to examine the relevance, sustainability, results and promising practices of gender related activities. The review found interesting differences in how projects were conceived and implemented; to what extent they aimed and succeeded in expanding women’s control over the value chain, whether it was possible to mobilize women in small collectives with regular meetings and to what extent women were able to obtain a sustainable income. The findings suggest that organizations should consider whether they could be more strategic, focused and ambitious in their work with women’s economic empowerment.
We agree with the assessment that «gender projects» frequently appear as “tick the women box” projects, with limited planning and ambitions for their results and impact. This is a general criticism for projects targeting women in Afghanistan,^[14] but one could have expected more from Norwegian NGO partners. That being said, we would like to emphasize that there were several innovative projects identified that deserve credit (not all of them included in the 2014 review that covered the framework agreements with NGOs). These projects helped further women’s economic prospects (as solar engineers and midwifes), secured their legal rights (as in the case of returning female refugees), and ensured women involvement in peace processes (Midwifes for Peace). 
 

^[9] LOTFA (2015) Project Summary December 2015, available at http://www.af.undp.org/content/afghanistan/en/home/operations/projects/crisis_prevention_and_recovery/lotfa.html visited 22.02.2016.

^[10] For details, see http://www.tawanmandi.org.af visited on 25.01.2016.

^[11] British Council webpage, https://www.britishcouncil.org/partner/track-record/tawanmandi visited on 25.01.2016.

^[12] The letter is available here http://tawanmandi.org.af/wp-content/uploads/2014/09/Letter-from-Tawanmandi-Donors-28-9-2014.pdf visited on 25.01.2016.
M&E and Anti-Corruption Procedures
Two areas of particular concern raised by the previous review were: i) the ability to document results and impact of the Norwegian development assistance, and ii) the capacity to address and prevent corruption. Recommendations included strengthening procedures and mechanisms in both areas.
Monitoring & Evaluation
The evaluation of Norwegian development cooperation in Afghanistan during the period 2001–2011 noted that M&E had been a weak point largely because Norway, due to the security situation, had to rely largely on the reporting of others. The absence of baseline data also meant that impact was difficult to measure. While this was acknowledged by MFA/Norad, it was also noted that the security situation in Afghanistan continued to deteriorate through 2011–2014—the period of this review. Similar challenges remained, with limited possibilities for staff to carry out monitoring in the field (no visits were made to Faryab since 2013) and a reliance on implementing partners M&E frameworks and reports. With the reduction of staffing at the Embassy and the transfer of responsibility for NGO contracts to Oslo, people interviewed notices that this issue has become even more of a challenge during the period under review.
This is not a concern unique to Norway, either for the previous period or for the period currently under review. Other donors face similar challenges, although some (e.g. Sweden) have maintained or even increased staffing at their embassies and made efforts to get staff out into the field. This practice shows that it is actually possible to conduct field visits, although the security assessment carried out by the Embassy in Kabul does not allow for doing it. As a result, working to strengthen the M&E and reporting procedures of implementing partners was the main channel to address the findings of the previous review.
As noted earlier in this report, multilateral organizations (primarily ARTF and LOTFA, the main multi-donor trust funds) continued to receive the majority (55%) of the Norwegian development funds to Afghanistan. While this means that strengthening the M&E and reporting of these organizations is critical for tracking the impact of Norwegian funds, it is also the area where Norway has the least capacity to effect change on its own.
However, during the period reported, an external review of ARTF was carried out, finding that more attention needed to be placed on strengthening M&E and reporting, as well as on providing gender disaggregated data. This has allowed Norway, as part of the ARTF Strategy Group, to have input into the process. These efforts have resulted in improved reporting on results, as well as in the development of an ARTF results matrix (launched in 2015), which provides at least some baselines against which progress can be measured, and some gender disaggregated indicators.
During the period under review, emerging concerns about mismanagement in LOTFA (which came to a head in 2012) also presented an opportunity for Norway—as one of the main contributors to the trust fund—to exercise pressure for the strengthening of M&E and reporting mechanisms. While baselines remained weak, Norway, together with other donors were heard in terms of more emphasis being placed by UNDP on strengthening M&E procedures and providing more adequate reporting.
The strengthening of M&E frameworks and reporting has also been a major emphasis in the dialogue with NGO implementing partners, with continued discussions between MFA/Norad and the NGOs in relation to their framework agreements at the annual meetings. While it was mentioned that baseline data is desired for all new project agreements, it was also recognized that given the security situation baseline data may not be available in all cases, and that it would be possibly absent for emergency/humanitarian assistance.  As such, the absence of baseline data does not have to mean that a project is automatically disqualified.
This emphasis on results reporting appears to have had results during the period under review, with all NGO implementing partners working towards establishing more robust M&E frameworks (with innovative ways of monitoring being adopted, such as documenting impact with digital camera or community reporting), improved reporting procedures, and the establishment of baselines against which to measure results. While these efforts are still ongoing, a majority of NGO implementing partners now have these mechanisms in place, and have frequently established a baseline for their projects. In some cases, partners are also moving towards developing theories of change (see Annex V for NAC’s theory of change) to guide their activities. These efforts go some way towards addressing the concern that there is too much reliance on reporting from implementing partners due to the inability to verify independently the impacts claimed.
Overall, interviews with NGO implementing partners indicate that signalling from Norway during the period under review, together with an overall trend towards a greater emphasis on M&E and impact reporting, has provided a push for them to invest more in this area. Similarly, it seems that Norway has been able to seize the opportunities presented to push for change also within the major multi-donor trust funds. While there is still scope for improvement, it does seem that MFA/Norad and the Embassy in Kabul have made a considerable effort to improve M&E frameworks and reporting on results, taking into account a very challenging context.
Anti-corruption procedures
Corruption remains a major concern in Afghanistan, threatening long-term development and stability. The Afghan government and the international community have repeatedly affirmed that addressing corruption is a key priority. Through interviews and the review of the relevant documentation, we are satisfied that the risk/threat that corruption poses to Norwegian development cooperation is recognized by staff within MFA/Norad and at the Embassy in Kabul.
Overall, under a very difficult context, the MFA/Norad and the Embassy in Kabul have by and large taken the measures that could be taken to safeguard against the misuse of Norwegian funds. Despite a potentially higher risk of corruption than in other partner countries, the Embassy actually had less means at their disposal to adopt safeguards than would have been the case in a more regular development context (e.g. field visits and on sight monitoring).
The overall framework for mitigating corruption risks in Norwegian development cooperation, along with a zero-tolerance of corruption policy, also apply in Afghanistan. This framework provides clear guidelines for how to address corruption allegations, including channels for reporting to the relevant units at HQ level. However, with the difficulties for carrying out monitoring in the field and a decrease in staffing at the Embassy in Kabul, the review of documentation from implementing partners increasingly became the main means of identifying cases of corruption during the period under review. The value of a zero-tolerance to corruption policy in a high-corruption context was also a concern raised. The risk is that partners may not report suspected cases of corruption in order to avoid having their funding discontinued—a situation that NAC experienced during the period. This concern was confirmed by several interviewees.
While corruption, when uncovered, should never be tolerated, cutting funding may not always be the most appropriate response if it ends up jeopardizing the implementation of critical development activities. Instead, applying the principle of proportionality would be desirable and allow for, to the extent possible, continued development efforts while working to prevent further cases of corruption.
Proportionality, in this case, entails adopting an approach that is appropriate given the scale of corruption encountered. For example, if an implementing partner staff member is found to have embezzled funds, cutting funding completely would be a disproportionate response. Requesting the organization to sanction the staff member and putting in place better safeguards, while continuing to implement development activities, would protect development funds in the future without having a negative impact on implementation. That said, this would also require close follow-up with the implementing partner to ensure that actions are taken. It is possible that this again would require sufficient staffing at the Embassy to be able to do this in a timely manner.
At the same time, the number of projects/implementing partners was decreased during the period, in an effort to minimize the risk of corruption/increase oversight. This meant that Norwegian aid was channelled through fewer organizations, with stringent due diligence carried out prior to entering into a funding agreement. All contracts included an anti-corruption clause and implementing partners (including NGOs) are expected to have in place adequate safeguards against corruption, including complaints mechanisms.
The review of the NGOs that Norway is working with (see section on “Review of NGOs and their activities” above) shows that all of them have either put in place or are in the process of putting in place specific policies and mechanisms for mitigating corruption. The effectiveness of these policies and mechanisms is however difficult to determine without carrying out a more in-depth assessment of the actual systems put in place and of the capacity of the staff responsible for putting them into effect. We do however share the view of key informants that over the period reviewed it appears that adequate controls have been put in place by the NGOs to safeguard Norwegian/donor funds. We do note however that more could be done in terms of providing support for NGOs in order to strengthen internal systems and building capacity of their staff to further increase confidence in the safeguards that they have put in place.
With the major multi-donor trust funds (e.g. ARTF and LOTFA), Norway has relied on adequate safeguards having been put in place by the administrative agent (WB for the ARTF and UNDP for LOTFA). In cases where this assumption has not held (e.g. in the case of LOTFA), Norway acted together with other development partners to seek to strengthen control mechanisms.
There is greater uncertainty as to the controls in place to prevent corruption once funds enter Afghan government systems or reach out into the field. This of course is beyond the capacity of Norway to address, but the Embassy worked with other development partners during the period to keep the issue high on the agenda, and supported projects/interventions aimed at strengthening Afghan government systems. This however remained a difficult undertaking given the perception of a lack of commitment to genuinely tackle corruption on the part of the Afghan government.
Case studies
We will present three brief case studies in this section to reflect the Norwegian priority areas of education, rural development and good governance. In these case studies, we have put the emphasis on reviewing the support for anti-corruption efforts.
Education
The GoIRA decided early on to prioritise education, and to resume the responsibility for the sector, and not outsource the implementation to NGOs (as with the NSP). The new Afghan Constitution, approved in 2004, states that “education is the right of all citizens of Afghanistan”. NGOs and private companies were allowed to build schools and provide teacher training and vocational/specialized training. In the field of higher education, private universities and institutes were allowed to be established, in parallel to the strengthening of the public universities. Finance Minister Ghani demanded direct budget support for the Ministry of Education (MoE), but most donors were reluctant and preferred to channel the funds through the ARTF. Denmark was an exception in this matter, as we will discuss below.
This led to the establishment of the Education Quality Improvement Project (EQUIP) in 2004, under the ARTF, with an EQUIP Coordination Unit tasked to coordinate the ARTF/WB support within the MoE and to liaise between the MoE and the WB. Key donors have been Australia, Canada, Germany, the Netherlands, Norway, Spain, Sweden and USA.
The budget for EQUIP I at the start-up, in 2004, was US$79 million, targeting 26 provinces. EQUIP II started in 2008, with initial funding for US$188 million, and with a subsequent cost extension of US$250 million. Planning for phase III is underway.  Support was temporarily suspended in 2011 due to a critical Mid Term Evaluation, but continued later, according to the Norad 2012 evaluation (p. 5), after WB and MoE interventions that addressed most of the concerns.
Between 2003 and 2015, Denmark provided direct support to the MoE through their Education Support Program to Afghanistan (ESPA). It aimed to secure larger sustainability and ownership by the Afghan government, and included secondment of staff to the Ministry. The initial expectation from the Danish side was that other donors would joint them in the initiative. As that did not happen, they were left with a heavy administrative burden, “especially as the MoE was rather fragmented in their structure, had limited capacity, and faced challenges related to corruption during the implementation.”^[20] Denmark therefore decided to channel their education support through the ARTF from 2015.
At the same time, in 2011 Afghanistan joined the Global Partnership for Education (GPE) (supported and funded by Norway), and the MoE received a three-year grant of US$55,7 million, starting in 2013. The GPE aim was to address some of the inequalities in the education system through targeting provinces that were “insecure, underserved, difficult to access, and have the lowest education and economic factors”. The project, that has established its own GPE Coordination Unit in the MFA, places large emphasis on ownership and social mobilisation through community and parent involvement, and on local teacher recruitment. While the EQUIP programme has been evaluated, so far there is no independent assessment of the GPE assistance, beyond a noticeable concern among donors due to a slow start, a lack of coordination, and the risk that of it ending up as a separate programme rather than an integrated part of the Afghan education system.
The Norwegian education support between 2001 and 2012 was primarily provided through EQUIP, but also through support for the WB’s Vocational Education and Training, UNICEF for basic education and literacy, UNESCO for educational planning, and later the NAC, NRC and FOKUS for basic, vocational and health sector training. With the termination of the priority given to education through the ARTF, there is only NGO support registered for the 2011–2014 period. Norad aid statistics shows that the NRC receives most of the support, a total of NOK 93 million, for their basic education project.
However, a case study is too narrow for reviewing the NGO support for the period 2011-2014, since Norway has given substantial support to the education sector over time and has been engaged with several coordination efforts. We will therefore provide a more general assessment of the Norwegian support to education, and its outcomes and impacts, drawing primarily on the research for a MFA funded study (2015).
The results are impressive, from less than 1 million children in school in 2001 (with a low proportion of girls), the MoE estimated that in 2015 the number of students had reached 8,35 million students (39% of them girls) in primary, lower secondary, and upper secondary government schools--including Islamic schooling. The school-aged population is 10,33 million. However, 3,3 million children, the majority of which are girls, are still out of school. There is a common concern over the reliability of the data and numbers provided by the MoE. Numbers are not independently verified, and some students remain in the system for several years even after they have dropped out. However, the number of 8,35 million students used here is regarded by the statistics department of the MoE, the WB and EQUIP as fairly reliable, and is far below numbers quoted on several occasions by the previous Minister of Education. 
There is a major lack of equity in the Afghan education system, measured by gender, geographic location, and language. Afghanistan has the highest level of gender disparity in primary education in the world, with only 71 girls in primary school for every 100 boys.[2] Only 21% of girls complete primary school, with important cultural barriers (such as early marriages) and a lack of female teachers as two of the main obstacles (GoIRA 2015).
There is also a major difference in enrolment in primary education between rural and urban areas. The Education Inequality profile for Afghanistan[3] shows that 58% of boys and 52% of girls in urban areas attend school, while in rural areas only 41% of the boys and 28% of the girls. To further highlight the gender and geographical disparities, 80% of the richest boys in urban areas completed primary school in 2011, while the same was true only for 4% of the poorest girls living in rural areas.   
There are numerous bottlenecks identified—including insecurity, limited human resources, infrastructure, qualified teachers, teacher training and teaching materials—, while demand side issues include economic factors, cultural barriers, and governance and capacity.
The limited capacity of and within the MoE to handle and report on progress is itself a barrier, especially in provinces that have not had the same attention, support and allocation of advisors as the ministry in Kabul.
As expected, given the major overhaul the education sector has been through (including the development of curriculums, the printing and distribution of textbooks, teaching of teachers and the task of building schools), it has taken time to increase the quality of education and to measure the quality of education and learning. The GoIRA points out that “…by most standards, the education quality in Afghanistan is very low. Learning outcomes are generally poor. A few sample studies suggest that about less than half of the children are able to meet the minimum required learning outcome at their level of study.” Furthermore, the GoIRA found that for technical training “most of the education is theoretical and of very little practical value” (GoIRA 2015).
However, the lack of evidence on results is not as bleak as was painted by the 2012 Norad evaluation. The first learning assessment for the Class 6 level released in 2015[4] stated  “…while there are small numbers of Class 6 students operating at the higher level of proficiency in each of the domains of reading, writing and mathematical literacy, there are substantial proportion of the population who are not able to perform simple reading, writing and mathematical tasks”. A comparison with three peer countries in the region indicates that their Class 4 students are performing at a similar or a higher level than Class 6 students in Afghanistan.
The assessment suggests that “what is needed is a focus on the quality of teaching, both through policy and planning in the wider level, and through the professional practice of individual teachers in classrooms.”
Interviews for the education report, including a number of donors, MFA officials, EQUIP and ARTF staff, established that the MoE and donors agree to continue to channel funding for education though the ARTF, as “this has proved a trusted mechanism that ensures a fair degree of influence and prioritization from the Afghan government” (p. 13).   
The report noted that
While there have been major achievements in Afghanistan since 2001, there is still concern over the quality of education. There is a broad recognition that more funding alone, if available, would not ensure quality education for all. Rather, quality education depends on a number of factors that must be addressed in parallel, and should be included in the new education strategy for 2015–2020, in the planning for EQUIP III, and any extension of the GPE.
The recommendations were to: a) start with the teachers and build their skills, and allow for NGOs to play a larger role; then b) strengthen the MoE, and in particular the data collection/verification and coordination efforts, for a better planning and management of the education assistance; and c) build in addition a domestic resource and support base, including community and parent involvement, and request for support from the private sector.
Norway, albeit not the largest donor, has played a major role in developing the Afghan education sector. Its involvement has gone beyond funding, both donors and MoE refer to a very active engagement of staff at the Norwegian Embassy in the various coordination bodies and in EQUIP and ARTF fora, pushing for a priority for education and in particular girls’ education. The NGO activities have from 2011–2014 complemented the GoIRA activities putting emphasis on improved teacher training to increase quality of education, vocational training to ease access to the job market, and literacy, numeracy and life skills training for those many that have not gained any basic education skills.
A question that emerges is whether quantity—and especially girls in school—trumped quality, and whether Norway and other donors could have done more to ensure the quality of education. There are two observations to be made in this regards, both indicating that there was an awareness at the Embassy of the prevailing challenges in the education sector, and due attention paid to this concern even if earmarking for education through ARTF has ended. One point is the diversity in funding for the education sector, including support for teacher training, vocational training and school building through the NGOs. More important, and emphasized by the Ministry of Education in an interview, is the scale of the Norwegian strategic and policy engagement during the period 2010–2013.  This engagement included participation from the Embassy in Kabul in the following bodies and fora:

^[13] For details see http://www.netpublikationer.dk/um/9103/html/printerversion_chapter03.htm , visited on 13.02.2016.
Norwegian development assistance in Afghanistan 2011-14 and the results
In this section, we will provide a schematic presentation of the NGOs and trust funds in light of the main ToR questions, and discuss the extent to which they report on and can document short and long term results and impact.
Review of NGOs and their activities
Based on the ToR we selected a number of NGOs and requested their reports, monitoring and evaluation, and anti-corruption guidelines. We were able to interview members of the NGOs in Kabul (except the Aga Khan Foundation that did not respond to our requests), while the Norwegian based NGOs were also interviewed in Oslo. Basic information about them is included in Annex III, but the NGOs selected are the following:

^[14] Afghan Research and Evaluation Unit. 2013. Women’s Economic Empowerment in Afghanistan, 2002-2012: Information Mapping and Situation Analysis. Kabul, AREU.
Review of Trust Funds
Afghan Reconstruction Trust Fund
The ARTF was established in 2002 to provide a coordinated financing mechanism for the Government of Afghanistan's budget and priority national investment projects. The fund is administered by the WB and supported by 34 donors. The trust fund is the largest single source of on-budget financing in the country. ARTF grants support to the Government of Afghanistan’s operational budget (recurring costs) and to 21 programmes, including education, agriculture, rural development, health, social development, infrastructure and governance. ARTF is the main channel used by Norway to support the priorities set by the Afghan government. In total, Norway has contributed US$395.656.635 to ARTF.
The ARTF reports on their results in their annual ARTF Scorecards.^[15] It reported the following accumulated total results by 2015:

^[15] ARTF. 2015. “ARTF Scorecard 2015: Integrated Performance and Management Framework”, available at http://www.artf.af/images/uploads/ARTF_FINAL_2015_SCORECARD_REPORT.pdf visited 22.02.2016

^[16] Government of Afghanistan. 2015. Afghanistan National Education For All. 2015 Review Report. Kabul, MoE.

^[17] Available at http://www.epdc.org/education-data-research/afghanistan-education-inequality-profile , visited on 03.06.15.

^[18] T. Lumley, J. Mendelovits, R. Turner, R. Stanyon and M. Walker. 2015. Class 6 Proficiency in Afghanistan 2013. Outcomes of Learning Assessment of Mathematical, Reading and Writing Literacy. Victoria, Australian Council for Educational Research.

^[19] Norwegian Embassy. 2013. “Norsk utdanningsbistand til Afghanistan”, memo, 12.12.2013.

^[20] For more details see Strand. A. (2015) Financing Education in Afghanistan: Opportunities for Action. Country Case Study for the Oslo Summit on Education in Development, available at http://www.osloeducationsummit.no/pop.cfm?FuseAction=Doc&pAction=View&pDocumentId=63328, visited on 25.01.2016

^[25] The CDC’s governance role was later disputed by the Independent Directorate of Local Governance (IDLG) and remain an issue for discussion, ref NPP debates.

^[26] For a presentation of NORPLANS activities, see http://www.norplan.af visited 15.01.2016

^[27] Available at http://www.nspafghanistan.org/Default.aspx?sel=109, visited on 26.01.2016.

^[28] There is a larger discussion about the advantages of CDCs compared to more traditional village structures, and about the   extent to which elites influence their decisions and priorities. AKF had an interesting experiences on the way in which regular re-elections of CDCs ensure a fairer representation (private communication, Kabul).

^[29] For details on the project and results, see FAO. 2016. “Afghanistan and FAO Partnering for food security through gend
Lessons learned and recommendations
The ToR ask for the review to provide recommendations for further development cooperation in Afghanistan, while a clarification from the Secretariat modified the request to develop a more general reflection about future learning, including new development engagements. We will therefore start with some of the suggestions made by the interviewees/Afghan stakeholders, and then reflect on the more general lessons learned from our findings.
The development partners request a continuation of predictable and flexible funding for the coming years, which they see as a prerequisite to provide quality humanitarian and development assistance in an increasingly challenging work environment. The NGOs and the ARTF argue for a continuation for the thematic areas they cover, but without suggesting cuts in other parts of the Norwegian engagement. Senior Norwegian bureaucrats have a similar position, but they also recommend more attention to M&E of the Norwegian funded assistance, and several of them emphasize the need to address grand corruption challenges (and the individuals influencing them) in order to ensure that the Norwegian assistance meets the required needs and the jointly agreed development goals.
One important observation is that the situation in Afghanistan changed substantially in 2011. With the US announcement of military withdrawal from 2014, a more definite timeline was set that led to the expectation among Afghans of a reduction of all types of international assistance. The new context established a new urgency and planning horizon for all actors involved, including those that aimed to benefit from the corruption and embezzlement opportunities the assistance provided. The Kabul Bank case, and the involvement of relatives of senior government officials in the fraud, is one example.   
Challenges to security, economic development and the establishment of a functional Afghan government increased in the 2011–2014 period and placed an even stronger urgency on working towards Norway’s strategic aims. These were to 1) strengthen Afghan institutions (to be in a position to handle international assistance as well as to increase and manage their own revenue); 2) contribute to a political settlement (to ensure a more peaceful future); and 3) contribute to sustainable and just development, humanitarian efforts, and promotion of governance, human rights and gender equality. It can be argued that these goals have been pursued consistently since 2001, including in the 2011-2014 period. The difference is that in this period these goals have been pursued through fewer development partners and projects, and with gradually less process involvement from the Kabul Embassy due to the reduction of Norwegian staff.  
This is where we identify the main challenge for development assistant, and where the Afghan case can offer insights for similar and future peace/state building efforts. The Norwegian support to Afghanistan has had multiple objectives since 2001. It included building a state structure and the development and strengthening of administrative capacities; building a judiciary, in competition with a traditional justice system; establishing a western democratic system and running regular elections; getting a free market economy in place; and running a military operation (while building a new army and police force).  Also relevant to this review, in collaboration with multiple donors and stakeholders, Norway sought to contribute to the achievement of major development tasks (and to building Afghan capacity to take them on over time), while ensuring the rights and opportunities of the most vulnerable, not least of Afghan girls and women. The Afghan government was held responsible—expected to be in the “driver’s seat”—for leading these efforts.
At the same time, it was acknowledged that most Afghan ministries and the newly elected parliament lacked the required management capacity to fulfil this role. The donor response was to establish trust funds to manage development activities, and to make use of NGOs and private companies to implement development programmes and projects, though in collaboration with and under control of Afghan ministries and donor coordination mechanism. Such a fragile framework, however, requires a continuity of process knowledge, and of awareness about how and why commitments were made. It has been challenging to maintain continuity in this area as Embassy officials (and UN, WB and International NGO staff) rarely stay on for more than 2 years in Afghanistan, and as key ministry staff is often changed when a new minister takes up.
The larger trust funds, as we have seen, had in place internal management procedures to safeguard donor funds. This was not necessarily the case when these funds were transferred to the implementing ministry for implementation and/or salary payment. Even when severe misuse was documented, as in the case of LOTFA, the response options for donor were limited, as the consequence of cutting funding could threaten their overall engagement in Afghanistan. In the LOTFA case, donors were able to put pressure on UNDP, but had very little leverage to effect changes in the MoI. The case of Tawanmandi was different, as support to civil society organisations was not seen as equally important to the achievement of the overall objectives for the Afghanistan engagement (particularly in  the security side), and was therefore more easily terminated.
However, fund disbursement is only one aspect of programme management. An equally important aspect is the Norwegian involvement in setting and ensuring strategic objectives in dialogue with the GoIRA and other donors. Also crucial is ensuring compliance with plans and priorities; coherence with Norwegian (or Nordic) priorities (as in ARTF, LOTFA and Tawanmandi); and follow up on implementation, M&E processes and anti-corruption safeguards.
Actually, there are stricter controls applied to NGO support than to support for the trust funds, both in terms of the selection of implementing partners, and of assessing their overall conflict analysis, risk mitigation, M&E and anti-corruption systems and routines before they are accepted as partners. Their project proposals are assessed as for development objectives and budget alignment. Budgets are cut or withheld if reports or accounts are not delivered, or if there are accusations of corruption. Such accusations result in a close dialogue with the NGO to ensure that they address the concerns identified, or an external investigation either confirms or acquits them of the allegations.  
We are drawing up this picture to identify a weakness of the aid management system in a fragile/weak state such as Afghanistan, where there are major concerns over weak management capacity and corruption on the government side. Successful implementation depends, in our view, equally on 1) donor administrative systems, approval and control of the aid funding – which is in place, but now located in Norway and administratively divided between the MFA and Norad. And 2) donors’ ability to engage in a “development dialogue” with the government of Afghanistan and a range of other stakeholders throughout the programme planning and implementation period. This dialogue is critical, as outcomes will only be achieved if there is willingness and ability on the part of the national government to ensure that shared goals are established and met through the selected and implemented programmes and projects (and some of them through other partners, as in the case of the NSP).
The first part of the aid management system is well in place for Afghanistan, and made easier with a reduced number of partners and projects. There is also a larger continuity and institutional memory as there is a more permanency in staff in the MFA and Norad than was deemed possible at the Embassy in Kabul, given the security situation. The NGOs are very satisfied with the handling of their contracts, recognizing that Norway has been able to combine a long term funding commitment with flexibility when required.
As for the second—crucial—part, the aid management system’s capacity for dialogue with the Afghan government has gradually been reduced over the last years with the reduction in Embassy staffing. In Figure 8, we have illustrated the different funding and potential dialogue channels in Afghanistan, taking into account that ultimately it is the GoIRA that is responsible for, or implements, the majority of the development programmes Norway funds.