Background

The Government of Tanzania is currently implementing a Results-Based Financing (RBF) scheme in several regions. The scheme is designed to improve health service use and equity, as well as the quality and efficiency of health care, particularly among primary health care facilities. The Ifakara Health Institute (IHI) in collaboration with the London School of Hygiene and Tropical Medicine (LSHTM) and Chr. Michelsen Institute (CMI) are conducting a rigorous impact and process evaluation of the RBF scale-up, focusing on Mwanza region.

The impact evaluation is a controlled before-after study involving a total of 150 health facilities and 3000 households in the intervention region (Mwanza) and the control region (Mara). The baseline for the impact evaluation was conducted in 2016. The process evaluation involves regular data gathering at the national level and at 9 dispensaries and 3 health centres in 3 intervention districts (Misungwi, Ilemela, and Sengerema) in the Mwanza region. To date, three rounds of district-level data collection have been conducted: April 2016, November 2016, and September 2017. The findings from the baseline and the process monitoring have been documented in a series of policy briefs and reports. 

As part of the process evaluation, our research team recently conducted phone interviews with all health workers (HWs) and community health workers (CHWs) who participated in the baseline survey in the Mwanza region. The main benefit of this phone-based learning platform is that is provides timely information from a representative sample of health care providers. Another three phone surveys will be implemented during 2018–19. 

This briefing note presents the methods and results from the first phone survey carried out in February 2018.

Topics covered 

The main topics covered in this survey are: 

• level of awareness and knowledge of the RBF programme and the incentivised indicators, both among HWs and CHWs.  
• implementation status (training received, RBF payments received, data reporting errors and penalties)
• nutrition-related services (monitoring of children in health facilities, perceptions among HWs about the level of stunting, nutrition messages by CHWs at home visits) 
• relationship between health facility and CHWs (reporting systems, supervision, training)
• performance (changes in activities/performances as a result of the programme) 

Data collection and sample

As part of the baseline survey for the impact evaluation, the research team collected the phone numbers of the HWs and CHWs in our sample, i.e. two HWs and one or two CHWs at each health facility.

The telephone survey aimed at interviewing all sampled HWs and CHWs in the Mwanza region, a total of 244 HWs and CHWs at 75 health facilities (8 health centres and 67 dispensaries). 

A total of 239 respondents were interviewed (i.e. 98% of the sample). 85% of them were part of the original sample. A total of 35 original, sampled respondents were replaced, most of them because they had moved to another facility. Replacement workers were identified by the in-charge of the facility. Out of the 239 respondents interviewed, 105 were CHWs and 134 HWs.

The phone survey was conducted by a professional survey company, Economic Development Initiatives (EDI), based in Bukoba-Kagera. Each call took approximately 30 minutes. 

Results: Health workers

Below we describe results from the survey among the health workers with a focus on 1) their knowledge of the programme and the incentivised indicators, 2) RBF payments, 3) services and perceptions related to nutrition, and 4) the relationship between the health facility and the CHWs.  

31% of the interviewed health workers were facility in-charges. The others were nurses (63%) and doctors (6%). 

Knowledge of the RBF programme and the incentivised indicators

Knowledge of RBF indicators was incomplete. Among the HWs, 66% had attended an RBF training. Health workers were able to mention on average only 5.7 out of the 14 RBF indicators on service coverage, that is 41% of the indicators. About one-third of the health workers cited four indicators or less, another one-third cited five or six indicators, and the rest cited up to 14 indicators. The trained HWs cited one more indicator than the non-trained on average (6 versus 5).  The most frequently cited indicators were institutional deliveries (mentioned by 66%), outpatient consultations (63%), and first antenatal visit before 12 weeks of gestation (63%). The least mentioned indicator was the number of TB suspects referred (12%) (Table 1).

Table 1. Knowledge about RBF

RBF payments

The majority of HWs (94%) reported that the facility RBF bonus had been received (beyond the start-up funds of 10 million) with an average of 2.78 payments having been made at the time of the survey (Table 2). The most common way to spend the facility bonus was on investments in infrastructure and renovations (69%), equipment (57%), and drugs and supplies (51%). Very few reported that the bonus had been used to incentivise CHWs or TBAs to bring more patients to the clinic. 

Half of the respondents reported that the final bonus received had been reduced due to penalties linked to data/reporting errors. Among these respondents, a majority thought that errors had arisen both at the district level (73%) and the facility level (66%). 

82% of HWs reported they had received a personal bonus linked to RBF, with an average of 2.87 payments received. The average amount of the last payment was around TSH 300,000 per health worker.

Almost all workers (95%) knew how the bonuses were distributed among staff and thought the distribution was fair (89%). The majority also recommended that the distribution of funds should be based both on worker attendance and responsibility level. The current practice thus seemed to be quite acceptable to the HWs in our sample. Note, however, that we did not interview anyone from the lower cadres.  

Overall, there seems to be a transparent environment related to both the receipt and the use of RBF payments.

Table 2. RBF payments

Nutrition related services and perceptions

Almost all HWs (99%) reported that their facility monitors the growth of children under 5 years. However, while 97% reported measuring weight, only 42% reported measuring the height of children (Table 3). The majority do not use growth charts.  

HWs severely underestimated the prevalence of stunting. Estimates from the baseline, as well as official estimates, indicate that more than 30% of children under the age of two are stunted. However, 79% of HWs believed that between 0 and 10% are stunted. These beliefs may be related to lack of monitoring.

Table 3: Nutrition-related services

96% of the respondents reported supervising CHWs on a weekly (69%) or monthly (26%) basis. 96% said that they ‘talk to Community Health Workers about what they should tell households to do to improve nutrition’. 

The HWs were unanimous (99%) in stating that the CHWs contribute to the facility’s performance on RBF indicators, and 67% thought that CHWs were escorting more patients than they used to escort before RBF.

Table 4: Relationship with CHWs

Results: Community health workers

The CHWs surveyed had an average of 14.6 years of experience. Each CHW reported supervising 308 households on average. 

Knowledge of the RBF programme and the incentivised indicators

Three out of four CHWs interviewed (72%) reported that they had received RBF training (Table 5). A high share knew 2 out of the 3 RBF indicators. When asked what they should do to get an RBF bonus, they answered: escorting women for delivery (87%), visiting households (86%), and reporting maternal and perinatal deaths (49%). They reported using performance-monitoring forms for RBF (in 87% of cases), and almost all knew where to report (99%) and reported on a monthly basis (93%).

Table 5: Knowledge of RBF

RBF payments

Almost all CHWs (94%) thought they were eligible for RBF payments, though only 41% had received RBF payments at the time of the survey (Table 6). Those who had received payments had received 1.8 payments on average. Some of these payments covered more than one quarter. The average amount received per CHW per round was TSH 110,000.

Table 6: RBF payments

CHW activities

In the previous month, CHWs reported escorting an average of five women to facilities for delivery (Table 7). However, only half of the CHWs (55%) felt that they were escorting more women compared to the previous year, and 34% felt they were escorting fewer than the year before.

CHWs reported carrying out an average of 16.7 home visits in the previous month. 76% of CHWs felt this was an increase compared to the year before, and only 16% felt it was a reduction.

The four most common issues discussed during home visits were antenatal care (53%), child nutrition (50%), nutrition for pregnant women (49%), and place of delivery (46%).

Nutrition-related services

When asked specifically about whether they provided messages about child nutrition during home visits, 97% of CHWs reported that they do so (Table 8). (Note that the question in the previous section was open-ended and did not mention specific activities.)

The most common messages were about children’s diet (77%), breastfeeding (75%) and the importance of colostrum (61%). Messages about the importance of specific nutrients were much less commonly mentioned. 

56% of the CHWs had received training on how to improve child nutrition during the past two years.

Relationship to the facility

All CHWs felt their relationship and communication with the health facility had improved in the previous 12-18 months (Table 9). 99% said they were supervised by health workers. Supervision was reported to include: submission of performance reports (90%), training (50%), joint home visits (35%) and meetings at the facility (20%). CHWs also reported ‘difficulties in communicating with the health facility’ as a major challenge to improving their performance (33%). This is an issue that needs to be followed up on in the next round of process evaluation to better understand what issues are being referred to. 

92% of CHWs reported challenges in increasing their performance, with transport issues (79%), communication with the health facility (33%), and geographical barriers (32%) being most frequently mentioned. There were also a number of “other” reasons.

Key messages

• The knowledge about RBF indicators among HWs seems to be weaker than desired. They are able to mention less than half of the incentivised service coverage indicators. A third of the interviewed health workers had not received training in RBF. These workers were able to mention slightly fewer indicators than those who had received training, but the difference was small, suggesting that those who have not been trained also have received information about RBF. The general low awareness of RBF indicators might negatively affect the HW’s ability to make targeted efforts to improve the indicators. Measures to improve the communication of the indicators to health workers and to support their retention of this information should be considered in order to improve awareness.   
• Among CHWs, knowledge of indicators was better (possibly because of the more limited number of indicators) even among those who had not been trained. This suggests that knowledge of the programme is being communicated through other channels (possibly through peers or supervision/contact with the facility).  However, the limited knowledge of the need for perinatal/maternal death audits among CHWs is worthy of note along with the fact that quite a few report that death registers are unavailable.
• RBF payments seem to be reaching most health facilities and health workers. The number of payouts seems to be somewhat behind schedule, though. There seems to be a transparent environment around the receipt and use of RBF payments. The way payments are distributed among health workers seems to be considered as fair by most of the interviewees. Note, however, that our sample does not include the lower cadres, which is where concerns about this issue have been raised in our previous process evaluation round.   
• Less than half of the CHWs have received any RBF payments to date. This resonates with our last round of process evaluation, where we found that CHWs were unable to report their performance due to a lack of forms and could not therefore be paid for RBF. However, availability of reporting forms seems to have improved substantially, which is likely to facilitate more regular future payouts to CHWs. 
• HWs strongly underestimate the prevalence of stunting in their communities. This is probably related to the lack of routine measurement of height in growth monitoring. There would be value in promoting the measurement of height as well as weight to increase HW capacity to detect stunting.
• At the same time, nutritional issues seem to be relatively high on the agenda of CHWs during home visits. Quite a number of the CHWs have recently received training on how to improve child nutrition.   
• CHWs were more likely to report an increase in the number of home visits in the previous year than an increase in the number of pregnant women they are escorting to the health facility for delivery. This difference may be related to reported obstacles with their funding of transport. Stronger incentives for escorting women may be one way of addressing this obstacle.  
• RBF payments to health facilities are being used mainly to improve infrastructure and drug supply. There was little evidence of other initiatives to attract patients to the health facilities. 
• Errors related to data reporting, both at the district and facility level, are reported in many places. It is not possible to tell whether efforts to reduce error rates have paid off yet.