Background 

The Government of Tanzania is currently implementing a Results-Based Financing (RBF) scheme in several regions. The scheme is designed to improve health service use and equity, as well as the quality and efficiency of health care, particularly among primary health care facilities. The Ifakara Health Institute (IHI), in collaboration with the London School of Hygiene and Tropical Medicine (LSHTM) and Chr. Michelsen Institute (CMI), is conducting a rigorous impact and process evaluation of the RBF scale-up, focusing on the Mwanza region.

The impact evaluation is a controlled before-after study involving 150 health facilities and 3000 households in Mwanza and the control region (Mara). The baseline for the impact evaluation was conducted in 2016. The process evaluation involves regular data gathering at the national level and at 9 dispensaries and 3 health centres in 3 intervention districts (Misungwi, Ilemela and Sengerema). To date, four rounds of district-level data collection have been conducted: April 2016, November 2016, September 2017, and September 2018. The findings from the baseline and the process monitoring have been documented in a series of policy briefs and reports. 

As part of the process evaluation, our research team is conducting phone-interviews with all health workers (HWs) and community health workers (CHWs) who participated in the baseline survey in Mwanza region. The first round of phone interviews was in February 2018 and second round in November 2018. The main benefit of this phone-based learning platform is that it provides timely information from a representative sample of health care providers. 

This briefing note presents the methods and results from the second phone survey carried out in November 2018.

Topics covered 

The main topics covered in this survey are: 

• training in and knowledge of the RBF programme and the incentivised indicators, both among HWs and CHWs. 
• implementation status (RBF payments received, spending and distribution of RBF payments, verification, perception around “carrot/stick” vs. “carrot/carrot” payments) 
• HW strategies and efforts for improved performance
• CHW performance (perceived activity volume, content of home visits, nutrition-related services 
• relationship between health facility and CHWs (reporting systems, supervision, feedback)
• experience with Direct Health Facility Financing (DHFF)

Data collection and sample

As part of the baseline survey for the impact evaluation, the research team collected the phone numbers of the HWs and CHWs in our sample, i.e., two HWs and one or two CHWs at each health facility.

The telephone survey aimed at interviewing all sampled HWs and CHWs in the Mwanza region, a total of 244 HWs and CHWs at 75 health facilities (8 health centres and 67 dispensaries). 

A total of 239 respondents (HWs and CHWs) were interviewed in round two, which is similar to the sample in round one. Among the 239, there were 134 HWs and 105 CHWs. 122 HWs and 99 CHWs from the baseline survey were interviewed, while 12 HWs and 6 CHWs were interviewed as replacements for those from the baseline that we were not able to reach. The replacements were identified by the in-charge of the facilities. The phone survey was conducted by a professional survey company, Economic Development Initiatives (EDI), based in Bukoba-Kagera. Each call took approximately 30 minutes.

Results: Health workers

The report based on health workers interviews focuses on 1) training and knowledge of the programme and the incentivised indicators, 2) efforts and strategies to improve RBF performance, 3) RBF payments, spending and verification, 4) the relationship between the health facility and the CHWs, and 5) experiences with Direct Health Facility Financing (DHFF). 

Training and knowledge about the RBF programme

A lower share of health workers report that they have attended RBF training in round 2 than in round 1 (48% vs. 66%). Nevertheless, their ability to recall RBF indicators seem to have improved slightly from round 1 to 2 (from 5.7 to 6 indicators). 

The pattern of which indicators health workers recall is quite consistent over time. Like in the previous round, frequently cited indicators were outpatient consultations (70%), first antenatal visit before 12 weeks of gestation (66%), vitamin A supplementation (58%) and institutional deliveries (57%) (Table 1). Note however that the share of health workers mentioning institutional deliveries has declined from 66% in Round 1. 

Interestingly, measles immunisation is now the third most commonly mentioned indicator (up from 43% in round 1 to 59% in round 2). Another indicator that is more frequently mentioned in round 2 is number of low-income households receiving outpatient care (up from 28% to 44%). 

A low share of the health workers recalls indicators related to HIV and TB. There is also a sharp reduction in the share of health workers who recall that there are incentives for preventive treatment of malaria for pregnant women (down from 38% to 28%). The share mentioning family planning is constant at 38% (Table 1).

Table 1: Training and knowledge about RBF

 

Efforts and strategies to improve RBF performance

After asking which indicators health workers could recall, we asked whether the facility had made any concrete efforts to improve any of these indicators, and if so, which ones. 

Table 2 reports the list of services in the same order as Table 1, i.e., from more to less frequently recalled services. We observe that the respondents reply to this question by repeating most of the services they mentioned in the previous questions. On average, they report that they had made concrete efforts to improve 4.7 indicators (while they recalled 6 indicators). 

It is hard to know how to interpret these results. One possibility is that the services they report here are actually those services they have given priority to, and that is also why they recall them. But it is also conceivable that they have given more or less the same priority to all services, and that they therefore in this question just repeat the services they recall. We should therefore interpret these results with caution. 

Particularly interesting observations are those cases where the reported priority is low, combined with a significantly higher share reporting that this is an incentivised service. This may suggest that the priority really is low. Post-natal care and preventive treatment of malaria in pregnant women may be examples. 24% report that post-natal care has received priority, while 41% know that this is an incentivised service. 18% report that malaria prevention has received priority, while 28% report that this is an incentivised service. 

Table 2: Any concrete efforts to improve RBF indicators

We then asked health workers an open question about which strategies they have used to increase the number of patients. The most frequently mentioned strategies were provision of community health education and sensitisation (77%) and more outreach services (51%) (Table 3). Other frequently mentioned strategies were better language and attitude towards patients (37%), direct encouragement of more patients to come (37%), better equipment (33%), and a nicer physical environment (30%). Perhaps surprisingly, only 25% mentioned better supply of drugs. Overall, these responses leave an impression of a wide variety of strategies being used to attract more patients. Thus, even though health workers were not very explicit on using the facility bonus to attract more patients (see above), they seem to adopt a variety of other strategies for this purpose.

Table 3: Strategies used to increase the number of patients

RBF payments, spending and verification

In terms of RBF payments, health workers report that their health facilities have an average of 4.4 rounds of RBF payments. In total, the Ministry of Finance has made seven rounds of disbursements, but not all facilities were enrolled from the beginning, and this may be why the average number of payments received is lower. There may also be recall bias involved. Not all health workers in our sample know when the facility was enrolled, but among the 79% who do know, almost 30% state that their facility was not enrolled until 2017. 

We asked health workers about their perception of how long they had to wait for the last RBF payment, counting from the end of the quarter when the payment was earned. The responses vary considerably: 30% say they waited 1-3 months, 23% say they waited 4-5 months, while 47% say they waited for 6-8 months or more. According to the government, the Ministry of Finance made the last disbursement in April 2018, 3-4 months after the end of the earning period. It is a possibility that some respondents may have been referred to the waiting period since the previous payment, which in November 2018 was 7 months according to the government (there was no RBF payment after April 2018). 

The most common areas where facility bonuses were spent include: investments in infrastructure and renovations (89%), purchasing drugs and supplies (71%) and purchasing equipment (48%) (Table 4). This pattern is like what we found in Round 1. 

Most health workers (93%) thought the spending of facility bonus money has been a good use of money to improve service delivery, while 7% replied that the money could have been spent even better. 

When asked who are involved in deciding how to spend the facility bonus, 93% mentioned the health facility governing committee, and 73% mentioned the facility in-charges (who are also part of the governing committee). In around half of the facilities, other facility staff also seem to be involved in the decision, while in some cases (10%) district level officials are involved. We also asked health workers whether they know what the facility wants to achieve by spending the facility bonus the way it does. Interestingly, only 34% said that the aim was to attract more patients to the health facility. Most health workers mentioned that the aim was to improve quality of services (91%), while 69% replied that the aim was to improve the working environment/job satisfaction. We cannot conclude from this that most health workers are paying limited attention to the aim of increasing service coverage, as high quality is a main channel through which higher service coverage is to be achieved. But this is an issue that it would be interesting to explore in more depth. 

The RBF scheme was changed in 2017 from a carrot-stick system with penalties/deductions in RBF payments when there are data or reporting errors, to a carrot-carrot system where penalties have been replaced by positive incentives for delivering high quality reports. Three rounds of RBF payments have been made with the carrot-carrot system, but health workers have apparently not understood this. Most health workers (88.1%) say that there are still penalties/deductions of RBF payment due to data or reporting errors. This is somewhat surprising, as they apparently acknowledge that the RBF payments (their individual bonuses) have increased substantially (see below). Efforts have been made to ensure that all facilities have a functioning computer for record keeping and data reporting. This seems to be in place in most facilities, but 20% of the facilities report that they still lack this equipment. (Table 4). 

To assess the quality of the verification process, we asked whether verifiers ever had gone to the community to check whether all reported patients are real patients. Most health workers (78%) replied that has happened, and among those who said that this had happened, 89% reported that it had happened recently (i.e., in 2018). It is however noteworthy that 18% said that tracing patients was not done. 

In terms of personal bonuses, there is an increase in the share of health workers who have received RBF staff bonuses from Round 1 (82%) to Round 2 (93%). Similarly, there is an increase in the number of payments received from 2.9 times to 4.2 times. The average amount paid has increased substantially from around TSH 300,000 to more than TSH 500,000 (Table 4). 

During our qualitative process evaluation, some facilities reported that they were sharing the RBF bonus among staff at a flat, equal rate, while the recommended practice is to distribute the bonus based on attendance and responsibility points. We were interested in how widespread the practice of equal bonus is. We therefore first asked what they thought about this idea. Opinions were mixed. While 58% thought it was a good or very good idea, 42% thought it was a bad or very bad idea. Only 11% report that they actually distribute the bonus at a flat, equal rate.

Table 4: RBF payments

Relationship with Community Health Workers (CWHs)

Almost all health workers (97%) acknowledged the fact that CHWs are escorting patients/clients to the health facility. Interestingly, 91% of health workers thought the number of escorted clients has increased compared to 12-18 months ago (Table 5).

Table 5: CHWs escorting patients to the health facility

Experience with Direct Health Facility Financing (DHFF)

Tanzania has introduced the Direct Health Facility Financing (DHFF) programme from 2017 onwards. This involves the transfer of health basket funds directly to health facility bank accounts. Among health workers interviewed in Round 2, the majority (83%) acknowledged that their facilities have received DHFF money (Table 6). Also, about 65% claimed that there are significant improvements in service delivery due to DHFF beyond RBF, while 35% though there were no significant improvements. These figures should of course be interpreted with caution, as health workers may have an interest in reporting positive effects of both DHFF and RBF. The responses should also be interpreted in light of the fact that there had been no RBF payments in the 6 months prior to this survey, while DHFF payments had continued to be received. 

Those who reported that DHFF had significant benefits beyond the effects of RBF were asked an open question about what these benefits were. The most commonly mentioned benefits were improved purchase of medicines and equipment (mentioned by 73%). Improved construction/renovation was mentioned by 13% of the respondents. 

With the stronger emphasis on decentralised financing, there is a need for technical support on financial matters. Interestingly, 89% of health workers acknowledged that their facility had received technical support from an accountant (Table 6).

Table 6: Direct Health Facility Financing (DHFF)

Results: Community health workers

The number of households that each CHW is supposed to visit declined from 308 in Round 1 to 210 in Round 2. This is a big reduction that is hard to explain unless there has been a significant increase in the number of CHW in this short period. 

Knowledge of the RBF program and the incentivised indicators

The share of CHWs who had received RBF training was 71%, which is about the same as in Round 1. Like in Round 1, a high share of CHWs knew 2 out of the 3 RBF indicators (Table 7). There is an increase in the share of CHW who know that they are incentivised for escorting women for delivery (up from 87% to 94%) and for visiting households (up from 86% to 92%). However, less than 50% of the CHW seem to know that they are incentivised for reporting maternal and perinatal deaths. In both rounds, 91% say that they report their RBF performance on a monthly basis.

Table 7: Knowledge about RBF among CHWs

RBF payments to CHWs

There is a significant increase in the share of CHWs who have received any RBF payments from round 1 (41%) to round 2 (75%) (Table 8). Those who had received payments, had received on average 1.8 and 2.2 payments in Rounds 1 and 2, respectively. While the average amount received in Round 1 was TSH 110,000, the average amount increased in Round 2 to TSH 265,400. A large share of the CHWs (63%) perceive that they had to wait for 6-8 months or more for their last RBF payments. The average (perceived) delay is larger than for the health workers (Table 8). 

Table 8: RBF payments to CHWs

CHW activities

CHWs report a higher number of women escorted for delivery the last month from Round 1 (five women escorted) to Round 2 (seven women escorted). (Table 9). A higher share of the CHWs also report that the number of women escorted has increased over the past year (up from 55% to 71%) 

The reported number of households visited also increased over time from 17 per month in Round 1 to 23 households in Round 2. Like in Round 1, three quarters of the CHWs felt that they do more frequent home visits now than one year ago. Hence, there is no indication in these data that the reported reduction in the catchment population has had any negative effect on the activity level.

The five most common issues discussed during home visits were antenatal care (63%) and issues related to nutrition (child nutrition (45%), breastfeeding (41%), nutrition for pregnant women (40%)) The pattern is largely the same as reported in Round 1, but we note that discussing place of delivery is less frequently mentioned (down from 46% to 34%) and information/education about newborn care is more frequently mentioned (up from 9% to 29%).

Table 9: CHW activities

Nutrition-related messages and training

Most CHWs reported that they give nutrition messages during home visits. The most common messages are about the importance of diversity of complementary diets (62%) and about exclusive breastfeeding (56%) (Table 10). A significantly higher share reported that they give messages about nutrition during pregnancy in Round 2 compared to Round 1 (53% vs. 36%). Due to some overlap between some of the categories in the table, results should be interpreted with caution. 

There is an increase in the proportion of CHWs who have received a refresher training in the past two years on how to improve child nutrition. Specifically, the share of trained CHWs has increased to 66% from 56% in previous round (Table 10). The majority of CHWs (84%) also reported to have any tools/ job aid on nutrition.

Table 10: Nutrition-related services and training

Relationship to the facility and challenges in improving RBF performance

CHWs are supposed to be closely linked with the nearby health facility. In both rounds, almost all the interviewed CHWs report to get supervision/feedback from the nearby health facility (Table 11). The supervision of CHWs was done either several times per month (45%) or once per month (46%). In the previous round, however, the majority received supervision several times per month (66%). When asked about the content of supervision, the CWHS mentioned submission of performance report (99%), joint home visits (46%), refresher training (32%), and meetings at the facility (23%) (Table 11). This pattern is similar to that found in Round 1. 

A high share of the CHWs experienced challenges in increasing their RBF performance (88%). The decline has moved from 92% to 88% (Table 11). Specifically, the most cited challenges include: lack of resources/funding for transport (70%), low RBF payment and delays in RBF payment (53%), and geographical access barrier (28%). The transport issues were also reported by many CHWs (79%) in the previous round. Then many CHWs mentioned ‘other’ challenges, which apparently was related mostly to low and delayed RBF payments.

Table 11: Relationship to health facility