Global Fund grants where the Principal Recipient (PR) is not a government entity are somewhat more likely to perform well, according to a recent statistical analysis published in The Lancet.

Authors Steve Radelet and Bilal Siddiqi of the Center for Global Development reviewed the performance ratings (A,? B1, B2 or C) that the Fund gave to 140 grants as part of the Fund's Phase? 2 renewal process. The authors then performed a detailed analysis of statistical correlations between these performance ratings and a variety of possible contributory factors.

They found that projects funded by Global Fund grants are more likely to achieve good performance ratings if:

1. The grant's PR is not a government entity
2. The grant is relatively small
3. The grant focuses on HIV/AIDS or TB rather than malaria
4. The original proposal received a strong ("Category 1") rating from the TRP
5. The LFA is a company other than KPMG
6. The country has a relatively high number of doctors per head, and/or relatively high measles immunisation rates
7. The country has relatively few health-sector donors
8. The country has relatively high disease-prevalence rates
9. The country is relatively poor
10. The country has relatively small government budget deficits
11. The country has, or once had, a socialist government

None of these factors was dominant, and there is no reason why a grant should not do well even if the grant, and the country in which it is based, has none of the above "success factors".

The authors do not suggest that the Fund use their findings to help it decide which proposals to approve. Instead, the authors suggest that the Fund use the findings to help assess the risks that approved grants face, and to guide allocation of management resources for oversight and risk management.

The study's findings can be summarised as follows. [Comments in square brackets are by GFO, not the study authors.]
 

SUCCESS FACTOR 1: PR IS NOT A GOVERNMENT ENTITY
Grants in which the PR is from civil society, private sector, or a UN agency were 17% more likely to receive an "A" performance rating than grants for which the PR is from government. The authors note that this is probably because government agencies often have problems with bureaucracy and capacity. [This in turn helps validate the board's recent decision that countries should be encouraged to assign non-government PRs to operate alongside government ones.]

SUCCESS FACTOR 2: GRANT IS RELATIVELY SMALL?
Performance ratings tended to rise as grant size (measured on a per head basis) decreased, possibly because smaller grants are less likely to have capacity constraints.

SUCCESS FACTOR 3: GRANT FOCUSES ON HIV/AIDS OR TB RATHER THAN MALARIA
HIV/AIDS and TB grants were 13% more likely to get an "A" performance rating than malaria grants. This might be partly because of the switch to artemisinin combination therapy (ACT) for drug-resistant malaria that started in 2004, which slowed implementation and added to costs. If so, performance on malaria grants might improve once ACT is fully introduced.

SUCCESS FACTOR 4: ORIGINAL PROPOSAL RECEIVED A STRONG RATING FROM THE TRP
Grants based on proposals that were graded "Category 1" by the TRP were 18% more likely to receive an "A" performance rating than grants that were graded "Category 2". The most obvious explanation for this is that the TRP is reasonably effective at identifying projects that have good chances of success. But it is also possible that Global Fund staff were partially influenced by the original TRP grades when they assigned grant performance ratings.

SUCCESS FACTOR 5: LFA IS A COMPANY OTHER THAN KPMG
25% of the grants for which the Local Fund Agent was a company other than KPMG received an "A" performance rating; but only 12% of those for which KPMG was the LFA received this rating. [Two conflicting forces are at work here. On the one hand, the LFA is supposed to advise the Fund about the strengths and weaknesses of the work done by the PR. When the LFA does this effectively and early, the chances should improve that problems will be fixed and the grant will do well. On the other hand, the LFA also provides the Fund with data at the time that grant performance ratings are assigned. If the LFA "hides the bad news" from the Fund, or doesn't even see it, this too might improve the chances of the grant being given a good performance rating, even if it doesn't deserve one. Either way, as the authors point out, there is a need for greater consistency between LFAs.]

SUCCESS FACTOR 6: COUNTRY HAS A RELATIVELY HIGH NUMBER OF DOCTORS PER HEAD, AND/OR RELATIVELY HIGH MEASLES IMMUNISATION RATES
Grants had significantly higher performance ratings in countries having more doctors per head and/or higher measles immunisation rates. This suggests, not surprisingly, that the strength of underlying health systems is a factor, and underscores the importance of not only aiming for short-term disease-specific targets, but also building strong health systems. It also suggests the need for greater Global Fund and CCM oversight in countries with weaker health systems and capacity.

SUCCESS FACTOR 7: COUNTRY HAS RELATIVELY FEW HEALTH-SECTOR DONORS
Grants tended to have higher performance ratings in countries were there were relatively few health-sector donors - that is, where Global Fund grants formed a larger proportion of health-related donor funding. This could be because management demands on recipients are much greater when there are multiple donors, and/or because recipients are less motivated to perform well when they have many funding alternatives. This might provide an incentive for the Fund to focus more of its efforts in countries where there are fewer donors, consistent with its role of filling funding gaps.

SUCCESS FACTOR 8: COUNTRY HAS RELATIVELY HIGH DISEASE-PREVALENCE RATES
Grant performance ratings were slightly higher in countries with higher prevalence rates for the disease (HIV, TB, malaria) on which the grant focused. This might be because there is a greater commitment to fight the diseases in high-prevalence countries; or because it is easier to achieve measurable progress in such countries; or because more realistic targets are set in such countries.

SUCCESS FACTOR 9: COUNTRY IS RELATIVELY POOR
Grant performance ratings tended to be somewhat higher in poorer countries, after adjusting for having a less well-developed health infrastructure. This result, which is the opposite of what one might expect, is hard to interpret; there are various possible explanations. But at least it drives home that there is no reason to expect that low-income countries will necessarily perform poorly.

SUCCESS FACTOR 10: COUNTRY HAS RELATIVELY SMALL GOVERNMENT BUDGET DEFICITS
Grant performance ratings tended to be somewhat higher in countries with smaller government budget deficits. One possible explanation is that a larger deficit could indicate generally weaker government economic and financial management.

SUCCESS FACTOR 11: COUNTRY HAS, OR ONCE HAD, A SOCIALIST GOVERNMENT
Grant performance ratings tended to be somewhat higher in countries that at some point in the last twenty years have been socialist. This might be because countries formerly in the Soviet bloc have stronger health systems and implementation capacity, even after accounting for the higher number of physicians and immunisation rates dealt with in factor 6 above.

NON-FACTORS
After taking account of the factors discussed above, the study found no significant links between grant performance ratings and other grant or country characteristics such as programme complexity (measured by the number of "service delivery areas"), quality of CCM operations (as measured by a survey), whether the country is a "fragile state", whether the country is suffering from internal conflict, certain measures of corruption and political stability, certain measures of civil liberties and political rights, certain measures of "red tape" (such as the number of days required to start a business), adult literacy rates, or girls' primary school completion rates. Although grants in certain continents or sub-continents, or in certain geographical settings such as being landlocked or in the tropics, had higher or lower average performance ratings than grants elsewhere, this was entirely explained, at a statistical level, by the non-geographic factors discussed in points 1 through 11 above.

Source: Global Fund grant programmes: an analysis of evaluation scores (The Lancet, Vol 369, pp 1807-13)