Background:

At the 1997 FAM Annual Meeting at Coolfont Resort in West Virginia, a monitoring and evaluation (M&E) working group was formed to help address the priority M&E issues that the FAM membership will face during the next five years and beyond. While the group recognizes that monitoring and evaluation are two basically separate processes, the needs of the larger FAM group necessitated that both issues be tackled simultaneously. Monitoring is understood in this context to be a management tool, while evaluation is defined as a measurement tool. It should also be noted that problem assessment and project design were also discussed as the M&E group drafted their scope of work. While these latter issues will not be a large focus of our work, we do recognize that they are closely related, and perhaps separate workshops can be held over the course of the next five years that address some of these concerns. With all of this in mind, the following objectives were established for the group:

The survey was completed by 75% of FAM's members. Some of the major points include:

General Objective: 

The general objective of the FAM M&E working group over the next five years will be to oversee the development of 1) a set of robust tools to monitor and evaluate Title II programs, and 2) instructions in how to use those tools under various settings. A sub-objective within the general one is to begin to link this work with the areas of problem analysis and project design. In order to achieve this general objective, the working group will rely on the services of individuals and groups both within and external to FAM. This objective is expanded on below.

Specific Objectives:

A tool kit and accompanying instructions (indications, counter-indications and cost analysis) will be developed for (in order of priority as per FAM member surveys): I) agriculture, II) health and nutrition, III) natural resources management, IV) small/micro-enterprise development, V) education, and VI) welfare/social safety. In keeping with this prioritization, the toolkit development will initially focus on the agriculture sector and proceed through the list in the order enumerated above.

For each of the program areas above, tools and instructions will be developed for each of the following M&E components: 1) pre-project rapid assessments, 2) project baseline surveys, 3) project monitoring, 4) mid-term project evaluations, and 5) final project and impact evaluations. Design issues will also be considered in the toolkit development.

For each of the components above, the tools and instructions will be developed based on the following set of tasks: a) review of current tools and instructions in Title II PVO community and expertise in using those tools, b) review of current tools and instructions outside of Title II PVO community and expertise in using those tools, and c) production of a detailed report on the review findings and presentation of toolkit recommendations to FAM members. These tools and instructions will also be made available to ADRA’s Professional Leadership Institute to be developed into courses for graduate students and added to the current curriculum of study.

The review above should include at least the following:

The following is a suggested brief outline of the process for achieving the above objectives:

Beginning in September 1998, a full-time M&E support person will be hired and placed within FAM for the first two years of the grant. The M&E working group will commit to finding the remaining three years funding for the completion of the SOW. It would be the responsibility of this person to complete the objectives of the project which would include the above cited analysis and recommendations for the remainder of the agriculture sector M&E components as well as all components for health, nutrition, S/MED, etc. by the end of LOP. This person will report to the director of FAM. However, the M&E working group will provide planning and oversight support in line with achieving the project's objectives. (For the full scope of work, see Appendix B).

Issues Related to Collaboration during the LOP:

At the end of five years, we anticipate producing, at a minimum, the following outputs:

Food Aid Management Monitoring & Evaluation Working Group

--- Survey of Current Monitoring & Evaluation System ---

1. Name & Title of person completing questionnaire

2. Very briefly (one short paragraph), how does your organization define:

Assessments -

Monitoring -

Evaluation -

3. Do you have a standard Title II program M&E system in your organization or for any of your sectors using Title II resources? If yes, briefly describe its components.

4. Does your organization currently collect baseline data for its Title II programs (or Title II assisted programs) and if so, how are they conducted? What type of technical assistance is utilized? If done in collaboration with other PVOs, how successful were they?

5. Does your organization have HQ staff whose primary responsibility is Title II program M&E? Field staff with the same responsibility?

6. Have you developed your own tools for your M&E activities? If yes, please list the various tools you use: (e.g., Rapid Food Security Appraisal, etc.)

7. Do your methodologies and instruments draw from the work of others external to your organization? If yes, name the organizations, research centers, institutions and/or methodologies from which they are adapted. (e.g., MSF - Food Basket Monitoring Methodology, IMPACT Sampling and M&E Guides, etc.)

Which organization’s work has been most useful to you? Why?

8. Is technical assistance in these areas provided via internal or external sources or both? If internally provided, please name the person(s) upon whom you rely.

If external, please name the agencies upon which you rely to provide technical assistance (e.g., CDC, etc.).

9. What are your most pressing needs or biggest problems with regards to monitoring and evaluating your Title II programs?

10. Are you or your country programs able to track impact with at least one impact indicator from each of the ‘Draft Title II Indicators’ specified by FFP/IMPACT and if not, what are the priority sectors for which indicators need to be revised?

Specific Objectives:

This position will work with the M&E Working group and FAM member organizations to produce a tool kit and accompanying instructions (indications, counter-indications and cost analysis) for the agriculture sector. Tools and instructions will be developed for project baseline surveys and, as time permits, for mid-project evaluations, post-project evaluations, project monitoring, and pre-project rapid assessments.

The tools and instructions will be developed based on the following set of tasks: a) review of current tools and instructions in Title II PVO community and expertise in using those tools, b) review of current tools and instructions outside of Title II PVO community (e.g., Impact Guides, KPC Survey Guide) and expertise in using those tools, c) production of a detailed report on the review findings and presentation of toolkit recommendations to FAM members.

The review above should include at least the following:

The following is a suggested brief outline of the process for achieving the above objectives:

Beginning in September 1998, a full-time M&E support person will be hired and placed within FAM for the first two years of the grant. The M&E working group will commit to finding the remaining three years funding for the completion of the SOW. It would be the responsibility of this person to complete the objectives of the project which would include the above cited analysis and recommendations for the remainder of the agriculture sector M&E components as well as all components for health, nutrition, S/MED, etc. by the end of LOP. This person will report to the director of FAM. However, the M&E working group will provide planning and oversight support in line with achieving the project's objectives. (For the full scope of work, see Appendix B).

FAM Budget for March-September 1998:

Salary and benefits: $39,083

Domestic Travel: $1000

Office Costs: $1000

Total for seven months: $41,083

_________________________________

FAM Budget for Life of ISA:

Laptop computer - $2,200

Salary w/ benefits

Y. 1 53,500 67,000

2 55,375 69,325
 
3 57,320 71,765

4 59,325 74,275

5 61,400 76,875

Travel

International $7,500/Y.

Domestic $2,000/Y.

Toolkit production

and dissemination: $3,000

Office costs (phone, fax and office supplies)

Approximately $2000/Y.

Total Costs:

Y.. 1 $80,700

2 $80,825

3 $83,265

4 $85,775

5 $88,375

LOA Total: $421,940

________________________________

The following are recommendations for PVOs to include in their ISA’s to address the ‘collaborative effort’ components of the grant:

Having been asked to estimate staff time/cost dedicated to DM&E (design, monitoring and evaluation, with the emphasis on M&E presently) dedicated to the Collaborative M&E Effort by PVOs for the next ISG, the following is estimated for an ‘average’ large versus small PVO. Costs are per year, while the grant is for 5 years; only staff salaries, overhead (including transport) and workshops are included.

A general estimate of the total amount allocated to DM&E per PVO is included, to contextualize that spent on the Collaborative Effort. We need to remember that for most PVOs, spending on M&E is not an added cost; this assumes that we CSs are already spending some resources on M&E. And the benefits of pooling our efforts are not estimated below -- benefits may supercede costs in terms of lower costs from pooled resources (e.g. lessons learned, not having to hire expensive consultants, etc) and from time/ staff/ consultancies saved due to the FAM M&E person and inter-PVO collaboration (and resulting saved time in USAID discussions) via this FAM initiative.

Budget #1 based on a Large PVO with an ISA grant of $4 million

(1) Lead HQ backstopping person -- 100% of time/ salary dedicated to development and field assistance for M&E [$ 50,000.00] (25% of that time allocated to Collaborative Effort) = $12,500

(2) 5 sectoral specialists in HQ -- 20% of time/salary dedicated to field assistance for DM&E [@$ 9,000 x 5 = $ 45,000.00] (20% of that time allocated to Collaborative Effort work, including work with regional field staff, being asked for data from FAM M&E person) = $ 9,000

(3) Overhead costs for transport/communications e.g. phone, fax, E-mail = $ 2,000

(4) Sponsoring (including providing space etc $2000), transport and participation (e.g. preparation of materials) for one regional dissemination workshop in the field (bringing in 4 staff at $2000 each) = $10,000.00

TOTAL FAM Collaborative Effort cost estimate for large PVO = $33,500.00 or 19%

This is based on an estimated $177,000 spent annually on M&E during the 5 year grant, totaling approximately $885,000.00 or 20% of the entire grant. (This includes $50,000 on 100% of the salary of the HQ person, $9,000 of each of the salaries (20%) per 5 sectoral specialists in HQ, $2,000 on overhead for FAM/USAID meetings, and sponsoring 2 workshops for field staff ($40,000 each) totaling $80,000 during the five years.) The large agency bears more of the cost of this initiative, percentage-wise.

Budget #2 based on a small agency with a total grant of $1 million

(1) Lead HQ backstopping person -- 30% of time/salary dedicated to field assistance for all DM&E [$ 15,000.00] (20-25% of that time allocated to Collaborative Effort) = $ 3,300.00

(2) Overhead costs for transport/communications e.g. phone, fax, E-mail = $ 2,000

(3) Transport and participation (e.g. preparation of materials) for one regional dissemination workshop in the field = $ 2,000.00

TOTAL FAM Collaborative Effort cost estimate for small PVO = $ 7,300.00 or 9.5%

This is based on an estimated $77,000 spent annually on M&E during the 5 year grant, totaling approximately $385,000.00 or 20% of the entire grant. (This includes $15,000 on 30% of the salary, $2,000 on backstopping FAM/USAID meetings, and sponsoring one workshop for field staff of $60,000 during the five years).