||Last Updated: Nov 17th, 2006 - 18:39:24
Preface to Guidelines for OSEP Leadership Program Competition Applications
August 20, 2004
The following document provides draft guideline and proposal evaluation recommendations for the Office of Special Education Program’s Personnel Preparation Leadership Program. The recommendations were developed by a ‘Blue Ribbon Committee (BRC)’ appointed by Assistant Secretary of the Office of Special Education and Rehabilitation Services to review the Leadership application and proposal evaluation process included in the Leadership Program’s annual request for proposals. The Committee was composed of a representative array of current, former, recent and new Leadership Program awardees. Dr. William Berdine, University of Kentucky and immediate past-president of the Higher Education Consortium for Special Education (HECSE), chaired the Committee. The current HECSE President, Herb Rieth, served as co-chair and scribe for the Committee. Members of the whole Committee were: Phil Burke (University of Maryland), Vivian Correa (University of Florida), Don Deshler (University of Kansas), Linda Flynn (University of New Orleans), Marty Kaufman (University of Oregon), Jennifer McComas (University of Minnesota), Doug Palmer (Texas A&M; University), Dan Reschly (Vanderbilt University, Mike Rosenberg (Johns Hopkins University), Charles Salzberg (Utah State University), Deb Smith (Vanderbilt University, Wilfred Wienke (Central Florida University), and Brenda Williams (William & Mary College). The final draft recommendations to OSEP were completed on December 15 & 16, 2003 by committee members Berdine, Burke, Deshler, McComas, Reschly, Rieth, and Salzberg. OSEP staff to the Committee was Bob Gilmore. The December 2003 ‘draft’ recommendations will be vetted with the HECSE Executive Board and members attending the 2004 annual HECSE membership meeting on January 22- 25 of that year.
Underlying Assumptions and Recommendations for the OSEP Leadership Program Application Process
1.The guiding principle of the BRC was to develop a set of recommendations for changes in the Leadership Program application, proposal evaluation, and paneling processes that would enhance the preparation of high quality leadership personnel in the field of special education and related disability service.
• The BRC recommends under the revised guidelines an applicant for an award under the OSEP Leadership competition must provide documentation of the existence of a comprehensive doctoral curriculum designed to meet program goals and attain mastery of required professional domains, including research methodology, personnel preparation, and policy/advocacy or professional practice. New Leadership Program applicants as well as previous applicants and awardees will be held to documenting the existence of a comprehensive doctoral program. Leadership Program funds will not be used to establish new doctoral programs.
2.The BRC discussed and scrutinized the paneling process used historically to determine Leadership award recipients.
• The BRC recommends the establishment of a separate standing panel for the Leadership Program competition. Membership in the standing panel would be open to any existing faculty member employed by an accredited university holding full graduate faculty status in a special education or related service area doctoral program. Additionally, all members of the standing panel must agree to a minimum of three years of service to the panel and to undergo training in the paneling processes. Panel members from IHEs with existing Leadership Program funding may serve on the standing panel but will be excluded from review and discussion regarding applications from their university.
3.After considerable review and consistent with the creation of a standing panel, the panel review process was extensively discussed.
• The BRC recommends the development of a new Leadership Panel Peer review form that includes a redistribution of points awarded in each of the eight evaluation categories. The redistribution of points is intended to concentrate 75% of the points on the quality of program evidence of program efficacy, adequacy of faculty and university resources and quality of project evaluation.
With regard to additional Leadership Program process, The BRC’s made the following recommendations.
4.. The Committee did not find convincing evidence for the continue use of “ competitive priorities”.
• The BRC recommends that they be eliminated as unnecessary and potentially divisive to the profession and professoriate.
5. The Committee did not find compelling evidence that doctoral programs in special education and related services were SEA- or LEA-based programs.
• The BRC recommends elimination of any required collaboration with SEA on anything but a voluntary basis The requirement for collaboration with state education agency was found to be irrelevant for Leadership Preparation as they should have a national, rather than local or state, focus. The state and/or local focus is more appropriate for teacher or clinician certificate preparation programs..
6. The Committee expended considerable time and solicited substantive input with regard to budget matters in the current Leadership Program grant award program.
• The BRC recommends that OSEP develop a provision to allow applicants to petition for exemption from program budget requirement requires 65% of the grant award must be allocated for student support. This proviso will allow program applicants to make the case for alternative student support strategies such as teaching assistantships, program assistants, and student teacher supervision to permit individual applicants to meet specific program needs.
7. The Committee found that requiring the inclusion of comprehensive syllabi for all program courses is unnecessary and added considerably to excessive paper use.
• The BRC recommends that OSEP drop its requirement for the submission of course syllabi for the Leadership application process.
With regard to the creation of new Peer Review Form and Application Packet for the OSEP Leadership Program the following underlying agreements or understandings were agreed to:
Underlying Guiding Agreements That Serve as the Framework for a New Peer Review Form and Application Packet for the OSEP Leadership Program
1. What Constitutes a Leadership Program
a. Applicants must:
i. Provide a list of program goals and objectives;
ii. Describe the comprehensive curriculum designed to meet program goals and attain mastery of the following required professional domains:
• Research methodology
• Personnel preparation
• Policy/advocacy OR Professional practice;
iii. Identify program faculty with expertise and provide a succinct listing of accomplishments germane to the grant proposal;
iv. Specifically designate key personnel for the grant; and
v. Describe how specific goals and objectives, as well as the comprehensive curriculum, will be delivered by identified program faculty.
2. Paneling Process
a. Qualifications of panelists
i. A member of a doctoral program faculty in special education and/or related services at an accredited institution of higher education
i. Explicit, standardized instructions to panelists;
ii. A separate standing panel for Leadership competition – each panelist is appointed for a 3 year term;
iii. Reviews are to conducted on site;
iv. Panels are to managed on site by OSEP staff;
v. Elaborate written feedback must be provided on each evaluation criterion;
vi. A score changed during the paneling process must be accompanied by a detailed written explanation; and
vii. Outlying scores are automatically submitted for OSEP administrative review.
3. Program vs. Project
a. High quality program – program emphasis.
4. Evaluation Outcomes Required of Leadership Training Programs
a. Information regarding graduates – required of previous grant recipients
ii. Institutional affiliation; and
iii. Professional accomplishments.
b. Student accomplishments
iii. Teaching; and
c. Graduation rate and average time to completion
d. Knowledge and skills of students/graduates
e. Incorporation of scientifically validated practices
f. Faculty and student contributions – development of scientifically validated practices and/or other evidence of scholarship
5. Student Recruitment
a. Describe plan and process for recruiting students with high potential for successful completion of the program - indicators may include grades, test performance, publications, and teaching evaluations.
b. Recruit, educate, and graduate students with attributes, knowledge, and
skills consistent with the career emphasis of program;
c. Diversity – how does the program address the issue of diversity of student body and demonstrate its commitment to recruit students from under-represented populations; and
Consistent with the above recommendations and guiding agreements, the following is a prototype for a recommended new OSEP Leadership Program Peer Review Form.
PEER REVIEW FORM
Need for Program - 5
Quality of Program - 30 (Note: this is a name change that must be approved at the Federal level)
Evidence of Program Efficacy 10
Adequacy of Faculty and University Resources 20 – (Note: to include all participating faculty – a change that must be approved at the Federal level)
Quality of Program Personnel 10
Quality of Management Plan 5
Quality of Project Evaluation 15
Adequacy of Budget 5
1. Need for Program – 5 points
In determining the need for the proposed program, please consider the following factors: (1) document the need for the leadership personnel program described in the proposal; and (2) The extent to which the proposed program will prepare personnel for fields in which shortages have been demonstrated.
2. Quality of Program – 30 points
In determining the quality of the leadership preparation program please consider the following factors: (1) The extent to which the leadership preparation program incorporates state of the art research and practice in the domains of research methodology, personnel preparation and policy/advocacy or professional practice; (2) The extent to which the proposed leadership preparation program is of sufficient quality, intensity and duration to lead to improvements in practice among the recipients of the leadership preparation. (3) demonstrate that the program incorporates scientifically validated practices, and (4) describe faculty and student contributions to developing scientifically validated practices and/or other evidence of scholarship
3. Evidence of Program Efficacy – 10 points
Evidence of Leadership Training Program is required. Evidence may include: (a) Information regarding program graduates including a list of positions taken by graduates, their institutional affiliation and their professional accomplishments; (b) A list of accomplishments of currently enrolled students including presentations, publications, teaching and honors; (c) data regarding student graduation rate and average time to completion, (d) the knowledge and skills acquired by students/graduates
4. Adequacy of Faculty and University Resources – 20 points
In determining the adequacy of faculty and university resources for the proposed program, please consider the following factors: (1) the qualifications of the faculty directly involved in the accomplishment of the program goals and objectives (e.g., teaching required courses, advising, mentoring, directing research, and supervising applied experiences); (2) the adequacy of support including facilities to support the proposed leadership preparation activities (e.g., Centers of Excellence, Research Institutes, and Policy Institutes), (3) equipment, (4) supplies, and (5) other resources from the applicant, and the potential for continued support of the program after Federal funding ends, including, as appropriate, the demonstrated commitment of appropriate entities to such support.
5. Quality of Key Program Personnel – 10 points
In determining the quality of key program personnel, please consider the following factors: (1) relevant training, and experience in leadership preparation of the Principal Investigator and/or Co-Principal Investigator.
6. Quality of the Management Plan – 5 points
In determining the quality of the management plan for the proposed program, please consider the following factors: (1) the adequacy of the management plan to achieve the objectives of the proposed program on time and within budget; and (2) the extent to which the time commitments of the program director and principal investigator and other key program personnel are appropriate and adequate to meet the objectives of the proposed program.
7. Quality of Project Evaluation – 15 points
In determining the quality of the evaluation, please consider the following factors: (1) the extent to which the methods of evaluation are thorough, feasible, and appropriate to the goals, objectives, and outcomes of the proposed program; and (2) the extent to which the methods of evaluation include the use of objective performance measures that are clearly related to the intended outcomes of the program and will produce quantitative and qualitative data.
8. Adequacy of Budget – 5 points
In determining the adequacy of the budget, please consider: (1) the extent to which the budget is adequate to support the proposed program.
Evaluation point values and attached descriptors
Weak – 0-1
Adequate – 2 - 3
Excellent - 4
Outstanding – 5
© Copyright 2006 HECSE
Top of Page