This checklist for panel administrators can help with setting up the assessment process of a policy quality panel.
This PDF is editable and can be completed electronically, or printed blank and filled out by hand.
Formats
- Decide the process the agency will use (e.g. internal panel, internal panel with some external members, cross-agency panel, with or without a follow-up external review, one-off annual panel or several sessions staggered throughout the year, independent or internal chair etc.).
- Set out the roles and responsibilities of the chair and panel members – possibly in a terms of reference.
- Document your panel review processes, including sample selection, panel meetings, feedback process, and make it available.
- Appoint a panel chair and administrator and identify who will be taking notes during panel meetings.
- Identify panel members who are open to exchanging constructive feedback. Get agreement from their managers (for external members, agree time commitment involved and contract for it if necessary).
- Identify back-up panel members, in case of sickness or unavailability.
- Ensure selected panel members have appropriate security clearances.
- With the panel chair agree:
- what population of papers is relevant and whether a random sample of the whole population should be drawn or a structured random sample by type of paper
- on the sample size and who will conduct the sampling (e.g. the administrator, the panel chair)
- who is responsible for writing up the assessments of each paper and the final report
- how and if you want to get context for the paper (e.g. by contacting managers before the panel review)
- if the author or their manager will be present, who is the appropriate lead contact
- the schedule for meeting and assessments, the paper distribution date, and due dates of the draft and final report
- how authors, their managers and the agency’s leadership will receive feedback.
- Schedule panel meetings for members and secretariat (ideally four three-hour sessions for 30 papers).
- Induct any new panel members by providing them with the guide for panels, any additional internal material, and providing a briefing by the chair.
- Collect electronic copies of papers along with the authors’ contact details.
- Collect from the manager or author the background and context of the paper.
- Distribute to all panel members printed copies of the papers and blank paper-scoring templates, which include the scale for scoring the quality of advice (next page).
- Have the panel chair brief the panel on process and signal when the papers need to be read.
- Organise sign off for draft and final reports.
- Report results to senior leadership, and scores to those responsible for annual performance reporting.
- Schedule feedback sessions with managers/policy teams/peer reviewers/authors on individual papers.
- Schedule feedback sessions with policy teams across your organisation on the results as a whole.
- Identify any best practice examples and make them available.
- Consider any initiatives you could undertake to improve quality by addressing the issues raised in reviews. This might include developing a policy quality improvement plan.