References, Tips and Tools


The Annotated Bibliography is a carefully selected set of research-based resources that practitioners and administrators can use to support their work on the Program Improvement/Professional Development (PI/PD) plans. This reference list allows practitioners to investigate independently high quality data resources. Criteria for selection includes being online, free (nearly all), practical, and research-based. The resources consist of articles, government reports, tables, videos, websites, book chapters, and entire books. Each annotation includes a brief overview and, when possible, the page numbers and video length. Most resources were culled from the internet and vetted so that they are practical and of high quality.

The Annotated Bibliography is a work in progress. As new resources are located, they will be added. As part of the Data Learning Community, please send any resources you wish added to the bibliography to Michael McCarty at  Please include the title, URL, and your reason for wanting the resource added.


This section contains tips for basic qualitative data collection. The tips sheets provide basic information on the method, such as when to use it, how to plan for it, and how to implement it.


This section will comprise of Tools that have been researched and found to have potential value to the field. Program staff who have developed tools that were successful for their PI/PD plan are invited to send them to Michael McCarty at for review and inclusion, so that other programs can benefit from the information.

Tools may include anything that relates to Qualitative Data Methods:

  • Interviews (protocols and questions)
  • Direct Observation (protocol)
  • Surveys and Questionnaires (protocols and questions)
  • Document analysis (protocol for analysis of portfolios, agendas, minutes, lesson plans, implementation guides, photographs, videos, etc.)

MIS Identified Tools


1. Parental Support Template
A seven-question Likert scale (“Not at all” to “A tremendous amount”) that explores how well parents perceive their involvement with their child (school, friends, learning).

2. Family Literacy Resources for New Parents feedback survey
A questionnaire for a program to ascertain how new families enjoyed a variety of resources (“Clear,” “Fun,” “Interesting,” “Valuable,” “Enjoyable”—Check All that apply); checklist of other resources they’d like; access and use of website; program size, etc. Mostly checklist, but some open-ended. (Actually, I think this is a better example of a not-so-good survey. Either include it with that caveat or delete it).

Interviews/Focus Groups

1. Sample In-depth Interview Guide
Protocol example (two pages) for conducting an interview with questions and probes.

2. Sample Focus Group Topic Guide
Protocol example (5 pages) for conducting a focus group. The topic here for discussion are the new standards.

Direct Observation

1. Sample Observation Instrument
Observation protocol (three pages) for observing faculty. Includes checklists, open-ended descriptions (e.g. hands-on activities, problem-solving activities, technology, assessment). Section Two (10 pages) is a Likert type scale of key indicators to rate the session. It also asks for “supporting evidence” (open-ended).

2. Classroom Observation Protocol
A very detailed descriptive and evaluative observation tool (17 pages). Includes open-ended items (e.g. description of activities, demographics, resources, behaviors) as well as continuum (e.g. interactions, engagement, teacher role). This tool is designed to solicit feedback from both the teacher and the observer. Ten pages of this document are dedicated to instructions and to observation protocol.

3. MIS developed tools

Tools from the Field

Submitted with permission:

1. Classroom Observation Rubric Mary Lou Friedline, PIC, Westmoreland County, PA.

2. Prior Schooling Inventory Dr. B. Allan Quigley, retired, St. Francis Xavier University, Nova Scotia, CA

Get the free download of Adobe Acrobat Reader


Explore the comprehensive range of resources and information tailored for agency administrators. Uncover valuable insights and tools to optimize your administrative role and ensure the effective functioning of your agency.


Explore a wide range of resources, including customized lesson plans and digital literacy materials, designed to elevate your teaching and tutoring experiences with adult learners.


Explore the world of precise and comprehensive data collection, entry, and analysis to enhance your program planning and foster continuous improvement.


Workforce/Career Pathways

Discover an array of resources and valuable information related to career pathways and workforce training. Explore opportunities that can empower your learners to make informed decisions and advance along their desired career journey.

Professional Development (PD)

Unlock a wealth of knowledge and strategies aimed at enriching your adult education practice. Explore a diverse array of methods and opportunities, equipping you with the tools to continuously improve and excel in your role as an educator.

Support for Programs