Common Evaluation Terms
A | B | C
| D | E | F
| G | H | I
| J | K | L
| M | N | O
| P | Q | R
| S | T | U
| V | References
| TERM |
DEFINITION |
SOURCE |
| Activities (as referred
to in logic models) |
The actions needed to implement a program, deliver a service,
carry out planned strategiesRelated terms: processes, strategies,
methods, action steps |
Innovation Network Logic Model Workbook p 10
<<Source
web site>> |
| Archival research |
Reviewing written documents (such as meeting
minutes, letters, reports) to obtain information |
|
| Assessment |
Often used as a synonym for evaluation |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Asset mapping |
A method that involves multiple stakeholders
in making a commitment to discover the community’s
capacities and assets instead of focusing solely on needs,
deficiencies, and problems
Related terms: participatory evaluation, culturally competent
evaluation |
CSSP Guide 6 p 76
<<Source
web site>> |
| Attrition |
Loss of individuals or other units of study (such as families,
schools, groups, or organizations) from the defined sample
during a longitudinal study |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Average |
A number between the smallest and largest
in a group or set of numbers, obtained by adding all the
values in the set and dividing the sum by the number of
values
Synonym: mean |
Development Leadership Network, Success
Measures Guide Book Glossary
<<Source
web site>> |
| Baseline |
Information describing current conditions, outcomes and
results; starting point or current situation against which
progress can be measured |
CSSP Guide 6 p 31
<<Source
web site>>
Development Leadership Network, Success Measures Guide
Book Glossary
<<Source
web site>> |
| Benchmark |
A measure that helps quantify the achievement
of a result, outcome or goal; performance data that are
used for comparative purposes (against a baseline, a target,
or the experiences of others)
Synonym: indicator |
CSSP Guide 6 p 21
<<Source
web site>>
UWA Outcome Measurement Resource Network, Measuring Program
Outcomes: A Practical Approach, Glossary
See below for ordering information. |
| Bias |
A consistent alignment with one point of view or one type
of experience |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Case study |
A story that describes a situation
specifically for the purpose of analyzing and learning from
it
Used to organize a wide range of information about a case
and then analyze the contents by seeking patterns and themes
in the data, and by further analysis through cross comparison
with other cases. A case can be individuals, programs, or
any unit, depending on what the program evaluators want
to examine through in-depth analysis and comparison. |
Development Leadership Network, Success
Measures Guide Book Glossary
<<Source
web site>> |
| Coding |
Translation of a given set of data into categories that
can be manipulated using computer technology, such as spreadsheets
(Excel) or statistical analysis packages (SPSS) |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Comparison group |
A group that provides a basis for contrast
with the study group – that is, the group participating
in the activity or program being evaluated
Comparison groups are generally selected to be as similar
as possible to the study group by matching the distribution
of characteristics that are believed to be important in
affecting the study group’s response to the activity
or program being evaluated
Related term: control group |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| Content analysis |
Process of systematically determining the characteristics
of a set of documents or materials |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Control group |
A group that has been selected not to
participate in the activities being evaluated, based on an
unbiased or random selection process |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<Source
web site>> |
| Correlation |
A statistical measure of the degree of relationship between
two variables |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Cost-benefit analysis |
Estimation of the overall costs and monetary
value of each alternative activity or program |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| Cost-effectiveness |
An analysis to compare the benefits or results of an activity
or program to its costs |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Cross-sectional study |
A study examining a population or group
at one point in time |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| Data |
Information, especially information organized for analysis
or used as a basis for decisions |
CSSP Guide 6 p 31
<<Source
web site>> |
| Data collection instrument |
A tool to collect information or data,
including survey questionnaires, interview protocols, focus
group guides, etc. |
Development Leadership Network, Success
Measures Guide Book Glossary
<<Source
web site>> |
| Data collection methods |
Ways in which data or information can be collected
Includes: in-depth interviews, focus groups, surveys, tests,
journals/logs, portfolio reviews, case studies, creative
expressions |
Innovation Network Evaluation Workbook p 12
<<Source
web site>> |
| Data system |
Collection of data from or about individuals,
families, groups, organizations, or communities that comes
from multiple sources
Related term: data warehouse |
CSSP Guide 6 p 60
<<Source
web site>> |
| Data warehouse |
Real or virtual storage place for data from multiple databases,
allowing multiple users to access data via computer links |
CSSP Guide 6 p 67
<<Source
web site>> |
| Database |
A standard set of information on a specific,
limited population and/or activities that is maintained in
electronic form |
|
| Dependent variable |
A variable in a statistical analysis that represents the
outcome or result to be achieved |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Descriptive statistics |
Products from statistical analysis that
summarize, tabulate, organize, and graph data for the purpose
of describing the population or group under study |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| Design |
The investigation procedures to be followed in doing a particular
evaluation |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Empowerment evaluation |
Use of evaluation concepts, techniques,
and findings to foster improvement and self-determination,
conducted by participants |
David M. Fetterman, Am J of Community
Psychology, Vol 30, No 1, February 1, 2002
<<Source
web site>> |
| Evaluation |
Systematic collection of information about an activity to
enable stakeholders to better understand the activity, improve
its implementation, and/or make decisions about future activities |
Innovation Network Evaluation Workbook p 1
<<Source
web site>> |
| Exit survey |
A survey administered to participants
following an activity or program to measure its effects |
|
| External evaluation |
An evaluation conducted by an evaluator from outside the
organization responsible for the activity or program under
study |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Field test |
Conduct of a research design in settings
similar to those in which it is ultimately to be used |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| Focus group |
A group selected for its relevance to an evaluation
that is engaged by a trained facilitator in discussions
to share insights, ideas, and observations on a given topic
Formal process of group discussion and analysis –
typically used for collecting background information, creating
new ideas, assessing how an activity or program is working,
or helping to interpret results from other data sources |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Formative evaluation |
An evaluation designed and used to improve
an activity or program, especially when it is still being
developed |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| GIS (Geographic Information
System) |
A computerized method to gather, integrate, and illustrate
data by geographic units (housing units, blocks, census tracts,
etc.) |
CSSP Guide 6 p 76
<<Source
web site>> |
| Hypothesis testing |
The standard model of scientific research
in which a hypothesis is formulated before the study is conducted;
the results are generally expressed in terms of the probability
that the observed results were due solely to chance |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| Impact |
The changes that result from an event, activity, or program |
Development Leadership Network, Success Measures Guide
Book Glossary
<<Source
web site>> |
| Implementation evaluation |
An evaluation focused on the extent to
which the planned activity or program was implemented |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| Independent variable |
A variable in a statistical analysis that represents conditions
or characteristics of the group under study that were not
the outcome or result to be affected by the activities or
program, including those representing the activity or program
itself |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| In-depth interview |
A method of collecting information
in which the interviewer has a set of open-ended questions
(questions without pre-determined answer categories) designed
to obtain very detailed information
Synonym: unstructured interview |
|
| Indicator |
A measure that helps quantify the achievement of a result,
outcome, or goal
Observable, measurable characteristics or changes that
represent achievement of an outcome
Synonym: benchmark |
CSSP Guide 6 p 21
<<Source
web site>> |
| Informed consent |
Agreement by participants for the use
of personal information, supplied by them in specified ways,
for stated purposes, by the evaluation |
EHR/NSF Evaluation Handbook, Chapter
Seven: Glossary
<<Source
web site>> |
| Input |
Resources that go into a program or activity to produce
its effects |
Development Leadership Network, Success Measures Guide
Book Glossary
<<Source
web site>> |
| Intermediate outcomes |
Changes that are expected to result
from short-term outcomes and link them to long-term outcomes
Synonym: interim outcomes |
Innovation Network Logic Model Workbook
p 16
<<Source
web site>> |
| Interval data |
Data that is organized into categories that can be placed
in order by size, amount or other quantity, with equal sized
intervals between the categories
Related terms: nominal data, ordinal data |
|
|
Level of significance |
The probability that the observations or results of an
evaluation occurred by chance |
EHR/NSF Evaluation Handbook, Chapter Seven: Glossary
<<Source
web site>> |
| Longitudinal study |
An investigation or study in which individuals,
groups, or organizations are followed over time to track
changes |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Long-term outcomes |
Changes expected to result from achievement
of the short-term and intermediate outcomes; more likely
to be affected by other influences and take longer to affect |
Innovation Network Logic Model Workbook
p 16
<<Source
web site>> |
| Mean |
A number between the smallest and largest
in a group or set of numbers, obtained by adding all the
values in the set and dividing the sum by the number of
values
Synonym: average |
Development Leadership Network, Success
Measures Guide Book Glossary
<<Source
web site>> |
| Median |
A number which is half way between the lowest
and the highest in a set of numbers |
Development Leadership Network, Success
Measures Guide Book Glossary
<<Source
web site>> |
| Mode |
The value that occurs more often than any
other in a set of data – if all values occur with the same
frequency, there is no mode; if the two highest values occur
with the same frequency, there are two modes |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Needs assessment |
Process used to gather information that
identifies the problems, challenges, opportunities, and
issues within a targeted population |
|
| No significant difference |
A statement, based on statistical analysis,
that an observed difference between two results occurred
by chance |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Nominal data |
Data that is organized into categories that
are not in order – such as regions of the country, college
attended, etc.
Related terms: interval data, ordinal data |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Observation |
Method of collecting information without
intruding by simply observing and recording |
Development Leadership Network, Success
Measures Guide Book Glossary
<<Source
web site>> |
| Observations |
Way of collecting data that involves watching
others and systematically recording their behaviors according
to preset definitions or categories |
|
| Ordinal data |
Data that is organized into categories that
can be placed in order by size, amount, or other quantity,
but the size of the intervals between the categories are
not known or cannot be measured
Related terms: interval data, nominal data |
|
| Outcome |
A statement about a condition of well being
for children, youth, families, or communities
Synonyms: result, goal |
CSSP Guide 6 p 21
<<Source
web site>> |
| Outcomes (as referred to in logic
models and theories of change) |
The results the activities
are intended to achieve if implemented as intended – the
changes that occur or the difference that is made for individuals,
groups, families, households, organizations, or communities
during or after the activity
Synonyms: results, impacts
Types: individual, family or community, systemic, organizational
|
Innovation Network Logic Model Workbook
p 13
<<Source
web site>> |
| Outputs (as referred to in logic
models) |
Measurable, tangible, and direct results
of activities
Synonyms: deliverables, units of service, products |
Innovation Network Logic Model Workbook
p 12
<<Source
web site>> |
| Participant-observation |
Method that involves joining in the process
that is being observed to provide more of an “insider’s”
perspective |
|
| Performance measure |
A measure of the effectiveness of agency
or program service delivery or other activity – data that
indicate whether a particular program, service, or activity
is achieving its intended results
Synonym: program measure |
CSSP Guide 6 pp 21 and 41
<<Source
web site>> |
| Pilot test |
A brief and simplified preliminary study
designed to try out methods to learn whether a proposed
activity seems likely to yield the expected results or whether
the study methods are feasible and yield useable information |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Population |
All persons or entities in a particular
group |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Post test |
A test to determine performance by an individual
after a program or activity was completed |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Pre-post survey |
A survey administered to participants twice,
both prior to the activity or program and following completion
of the activity or program |
|
| Pretest |
A test to determine performance prior to
the program or activity to establish a baseline or to assess
the feasibility and value of the proposed evaluation methods |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Process evaluation |
An evaluation that focuses on the way in
which the planned program or activity was implemented, with
special attention to the link between inputs and outputs
Synonym: implementation evaluation |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Program measure |
A measure of the effectiveness of agency
or program service delivery or other activity
Synonym: program measure |
CSSP Guide 6 p 21
<<Source
web site>> |
| Qualitative (methods,
data, information, analysis) |
Evaluation methods, data, information, and/or
analysis that use observations, interviews, focus group
information, and surveys to assess change by reporting on
general trends and patterns not based on statistical analysis
Related terms: process evaluation, implementation evaluation |
CSSP Guide 6 p 56
<<Source
web site>> |
| Quantitative (methods, data, information,
analysis) |
Evaluation methods, data, information, and/or
analysis that relies primarily on numeric data and statistical
analysis to determine trends and patterns |
CSSP Guide 6 p 56
<<Source
web site>> |
| Random sampling |
The selection of a limited number of entities
from a larger group or population so that every individual
entity has a specified (usually equal) probability of being
chosen |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Reliability |
Consistency of data values across measurement
instruments or human observers |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Replication |
Repetition of a program, activity, or evaluation
with all essential elements unchanged |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Response bias |
Error due to incorrect or inaccurate information
provided by the subjects of the evaluation that yields results
at the individual, and in some cases aggregate, level different
from the “true” information |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Result |
A statement about a condition of well being
for children, youth, families, or communities
Synonyms: outcome, goal |
CSSP Guide 6 p 21
<<Source
web site>> |
| Results-based accountability |
The steps and processes for making decisions
using results to measure accountability |
CSSP Guide 6 p 17
<<Source
web site>> |
| Sample bias |
Error due to collecting and analyzing information
from a sample rather than the entire population |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Self-evaluation |
An evaluation, performed by the group or
organization whose efforts are being evaluated, to assess
the degree to which it is achieving its goals and realizing
its desired results and to use this information in measuring
progress and making decisions
Related terms: participatory evaluation, action evaluation,
formative evaluation |
CSSP Guide 6 p 55
<<Source
web site>> |
| Short-term outcomes |
Changes expected to occur either immediately
or very shortly after implementation of activities |
Innovation Network Logic Model Workbook
p 16
<<Source
web site>> |
| Significance level |
The likelihood that the results observed
from a study were due to chance – the significance level
of one chance in twenty (probability or p =< .05) or
one chance in 100 (p =< .01) is a high degree of improbability |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Standard deviation |
A measure of the range of values for a variable,
based on the degree of difference of these values from the
mean |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Strategies |
A systematic plan of action to reach predefined
goals
Coherent collection of activities that have a reasonable
chance of improving results or outcomes, based on a set
of assumptions drawn from experience and evidence |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Summative evaluation |
An evaluation designed to present conclusions
about the merit or worth of a program or activity and recommendations
about whether it should be continued, revised, or discontinued |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Survey (form, instrument) |
A structured set of written questions to
collect standardized information – that is, every person
(respondent) who is in the survey is asked exactly the same
questions
Synonym: questionnaire |
Innovation Network Evaluation Workbook p
12
<<Source
web site> |
| Surveys |
A method for gathering both qualitative
and quantitative data from individuals |
CSSP Guide 6 p 76
<<Source
web site>> |
| Targets |
Numerical objectives for a level of achievement
on outcomes |
UWA Outcome Measurement Resource Network,
Measuring Program Outcomes: A Practical Approach, Glossary
See below for ordering information. |
| Unanticipated outcomes |
A result of a program or activity that was
unexpected |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Unstructured interview |
A method of collecting information in which
the interviewer has a set of open-ended questions (questions
without pre-determined answer categories) designed to obtain
very detailed information
Synonym: in-depth interview |
|
| Validity |
The degree to which information
collected in a particular way accurately represents the
phenomenon under study |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| Variable |
An item of information the values of which
vary among the population under study |
EHR/NSF Evaluation Handbook, Chapter Seven:
Glossary
<<Source
web site>> |
| |
|
|
References
The 2002 User-Friendly Handbook for Project Evaluation, J. Frechtling
and C Katzenmeyer, The National Science Foundation, Directorate
for Education & Human Resources, Division of Research, Evaluation,
and Communication: Washington DC, January 2002. www.nsf.gov/pubs/2002/nsf02057/nsf02057.pdf
Empowerment Evaluation: Building Communities of Practice and
a Culture of Learning, D. M. Fetterman, American Journal of Community
Psychology, February 1, 2002. www.nexis.com/research
IAC-ACC-NO: 84547036.
Logic Model Workbook, Innovation Network, Washington DC, no date.
www.innonet.org/files/view/LM_workbook.pdf
Measuring Program Outcomes: A Practical Approach, United Way
of America, Alexandria VA, 1996. To order, contact Sales Service/America
at 800-772-0008 (toll-free U.S.) or 703-212-6300. Item No. 0989.
Price: $5 (plus shipping and handling).
Success Measures Guide Book Version, Development Leadership Network,
Boston MA, 2000. www.developmentleadership.net/smp/manual/toc.htm
Using
Data to Ensure Accountability, Number 6 in Series of Learning
Guides for Building Capacity for Local Decisionmaking, Center
for the Study of Social Policy, Washington DC, June 2001. www.cssp.org/uploadFiles/LearningGuide_6.pdf
|
