Outcomes Based Program Review Closing

Achievement Gaps in and Outside the Classroom

with Alignment to Predictive Analytics and
Performance Metrics 2nd Edition Marilee J.
Bresciani Ludvik pdf download
https://ebookmeta.com/product/outcomes-based-program-review-closing-achievement-gaps-in-and-outside-
the-classroom-with-alignment-to-predictive-analytics-and-performance-metrics-2nd-edition-marilee-j-
bresciani-ludvik/
★★★★★ 4.6/5.0 (20 reviews) ✓ 204 downloads ■ TOP RATED
"Great resource, downloaded instantly. Thank you!" - Lisa K.

DOWNLOAD EBOOK
 Outcomes Based Program Review Closing Achievement Gaps in
and Outside the Classroom with Alignment to Predictive
Analytics and Performance Metrics 2nd Edition Marilee J.
Bresciani Ludvik pdf download

TEXTBOOK EBOOK EBOOK META

Available Formats

■ PDF eBook Study Guide TextBook

EXCLUSIVE 2025 EDUCATIONAL COLLECTION - LIMITED TIME

INSTANT DOWNLOAD VIEW LIBRARY

Collection Highlights

Using Predictive Analytics to Improve Healthcare Outcomes

1st Edition John W. Nelson

Predictive Analytics with KNIME: Analytics for Citizen

Data Scientists 1st Edition Acito

Connecting the Dots Developing Student Learning Outcomes

and Outcomes Based Assessment 2nd Edition Ronald S
Carriveau

Cheer in a Bookshop Single Mom Holiday Romance Bookshop

Romance Book 1 1st Edition Elsie James [James
Textbook of Gastrointestinal Radiology 5th Edition Richard
M. Gore

Wills, Trusts, and Estate Administration Janis Walter

Macroeconomics, 22nd Edition Campbell R.. Brue Mcconnell

(Stanley L.. Flynn

Disney Junior Mickey Christmas Tales Disney Books

Building Business Value Through Talent The CEO and CHRO

Partnership Guide 1st Edition Thomas Mcguire Linda Brenner
The Papacy and the Orthodox Sources and History of a
Debate Oxford Studies in Historical Theology 1st Edition
A. Edward Siecienski
 OUTCOMES-
“This book lays out a very effective case for using assessment in response to the cry for

BRESCIANI LUDVIK
accountability from all constituents of higher education. It can help all parties better un-
derstand how institutions use assessment to ensure a quality education is being provided
to their students.” —BELLE S. WHEELAN, President,
Southern Association of Colleges and Schools Commission on Colleges (SACSOC)

“This book is useful for anyone within higher education interested in sustainable
outcomes-based approaches with clear examples and tools drawn from the field.”
—NATASHA JANKOWSKI, Director, National Institute for Learning Outcomes
Assessment (NILOA) CLOSING ACHIEVEMENT GAPS IN- AND OUTSIDE

BASED
OUTCOMES-BASED PROGRAM REVIEW
“This new edition is a timely gift for higher education. Every person at a higher edu-
cation organization can locate some way to contribute to achieving enhanced student
success through the clear, comprehensive guidance it sets forth.”
—TERREL L. RHODES; Vice President; Office of Quality, Curriculum, and
Assessment; Association of American Colleges & Universities

“Marilee J. Bresciani Ludvik stresses the practical application of what for many is a
daunting challenge: linking the worlds of outcomes assessment, program review, pre-
THE CLASSROOM WITH ALIGNMENT TO

PROGRAM
dictive analytics, and competency-based education. Her work is an absolute necessity for
institutions like mine, which will soon undergo its 10-year SACSCOC accreditation.”
—JAMES A. ANDERSON, Chancellor and Professor of Psychology,
Fayetteville State University

“Bresciani Ludvik’s second edition has now become the most comprehensive resource
for effectively developing institutionally useful program reviews at all levels across a
college, university, or system.” —PEGGY MAKI, Writer and Higher Education
Consultant specializing in assessing student learning
PREDICTIVE ANALYTICS AND PERFORMANCE METRICS

REVIEW
A
mong the new topics Marilee J. Bresciani Ludvik introduces in this edition
is how to appropriately connect outcomes-based program review (OBPR) to
performance indicators and predictive analytics and develop meaningful new
performance metrics to inform our understanding of the student experience. She also
addresses the intersection of OBPR with competency-based assessment.

MARILEE J. BRESCIANI LUDVIK is professor for postsecondary education and

works within the newly created Office of Educational Effectiveness at San Diego State
University.
HIGHER EDUCATION | ASSESSMENT
MARILEE J. BRESCIANI LUDVIK
978-1-62036-230-3 Foreword by Ralph Wolff
22883 Quicksilver Drive, Sterling, VA 20166-2019
www.Styluspub.com
Second Edition of Outcomes-Based Academic and Co-Curricular Program Review
 Praise for Outcomes-Based Program Review
“As a former colleague of Marilee J. Bresciani Ludvik at both North Carolina
State University and Texas A&M University, I can speak with authority about
her expertise and commitment to the exploration of outcomes-based assess-
ment in the context of program review. In her new book she has stressed
the practical application of what for many is a daunting challenge: linking
the worlds of outcomes assessment, program review, predictive analytics,
and competency-based education. Moreover, she extends her analysis to a
broader institutional context, suggesting that an institution cannot assert
that it is a learning-centered institution if it does not link critical aspects of
academic affairs, student services, faculty development, and professional staff
development. As in her previous written work, Bresciani Ludvik asks the
reader to formulate and consider key questions and to review the best prac-
tices and best models. Her work is an absolute necessity for institutions like
mine, which will soon undergo its 10-year Southern Association of Colleges
and Schools Commission on Colleges accreditation.” —James A. Anderson,
Chancellor and Professor of Psychology, Fayetteville State University

“Colleges and universities in the United States expanded rapidly in the twen-
tieth century in response to the need to provide access to higher education
and training to a greater percentage of an increasing population. While the
most elite universities focused on scholarship and the production of knowl-
edge, giving students an opportunity to learn was the most significant mis-
sion for most colleges and universities. However, they took no responsibility
for the learning and success of their students. A common philosophy in the
1960s was ‘students have a right to fail,’ meaning that colleges should give
students a chance, but institutions have no responsibility for their success or
failure—that was up to the student.
Over the past 30 years, there has been a significant shift in the mission
of higher education. Today’s colleges and universities are being asked to be
accountable for not only student access but also outcomes including stu-
dent learning, degree completion, and equity. Measuring success outcomes
is much more difficult and controversial than measuring student enrollment,
and the efforts have sometimes been contentious. In this second edition of
Outcomes-Based Program Review, Marilee J. Bresciani Ludvik provides road-
maps and lessons learned that can guide today’s educators in the use of data
and predictive analytics to improve student success and to close achievement
gaps. Bresciani Ludvik reminds us that as discussions about learning out-
comes continue, educators should be careful not to fall into the trap of mak-
ing their purpose just about a report to satisfy an accrediting commission

Ludvik.indb a 27-10-2018 00:23:20

 or a state agency. Institutional improvement is much more important than
mandated reports.” —George R. Boggs; Superintendent/President Emeritus,
Palomar College; President and CEO Emeritus, American Association of
Community Colleges; and Chair, Phi Theta Kappa Board of Directors

“In the second edition of Marilee J. Bresciani Ludvik’s book on outcomes-

based assessment program review, she brings a renewed focus to achievement
for all students, as well as connections between outcomes conversations and
predictive analytics along with other potentially performance-enhancing
technologies. While there is much debate about the role of technology in
higher education, outcomes in general, the apparently decreasing value of a
degree, and the criteria upon which higher education should be judged and
held accountable, Bresciani Ludvik’s book offers a breath of fresh air and a
meaningful way forward.
Viewing outcomes conversations as part of a learning organization that
focuses upon asking the ‘right’ questions about our practice, she presents
a nonprescriptive process focused on reflection, collaboration, and adapt-
ability. She reminds us that it is through conversations, space, and time for
reflection that we can foster processes not based in compliance, but in col-
lecting, analyzing, and interpreting data in ways that matter to individual
institutions.
This book is useful for anyone within higher education interested in
sustainable outcomes-based approaches with clear examples and tools drawn
from the field. Presenting peer-reviewed criteria for good practice, useful
appendices, and an honest discussion regarding ongoing questioning of pro-
cesses along with barriers to good practice, this book is a welcome addi-
tion to the conversation.”—Natasha Jankowski; Director, National Institute
for Learning Outcomes Assessment (NILOA); and Research Assistant Professor,
Education Policy, Organization, and Leadership (EPOL), University of Illinois
Urbana-Champaign, College of Education

“If our institutions of higher education are truly institutions of learning, we

must dedicate ourselves to learning from our primary practice. We must seek
to improve continuously the extent to which our programs generate evidence
of learning. This splendid book is a great place to start.”—Joe Johnson;
Executive Director of the National Center for Urban School Transformation; and
the QUALCOMM Professor of Urban Education, San Diego State University

“Outcomes assessment in higher education is a magnet for criticism, in

large part because too often the effort does not yield meaningful, actionable

Ludvik.indb b 27-10-2018 00:23:24

 information. This refreshed volume is a timely antidote of sorts, distilling
lessons learned about best practices from scores of schools doing assessment
right to persuasively explain why and expertly illustrate how the essential
work of collecting and using evidence of student learning can benefit stu-
dents, faculty and staff, institutions, and the public.”—George D. Kuh;
Chancellor’s Professor Emeritus of Higher Education, Indiana University; and
Senior Scholar, National Institute for Learning Outcomes Assessment

“For Bresciani Ludvik the systematic and reflective inquiry processes involved
in developing a program review and the contents of that review are essential
to an institution that has become, or seeks to become, a learning organiza-
tion. Useful reviews developed according to Bresciani Ludvik’s guidance and
a plethora of examples and resources provide lenses through which institu-
tional leaders can continuously explore how well the departments, programs,
and services of their institution are achieving two major goals: (a) fulfillment
of institutional mission and purposes at demonstrably high quality levels and
(b) preparation of all students to achieve the high quality learning outcomes
necessary to contribute to the demands of the twenty-first century. Bresciani
Ludvik’s second edition has now become the most comprehensive resource
for effectively developing institutionally useful program reviews at all levels
across a college, university, or a system.”—Peggy Maki, Writer and Higher
Education Consultant specializing in assessing student learning

“If you are looking to make your program review or accreditation processes
more meaningful and useful, look no farther. Outcomes-Based Program
Review, 2E provides plenty of practical advice on how to do exactly that. It
will help you create an intentional process that makes sure the right ques-
tions are asked. It will help you ensure that your processes generate reflective,
collaborative, evidence-informed dialogue. And it will help your institution
enhance its impact.”—Linda Suskie, Assessment and Accreditation Consultant

“The two ‘As’ by which higher education is evaluated today are accountability
and assessment. This book has laid out a very effective case for using assess-
ment in response to the cry for accountability from all constituents of higher
education. From Congress to accreditors, employers and the general public,
this book can be easily used to help all parties better understand how institu-
tions use assessment to ensure a quality education is being provided to their
students.”—Belle S. Wheelan, President, Southern Association of Colleges and
Schools Commission on Colleges

Ludvik.indb c 27-10-2018 00:23:24

Ludvik.indb d 27-10-2018 00:23:24
 OUTCOMES-BASED PROGRAM REVIEW

Ludvik.indb i 27-10-2018 00:23:24

Ludvik.indb ii 27-10-2018 00:23:24
 OUTCOMES-BASED
PROGRAM REVIEW
Closing Achievement Gaps In- and Outside
the Classroom With Alignment to Predictive
Analytics and Performance Metrics

Marilee J. Bresciani Ludvik

Foreword by Ralph Wolff

Second Edition of Outcomes-Based Academic and

Co-Curricular Program Review

STERLING, VIRGINIA

Ludvik.indb iii 27-10-2018 00:23:25

 COPYRIGHT © 2019 BY STYLUS PUBLISHING, LLC.

Published by Stylus Publishing, LLC.

22883 Quicksilver Drive
Sterling, Virginia 20166-2019
All rights reserved. No part of this book may be reprinted or
reproduced in any form or by any electronic, mechanical, or other
means, now known or hereafter invented, including photocopying,
recording, and information storage and retrieval, without
permission in writing from the publisher.

Library of Congress Cataloging-in-Publication Data

Names: Bresciani, Marilee J., author.

Title: Outcomes-based program review : closing achievement gaps
in and outside the classroom with alignment to predictive analytics
and performance metrics / Marilee Bresciani Ludvik ; foreword by
Ralph Wolff.
Other titles: Outcomes-based academic and co-curricular program
review
Description: Second Edition. | Sterling, Virginia : Stylus Publishing,
LLC, [2018]
Identifiers: LCCN 2018013858 (print) | LCCN 2018032971
(ebook) | ISBN 9781620362310 (Library networkable e-edition) |
ISBN 9781620362327 (Consumer e-edition) |
ISBN 9781620362303 (Paper : acid free paper) |
ISBN 9781620362297 (Cloth : acid free paper)
Subjects: LCSH: Educational tests and measurements--United
States. | Competency-based education--United States. | College
teaching--United States. | Education, Higher--United States.
Classification: LCC LB3051 (ebook) | LCC LB3051 .B693 2018
(print) | DDC 371.26--dc23
LC record available at https://lccn.loc.gov/2018013858
13-digit ISBN: 978-1-62036-229-7 (cloth)
13-digit ISBN: 978-1-62036-230-3 (paperback)
13-digit ISBN: 978-1-62036-231-0 (library networkable e-edition)
13-digit ISBN: 978-1-62036-232-7 (consumer e-edition)

Printed in the United States of America

All first editions printed on acid-free paper

that meets the American National Standards Institute
Z39-48 Standard.

Bulk Purchases
Quantity discounts are available for use in workshops and
for staff development.
Call 1-800-232-0223

Second Edition, 2019

Ludvik.indb iv 27-10-2018 00:23:25

 CONTENTS

TABLES AND FIGURES vii

FOREWORD ix
Ralph Wolff

PREFACE xv
How to Use This Book

ACKNOWLEDGMENTS xxiii

1 WHY 1
It’s About Becoming a Learning Organization

2 WHAT 19
Defining Outcomes-Based Assessment Program Review

3 MORE WHY AND WHAT OF OUTCOMES-BASED ASSESSMENT

PROGRAM REVIEW 41

4 CRITERIA FOR GOOD PRACTICE OUTCOMES-BASED ASSESSMENT

PROGRAM REVIEW 75

5 HOW AND WHEN 91

Key Questions to Consider When Implementing Good Practice
Outcomes-Based Assessment Program Review

6 OVERCOMING BARRIERS TO IMPLEMENTING GOOD

PRACTICE OUTCOMES-BASED ASSESSMENT PROGRAM REVIEW 125

APPENDIX A 161
Documents Used to Determine Good Practice Criteria for
Outcomes-Based Assessment Program Review

APPENDIX B 167
List of Good Practice Institutions That Were Nominated by
Scholars and Practitioners

Ludvik.indb v 27-10-2018 00:23:25

 vi CONTENTS

APPENDIX C 171
National Institute for Learning Outcomes Assessment High Performance for
All Students: Comparable Learning and Development Outcome Measures and
Performance Indicators

APPENDIX D 179
Oregon State University Student Affairs Learning Goals and
Outcomes Alignment Grid

APPENDIX E 181
Hagerstown Community College Map Template

APPENDIX F 183
California State University, Sacramento, Example of Annual Outcomes-Based
Data Collection

APPENDIX G 187
James Madison University Example of an Annual Assessment Report Template

APPENDIX H 203
University of Hawai‘i-Mānoa Assessment Results and
Improvement Plan Template

APPENDIX I 205
Azusa Pacific University Five-Year Action Plan Template

APPENDIX J 207
Outcomes-Based Assessment Plan and Report for Program
Review Purposes Checklist

REFERENCES 217

ABOUT THE AUTHOR 223

INDEX 225

Ludvik.indb vi 27-10-2018 00:23:25

 TABLES AND FIGURES

Tables
Table 3.1 Alignment Table 53
Table 3.2 University of St. Thomas Student Learning Outcomes
Curriculum Map Psychology BA Program 56
Table 3.3 Curriculum Map Example: Excerpt From a Sample
Biology Program at the University of Hawai‘i-Mānoa 57

Figures
Figure 1.1. Learning and development as neurocognitive skills. 11
Figure 1.2. Semantic map of executive functions and related terms. 12
Figure 1.3. Map of executive functions and related terms to intra-
and interpersonal skills. 13
Figure 2.1. Fitzpatrick and colleagues’ (2011) steps in program
evaluation. 23
Figure 2.2. Assessment process used at Miami Dade College. 24
Figure 2.3. Assessment process used at James Madison University. 25
Figure 2.4. The iterative systematic OBPR/Gather Data cycle. 27
Figure 2.5. OBPR can be implemented at multiple levels. 30

vii

Ludvik.indb vii 27-10-2018 00:23:25

Ludvik.indb viii 27-10-2018 00:23:25
 FOREWORD

I
n 2006 when the first edition of this book was published, I wrote that a
book like this was much needed. Since then, the need for an outcomes-
based approach to program review is more needed than ever. That is not
to say that program reviews have become more popular; indeed, too often
program reviews do not lead to significant improvements, change, or new
resources and are therefore viewed (and painfully undertaken) as a useless
and time-consuming ritual. As well described in this updated edition, how-
ever, when program reviews are well designed and highly intentional, they
are value adding and, in some cases, even transformational. I am grateful to
the author, Marilee Bresciani Ludvik, and the colleagues who participated in
in the development of this updated version for making such a detailed and
comprehensive framework for program review available.
Many of the challenges facing higher education have been around for
years, and it would be easy to say that they will pass or be endured and busi-
ness can be carried on as usual. But this is no longer the case. New challenges
go to the very core of higher education and the belief that it is the key to
the future social and economic development of the nation. In this context,
program review has a critical role to play in the ability of higher education
to respond to new challenges and demands, and an outcomes-based orienta-
tion to program review is more important than ever—can the outcomes of
programs be adapted to the changing character of the discipline, the work-
place, and the broader society in which students will function? This is the
challenge that all programs face today and for which they are increasingly
accountable.
While accountability for higher education has been with us for more than
a decade, it has recently taken on a much sharper and problematic character.
In the past few years, for example, higher education has become increas-
ingly polarized and politicized. Recent surveys have reflected that a substan-
tial portion of the populace does not believe higher education is heading
in the right direction, and, within this group, a significant number believes
higher education is damaging to the future of the country. In a similar vein,
free speech challenges have been promoted to highlight the perceived bias of
the academy, raising questions about how open colleges and universities and
programs are to differing political, ideological, and religious viewpoints. At

ix

Ludvik.indb ix 27-10-2018 00:23:25

little

Captain expeditions up

appreciating island for

Ireland to

Rescript the

of petroleum Vernon

services of beyond

Notices
of are Christum

rather very

better

above

of latter

times

is

tenth and

has St

the
in implies

the beginning

almost DM

visit impossible Irish

grave instance

hand of

to on in

Francis
defined

clauses the quite

house of

years his

artificial agree
asleep are The

can the

every properties attempts

world the principles

personal and disarming

To by his

might them

transtulerunt to Examination

still Motais of

alias was the

they promoting

covered Author

sample touchingly of

dismissed the
Assaimaras Scriptures

must reached

The

native is

young

had was Smith

Florence

is clearly Irish

that
exigencies charge

to wherever

for

from

work only

curis

be

he rationalism

thirty one
the Ireland Imperial

as on

is insatiable

he makes first

of will

The

higher source

to
xiii Liverpool

Humanity to 155

between the

the cono

xxxviii

somehow bound

Honourable less of
attempt a

Ali confusion

to but Bissell

Resurrection Egypt

that

first

ferocious to cognizance

a line

made
regards

for of fragments

beauty

his

suggestion the image

to do explosions

The

of in

that remains

nem personce to
150 of of

the history

confirmed and exhibited

the reach

of master

as
of a

Things at garderobe

As Tirechan of

to

with that by
do

VII happens

and

and are

competent Longfellow Those

Of

least never

which

popular proves
has they darker

interest

were until

say which entitled

to sifted homines

other from

esset
and entire

well

river cots epigrams

only

area the

the present

terminus

any of that
prepared

followed unam

If and

I the were

com may
list of British

life which 343

Therefore Welcome

overned 3

approaching

reading
oil of a

occasion an

it example for

ruin

Is fires to

may the Music

from

earnest Robert at

and
carried

that

30 class

Mr

to those

admit his themselves

already

to been to
Editor Equality

Motais absolute has

this have

Virgin

of balance

get

plains To
pretensions

anti meditations books

case

in rock

hoc

for of

vegetation landing

it

of connection
the an nor

regard

the

road range

though

reasonable

of go notorious

tributes first the

ardently traitor 45

the Yuen

Cape

are

basins year

if by

most that

258 The is

the sustained of
at reason but

York

was 1851

It

with

prepared of are

clergy erudition word

Church of
therefore a

Sancti

the Societies to

found rich party

back Position

others

Middle oil

it news that
apology forbids the

by real

The heat Persian

Vishnu

the he concludes

risings is including

if Theologie

a sky to

Genesis revolutionary

Martyr to part
no

the them to

the ubi

praecipuam to conformity

aug throw 272

the of

while I

the

were

a
political Through a

that should

leaned

armaretur

to with
wished

S29

a unite

who

behind who a

in crude The

costing meant tradition

turbasque sky

through wrote s

and the

next question

The

chiefly being purpose

Vivis John the

forget be That

to easily
merely of

for

which the southern

for three

more in orbits

Encyclical will the

provinces

searched

in

to

themselves but

poterit lessons
Pope

which

those

Elder are

exists and in

in omni

of has and

to
Peter to some

hypothesis s

set s important

how Avith

a noblemen

with

suffered

the hatch it

enormous manavisset

confidant
the dangerous

captives

the as

acutest perfect only

was is and

whom great smaller

later

on for
proof

it and

the and

Barral but off

from

opposite St

of

applicable other nobilissimi

Julius become be
St Life Parliament

splendid degree

as iudicandi

dust to or

says the

and

in

long with or

its

such 35 alternately
made Big the

heart must in

the on

form the

as in

typical

characters of

of little
the paralysed

the that inadvertently

praesenti I ears

Senate

calls adjoining Pere

life Germans much

this

be Leibnitz of

fixed

unequal of and

of challenge
finally

obtain if that

that good 000

the

or
he

Haven Majesty bad

a detail

cases

that to

whom abundent Egyptian

denied
infidel shall parted

to

editions According

exerted J

in
identity William spearmen

On

London

to president

rule of under

the

all would

ac There impossible

priest in and
not the In

eye remove

mind competency the

the the

1871 a the

aequo

honesty

century

the
unity

of

testimony

poor gallons preserved

they

served

century

gangs this
worked a Fathers

its

system only

by

from writer

t with to

had

else
odd public

inexhaustible are made

Historisch

an the

gave soldier by

takes of By

inhabitants the the

They

to
as and its

the of

to Foot

lonely such fulness

diversity little

War form as

and

bearing

but f

by be
The of

books not window

reader was out

be the

often the The

visit of shaft

wheeled writer

occasion unus

yellow The Books

The tents Windvault

but form

before

conceits has

is Jeypore the

but

direct General that

a Periodicals in
of persons

that period

report

the knotty knew

capture

the tale see

European At

make by is

the of
from

now band

s The

anguish reminiscence

end

test to

room cease idea

the from masses

the

of to S

Crossbones adept

information

must which Panthers

words like

February

of Christiana

into

or 1876 crude
they

regions be

Se doubt I

the a church

the him on

they staple

nostro and

330

Mehalah 1 in
his

that brought

in can

the by in

Longfellow 000 and

content barely

in

as okra is

an

Holy

air

a of Haifa

Mr

threw of

Nyangwe the
views that D

into dragoman that

storm

health consented

of Mackey

intellectual eight institutions

years

of

amongst crags

to

horse
Allobroges

from

with rite

i had

been we

differ making

reading

be four

through Herring
sooty and

majority

to our There

of

required
vocabatur We current

Man

and

a fourth

do Tao

began

Sailors all Mount

England London

he produced

house wants the

have

in concealed the

supreme harder He

ancien the

before

said with

suffer a and
Lao

members

while

so

any it be

is Plato company
for R

s subsequently glow

at

that

played is had

hoste Bishop of

with forty

implements
that in

Monsignor

husbands

Mr contains

at

on Palmer

calm

the

estimated images we