Become a Client

Access the best custom research to help hit your organization's goals. Set up your custom consult below in 60 seconds or less

  • This field is for validation purposes and should be left unchanged.
PUT THE POWER OF DATA See the stats TO WORK FOR YOUR ORGANIZATION

Become a Hanoverian

Join the second fastest growing market research firm worldwide and a top 50 fastest growing company in D.C. area – embark on a career with Hanover Research. Hanoverians hail from top educational institutions and firms in the U.S. and globally. Our staff have degrees ranging from bachelors to advanced masters and doctorate degrees in a variety of arts and sciences fields as well as professional experience in diverse industry areas. More than anything, we seek out candidates who approach questions with curiosity and precision and who delight in providing clients with high-quality, accurate, and objective work.
  • This field is for validation purposes and should be left unchanged.

Get the answers you need.

to general questions, press inquiries, references, citations & more. Just fill out the form below and we.ll get back to you within 48 hours or less

For inquiries relating to press and media coverage, reference and citations, content licensing, and other matters, please fill out the form below and we will contact you within 48 hours.
  • This field is for validation purposes and should be left unchanged.

View Our Fact Sheet SEE THE STATS: FILL OUT THE FORM FOR YOUR FREE FACT SHEET

Hanover

Law School Strategic Planning and Outcomes-Based Assessment

July 16, 2013 by Hanover Research


law

The importance of accreditation status to a law school’s success is clear and can therefore be a driving force behind strategic planning. The primary accrediting body in legal education, the American Bar Association (ABA), has recently entertained proposals to alter its accrediting standards. In October 2007, ABA’s Legal Education Section appointed a special subcommittee of the Standards Review Committee “to examine ways to revise the accreditation process to rely, to a greater extent than it currently does, on output measures.” The final report, released in 2008, was essentially pro‐assessment and resulted in numerous proposals to revise the ABA’s accreditation standards. The most recent proposal, presented at the Standards Review Committee Meeting in January 2013, reveals potentially dramatic changes to the third chapter on academic programming. A comparison of current standards to proposed standards reveals a new concentration on learning outcomes and assessments, as shown in the figure below.

With changes to ABA accreditation standards potentially looming, law schools have addressed outcomes‐based assessments in a variety of ways. Their primary focus remains on input factors such as attracting quality faculty and incoming students – unsurprising given the importance of student LSAT scores and faculty resources in the U.S. News and World Report law school rankings. However, it is also evident that the legal profession is increasingly demanding highly‐prepared graduates. Consider the comments of Phillip Bradley, senior vice president and general counsel for Duane Reade, who asserted that “law firms are developing core competencies they expect of their lawyers, but many law schools aren’t delivering graduates who come close to meeting them.” Law schools are clearly under pressure to use outcomes‐based assessments, but their processes for integrating these assessments into their programs of instruction are not highly developed.

General Approach

How can law schools do a better job of this? Professor Roy Stuckey, a prominent figure in legal education reform, recommends for law schools’ programs of instruction to articulate goals in terms of desired outcomes. Law schools seem to vary in how they express these “desired outcomes” in their strategic plans, though plans generally emphasize graduates’ career placement rather than development of skills and knowledge. For instance, it is common for schools to establish goals for alumni employment rates at predetermined times after graduation. In contrast, strategic plans typically refer to learning outcomes in broad terms and without specific measures of accountability.

Measuring Learning Outcomes

With law schools tending to be vague when addressing learning outcomes in their strategic plans, it therefore comes as no surprise that details are scant regarding how law schools actually measure their desired learning outcomes. Stuckey, however, provides a useful set of practices for assessing student learning. He stresses that effective assessments exhibit qualities of validity, reliability, and fairness. Stuckey’s recommendations are as follows:

  • Be clear about goals of each assessment
  • Assess whether students learn what is taught (validity)
  • Conduct criteria‐referenced assessments, not norm‐referenced (reliability)
  • Use assessments to inform students of their level of professional development
  • Be sure assessment is feasible
  • Use multiple methods of assessing student learning
  • Distinguish between formative and summative assessments
  • Conduct formative assessments throughout the term
  • Conduct multiple summative assessments throughout the term, when possible
  • Ensure that summative assessments are also formative assessments
  • Require students to compile educational portfolios

Assessment methods can be qualitative and/or quantitative, but oftentimes the data collection process can be embedded into the course so that there is little extra effort involved. Examples of assessment instruments identified by Gregory Munro, Director of Professional Skills at the University of Montana School of Law, include:

  • Interviews
  • Statistical indicators
  • Performance appraisals
  • Faculty teaching portfolios
  • Employer reports
  • External examiners
  • Questionnaires and surveys
  • Examinations and papers
  • Student portfolios
  • Alumni follow‐up reports
  • Student self‐assessment

Law schools typically do not address specific assessment methods in their strategic plans, likely because ABA standards (or proposed standards) do not explicitly require the use of any particular measure or instrument. However, this is not to say that assessment of student learning does not play an integral role in the strategic planning of institutions. The specific methods of outcomes‐based assessment may vary between schools, but the key element is that results are used to evaluate effectiveness and to shape school strategy, planning, and policy moving forward.

Implementation Efforts

Once a law school’s strategic plan is finalized, the primary challenge becomes implementing the changes. Many strategic plans include “action items” or “indicators of progress” to ensure accountability. Furthermore, some schools are proactive in reporting progress to their respective communities in the form of annual reports. Such practices are consistent with Stuckey’s recommendation to create a feedback loop. In a feedback loop model, the school collects data on student outcomes, disseminates the information to stakeholders (faculty, administration, etc.), and utilizes the information to evaluate effectiveness and make changes if necessary.  This model is particularly useful for implementing outcomes-based assessments, a process that demands thoughtful strategy, faculty onboarding, and certain institutional support structures.

In a presentation at the 2009 Legal Education at the Crossroads conference, a trio of faculty from Hamline University School of Law explained that there are two broad strategies for incorporating outcomes‐based assessments into school curriculum and operations. The first is a bottomup approach that begins with a small group of professors implementing outcomes assessments at the individual course level. After these pilot programs identify success factors, the system is gradually expanded throughout the institution. One obvious advantage of this process is that not as many stakeholders are involved from the outset, before the process is perfected. The pilot‐teachers can be selected through a self-nomination process, and are therefore likely to be “true believers” in the assessment philosophy.

A disadvantage to the bottom‐up approach is the potential risk of isolating teachers and the inability to provide dependable resources. In contrast, the topdown approach requires an institution‐wide commitment. Outcomes assessments are rolled out by central leadership and filtered down to the course level. Hamline Law ultimately decided to use the top‐down approach, in part because the institution felt that the bottom‐up approach meant delaying the inevitable – campus‐wide installment.

Regardless of strategy, implementing a system of outcomes‐based assessment requires an element of faculty onboarding. Lori Roberts, a director at Western State University and supporter of the aforementioned ABA proposals, explains that faculty resistance to the practice of outcomes assessment can significantly hinder the implementation process:

“Unfortunately, assessment is often not conducted because educators are understandably resistant and defensive when it comes to assessing themselves. Fear in realizing the results of assessment, time and resource constraints, and the perception that imperfect assessment is not valuable can interfere with legal educators’ internal motivations to improve student learning and the external motivations of the law schools’ accrediting agencies. These realities prevent legal educators from buying into assessment and enhance resistance to the process.”

In response, Roberts advises administrators to emphasize that assessments are being used to evaluate the institution, not the faculty therein. Additional strategies for encouraging faculty acceptance of outcomes‐based assessments, as described by the Hamline Law presenters, include:

  • Creating a new committee to coordinate the implementation of learning outcome assessment objectives
  • Including learning outcome assessment information in faculty recruitment efforts
  • Encouraging faculty and staff to report their learning outcome assessment‐related activities as part of their annual reviews
  • Reconfiguring the law school schedule for faculty colloquia focused on, among other issues, learning outcome assessment related discussions
  • Adopting principles for faculty and staff evaluations to ensure the appropriate use of assessment data