Access the best custom research to help hit your organization's goals. Set up your custom consult below in 60 seconds or less
to general questions, press inquiries, references, citations & more. Just fill out the form below and we.ll get back to you within 48 hours or less
July 16, 2013 by Hanover Research
The importance of accreditation status to a law school’s success is clear and can therefore be a driving force behind strategic planning. The primary accrediting body in legal education, the American Bar Association (ABA), has recently entertained proposals to alter its accrediting standards. In October 2007, ABA’s Legal Education Section appointed a special subcommittee of the Standards Review Committee “to examine ways to revise the accreditation process to rely, to a greater extent than it currently does, on output measures.” The final report, released in 2008, was essentially pro‐assessment and resulted in numerous proposals to revise the ABA’s accreditation standards. The most recent proposal, presented at the Standards Review Committee Meeting in January 2013, reveals potentially dramatic changes to the third chapter on academic programming. A comparison of current standards to proposed standards reveals a new concentration on learning outcomes and assessments, as shown in the figure below.
With changes to ABA accreditation standards potentially looming, law schools have addressed outcomes‐based assessments in a variety of ways. Their primary focus remains on input factors such as attracting quality faculty and incoming students – unsurprising given the importance of student LSAT scores and faculty resources in the U.S. News and World Report law school rankings. However, it is also evident that the legal profession is increasingly demanding highly‐prepared graduates. Consider the comments of Phillip Bradley, senior vice president and general counsel for Duane Reade, who asserted that “law firms are developing core competencies they expect of their lawyers, but many law schools aren’t delivering graduates who come close to meeting them.” Law schools are clearly under pressure to use outcomes‐based assessments, but their processes for integrating these assessments into their programs of instruction are not highly developed.
How can law schools do a better job of this? Professor Roy Stuckey, a prominent figure in legal education reform, recommends for law schools’ programs of instruction to articulate goals in terms of desired outcomes. Law schools seem to vary in how they express these “desired outcomes” in their strategic plans, though plans generally emphasize graduates’ career placement rather than development of skills and knowledge. For instance, it is common for schools to establish goals for alumni employment rates at predetermined times after graduation. In contrast, strategic plans typically refer to learning outcomes in broad terms and without specific measures of accountability.
With law schools tending to be vague when addressing learning outcomes in their strategic plans, it therefore comes as no surprise that details are scant regarding how law schools actually measure their desired learning outcomes. Stuckey, however, provides a useful set of practices for assessing student learning. He stresses that effective assessments exhibit qualities of validity, reliability, and fairness. Stuckey’s recommendations are as follows:
Assessment methods can be qualitative and/or quantitative, but oftentimes the data collection process can be embedded into the course so that there is little extra effort involved. Examples of assessment instruments identified by Gregory Munro, Director of Professional Skills at the University of Montana School of Law, include:
Law schools typically do not address specific assessment methods in their strategic plans, likely because ABA standards (or proposed standards) do not explicitly require the use of any particular measure or instrument. However, this is not to say that assessment of student learning does not play an integral role in the strategic planning of institutions. The specific methods of outcomes‐based assessment may vary between schools, but the key element is that results are used to evaluate effectiveness and to shape school strategy, planning, and policy moving forward.
Once a law school’s strategic plan is finalized, the primary challenge becomes implementing the changes. Many strategic plans include “action items” or “indicators of progress” to ensure accountability. Furthermore, some schools are proactive in reporting progress to their respective communities in the form of annual reports. Such practices are consistent with Stuckey’s recommendation to create a feedback loop. In a feedback loop model, the school collects data on student outcomes, disseminates the information to stakeholders (faculty, administration, etc.), and utilizes the information to evaluate effectiveness and make changes if necessary. This model is particularly useful for implementing outcomes-based assessments, a process that demands thoughtful strategy, faculty onboarding, and certain institutional support structures.
In a presentation at the 2009 Legal Education at the Crossroads conference, a trio of faculty from Hamline University School of Law explained that there are two broad strategies for incorporating outcomes‐based assessments into school curriculum and operations. The first is a bottom‐up approach that begins with a small group of professors implementing outcomes assessments at the individual course level. After these pilot programs identify success factors, the system is gradually expanded throughout the institution. One obvious advantage of this process is that not as many stakeholders are involved from the outset, before the process is perfected. The pilot‐teachers can be selected through a self-nomination process, and are therefore likely to be “true believers” in the assessment philosophy.
A disadvantage to the bottom‐up approach is the potential risk of isolating teachers and the inability to provide dependable resources. In contrast, the top‐down approach requires an institution‐wide commitment. Outcomes assessments are rolled out by central leadership and filtered down to the course level. Hamline Law ultimately decided to use the top‐down approach, in part because the institution felt that the bottom‐up approach meant delaying the inevitable – campus‐wide installment.
Regardless of strategy, implementing a system of outcomes‐based assessment requires an element of faculty onboarding. Lori Roberts, a director at Western State University and supporter of the aforementioned ABA proposals, explains that faculty resistance to the practice of outcomes assessment can significantly hinder the implementation process:
“Unfortunately, assessment is often not conducted because educators are understandably resistant and defensive when it comes to assessing themselves. Fear in realizing the results of assessment, time and resource constraints, and the perception that imperfect assessment is not valuable can interfere with legal educators’ internal motivations to improve student learning and the external motivations of the law schools’ accrediting agencies. These realities prevent legal educators from buying into assessment and enhance resistance to the process.”
In response, Roberts advises administrators to emphasize that assessments are being used to evaluate the institution, not the faculty therein. Additional strategies for encouraging faculty acceptance of outcomes‐based assessments, as described by the Hamline Law presenters, include: