Friday, January 15, 2010

9 Principles of Good Practice for Assessing Student Learning

Developed by Alexander W. Astin; Trudy W. Banta; K. Patricia Cross; Elaine El-Khawas; Peter T. Ewell; Pat Hutchings; Theodore J. Marchese; Kay M. McClenney; Marcia Mentkowski; Margaret A. Miller; E. Thomas Moran; Barbara D. Wright

This document was developed under the auspices of the AAHE Assessment Forum with support from the Fund for the Improvement of Postsecondary Education with additional support for publication and dissemination from the Exxon Education Foundation. Copies may be made without restriction.

1. The assessment of student learning begins with educational values. Assessment is not an end in itself but a vehicle for educational improvement. Its effective practice, then, begins with and enacts a vision of the kinds of learning we most value for students and strive to help them achieve. Educational values should drive not only what we choose to assess but also how we do so. Where questions about educational mission and values are skipped over, assessment threatens to be an exercise in measuring what's easy, rather than a process of improving what we really care about.
2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time. Learning is a complex process. It entails not only what students know but what they can do with what they know; it involves not only knowledge and abilities but values, attitudes, and habits of mind that affect both academic success and performance beyond the classroom. Assessment should reflect these understandings by employing a diverse array of methods, including those that call for actual performance, using them over time so as to reveal change, growth, and increasing degrees of integration. Such an approach aims for a more complete and accurate picture of learning, and therefore firmer bases for improving our students' educational experience.
3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. Assessment is a goal-oriented process. It entails comparing educational performance with educational purposes and expectations -- those derived from the institution's mission, from faculty intentions in program and course design, and from knowledge of students' own goals. Where program purposes lack specificity or agreement, assessment as a process pushes a campus toward clarity about where to aim and what standards to apply; assessment also prompts attention to where and how program goals will be taught and learned. Clear, shared, implementable goals are the cornerstone for assessment that is focused and useful.
4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes. Information about outcomes is of high importance; where students "end up" matters greatly. But to improve outcomes, we need to know about student experience along the way -- about the curricula, teaching, and kind of student effort that lead to particular outcomes. Assessment can help us understand which students learn best under what conditions; with such knowledge comes the capacity to improve the whole of their learning.
5. Assessment works best when it is ongoing not episodic. Assessment is a process whose power is cumulative. Though isolated, "one-shot" assessment can be better than none, improvement is best fostered when assessment entails a linked series of activities undertaken over time. This may mean tracking the process of individual students, or of cohorts of students; it may mean collecting the same examples of student performance or using the same instrument semester after semester. The point is to monitor progress toward intended goals in a spirit of continuous improvement. Along the way, the assessment process itself should be evaluated and refined in light of emerging insights.
6. Assessment fosters wider improvement when representatives from across the educational community are involved. Student learning is a campus-wide responsibility, and assessment is a way of enacting that responsibility. Thus, while assessment efforts may start small, the aim over time is to involve people from across the educational community. Faculty play an especially important role, but assessment's questions can't be fully addressed without participation by student-affairs educators, librarians, administrators, and students. Assessment may also involve individuals from beyond the campus (alumni/ae, trustees, employers) whose experience can enrich the sense of appropriate aims and standards for learning. Thus understood, assessment is not a task for small groups of experts but a collaborative activity; its aim is wider, better-informed attention to student learning by all parties with a stake in its improvement.
7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about. Assessment recognizes the value of information in the process of improvement. But to be useful, information must be connected to issues or questions that people really care about. This implies assessment approaches that produce evidence that relevant parties will find credible, suggestive, and applicable to decisions that need to be made. It means thinking in advance about how the information will be used, and by whom. The point of assessment is not to gather data and return "results"; it is a process that starts with the questions of decision-makers, that involves them in the gathering and interpreting of data, and that informs and helps guide continuous improvement.
8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change. Assessment alone changes little. Its greatest contribution comes on campuses where the quality of teaching and learning is visibly valued and worked at. On such campuses, the push to improve educational performance is a visible and primary goal of leadership; improving the quality of undergraduate education is central to the institution's planning, budgeting, and personnel decisions. On such campuses, information about learning outcomes is seen as an integral part of decision making, and avidly sought.
9. Through assessment, educators meet responsibilities to students and to the public. There is a compelling public stake in education. As educators, we have a responsibility to the publics that support or depend on us to provide information about the ways in which our students meet goals and expectations. But that responsibility goes beyond the reporting of such information; our deeper obligation -- to ourselves, our students, and society -- is to improve. Those to whom educators are accountable have a corresponding obligation to support such attempts at improvement.

To request more information about the Assessment Forum, contact:
One Dupont Circle, Suite 360 Washington DC 20036-1143 Phone 202/293-6440 Fax 202/293-0073 info@aahe.org © 2003 American Association for Higher Education

Friday, December 11, 2009

Reference Class Learning Reflections

Introduction

In the last class session of the semester, my discussion group responded to the question, “A GSLIS student is thinking about taking this course and wants to know what to expect. What are the three most notable things they will learn or should expect from the class?” My classmates agreed with these suggested answers: 1. How to conduct the reference interview; 2. How to use excellent reference sources in print and online; and 3. Search strategies for best results. That summary encapsulates my own learning experience this semester.

Conducting the Reference Interview

The reference interview process follows many principles I had previously learned, that were refreshed by this course. I freely admit my bias but, in my mind, librarianship begins at the reference desk. This is why I signed up for this course early in my curriculum plan.

Our text is truly “a how-to-do-it manual for librarians”, as the publisher’s trademark states at the top of the cover. I plan to use this manual throughout my library career and refer back to it frequently. It is chock-full of examples, detailed instruction and step-by-step approaches on how to respond to patrons in reference transactions. I have already been putting that learning to use, but obviously more practice is needed to get it right.

The Readers’ Advisory situations are especially difficult for me. Upon hearing my first RA question in the library, I froze. I didn’t know the author, the genre or anything else, I was totally lost, and forgot to ask questions to get the patron to tell me those things. Instead, I thought maybe I should access a reference source (Novelist) and try to respond. By the time I got the application open and entered the author’s name, the patron lost interest in waiting for the “rookie librarian” to figure it out, and had moved on. Failure, in my first opportunity! Following the suggestions of Joyce Saricks, quoted in our text, I have reminded myself since this experience to be a better listener and get a conversation going with the patron that reveals interests. Saricks calls it having a fun conversation about books. What could be better than that? My next RA opportunities yielded better results.
Reference Sources

The source assignments were exceptionally well crafted to force me to use some sources I had never used before. While I retain a healthy admiration for Google and Wikipedia, I have learned that they are rarely the “perfect” sources for serious information-seeking. Learning about MedlinePlus, the Statistical Abstract for the United States, Books in Print, the Gale Learning Center resources has been a significant milestone for me. These are sources I had never used, certainly not online anyway, and this was a major revelation that so much detailed information is at our fingertips, on authoritative databases.

The assignments also took me to the physical collection in a way I had never experienced. I discovered new encyclopedias, such as the Gale medical encyclopedias, which were completely unknown to me previously. Also, I learned for the first time how to use certain Quotations and Dictionary sources, like the McGraw-Hill Dictionary of Scientific and Technical terms. The diversity of encyclopedias was also a new experience. I had used large sets, like Britannica and Americana. Through the assignments, however, I was introduced to single or fewer volume encyclopedias that provided excellent articles on specific topics, such as The New Century Cyclopedia of Names, The Concise Encyclopedia of Psychology, The Columbia Encyclopedia and several others including the print editions of some of the online sources mentioned in the previous paragraph.

The geographic sources were used in a more in-depth manner than I had ever experienced. In particular, I found the CIA Factbook and The Columbia Gazetteer very useful.

Search Strategies

A third key lesson for me this semester was learning how to combine terms to make a database search effective and efficient. Through the Campbell and Fyfe article, I also learned that teaching others how to use the available information tools goes hand-in-hand with reference desk work. Knowing how to search is good, but knowing how to teach others how to search is even better.

This class has helped me to understand the importance of teaching during sort interactions, like using the OPAC or a database to find information. This is the very point of taking this class. For me, it is all about helping others become “information literate”. The information is available, but only if one knows how to retrieve it.

Further, I have learned to keep searching until the question is completely answered. An additional source may provide information not found in the first source. “Special topic” sources may give insights not available elsewhere, but still not contain the broad view of a more general source. At times, both are needed. It is also important to keep searching and verify the information in multiple sources in order to avoid biased or dated information.

Conclusion

Articles and statements from several contributing authors of the many works assigned for reading this semester, such as Catherine Ross and Marie Radford, have encouraged me that the future remains bright for reference librarians. They have eloquently articulated the point that information seekers will continue to need assistance and that the librarians who learn the basic techniques and skills and who are open to adopting new technology and methods, will be preeminently positioned to offer that assistance. It is exciting to hear experienced veterans express such enthusiasm for the profession and instills in me a deeper desire to be a part of it. I feel that my decision to take this reference class at the start of my study of Library and Information Sciences has been validated as an excellent choice, and I would encourage others with similar goals to do the same. In the spirit of Billie Walker at Penn State University (Radford article), I expect to enjoy being a “library dude” on campus someday!


Cited Works:

Ross, Catherine Sheldrick, Kristi Nilsen and Dewdney. Conducting the Reference Interview: A How-To-Do-It Manual for Librarians. New York: Neal-Schuman, 2009.

Campbell, Susan and Debbie Fyfe. “Teaching at the Computer: Best practices for One-on-One Instructions in Reference.” Feliciter 1 (2002): 26-8

Radford, Marie L. “A Personal Choice: Reference Service Excellence.” Reference & User Services Quarterly 48:2 (Winter 2008): 108-15.

Thursday, December 10, 2009

Final Project

Group Assignment: Your information organization has received a grant to develop a small collection. All of the items in the collection will be of a single genre (Rare Books in our case). You will post the URL containing your report that explores the important issues and approaches to dealing with this collection.

And, here it is: The Corporis Humani Digital Collection

.

Friday, November 20, 2009

Models and Standards for Authority Control

Models and standards work well when agreement is reached on which models and which standards to use for an activity or process. An additional requirement for enabling models and standards to meet the objectives of their design is that those working within the models should also adhere to the standards. This paper will synthesize the concepts found in the text with the comments from two information researchers about metadata quality and the problems of not adhering to standards.
In recent weeks, our class discussion focused on the description of information resources using metadata that allows for retrieval of data related to names, titles and subjects. Further, in order for relevant, or the best, information resources to be retrieved, the metadata and the retrieval system have to be subject to the same description process and authority control. As the vast volumes of information resources expanded, it became necessary to develop models and standards for this authority control.
Over time, superior models have been adopted and eventually accepted by the professional community of information scientists, managers and users. So that today, we have the general bibliographic models of Functional Requirements for Bibliographic Records (FRBR) and the Functional Requirements for Authority Data (FRAD). These two models outline the way that resource titles, their associated names, and associated subjects are related to one another and to other resources and also establish the way that those relationships are organized, categorized and classified. Like the models, the standards have also experienced lengthy development over time by way of constant improvement and upgrading until today several general standards exist. They are: Anglo-American Cataloguing Rules, Second Edition, 2002 Revision (AACR2); Dublin Core Agents, Metadata Authority Description Schema (MADS) and the soon to be launched Resource Description and Access (RDA) and the Statement of International Cataloguing Principles. These models and standards have been established, among other reasons, in order to improve metadata quality.
Many studies (cited in the Park and Salo articles) have shown that following these models and standards make retrieval of information, especially about bibliographic resources, possible and more accurate than if there were no standard methods of organization and classification. When the models and standards are followed, metadata quality is enhanced and information seekers have greater success in their information searches as more relevant information and greater amounts of information are found.
Examples of information sources that have been lacking in metadata quality and thus have led to ineffective searching for users are digital repositories and institutional repositories. Jung-ran Park identifies a number of studies on metadata quality that identify principles of “good metadata” and several other studies that help to identify the problems with metadata in digital repositories.
According to the National Information Standards Organization (NISO) there are six principles of good metadata, which are found on pg. 215 of the Park article. These principles are heavily related to standards and authority control. Obviously, following a standard makes the retrieval of data more successful because the standard has been intelligently established over time and will lead to better results in a search than if the data is organized, classified and indexed in a haphazard way. The essential key is to get submitters of data and the catalogers of data to follow the prescribed standards consistently and avoid making errors.
As mentioned, Park cites studies that uncover the problems with metadata quality in repositories. In summation of the many different problem criteria cited by the studies, Park simplifies our thinking on the subject by consolidating them into three common criteria: completeness, accuracy and consistency. She explains that by completeness she means that the metadata is complete enough to facilitate its purpose of making the resource to which it refers fully accessible in a search. Accuracy means correct entry of metadata and avoidance of mismatched metadata that is imported from data providers. She calls for consistency to be measured on two levels, the conceptual or semantic level and looking at the data format on the structural level.
Dorothea Salo, in her article “Name Authority Control in Institutional Repositories”, highlights these same problem criteria by showing how the methodology designed for the institutional repository did not protect against them. She may not use the same words, but she is emphasizing throughout her comments the problems with incomplete, inaccurate and inconsistent metadata.
According to Salo, the factors causing the problem in institutional repositories include the process of self-archiving by submitters of resources, ingesting material possessing no metadata control, and harvesting materials from aggregators and e-indexers which possess inconsistent metadata.
Obviously, Park’s recommendations for the improvement of repositories are to establish methods to improve the completeness, accuracy and consistency of metadata. Her recommendations include establishing metadata guidelines as a best practice for information organizations. This suggestion also applies to mechanized or automatic generation of metadata, leading into the further development of the semantic web technology. She cites several projects currently underway towards this objective. While Salo mentions several barriers to solutions to these problems with metadata inconsistencies, she does recommend finding a method to pass corrected metadata around between generators and managers of metadata. (I might add even involving users.) Salo suggests that use of the Open Archives Initiative and JISC are two paths to pursue vigorously.
Finding solutions to these problems is the challenge for the current generation of information scientists. Launching RDA and developing the Semantic Web will no doubt be the main focus for the near term. Establishing guidelines as best practices is one way, but this only gets results when others follow along. Development of the semantic web provides hope on the horizon, but again, information professionals engaged in the organization, cataloging and indexing of information resources must individually contribute to the completeness, accuracy and consistency of metadata in every aspect of their jobs in order for models and standards to take hold and take full effect. It will take personal commitment and engagement of all concerned.
Resources:
Taylor & Joudrey, Chapter 8. Metadata: Access and authority control (pp. 245-301).
Park, J.-R. (2009). Metadata quality in digital repositories: A survey of the current state of the art. Cataloging & Classification Quarterly 47(3/4): 213-228.
Salo, D. (2009). Name authority control in institutional repositories. Cataloging & Classification Quarterly 47(3/4): 249-261.

Wednesday, November 11, 2009

Reader's Advisory Services

In the Catherine Ross text on Conducting the Reference Interview (pg. 237), Joyce Sarick is quoted as having said, “There are fifty thousand answers to the request, ‘Give me a good book’.” Sarick further suggests that the readers advisory interview requires some knowledge development activity before the patron ever arrives. I suppose in the past this was done by reading a lot of books, book commentaries, reviews, etc.. It seems to me that today, this process of knowledge development requires the same activity of reading like it did in the past, although today, for the most part, it is done by using electronic resources. By becoming familiar with the many different electronic resources, a librarian can understand the strengths and weaknesses of each and deftly use the right resource when the requests are made. In this regard, reader’s advisory is exactly the same as the reference advisory, in that knowing the best resource for the real information need makes the advisory process so much more efficient and efficacious.

Sarick goes on to describe the reader’s advisory interview as having a fun conversation about books, primarily by getting the patron to do most of the talking and the interviewer doing some careful listening. This approach makes the process seem much less daunting and after all, aren’t most librarians and information specialists in the field exactly because they enjoy having this kind of conversation about books? What a perfectly appealing way to describe it.

Another key point for me in this weeks’ reading was the cautionary advice to refrain from falling into the trap of responding with one’s own opinions about a book or an author. Although the question sometimes comes out that way, the requester is not really asking for the librarian’s opinion on another “good author”, “good book” or “good read”. What the librarian might like may not be what the requester will like. Staying focused on finding out what the patron likes came across to me as very sound and key advice.