Assessment Proposal: A Partnership with Dark Horse Comics Emporia State University Betsy Summers

Assessment Proposal: A Partnership with Dark Horse Comics Program Overview The Ledding Library conducted a thorough community analysis in April 2010. The data collected from that analysis indicate two significant facts that are highlighted in this paper: 1) boys ages 13-18 make up the lowest statistical percentage (9 %) of any group of library users, and 2) the library is located near many businesses, offering the opportunity for partnerships. Grover, Greer, and Agada (2008) define assertive service as "applying collected data about a community to address[ing] the needs of the clientele" (p. 174). The Youth Services department, along with the Teen Advisory Board, decided to use assertive service by creating a new program that both attracted teenaged boys and fostered a partnership with Dark Horse Comics, a nearby business. Local Milwaukie resident Mike Richardson founded Dark Horse Comics in 1986, and since then, it has "transformed the creative landscape in the Pacific Northwest, turning the Portland area into the country's largest community of cartoonists and graphic artists outside of New York" (Oregon Encyclopedia, 2011). The Youth Services department saw a unique opportunity to create new programming, called Draw Me Graphic, a graphic arts class aimed at teenage boys. A partnership was formed with Dark Horse Comics, and classes began in September. Assessment Questions The overall objective in creating a partnership with Dark Horse Comics was to bring more teenaged boys into the library. Therefore, the questions used to assess this new program were as follows: 1) Did the Draw Me Graphic classes actually draw in more teenaged boys to the library? 2) Did the boys enjoy coming to the library and participating in the Draw Me Graphic classes and would they participate again? 3) How could we improve on the Draw Me Graphic class? There were a number of other questions that could have been asked, such as have the teenage boys participating in the Draw Me Graphic classes been interested in using other library programs and services, or was there a seen positive influence on the behavior of the boys, outside of the library, while engaged in the program? Because of 2

Assessment Proposal: A Partnership with Dark Horse Comics the complexity involved in arriving at those answers, this paper will not deal with those questions. Methodology Used Once the questions were chosen, a research methodology was needed to help find the answers. Because of the small sample size in this study, the methodology could be qualitative. "Qualitative research data are derived from interviews, observations, or in-depth case studies" (Greenblatt, Dickinson, & Simpson, p. 75). At the same time, by including observations in the study, a quantitative measurement would be used to assess whether the library was in fact attracting more teenage boys. By applying three different types of methodology, the library was able to get a truer representative picture of the attendance numbers, the quality of programming, and the overall interest in the library's services. Beck and Manual (2008) state that triangulated "findings from each data collection group . . . provide a well-rounded view of the research" (p. 93). The three methods used for data collection included: 1) Observation. The Youth Services staff used this method at the time the class was held. The technique implemented was a simple ticking off of attendance numbers, bystanders, and also age range and gender. 2) Focus Group. This method was conducted near the conclusion of each of the 6-week session of classes, asking for overall impressions and comments about the program. 3) Survey. A brief SurveyMonkey questionnaire was sent out via email to each of the participants, as well as their Dark Horse Comic instructors who taught the Draw Me Graphic classes. The first method used observation to quantitatively collect data about the Draw Me Graphic classes. An online sign-up was utilized to limit the number of participants in each session to ten, which also offered specific information about who was supposed to participate in the class. The class met for six consecutive Thursdays from 3 - 5 p.m., and was under the constant observation of the Youth Services staff on duty during those times. The observation included a broad scan of the Youth Services area. Staff simply 3

Assessment Proposal: A Partnership with Dark Horse Comics put check marks which indicated a patron next to the appropriate boxes: male or female; under or over 15 years of age; and, whether they were participating in the class or using other services. This gave a more detailed picture of who was doing what in the library during those times. The second method used was the focus group, which helped in the discovery of the thoughts and feelings of the Draw Me Graphic participants. At the very beginning of the last class held during the 6-week session, students were asked to sit through a brief focus group discussion. The questions asked were as follows: 1) Have you enjoyed the classes, so far? 2) What is the best thing about the classes? 3) If you could change one thing about the class, what would it be? 4) Would you participate in the class again? 5) Have you learned a lot about drawing that you didn't know? 6) Would you tell a friend about the class? The focus group was held in an informal setting, was designed to be interactive, and to get the students talking freely. The instructors in the class were also invited to participate and share their thoughts. The youth librarian, or other Youth Services staff on duty during that time, helped to record the answers. The last method employed was the survey. A SurveyMonkey questionnaire was sent out by email address to each participant, as well as the Dark Horse instructors. They were asked to reply to the following questions: 1) How satisfied were you with the Draw Me Graphic class? Extremely satisfied Very satisfied Somewhat satisfied Slightly satisfied Not at all satisfied

Assessment Proposal: A Partnership with Dark Horse Comics 2) How would you rate the library in terms of overall support for the Draw Me Graphic classes? Extremely supportive Very supportive Somewhat supportive Slightly supportive Not at all supportive 3) If you could change one thing about the Draw Me Graphic class, what would it be? Please be specific.

4) Do you think it's likely that you'd participate in a Draw Me Graphic class again? Extremely likely Very likely Somewhat likely Slightly likely Not at all likely 5) Do you think it's likely you would tell a friend about the library's Draw Me Graphic class? Extremely likely Very likely Somewhat likely Slightly likely Not at all likely 6) What was your favorite part of the Draw Me Graphic class? Please, be as specific as possible.

Assessment Proposal: A Partnership with Dark Horse Comics Methodology Pros and Cons The three types of methodology used in assessing the Draw Me Graphic classes might be known as "convenience sampling" which is defined by Beck and Manual (2008) as "using what is available" (p. 85). It is easy to conduct and therefore prone to bias; however, Youth Services can be "more confident with [the] result if different methods lead to that same result" (p. 85). Choosing to apply a triangulation of methodologies has helped to create a more complete picture of the class and thoughts of its participants. The observation methodology in this assessment was used to corroborate the online sign-up sheet with the actually participation of students. Because this type of quantitative data collection cannot discern "what users think or feel about [the] library['s] services or resources" (Beck & Manual, p. 117), it was important to also conduct methodology testing to analyze the cognitive aspects of what the teenagers were thinking. Using a focus group is a concrete way to get genuine, direct feedback from the teen participants enrolled in the Draw Me Graphic classes. "In a usability study, a focus group helps researchers and designers understand users' reactions" (Beck & Manual, p. 125). The Raby (2010) study of focus groups using teenagers, cites a number of compelling reasons to use them: Although focus groups are staged social events, they nonetheless are an attempt to capture the way meaning is made and negotiated: Participants explain and defend their positions (Warr, 2005), challenge others comments (Hyde, Hewlett, Brady & Drennan, 2005) and share excitement (Vaughn et al., 1996), teasing, and joking (Kitzinger, 1994). Kitzinger particularly values turning points, when participants change their minds through group interaction. (p. 2) The last piece in the triangulation of methodologies was the use of the survey. Grover et al. (2008) indicate, "questionnaires are appropriate for uses in certain situations, and a waste of time in others" (p. 69). Because of the small sampling, and the specificity of the SurveyMonkey questionnaire, it was thought that valuable information 6

Assessment Proposal: A Partnership with Dark Horse Comics could be collected. In a recent reliability study on surveys, researchers found that "participants may differ in interest and awareness throughout long questionnaires, which can affect reliability of their answers" (Wentzel-Larsen, Norekval, Ulvik, Nygard, & Pripp, 2011). Therefore, it was important to make the questionnaire as brief as possible. Arlene Fink, author of The Survey Instrument, said that surveys are used to "find out about people by asking questions about feelings, motivations, plans, beliefs and personal backgrounds" (Beck & Manual, p. 28). The SurveyMonkey questionnaire tried very hard, in a very simple way, to find out those exact determinants. Application of the Data While the triangulation of methodologies strengthened the assessment study of the Draw Me Graphic classes, it is important that a practical application of the data be employed. In the article Evidence Based Practice: Science? Or Art?, Koufogiannakis (2011) states that librarians need to recognize the science as well as the art of applying collected data to library programs and services. She emphatically states: Without the art side of the equation, meaning and context may easily be lost. Someone can gather all the best evidence, but not know how to effectively implement it. Or if you are overly reliant on perfect evidence before you make a decision that is in the best interest of your community, you will likely lag behind and not serve peoples needs. No research study, no matter how perfectly conducted is going to provide all the answers for what we do in practice. (p. 2) The actual numbers derived from the assessment of Draw Me Graphic classes are remarkable. Since its inception in September 2010, the library has seen a 54 percent increase in usage by boys ages 13-18. Youth Services concluded this through the tally of actual attendance numbers and observation gathering methods, compared to that of the community analysis which showed a 9 percent usage by boys ages 13-18. The focus group responses and the SurveyMonkey questionnaire reflected nothing but positive feedback about the classes. Both instructors and students agreed that the class was educational and enjoyable. The number of participants 76 in the 8 classes offered so 7

Assessment Proposal: A Partnership with Dark Horse Comics far is quantitative evidence in itself, and indicates how a successful partnership with a local business can increase usership of the library. This extraordinary increase in the number of boys using the library is significant, but perhaps as importantly, Youth Services must provide their findings to the library's administration and Board. According to Wilson (2010), "presenting evidence either self-generated or found in the literature at library meetings or other internal venues can be the spark needed to push new ideas forward, and can contribute to an evidence based culture shift within the organization" (p. 100). It is also necessary to engage those boys who are actively participating in the Draw Me Graphic classes by inviting them to be on the Teen Advisory Board, asking them for their ideas, and helping them understand that they can influence the library's services and programming. Conclusion The Draw Me Graphic classes and partnership with Dark Horse Comics has been an overwhelming success. The community analysis diagnosed the needs of Milwaukie; Youth Services assertive service created a treatment. The desired outcomes of increasing the usage of the library by teenage boys and creating a partnership with a nearby business were both achieved. The partnership with Dark Horse Comics will continue into the future, as programming expands to cater more to this underserved population group. Representatives from Dark Horse Comics have expressed an interest in starting up new classes, aimed at 9-12 year olds. The Youth Services department will strive to seek the opinions of youngsters, while continuing to offer quality programming and look to form new, exciting partnerships with the community's businesses.

Assessment Proposal: A Partnership with Dark Horse Comics References Beck, S. E., & Manuel, K. (2008). Practical research methods for librarians and information professionals. New York, NY: Neal-Schuman. Greenblatt, M., Dickinson, G., & Simpson, C. (2004). Implementing the research question. Knowledge Quest, 33(2). Grover, R. J., Greer, R. C., & Agada, J. (2010). Assessing information needs: managing transformative library services. Santa Barbara, CA: Libraries Unlimited. Koufogiannakis, D. (2011). Evidence based practice: Science? or art?. Evidence Based Library and Information Practice, 6(1). The Oregon Encyclopedia (2011). Dark Horse Comics. http://www.oregonencyclopedia.org/entry/view/dark_horse_comics/ (accessed November 17, 2011). Quinones, V., Chew, I., Bon I., & Muller, P. (1996). Guidelines for library services for young adults. International Federation of Library Associations and institutions (IFLA) Professional Report. PDF file. Raby, R. (2010). Public selves, inequality, and interruptions: The creation of meaning in focus groups with teens. International Journal of Qualitative Methods, 9 (1). 1-15. SurveyMonkey (2011). www.surveymonkey.com/ (assessed November 13, 2011). Wentzel-Larsen, T., Norekval, T. M., Ulvik, B., Nygard, O., & Pripp, A. H. (2011). A proposed method to investigate reliability throughout a questionnaire. BMC Medical Research Methodology, 11(137). doi: 10.1186/1471-2288-11-137 Wilson, V. (2010). Disseminating your research. Evidence Based Library and Information Practice 6(1). Young Adult Library Services Association (2010). Competencies for librarians serving youth: Young adults deserve the best. www.ala.org/ala/mgrps/divs/ yalsa/guidelines/yacompetencies2010.cfm (accessed November 12, 2011).