Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Demystifying assessment: Crafting a message to communicate results that address stakeholder concerns.

•Download as PPTX, PDF•
•
Lynn Connaway
Lynn Connaway

"Mikitish, S., Connaway, L. S., Radford, M., Kitzie, V., Floegel, D., & Costello, L. (2019). Demystifying assessment: Crafting a message to communicate results that address stakeholder concerns. Presented at ALA Annual, June 23, 2019, Washington, DC. "

1 of 21
ALA 2019, Washington, DC June 23, 2019 Update on ACRL's Value of Academic Libraries Initiative: Communicating Assessment Results to Stakeholders Stephanie Mikitish, Ph.D., Library of Congress Lynn Silipigni Connaway, Ph.D., OCLC Marie L. Radford, Ph.D. Rutgers University Vanessa Kitzie, Ph.D., University of South Carolina Diana Floegel, MLIS, Doctoral Student, Rutgers University Laura Costello, MLIS, Doctoral Student, Rutgers University
Today’s Agenda Assessment defined Stakeholder categories Common concerns/goals of stakeholders Library data Communication channels/methods/strategies Group activity
Assessment
Assessment Defined • Process of… • Defining • Selecting • Designing • Collecting • Analyzing • Interpreting • Using Information to increase service/ Program effectiveness Interpreting Analyzing Collecting
Stakeholders
Stakeholder categories Type Definition Examples Steering/ executive Leaders who develop, coordinate, & gather feedback Task force members, officials Advisory Members who gather feedback & speak for their groups Representatives (e.g., school board), policymakers, researchers, teachers, parents, students Associations Engage, promote, & gather feedback regarding library ACRL, ALA
Stakeholder categories Market name Definition Examples Customer Use library’s resources Adults, children, students Internal Employees Librarians, staff, pages Supplier/ Alliance Provide resources to library Publishers, vendors, board members, provosts, CAOs Referral Refer customers to library Parents, teachers, faculty Recruitment Are or refer potential library employees MLIS/MI students, interns Influence Affect institution Accreditors, governing bodies, consortia
Concerns/Goals of Stakeholders
• Stewardship of funds • Effect on institutional objectives • Recruitment • Achievement • Retention • Job placement • Learning • Literacy • Learning Stakeholder Concerns/Goals
Library data
Library data Important factors: •What data exist, and what questions do they answer? •How is it measured, and why? •What is being done with it? •Based on the above, are there alternative data? Data types: Qualitative, quantitative, and mixed From data to narrative (Grieves, 2017) Collection Analytics Insights
Communication
Communication methods • Factors to consider: • Platforms (e.g., electronic, physical) • Targeted audience • Findability (i.e., is it Google- able?) • Frequency
Communication content Name Text Image Sound Website Report Blog
Communication strategies Primary Consideration – Stakeholder Group Next – What are your assessment findings that are of greatest importance to groups? • Decide which data points are important for each group. • Think about their priorities - what in your data speaks to these? • Design visualizations that are attractive and can easily be grasped. • Bring data to life - select juicy quotes (or stats!) for illustrations that pertain to that group. • Which method will reach each group most effectively?
Group activity
Group activity • Choose a group from worksheet • Individual work • Share with a partner • Report results
Closing
Thanks & Questions? • Stephanie Mikitish, Ph.D. (smik@loc.gov) • Lynn Silipigni Connaway, Ph.D. (connawal@oclc.org; @LynnConnaway) • Marie L. Radford, Ph.D. (mardford@rutgers.edu; @MarieLRadford) • Vanessa Kitzie, Ph.D. (kitzie@mailbox.sc.edu; @vkitzie) • Diana Floegel, Ph.D. student (diana.floegel@rutgers.edu; @floeginator) • Laura Costello, Ph.D. student (laura.costello@Rutgers.edu)
References Identifying audience/stakeholders Association of College and Research Libraries. (2017). Provost semi-structured individual interviews. In Academic library impact: Improving practice and essential areas to research (pp. 31-36). Prepared by L.S. Connaway, W. Harvey, V. Kitzie, and S. Mikitish. Chicago: Association of College and Research Libraries. Murray, A. L., & Ireland, A. P. (2017). Communicating library impact on retention: A framework for developing reciprocal value propositions. Journal of Library Administration, 57(3), 311-326. Robertson, M. (2015). Perceptions of Canadian provosts on the institutional role of academic libraries. College & Research Libraries, 76(4), 490-511.
References Communication do’s/don’ts and best practices Association of College and Research Libraries. (2017). Communicate the library's contributions. In Academic library impact: Improving practice and essential areas to research (pp. 45-50). Prepared by L.S. Connaway, W. Harvey, V. Kitzie, and S. Mikitish. Chicago: Association of College and Research Libraries. Institute of Education Sciences. (2011). Stakeholder communication. SLDS Best Practices Brief, 1. Retrieved from https://nces.ed.gov/programs/slds/pdf/best_practices.pdf Library communication examples Grieves, K. (2018). Generating bespoke value and impact evidence to inform a thought leadership approach to service engagement at The University of Sunderland. Performance Measurement and Metrics, 19(1), 53-65. Kelly, M., & O’Gara, G. (2018). Collections assessment: Developing sustainable programs and projects. The Serials Librarian, 74, 19-29.

More Related Content

Demystifying assessment: Crafting a message to communicate results that address stakeholder concerns.

  • 1. ALA 2019, Washington, DC June 23, 2019 Update on ACRL's Value of Academic Libraries Initiative: Communicating Assessment Results to Stakeholders Stephanie Mikitish, Ph.D., Library of Congress Lynn Silipigni Connaway, Ph.D., OCLC Marie L. Radford, Ph.D. Rutgers University Vanessa Kitzie, Ph.D., University of South Carolina Diana Floegel, MLIS, Doctoral Student, Rutgers University Laura Costello, MLIS, Doctoral Student, Rutgers University
  • 2. Today’s Agenda Assessment defined Stakeholder categories Common concerns/goals of stakeholders Library data Communication channels/methods/strategies Group activity
  • 4. Assessment Defined • Process of… • Defining • Selecting • Designing • Collecting • Analyzing • Interpreting • Using Information to increase service/ Program effectiveness Interpreting Analyzing Collecting
  • 6. Stakeholder categories Type Definition Examples Steering/ executive Leaders who develop, coordinate, & gather feedback Task force members, officials Advisory Members who gather feedback & speak for their groups Representatives (e.g., school board), policymakers, researchers, teachers, parents, students Associations Engage, promote, & gather feedback regarding library ACRL, ALA
  • 7. Stakeholder categories Market name Definition Examples Customer Use library’s resources Adults, children, students Internal Employees Librarians, staff, pages Supplier/ Alliance Provide resources to library Publishers, vendors, board members, provosts, CAOs Referral Refer customers to library Parents, teachers, faculty Recruitment Are or refer potential library employees MLIS/MI students, interns Influence Affect institution Accreditors, governing bodies, consortia
  • 9. • Stewardship of funds • Effect on institutional objectives • Recruitment • Achievement • Retention • Job placement • Learning • Literacy • Learning Stakeholder Concerns/Goals
  • 11. Library data Important factors: •What data exist, and what questions do they answer? •How is it measured, and why? •What is being done with it? •Based on the above, are there alternative data? Data types: Qualitative, quantitative, and mixed From data to narrative (Grieves, 2017) Collection Analytics Insights
  • 13. Communication methods • Factors to consider: • Platforms (e.g., electronic, physical) • Targeted audience • Findability (i.e., is it Google- able?) • Frequency
  • 14. Communication content Name Text Image Sound Website Report Blog
  • 15. Communication strategies Primary Consideration – Stakeholder Group Next – What are your assessment findings that are of greatest importance to groups? • Decide which data points are important for each group. • Think about their priorities - what in your data speaks to these? • Design visualizations that are attractive and can easily be grasped. • Bring data to life - select juicy quotes (or stats!) for illustrations that pertain to that group. • Which method will reach each group most effectively?
  • 17. Group activity • Choose a group from worksheet • Individual work • Share with a partner • Report results
  • 19. Thanks & Questions? • Stephanie Mikitish, Ph.D. (smik@loc.gov) • Lynn Silipigni Connaway, Ph.D. (connawal@oclc.org; @LynnConnaway) • Marie L. Radford, Ph.D. (mardford@rutgers.edu; @MarieLRadford) • Vanessa Kitzie, Ph.D. (kitzie@mailbox.sc.edu; @vkitzie) • Diana Floegel, Ph.D. student (diana.floegel@rutgers.edu; @floeginator) • Laura Costello, Ph.D. student (laura.costello@Rutgers.edu)
  • 20. References Identifying audience/stakeholders Association of College and Research Libraries. (2017). Provost semi-structured individual interviews. In Academic library impact: Improving practice and essential areas to research (pp. 31-36). Prepared by L.S. Connaway, W. Harvey, V. Kitzie, and S. Mikitish. Chicago: Association of College and Research Libraries. Murray, A. L., & Ireland, A. P. (2017). Communicating library impact on retention: A framework for developing reciprocal value propositions. Journal of Library Administration, 57(3), 311-326. Robertson, M. (2015). Perceptions of Canadian provosts on the institutional role of academic libraries. College & Research Libraries, 76(4), 490-511.
  • 21. References Communication do’s/don’ts and best practices Association of College and Research Libraries. (2017). Communicate the library's contributions. In Academic library impact: Improving practice and essential areas to research (pp. 45-50). Prepared by L.S. Connaway, W. Harvey, V. Kitzie, and S. Mikitish. Chicago: Association of College and Research Libraries. Institute of Education Sciences. (2011). Stakeholder communication. SLDS Best Practices Brief, 1. Retrieved from https://nces.ed.gov/programs/slds/pdf/best_practices.pdf Library communication examples Grieves, K. (2018). Generating bespoke value and impact evidence to inform a thought leadership approach to service engagement at The University of Sunderland. Performance Measurement and Metrics, 19(1), 53-65. Kelly, M., & O’Gara, G. (2018). Collections assessment: Developing sustainable programs and projects. The Serials Librarian, 74, 19-29.

Editor's Notes

  1. SM
  2. SM
  3. SM
  4. From IES (2011) SM
  5. LSC
  6. From IES (2011) LSC
  7. Murray & Ireland (2017) on Payne, Ballantyne, & Christopher’s (2005) markets customer [students] internal [library employees] supplier/alliance [publishers, vendors, provost/chief academic officer, university president, board members] referral [customers and non-customers, faculty] recruitment [potential employees] influence [groups/individuals including student government, deans, CFOs, other libraries, consortia, associations] LSC
  8. LSC
  9. 2 basic ones according to Grieves (2017): value for money; value/impact in achieving institutional objectives Libraries have been slow to frame things in these terms because according to Kelly & O’Gara (2018): “Far too often, libraries report and evaluate based on what has historically been done – collecting stats because they are available, because “we always have,” or because they answer a question the parent institution used to have” Linking the library to these concerns/goals can be difficult! According to a study by Murray & Ireland (2017) of 68 university (Master’s as of 2013) library directors: “A final, strongly prevalent theme identified in this data was the lack of any known methods to document library impact on retention. Several respondents commented that, while they desire to initiate a process, they do not know how to start.” LSC
  10. Kelly & O’Gara (2018): “libraries have access to more data than they can ever hope to use. That said, understanding the data that already exist, and what questions data can help to answer about the use of a collection, is key to developing a sound assessment approach” VK
  11. Grieves (2017): data vs analytics (evidence ordered by pattern/trend) vs insights (value/intelligence gained through analytics, the story/narrative) Kelly & O’Gara (2018): data inventory fields: metric (name of thing being measured), category [e.g., collection, space, service], difficulty to obtain (1-5, easy-difficult), data source, compiled (how frequently), purpose; mapping assessment to tools/techniques and data fields: question, tools/techniques, readily available data, wish-list data VK
  12. Grieves (2017): relationship management “Makes a conscious shift from the academic support role to one of expert, active and trusted partner” Murray & Ireland (2017): reciprocal value propositions can make “’explicit the benefits expected to be gained and given up’ by all involved parties, may be informal or formal, allows for coordinated meaning-making, and serves as the basis of trust for longer=term relationships” (Ballantyne et al., 2011); ideally reciprocal rather than unidirectional 68/271 library directors/deans at public comprehensive US universities w/ a master’s level Carnegie classification as of Jan 2013 RQ1: How do academic library deans/directors document the impact of library services and resources on student retention? Relating info lit efforts to retention; User/student satisfaction/feedback surveys on services/resources RQ2: How do academic library deans/directors communicate the impact of library services and resources on student retention? Often none!; Formal presentations and reports>annual reports>informal communication VK
  13. Kelly & O’Gara (2018): “The choice of medium should be based on stakeholder preferences…When determining how results are communicated, consider what stakeholder actions the results should prompt. It can be helpful to think of communication as storytelling” VK
  14. Twitter: http://www.stickpng.com/assets/images/580b57fcd9996e24bc43c53e.png Instagram: https://diylogodesigns.com/wp-content/uploads/2016/05/instagram-Logo-PNG-Transparent-Background-download-768x768.png Apple podcast: https://communities.apple.com/kb20181115/securedImage.jsp?productid=PODCASTSFORIOS&size=240x240 Facebook: https://www.jolttx.org/wp-content/uploads/2017/04/photos-facebook-logo-png-transparent-background-13-300x169.png Checkmark: http://clipart-library.com/images/dc9rkjRc7.png VK
  15. VK
  16. SM
  17. SM