Accepted Paper Abstracts
Monday, 17 June
Using routine evidence within health libraries (Session A, 10:45am-12:15pm)
Chloe Stewart. How are help and information seekers really searching? Analysing user routes through an enquiry management system
This paper analyses how users seeking assistance from the library services of a large Scottish health board area make use of an online enquiry management system.
Background
Zendesk has been used since May 2017 to manage most NHS Greater Glasgow and Clyde Library Network service requests (including enquiries, training and literature search requests). User support and documentation is also provided via Zendesk and is continually added to, and potentially provides an alternative for users to some queries and training needs that they place; however the level of usage has hitherto not been assessed. An analysis of user information seeking behaviour is timely as this year Zendesk will be replaced by ServiceNow as our enquiry management system.
Methodology
Search strings entered, help material viewed and service requests placed by users were analysed over a 3 month period in 2018 to ascertain:
the categories of topics searched on
the behaviour of users when presented with search results
the level of usage of help and support material
the behaviour of users on the system leading up to the placing of service requests
Results
Local unofficial practice in enquiry deletion limited the amount of data available for categorisation and session analysis of quick/routine enquiries as opposed to service requests. The route used by a high proportion of users in making requests (email) diverts them from support material. Session analysis indicates that only a minority of users search or browse before placing service requests. Much support material has a low usage. Few users make use of the search facility, and a substantial number confuse it with the combined catalogue and article search resource which is accessed through a different website.
The results will inform local practice in providing and arranging support material on ServiceNow and the library service websites, inform marketing practice, and assist local workflow and request handling protocols. It should be of wider interest to those who wish to analyse what users (especially remote users) really do when seeking information and support via online enquiry and support systems.
Stacey Astill. Statistical Shields: drawing Manx libraries together in uncertain times
Objective
Keyll Darree library, Isle of Man routinely collects statistics as part of the national quality assurance system for English health libraries. We regularly use these statistics to build business cases and defend our own service and wish to build a similar body of evidence which can be used by all types of Manx library services to support their: purpose, use, and funding/service needs. Although research evidence can be used to make decisions, it is important to also represent local situations – there is little research into services on the island (especially as a full body of libraries), and therefore this leaves a significant knowledge gap around the use of Manx libraries. This initiative is framed by a political interest in the Manx library services brought about by a national Isle of Man Government debate containing some fundamental misunderstandings of modern libraries which has included combining or removing services as well as funding changes.
Methods
Two phases of survey were used to gather information on the statistics recorded across all Manx library services. The complete set of potential statistics were circulated to all libraries for feedback regarding their ability to collect the identified data. This feedback was used to make recommendations regarding local evidence collection in the future (including specific recommendations for school and teaching libraries), and design a form to facilitate this process.
Results
The survey showed that the majority of services are not recording footfall, events, teaching sessions, staff engagement, renewals, literature searches, or other information which demonstrates service diversification and value. However most would easily be able to do so.
The first statistics will be collected in April 2019, and preliminary results will be presented at the conference.
Discussion
As a small island we should be able to build a pool of statistics for all Manx libraries to drawn on for advocacy in uncertain times. The process of implementation has been met with enthusiasm, suspicion, and concern in turn, but a small working group has formed to help process the information and make positive steps towards building evidence of our worth as a complete Manx library service.
Dominic Gilroy. National collation of quantitative evidence from an NHS Library and Knowledge Services Impact Questionnaire: challenges and uses
Aim
To determine whether the results of a National Impact Questionnaire, implemented at local level within NHS organisations, can be collated centrally in a meaningful way to help build a national picture of evidence on the impact of NHS libraries.
Methods
•A working group made up of library professionals working in the health sector researched and compiled a 5 question Impact Questionnaire consisting of 5 questions based on existing questionnaires published in the professional literature and in use within the sector. Initial validation and piloting was undertaken prior to dissemination.
•This questionnaire was disseminated through various channels with services asked to make use of the questionnaire without altering the basic questions.
•In late 2017 a call was issued for those sites which had collected impact data using the questionnaire to submit their data centrally for analysis.
•Resulting data was analysed to determine whether collation was possible, to identify limitations of the process, and to identify what conclusions, if any, could be made.
Results
Analysis of the resulting data indicated the following limitations to the collection process:
•Although intended for implementation following receipt of a specific library service by users (eg: training session), on occasions the questions were inserted within a wider library survey resulting in broader, less focused results and affecting collation.
•A variety of online questionnaire tools used locally (SurveyMonkey, BOS, etc) resulted in different output styles and complications when attempting to collate data centrally.
Nevertheless, it was possible to collate much of the data and make some general statements regarding the impact of health libraries on a national scale.
Conclusion
There is potential to collect impact data nationally using a locally implemented generic questionnaire. To ensure that the collation of this data is possible in an efficient manner the following need to be in place:
•Common understanding of the context in which the questionnaire is used
•Use of a common format/tool for the submission of local results to streamline collation
•Clarity over the use for resulting impact data to ensure efforts are targeted appropriately.
Using qualitative methods to demonstrate value and impact (Session A, 10:45am-12:15pm)
Susan Archambault. In Their Own Words: Using Student Feedback to Improve LibGuides
Aim
Lengthy descriptions of research databases can unintentionally cause extraneous cognitive load at a moment when a student researcher is already taxed. Furthermore, publisher descriptions of databases often don’t conform to students’ mental models. To address this problem, librarians at a medium-sized private university combined data from established LibGuides best practices and student vocabulary mined from chat reference transcripts to design a new formula for database descriptions in LibGuides.
Methods
The new database description formula was tested through intercept task-based usability on 10 students. The test compared the success rate and preference of students using database descriptions written in the old format (one long paragraph) vs. the new format. Students using the new style of the database descriptions performed better than students using the old style for the variables of time on task, number of wrong paths, and success rate. Also, student preference for the new style over the old style was statistically significant using the t-test for two dependent samples.
Results
The new formula for database descriptions was applied on all the LibGuides at this university. Recognizing that online users naturally “skim” websites, the new descriptions are broken down into shorter chunks of information with separate headings for “subjects,” “description,” and “contents.” The subjects are based on the majors and minors at this particular university, while the content types emphasize items that students overwhelmingly search for, such as scholarly (peer reviewed) journal articles and primary sources. The description emphasizes important information such as whether there is full text, the tool type, the vendor name, time period of the content, and whether there is a geographic area of focus.
Nicola Pearce-Smith. Demonstrating the value and impact of literature searches conducted for Public Health England (PHE): collecting Impact Stories
Aim/Objectives
• to discover how our Knowledge and Library Services (KLS) literature searches contributed to the knowledge, research and policy-making of our users
• to have access to a collection of stories enabling KLS to demonstrate the value and impact the library service has on informing public health decision-making
Methods
The PHE KLS record statistics on the number of literature searches conducted each month, but what are these searches used for? Do they contribute to policy-making or research publications, do they lead to cost or time savings, do they result in new tools or services – ultimately, what is their impact? KLS began collecting case-studies in an effort to reveal the impact of our literature searches. We adapted a case-study template produced by CILIP, the UK Library and Information Association, which we called an Impact Story template. Instructions, an example completed template and standard email text were produced to enable KLS staff to undertake a consistent approach to obtaining completed Impact Stories. KLS staff contacted users who had been sent literature search results more than 3 months ago (to allow time for the search to have impacted on their work), with a request to fill in an Impact Story.
Results
KLS and PHE staff collaborated to produce Impact Stories using the template. Targeting users who had already given us feedback about the literature search they received proved successful in getting users to complete an Impact Story template. Tips on how to complete each template heading proved useful especially if they were tailored to the specific literature search. 15 Impact Stories have been completed since October 2017. All completed Impact Stories are available on our library web pages.
Discussion
KLS can now demonstrate the impact that our literature search service has, including providing essential support for important research publications, supporting funding applications, identifying gaps in evidence and contributing to the development of new tools.
Louise Graham. Storing Stories - answering the So What? question
Aim
Using digital technology to engage with partners, developers and stakeholders to capture qualitative data to identify and collate evidence of impact of library services.
Methods
We were aware of a constant struggle within our own service to measure, record and evidence impact; potentially in common with our profession as a whole. It became apparent that we did manage to record impact, but that it was mainly in anecdotal form which, although authentic and valuable, was hard to categorise, search, cross-reference. We wanted to develop a tool which would allow library customers to provide feedback (evidence of impact) in an easy way and record how libraries had made a tangible difference.
Results
The tool was successfully developed as part of a Hackathon process, receiving funding from SLIC as runner up in the process, and ran initially as a limited pilot with Stirling Libraries where it returned responses of a quality and quantity that outstripped any previous written surveys. On the basis of the responses, SLIC agreed to increase the funding to allow an extension to the pilot within Stirling and the results were replicated across the rest of the branches. We applied for PLIF funding to extend the pilot nationally and this was successful; this extended pilot is currently running across several local authorities. In addition, Stirling Libraries used the evidence of impact gathered by Storing Stories to inform the recent Service Review and refreshed Library Strategy, and also adapted the core tool to produce a digital events module to evaluate the annual Off the Page book festival, producing compelling and authentic information on how this makes a positive difference to communities.
Discussion/Conclusion
I would like to talk about how the tool has been designed and adapted for use in public libraries, to showcase how it works, and demonstrate how it has met the stated aims of being able to capture evidence of impact by sharing some of the data we have collected.
We believe this could be a tool used nationally to connect with stakeholders and gather feedback to intelligently inform service development and strategy.
Building skills (Session A, 10:45am-12:15pm)
Lorie Kloda, Jodi Philbrick and Susan Lessick. Reducing Uncertainty: Building Health Sciences Librarians’ Capacity for Evidence-based Practice through a Research Training Institute
Aim
This paper assesses the effectiveness of the 2018 Medical Library Association’s Research Training Institute (RTI) curriculum model to increase the research output of participants and support evidence-based practice. The RTI is a program funded by the Institute of Museum and Library Services to equip practicing health sciences librarians in the United States with key research competencies related to scholarly research, inquiry, and publishing. It includes online and face-to-face content and activities for a group of 20 health sciences librarians. The centerpiece of the RTI is a five-day immersive face-to-face workshop held in the summer, where health sciences librarians develop the skills and confidence necessary to conduct a rigorous research study during the ensuing year. This paper focuses on program outcomes related to participating librarians’ research and evidence-based practice knowledge from the first RTI held in Chicago in July 2018.
Methods
This paper will compare participants’ confidence with respect to research and evidence-based practice skills before and after the workshop. The instrument used for the pre- and post-assessment survey is based on methods of Brancolini and Kennedy (2017) and adapted for use with the RTI. A paired-sample t test will be performed to compare all 20 participating librarians’ self-reported ability on these skills. The RTI program content and structure will be described.
Results
The results of the pre- and post-assessment survey of research skills and confidence will be shared. Findings from a post-workshop evaluation will also be described.
Discussion/Conclusion
The uses of the pre- and post-assessment survey and the post-workshop survey will be discussed, including changes that will be made to the 2019 RTI program based on assessment results. Unexpected benefits and drawbacks of the RTI experience will also be discussed. The effectiveness of the RTI in building capacity of the first cohort of health sciences librarians to use evidence in practice will also be reviewed.
This paper will present results of research analyzing the impact of continuing professional develop (CPD) workshops on librarians’ practice in the workplace. When times are tough, funding for CPD is often a casualty and yet training and development can equip librarians with the skills to weather uncertain times and develop new, innovative approaches to better serve their customers. CPD workshops, specifically, have been found to have a range of advantages in this regard because they can be tailored quite specifically to participants’ existing skills and needs and they also provide valuable opportunities for networking with fellow professionals, thus gaining fresh perspectives on challenges faced in the workplace. The effectiveness of short workshops in changing practice has been questioned, however (Lydon and King, 2009), and the difficulty of assessing the impact of this type of CPD has been noted (Nicolaidou and Petridou, 2010). Evaluation usually takes the form of one-off, immediate participant self-reports of their reactions to the event and self-assessment of their learning. Evidence of participants’ use of knowledge gained is more difficult to gather and can only really be evaluated after they have returned to the workplace and have applied their learning. More general concern about the positive transfer of training to the workplace is widespread (Baldwin and Ford, 1988) but there has been little focus on the “transfer problem” (Michalak, 1981) in the librarianship literature. This research aims to address this research gap. Using a questionnaire based on an adaptation of Guskey’s (2000) five level framework for evaluating teacher CPD, librarians who participated in four CPD workshops were surveyed to explore their reactions to the workshops, their evaluation of their learning, their use of new knowledge and skills in the workplace and their view of the impact of their learning on library users. The results provide evidence of whether and, if so, how library professionals implement their learning from CPD workshops in the library workplace and the impact it has on their practice and service delivery.
Meeting user needs (Session B, 1:00pm-2:30pm)
Lee Ann Fullington and Matthew Harrick. New campus, new information: Library support for transfer students
Aim
Transfer students--students in their first term at a new institution who have completed credits at another college or university after high school-- are the largest population of new students each semester at our institution, a bachelor’s degree granting college in an urban public university system. A recent restructuring of our institution’s orientation program saw the library lose its place at annual transfer student orientation events, pointing to an uncertain future for library participation. This study aims to provide evidence for reintegrating the library into transfer student events. We also look for evidence to show how the library can collaborate with other campus departments and services to increase transfer student success through sharing of information.
Methods
We employed a screening survey to recruit first-semester transfer students for focus groups in the fall and spring semesters. We used open-ended questions to foster discussion about their experiences transferring and adjusting to their new campus. To analyze the transcripts, we used thematic analysis.
Results
Students were candid in discussing their stories of being transfer students, and pointed out numerous opportunities for the library to support them in their acclimation to campus. Themes included: feeling confused and overwhelmed with campus processes and policies and frustration with a lack of local knowledge, both of which contribute to unnecessary stress. Students see the library as a source of information and guidance, and noted that library presence at orientation would have been useful.
Discussion/Conclusion
In the focus groups, students discussed their experiences with the transfer process, including attending orientation and acclimating to their new campus. Analysis of their comments allowed us to identify new ways the library could assist them and to improve current support. Library participation in orientation events is key to disseminate local knowledge to incoming students, and we discuss different modes of participation, both new and revised. Much of the frustration and stress for transfer students could be alleviated by more opportunities for the library and librarians to share information by participation in events for transfer students.
Esther E. Carrigan, Heather K. Moberly, Eric G. Riddle, L.M. Rey and Molly Crews. Using Pilot Program Evidence to Determine Feasibility and Parameters of a New Service
Background
Veterinarians are trained in evidence based veterinary medicine (EBVM), a parallel to human evidence based practice. This methodology is generally described with five steps, including the essential steps of identification of published evidence and its acquisition for further appraisal and application. Librarians are well-positioned to lessen the challenges to practicing evidence based veterinary medicine: identifying published evidence and acquiring it.
Aim
This research, continuing at the time of this abstract, is building evidence-in-practice data through pilot studies about the use of EBVM support services provided by an academic medical library. Specifically, it focuses on veterinary practitioners’ use of these services by partnering with selected professional veterinary associations to offer this service as a membership benefit. Pilot data, shared with participating associations, will be used to define service parameters and to clarify mutually acceptable terms of service between the associations and the library, including costs and limitations of the service. Questions surrounding the establishment of this service include: What service demand will be generated by practicing veterinarians? What is the impact on existing library staffing and workloads? Can we create a sustainable, affordable EBVM service?
Methods
Principal investigators developed a standard Memorandum of Understanding (MOU) to clarify the parameters and responsibilities of each party in the service pilot. The initial association partnership was triggered by repeated service requests from a veterinary practitioner professional association. Once that pilot was underway, other association requests for the service quickly followed. With new association service inquiries continuing and seven association partners already in the pilot, principal investigators put a hold on adding any additional service partners. The Illiad software was used for transaction management and as a data source for service activity analysis, with data export into Excel for additional analysis.
Results
Selected information from pilot data over several years of activity will be presented. Data sources include partner association total membership, active participant numbers, individual requestor activity levels, cumulative association activity levels, published materials requested, and library impacts. Challenges and modifications of the pilot MOU will also be presented.
Conclusions
Preliminary parameters of the resulting EBVM service will be presented.
Laura Kuo. Evidence on how Libraries Enhance the Academic Success of At-Risk Students
Aim
The aim of this study is to provide evidence on how libraries enhance the academic success of at-risk students through campus partnerships and embedded services. The author defines one aspect of academic success as information literacy (IL) skills as measured through IL self-efficacy and self-reported student reflections.
Methods
In the summers of 2017 and 2018 the library participated in a four-week pre-college summer orientation program for incoming economically and educationally disadvantaged (“at risk”) students. The program is designed to simulate an academic semester and includes courses in math and writing. In the summer of 2017, the librarians developed and delivered three 50-minute active learning workshops in which students worked collaboratively and were introduced to the following research concepts: developing a research question; search strategies; evaluating sources; and citations. In the summer of 2018, the librarians were given an expanded role and provided four 50-minute workshops introducing the same research concepts, but with culturally relevant topics and inclusive IL pedagogical practices.
Prior to the first workshop, students were asked to respond to the following three questions: (1) please describe your high school library; (2) what, in your opinion, are libraries for; and (3) what do you want to learn during the library workshops this summer. At the beginning of the first and last workshops, students completed a 15-question adapted version of Kurbanoglu’s IL Self-Efficacy Scale based on the content presented. Students were also asked to complete a short two-question One Minute Paper survey at the end of each workshop asking (1) when searching for information, what will you do differently and (2) what questions do you still have about doing research?
A mixed-methods approach will be used to analyze and summarize the quantitative (paired sample t-test) and qualitative data (opening coding and thematic organization) from the self-efficacy scale and one-minute paper.
Results
The evolution of the collaboration, including the initial design, revisions to the instructional content, survey data analysis, embedded reference services throughout the semester, and outcomes for all partners will be discussed.
Discussion/Conclusion
This research asserts the importance of developing campus partnerships and embedding support for at-risk students.
Digital and media literacy (Session B, 1:00pm-2:30pm)
Katie Edwards, Christine Gallagher and Kristi Long. Supporting digital information literacy in post-integration Scottish health and social care
Aims
We are evaluating approaches to digital information literacy training provided by Library Services at territorial and national health-board level in Scotland. We are comparing face-to-face with online training and generic approaches versus targeting of specific groups. This builds on previous training aimed specifically at social services workers (Downie et al, 2018). The health and social care landscape is rapidly changing and national introduction of Realistic Medicine (Chief Medical Officer, 2018) highlights the importance of good quality information in supporting shared decision-making and evidence-based practice. Digital information literacy skills are essential to deal with an overload of information of variable quality and to meet complex information needs, especially in the post-integration health and social care context. This work is valuable as a general evaluation of services and training, as well as horizon scanning to establish how best to provide training for integrated teams.
Methods
A search of current literature on good practice for digital information literacy teaching approaches, with reference to specific populations to establish evidence gaps. This plus previous evaluation responses has been used to ascertain a baseline.
We are utilising current evaluation methods for training including online and paper surveys and have begun to standardise methods used in territorial and national services, with reference to outcomes of the literature search. This standardisation process will be completed during January 2019. Evaluation responses will be collated and analysed over a period of three months.
Results
Qualitative and quantitative analysis and comparison of completed surveys from users/trainees over the period specified. Themes will be identified from analysis to inform conclusions.
Discussion/Conclusion
Using themes identified, we will generate conclusions for training strategies which are evidence-based. These will be used to inform training for an integrated workforce and provide standardised training evaluations to consistently measure impact. This will reflect different user types, professions and other relevant demographics and take into consideration the pros and cons of multi- versus single-profession sessions in addition to online versus face to face or local versus national. This may also have implications for other sectors.
Angela Short and Lindsay McKrell. Work IT – how libraries can deliver digital support for employability and the lessons for wider digital inclusion
Stirling Council Libraries identified a growing demand for digital employability support in libraries and applied for grants so we could offer help and evaluate its effectiveness. We received a bursary from the Information Literacy Group of CILIP funding a Digital Inclusion Officer for four months. This post was extended by ten months of additional funding from the Digital Participation Unit of Scottish Government.
From mid-September 2017 until November 2018, we worked with jobseekers, helping people register with Universal Jobmatch and maintain their jobsearch journal for the UK Department of Work and Pensions. We worked extensively with partners including the Citizens Advice Bureau and our Council Learning and Employability service. Our Digital Inclusion Officer worked with eight job clubs, attending Work IT sessions in rural and urban settings. Over the course of 10 months over 156 drop-in sessions were delivered with 229 attendees.
We assessed participants’ digital skills, confidence, frequency of internet use and aspirations for the future when first attending our regular jobseeking drop-ins and again after they had attended several sessions. The improvement was marked. Among the 32 learners who attended regularly and signed up for our research, weekly internet use increased from 44% to 68%. 57% said they were using the internet to look for and apply for work compared to 41% initially and 54% reported that they now send and receive emails, compared to 28% at the start of the sessions. Asked what difference the club has made to them, people said, “It has made me more aware of what is out there”, “It has helped me in applying for jobs by email” and “It has helped me a lot because I didn’t even know how to switch a computer on.” In 27 comments received, 12 respondents used the words “more confident” or “boosted confidence” when talking about the impact attending Work IT had for them. Seven participants said they valued the chance to mix with others, making friends and talking to people.
We now have two funded Digital Inclusion posts. We are continuing to evaluate so we can develop and improve the support we offer.
Doreen Bradley and Craig Smith. How Do College Students Find and Evaluate News? Using Evidence to Improve Teaching Methods for Media Literacy
Aim
This study seeks to understand students’ beliefs and practices related to news sources in order to inform teaching methods and improve student media literacy. In particular, we investigated whether certain practices related to the rigor of engaging with news (e.g., engaging in fact checking, relying on Fox News, etc.) were associated with certain beliefs about news (e.g., it’s difficult to differentiate real from fake news).
Methods
In Spring 2018, 817 undergraduate students at a research university participated in a national study examining how students find, use and evaluate news. These data were used to identify certain student beliefs and practices that could be improved with modified teaching methods. Researchers conducted statistical analyses, including correlations and t-tests, to explore of relations between student beliefs and self-reported practices.
Results
Analyses of the survey data led to several significant findings. Respondents who preferred Fox News over other sources were significantly more likely to agree that journalists deliberately insert their own bias into news stories, t = -7.49, p < .001. Those who indicated that they fact check news tended to agree less strongly that it is difficult to tell real from fake news, r = -.14, p = .006, and also tended to feel more confident in recognizing fake news, r = .37, p < .001. Students who feel more confident in recognizing fake news also tend to agree more strongly that news is necessary in a democracy, r = .24, p < .001.
Discussion
Library instructors teach media literacy in many venues across campus. Understanding how students use, process, and evaluate news facilitates the improvement of pedagogy for media literacy. Many of these findings, as well as others that will be shared, involve issues of trust. Although conclusions about causal relations between the variables described above cannot be reached, the findings speak to the importance of helping students develop explicit strategies for finding and evaluating news. Guided by these findings, media literacy instruction, including source evaluation, was enhanced in two credit-bearing library courses and in course-integrated instruction sessions.
New roles for health librarians (Session B, 1:00pm-2:30pm)
Michelle Kirkwood. The role of the Knowledge Broker in the development of Clinical Decision Support
NHS Greater Glasgow and Clyde will in 2019 become an NHS Scotland Center of Excellence for Clinical Decision Support. Building on previous work the organisation is developing the range and reach of Clinical Decision Support (CDS) by adopting The Right Decision platform. Using this platform will allow NHS Scotland staff and partners to create shareable and customisable CDS across both mobile and web channels.
As an early adopter of The Right Decision the NHSGGC Library Network is working in partnership with clinical and technical leads to translate the existing emergency care pathway for Chest Pain into a set of mobile and web tools, including visual and narrative versions of the pathway, and a HEART calculator.
In addition to having technical outcomes, this project is also an opportunity to map out a framework of roles and skills for the continued development of Clinical Decision Support including that of the knowledge broker. It is important within the dynamic digital landscape of the NHS that the value of the librarian as a knowledge broker is highlighted (and continuously developed) as pivotal to the success of delivering evidence to the point of care, as well as reducing waste and unwanted variation in patient care in line with the national priority of Realistic Medicine.
This paper will present the role of the knowledge broker in Clinical Decision Support from the strategic to the practical level, using this project as an example.
Diane Pennington and Laura Cagnazzo. Library linked data implementations and perceptions: Implications for practice
Aim
This presentation will discuss evidence surrounding the use of linked/open data applications in libraries and what this might mean for the evolution of librarians' current and future practice.
Paul Levay and Jenny Craven. Reflections on the role of expert searcher in times of uncertainty: a NICE case study
Aim
To stimulate discussion on the evolving role of the expert searcher as practice is extended beyond reviews of medical interventions and different types of evidence are gathered using new technologies.
Methods
The authors will show how their team is meeting the challenge of providing search services adapted to the evolving needs of the National Institute for Health and Care Excellence (NICE) in the UK.
The presentation will consider how services have been developed now that the remit of NICE has broadened from appraising pharmaceutical products to producing guidelines on public health, service delivery and social care. NICE requires the best evidence available within the time and resources available but what does this mean in practice?
Results
The presentation will discuss the methodological, interpersonal and technological challenges of supporting a wider range of guidelines. Expert searchers at NICE must adapt methods developed for medicine to different disciplines, with their own sources, types of evidence and ways of working.
We are facing rapid technological change. Text mining and machine learning have been deployed to develop searches and to analyse the results. Fully automated systems, driven by linked data and, potentially, artificial intelligence, are already being developed.
The presentation will show how the team has collaborated with multi-disciplinary groups, undertaken research and designed new processes to deliver high-quality searches.
Discussion
In times of uncertainty, could these challenges be seen as threats to the existence of the expert searcher? NICE has found that expert searchers can lead developments in ways that are helpful to the information profession.
Technical knowledge is important but the key skills required are ones inherent to the information profession: communication and collaboration. We must collaborate effectively with the teams we support to show them what skills we can bring. We must also collaborate with one another across the profession to ensure we promote innovative practices.
Conclusion
Expert searchers can thrive in these challenging times if they are flexible, creative and at the forefront of innovation.
Developing methods for EBLIP (Session C, 3:00pm-4:30pm)
Clare Thorpe and Alisa Howlett. Developing certainty via a maturity model for evidence-based library and information practice in university libraries
Aim
This paper addresses the questions:
• how does a university library leader know the library’s service and practice is evidence-based?
• how can a university library measure and progress its maturity in evidence-based practice?
Current models of evidence-based library and information practice (EBLIP) apply predominantly to individuals. Little is known about how an organisation can enhance its maturity level in being evidence-based, despite a growing demand from institutional and library leaders for evidence to demonstrate why investments in libraries should continue (Baker and Allden, 2017; Council of Australian University Librarians, 2016). With the exception of Gillespie et al (2016), there has been limited research into experiences of evidence-based practice in an organisational setting, either within libraries or within other domains.
Method
Twenty semi-structured interviews were conducted with two cohorts – four senior leaders at the authors’ institution; and sixteen library professionals employed at Australian and New Zealand university libraries. Transcripts were analysed using thematic analysis. Interviewees reflected on their own experiences with evidence-based practice, and the level of evidence-based practice maturity demonstrated by their library. The interviews, combined with a literature review and environmental scan of evidence-based practices in Australian university libraries, informed the development of the draft maturity model.
Results
At the time of writing, thematic analysis is still underway. Based on literature about the development and use of maturity models in higher education and libraries in areas such as e-learning (Marshall, 2012), user experience (MacDonald, 2016) and quality cultures in libraries (Wilson, 2015), the authors anticipate the model will include up to five stages of maturity. The model will identify characteristics of evidence-based practice at different levels of maturity, including strategies to progress maturity on a whole-of-organisation level.
Discussion/Conclusion
The development of a maturity model for academic libraries is the first to conceptualise and identify practical steps to cultivate a culture of EBLIP within organisations. Library leaders and library practitioners will benefit from the model as they seek to identify and build upon their evidence-based practice maturity, enabling more robust decision-making, a deeper understanding of their clients and demonstration of value and impact to their stakeholders.
Thane Chambers and Sharon Murphy. The Big Bang: When evidence and affect collide. A case study in group decision making
Aim
This paper presents an analysis done at the University of Alberta on three research evaluation tools and the role evidence played in the decision making process around licensing one of them. We explore the role evidence plays in the social phenomena of decision making. We look to theoretical concepts to understand affect, power structures, and interpersonal dynamics that also play strong roles alongside evidence in this process.
Methods
The University of Alberta Libraries’ used an evidence based approach with stakeholder feedback in the assessment of Elsevier’s SciVal, Clarivate Analytics’ InCites, and Digital Science’s Dimensions Analytics. The goal of the assessment was to recommend one tool for licensing. When the Library presented the data and its recommendations, the decision making conversations around which product should be licensed were not situated around the evidence presented, rather they were focused on components related to politics and heuristics. In essence, our in-depth analysis was less important than other elements. In trying to understand why this occurred, we explore theoretical models of group decision making, the roles of evidence, information, affect, and power. This paper explores ways in which factors, besides logical reasoning, need to be incorporated into the presentation of data when dealing with outside groups.
Results
We found a number of decision making theories that describe a non-linear process strongly influenced by power structures and interpersonal relationships. Our communication was based on the evidence of bibliometrics analysis. Including real world narratives instead of technical analytics would have enhanced our evidence. Although we were viewed to have expertise in these tools, our evidence was less compelling than emotional information from group members. Essentially our neutrality made our views less convincing than those from other group members, who lacked expertise, but were passionate.
Conclusion
Logical persuasion can work. But politics and affect are also important factors in how groups make decisions. This paper will help librarians working with non-librarians to be better able to present evidence in ways that are digestible and convincing. We hope to provide librarians with a toolkit of strategies to have our evidence heard and understood outside of the library.
Paula Ochôa and Leonor Gaspar Pinto. Gathering evidence for Sustainable Development Goals: an alignment perspective
Aim
Initiated in November 2016, the project Public Libraries and Sustainability: Gathering Evidence of Contribution to SDGs (Project PLS) aims to develop a framework for evaluating public libraries’ contribution to Sustainable Development Goals (SDGs). This Model for the Alignment and Evidence Gathering of Libraries’ Contribution to Sustainable Development was consolidated and expanded within the Project Libraries for Development and the UN Agenda 2030 (April-July 2018) - an initiative of the Portuguese Association of Librarians, Archivists and Documentalists in conjunction with IFLA - International Federation of Library Associations and Institutions (International Advocacy Program). Based on the outcomes of Project PLS, this paper discusses the role of alignment strategies and impact evaluation practices and competencies in the processes of gathering evidence and advocacy towards libraries’ contribution to the UN 2030 Agenda.
Methods
An integrated and holistic approach was used to build the Model for the Alignment and Evidence Gathering of Libraries’ Contribution to Sustainable Development. In the alignment process, the use/adaptation of library assessment data was valued: UN Global Indicators and the corresponding Portuguese national indicators, as well as of ISO 11620 – Library performance indicators and ISO 16439 - Methods and procedures for assessing the impact of libraries, complemented by the Global Libraries Impact Planning and Assessment Guide, the Generic Learning Outcomes and Generic Social Outcomes (Arts Council England). One of the Model´s tools – the Roadmap for the alignment and evidence gathering of libraries’ contributions to the 2030 Agenda – was tested in two workshops.
Results: The cornerstone of the model is a Roadmap that is anchored in the identification of potential (inter)relations between the Dimensions that frame impacts of library activities/projects and the SDGs. The Roadmap supports the alignment cycles in four steps: Pre-planning and learning; Planning in the face of alignment; Implement measures and indicators; and Using evidence to advocate for libraries.
Discussion/Conclusion
This model fosters a set of tools that values processes of gathering evidence in alignment strategies and impact evaluation practices. One of those tools - the Roadmap – proved to be useful in developing librarians’ alignment and assessment competencies, as well as advocacy efforts.
User needs (Session C, 3:00pm-4:30pm)
Christine Wolff-Eisenberg and Janet Fletcher. Collectively Supporting Faculty: A national study of research and teaching practices and needs
Aim
Libraries are increasingly called upon to provide evidence that their decisions—about the services they offer, the spaces they create, and the resources they purchase—are supporting the mission of their institutions and the needs of their faculty and students. A group of universities in New Zealand recently collaborated with a strategy and research organization to better understand the communities they serve and gather evidence to inform their strategic decisions.
Methods
In 2018, six of the eight New Zealand universities surveyed their faculty on their research and teaching practices and perspectives. The surveys covered topics on discovery and access, data management and preservation, scholarly communication, instruction and perceptions of student research skills, and the role of the library. Over 2,000 faculty members completed the survey for a 30% response rate, providing a rich dataset for exploration.
Results
Aggregate analysis yielded valuable findings on how NZ universities are similar and different from one another, how NZ faculty are similar and different from those in other countries, and how subgroups of NZ faculty differ from one another. Since fielding the surveys, the group of universities has considered both collective and individual action including working with faculty members to better support the publishing lifecycle, determining the library’s role in managing and preserving research data, developing strategies for promoting use of existing services, and developing new library services.
Discussion/Conclusion
In this session, attendees will learn both about the research and teaching practices of faculty members internationally as well as the value of engaging in a cross-institutional research initiative to develop and strengthen collaboration at a national level. Presenters will share results on New Zealand faculty practices and perspectives as well as how these compare to those from other countries including the US, UK, Canada, and Australia. Presenters will also discuss how survey results like these can be used effectively for decision-making both within individual universities and collectively.
Jung Mi Scoulas and Sandra De Groote. Factors affecting University Students’ Library Visits in person and online using a Multiple Regression Approach
Aim
The purpose of the current study is to identify factors influencing students’ library visits in person and online based on survey results of university students’ library experiences.
Methods
In spring 2018, a total of 28,725 students in a public research university in the United States were invited to participate in an on-line web-based survey with multiple choice questions, and 2,277 students responded (a 7% rate of response). Multiple regression analyses were conducted to examine the relationship between students’ library visits in person, their online use, and various potential predictors (students’ age, years at the university and GPA).
Results
To test if three factors (age, years, and GPA) significantly predicted participants' library visits in person, multiple regression analyses were conducted. The results of the regression indicated that three predictors explained 9.4% of the variance (R2 =.094, F (3, 2082)=71.74, p=.000). It was found that students’ age, years at the university and their GPA significantly predicted students’ library visits in person (β = -.24, p=.000; β = -.072, p=.001; β = -.099, p=.000). To test if the same predictors significantly predicted students’ online use, multiple regression analysis was followed. The results showed that the three predictors explained 15.5% of the variance (R2 =.155, F (3, 1790) = 109.09, p =.000). It was found that both students’ age and their GPA significantly predicted students’ library visits in person (β = .368, p=.000; β = .063, p=.005); however, students’ years at the university did not predict students’ online library use (β = .023, p=.313).
Discussion/Conclusion
In sum, younger students with fewer years at the university and a lower GPA tend to visit the library in person, whereas older students with a higher GPA tend to use the online library. Older students’ years at the university did not contribute to the multiple regression model. Overall, the findings suggest that understanding library users’ characteristics can be effective predictors of their library use.
Aim
An increasing number of medical schools are teaching clinical reasoning skills to first- and second-year medical students. Our medical school introduced a three semester-length clinical reasoning course in 2015. The course uses a problem-based learning, facilitated small group approach. Prior to their small group sessions students must complete online pre-work in identifying probable diagnoses. Students must support each diagnosis with the specific information resources that they used for supporting evidence. We sought to identify those information resources that students valued most highly halfway through their second semester.
Methods
Retrospective cohort study. We presented a list of possible information resources on a survey to medical students administered during their second semester. We asked them for additional details in some categories such as for specific textbook titles. We then queried them on which sources they valued most highly. The researchers expanded a list developed by Eldredge et al (2016) for use originally with rural practitioners in creating their own survey instrument. These results were cross-validated using aggregated, de-identified usage data from the library.
Results
Our results will be reported at EBLIP 10. We anticipate that students will have become far more sophisticated in their choices of information resources, tending toward more authoritative textbook and website sources. We do anticipate a slight uptick in reported usage of the primary literature in the form of original research articles.
Methods - using electronic media to collect evidence (Session C, 3:00pm-4:30pm)
Deborah Charbonneau. The e-Delphi Method: Opportunities for Soliciting and Analyzing Expert Opinions
Aim
To help build the research skills of librarians for evidence-based practice, it is important to understand how to optimize e-research methodologies to collect data from groups. The Delphi technique has traditionally been used to solicit opinions from experts on a number of important issues. The e-Delphi method leverages technologies to gain insight from diverse groups of experts in the digital environment. The objective of this presentation is to provide an overview of the e-Delphi method to solicit, collect, and analyze expert opinions online. More specifically, the presentation aims to 1) describe the e-Delphi method, 2) highlight considerations when conducting e-Delphi research, and to 3) provide examples of applications in practice.
Methods
The presentation draws on the author’s experience of conducting an e-Delphi study with a panel of health experts using multiple data collection rounds. It is anticipated that lessons learned about conducting e-Delphi studies, along with considerations for conducting e-Delphi research, will be shared with attendees with the goal of raising an awareness of the e-research method.
Results
The e-Delphi method can be a useful technique for collecting and analyzing data online from diverse and geographically distributed groups. This research approach can be used in future-oriented planning, policy development, and decision-making in libraries. The overall logistics of conducting e-Delphi studies, managing iterative rounds of expert feedback, and the strengths and weaknesses of the e-Delphi approach will be discussed.
Discussion/Conclusion
The presentation supports the conference themes of preparing information professionals to thrive in complex information environments and building skills for evidence-based practice. In particular, the presentation advances conversations about e-research methodologies in libraries, builds knowledge of the e-Delphi method, and illustrates opportunities for e-Delphi studies to gather expert feedback. Librarians and information specialists wishing to enhance their research skills or wishing to learn more about the e-Delphi method as a way to collect meaningful data from groups will find this presentation to be of interest.
Erika E. Smith and Richard Hayman.Meaningfully Mixing Methods for Large Social Media Datasets
Aim
This presetation outlines strategies for intentionally combining qualitative and quantitative research to meaningfully understand and use evidence from large social media datasets. Though quantitative methods are often assumed to necessarily drive large projects, we discuss how qualitatively-driven mixed methods approaches can also provide valuable insights. Presenters and participants will engage with the following question: How do mixed methods research (MMR) methodologies help us to better represent meaning in large social network datasets to respond to complex information needs in uncertain times? Informed by the literature (e.g., Creswell & Plano Clark, 2018) and drawing our recent study of undergraduates’ online information behaviours (Authors, 2018), we illustrate the benefits and challenges of mixing methods for large-scale social data.
Methods
Methodological considerations for designing large-scale MMR studies will be illustrated using a recent social media research case study. Focusing on evidence-informed recommendations connected to practice, presenters will demonstrate strategies for research design, data-mining, software, and collaboration. The case study reflects our application of these approaches in the context of an exploratory MMR content analysis of 2,712 posts to a public, university-oriented Facebook page over a full academic year.
Results
Methodological results include meta-research outcomes, including research design elements resulting from our study of a large-scale social dataset. The illustrative case study links these meta-research items to practical examples. We will show the ways in which MMR approaches provided additional insights into the complex nature of online interactions, enabling researchers to investigate not only what, how, and when students sought anonymous help online, but also the range of learning help they needed.
Discussion/Conclusion
Qualitatively-driven approaches that place meanings at the core of research projects can be beneficially combined with well-known quantitative methods to reveal important insights within mixed datasets. MMR strategies can help those undertaking assessment and evaluation not only to describe results, but also to discover relevant meanings that might otherwise be invisible in large studies relying solely on quantitative measures.
References
Authors. (2018). [Anonymized for peer review].
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Los Angeles, CA: Sage.
Tuesday, 18 June
Value, impact, and outcomes (Session D, 11:00am-12:30pm)
Clare Hemsworth and Andrew Givan. The Scottish Attainment Challenge in Renfrewshire school libraries: using evidence based programmes in school libraries to help close the poverty related attainment gap.
Aims
The school libraries attainment team use library based literacy interventions to help close the poverty related attainment gap. All work is evidenced to support two main aims: increasing positive reading attitudes and skills development. This programme links into the University of Strathclyde’s ‘Renfrewshire Literacy approach’, and ‘Vibrant Libraries, Thriving Schools’, a national strategy to show the impact of school libraries in Scotland.
Methods
Multiple projects are run across various primary and secondary schools. Projects include reading support, information literacy sessions, EAL support and the development of library spaces in primary schools. A targeted approach is conducted with pupils meeting a clear justification for being included or meeting the criteria of the Scottish Attainment Challenge with the aim of improving reading attitudes or skills development. The impact of each project is recorded on an individual pupil, project and authority wide level using standardised evaluations. Evidence collected is varied but includes surveys pre/post intervention which are based on the reading agency toolkit (https://readingagency.org.uk/news/blog/reading-outcomes-framework-toolkit.html), and attendance monitoring along with observational changes to attitudes. The attainment team work closely with the core school library service to provide an integrated offer.
Results
Currently in year 2 of the project. All interactions with pupils are recorded and evidence collected is submitted alongside other literacy workstreams. Research methodology was changed at the end of year 1 to improve evidence collection as non-standardised evaluations had proved ineffective. This change has allowed greater analysis of results to be made in year 2 and has enabled even minimal changes in attitudes to be recorded ensuring a strong bank of quantitative and qualitative evidence is collected on the impact of school libraries. Results for year 2 are expected to be available late June 2019.
Conclusion
The school libraries attainment team is part of a wider approach to close the attainment gap including interventions related to literacy, numeracy, wellbeing and transitions. The programmes have demonstrated a positive impact on reading and information skills. It would be beneficial for further research to be carried out into how library engagement complements other attainment approaches and makes a significant contribution to raising attainment.
Pip Divall and Cathryn James. The impact of the Clinical Librarians in the NHS: findings of a national study
Introduction
The evidence on the impact of Clinical Librarians (CL) has previously proved limited with varied evaluations of the clinical librarian services. Previous systematic reviews have found limited evidence on the impact of the CL service and existing research has discussed the difficulties of measuring impact and the poor quality of reporting. Brettle et al (2016) was the largest UK evaluation of the CL service across health care organisations, this study sought to build on that work at a national level.
The Quality and Impact Working Group (QIG) of Health Education England (HEE) set up a Task and Finish Group in late 2016 to undertake a national study looking at the impact of clinical librarians within the NHS. The group includes representatives from the NHS regions in England.
Method
A mixed-methods approach was used to evaluate CL impact. A qualitative approach was taken with an online questionnaire, followed by semi-structured interviews. The questionnaire was adapted from the toolkit by Brettle et al (2016) study. Strategic Leads invited libraries in England, Scotland and Wales to participate in distributing questionnaires to their users who had received a literature search, journal club or training. An email was sent to all users with a link to an online resource named SmartSurvey for responses. Interview participants were randomly selected post the questionnaire returns.
Results
The number of returned surveys was 697, giving a response rate of 35%. The number of participating organisations supporting this project is 41, with 8 undertaking the interview phase. In total, there were 25 interviews with clinical staff. Transcriptions of the interviews will be analysed using a thematic approach for supportive evidence on impact.
Conclusion
CL services were shown to be timely, making an impact on healthcare outcomes, patient care experience, leadership and supervision, improved clinical collaboration and improving patient and staff safety. Findings were broadly in line with the earlier study by Brettle et al (2016) and add to the existing evidence base around clinical librarianship. Our study confirms previous results and shows more detail of the complexity of the CL role.
Maria Beatriz Marques and Leonor Calvão Borges. From the management of assumptions to the management of evidence: the evaluation of the outcomes of Information Systems
Aim
The main objective of this study is to analyze the concept of competitive advantage of organizations based on evidence and correspondent financial and social sustainability of Information Systems for its stakeholders. This qualitative approach, which emerged in the 1990s, focuses on the study of the market, and aims to demonstrate the value and quality of organizations in the community, depending on the impact they have on individuals.
Methods
Based on the theoretical framework, where fundamental concepts related to the new management models are approached and defined, we describe some of the most recurrent cost-benefit evaluation methods in international studies, used to assess the economic and social value of Information Systems, including Return on Investment (ROI), Social Return on Investment (SROI), Contingent Valuation (CV), and the Multiplier Effect. In order to confirm and/or refute this change in the Public Services vision, a qualitative research methodology was used, with a case study in two public libraries selected from the central region of Portugal - a Municipal Library (Anadia) and an Itinerant Library (Proença-a-Nova).
Results
The results of the research carried out emphasized the need to ally the symbolic power of these thousand-year-old institutions to the real and effective needs of people / citizens who every day seek their "products / services", sensations and emotions for their social, human, economic and environmental. Anadia’s Library Return on Investment case study has allowed us to conclude that the more a Library is used, the more profitable it becomes. As for the results of the Social Impact of the Itinerant Library, the general conclusion drawn is that citizens consider it an essential service for the surrounding community.
Discussion/Conclusion
As a conclusion, we can say that the importance of evaluating an Information System results from the need to know the inputs that produce the outputs corresponding to the expectations of the stakeholders. It is also necessary to evaluate the impact or effect of these products and/or services on individuals and the community (outcomes).
Systematic reviews (Session D, 11:00am-12:30pm)
Lynda Ayiku and Jenny Craven. Developing validated geographic search filters to search efficiently and effectively for context-sensitive topics
Aim
Many organisations are facing resource constraints and are exploring methods to work more efficiently by reducing information overload. Using the National Institute for Health and Care Excellence (NICE) United Kingdom (UK) filters for MEDLINE and Embase (OVID) as an example, this presentation will discuss how geographic search filters can enable efficient and effective searches for topics that require research about a specific location.
Methods
The UK filters were created in accordance with best practice for search filter development through the use of a gold standard (‘GS’ - set of relevant references that is used to test how effectively search filters retrieve research). The ‘relative recall method’ was used to generate the gold standard. The GS was made up of references to UK studies which had informed a range of NICE guidelines and it was split into three sets; development, test and validation set. A word frequency counter (WriteWords) was used to identify relevant search terms and fields for the filters from the ‘development set’. Draft filters were then created. The ability of the draft filters to retrieve the references from the ‘test set’ was then examined and the filters were modified to improve their effectiveness. Lastly, the final versions of the filters were validated by calculating their recall (percentage of references retrieved by the filter) against references in the ‘validation set’.
Results
The MEDLINE UK filter achieved 99.5% recall of identifiable UK references (233 out of 234) and the Embase UK filter achieved 99.8% recall (560 out of 561).
Discussion/Conclusion
The filters have demonstrated their ability to retrieve evidence about the UK efficiently and effectively. The filters retrieve the vast majority of UK evidence for research topics and their application to literature searches reduces search result volumes because they restrict results to specific geographic regions. The methods used to develop and validate the UK filters are transferrable to geographic search filters for any location. There are currently only two other validated geographic search filters that retrieve evidence about Africa and Spain. It is hoped that this presentation will encourage the development of search filters for more geographic locations.
Amanda Click, Meggan Houlihan and Claire Wiley. Distilling the evidence: A systematic review of business information literacy
Aim
As librarians make the transition from the ACRL Information Literacy Standards to the Framework, we need more evidence that informs effective teaching of information literacy skills in the disciplines. This session shares the findings of a multimethod study that includes a systematic review and online survey, exploring how librarians develop the information literacy skills of business students. We will share new ideas and best practices in order to increase the value of the Framework for teaching librarians, support implementation in business information literacy, and consider applications for other disciplines.
Methods
The researchers first conducted a systematic review of the LIS literature related to information literacy in the business disciplines. This robust method supports evidence-based practice and is growing in popularity in the LIS field. These findings were used to design an online survey administered to librarians who support business students and faculty. Both quantitative and qualitative survey questions focused on practical examples of Framework implementation, as well as the development of best practices related to the implementation of threshold concepts in information literacy instruction, courses, and curricula for business students.
Results
The systematic review covered LIS literature on information literacy in the business disciplines published between 2000 and 2018. Analysis focused on research populations (e.g., MBA students, first year business students), topics (e.g., information seeking behaviors, instruction impact), and key findings and/or recommendations. Survey respondents shared perspectives on the Framework in business information literacy, including current practices and future plans.
Discussion/Conclusion
Findings will not only be of interest to business librarians, but to any librarians interested in integrating the Framework into their information literacy work in the disciplines. The goal of the session is to share new ideas from the instruction classroom and evidence-based practices from the literature, thus increasing the value of the Framework for all teaching librarians and advancing implementation. Especially when shared effectively, innovative practice benefits both those in the field and those that they support.
Aims
This paper presents the results of a systematic review and meta-analysis that aims to synthesize the findings library impact studies that examine the relationship between academic library use and instruction on students’ grade, retention and graduation outcomes. Using the results of this review and analysis as a starting point, this paper then discusses the benefits and risks of these types of studies to student participants and their institutions within the ethical framework of human subjects research standards and responsibilities.
Methods
Systematic review and meta-analysis.
Results
The systematic review indicated a need for increased use of effect size measures that evaluate the real-world impact of statistically significant findings. Without these measures, studies evaluating library impact on student outcomes risk overstating the library’s contribution or reporting relationships based on relatively small statistical differences that may not, in fact, exist (a Type I error). The results of the meta-analysis supported this argument by demonstrating in aggregate a very small over all effect of library use and instruction on student outcomes. Together, these findings suggest that researchers may need to reconsider the benefit of these studies given the risks students are asked to bear.
Conclusions
By conducting research that potentially puts students at risk-- even if that risk is small—library impact studies assume the ethical responsibility to conduct research that maximizes potential benefits and utilizes the highest quality methods available. Demonstrating a large effect would support tolerating a higher level of potential risk posed by connecting library data to student outcome data, because it provides greater benefit by generating findings that are more meaningful and support more confident decisions by library administrations. However, given the very weak aggregate effect observed in this review and meta-analysis, this research suggests that library impact studies’ present approaches may not be sufficiently meeting this ethical standard of beneficence. In order to address this problem, this paper concludes with recommended adjustments to data collection methods and measures that may provide a more accurate and rigorous demonstration of the library’s effect on students’ academic outcomes while also respecting enhancing students’ privacy and autonomy.
Planning and evaluating services (Session D, 3:00pm-4:30pm)
Christine Wolff-Eisenberg. Addressing Unmet Needs: Developing and testing services for important campus communities
Aim
How do you identify the needs of important campus communities? Once unmet needs have been identified, how do you develop and test service concepts that might effectively address them? How do you know when the library is the right provider of those services?
Methods
This session will detail how mixed-method approaches – including both semi-structured interviews and large-scale surveys – can be deployed in a variety of institutional contexts to uncover the practices, preferences, and needs of important campus communities. These mixed-method methods can also be used to test whether existing services are adequately meeting the needs of various campus communities. This session will illustrate successful utilization of these methods through a recent multi-year, cross-institutional, and collaborative research initiative aimed at understanding how students define success, what challenges they are facing, and what library and other academic support services might help them succeed.
Results
Research from this project has already yielded valuable insights on the practices, preferences, and needs of students. Based on these findings, we have developed and are currently testing via survey a series of service concepts that aim to help institutions better meet the needs of their students. These service concepts are designed to help students more effectively navigate college, develop their capacity as a member of their community and society, operate effectively and safely in today’s digital world, and showcase work from their classes. Results from these surveys will provide strong evidence for the community college and higher education sector to create new and evaluate existing services.
Discussion/Conclusion
In this session, methodologies and results from this initiative will be shared to illustrate the ways in which evidence - both qualitative and quantitative - can be effectively used in decision-making and advocacy for critically important communities. When attendees leave the session, they will not only have a better understanding of the goals, practices, and perspectives of these students, but they will also be able to examine mixed methods for assessing the needs of their own communities and redesigning and creating services to meet these needs.
David Morgan. Evidence gathering for project management – supporting planning, executing, and evaluating.
Aim
To examine and illustrate the benefits of effective evidence gathering for project management in libraries, to suggest that wider adoption of best practice would be of benefit to library practice, and to illustrate practical examples of these benefits.
Methods
Draws on a literature review and survey of formal project management practices in UK academic libraries, conducted as part of the author’s recent LIS master’s degree dissertation, with an examination of best practices and examples from the author’s and colleagues’ work on collection management projects at Royal Holloway, University of London.
Results
The author’s research reveals a mixed picture of approaches to project management in UK academic libraries. Some projects and libraries employ highly formal methods, including recognised project management methodologies such as PRINCE2, to capture, record, disseminate and reuse project data and lessons learned, while others show much lower use of formal processes, practices, and documentation. The results show a correlation between respondents’ assessment of their service’s project capability maturity, use of formal methods and documentation, and the frequency with which project success criteria are achieved. More formal methods tend to be associated with more successful outcomes. The research also examines the challenges to delivering successful projects, finding that issues of scheduling and estimating are amongst the chief concerns of respondents.
Discussion/Conclusion
Estimating is a core project skill, and essential to key project functions such as budgeting and scheduling. All estimating methods, but especially comparative (or analogous) and parametric estimation depend on accurate data from prior work.
Formal project management methods can encourage and support the gathering, recording, dissemination and reuse of important project data and contextual information. In times of uncertainty such data can be used in improved estimating for budgeting and scheduling of future projects, reducing waste and improving project outcomes, and supporting project proposals. Illustrated by recent examples from the author’s recent work and that of his colleagues.
Håkan Carlsson and Tore Torngren. General User Surveys and Other Methods for Quality Assessment – User Knowledge and Evidence-Based Library Development
Aim
A key component of library management is meeting the needs of our users. This requires continuous analysis of user experience followed by library development. The aim of this study was to better understand the use and effect of general user surveys (GUS) and other methods of gathering user input for quality assessment and improvement of library activities.
Methods
Data collection was done via surveys, which were sent out to the library directors of all 308 European LIBER university libraries. The response rate was 42%. After survey collection the free-text answers were coded and categorised and then the results were analysed.
Results
A majority of the responding libraries (77%) have performed a GUS during the last four years. Of these libraries, 54% reported changes to services based on the surveys. The most common method of obtaining user input was by placing a digital “user suggestion box” on the library website. The next most common method entailed using systematic processes to ensure continuous quality improvements.
Discussion/Conclusion
Nearly half of libraries develop their own survey instruments. Appropriate follow-up after the survey was important in order to assure library development. The key step was producing a written report, which increased success in changes to services by nearly 50%. Changes were observed in all traditional library areas, with most changes reported for the areas of physical library standard, communication/marketing and quality of information literacy education. The changes implemented were typically small steps leading to substantial improvements, rather than larger strategic shifts.
Small libraries have limited resources and difficulty arranging large user surveys. They reported higher usage of methods performed in the physical library.
The library strengths most valued by users were in the areas of the physical library, researcher support - especially in the area of scholarly communication - and information literacy education. User feedback helps build institutional profile and image. Analysis of the different methods revealed information regarding which methods give the best understanding of the users.
Publishing (Session E, 1:30pm-2:30pm)
Lorie Kloda, Denise Koufogiannakis and Virginia Wilson. Research for librarianship: A study of a decade of Canadian faculty publications
Aim
The objective of this research is to determine if, and if so to what extent, library and information studies (LIS) faculty research in Canada is related to the practice of librarianship. This research also examines in what topic areas LIS faculty are publishing. The researchers conducted a content analysis of journal articles published by current faculty at Canadian LIS schools in the 10-year period between 2008-2017. This paper reports on an update of a previous content analysis conducted in 2013 and presented at EBLIP8 of the literature from 2008-2012.
Method
A content analysis of journal articles published by current faculty at all eight Canadian LIS schools between 2008-2017 was undertaken. The databases Web of Science, Scopus, LISA, LISS, and LISTA were searched for publications by tenure-stream teaching faculty members, and data were extracted from the articles independently by two researchers.
Results
A total of 745 journal articles published in English or French were included for analysis. The findings will describe the set of research produced during the 10-year period, including: 1) whether LIS faculty at Canadian institutions publish research articles on topics in the field of librarianship; 2) which research topics the LIS faculty write about; 3) in which librarianship domains their journal articles fall; 4) how many LIS faculty co-investigate and co-author with librarians and information professionals; and 5) in which journals LIS faculty publish.
Discussion/Conclusion
The issue of whether or not LIS faculty are researching in areas that are significant to the future librarians they are teaching is a perennial one. The on-going perceived disconnect between research and practice – the research-practice gap – suggests that the research being conducted and published by LIS faculty is not meeting the needs of practitioners in the field. But is this true? The results of this study will provide answers to the questions above, demonstrating if research output from Canadian LIS schools is in fact related to librarianship, and on which topics.
Richard Hayman. Is it illegitimate evidence? A facilitated discussion on predatory publishing and EBP
Predatory publishing has disrupted our scholarly publishing environment, providing problematic venues that share research without rigorous peer-review. Predatory practices create tensions across disciplines and institutions, and can directly impact researchers trying to disseminate and advance their careers. One need only consult the countless editorials and opinion pieces from respected journals to get a sense of the perceived scale of the problem.
In this session, participants will discuss big questions concerning some of the effects that predatory publishing has on evidence based practice, such as:
- In what ways can librarians and information professionals support and inform our communities, including our researcher and educator colleagues, in recognizing predatory publication practices? And, are people open to these conversations and opportunities for training?
- Are library and information professionals truly in the best position to be advocates, educators, and gatekeepers for identifying suspect research?
- What are the implications or consequences when our communities draw on illegitimate evidence in their own research?
- What kind of damage is predatory publishing causing outside of academia?
These questions do not have simple answers, but engaging in a discussion of these key issues is an essential part of informing ourselves on the perils of predatory publishing for evidence based practice and related topics.
Digital services (Session E, 11:00am-12:30pm)
Karim Tharani. Of evidence and technology: How EBLIP helped safeguard an oral tradition
Aim
This paper is based on a case study that took an evidence-based approach to digitally curate an oral tradition for research and education at an academic library. The purpose of the paper is to share practical tips and insights gained during the course of the case study.
Methods
The researcher worked with members of the Ismaili community to gather evidence (source materials) pertaining to its oral tradition of ginans (gnostic and devotional hymns). The evidence comprising over 500 items were gathered and categorized into one of five categories, namely; Oral (audio recordings and renditions), Archival (manuscripts and lithographs), Canonical (printed publications), Communal (community resources), and Academic (research studies). The analysis of the evidence led to identification of over a thousand unique oral tradition works (OTWs).
Results
A key outcome of this research was the development of an evidence-based master index, which served as a scaffolding to develop a digital curation framework to digitally preserve and provide access to oral traditions using existing library infrastructure. In addition, the index served as a tool to provide online, integrated, and full-text access to the underlying evidentiary sources of the OTWs to facilitate research, teaching, and learning of the tradition.
Discussion/Conclusion
Oral traditions by their very definition and nature are alive and current, but there is a tendency in the western academia to regard them as static or historical knowledge. To this end, the researcher identified five specific areas for librarians to be cognizant of while working with oral traditions, namely; 1) Embracing plurality of sources and resource; 2) Accommodating foreign scripts and languages; 3) Moving from carriers to contents; 4) Working with works and variants; and 5) Revisiting notion of authority and authorship. The study demonstrates the possibility and practicality of how librarians can help privilege oral knowledge alongside western knowledge in libraries.
Kathryn Barrett and Sabina Pagotto. Local Users, Consortial Providers: Seeking Points of Dissatisfaction with a Collaborative Virtual Reference Service
Aim
Many libraries are reluctant to trust the research needs of their local users to external service providers through a collaborative virtual reference service. As a consortium providing a chat reference service, we collect and provide evidence on service quality to reduce partner institutions’ uncertainty. For this project, researchers examined whether our service model, staffing choices, and policies were associated with user dissatisfaction, aiming to identify areas where the collaboration is successful and areas that could be improved.
Methods
The researchers examined transcripts, metadata, and exit survey results from 437 chat interactions originating from 13 universities between June and December 2016. Transcripts were coded for mismatches between the chat operator and user’s institutions, and reveals of such a mismatch; user, staff, and question type; how busy the shift was; and proximity to the end of a shift or service closure. Chi-square tests and a binary logistic regression were performed to compare variables to user dissatisfaction.
Results
There were no significant relationships between user dissatisfaction and institutional mismatch, user type, question type, shift activity levels, chats initiated near the end of a shift, chats initiated when the service was closing, or reveals that the chat operator was working with other users. However, revealing an institutional mismatch was correlated with user dissatisfaction. Operator type was also a significant variable; users expressed the least dissatisfaction with graduate student staff, followed by paraprofessional library staff and librarians.
Discussion/Conclusion
The study largely reaffirmed the consortium’s service model, staffing practices, and policies. Users are not dissatisfied with the service received from chat operators at partner institutions, or by service provided by non-librarians. Current policies for scheduling, service closure, and handling shift changes are appropriate, but best practices related to disclosing institutional mismatches may need to be changed. This exercise demonstrates that institutions can trust the consortium with their local users’ needs, and underscores the need for periodic service review.
Meeting user needs in non-traditional communities (Session E, 1:30pm-2:30pm)
Joanna Hare and Mengjie Zou. Into the unknown: developing an evidence-based information literacy program for a non-traditional undergraduate program
Aim
This paper will describe how a Sino-foreign university library is developing an information literacy program to complement and enhance a brand new, undergraduate program. The program is being developed in a professional climate that embraces the new and unknown. At a very young university, with a campus under construction and a brand new faculty and student body, the curriculum of the undergraduate program presents another layer of complexity: each semester is only seven weeks long. This unconventional curriculum structure presented challenges the author had not previously faced in designing both embedded and general information literacy workshops.
Methods
A variety of methods were used to collect data to aid in the development of a balanced and meaningful information literacy program that will complement the seven week curriculum structure. The methods included curriculum mapping, a needs assessment survey of the inaugural undergraduate students, in-class questionnaires and faculty focus groups.
A variety of methods were used simultaneously to aid rapid data collection and allow the authors to implement the program incrementally, while at the same time allowing space for experimentation and change.
Results
In this paper the results of the various collection methods will be shared with a particular focus on the results of the inaugural undergraduate student survey, as well as how the information collected via the various methods were used to design and implement the information literacy program.
Discussion/Conclusion
This paper will be useful for people working on time-sensitive projects that need rapid data collection to maintain the momentum of the project. It will be relevant to people working on the development of information literacy programs, but also anyone seeking to apply simple, adaptable data collection methods to a unique and changing work environment.
Jennifer Zhao and Tara Mawhinney. Assessment of Multilingual Library Orientation Videos
Aim
Since 2015, McGill University Libraries has used multilingual orientation videos to engage its large non-native English-speaking student population. This paper will report on the findings from an assessment study of these videos, including (1) how effective these multilingual videos are in delivering library orientation to non-native English-speaking students and (2) how the various features used in the videos positively or negatively contribute to such effectiveness.
Methods
The study employed a checklist exercise and a semi-structured qualitative interview to gather data. Non-native English-speaking student participants were recruited and divided into two groups. One group watched the English video and the other watched the vernacular language video that was translated in the student’s native language. After viewing the video, participants were asked to complete the checklist, indicating when they learned about the services introduced in the video, and were interviewed with questions related to the video’ content, features used, and participants’ affective responses. All the interviews were audio-recorded and analyzed thematically. Data from the two groups were compared.
Results
Results of the checklist exercise suggest that the multilingual videos are slightly more effective than the English video in delivering the content, and the interview findings further prove the multilingual videos to be more successful in invoking positive emotions among participants. Most participants rated the quality of the video as high and were in favor of different features used in the video, but they had varied perceptions of the level of detail of the information provided. While many participants found the translated video to be easier to understand, they recommended embedding English subtitles and vernacular language closed-captioning in the English edition to help international students comprehend the video. Finally, the participants’ limited awareness of these multilingual orientation videos suggests a wider promotion to the university’s large multilingual community.
Discussion/Conclusion
These findings are useful for librarians who are interested in using videos to engage a linguistically diverse student population and may inform the work of other librarians who play a role in creating library orientation materials and promoting library services in general.
National initiatives for health libraries (Session F, 2:30pm-3:30pm)
Clare Edwards and Dominic Gilroy. The development and implementation of Quality Improvement Standards for NHS Library and Knowledge Services
Aim
To develop a new approach and set of quality assurance and improvement standards for NHS Library and Knowledge Services (LKS) reflecting the ongoing priorities of Health Education England's Knowledge for Healthcare Framework. The presentation will focus on the evidence based approach and evaluation methods used to used to underpin the development, lessons learned, and approach taken.
Methods
A Plan, Do, Study Act (PDSA) methodology was undertaken, supported by evidence review, piloting and evaluation.
An initial literature search to scope out the definitions of “quality” was followed by a series of focus groups with NHS LKS staff and stakeholders to identify perceptions relating to the former Library Quality Assurance Framework (LQAF) to identify areas of weaknesses with the aim of ensuring these were addressed in developing the new standards.
The project group consulted with Sharon Markless who’s work in developing outcomes- based development frameworks for libraries assisted us in focusing on an outcomes and impact approach for the standards development.
Following the development of seven new Quality Improvement Standards (QIS), twelve pilot sites trialed the standards during 2018 to test fitness for purpose.
A retrospect review of the pilot process was then undertaken to capture lessons learned.
Results
The study process confirmed that the new QIS were developing into a workable tool and that the outcomes approach to assessment will support service evaluation and improvement.
Areas identified for further enhancement included the guidance provided with the standards, a review of some duplication and overlap, and a focus on further development of the levels used to assess progress. The need for training for those using the new standards also became clear.
Conclusion
The use of a PDSA approach and the results of the trial indicate that the draft NHS LKS QIS are on track to meet the objectives of the project. Following further review and refinement of the standards and guidance a launch is planned for April 2019.
Cecily Gilbert, Kathleen Gray, Kerryn Butler-Henderson and Ann Ritchie. Digital health and professional identity in Australian health libraries: Evidence from a census of the Australian health information workforce
Aim
This paper presents insights into the health library workforce, using evidence from the 2018 Australian Health Information Workforce Census, and compares this workforce with other health information occupational groups included in the Census.
Background
The health information workforce overall is being affected by digital transformation in the health sector. This workforce is poorly defined in general, and specialisations within it such as health librarianship are often invisible to those responsible for digital health implementations and the consequent quality and safety of patient care.
Methods
The Census was available online during May 2018 for anyone who self-identified as part of the health information workforce and worked in the Australian health sector. 1,597 usable responses were received, 14.4% from health library respondents. Descriptive statistical analysis and deductive thematic analysis were used to identify their comparative demographic profile, geographic location, employment, experience, qualifications, continuing professional development, and interest in further training.
Results
Initial results [1] indicate that health library respondents are more likely than other health information occupations to be credentialled and to hold membership of specialist professional associations. Research continues to explore the implications for this part of the health information workforce of its distinctive characteristics and roles, sense of cohesiveness and sense of the future.
Discussion/Conclusion
Following on an Australian health library workforce census by Health Libraries Australia in 2012, this broader Health Information Workforce Census provides timely evidence about the changing nature and context of health library work. We need this information as a basis for the workforce planning, training and advocacy that will enable such work to adapt successfully in relation to other health information workforce roles and functions, and to contribute to the contemporary digital health environment. We encourage international adoption of the Census tool to strengthen global evidence about the status and prospects of health library work.
________________________________________
[1] Butler-Henderson K and Gray K 2018, Australia’s health information workforce: census summary report 2018, University of Tasmania.
Graduate student needs (Session F, 2:30pm-3:30pm)
Hilary Bussell, Jennifer Schnabel and Amanda Rinehart. To Collaborate or Not to Collaborate: Subject Librarians Re-Examine Graduate Student Outreach
Aim
Academic librarians across institutions continue to face times of uncertainty. Our bandwidth for in-depth engagement with faculty and students is affected by shifting priorities in the library and on campus. Subject librarian roles, in particular, are continually redefined. We are not alone; graduate students and their departments are also experiencing high levels of uncertainty. Job prospects have dwindled, and there is a growing push to provide graduate students with skills that prepare them for careers outside of academia. How should subject librarians strategize to support the evolving and broadening research needs of graduate students while functioning within their own uncertain professional landscape?
For the past two years, we organized a graduate student workshop series to introduce a variety of research support we offer across disciplines and to highlight teaching and writing support provided by our campus partners. However, attendance has been low, and we are concerned that the series is not worth the organization, preparation, and promotion time. This has led us to the unconventional hypothesis: that it might be time to stop working across campus and instead focus on our own silos. When is it more efficient and impactful to not collaborate on graduate student outreach?
Methods
We conducted two evaluative surveys after the 2017 and 2018 series as well as attendance evaluation. We will conduct a purposive sample survey targeting graduate students and program coordinators in our respective subject areas in the social sciences and humanities in early 2019.
Results
Preliminary results indicate that, although graduate students were unaware of many library services, the information that they found most useful could have been better conveyed and tailored to their discipline by their subject librarian. Subject librarians were specifically mentioned as helpful resources. We will have data from the purposive survey we conduct in early 2019.
Discussion/Conclusion
We will consider whether an individual, discipline-specific focus is more beneficial and impactful for communicating with graduate students than a broader workshop series. We will also interrogate whether formal evidence acquired through surveys is actually any more useful for informing our practice than anecdotal data and personal experience.
Joe Lenkart. Building Long-Term Research Relationships: Reference Services for Graduate Students
Aims
This study investigated the research experience of graduate students and their interactions with subject librarians and faculty advisers. Through semi-structured interviews with graduate students, this study gained insights into how graduate students mediate between reference services, subject specialists, and faculty advisers. Research questions include:
How do they mediate between short-term and long-term research strategies recommended to them by faculty advisers versus librarians? Are these strategies contradictory or complimentary in nature?
What initial strategies or steps do they take before consulting reference services or subject specialists?
What library services are effective in their view?
How do they evaluate sources and resources in their respective areas of specialization?
Are there any gaps in their research skills set in terms of locating resources beyond their institution, language comprehension, and academic terminology associated with their areas of specialization?
What roles (if any) do librarians and faculty advisers play in supporting their research projects and pre-thesis/pre-dissertation research?
Research libraries and academic departments invest enormous amount of resources in designing or redesigning services and programs, and hiring professional staff to support graduate students. The intent of the project is to improve the research experience of graduate students, library research services and address potential areas for improving graduate advising.
Methods
Study participants were recruited at library service points, academic department/intradepartmental graduate students, listservs, and university-wide listservs. Interview audio files were transcribed for thematic analysis.
Results
Preliminary results show a need for a hybrid model of advising for graduate students. This model would see subject librarians and faculty advisors working with graduate students for the duration of their graduate program.
Conclusion/Discussion
Graduate students are one of the most important patron groups for reference services at academic libraries. Throughout their academic career, these emerging scholars form partnerships and relationships that have a lasting impact on their overall graduate career. Two relationships in particular are instrumental (and sometimes lack thereof) for shaping their research experience: relationships with librarians and faculty advisers. Understanding these research relationships is critical to user services.
Collections (Session F, 2:30pm-3:30pm)
Jaclyn Mclean, Dede Dawson and Charlene Sorensen. Communicating Collections Cancellations to Campus: Qualitative Evidence to Inform Practice
Aim
In recent years, academic libraries across Canada have experienced severe budgetary challenges due to de-funding, currency instability, and the unsustainable rising costs of e-resources subscriptions. All of this is taking place in a volatile scholarly publishing environment. In these uncertain times, many libraries are being forced to make major collections cancellations decisions, and the focus is increasingly on the “big deal” journal packages. At some institutions, these cancellations attract considerable media coverage and negative backlash from faculty against the library, but at other institutions libraries receive support from their campuses. How can libraries effectively communicate cancellations so that their campus communities understand and support these decisions? The aim of this study was to collect qualitative evidence to answer this question.
Methods
We conducted semi-structured interviews with 17 librarians from large research libraries across Canada. Each of the participants had some experience with major collections cancellations decisions and the communications of these decisions to campus stakeholders at their institutions. The interviews allowed participants to reflect on what worked well and what didn’t in their communications strategies. We analyzed the transcripts using qualitative coding methods to reveal recurring themes, lessons learned, and advice for others facing similar situations.
Results
Recurrent themes from the data include practical strategies, but also higher level thoughts on topics such as stewardship, building trust, and the importance of tying collections decisions to broader issues in scholarly communication.
Conclusion
Cancellations communications are becoming commonplace for academic libraries in these times of uncertainty. Typically, these communications are reactive in response to circumstances driven by external forces. We hope that the evidence collected in this study supports libraries in the preemptive development of effective and strategic communication practices that results in support and understanding from their campus communities. Furthermore, we anticipate that the results of this project will encourage libraries to raise the awareness of faculty and administrators about the challenges of the current publishing ecosystem and their role in it – therefore advocating for a transition to a more sustainable scholarly communication system.
Pearl Herscovitch and Madelaine Vanderwerff. If you build it, will they (really) come? Proximity and Other Factors Affecting Education Students’ Use of Mount Royal Library's Curriculum Collection
Mount Royal University Library relocated to the newly built Riddell Library and Learning Centre (RLLC) in the summer of 2017. The new integrated facility offers exciting new technologies, tools and service opportunities for the library and brings together several building partners, including the Education Department, with the intention of fostering student and faculty teaching and learning. Many education courses are scheduled in the new building in close proximity to Mount Royal University Library’s Curriculum Collection.
A one-time fund for a Curriculum Collection was granted in 2011 after a bachelor degree in elementary education was approved at the University. Ongoing collection funds have also been allocated since the start of the degree. A core collection became available in 2013 and the collection continues to grow, but usage stats reflect minimal use between 2013 and 2017. We hoped that the change in the Education department’s proximity to the curriculum collection would result in an increase in use. A mixed survey was deployed consisting of both quantitative and qualitative questions to students enrolled in third and fourth year courses in the winter of 2017 (before the move) and winter of 2018 (after the move). We also analyzed circulation statistics to determine trends in use of the curriculum collection. We examined the extent to which proximity, as well as other factors, such as comfort, enhanced workspaces and openness increased use of the Curriculum Collection.
Our analysis of student responses and circulation data indicates an increase of use post move to the RLLC. The data also reveals connections between a student’s declared minor, characteristics of the collection and commutability to their practicum placement. This paper will address whether improved library facilities, proximity and other factors have a measurable impact on library use, exploring the unique challenges of specialized collections or branch collections within a larger academic library setting. The presentation should be of interest to academic branch librarians, education librarians and those interested in the impact of library design on collection use.
Emerging roles for librarians (Session G, 4:30pm-5:30pm)
Jordan Nielsen. New Venture Librarianship: Practical Implications from the Entrepreneurship Librarian Job Market
Aim
Entrepreneurship has emerged as a major force on university campuses. It has influenced educational programming, research output, and even commercial activities connected to universities. Universities have put infrastructure in place to support entrepreneurial engagement and success. This infrastructure include spaces and services such as business incubators where students and faculty can seek mentoring, funding, equipment, and other resources to support new business ideas. Additionally, universities have technology transfer offices where researchers can access resources that will help them spin off discoveries into commercial enterprises. University libraries are also contributing to the entrepreneurship infrastructure by creating entrepreneurship librarian positions. Often, these entrepreneurship librarians liaise with offices and units that make up the entrepreneurship infrastructure and provide research support to faculty, staff, and students engaged in entrepreneurship activities. What conditions need to exist in order to support entrepreneurship within a university? How are libraries contributing to this entrepreneurship support? This poster will use evidence from the job market for entrepreneurship librarians to shed light on entrepreneurship activities in the university environment and highlight the ways in which libraries are crafting positions to respond to these activities. Finally, this poster will discuss the practical implications of this research for librarians/libraries seeking to engage entrepreneurs on their campuses.
Methods
A content analysis of North American academic entrepreneurship librarian job advertisements will be performed. Job advertisements will be collected from multiple job boards and listservs, and they will be coded and analyzed to identify patterns in position descriptions, expectations, skill requirements, and institutional contexts.
Results
The results of this research will reveal commonalities in the approaches libraries are taking to provide support for entrepreneurship on their campuses. Additionally, the results will highlight activities and perceived information needs of entrepreneurs in the university environment.
Discussion/Conclusion
The research presented in this paper will serve as a guide for libraries seeking to engage entrepreneurial communities on their campuses. It will highlight common areas where support is needed, the types of services provided, and the institutional contexts in which entrepreneurship librarian positions have been created.
Tom Hudson and Liz Walton. Can a searcher become a screener? Evidence of a search team’s capability to perform title/abstract screening of literature search results for NICE common infections evidence reviews
Aim
To test whether information specialists (IS) who undertake literature searchers could perform title/abstract screening of results to a comparable standard and timescale as the analysts who currently perform the screening. Also to investigate the IS’s attitude to this extension of their role.
Methods
Testing was carried out prospectively on a National Institute for Health and Care Excellence (NICE) common infections guideline topic. IS “test” screening took place in parallel to the analyst’s “real” screening, to allow comparison of screening quality in terms of include/exclude decisions on which articles to screen further in full text, plus time taken. All were blind to each other’s work. A senior medicines adviser provided advice as needed. Each IS first screened a number of small batches of records, received feedback on how their results compared to the analyst, then discussed any discrepancies with the medicines adviser, thus allowing the opportunity to improve their screening skills. The ISs were asked to rate confidence in their screening skills during the process, and to comment on screening as an extension to their role.
Results
Mean agreement on all coding decisions was 93%. Agreement on the key measure of which papers to include for further screening was 92%. The IS team on average completed screening faster than the analyst in the final larger batch, with 116 compared to 108 records screened per hour. The IS team’s rating of their own confidence in their screening skills increased during the process, though enjoyment of the task decreased slightly! The IS team made several recommendations to facilitate the success of any future screening.
Conclusion
Positive results allowed the decision to be made that the IS team would extend their role to take on routine title/abstract screening of results for this programme. The IS team’s recommendations on support for screening were implemented.
Needs and behaviour of nursing students (Session G, 4:30pm-5:30pm)
Diane Ingram. User Needs – User Gets: Developing academic and information literacy skills modules for nursing students at an Australian University, using an evidence-based approach for student success
Aim
During 2016 and 2017, data sourced from academic skills drop-in sessions identified a developing need by entry level nursing students at Edith Cowan University. The result was creating an online series of self-paced academic skills modules available at point of need. Hosted on the SpringShare LibGuides Platform the learning support team source data as contributing evidence that having academic and information literacy skills has a positive impact student success and retention.
Methods
The inbuilt statistical functionality of the LibGuides Platform produces quantitative data of the views per module, with the inclusion of quizzes and surveys within modules providing qualitative data. The end of semester 2, 2018 sample review of assessment results where then compared to results received by students in previous years, who had not had access to the modules. When synergised with the other learning support activities, this data contributes to a comprehensive picture of the valued added.
Results
Launched in Semester 1, 2018 the modules have been accessed 36,697 to year end, by a cohort of 3,135 students. The quiz results aim to evidence assurance of learning with survey feedback used to improve the modules and meet the ongoing developing skills needs.
Initial comparisons between previous year’s assessment results and those for 2018, along with Unit coordinator feedback, indicate an increased use of credible information in assessments and improving academic writing skills. Initial discussions with coordinators suggest a positive impact on the number of students passing the assessments. Some slight changes to the data collection in 2019 aim to further evidence this hypothesis.
Discussion
As the Higher Education sector moves to a performance-funded model with a heightened focus on student success and retention (Norton, 2018), Almquist (2014) suggests the academic library must shift to a bio-directional marketing process. User needs are identified, services developed and feedback obtained to determine how effective the library addressed the need. The academic skills online modules is one example of how a collaborative academic library identified and delivered on a user need, while simultaneously collecting data to demonstrate a relationship and value correlation to student success (Gaha, Hinnefeld, & Pellegrino, 2018).
Peter Stokes. Profiling the information seeking behaviour of nursing students
Aim
Finding information is necessary for students studying at university. Qualified nursing students are expected to know how to locate up-to-date evidence to administer evidence-based practice. Information seeking is a vital skill for these students and as the body of nursing literature grows, so does the need for students to attain and develop their skills to search effectively. Individual characteristics impact on the way users seek information, with personality, self-efficacy, and learning styles three of these attributes. Stokes and Urquhart, (2011; 2015) investigated the information seeking behaviour of nursing students with the aim of creating a profile of these students to enable information skills training to be tailored to their preference to create a better learning experience. The research used Foster’s (2005) non-linear model of information seeking along with self-report scales on personality (Saucier, 1994), self-efficacy with information literacy (Kurbanoglu, Akkoyunlu and Umay, 2006), and learning styles (Entwistle, 1997) to form a questionnaire. Data generated seven distinct profiles. These profiles however, were created from a single study and it is prudent to replicate the research to determine their validity and generate more robust findings. The aim of this study is to replicate the quantitative aspect of the original research (Stokes and Urquhart, 2011; 2015) to determine which of the relationships exist and thus the validity of the profiles.
Methods
175 students have been invited to complete an online survey which will be analysed using Chi-square, odds ratios and logistic regression.
Results
So far 51 students have responded to the survey and initial results show a significant relationship between Learning Style and Self-efficacy with information literacy, and strong links for Deep learners with Eclecticism, Keyword searching, Browsing, Serendipity, Picture Building, Reviewing, Identifying keywords, Incorporating and Verifying; Strategic learners with Keyword searching, Browsing, Chaining, Serendipity, Problem definition, Reviewing, Identifying keywords, Knowing enough, Refining, and Sifting; and Surface learners with Networking. There are also links between Deep learners and the Openness trait; Strategic learners with the Agreeableness, Conscientiousness and Emotional stability traits; and Surface learners with the Extraversion trait.
Discussion
The final results of the study will be disseminated at the conference.
Assessment (Session G, 4:30pm-5:30pm)
Elizabeth Brown and Maureen Rust. Creating reason from results: interpreting student assessment data at scale
Aim
The rationale for this research stems from a need to develop and offer standardized information literacy instruction to all first-year university students and maximize its effectiveness. While often viewed as unequivocally valuable by librarians, others in the academy may be less convinced of a need to teach information literacy skills. In this research, student assessments prior to and after their instruction sessions were used to draw conclusions about student learning, as well as inform changes to future instruction sessions. Discussion of these results highlight student information habits with strategies for interpreting instruction assessments at scale.
Methods
During the 2017-18 academic year, 1,007 first-year students participated in an online pre-assessment on information literacy knowledge prior to attending an in-person library session. Student assessments from the 43 sections that completed the pre-assessment were then compiled and analyzed to look for trends in knowledge or knowledge assumptions.
Results
Analysis of this assessment resulted in a number of discoveries, some of which included: students generally understand the value of reliability, but are less sure of applications; satire is difficult for students to detect; fewer students used their secondary school library than one might assume; student perspectives on the stages of the research process vary widely and many students brushed over the need for external sources. This paper will discuss other conclusions from the research.
Discussion/Conclusion
Finding the meaning within thousands of data points requires creative interpretation and a willingness to let go of data that forces assumptions not based in evidence. The research discussed in this paper will report on an ongoing instruction assessment project that influences new iterations every year. Authors will share methods of data collection, analysis procedures, and interpretation of results.
Sally Bell and Fiona Tinto. Developing an eResources evaluation tool at the University of Strathclyde
Aim
This project aims to develop an evaluation tool which can be used for academic and research library eResource products. Emphasis on usage figures in relation to collection management can restrict the conversations around the value of a product. This tool aims to collate in a single location key information about eResource products and the library’s evaluation commentary and decisions.
Method
Data required to fully consider the overall value of an eResource was identified as being supplier provided, user experience related, or related to the library’s experience of supporting and managing a product. Staff from across the library were consulted to identify priorities for eResources relating to their service area. These formed the basis of a standard form which suppliers would be requested to complete. The UX elements of a resource, factors related to content coverage within our wider collection, and elements related to service delivery were used to construct an internal-facing evaluation template.
Results
Pilot results have been good, with suppliers positive about having a structured format for providing information. Consistency in how that information is delivered to the library removes the risk of information loss through failures in communication or staff absences. Library and Information Studies students will trial the internal UX focussed side of the tool at the start of 2019 with results from this to follow.
Discussion/Conclusion
As this is a work in progress results at this stage are not conclusive. It is anticipated that the tool will provide a consistent approach to eResource evaluation while also enabling suppliers to better understand the needs of their customers. Stronger evaluation, beyond looking at usage figures, should enhance our collections and enable staff to have a wider understanding of the products we purchase for our users.
Wednesday, 19 June
Academic libraries (Session H, 9:30am-10:30am)
Laura Rocco and Elise Feltman. Understanding How International Students Interact with Library Spaces, Staff, and Services
Aim
We will present the preliminary results of a currently ongoing study which aims to investigate how international students interact with library spaces, staff, and services, to be completed by May 2019. Libraries invest significant effort into designing orientation and training sessions, information literacy programs and workshops, outreach campaigns, and library space redesigns in order to support various student groups, such as for freshmen and graduate students. International students may have different and specific needs and challenges that should be considered when these projects are undertaken. We will use the results of these interviews to create evidence-based recommendations for more successful reference and instruction for international students. Research questions for our study include:
·How do international students interact with the library? How do they utilize library spaces, staff, resources, and technology? How do they describe these experiences?
·What positive or negative feelings do they attach to these interactions?
·What kinds of library training do international students receive while they are enrolled? Are these trainings facilitated through the University Library or through academic departments?
·Do international students feel that they can utilize library resources to be successful in their academics?
·Are there common barriers that affect international students who use the University Library?
Methods
We are currently conducting up to 20 semi-structured interviews with both graduate and undergraduate international students enrolled at the University of Illinois at Urbana-Champaign. Interviews will be recorded, transcribed, and coded for positive and negative values that describe experiences with the library.
Results
We will use the results of the data to identify commonalities among international students about how they perceive the quality of their interactions with library spaces, staff, and services.
Discussion/Conclusion
The University of Illinois Library does not conduct regular or updated assessment for the international student demographic, though they make up 22.6% of the current student body. It is vital that we obtain firsthand data about their user experiences in order to better shape future summative instruction assessments, larger-scale library assessments, and other campus collaborations and projects.
Jonathan DaSo and Jessica Barmon. Analyzing the Impact of Library Sponsored Tutoring on Student Success
Aim
This study will examine historical data from a library-funded tutoring program at Florida State University to determine if there is a significant impact on student performance and retention. Following previous studies on student performance and library usage, and the return of investment (ROI) in the library tutoring program, this study will examine actual and perceived impacts on students who make use of the library’s tutoring service and be used to inform future decisions regarding the program.
Methods
This study will examine two years of attendance data for academic years 2016 – 2017 and 2017 – 2018. This data will be compared to turnstile data from the same time period, examining the GPAs and rates of retention for students that make use of the library’s tutoring service versus those that do not. Further, the authors will be sending out a two-question survey to tutoring patrons to gather qualitative information about the perceived impact of the service on the students’ experience at Florida State.
Results
We expect results to be gathered by April, allowing for the data to be used in consideration of services being offered in the following fall semester.
Discussion/Conclusion
The results gathered from this study will be used in conjunction with the previously mentioned studies on ROI and library usage to gauge the actual impact of the tutoring program’s effect on student success and the benefits of the library maintaining the program. If impact is determined to be high, the library will need to consider how to best support and continue the program, and locate funding to continue growing the program. If impact is determined to be low or non-existent, the library will need to consider shuttering the program and developing stronger partnerships with external campus tutoring services. The study will offer an example of how libraries can approach tutoring services in their spaces, and to gauge whether the presence and funding of these programs benefit the goals and mission of the library.
Instruction (Session H, 9:30am-10:30am)
Veronica Wells. Using Participatory Design to Develop Strategies for the Library’s Information Literacy Program
Aim
Advocating for the importance of information literacy within our institutions of higher education can be difficult when budget cuts, challenges with student enrollments and retention, and other competing priorities are at the forefront of our faculty, staff, and students’ minds. How do we get the attention of our stakeholders and how do we get their input? At the University of the Pacific, we adopted a participatory design approach, which actively seeks to involve all stakeholders, to inform them about the results of the SAILS Test and to gather feedback on their perceptions of information literacy.
Methods
Librarians at the University of the Pacific administered the SAILS Test to 368 senior students in 2017-2018. A representative from the library and the University Assessment Committee disseminated the results in meetings with stakeholders and held a forum to gather feedback on the SAILS Test results. The forum especially proved to be a valuable event. Approximately 40 faculty, staff, and students viewed select questions from the SAILS Test and discussed what aspects of information literacy is most important to them and their respective disciplines. The feedback gathered from all events is being used to develop new strategies and initiatives for the library’s information literacy program.
Results
From the SAILS Test we learned that our students scored a little higher than students at our peer institutions, transfer students score significantly lower than non-transfer students, and students who reported writing more papers scored higher. Through the participatory design activities, we learned that stakeholders are most concerned with students’ ability to evaluate sources and develop a search strategy.
Discussion/Conclusion
Using the feedback received at the meetings and forum, we will be focusing our information literacy curriculum on evaluating sources and developing search strategies in our teaching and online tutorials. In addition, we are exploring new ways of supporting transfer students, as well as developing a media literacy initiative, which will include lesson plans and activities that can be used for by faculty and/or librarians in the classroom.
Britt Foster. Faculty Information Literacy Teaching Practices: A Data/Methods Triangulation Approach to Investigating Instruction Independent of Librarians
Aim
At a large, public university in the United States, the desire to provide students information literacy (IL) instruction in higher order IL skills and concepts is challenged by low librarian-to-student ratios. In addition, while library literature contains many studies examining faculty perceptions of the value of librarian-led IL instruction, there is little evidence regarding IL instruction practices of disciplinary faculty independent of librarians. In a climate of uncertain budgets, increasing student enrollment, and increased conversation around the need for IL, media, and digital literacy skills, this study aimed to investigate a little-researched area of the IL instruction, learning, and development milieu.
Methods
In collaboration with the institutional research office, a data and methods triangulation approach was used. A survey of disciplinary faculty was administered; disciplinary faculty focus groups were also conducted. Student outcomes and annual assessment reports, documents that describe teaching and assessment methods for courses across the university, were analyzed. Voyant, a text-mining tool, was also used to determine key phrases and terms related to IL in these documents.
Results
Results revealed that disciplinary faculty highly value skills and understandings affiliated with IL competency. Faculty provide the majority of IL learning opportunities independent of librarians, though these learning opportunities are generally provided through implicit, rather than explicit, methods. Pedagogical methods that may enable explicit practices, such as the use of standards and competencies, IL outcomes, and transparent lessons/assignments, are infrequently used.
Discussion/Conclusion
Evidence and findings from this study are being used to inform several initiatives. Collaborations with the institutional research office, members of the Library Subcommittee of the Academic Senate, instruction librarians, and the Center for Faculty Excellence are resulting in the development of new services, resources, and instruction models to support IL competency in students.
Gender and methods (Session H, 9:30am-10:30am)
Hilary Bussell and Tatiana Bryant. Lived Experience as Evidence: Reflections on a Qualitative Study of Gender in Librarianship
Aim
This study is a qualitative, phenomenological exploration of perceptions of career resources and obstacles for US library workers. We explored the ways in which our gender identities as women, men, or nonbinary workers impact our career paths in a female-majority profession. The aim of this presentation is to understand the value of the type of evidence we gathered through our qualitative study as distinct from other types of evidence, and what it can tell us about organizational culture and change. When living in times of uncertainty and flux, as libraries are today, the need for nuanced and varied data become more important; after all, when living in a time of change, we often need to ask new questions. Qualitative studies help to understand what questions we need answered.
Methods
We conducted 29 in-depth phone interviews with librarians from public, academic, and special libraries across the U.S. The data was analyzed using a constructivist grounded theory approach.
Results
From our analysis, we developed four overarching themes: Visibility and Connection to Library Users; Credibility and Presumed Competence; Lack of Awareness and Hyperawareness; and Being Your Authentic Self and Concealing Yourself. Many of our participants of color and nonbinary participants noted that being a visible minority was an important, if not radical, aspect of their professional work. Our findings provide a deeper understanding of how librarians with multiple marginalized identities see these identities as ways of connecting with communities, and in other ways see them as barriers.
Discussion/Conclusion
After discussing the results of our study, we will consider the value of qualitative, phenomenological data we collected in this study for professional decision-making. We will discuss what normally counts as evidence, and consider how lived experiences provide a necessary complement to the type of quantitative evidence that our institutions often value. Specifically, we will highlight how the type of data we gathered offers a nuanced, surprising, and at times counterintuitive picture of how gender intersects with race, sexuality, and class privilege in librarianship. We will conclude by considering how our findings can be translated into decision-making at interpersonal and organizational levels.
Emily Kingsland. Unexpected paths: Undercover feminist pedagogy in information literacy
Aim
Feminist pedagogy in library instruction presents a new approach to actively engaging students in the research process. While feminist pedagogy in universities found early adoption in the late 1980s and early 1990s, it is a newer phenomenon in library instruction, finding its early roots in works by Ladenson (2009) and Accardi (2010).
By fostering active engagement and critical thinking skills, feminist library instruction sessions encourage students to question authority, actively participate in the knowledge production process, and become aware of their power and information privilege as they navigate increasingly complex information environments. At its core, this specific pedagogical approach subverts traditional classroom dynamics by focusing on diversity and inclusion.
This literature review presents the feminist pedagogical approach in academic libraries and examines how this particular instruction style has been assessed. This paper is intended for those who wish to build an evidence-based approach to their practice by demonstrating how feminist pedagogy can be applied to library information literacy instruction sessions.
Methods
Practitioners of feminist pedagogy draw on techniques and methodologies designed to emphasize and value different experiences, such as cooperative learning, collaborative learning, inquiry-based learning, and inquiry-guided learning. These techniques and methodologies are used to develop students’ information literacy skills, to take ownership of the research process, and to stimulate critical inquiry.
Case studies, book chapters, literature reviews, handbooks, research papers, surveys, and papers based on statistical and qualitative analysis are being consulted. Their methodologies are being evaluated and conclusions of their assessments will be tabulated to determine if the evidence supports practicing feminist pedagogy in the academic library classroom.
Results
Preliminary results indicate that while many librarians lack familiarity with feminist theory, feminism writ large influences academic librarians’ professional practice (Schroeder & Hollister, 2014). However, librarians who focus on feminist pedagogy face obstacles in their teaching, which has resulted in gaps in pedagogical and professional discourse within library and information studies literature (Fritch, 2018; Hackney et al., 2018).
Conclusions
Forthcoming.
Value and impact (Session I, 10:45am-11:45am)
Cheryl Stenstrom and Natalie Cole. Measuring the Value of California's Public Libraries
Aim
The goal of this project is to identify the unique value framework (social and financial) offered by public libraries, and subsequently garner the perceptions of Californians and government decision makers of the facets of the framework.
Methods
The study design includes a multi-part literature review, a survey of a representative sample of Californians, interviews with decision makers, and an analysis of library programs and services across the state.
Results
It is anticipated that the results will reveal the presence or absence of the alignment of the value framework with decision makers' priorities, Californians' preferences of public expenditure priorities, and the activities and services provided by the state's public libraries.
Discussion/Conclusion
The results of the study might be applied in practice in different ways, e.g., as part of the messaging that used with local decision makers when considering requests for library support, and as confirmation of libraries' activity and services with stakeholders' priorities. The conceptual impact of this study could result in the development of a reliable, clearly defined and unique value framework for public libraries in California and other jurisdictions in North America.
Alyson Tyler. “Books and banter”: evidence from Scottish mobile library users of the service’s impact on their lives
Aim
This paper explores qualitative evidence from mobile library users across Scotland regarding the difference the service makes to their lives. It considers the impact made on a number of areas including literacy, health and wellbeing, social isolation and community engagement and considers how service managers can use this impact evidence for advocacy purposes as well as for enhancing the service. When many public services are under threat, gathering impact evidence can help show the wider value of a service.
Methods
The Scottish Library and Information Council (SLIC) commissioned research to review mobile library provision in Scotland and to gather evidence of the impact mobile libraries have on people and communities. The research comprised a literature review, online survey with library service managers (with quantitative and qualitative questions) and a sample of follow-up depth interviews, a survey of mobile library users and a fieldwork day on a mobile library.
Results
There were over 300 survey responses from mobile users. Analysis showed that using the mobile library service has a positive impact on people’s health and wellbeing, finances, learning and development, their sense of community, and also reduces feelings of isolation. The research also found significant appreciation by users for the friendly, helpful and knowledgeable library staff operating the mobiles. The findings can be used by service managers to demonstrate the beneficial impact of mobile libraries across a number of policy areas, and to consider potential new partners for collaboration.
Conclusion
This evidence of impact from users raises points for discussion around the role and future of mobile libraries. Mobile libraries have a positive impact on users, and contribute towards many local authority objectives, as well as to national strategies. The evidence can be used by service managers to illustrate the value of the service to other managers and policy makers, including for advocacy purposes.
“I wait to see the yellow van wind its way down the … hill and know it is full of treats. It lifts my spirits and makes me smile.”
Collecting users' experiences qualitatively (Session I, 10:45am-11:45am)
Robin Bergart and Juliene McLaughlin. Can user experience research be trusted?
We have had a user experience research team in our library for a decade. Over this time we have had a continuous dialogue with our colleagues in our library about the rigour, validity, and trustworthiness of our research studies. Do our findings accurately reflect the experiences of our user communities? Do our studies pander to the neoliberal agenda of treating students and faculty as customers seeking a ‘delightful experience’? How much do our own biases taint the results of our research? This paper does not report on a particular research study; rather we want to use this time to reflect on the various critiques of user experience research in libraries, and how we are becoming more critical of our practice in order to deliver better evidence to our colleagues and to the library administration. As most of our studies rely on qualitative research methods, part of our reflection is on the place of qualitative research at a time when quantitative data is privileged.
Laura Spears and Bess de Farber. Assessing Collaboration Workshops: Revealing Hidden Values of Unsuspecting Library Stakeholders
Aim
This study examines data from multiple Collaborating with Strangers Workshops (CoLABs) to explore the use of Appreciative Inquiry (AI) as a mode that facilitates action research. Integrated with the AI approach, action research offers generative capacity for knowledge building and a rigorous inquiry mode for effective examination of social organizations. By directly participating in an agency-building process, participants conducted peer-to-peer interviews as three-minute “speed meetings,” combining the aims of action research with the participative/reflective model of AI, and promoting themselves and their assets from which substantial amounts of qualitative data and social insights could be efficiently extracted.
Methods
CoLAB workshops break down barriers and provide a structured environment where participants connect during one-on-one, three-minute speed-meetings, using profile cards produced during the workshop to practice 1) conversing with others; 2) discovering untapped resources; and 3) initiating cooperative, collaborative, or mentoring partnerships. At the end of these conversations, participants share reflections of the experience. This study examines the AI method of data collection in three different contexts including collaboration building for a university journalism department with a large number of new faculty; understanding librarians as agents in a regional library cooperative’s strategic planning process; and, examining the assessment knowledge, skills and abilities of an academic library’s professional faculty and staff. We analyze the participant profile cards and the reflective responses specific to each workshop, generating collaborative outputs and stakeholder profiles for these three unique purposes.
Results
The library cooperative workshop revealed the assets of the library professionals in the region, generating 765 statements about participant assets, desires and intent to act. The college department workshop is in the analysis phase and the assessment CoLAB workshop will be conducted and analyzed in early 2019.
Discussion/Conclusion
User assets emphasize the access and use of a community of practice’s (CoP) expertise, knowledge, skills and other resources already available but hidden to all but those who possess them. In addition, the safe environment created by the workshop facilitator was revealed as a hidden asset, producing social capital within the CoP by interactions that generate trust, reciprocity and cooperation in uncertain times.
E-resources and social media (Session I, 10:45am-11:45am)
Richard Hayman, Erika E. Smith and Hannah Storrs. "I’ve never felt so done with school before": Connecting information needs and behaviours from students’ Facebook Confessions
Aim
What can interactions on Facebook Confessions pages (FBCs) tell us about student learning needs? This research examines the information behaviours of undergraduates using an anonymous university FBC page by 1) evaluating social information sharing and help seeking behaviours related to undergraduate learning, and 2) exploring the academic needs as expressed in students’ posts. This presentation will highlight the relationships and patterns between four information behaviours and students’ specific information needs. Understanding these relationships will inform evolving evidence-based practices for instructors, librarians, counsellors, and others
Methods
Using an exploratory mixed methods research (MMR) methodology, researchers conducted a mixed content analysis (Hamad et al., 2016) of 2,712 posts made to a university’s FBC page during one academic year. Qualitative content analysis focused on identifying latent and emergent meanings reflected in information behaviours. Quantitative content analysis used descriptive statistical procedures to identify trends and patterns of information behaviours and their relationship a range of learning needs.
Results
Approximately 26% of Confessions supported a range of peer-to-peer learning exchanges, much higher than found in previous studies. Analysis revealed four information behaviours (help seeking, advice giving, information seeking, and moderating behaviours) related to seven emergent learning themes (academic standing and performance; administrative and technology processes; courses and studying; programs and degrees; student identity and emotion; tuition and finances; and university transitioning). Results demonstrate notable relationships between the types of information behaviour exhibited and students’ particular learning needs.
Discussion/Conclusion
While FBCs enable rich peer-to-peer academic help and information behaviours, those in official post-secondary roles should use caution when considering whether to engage in student-driven social media. Presenters will outline findings and implications for those in campus support roles, including ways to foster digital literacy for social media environments within and beyond the formal information literacy curriculum.
References
Hamad, E. O., Savundranayagam, M. Y., Holmes, J. D., Kinsella, E. A., & Johnson, A. M. (2016). Toward a mixed-methods research approach to content analysis in the digital age: The combined content-analysis model and its applications to health care Twitter feeds. Journal of Medical Internet Research, 18(3), e60. https://doi.org/10.2196/jmir.5391
Andrew Asher, Kristin Briney, Abigail Goben, Kyle M. L. Jones, Michael Perry, Dorthea Salo and M. Brooke Robertshaw. Do students care? Student perspectives on personal data use in library learning analytics
Aims
This presentation will discuss the findings from phase one of a 3-year study examining student perspectives on library participation learning analytics initiatives at 8 higher education institutions in the United States. This study specifically asks students to share their opinions and experiences with regards to privacy, autonomy, and the use of their personal data. The presentation will also briefly cover the next two stages of the study.
Methods
Semi-structured qualitative interviews analyzed using a modified grounded theory approach.
Results
This presentation will focus on results from two core themes:
1. Conceptions of privacy as they relate to learning analytics and data use.
2. Awareness and reactions to libraries’ participation in learning analytics.
While the data is currently under analysis, common discussions across the interviews indicate that students know very little about the data collection practices by universities and academic libraries. However, further discussion of principles of data collection and use revealed ambivalence and skepticism toward widespread learning analytics practices, as well as nuanced opinions about the costs and benefits of learning analytics data.
Conclusions
The efficacy of learning analytics depends in part on an institution’s ability to connect campus information systems—including those under the purview of libraries—to aggregate and analyze student data. As institutions continue to surface granular data and information about student life, the risk to student privacy grows and the tension on professional ethics commitments increases.
Librarians value user privacy and intellectual freedom, and the profession has historically been averse to sociotechnical practices that track user behaviors in information systems. Given the movement towards learning analytics, a need has emerged to research issues related to data ethics, and information privacy and policy. One key to answering these questions is discovering how students perceive these practices and how they impact their privacy and autonomy. Extrapolating from this data, we will make recommendations regarding information policy, and data and research practices to help participants engage in conversations in these areas at their organization.
Accepted Poster Abstracts
1. Elisabeth Nylander and Margareta Hjort. PhD students and information literacy in the health sciences: A scoping review
Introduction
In the current socio-cultural climate of “alternative facts”, the need for rigorous research is greater than ever. Doctoral studies offer a unique phase in the development and legitimization of researchers, in which PhD students shift from the consumption to the production of knowledge. If librarians are to support this process in an evidence-based manner, it is essential to understand the distinct practices of this population. While recent literature reviews exist concerning the information practices of graduate students and researchers, there is little knowledge synthesis which focuses solely on the information literacies of PhD students within the health sciences.
Aim
The aim of this project was to explore the depth and breadth of evidence which describes the information literacies of students pursing a doctoral degree within the health sciences.
Method
In order to investigate the research activity concerning PhD students within the health sciences, we performed a scoping review (Arksey & O´Malley, 2005). Relevant literature was identified through strategic searches in databases, hand-searched key journals, and reference lists. References were screened independently by both authors based on pre-determined inclusion criteria. General trends within the literature were mapped based on the extraction of the following data: geographic location, population, academic discipline, and method of investigation.
Results
Many studies fail to treat doctoral studies as a unique process. PhD students are often grouped together with other graduate students or researchers. Studies tend to be based on small populations, and the number of PhD students involved is either unclear or only equals a few individuals within the entire group of study. In addition, of the limited number of studies which focus on PhD students only, few conduct explicit examination of information practices within the health sciences. The result is that PhD students in the health sciences are underrepresented as a distinct group within the recent literature.
Conclusion
This review highlights the need for more primary research on the information literacies of PhD students in the health sciences.
Reference
Arksey, H., & O'Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19-32. doi:10.1080/1364557032000119616
2. Doreen Bradley, Angie Oehrli, Soo Young Rieh, Elizabeth Hanley and Brian Matzke. Advancing the Reference Narrative: Assessing Student Learning in Research Consultations
Aim
As reference services continue to evolve, libraries must face this challenge by making evidence-based decisions about our services. In order to determine the value of reference services provided through consultations, this study sought to assess student learning that happens during research consultations.
Methods
Researchers conducted interviews with librarians who provided the consultation service and also developed an online survey for students who had a consultation in the 2017-2018 academic year. Interviews were conducted with 8 librarians. Online surveys were sent to 38 students, and 20 surveys were completed, resulting in a 53% response rate. Researchers focused on the questions such as Do students achieve their self-identified learning goals through consultation with a librarian and Do they successfully apply these knowledge and skills to their course projects? Librarian interviews were coded for level of student’s prior research experience, librarian learning objectives, librarian’s perception of student learning outcomes, and consultation success. Student surveys were also coded for demographic data, student learning objectives, student perception of learning outcomes, and student perception of success.
Results
Analysis indicated that 70% of the consultations were for class-related projects; 93% of the students scheduled a consultation because they needed help finding sources; approximately half of students also wanted help learning to use library databases; and about a third of students wanted help understanding library organization. Overall, 86% of students were able to apply something they learned during the consultation to their project; and 50% were able to apply something they learned to other academic projects. 70% of students believe the consultation significantly improved the quality of their work, and 100% of students believe the consultation met their expectations. Additional specific student learning outcomes will also be shared.
Discussion
This study offers intriguing opportunities to compare librarian perspectives with student perspectives. Overwhelmingly, the data demonstrate that individual research consultation are highly effective in meeting student learning needs. As reference models continue to change and reference desk usage declines, research consultations are a valuable element in our service model and an efficient use of human resources.
3. Judith Logan and Kyla Everall. How would users prioritize library services in a Hierarchy of Library Needs?
Aim
To test a theoretical framework for allocating financial and staffing resources according to users’ priorities.
Methods
In a forthcoming publication in Weave, we developed a hierarchy for prioritizing user needs that uses the structure of Maslow’s Hierarchy of Needs:
Library as Minimum Viable Product
Library as Convenience
Library as Connector
Library as Incubator
Community as Library
We hypothesized that library services and resources would be placed on this hierarchy according to how fundamental they are to other parts of the library, the degree to which users are affected by them, and the scope of users in the community who access them. This model was intended to help libraries dedicate resources to services in proportion with their placement in the hierarchy. To test this model, we piloted a study where we provided users with a representative sample of library services and asked them how often they use the services and how it would affect them if these services were not offered. We analyzed the results by sorting the services into the five tiers of the Hierarchy of Library User Needs. We created a composite summary of the responses and compared these to the services the authors hypothesized would belong in each tier to show the differences between library staff members’ perspective of the library and it’s users’ perspective.
Results
The study has not yet been completed. Results will be presented at the conference.
Discussion/Conclusion
It is important to make sure that the resources and services that libraries dedicate the most energy and budget to are those that are most valued by our users. The Hierarchy of Library User Needs was designed to provide libraries with a framework to help align their offerings with their user community’s values. Testing this theoretical model with real users is the next step in determining if it can be used in practice.
4. Dominic Gilroy and Rebecca Williams. The development of a formal network for Library and Knowledge Services staff working in health both in the NHS and non- NHS within the Yorkshire and Humber region (YOHHLNet)
Aim
To develop a strong active self-supporting network for library and knowledge services staff working within the Yorkshire and Humber region based on the needs of potential members and utilising best practice, expertise, and proven method.
Methods
- Survey of potential member
- Best practice from elsewhere
- Focus Groups with skilled facilitator
- Proposals with feedback
Results
The Yorkshire and Humber Health Libraries and Knowledge Network (YOHHLNet) held its first committee meeting in March 2016. In the 3 years since then its achievements include:
- Active YOHHLNet committee with membership from NHS and non-NHS library staff. Governance provided through a constitution and AGMs
- Provision of a range of CPD events and study days, many in partnership with other organisations
- Bursary support for members attending conferences and other events
- Access to a library professionals books collection including a range of e-books
- Social media presence
- Strong local network with members from NHS, Higher Education, and Public Library services
- Number of members
- Healthy budget from membership fees and support
- Communities of Practice
Conclusion
YOHHLNet has been developed based on the requirements of its members and stakeholders and represents a strong and supportive network for those working in the field of healthcare library and knowledge services both within and outside the NHS.
The model used in the development of YOHHLNet is transferrable to others looking to establish and develop networks within the field of library, information and knowledge work and beyond.
Academic libraries subscribe to numerous databases and often invest in discovery systems, all in the quest to facilitate the search and retrieval process for students, faculty, and other members of the college community. Electronic resources, which already constitute a substantial percentage of acquisition budgets, will continue to grow. Therefore, academic librarians must manage these costly resources wisely. In an effort to consolidate our continuous evaluation of existing and new resources, the Berkeley College Libraries formed the Electronic Resources & Technology Committee, which was tasked with evaluating the often overlapping content within databases and making recommendations based on usable data.
Aim/Method
This poster will illustrate the Committee’s workflow as well as the content created in the form of evaluation rubrics and database comparison spreadsheets to assess the following: vendors, consortia license agreements, renewal dates, and coverage, and academic program alignment, notable features, overlap with free or existing resources, competing products, and ultimately relevancy with our constituency. Berkeley College is a private 4-year multi-campus academic institution with a library on each of its 8 campuses, which includes the online campus. The campuses also span two states which poses some challenges when reviewing contracts and access to specific databases.
Results
Based on the Committee’s evaluation tools, which provides us with up-to-date reliable data, we are able to make decisions that are more informed on which products to keep, which results in a strategic method to budgeting and planning. It also allows us to conduct more informed negotiations with vendors. In this information rich age, students can easily come upon unreliable sources if trusted databases are not easily accessible. Librarians need to stay one step ahead by continually evaluating their products to ensure they are providing the best possible resources. The hope is that fellow librarians will have some takeaways as to how to start a similar project, some tools to use, and to use their own data to aid in the decision making process.
6. Hanna Primeau and Katie Blocksidge. Generation Z: Facts and Fictions Revisited
Aim
The purpose of this study was to examine if Dervin’s ten information assumptions are present in Generation Z students from The Ohio State University at Newark; a secondary objective was to examine how the early use of technology influences students use of information.
Methods
This study made use of a survey instrument which was designed to examine the assumptions generation Z students make regarding information. Study participants were recruited through their University Survey course, a required first-year course; a link to the survey was placed in their LMS, and an email was sent to all class members with an introduction to the survey. Participation in the survey was voluntary.
Results
Data from 179 students, indicates that while Generation Z does partly align their information assumptions with previous generations, they are also starting to deviate from those assumptions in intriguing ways. More than half of students surveyed agreed that opinion-based pieces are useful, which contrasts Dervin's assumption that only objective information is valuable. Students surveyed who started utilizing a computer later for non-school related activities, between the ages of 11-15, do not on the whole believe that there is an answer to every question. On the other hand, students who started utilizing at an earlier age between 0-10, on the whole, do believe there is an answer to every question, it just has to be found. While this information is self-reported, it provides insight into how students view their own computer use.
Discussion/Conclusion
The trend of web resources being more visually more homogenized may have also played into the reason why most of our students agreed that is sometimes difficult to choose what information to keep when searching. Based on our data, Generation Z partially align their information assumptions with previous generations, but they are also starting to deviate from those assumptions in ways that have implications for how we design information literacy instruction. Discoveries like these should lead to changes in how we approach instructing our first years, and in how we think about students who we assume, often incorrectly, are the most technologically savvy generation.
7. Lisa Gardinier, Manuel Ostos, Austin Smith and Hilary Thompson. Using Consortial Data to Inform Cooperative Collection Development in Area Studies
Introduction
The Big Ten Academic Alliance (BTAA) Library Initiatives is a consortium of fifteen U.S. university libraries that collaborate to optimize access to the group’s combined resources. Cooperative collection development is an emerging area of interest, as evidenced by the BTAA’s 2017 conference on this subject, and area studies is a prime area for interinstitutional collaboration given its uncertain funding and support.
Aim
To better understand the consortium’s resource sharing needs for Spanish and Portuguese materials published in Latin America and to develop data-informed models for cooperative collection development of these publications.
Methods
The presenters developed a scalable methodology that uses ILLiad queries, Python scripts, Google’s Language Detection Library, and WorldCat Search API to gather and analyze interlibrary loan (ILL) and collections holdings data from member libraries. Data examined include:
•Request Type, Format, Publication Date, and Country of Publication for ILL borrowing requests; and within those categories, fill rate and reliance on the BTAA collective collection.
•Collection size by country of publication
•Rate of duplication among BTAA collections by country and call number
Results
At least 29,000 Latin American publications were requested by BTAA users via ILL in the past six years. Most of the requests were for loans of books (80%) requested by faculty or graduate students (91%) and filled by other BTAA member libraries (61%), demonstrating a strong reliance on the consortial print collection to meet research needs. Mexican literature, Mexican history, and Argentine literature emerged as the areas of greatest demand, while collections data analysis revealed high duplication in Cuban and Peurvian history. Multi-institutional collecting pilots for Mexico and Brazil are currently underway, and preliminary results for Mexico should be available by June 2019.
Conclusion
Given the uncertain funding and support for area studies, it is critical that academic libraries collaborate to build distinctive collections of international publications to support our users’ research. Leveraging ILL and holdings data to inform these collecting efforts allows librarians to build upon their institutions’ existing collection strengths, to address areas of collective need, and to minimize duplication among individual collections, thereby providing access to a broader array of global resources.
8. Lee Ann Fullington, Melissa Fuster, Margrethe Horlyck-Romanovsky and Susan Jakuboski. Evidence-based library instruction for nutrition research: Librarian and faculty collaboration
Aim
Undergraduate nutrition and dietetics students need to develop research skills and professional competencies to identify evidence-based nutrition and food studies research. Research skills are essential for student success in dietetic internships, graduate school, and future professional practice as nutrition and dietetics professionals. In a collaborative effort to improve student research skills, teaching faculty, the program director, and the liaison librarian developed a series of research skills one-shot sessions for an accredited undergraduate program in Foods and Nutrition at a public liberal arts college. Using a scaffolded approach, the library sessions were designed to help students build research skills essential to preparation for graduate school, dietetic Internships and to practice as health professionals. The program was piloted in Fall 2016 and presently continues.
Methods
The librarian reviewed nutrition curriculum map for all required nutrition courses; conducted content analyses of syllabuses and library related research assignments for projects; and facilitated formal and informal discussions with faculty and program leadership. The librarian developed appropriate workshops for the curriculum. For assessment purposes, the librarian sent a short survey to collect feedback from the students at the end of each semester. To assess evidence of library research skills at the end of the academic program, the librarian observed student capstone presentations.
Results
Five required courses in the program now include a specific library session that contributes to developing students’ research skills. Topics include online health information literacy, scholarly research using databases, citation tracking, and finding government data. Each lesson adds to a student’s toolkit and the assignments in the relative course incorporate their new skills. Observations of presentations indicate that students are adept at finding information using databases and data sources, and using discretion when selecting resources found through online search engines.
Discussion/Conclusion
Librarian and faculty collaboration is essential to ensuring that content of workshops is aligned with the assignments and course objectives. Reflective consideration of faculty and student feedback is crucial for librarian to revise and enhance workshop content. Research skills instruction scaffolded throughout a curriculum builds crucial professional competencies.
Aim
Booth indicates that question formulation drives the entire EBP process. Elsewhere, he notes a general lack of empirical research on question formulation skills (2006), as later confirmed by a 2010 systematic review by Horsley et al. We sought to improve on the existing training protocols for question formulation through a multiple year trial and error development process primarily based on anonymous student evaluations. Our training of first-year medical students included the use of a rubric for student self-assessment and for faculty members to assess students’ performance. Previous anonymous student evaluations had rated the question formulation training highly so we wanted to measure the degree of improvement of student performance in question formulation skills.
Methods
Prospective cohort study. Students beginning the course were presented with a clinical vignette and asked to formulate a question based on that vignette. Two weeks later they were trained in question formulation skills in a one-hour session coupled with a one-hour searching skills lab. They then were post-tested on another clinical vignette and asked to formulate a question. Students were assessed by three faculty instructors using the rubric that had been incorporated into the training. This rubric measured student performance on question formulation for both the pre- and post-test.
Results
Our results will be reported at EBLIP 10. We anticipate that students will improve their question formulation skills as measured using the same rubric by at least 20%. We expect to use a two-tailed t-test or a similar statistical analysis to compare the pre- and post-test scores on the same rubric.
10. Lee Ann Fullington, Jane Cramer and Mariana Regalado. Anecdotes, Barriers, and Cooperation: The ABCs of a Library/IT Collaboration
Aim
Our library’s three main computing areas are the primary computer labs on campus, with two of them managed by librarians. Our staff mediated sign-in system made for long lines, delays, and more than a few misplaced student ID cards. After several semesters of monitoring issues and securing funding, the library implemented a new sign-in system for computers in the library’s labs. This presentation discusses the types of evidence we collected and used over several stages of this transition to launch and continually improve the self sign-in process.
Methods
After selecting and deploying the new self sign-in software, library staff at computer labs collected evidence to help improve the process: logging student complaints and comments, observing traffic at sign-in stations, inviting IT staff to troubleshoot on the spot during busy times, collecting articles from student-run campus newspapers, surveying students, and using our own gut instincts to make decisions for quick fixes.
Results
By collecting data and documenting issues immediately, librarians and staff in the labs promptly engaged IT staff in ongoing monitoring and identification of problems during the first semester rollout. By being nimble in using evidence for making changes we could handle without IT intervention, the self-sign-in process became more efficient. For more complex issues, we used our evidence for communication with IT staff for resolution. The improvements we made in time for the beginning of the second semester of the roll out were met with approval from students and staff.
Discussion/Conclusion
By collecting several types of evidence, the librarians and staff were able to better communicate issues to IT staff for identification and resolution. Some decisions could be made quickly, but others required ongoing collection of evidence. By involving IT staff in our evidence collection, we developed a better understanding of how to communicate between our two departments in weekly meetings and in the moment in order to make the computer labs more efficient for students. As a result of this experience, we developed an online reporting form and established a triage hierarchy for reporting critical computer lab issues.
11. Jung Mi Scoulas and Sandra De Groote. Assessing the University Library’s Impact on Students’ Academic Performance
Aim
This study aims to examine if university students’ library experiences impact their academic performance. The current study used data from student surveys conducted in 2016 and 2018.
The institution is a large public research university in the United States. More than 30,000 students with access to four libraries. The total number of respondents was 1,087 in 2016 (response rate= .04), and 2,277 in 2018 (response rate= .07). The independent variables of students’ library visits in person and library resource use were coded as yes (1) and no (0). The dependent variable was students’ grade point average (GPA).
Results
Multiple regression analyses were conducted to test if university students’ library visits in person and library recourses use predict students’ academic performance (GPA) using the 2016 and 2018 student surveys. The results from the 2016 survey showed that students’ library experiences did not contribute to the multiple regression model, F (2, 941) = 0.73, p =.48. However, the findings from the 2018 student survey indicate that students’ library experiences significantly predicted students’ GPA, F (2, 2083) = 12.42, p =.000: students’ library visits in person (β = -.06, p=.000) and students’ library resource use (β = .09, p=.006). In other words, the average GPA of students who did not visit the library is 3.31. The average GPA of students who visited the library is 3.15, whereas the average GPA of students who used the library resources is 3.51.
Discussion/Conclusion
The goal of the current study is to test if students’ library experiences (library visits and library resource use) predict students’ GPA using the data from multiple student surveys. The findings from the 2016 survey showed that students’ library experiences did not predict their GPA. The findings from the 2018 survey indicate that students’ library visits were a negative predictor of students’ GPA; however, students’ library resource use was a positive predictor of students’ GPA. Given that this finding is contrary to earlier studies, further investigation is needed to better understand the negative relationship between students’ library visits and their GPA.
12. Anthea Sutton, Mark Clowes, Helen Buckley Woods and Andrew Booth. Articulating uncertainties: a comparison of questions asked by LIS practitioners participating in an EBLIP online course in 2010 and 2018
Introduction
The questions asked by library and information practitioners can provide a valuable insight into the nature of current uncertainties in the profession. As times become increasingly uncertain, do the types of question change and what can that tell us about current concerns?
Aim
To analyse the “burning questions” posed by a cohort of library and information practitioners, working in Australia and undertaking an online evidence-based library and information practice course in 2018, compared with an equivalent cohort in 2010.
Methods
Information Specialists based in the School of Health and Related Research (ScHARR) at the University of Sheffield developed and delivered an online course as an introduction to EBLIP. The course has most recently been run in collaboration with the Australian Library and Information Association (ALIA) in 2018. The previous iteration of the course was in 2010. Participants on the course identify an issue of current concern in their day-to-day practice and use this topic to formulate a research question, then identify and appraise the evidence to answer their question. The questions identified by the 2010 cohort were collated and categorised by EBLIP domain. This study will be repeated with the 2018 cohort and the findings will be compared to see if the questions posed have changed over time.
Results
The questions will be presented and will be categorised by EBLIP domain (management, information access and retrieval, professional, collections, reference enquiries, education). The questions will also be categorised by library and information sector to identify trends.
Discussion/Conclusion
In 2010, management was the most common domain of the library and information practitioners, with the recurrent themes of the EBLIP questions relating to efficiency, innovation, and service improvement. Is this the case in 2018? Are any domains now redundant, and are new domains to reflect the increasing uncertainty required? We will explore such issues in this study to provide a snapshot of the current concerns in the library and information profession in 2018.
13. Derek Boyle, Annette Thain and Gillian Flett. Delivering a national digital library in times of austerity
Aim
In any digital library service, a user-friendly discovery platform and access to online content are fundamental to ensure a service that is valued by its users. In this presentation we will highlight recent work undertaken to implement a new discovery platform and the procurement of online content and the processes used to capture and use evidence to maintain and deliver a quality service in times of financial constraint and limited capacity.
Methods
In March 2017, a new search engine was implemented to support the discovery of print and electronic resources via The Knowledge Network, the national knowledge management platform for health and social care staff in Scotland. This implementation enabled us to review and rationalise our current digital infrastructure along with a user-centred approach working alongside our network of NHSScotland library services, representing the voice of our users.
Post-implementation we continue to work with our network along with key partners in health and social care to identify and prioritise enhancements to our digital services. We have also taken an Agile approach to reviewing our infrastructure, improving our discovery platform and streamlining existing digital systems.
In the past year our team have undertaken the renewal of our online subscription content to identify resources that provide the best value for money, taking into account the needs of a range of staff and subjects. Key activities include:
• Gathering evidence through a survey, focus groups and the experience of others
• Reviewing relevant policy documents
• Horizon scanning for trends in delivering digital libraries
• Statistics of use
Discussion/Conclusion
Communication and involvement of the wider librarian network and users is essential. We must also consider current policy drivers and the needs of minority groups. There needs to be a transparent process between the evidence gathered and outcomes to ensure value for money and staff resources are used effectively. Lessons can be learned from other services. The Agile approach to redesign and delivery of digital products offers a number of benefits for the process, the team and our users.
14. Jaclyn Mclean and Elizabeth Stregger. Unaffiliated Users and Access to Academic Library Collections
Aim
To investigate how unaffiliated (walk-in, community) users access electronic resource collections at academic libraries. As library collections have changed from mostly physical collections to hybrid collections with rapidly growing online collections, community members and others who historically had equitable access to the collection by virtue of visiting campus have had that access restricted. In the increasingly complex environment of online scholarly publishing, the provision of access to unaffiliated users by libraries has become complex and difficult to manage.
Methods
A thirteen-question online survey was conducted in October and November 2018. Participants were recruited through listservs including CARL, Serialist, ALCTS, and COPPUL. Those who work in academic libraries in North America, and who have knowledge of library collections and technology, were encouraged to participate.
Results
Survey data will be visualized with Tableau. Participants from 128 institutions across North America provided details about networked connections, wireless access, and electronic resource license analysis in the context of unaffiliated users of their libraries.
Discussion / Conclusion: Libraries have implemented a range of solutions, from library staff temporarily logging in to computers in public areas to open wireless access for a refined list of electronic resources. There is a high level of interest in continuing this conversation as authentication practices and library spaces evolve.
15. Brianne Dosch. How to save assessment: documenting, re-evaluating, and implementing assessment after changes in funding and leadership
Aim
An IMLS funded grant, Experience-Assessment (UX-A), which focused on supporting MSIS students in developing UX and assessment skills across information environments, successfully concluded in May 2018. The success of the grant was measured through rigorous assessment including surveys, interviews, and focus groups. The grant team then wanted to take the successful elements of UX-A and implement them for a new groups of students. However, it became difficult to assess the success of the grant’s continuing efforts due to limited funding and the turnover of two program managers in one year. This poster is the summary and results of the current program manager’s efforts to document, re-evaluate, and implement assessment, while building upon previous assessments, and focusing on the sustainability of future assessments.
Methods
In-depth interview questions were developed from reviewing literature and past assessment efforts. Interviews were then conducted with current UX-A students and leadership team. The interviews were then coded for themes and analyzed to assess the current climate, and to decide which assessment methods to use moving forward for the sustainability of the UX-A program. Detailed documentation of all assessment efforts, both past and present, were pivotal in finding a cohesive and sustainable assessment strategy.
Results
Preliminary results show frustration and decreased motivation come from changes in funding, leadership, and intermittent and inconsistent assessment methods. This in turn limits the ability to demonstrate value, and to encourage a team and organization to participate in future assessment. Overall results are expected to be the need for consistent and detailed documentation, clear communication, and the willingness to re-evaluate assessment strategies (or the lack thereof) to be able to save past assessment efforts and propel them into a dynamic future of demonstrating impact and value.
Discussion/Conclusion
Hope is not lost when funding is lost, leadership changes, or buy-in has been lost from team members and organizations. If detailed documentation, clear communication, and rigorous relationship-building is at the core of new assessment efforts, any assessment can be “saved” and made sustainable for the future of library school and library programs alike (Sputore & Fitzgibbons, 2017).
Aim
In 2018 the Royal Society of Edinburgh awarded a grant to support Research Impact and Value in LIS (RIVAL). The aim of RIVAL is to create a network of around 30 Library and Information Science (LIS) researchers and practising library and information professionals who have an interest in maximising the value and impact of LIS research. The network members will address issues associated with the research-practice gap in LIS, e.g. needs for LIS researchers to produce accessible research outputs that are applicable in practice for the improvement of library and information services provision, and for LIS practitioners to engage with LIS research.
Methods
Four knowledge-exchange network events will be hosted. A Project Board will determine the content of the first three events. The fourth and final event will be determined by the network membership, with a proportion of time in the three earlier events dedicated to its planning. Thus the network members will be involved in the co-production of the RIVAL project. In addition an extensive open access online presence for RIVAL will be built.
Anticipated outcomes (results)
Amongst the anticipated outcomes of the RIVAL project are:
- New, and strengthened, relationships between LIS researchers within universities, and LIS researchers and LIS practitioners who work across a range of LIS contexts
- Increased confidence and self-efficacy amongst the practitioner participants as research users and partners
- Evidence of efforts amongst practitioner members RIVAL to integrate learning from the network into services provision
- Evidence amongst LIS researchers to incorporate research ideas inspired by the needs expressed by the practitioner community into future research bids
Possible future outcomes (more likely to become evident after the funded period) may include:
- Improvements in LIS services that can be traced back to knowledge exchange activities undertaken at RIVAL events
- Active collaborations between RIVAL network members e.g. joint project funding bids, co-authored publications, and other initiatives, e.g. service on committees and event organisation
Conclusion
The content of the poster will introduce the RIVAL project and the series of events to take place between July 2019 and July 2020.
17. Anna Brown, Natasha Chowdory, Bridget O'Connell and Amber Dunlop. CEBIS: Tell us what you want, what you really, really want
The Clinical Evidence Based Information Service (CEBIS) is a small team of Information Specialists based at University Hospitals Coventry and Warwickshire (UHCW) NHS Trust. CEBIS Specialists search and summarise relevant information in response to queries raised and facilitate appraisal of research evidence and its application to day-to-day healthcare practice, including patient management, professional development and research.
Aims
An exploratory study to:
• Investigate the information needs of clinical staff at UHCW NHS Trust
• Gauge demand for CEBIS across the organisation, and extent of the need for such a service in a time where funding is stretched and services are under scrutiny
• Assess the accessibility and usability of the CEBIS service
• Assess whether the service meets the needs of clinical staff requiring evidence-based information to support patient care, service development or research
• Find out whether there is a need for the CEBIS service to expand into providing information skills training.
Methods
The primary approach will be an information needs survey, piloted with 5-8 colleagues before sending out to the population (initially UHCW clinical staff, with the intent to adapt and extend to non-clinical staff such as managers and support departments in future). The survey will investigate the types and spread of information needs across the organisation. Results will be used to map demand for support in accessing and using evidence-based information against the current CEBIS offer. This will form the basis for future in-depth, targeted interviews and/or focus groups to dig deeper into reasons for non-usage and gaps in CEBIS provision.
Results
Results of the clinical staff information needs survey will be presented at the conference, alongside proposals for evolution of the CEBIS offer based on survey findings. Further results and ongoing developments will be shared with participants and the library and information community via UHCW Library website, Twitter and relevant publications.
Conclusions
The research findings will be used to advocate for continued investment in CEBIS. They will inform further research, as well as development of the service to better meet user needs and ensure accessibility, thereby providing an efficient service and continuing value for money.
18. Gillian Siddall and Hannah Woods. Challenging assumptions: Using evidence to scaffold students’ developing information skills.
Aim
To explore the previous educational experiences of first year students transitioning into Higher Education (H.E.), specifically looking at how the library can support them to progress in their degree.
Method
This collaborative inter-disciplinary research project builds on earlier research (Lumsden et al., 2010) and has been expanded to look at three undergraduate programmes: Early Childhood Studies; Childhood and Youth; and Social Work.
The research began with an online questionnaire to collect evidence about students' prior educational experiences and career aspirations. This paper will focus on the questions exploring students’ previous experience of libraries and study skills support. The research is ongoing and the next stage is a series of degree specific student focus groups to further explore key areas such as academic skills development and employability.
Results
Findings from the initial questionnaire in October 2018 showed that most students felt they had received good to excellent support on how to use a library during their previous studies. There was a more varied pattern of library usage. Many students felt they had received good to excellent support in how to search the internet.
The research highlights the varied educational experiences of the students. Therefore, it challenges the team to avoid making assumptions and to consider the varied experience of students when planning their interventions.
Discussion
We will discuss how we are using the evidence to inform our learning and teaching practice. For example, designing and delivering targeted information skills sessions based on students’ previous educational experiences.
The research is pertinent given the changing landscape of H.E., especially as the University has just moved to a new campus. The researchers thought it was a key point to reflect on their teaching and to make sure they are supporting and enabling students to succeed in their course.
Furthermore, the presenters will reflect on their experience of working with a large inter-disciplinary research team, with different agendas and levels of experience.
References
Lumsden, E., McBryde-Wilding, H. and Rose, H. (2010) Collaborative practice in enhancing the first year student experience in higher education. Enhancing the Learner Experience in Higher Education, 2(1), pp. 12-24. 2041-3122.
|
Aim
To make assessment a fully integrated part of operations in order to produce more data driven decision-making.
Methods
This poster will be a case study of integrating assessment and analysis of data into operational decision making at an Academic Health Sciences Library. It covers roughly the last five years and will illuminate multiple types of assessments conducted, types of data collected, and the inclusion of assessment as part of operations in nearly every area of the library.
Results
Nearly all operational decisions at the library now include some assessment or analysis of data. This data is documented evidence of both our good stewardship of funding and resources, and our commitment to good decision making for the benefit of our users.
Discussion/conclusion
East Carolina University’s Laupus Library was experiencing a shift in user needs, away from physical books and journals, and making justification of our physical space a priority. The planning and construction of a large student center next door to the library, with a gym, food options, and study space was a wake-up call that we needed to establish clear plans for assuring that our user needs were being met. We also needed to integrate data and evidence into decision making. Many decisions were at the time being made by department heads or committees of librarians who often assumed they knew what users wanted and needed. Conducting assessments showed us that what librarians think users want and what users actually want are often not the same. Our culture change began with establishing a User Experiences Committee and expanded to collecting more granular gate count data, mining collection use data from our catalog, collecting ID card swipe data, conducting Student Advisory Group meetings, and observational surveys to determine space use patterns among users. Establishing a culture of assessment and decision making driven by evidence and not assumptions or gut feelings is not easy, but the results are better and less risky operational decision making in nearly every area of the library from the website to hours of operation.
20. Natalie King, Judy Wright, Masyam Abdulwahid, Rebecca Randell, Joanne Greenhalgh and Justin Keen. How Information Professionals can support a realist review: a case study involving health information technologies (HIT) and patient safety.
Aims
RRs are a type of systematic mixed methods review, gaining popularity, that look beyond ‘what works’ to generate an evidence-based understanding of ‘what works for whom and in what circumstances’. RR’s can use multiple searches that support one another to uncover and test explanatory theory - unlike a single SR search. Flexibility and teamwork are key, along with lateral thinking in identifying appropriate resources and search techniques for each search iteration. Working closely within the research team provides a fantastic opportunity to develop communication and team-working skills while developing skills in using less traditional databases, supplementary search techniques and complex reference management.
Results
Search methods and databases used in the SafetyNet RR using RAMESES guidance were compared with those of two conventional SRs adhering to PRISMA guidance on the same topic. We reflected on our role embedded in the project team.
Methods
When designing public services or policies a realist review (RR) can help to identify what works best under different circumstances. We compared search methods used and the Information Professional’s role when undertaking a realist versus conventional systematic review (SR).
Discussion
All reviews used structured sensitive searches, similar date ranges and common patient safety terms. All searched Medline, Embase and CINAHL plus reference checking. Reviews showed wide variation in terms particularly HIT search terms. Key differences emerged; the RR conducted 12 searches throughout the review rather than one search at the beginning of the SR; the RR searched 17 databases compared with 3-4 in the SRs; only the RR searched grey literature resources; topics covered by searches expanded throughout the RR project unlike the SRs; Search precision varied between individual searches in the RR; Supplementary search techniques (forward citations, author searches, project searches) were performed in the RR but not reported in the SRs. Study types excluded from the SRs (letter, editorials, conceptual frameworks) were actively sought in the RR (for theory identification). Personal reflections noted a higher time commitment for meetings, searches and complex reference management for RR compared with conventional SR. Also, a much closer working relationship within the project team to plan multiple iterative searches.
21. Doug Knock and Alison Brettle. Can the impact of health libraries be measured routinely and robustly?
Aim
To validate a questionnaire to routinely measure the impact of health library services. Specific objectives are to measure internal consistency (reliability) and content validity amongst end users.
Methods
A short generic questionnaire has already been developed based on evidence from theory, the literature and best practice. Initial testing has demonstrated its acceptability for routine use across the UK NHS and that it captures a range of evidence of impact on NHS priorities. A 3 part mixed methods study will be conducted (convergent design). Part 1 will include statistical analysis in SPSS of a purposive sample of 100 questionnaire responses using Cronbachs alpha to determine internal consistency (reliability).
Part 2 will involve semi structured telephone interviews of a purposive sample of 15 users of the questionnaire and deductive-inductive thematic analysis of the responses to assess the relevance and clarity of the items on the questionnaire (content validity).
In part 3 the results will be combined and fed back to a stakeholder group to determine the final version of the questionnaire.
Results
The results of the validity and reliability testing will be reported at the conference, prior to publication of the questionnaire and a paper in a peer reviewed journal.
Discussion/Conclusion
A number of surveys and studies have sought to measure the impact of health libraries, but these have often been limited either in their rigour or the focus of the outcome data collected. This questionnaire seeks to overcome both these weaknesses and provide evidence of impact that can be used at either a local level for advocacy, and because of the unique way UK health libraries are organised and have to routinely demonstrate impact as part of quality assurance mechanisms has a potential to build a robust UK wide evidence base. To have academic acceptability, it is essential that as well as being acceptable and useful to library practitioners, it is also valid and reliable.