Abstract

Core facility laboratories are an essential part of the successful research enterprise of many universities around the world. Core facilities provide state-of-the-art instrumentation and technologies to support research of all faculty, postdocs, and students on a fee-for-service basis. Academic next-generation sequencing cores are typically “full service” facilities, and access to and training on their instrumentation is limited to core staff. To address these limitations, we provided graduate students with technical training at our core facility. We developed a 1-week noncredit-bearing workshop and recruited 6 graduate students (N = 6) as part of a pilot program. The program involved online teaching, classroom-based teaching, and hands-on training in next-generation sequencing library preparation and sequencer operation. A post-participation survey revealed highly positive outcomes in terms of skill development and increased awareness of technologies offered by the core facility. A workshop of this scale could be incorporated into the graduate curriculum and extended to core facilities that focus on other technologies. We believe that introducing formal standardized teaching spearheaded by core facilities would improve the graduate student curriculum and hope that this study can provide guidance on curriculum design for similar workshops.

Conflict of Interest: The authors declare no conflicts of interest.

A Human Subjects Protocol for the survey was submitted and deemed compliant by the Baylor College of Medicine Institutional Review Board.

ADDRESS CORRESPONDENCE TO: Daniel C. Kraushaar, Baylor College of Medicine, 1 Baylor Plaza, Houston, Texas 77030 (Phone: 713-798-7787; Email: [email protected])

INTRODUCTION

PhD programs in the biological sciences are meant to equip students with the skills necessary for solving their research questions and the fundamentals required for finding employment in academia and research but also, increasingly, careers in other sectors.[1] Universities in the United States and the United Kingdom have traditionally trained graduate students for careers in academia and industry through similar processes: coursework that must be completed before continuing to the dissertation phase and hands-on laboratory PhD research projects. In many European countries, PhD students work on their research projects during the entirety of their PhD program.[2] Learning outcomes of the PhD are driven by an increasing demand for workers with specialized knowledge and skills. Among core competencies that are expected to be developed through doctoral training, “technical skills” rank in the top 4.[3]

In the life sciences, graduate students that train at the bench must apply molecular biology techniques effectively to their thesis research projects in order to complete their PhD. Graduate students should know what tools are available to them and when to use them, and they have to learn the skills and protocols to apply them. Awareness of the availability of research tools and understanding their underlying principles and their applications are critical for students to achieve their research goals. The possession of sound technical skills is also important for career progression, especially to bench scientist roles in research and development or academic laboratories in which many PhD-level scientists are expected to use the technical molecular skills acquired during their PhD study.[4] Core facility directorship opportunities that typically require a PhD are also expected to continue to grow and fill openings with candidates that have a strong foundation in biotechnology.[5]

But despite the requirement for technical skills for PhD students, formal training and structured hands-on courses are still not commonly part of graduate programs.[2] In addition, acquiring skills and molecular techniques can be limited by the lack of “best practices,” a shortage of resources, limited access to high-end instrumentation, and outsourcing.

Students today rely increasingly on core facilities to support their laboratory work and/or data analysis. The mission of academic core facilities is to provide expert services and consultation. They act as support units, to which individual scientists outsource technology‐demanding projects that require expertise and instrumentation beyond that of the research laboratory.  Shared core facilities play a central role in fostering collaborations and interdisciplinary research, and they increase the productivity of research teams by providing scientists access to sophisticated technology and expertise.[6] They are equipped with advanced instrumentation, and their staff is highly qualified and experienced. Core facilities offer unbiased experimental design and analysis, and they promote transparency, rigor, and reproducibility as part of their operation. In addition to managing existing technologies, core facilities can also act as incubators for new innovations.[7]

Core facilities in the United States are substantially supported by the U.S. National Institutes of Health through Center Core grants and its Shared Instrumentation Support program that enables the acquisition of high-end instrumentation.[8] Core facility operational models range from “user-access laboratories” to “all-inclusive services.”[9],[10] In the case of “all-inclusive services” or “full-service” facilities, the researchers provide their samples to the core facility and receive the results. Core facility personnel perform the experiments on the core facility's equipment and may even analyze the data; no unassisted operation of equipment is typically provided. In the case of “user-access laboratories,” the researchers come to use the core facility's equipment on their own after they have been trained and advised by the core facility's technology experts. Core facilities take on important educational roles through providing consultation, training, and seminars to their user base and the broader scientific community.[11] Traditionally “all-inclusive services” cores such as many next-generation sequencing (NGS) core facilities are limited in the training they provide. Consequently, students and postdocs that pass on their samples for processing to a core facility may miss out on potentially beneficial technical skills and knowledge. It therefore becomes important for “all-inclusive services” cores as well as “user-access” cores that offer limited training to take on a larger role in student education by offering comprehensive technical skills training.  Hence, formal and mentored biotechnology-focused workshops spearheaded by core facilities may create a high-impact learning environment and may achieve effective learning outcomes through reinforcement of theoretical concepts. They may also strengthen a participant’s resume and lead to higher success in finding postgraduate employment.

To address the aforementioned objectives, we developed a 6-day workshop curriculum for graduate students. The objectives of the curriculum were to equip students with a basic understanding of the underlying principles of current NGS technologies with a strong emphasis on hands-on experience in sample preparation (NGS library construction) and sequencer operation, enabling students to perform quality control, library preparation, and sequencing independently and with good laboratory practices. Furthermore, we wanted to raise awareness of the technologies available to early career scientists at the core facility, provide access to high-end instrumentation, and elevate the educational mission of our core facility. The framework for the learning outcome assessment was provided in the form of quizzes and a poster presentation. The evaluation of the program was performed with the results from a post-participation survey.

MATERIALS AND METHODS

Pilot site

The study was conducted at the Baylor College of Medicine (BCM) in the Genomic and RNA Profiling Core facility and led by the Department of Education, Innovation and Technology. Among 19 applicants, 6 graduate students were enrolled in the workshop. The Next-Gen Sequencing workshop (https://www.bcm.edu/research/atc-core-labs/genomic-and-rna-profiling-core/next-generation-sequencing-workshop) was offered free of charge, and the course participants were selected based on the completion of an application that requested information on experience and a short justification for participation that each student provided. Participants with little or no experience in NGS were selected over more experienced students.

Curriculum design

The workshop curriculum was designed to incorporate both theoretical classroom-based content and hands-on wet laboratory training to graduate students (Table 1). This approach aimed to introduce learners to foundational knowledge while also providing opportunities for hands-on experience and scaffolding.

Two e-learning modules were developed in collaboration with the Center for Teaching and eLearning, Department of Education, Innovation and Technology, following e-learning course development best practices, using the backward design framework and including the incorporation of the 5Es of instruction: engage, explore, explain, extend, and elaborate.[12] The first module (Module 1) focused on the Illumina/Solexa and Oxford Nanopore NGS platforms, their underlying chemistries, and a comparison of NGS to Sanger sequencing. The second module (Module 2) centered around bulk RNA sequencing (RNA-seq), single-cell RNA-seq, and spatial transcriptomics chemistries. Each module provided learners with opportunities to explore the content and comprehend the concepts through detailed explanations and illustrative examples.

To gauge learners’ progress and understanding, both formative and summative assessments were integrated into the modules. The online lectures were created using the Articulate Rise 360 eLearning development tool (www.articulate.com) and were accessible through the BCM learning management platform Blackboard (www.blackboard.com).

 An instructional video on the operation of an Illumina NextSeq 500 instrument was created in collaboration with the BCM Audio–Visual Technology and Services. Our video was 11 minutes long and segmented into 6 major steps of the workflow. We paired the video directly with tasking each student to set up their own Illumina runs independently under supervision of our staff. The wet laboratory training was conducted in the Genomic and RNA Profiling Core facility laboratory and took place over 4 days. For every 2 students, 1 instructor was on hand, and each student was allowed to set up their own independent sequencing run (Module 4) with their prepared libraries (Module 3). The classroom-based portion focused on the analytical review of technical run performance (Module 5) and sequencing content (Module 6) quality checks. Technical run performance was reviewed with the Illumina SAV software (www.illumina.com) and sequencing content with MultiQC reports[13] that were generated for each run and included quality metrics on a per sample basis. Upon completing eLecture (Modules 1 and 2) and classroom modules (Modules 5 and 6), the students were required to undertake a summative assessment in the form of a knowledge quiz, each with around 30 questions.  Following the classroom-based session, the students were asked to review their own runs and present their results in front of the instructor and peers in a poster format at the end of the course (Module 7). The student posters were assessed rubric style based on the understanding of key concepts and clarity of presentation.

Survey and course evaluation

Course participants were asked to complete an anonymous online post-participation survey. All 6 students (100%) completed the survey. The survey included questions regarding program satisfaction and feedback on structure and content as well as course impact. The survey included Likert-type questions as well as open-ended questions. Considering the small sample size, no inferential statistics were performed. While the results are discussed in detail below, Table 2 and Table 3 list the mean and standard deviation for each of the Likert-type questions. A Human Subjects Protocol for the survey was submitted and deemed compliant by the BCM Institutional Review Board.

RESULTS

On a scale from 1 (high quality) to 5 (low quality), the participants rated the course components with overall positive scores (Table 2). The poster session received moderately positive ratings. Based on open-ended follow-up questions, this was mainly because of content redundancy across poster presentations. Although each student’s run generated its own data, the “introduction” and “methods” sections remained largely the same for each student and created repetition during the poster presentations.

Instructional videos are increasingly part of teaching practices and are considered effective educational tools when kept short, kept segmented, and paired with learning activities.[14],[15] As part of our survey, 5 out of 6 students felt that the video was very helpful (1), and 1 student felt it was somewhat helpful (2 on a 5-point scale in which 1 = very helpful and 5 = very unhelpful).

The participants rated the level of content presented as appropriate. Out of 6 students, 5 stated that the content was perfect for their prior knowledge and understanding, and 1 student stated that he/she found the content slightly advanced. The open-ended portion of the survey suggested interest in additional content such as other types of NGS library preps that the course was not able to cover. Notably, 3 out of 6 students highlighted that they had wished for a deeper analysis of the data. Although we limited the data analysis to quality control, it is entirely feasible to deepen the analysis and include a secondary bioinformatics analysis by using data analysis platforms for bench scientists without a requirement for command line competence or, alternatively, by offering a complementary bioinformatics course with emphasis on coding and running tools from the command line. In any case, it is important to clearly communicate the agenda and learning objectives from the outset of the workshop.

In order to assess the effectiveness of the course, we asked students to rate their confidence and knowledge levels before and after taking the class. Both hands-on components and the understanding of theoretical concepts as well as the analytical competencies were rated substantially higher for “after-course” categories, suggesting that the course was effective at enhancing the skills of the course participants (Table 3). To assess the longer-term impact of the workshop, we asked the participants to complete another survey 8 months after the workshop. Questions related to collaboration with the core facility, awareness, and application of technical and analytical skills were included (Figure 1). Overall, we found that 3 of the students had since reached out to the core for information, but none had actually committed to consultations or sample submission, and there were few that were planning to take advantage of core services in the future.

One important goal besides providing training to students was to raise awareness of the technologies available at our core facility, ideally early in the student’s graduate program. All of the students felt that the course increased their awareness of core technologies and that they since had explored ways to include NGS in their research.

In the category technical skills, we found that the students applied their learned skills through increased attention to “best practices” and quality review. Around half of the students already applied their technical skills to independent NGS library preparation, whereas the other half of students stated that they planned on making NGS libraries in the near future.  Out of 6 students, 5 stated that they have become more proficient in setting up and analyzing their own sequencing runs, indicating that the students have access to sequencers outside of the core facility. All 6 students stated that the course had a positive impact on their thesis research by either increasing awareness of tools, incorporating NGS applications that they learned about, or increased confidence in their technical and analytical skills based on follow-up questions on the survey. Both the hands-on components and specifically the opportunity for sequencer setup were rated favorably and helped to reinforce the concepts learned in the classroom. Out of the 6 students, 4 thought that the course should be offered in this or a similar format as part of the graduate student curriculum. Even though only 3 students thought the course could have an impact on career progression, at least 4 students stated they would include course completion in their resumes.

DISCUSSION

This workshop was considered a pilot study, with the potential for other institutional core facilities to adopt similar approaches in the future. NGS has grown in importance in many areas of basic and translational research and was therefore considered an ideal technology for the pilot study. Overall, our goals were to teach students technical knowledge and transferrable skills they can apply to their own thesis projects and beyond. Second, we wanted to provide an opportunity for the students to operate high-end instrumentation not typically found in research laboratories. Lastly, we wanted to increase awareness, foster collaborations between students and core facility, and elevate the educational mission of our core facility. Based on our survey results, we achieved most of our goals, and we consider this format a success. However, the cohort of participants was small, and the conclusions we draw from this pilot study should be interpreted with caution.

One of the challenges we encountered was that the program absorbed substantial personnel effort for several days of the hands-on training and prior workshop preparation. Core facilities are under pressure to meet turnaround times, and depending on the number of staff and number of instruments available, some downtime will be expected and may lead to slower than usual customer service. Several factors required the class size to be kept relatively small for this study. For one, the cost for reagents was substantial at around $1500 per student. From a budgetary standpoint, this type of program can be offered as fee for service by core facilities, or it could be internalized by academic departments. Funding for the study here was provided by a Department of Education, Innovation and Technology and Huffington Center grant. External funding opportunities exist through National Institutes of Health R25 and National Science Foundation (NSF) grant mechanisms that support curriculum development and other educational activities. A second factor we had to consider was laboratory space, which only allowed for a small group of trainees. Third, our core facility only has one NextSeq 500 sequencer. Because each run takes approximately 16 hours to complete, we needed to schedule each student’s run ad hoc on different days. This would have presented an organizational challenge with a larger group of students. A simulation approach to sequencing could mitigate this challenge as well as substantially lower the cost of the workshop. However, the majority of the students felt that providing each student with an opportunity to independently set up their own sequencing runs was a particular strength of this course (Figure 1).  Shared resources such as many NGS cores typically shy away from offering training on high-end sequencers. The reasons for this are multi-faceted—improper operation by an external operator can lead to instrument outages and costly delays, loss of reagents, and loss of data. A centralized consolidation of samples and sequencing enables cores to offer cost-competitive sequencing rates. In this study, we provided every student with the opportunity to sequence their libraries on a NextSeq 500 midrange sequencer. Although almost all sequencing in the core facility is done on an Illumina NovaSeq 6000 sequencer, we considered the NextSeq 500 the ideal instrument for training purposes, as it allowed each student to load their own mid-output flow cell, sequence relatively cost economically, and kept the risk of affecting other core business to a minimum. The workshop in this study was offered to a small group of students, primarily due to a limited budget, laboratory space, and instrument capacity. Increasing the number of participants substantially would have required a larger laboratory space and either more sequencing instruments or an altered format in which all of the students libraries would have needed to be combined into a single sequencing run.

 Although other formal RNA-seq workshops are available, the majority focus is on data analysis and less on wet bench aspects. As far as we are aware, the most comprehensive network of biotechnology workshops is offered by the Cold Spring Harbor Laboratories and the commercial entity Bio-Trac^®. Such workshops offer lecture-based and in silica training and often include hands-on training at remote laboratories. Enrollment and travel expenses coupled with the time investment to travel can be inhibitory for participation.[16] In some cases, the courses are for virtual attendance only and may not provide the same or similar experience. Once completed, commercial workshops do not continue to provide support. In contrast, any given core facility is in a position to remain accessible for consultation and mentorship. The small class size of our workshop allowed for individual attention and the possibility of the students setting up their own sequencing run. Commercial workshops aim for larger class sizes and may not offer the same individual attention.

NGS has grown in importance for many areas of basic and translational research.[17] RNA-seq is one of the most widely-used NGS assays, and hence, we decided to focus our workshop on this particular application. Needless to say, other applications are in demand and can justify workshops such as this one. One may argue that training students to become more independent is counterintuitive to a core facility’s mission to free up time for students and postdocs already encumbered with other laboratory tasks. In addition, many workflows can be automated and are available at a low cost from service providers. However, our survey results suggest that some students prefer to make their own NGS libraries independently over outsourcing. In this regard, there is value in learning NGS skills according to “best practices” as well as in applying basic concepts common to other NGS preps.

Shared resource core directors and staff are technology experts and involved in various educational activities such as lecturing of core technologies and training of core users in the operation of core instrumentation. It is therefore appropriate for academic institutions to work closely with their core facilities to develop formal workshops that combine both classroom-based teaching with hands-on components as well as career development within a coherent, standardized framework. We see mutual benefit to both student and core facility in providing formal workshops and see it as being most effective when being offered early on in the student’s PhD program. Core facilities that consider hosting a workshop of similar nature can review both costs and benefits (Table 4).

In summary, our pilot study suggests that a workshop hosted by a core facility combining hands-on and classroom-based components can provide an effective learning environment. Our survey results indicate a gain of knowledge and improvement of technical skills as well as the direct application of learning content to research projects. Specifically, greater attention to quality review and “best practices” were among positive outcomes highlighted by the students. We did not find higher rates of collaboration between students and the core facility despite an increase in awareness of the core’s technologies among course participants.

Ultimately, we think that hands-on workshops taught by core facilities will more effectively provide students with biotechnological tools for their PhD projects and, depending on scope, could even facilitate career advancement. In turn, core facilities benefit from building their educational portfolio, in some cases from publishing data generated during the workshop and from raising awareness of core technologies. We therefore advocate for the integration of structured courses and hands-on workshops taught by academic core facility staff into the PhD curriculum.

ACKNOWLEDGMENTS

Research reported in this publication was supported by a grant from the Department of Education, Innovation and Technology of BCM and the Huffington Foundation. The content is solely the responsibility of the authors and does not necessarily represent the official views of BCM or the Huffington Foundation. We thank Rob Dickehuth with BCM Audio–Visual Technology and Services for assistance with the creation of the instructional video and Chris Huang for the provision of laptops.