A Carpentries Approach to ACRL Framework Instruction

Objective: This paper compares the pedagogical theory driving current norms towards instruction of novices in both fields, specifically focusing on The Carpentries and ACRL Framework instruction. I identify key areas of difference in theoretical and practical approaches towards education of learners entirely new to a topic, focusing on a choice to pursue constructivist or experiential learning versus providing direct instructional guidance. Methods: Two case studies are explored through the lens of the Dreyfus Model of learning for their theoretical underpinnings for engaging novice learners: the ACRL Framework and Carpentries’ Instructor Training. Results: Applying the Dreyfus Model of learning and cognitive load theory shows theoretical benefits to direct instructional guidance over constructivist or minimally guided instruction. Conclusions: The ACRL Framework and Carpentries workshops share teaching goals of creating new mental models and core skills to support future learning, but differ in their pedagogical approaches. For novice learners of information literacy, there may be value in considering a more guided approach. Concrete lesson-planning strategies are proposed. Correspondence: Ari Gofman: ari.gofman@tufts.edu


Introduction An Introduction to the Case Studies
What is the ACRL Framework?
The ACRL Framework was adopted by the Association of College and Research Libraries (ARCL) board in 2016 as a national guideline for academic librarians teaching information literacy. It identifies information literacy as a core skill taught by instruction librarians, defined as "the set of integrated abilities encompassing the reflective discovery of information, the understanding of how information is produced and valued, and the use of information in creating new knowledge and participating ethically in communities of learning." (ACRL Framework 2015). There are six frames: • Authority Is Constructed and Contextual • Information Creation as a Process Each frame is composed of a: description, knowledge practice, and dispositions. The Framework identifies dispositions and practices that demonstrate having successfully learned the frames, but does not provide specific guidance on educational practices towards achieving that goal. Appendix 1 of the Framework, Implementing the Framework, explains that "the frames are intended to demonstrate the contrast in thinking between novice learner and expert in a specific area; movement may take place over the course of a student's academic career." (ACRL Framework 2015, 25). The Framework identifies the expected trajectory of learners to start at the position of novice.
Considering implementation, it continues ACRL realizes that many information literacy librarians currently meet with students via one-shot classes, especially in introductory level classes. Over the course of a student's academic program, one-shot sessions that address a particular need at a particular time, systematically integrated into the curriculum, can play a significant role in an information literacy program. It is important for librarians and teaching faculty to understand that the Framework is not designed to be implemented in a single information literacy session in a student's academic career; it is intended to be developmentally and systematically integrated into the student's academic program at variety of levels. (10) Implementation of the ACRL Framework is up to the discretion of librarians and broader departments (Oakleaf 2014); recommendations are discussed in professional literature and conferences, but rarely formally endorsed by ACRL and never mandated as a criterion for becoming an instructor.

What are the Carpentries?
The Carpentries are two-day workshops in Data, Software, and Libraries that aim to teach computational competence to learners by taking an applied approach, avoiding the theoretical and general in favor of the practical and specific. The Carpentries are guided by the principle that "by showing learners how to solve specific problems with specific tools and providing hands-on practice, we develop learners' confidence and lay the foundation for future learning." (Koch and Wilson 2016). Instructors are required to attend a two-day training, successfully perform a sample lesson to expert trainers, and participate in collaborative lesson-plan development to become certified. All materials are open source and available for repurposing, but the organization provides strict guidelines about which lessons must be included to be publicized under their name.

The Topic: Novice Learners
This paper particularly focuses on these two approaches as case studies for novice instruction. Different learners experience the process of learning new information differently, and instructors need to be aware of their audience when developing lessons. The Encyclopedia of the Sciences of Learning defines novice learning as "a person learning content about which [they do] not hold any previous knowledge or experience" (Laakso et al. 2012).
The question of novice learners is core to both the ACRL Framework and the Carpentries. The Carpentries' instructor training identifies three key experience levels in the context of learning software and data skills: a novice, who does not know what questions to ask; a competent practitioner, who is confident in their ability to complete tasks but regularly looks up information; and an expert, who tends to have been doing the task on a daily basis for years, and has a deep understanding and store of knowledge (Koch and Wilson 2016). The stated goal of the Carpentries is to facilitate the development of a learner from novice to competent practitioner, drawing on learning theory to answer the question "How can we help novices become competent practitioners?" (Koch and Wilson 2016). The ACRL Framework discusses Novices and Experts in each of the Frames. For example, in Searching as Strategic Exploration, the description includes the distinction that "Novice learners may search a limited set of resources, while experts may search more broadly and deeply to determine the most appropriate information within the project scope" (2016).
Both the ACRL two-part (Novice and Expert) and Carpentries three-part (Novice, Competent Practitioner, and Expert) models are simplified from the Dreyfus model of skill acquisition, originally developed from studying the learning processes of pilots and chess players and used broadly in scientific and medical education literature (S. E. Dreyfus and Dreyfus 1980). While recent empirical literature has contested the role of intuition in expertise, the model continues to be useful in distinguishing different levels of competence (Gobet and Chassy 2009).
As learners develop competence and expertise, they transition from what the Dreyfus model calls non-contextual, detached, non-involved, analytic behavior-which follows theoretical rules in ways that can seem mechanical-to expertise that is intuitive and contextually situated (H. L. Dreyfus and Dreyfus 2005).
The five levels of learning have been adapted to technical education in a variety of settings, including a succinct summary by Patricia Benner in the context of nursing:  (Benner 1982).
Information literacy instruction has been discussed in the context of the Dreyfus Model (Farrell 2012;Hall-Ellis and Grealy 2013). In a two-article piece applying the Dreyfus Model to formal Information Literacy learning, Robert Farrel describes teaching novice-level information literacy: "providing students with the kind of context-free rules they need to begin making decisions about information use and information seeking is one of the most common activities in librarians' instructional repertoire, particularly in the area of information evaluation."

Theory of Learning
Many factors affect learning, and this paper will not attempt to incorporate all of them; rather, it will focus specifically on one salient consideration that provides a useful lens for the two case studies: the concept of cognitive load. As originally developed by educational psychologist John Sweller, cognitive load theory posits that as people learn, there are three things that can occupy their mind: 1. Intrinsic load is the concepts and facts necessary to carry out a learning task 2. Germane load is the effort of creating connections between existing knowledge and new knowledge, which creates substantial learning 3. Extraneous load is everything else that can occupy a learner that is not contributing to the two above tasks (Sweller, Ayres, and Kalyuga 2011) Level I (novice): cannot use discretionary judgement and learns rules for action according to specific characteristics of a situation.
Level II (advanced beginner): can perform acceptably and, from prior experience, will notice recurrent, relevant, general characteristics of a situation, but needs support to prioritize.
Level III (competent): lacks speed and flexibility but analyses, prioritizes, and plans action, and assumes mastery and ability to cope with contingencies.
Level IV (proficient): perceives situations as wholes, not just aspects, is guided by situationally dependent maxims, and recognizes abnormality.
Level V (expert): only resorts to analytical tools, rules, and maxims in novel situations, and can see what is possible and what is not worth pursuing.
Sweller and colleagues argue that to promote effective learning, instructional design should be used to minimize extraneous load and focus on directing learners' attention to the germane load (Sweller, van Merrienboer, and Paas 1998).
However, how this translates into instructional design continues to be an ongoing debate in educational psychology, taking form in what Koedinger & Alven describe as an "assistance dilemma" (2007). As succinctly summarized by Lee & Anderson, the question asks "Is it better to try to tell students what they need to know, or is it better to give students an opportunity to discover the knowledge for themselves?" (2013). The debate has been rigorous: proponents of minimally guided instruction include many historically prominent figures like Dewey, Piaget and Rousseau (Brunstein, Betts, and Anderson 2009) and several more recent researchers (e.g. Bonawitz et al. 2011;Cobb et al. 1991;Hiebert et al. 1996). There are, however, also many studies identifying direct instruction as more effective in a huge range of fields including programming (Fay and Mayer 1994; M. O. C. Lee and Thompson 1997). Various experimental methodologies have identified that worked examples, where learners are introduced to a pre-developed solution to a problem with steps to the conclusion followed, are more effective than self-guided practice after instruction (Kirschner, Sweller, and Clark 2006;Atkinson et al. 2000).
Discovery-based learning has been found to be most effective when provided with a substantial opportunity for practice. Brunstein, Betts, and Anderson provide an experimental approach particularly relevant to university-level instructors of one-shot instruction sessions, 'boot camps,' and other introductory formats with limited time and substantial amounts of content. In two experiments, college students were presented with a set of algebra problems in a novel format and a digital tutoring program. In the first experiment, when given substantial time (1-1.5 hours, with 174 practice problems), participants in the discovery condition with no instruction-neither minimal or guided-performed best in follow up tests. In the second experiment, however, participants were given a more limited opportunity to practice-only 44 practice questions, about one per topic covered. Nearly half the participants were so confused that they asked to quit the study; of the remaining participants, those in the discovery condition did substantially worse than those given direct instruction.
This article will explore a comparison of the Carpentries and the ACRL Framework as part of this existing debate between direct instruction and discovery-based learning, informed by cognitive load theory. The method focuses on the requirements and recommendations in the foundational documents for each case study.

ACRL Framework and Carpentries: Similarities
Looking at our two case studies, both educational models have significant similarities.
They both have a goal to support future learning by helping learners create a mental model and habits of mind, both types of schema, or "knowledge structures that represent objects or events and provide default assumptions about their characteristics, relationships, and entailments under conditions of incomplete information" (DiMaggio 1997). The Carpentries teach mental models for data/software skills to facilitate future programming, while the Framework identifies habits of mind as a core part of each of the frames. While seemingly straightforward, this priority is not an obvious one: it is easy for instructors to want to teach students how to solve specific tasks, and most want to facilitate competence and nuanced, complex understanding in their graduates. However, both recognize that given the enormous range of possible future tasks and a limited amount of educational time, establishing mental models and core skills facilitates future learning and doing. The ACRL Framework specifically identifies threshold concepts as "core or foundational concepts that, once grasped by the learner, create new perspectives and ways of understanding a discipline or challenging knowledge domain. Such concepts produce transformation within the learner; without them, the learner does not acquire expertise in that field of knowledge" (Meyer, Land, and Baillie 2010).
Second, both the Carpentries and ACRL Framework use an approach of developing and identifying small, distinct skills into larger programs and results. For the Carpentries, these skills might include concepts like pipes and filters to build into more complex programs in the command line; for ACRL, tools like binary searching can be expanded and combined to build complex searches and explore different facets of a research question.
Next, an important and mandatory aspect of Carpentries instruction is that instructors live code the material they are teaching, rather than teaching off of screen-shots. This intentionally opens the door to errors ranging from misspellings to substantial mistakes like skipping a step or even technical ones like a computer malfunctioning. While this introduces an element of unpredictability into the process, it is an intentional set-up to acknowledge instructors as learners to (a) demystify a complex skill set, and (b) to ensure that there is ample opportunity to demonstrate problem-solving strategies that will occupy a substantial portion of the learner's time in future, self-directed projects. Instructors use positive error framing as "presenting errors as an integral part of the learning process and using them as teaching opportunities. Error framing encourages learners to understand that making errors provides valuable learning opportunities instead of having negative consequences" (Koch and Wilson 2016). Research has demonstrated that this can have positive effects on self-efficacy and metacognition, as well as performance quantity and quality (Koch and Wilson 2016). Somewhat similarly, the ACRL Frame of Research as Inquiry states that "Research is iterative and depends upon asking increasingly complex or new questions whose answers in turn develop additional questions or lines of inquiry in any field" (ACRL Framework 2015). While this does not mandate that instructors demonstrate live and make mistakes, it does emphasize that learners should be led to identify that they will make mistakes and have to adjust research questions as part of the expected standard research process.
Fourth, both emphasize hands-on practice with real problems as a strategy for reinforcing practical skills. The Carpentries have all learners follow along with lessons on their own laptops, using the same code that the instructor demonstrates on the class screen. The Framework does not provide guidance on implementation, but professional literature and norms firmly support interactive learning as opposed to the infamous "point and click" lesson plan.
Finally, both the ACRL Framework and Carpentries include an emphasis on inclusivity and universal design. Carpentries instructor training include a firm code of conduct and substantive unit on imposter syndrome, accessibility issues, and stereotype threat, which is the psychological demotivator where "Reminding people of negative stereotypes, even in subtle A Carpentries Approach to ACRL Framework Instruction JeSLIB 2019; 8(2): e1173 doi:10.7191/jeslib.2019.1173 ways, can make them anxious about the risk of confirming those stereotypes, in turn reducing their performance" (Koch and Wilson 2016). Instructors are urged not to highlight learners based on minoritized identity, and materials are screened for accessibility to screen readers and color blindness. All teaching materials are actively accessed for inclusivity, which is broadly defined to mean "making a positive effort to be more welcoming to women, people of color, people with various sexual orientations, the elderly, people with physical disabilities, the formerly incarcerated, the economically disadvantaged, and everyone else who doesn't fit Silicon Valley's white/Asian male demographic" (Koch and Wilson 2016). An example in practice can be seen in Figure 1 earlier in this paper; where the visual graphics intentionally subvert the cultural norms for who is assumed to be an 'expert' in technical fields. The novice illustration presents as a white man; the competent practitioner seems to be an Asian woman, and the expert is a queer-presenting black woman.
The ACRL Framework recognizes structural inequality throughout the framework, most explicitly in the frame Authority is Constructed and Contextual. This frame recognizes the role of power structures in the creation, dissemination and interpretation of information: "Experts understand the need to determine the validity of the information created by different authorities and to acknowledge biases that privilege some sources of authority over others, especially in terms of others' worldviews, gender, sexual orientation, and cultural orientations" (ACRL Framework 2015).

Differences
The Carpentries and ACRL Framework share goals of long-term understanding & ability to utilize skills learned in practical use but have different pedagogical approaches.

A. Experiential Learning, or Minimally Guided Instruction
While the ACRL Framework does not provide binding guidance on instructional design that librarians are required to follow, several aspects of the Framework and surrounding professional literature indicate support for experiential learning. The co-chairs of the ACRL Taskforce that developed the Framework, Trudi E. Jacobson and Craig Gibson, explained that they expect learners to struggle with threshold concepts, and "posits that students must pass through a portal of struggle and difficulty, in order to develop increased understanding of a discipline or knowledge domain" (Gibson and Jacobson 2014). The Framework emphasizes using information to create new knowledge; one of the dispositions for Research as Inquiry, for example, emphasizes that learners should "consider research as open-ended exploration and engagement with information" (ACRL Framework 2015). In some ways, the presentation of a professional set of standards without clear implementation or worked examples of model lessons and lesson plans is in itself a form of experiential learning for librarians.
Research, summarized earlier, supports this in some contexts of higher education, particularly in situations that allow for substantial amounts of practice. Several practitioner's guides to teaching information literacy advocate for discovery-based learning (Jacobson and Xu 2004). A common library instruction session at the undergraduate level, for example, includes a period of time where students are to pursue research topics individually as the instructor circulates after providing instruction to the group; other lesson plans, involving even less guidance, may provide undergraduates with a list of questions about evaluating information, an A Carpentries Approach to ACRL Framework Instruction JeSLIB 2019; 8(2): e1173 doi: 10.7191/jeslib.2019.1173 article, and leave them in groups to develop their own strategies for answering the questions.
Such lesson plans are appealing-students explore fundamental concepts in the Framework through practice. The questions they are tasked to answer may lead them to use Searching as Strategic Exploration in an attempt to answer questions about the article's authors, publication, funding, and methodology. When successful, they may engage in discussions that address other frames like Information Has Value and Authority is Constructed and Contextual.
Following Brunstein et al.'s research, however, the concern is that in a situation of limited time and limited practice-like a one-shot instruction session-the benefits of constructivist learning may not be achieved (2009). If given insufficient time to not just fail multiple times before developing a successful strategy, but to implement that successful strategy multiple times, students are more likely to become overwhelmed and want to give up, not having developed new knowledge or skills, as in Brunstein's 2009 study. By contrast, students with more opportunity to develop schemas and implement them in multiple situations, perhaps in a credit-bearing class or through embedded instruction, may be in a better situation to genuinely understand the threshold concepts of the Framework.

B. Direct instructional guidance
As discussed earlier, direct instructional guidance is an approach that provides information that fully explains the concepts and procedures that students are required to learn. Kirschner et al. argue that this is more effective than experiential learning in long-term memory changes for learning (Kirschner, Sweller, and Clark 2006). The Carpentries very explicitly acknowledge their use of direct instructional guidance as the theoretical framework of their pedagogy, including citations to Kirschner et al. during required instructor trainings and instructor training materials.
The Carpentries use live coding of pre-prepared code so instructors know that it will teach the concepts needed and students can follow along through the same process; live coding allows for mistakes and problem-solving. All instructions are presented in multiple modalities for different types of learners: each step is presented verbally as the instructor talks through the steps; visually by live coding on the screen; and in text through the online transcript that is prewritten and shared with participants. The problems and solutions are pre-scripted and the instructor and students work through them together. Regular formative assessment interspersed with the worked examples provide opportunity for learners to practice applying strategies and schemas.
Participants leave with an introduction to the tools and methods described in class, as well as perpetual access to both course materials and instructors. Research has been ongoing since 2017 to explore the long-term learning effects of the Carpentries; the most recent report finds that 66% of respondents reported using programming languages or command line to automate repetitive tasks, one of the major lessons of most workshops (Jordan 2018).

Discussion: Experiential Learning v. Direct Instructional Guidance
The Carpentries and ACRL Framework have different philosophies of guidance for learners. Having explored the case studies focusing on novice learners in time-limited learning opportunities, the evidence seems clear that direct instructional guidance is more appropriate for these contexts. Providing students with guided practice that creates a structure for applying skills and getting feedback without becoming distracted or overwhelmed, following the theory of cognitive load. In practice, this may be familiar to information literacy instructors: telling students to "write something, it doesn't matter what" will often end up with learners spending substantially more energy deciding what to write then practicing the desired skill. Similarly, even students with finely honed topics, without clear guidance, will spend research time reading an article rather than practicing search strategies, for example.
In a 2015 summary of applications of cognitive psychology to information literacy, Cook and Klipfel propose principles for information literacy instruction based on the same body of research discussed in this paper. One of their principles is "Do Less," to avoid information overload, and to encourage deliberate practice where "individuals are motivated to focus on a task, put effort into improving their ability, find the task only slightly more difficult than what they are already capable of, and receive immediate and constructive feedback" (Cook and Klipfel 2015). Creating tasks that are clearly scaffolded and provide immediate, constructive feedback requires guidance and preparation.
Both the Carpentries and ACRL Framework acknowledge that learning is a difficult task; learners are faced with a barrage of new information, skills, and concepts to integrate into their existing worldviews and other tasks. In either framework, instructors should acknowledge that challenge without emphasizing it. The Carpentries instruct teachers to avoid demotivating words like: need, must, can't, easy, just, only, and fast. Based on a list by Jason Fried (Fried 2015), "these signal to the learner that the instructor thinks their problem is trivial and by extension that they therefore must be stupid for not being able to figure it out" (Koch and Wilson 2016). Unfortunately, words like these are in fact often used by information literacy instructors: many a librarian has tried to convince learners that these skills are in fact easy and fast, and that they can "just" do something. Instead, it is helpful to acknowledge that whatever skills are being taught are in fact skills, and require effort and learning-and to respect the effort that learners have to put in to become competent.
Given that practice and repetition is a crucial part of learning in both discovery and direct instructional guidance, instructors should choose a limited number of learning objectives per lesson. As discussed earlier, Cook & Klipfel encourage information literacy instructors to "do less" and not pursue all the Frameworks in a single session. The Carpentries suggest a mechanism to choose between many valuable options by considering the tradeoffs between how useful a skill is once mastered and the mean time to master it, as in the graph below.
The Carpentries take an approach of teaching the most immediately useful skills first, rather than building up to them. Learners should learn something that they think is useful to their daily work within 15 minutes of each lesson, which both motivates their learning and builds confidence for more difficult material. Complicated material that is not useful-even if considered foundational concepts in other curricula-can be avoided. While the ACRL Framework intentionally does not prioritize between the different frames, this may be a useful way to decide among different learning objectives for different audiences. To support the cognitive load of novice learners, instructors may consider live searching a pre-prepared topic with students doing the same search for class exercises, rather than having students explore individual topics. Instructors may also provide step-by-step instructions to learners in at least one worked example for short instruction sessions. Although worksheets and checklists receive substantial critique in the professional literature (including Meola's call to "chuck it"), checklists can provide a first step for learners to move through, rather than a final state (Meola 2004). Farrel explains: Dreyfus' model helps us understand the role such checklists play for novices who need rules to guide their actions. Rules prevent novices from becoming overwhelmed by the complicated details of real world information contexts. Within the skill environment in which evaluation is first practiced, simple predefined features such as "currency," "relevance," "authority," etc. (Meriam Library, 2010) can be applied…activities that challenge students to evaluate websites and articles according to various rubrics, whether in class or online, can provide them with the experiences needed to develop a facility for acting according to abstract concepts. (Farrell 2012) While critiques of highly guided instruction argue that "The checklist model is difficult to implement in practice and encourages a mechanistic way of evaluating that is at odds with critical thinking" (Meola 2004), highly structured guidance can instead recognize that mechanistic following is a useful first step in education, and use that to guide learners through repeating and into critical thinking, rather than neglecting to set the core skills necessary by jumping immediately into advanced and unstructured activities.

Conclusions
The ACRL Framework and Carpentries workshops share the goals and formats of teaching new mental models to novice learners in a short period of time. Their differing pedagogical techniques provide useful case studies in the ongoing debate between experiential learning and direct instructional guidance. For novice learners of information literacy, there may be value in considering a more guided approach than is currently discussed, and concrete lesson-planning strategies are discussed in context of pedagogical theory.
Additional research in this field is needed. Comparisons of pedagogical methods could be pursued through other methodologies; a card-sorting exercise with multiple instructors who teach information literacy and/or Carpentries workshops could incorporate other perspectives and develop a further cross-pollination of strategies and content. Additional research could also focus on practical applications of both sets of foundational documents: rather than discussing what should be done, as in this article, future researchers could explore how different academic librarians and Carpentries instructors develop and implement actual lessons in a variety of contexts. Effectiveness could be measured through a variety of assessment techniques.