A Spider, an Octopus, or an Animal Just Coming into Existence? Designing a Curriculum for Librarians to Support Research Data Management

The paper explains the approach taken in the UK Jisc-funded RDMRose project to developing a study module/Open Educational Resource about Research Data Management (RDM) for librarians. The resource was developed collaboratively between the University of Sheffield Information School and the libraries at the Universities of Leeds, Sheffield, and York. Curriculum design principles such as an emphasis on exploring the nature of research and on other professional services supporting RDM were based on requirements gathered from focus groups and the literature. The content of the eight half-day sessions is briefly outlined. The paper goes on to explore how the learning materials were evaluated by this first cohort of learners and readjusted to respond to feedback. Future plans for co-producing an RDM related learning resource through a student-centered process and to create a sustainable learning network are discussed.


Introduction
As an ice breaker activity in training sessions run by the authors, librarians were asked to draw a picture that represented Research Data Management (RDM) as if it were an animal. Among the drawings produced were a frightening looking spider, a rather happy octopus, a millipede, and a scarab beetle pushing a ball of dung up a slope. The imagery suggests the multiplicity of RDM and also hints at anxiety around the subject. One other drawing was of an "animal that is just coming into existence." This aptly captures the emergent character of the RDM agenda in early 2013. In the institutions from which the librarians came, it was still unclear what the institutional policy on RDM was and how this policy was to be realized as a support infrastructure. It was also un-certain what the Library's role in providing RDM services would be in relation to other professional services. At the same time it was perceived to be an important, pressing, complex, and far-reaching agenda.
There is a growing body of literature and practice that delineates an important role for librarians in RDM (Pryor et al. 2014, Cox and Pinfield 2013, Auckland 2012, Corrall 2012, Cox et al. 2012, Lyon 2012, Alvaro et al. 2011, Brewerton 2011, Lewis 2010, Gabridge 2009, Garritano and Carlson 2009, Henty 2008. It encompasses specialist roles such as those relating to the technical aspects of running a data repository and to the expertise in change management that is needed to effect cultural changes in attitudes to data management and data sharing. But as institutions develop Research Data Ser-vices (RDS) in the UK in response to funders mandating improved RDM (Research Councils UK 2011;Pryor 2012;Pryor et al. 2014), there is also a need for general awareness among library staff as a whole. Liaison staff in particular -who work closely with academics in departments -are likely to need understanding of the issues. RDM has the potential to touch the roles of many library staff, not only those in liaison roles. A survey of UK academic libraries' orientation to RDM by Corrall et al. (2013) found that institutions recognized the need for training, though it often seemed that they expected staff to develop the skills in their own time. Another survey by Cox and Pinfield (2013) found that the institutions they surveyed recognized a skills gap that was a major obstacle to planned RDS. There is a need, therefore, for learning materials about RDM for continuing professional development (CPD) of librarians and for library and information science (LIS) degrees, so that all new entrants to the profession have a good understanding in the area.
In this context, as part of the second Managing Research Data Programme 2011-2013 (Jisc 2013), Jisc funded the RDMRose project to produce learning materials in RDM for taught courses and CPD tailored for information professionals (RDMRose 2013). RDMRose brought together the Information School at the University of Sheffield with a practitioner community based on the White Rose University Consortium's libraries at the Universities of Leeds, Sheffield, and York. The consortium already works together on a number of projects, including a shared institutional repository.
The approach of RDMRose was to base the creation of the learning materials on a highly participative curriculum development process with a strong strand of continuous evaluation by participants. The learning materials were intended to be multi-purpose: suitable for practitioners' self-directed CPD, for internal training by libraries to their staff, and for embedding into the postgraduate taught (PGT) curriculum. They were intended for librari-ans working in any context, not just escience. A range of materials was developed that had content for at least eight halfday sessions, including presentational material, recommended readings, and activities. These materials were initially trialled with librarians in one of the White Rose Consortium's three institutions. Based on extensive feedback from this iteration, the materials were revised and delivered to librarians from the other two institutions. Feedback from this delivery was used to further revise the materials. The whole set of learning resources was then made available for (re)use under a Creative Commons share-alike licence (CC BY SA) via the project web site (RDMRose 2013) and by being deposited in Jorum, the Jisc-funded UK learning materials repository (http://find.jorum.ac.uk/ resources/18017).
This paper describes in more detail the principles on which the learning materials were based, and gives an overview of the content. It also reports the detailed evaluations collected during the delivery of the module and how these influenced the design of the current set of learning materials. The paper concludes with some reflections on lessons learned through the process of the project.

The Learning Approach
The initial design of the content was derived from three sources: focus groups at the participating libraries (RDMRose 2012) combined with a review of the literature on the library role in RDM and a study of existing RDM curricula. On this basis the module was designed around a number of core principles. This section discusses these principles.
The literature suggests a wide range of possible roles for libraries in RDM. Perhaps over time some of these roles will emerge as more important and particular institutions will gradually develop their own range of specific services, but at the time the learning materials were being developed none of the partici-   (Cox, Verbaan and Sen, 2012) Another central assumption made in the module design was that librarians themselves often do not have in-depth experience of research. RDM and an increasing number of other roles to support research require more understanding of the perspective of the researcher. Therefore considerable time in the module was devoted to actively exploring the nature of the research process and the nature of the research data.
The wider literature ) -including studies by the authors (Cox et al. 2014, Verbaan and) -indicates that RDM services require various support teams within the institution to collaborate, since no professional group has the expertise to cover the whole gamut of the required RDM infrastructure. The library, IT services, and research administration are the most likely partners in this joint venture, possibly with collaboration from records management; but it can also include other groups such as staff development, finance, and the legal department. As a result, the module was designed to also encourage the participant to think about the potential role of the library in RDM in relation to other professional services, considering the professional culture of these other groups.
It was also decided to place a strong emphasis in the module on practical hands-on activities, often engaging with real documents such as institutional RDM policies or existing data management plans (DMPs). This reflected participants' expressed desire for a focus on what is pragmatically useful. It was impossible to incorporate institutional specifics because these were not yet known. For example, none of the participating institutions had a data repository when the learning materials were being developed. At the same time, and while acknowledging the orientation of the library profession to practical skills, it was important to engage with theory. For example, the module introduces some theoretical perspectives on the nature of disciplinarity and interdisciplinarity -forces potentially shaping RDM -as well as directly pating libraries had a well-established RDS. Therefore, it seemed important that aspects of all library roles in RDM were explored in the module. This ensured that the material was relevant to any library regardless of its maturity in RDS and that it accommodated all individuals who might have a specific role in RDM. A structure of potential RDM roles of librarians was developed as the foundation for the module, and reused in a number of ways throughout the learning materials, including in evaluation (Table 1). For the final version of the learning materials, an index was provided organized around these roles. This facilitates use where a participant has a very specific role or area of interest in RDM. Using this structure as a point of departure, the module was designed around discussing the role of libraries in RDM within institutions in an open-ended way. It was also based on an emphasis on individual reflection on the place of RDM within the participants' own role. Finally, the module was also designed to stimulate open -ended reflection on the role of the profession as a whole in a complex and fluid strategic context. The overall aim could be summarized as "confidence building." It was never the intent to offer specialist datacuration training.
Another belief that guided the design of the module was the idea that librarians' prior conception of their role would strongly shape their responses to the new RDM agenda. For example, some liaison work is about collection; in this context RDM could be seen as expanding the collection to encompass data. For others, liaison is about networking and influence; here the focus might be much more on policy and the wider context of open access. New roles in RDM might also imply a change in professional identity. This led us to focus on the individual reflecting on what RDM meant or might mean for them. Indeed, reflection was a core part of the module, with the idea that the learning materials should support a personal learning journey.  Finally, the University of Sheffield has a strong culture of promoting research-led teaching and inquiry-based learning (http:// www.sheffield.ac.uk/ibl). An element of inquiry-based learning is offered in the RDMRose module through a scaffolded exercise to plan and conduct an interview of a researcher that spans most of the length of the module. A further element of problembased learning is offered through a number of case studies based on documents and audio recordings relating to specific real research projects, as well as a fictional case which prepares participants for investigating a researcher later in the module.

Section 1 RDM and the role of LIS
Once the foundations have been laid, the third session is designed to look at research data lifecycle models, and who is involved at each stage. This includes a discussion of DMPs and an exploration of the point of view of different stakeholders in RDM. The fourth section is then devoted to different ways of keeping up-to-date, with a strong focus on the UK's centre of excellence in digital curation, the Digital Curation Centre (DCC, http:// www.dcc.ac.uk/) and the resources it makes available on its website.
The topic of the fifth session introduces participants to the world of research data. In previous sessions participants have prepared an interview with a researcher, and this section kicks off with discussions of the findings of this exercise. Participants are also introduced to a framework that outlines different ways of looking at the data used by the researchers they interviewed and others. study of RDM provision in the fictitious University of Poppleton.

The Content
The content of the RDMRose module is distributed over eight half-day sessions building from introductory material in the first sessions to in-depth case studies later in the course (Table 2). Each half-day session usually has four hours of sets of activities. In the first session participants are introduced to the core concepts of RDM, and there is an opportunity to start thinking about the different roles that libraries could play in RDM and how well they are prepared for them. The second session zooms in on research and the differences between academic disciplines in order to better understand the variety and complexity of research data. The second session also looks at the context in which RDM has become an issue, including funders' expectations and institutional RDM policies. Finally, participants are introduced to research data audits and data profiling, 20 in different academic disciplines. These case studies are based on recordings of interviews with real researchers and actual documents related to their projects. Taken together these case studies follow the processes of research and handling research data: from project proposal and initial data management planning, via the reuse of existing datasets, to publishing research outputs and depositing relevant data in repositories. Finally, in the eighth session participants examine how everything related to RDM fits together at an institutional level.
The sixth session moves from the data to the management of data. It includes practical data management guidelines such as file naming conventions and backing up files, which cover live data, and a range of other topics that cover archived data, such as institutional data repositories and national subject repositories. Closely linked to this is an exploration of metadata and data citation.
The last two sessions are devoted to case studies. The seventh session looks at case studies of researchers and research projects

Evaluation of the Curriculum Content
The structured list of potential roles in RDM (Table 1) was expanded to gather systematic feedback from the participants throughout the sessions about their areas of learning.
The results are represented in Table 3. Participants were asked to rate (1) their current They do this by exploring a case study of a fictional institution, looking closely at the point of view of particular stakeholders, including senior academics and professional services. After presenting their views on different stakeholder perspectives they then discuss a number of plausible scenarios to explore how the stakeholders are likely to interact around RDM as an agenda.  Table 3 shows that participants generally had a low estimate of their knowledge of RDM at the beginning of the module, with values ranging from 0.1 to 1.9 and an average of 0.8. But in most areas participants' self-rated understanding had improved at the end of the module by an average of 1.2. Improvement was highest in those areas that were explicitly covered by the learning materials. For example, self-rated understanding of the "basics of RDM" had risen from 1.3 to 2.9. Given the focus of the project on awareness raising and confidence building knowledge and (2) the importance of the topic for the RDMRose module on a Likert scale of 0 to 4. They were asked to do so both in the first and in the last session of the course. i The columns in grey indicate the change in rating over time. The relatively small numbers of respondents precluded comparisons between responses from Sheffield and the other institutions or between students and practitioners.

Current Knowledge
The figures should be viewed with caution because none of the changes noted is actually statistically significant. est where the topics were not or not explicitly covered. Some items on the list were from Auckland's inventory of roles and skills that liaison librarians need to support research (2012) but they were clearly not core to RDM, so there was no intention to cover these in any detail. This was in particular true for conducting data analysis (number 28), measuring impact (number 30), bibliometrics (number 31), and archiving project records such as correspondence (number 33). Importance

23
Just as striking as the relatively low scores for self-rated understanding of RDM topics, this was a very positive finding. Participants also felt they understood "researchers' perspectives from the inside" better, rising from 1.0 to 2.5. Participants' knowledge to persuade a researcher that RDM is important had risen from 0.9 to 2.5 and key messages for researchers rose from 0.6 to 2.2. This reflects the module's emphasis on understanding the researcher's perspective. Finally, participants also felt more knowledgeable about how to keep up-to-date, their scores rising from 0.9 to 2.6. This means that the core intended learning outcomes had been achieved.
Not surprisingly, the improvement was low-24 cess to RDM (+0.4), and knowledge of who is who in the different professional services such as research administration and IT (+0.4). This means that the participants had become more aware of the relevance of the policy context to be able to operate in an RDM role, and that they had begun to see RDM as a collaborative effort shared by the professional services. These are certainly positive changes reflecting the philosophy of the RDMRose materials.

Evaluation of the Learning Approach
In the final session of the module, participants were also asked for written comments on the elements of the module they found most helpful, and what could be improved. Such data are hard to evaluate with confidence because comments in each case tended to be from just a few individuals and may not reflect the dominant feeling of the entire cohort.
Overall, the RDMRose module was well received. As one member of staff commented about her colleagues who had participated in the training: What all participants valued especially highly were the "hands-on & practical" activities such as designing a web page with RDM advice (session 4), and discussions "gaining insight into my colleagues' perspectives on RDM." They also remarked favourably on the exploration of the perspectives of different stakeholders in RDM, especially the researchers' perspectives (sessions 7 and 8).
In addition, the overarching activity where participants were asked to conduct an interview with a researcher (that occurs in ses-is that all topics were considered to be important. The highest scores for importance (Table 3) related to the participants' selfperceived lack of understanding of RDM and research, and to their point of view as staff in a liaison role, such as:  the basics of RDM, and the potential library roles in RDM;  institutional policies on RDM, especially in the local institution; and  looking at the researcher's perspective: the role of research in higher education institutions (HEIs), perspectives from researchers, and key messages about research data best practices for researchers.
Relatively unimportant was the Open Archival Information System (OAIS) reference model. Though it lies at the foundation of the research data lifecycle from a curatorial perspective, it was relatively unknown to participants, which may partly explain its perceived lack of importance at the start of the module (1.7) and its increased importance towards the end (2.5, a rise of 0.8). Other less important topics were ways of analysing data (2.5), and archiving of project records such as correspondence (2.7 to 2.8). These topics were -as stated above -mentioned by Auckland as possible roles of liaison librarians in supporting research but possibly not core to RDM.
In general these ideas about importance did not seem to depend on the know-how of the participants: Participants did not alter their views on the importance of the varying RDM issues significantly as their knowledge of these issues increased. This could simply be because all the issues were seen as of importance, i.e. with an average score of over 3 out of 4. But there are exceptions. In particular, participants became more convinced of the importance of institutional policies (+0.5), licensing of data in relation to data reuse (+0.5), the relation of open ac-25

Areas for Improvement Suggested by Participants
A number of areas for possible improvement were suggested by the participants. Firstly, some participants expressed the desire to involve colleagues from other institutions, or researchers and people from computing services and research offices in the teaching. They also suggested including recorded interviews with someone already employed in an RDM role. In response, the material from other stakeholder groups than the Library was strengthened, although the philosophy of producing a self-contained reusable resource went against having "guest speakers." There would be nothing to stop local presentations of the module to involve such speakers.
A second point was that for some the institutional context was lacking: "this would have helped us to make much more sense of how the theory might be translated into reality." As one respondent wrote, "Less background info; just focus on what we need to do." These concerns relate to participants' remarks that local practices around RDM were only just coming into place. As one respondent wrote, "[The RDMRose module] came too early -would have been far more useful if we'd known what our responsibilities would be before we started this." The project steering group discussed this point, but concluded that in fact the timing of the course was good. Simply waiting for institutional arrangements to be clear was not adequate: a proactive approach was really required for staff to go out and discover researchers' requirements in order to construct RDS organically. Yet the point does reinforce the extent to which training has to be integrated with wider planning.
The lack of institutional context not only referred to the lack of clarity of likely roles participating staff might need to undertake in the future, but also to the technical infrastructure to support the implementation of an institutional RDM policy. As soon as it becomes sions 1, 4, and 5) was highly rated. Indeed, actually talking to a researcher about their research and their potential research data management needs was an eye-opener for many participants. Also highly valued were the examples presented in the researcher and institutional case studies, the analysis of examples of DMPs, and the introduction to websites of other libraries and of the DCC.
Not only were the hands-on and more practically oriented activities viewed positively, but some participants also mentioned they had no substantial previous knowledge and that they therefore appreciated the explanation of basic concepts in the first two sessions. Similar remarks were made about the RDM lifecycle discussed in session 3, although many found the lifecycle models "confusing & overly complex" and "difficult to engage with," which made "a dry subject less enticing." The same person remarked that this was also the case with Data Management Plans, because these "feel remote from my role." Many comments reflected a desire for the material to be immediately and practically useful. Some participants, for example, did not see the relevance of some of the activities: "I would never have to write an institutional policy or design a website or write a job description." These participants missed the point that these activities were not meant solely to develop skills on how to write a policy, but to build general knowledge of RDM.
Participants in Sheffield remarked that four hour long sessions were too long to concentrate, even though they said that "there's a good variety of activities." Similar remarks came from the participants to the second iteration, who attended four whole day sessions. Bresnahan and Johnson (2013) found that the most popular way to learn about RDM in their institution was a singleday workshop; the least popular workshops spanned multiple days. Yet it is hard to see how with a wholly new topic that knowledge and confidence can be built except over a series of events. 26 for people to find the time and mental space to reflect on their work.

Reflections on RDMRose
A strength of the RDMRose project is that because the module is embedded in the Sheffield Information School Masters programmes, it will be routinely updated. Sustainability is in some sense built in. However, it is such a fast moving field that fundamental changes will rapidly point to ways the material has to be restructured. The quantity of the material to some degree precludes doing this easily. From the perspective of 2014, there are elements in the learning that would be more central now. In particular, there is the emergence of a number of patterns of good RDS practice.
Having acknowledged that, the most stimulating part of the module has always been the investigation of a researcher and this remains highly relevant because it is grounded in participants going and undertaking a study themselves.
Trying to design a multi-purpose module usable for face-to-face teaching (by ourselves and also by other information schools) as well as self-directed CPD was the challenge of the project. The decision was made not to turn the materials into a click-through online learning package using software such as Xerte (http://www.nottingham.ac.uk/xerte/). Rather the form of the open educational resource (OER) was resources such as presentations (in PowerPoint), activity sheets (Word documents), and discrete media files (mp3). Although an online tutorial is quite attractive for self-study, it was thought it would not be very reusable because it may have local references to specific institutions embedded, and the form is not suitable for face-to-face learning scenarios. The form of the materials that were produced maximized their reusability, at least for other educators and including libraries reusing the materials in their own training programmes. In retrospect, however, the packaging of the materials may not have been very engaging. Nei-clear what software will be put into place to support RDM -such as data stores, data catalogues, data repositories, and links to Current Research Information Systems such as Symplectic -training needs will become clearer, not only regarding the actual roles staff will have to perform, but also regarding the technical and procedural infrastructure in which they will have to operate.
A third and related point made by participants was to include discussions of team objectives and creating an action plan for moving forward, especially in the last session, thus linking the theory and generic hands-on activities directly to the participants' institutional and specific situation. In the second iteration, time was indeed reserved for discussion on the way forward for each of the two participating institutions. Thinking about how to integrate local experience and practice is vital to the success of education embedded in a library service; yet it is challenging in contexts where service models across the whole sector, let alone in one library are just emerging.
Finally, some participants wrote that the module seemed to be geared towards students rather than professionals, or at least it suffered from trying to target both audiences: "It needs to be tailored to appeal to information professionals. It felt very studenty." This feeling seemed to be linked to the emphasis given to reflective exercises. A number of participants recoiled from the idea of reflection. As one participant noted: "We do tend to be very busy. So, we're used to a fast pace and we get on and do stuff a lot of the time. I'm not saying it's good practice, but we don't stop and think a lot about it." The steering group felt that reflection on professional practice was a valuable part of the module, but perhaps the explanation of its value needed to be strengthened even further. Again, this point relates to a generic issue with training in the workplace: it is hard 27 The best indication of how the project team thinking has changed through the experience of the project is apparent in existing follow-up plans. The Information School team were fortunate enough to win funding from the Leadership Foundation for Higher Education (http://www.lfhe.ac.uk/) for a second project called "Wicked Ways In RDM" (http://www.shef.ac.uk/is/research/ projects/wickedways). This project will produce an OER related to RDM, focusing specifically on conceiving of it as a "wicked problem" (Horn and Weber 2007, Rittel and Webber 1973, working with a multi-professional group including research administrators and computing services staff, as well as librarians across eight participating institutions that will get together for two full-day workshops. The approach differs from RDMRose in quite a few respects, but two are very salient for revealing how the authors' approach has changed as a result of the RDMRose experience.
Firstly, one of the aims of Wicked Ways is primarily to build a network of participants: in this sense Sheffield information school staff enter into the workshops in a much more open ended way. In particular, it seeks to reinforce existing networks across a number of participating institutions in the North of England. This is useful to the professionals concerned, builds on the collaborative instincts of the library profession and the existing local inter-professional collaborations, and also embeds us as LIS educators in the community who are currently inventing RDS in parts of the north in the UK. At an individual level it is hoped that some will make useful contacts that they will go on using; this happened in RDMRose because librarians from a number of libraries were taught together, but in Wicked Ways it will be one of the main objectives. It is also hoped that the group as a whole will find a future for itself.
Secondly, the authors want to engage the participants of Wicked Ways in a coproduction model rather than a relatively directive framework: the aim is to produce an ther linear navigation through an online tutorial nor a body of separate files really offers a convincing model of how to offer a reusable and multipurpose online resource. A current plan is to rework the materials in a simplified, restructured form for the iTunes U platform. This will be more limited in its learning objectives, but easier to reuse for self-directed learning.
The participative approach adopted in RDMRose offered an exciting but resourceintensive model of curriculum design. RDMRose funded a Research Associate for a whole year and the Jisc Managing Research Data Programme acted as a wider community of activity within which to operate (Jisc 2013). Thus it was possible to spend a lot of time developing materials and to undertake in-depth engagement with LIS staff. Their interest in RDM meant that the libraries in the consortium were eager to participate and shared their time generously. This created a very stimulating context for the educators in RDMRose to develop the learning materials. The good resourcing of the project also meant that a lot of learning materials could be developed. While this reflects the wide-reaching ramifications of RDM, it is perhaps intimidating for the potential user that the materials are so extensive.
Although maintaining this level of engagement with the practitioner community for maintaining the RDM curriculum is not practical or necessarily required, the value of working closely with the profession to develop learning materials is something that the authors have learned from the project experience. Indeed, thinking how to continue this form of collaboration has been a preoccupation since the close of the project. When the RDMRose module was re-run for taught students in the Information School, some local library staff attended a number of sessions. It has also given incentive to the authors to work with a number of institutions/library consortia to re-run the RDMRose module or repackage it for shorter sets of sessions. 28 OER in full and equal collaboration. In doing so, the new project is heavily influenced by the concept of a "student-centered process model" to curriculum design where the tutors negotiate with the learners what the curriculum should be (Fraser andBosanquet 2006, Knight 2001), rather than defining learning outcomes in advance as expected in the model of constructive alignment (Biggs and Tang 2011). Key decisions in the project will be made within the group. One of the outcomes of the workshops will be an OER that is much more lightweight than RDMRose. In RDMRose the outcome of the project was primarily conceived in terms of producing a "thing:" a body of high quality, carefully designed, and quality checked resources that addressed the subject in a comprehensive way. In Wicked Ways, the output is conceived of more as a small, very engaging OER, co-produced in the project, and a sustainable network of people. If RDM is "a creature just coming into existence," the curriculum and the learning group itself need to be fluid and agile.