Presentation and discussion of findings

4. Presentation and discussion of findings

4.1 Introduction

This section will be split into two parts; the first section will simultaneously provide an overview and analysis of data from the self-administered questionnaire, and the second part will do the same for the key findings from the semi-structured podcast interviews. This structure corresponds with the sequence adhered to in the data collection phase, and it allows for key themes to emerge from both data sets.

4.2 Questionnaire

In total the questionnaire generated 99 responses. As outlined in the methodology eight out of the nine questions were MCQ-style questions predicated on generating responses across three domains; demographic and professional information, and attitudinal data relating to FOAMed’s relationship with the curriculum and reflective learning. The full breakdown of responses is available in the Appendix. The table below provides an overview of the top responses to the first eight structured questions:

Question Top response
1) Please confirm your age 30-39 = 45%
2) Please confirm your current grade Consultant = 34%
3) Are you a RCEM member Yes = 66%
4) Please confirm your deanery or region North west = 13%
5) Is curriculum mapping a positive feature of the RCFN It helps = 55%
6) Should RCFN resources be recognized or included within the RCEM curriculum? Yes = 84%
7) Would a reflective learning log improve your experience of the RCFN? A little bit = 47%
8) Do you struggle to get sufficient recognition for activity undertaken on the RCFN? I don’t use the RCFN for assessment = 42%
9) What would improve your experience of the RCFN? Please enter some general comments below See pie chart

Image 12: Overview of top responses to structured questions

The demographic information captured in the first three questions aligns with organisational assumptions about FOAMed consumers. Respondents were from the second youngest cohort of clinicians available on the questionnaire (ranges included 20-29, 30-39, 40-49, 50-59 and 60+) and the most popular professional role was consultant; other options for this question included undergraduate, foundation doctor, core trainee, specialist trainee, non-trainee and retired. This aligns with the question about maturation of the movement as it seems FOAMed is especially well received by a younger generation of medics and learners. It was not surprising that the majority of respondents (66%) were RCEM members; more needs to be done to find out who amongst consumers of RCFN resources are not members and what the College can do to engage with them.

Responses to the questions concerning FOAMed’s relationship with the curriculum and reflective learning reveal a degree of ambivalence. I was surprised by this as I assumed recognising or signposting FOAMed resources within the curriculum and developing a tailored reflective learning diary would help to support the continued development of the RCFN. 55% of respondents claimed that curriculum mapping is a positive feature of the RCFN, yet 84% believe that it should be recognised or included within the RCEM curriculum. The tension between the two – a somewhat timid endorsement of curriculum mapping in the first question yet a pronounced need for recognition in the latter – could very well be attributed to the ambiguous wording of the question. On reflection I should have been much clearer about what I meant by ‘recognised’ and ‘included.’ It is now apparent to me that I should have asked if the RCFN should be signposted in the curriculum so learners know they are ‘endorsed’ resources that allow them to follow or reflect on what can be disparate learning pathways. The data suggests that mapping resources is enough and that I over-determined the importance of FOAMed to curriculum development. However the manner in which the curriculum is presented could learn some composition rules from FOAMed and become more multi-modal to align with leaners’ expectations.

A similar degree of ambivalence was also evident in the questions relating to the RCFN’s relationship with reflective learning. 47% of respondents claimed that a reflective learning log would improve their experience of the RCFN but only a modest 21% stated that it would ‘very much’ improve their experience. This brings one of the major tensions of the FOAMed movement to the fore as learners are using these uncredentialised resources in heavily credentialised educational environments. This is supported with the most popular response to the next question of ‘do you struggle to get sufficient recognition for activity undertaken on the RCFN’ where 42% of respondents claimed that ‘I don’t use the RCFN for assessment purposes.’ Although this is a small data set responses from the questionnaire suggest that RCFN resources should be free from assessable components and that any reflective diary should be flexible and voluntary. The data made me aware that more work needs to be done to establish the efficacy of FOAMed in our learning communities in order to establish if it leads to tangible improvements in learning outcomes (for exams) and to improved experiences between learners and their faculty (Fieldstein et al 2012, Petrides et al 2015).

The final question was an open-ended one designed to capture broad data and suggestions about how the RCFN could be improved. Completion was voluntary and there were sixteen responses that generated twelve categories of answers; there were 20 occurrences of these categories with the most popular category appearing 4 times, with 7 occurring once each. No coding was implemented for these responses given the low volume of data involved. The pie chart below presents the narrative categories and their number of occurrences:

screen-shot-2016-11-23-at-10-53-38

Image 13: Occurrences of comments to free text question in SAQ

Although a small data set the narrative responses to the ninth and final question (‘What would improve your experience of the RCFN. Please enter some general comments here’) generated some valuable responses. Three themes emerged from the narrative data and they are; learners’ desire for tangible proof of activity, which contrasts with the sufficient recognition question from earlier in the questionnaire where users claimed they don’t engage with the RCFN for assessment purposes; several responses also suggest that we need to do more about educating stakeholders about the purpose of FOAMed as they were conflating it with features (especially related to assessment) traditionally associated with VLEs. The final theme anticipated something that emerged from the interviews, and that is FOAMed’s status as a discourse. A great deal of attention – here and elsewhere – has focused on FOAMed’s attempts to critique and re-shape established discourse, but there’s an inherent danger. FOAMed is constructing a certain type of learner or learning experience that could be problematic and result in potentially marginalising significant demographics within our cohort of learners.

The highest number of responses expressed a desire to implement a tracking or recording system for activity undertaken on the RCFN. This of course is misaligned with the self-directed reflective philosophies which inform the movement. One user requested a ‘yearly one click download of time spent’ (29/3/2016 8:50 PM)[1] accessing the site, whereas another asked for a function to ‘get recognition/official log of time spent accessing it’ (21/3/2016 11:48 AM). A paradox emerges from the data here as the closed-ended questions suggest proof of activity is not required, yet the narrative responses demand it; it seems that the cultural requirement to document all learning is not an easy one to shake-off or accommodate. It is also evidence of cultural lag or misalignment between the types of resources learners are engaging with and those ‘approved’ by credentialing and assessment bodies. Other users asked if there was some way for it to be ‘recognised in appraisal. In time as these forms of education…become the norm I am sure it will be accepted within appraisal,’ whilst another asked for ‘guidance on what counts as CPD and how to demonstrate it. Currently all done for personal interest and not recorded formally by me’ (18/3/2016 6:02 PM, 18/3/2016 9:16 AM). These two comments are crucial as it reveals that the RCEM has an important role to play in delivering a viable system for capturing and recognizing (without overly formalizing) learning activity undertaken on the RCFN in a manner that will be validated for appraisal purposes.

Several responses were significant as they highlighted a degree of confusion about the type and purpose of the RCEM’s elearning portfolio. In other words it appears that we have to think about how we educate members about some of our educational resources. One user requested ‘online questions creating CPD’ (18/3/2016 7:59 AM); RCEMLearning is home to a substantial number of sessions which do just this, so I feel encouraged to reexamine how we differentiate between assessable elearning and FOAMed resources as learners may be struggling to tell them apart. Other respondents also conflated the functions of VLEs and FOAMed by asking for a way to ‘record things completed – without having to reflect’ (29/3/2016 4:26 PM), whereas another stated that ‘reflective bits at the end do not help but hinder me continuing…More exam resources would also be helpful’ (18/3/2016 10:07 AM). Of course as an organisation we have a responsibility to accommodate our stakeholders who possess a diversity of learning preferences, but the problematizing of reflection alluded to here and the request once again for learning resources which are quantifiable suggest more awareness raising exercises need to be carried out.

The final theme that emerged from the free text responses was significant if unanticipated. As outlined a great strength of FOAMed is its ability to critique and question emerging and longstanding clinical practices in formats that are educationally and technologically innovative. FOAMed advocates are often highly accomplished clinicians, educators and users of social media but this could potentially alienate as many learners as it attracts. One respondent claimed that ‘older docs have no idea of the resource or scope’ (26/3/2016 6:14 AM). Whilst the RCEM is working hard to attract digitally literate learners it may be alienating or under-using demographics of its membership who are reticent to engage with the movement. This will be discussed in the analysis of the podcast interviews, but it represents a significant area for further research and strategising at an organisational level.

Although the data set was relatively small the SAQ was a valuable exercise. It generated some useful demographic and attitudinal data to be triangulated into the podcast interviews, although it paradoxically revealed a degree of ambivalence rather than clarity in terms of attitudes towards the RCFN, especially when contrasted with usage patterns. However it illustrated that at an organisational level we need to bring FOAMed into reflective practice to enable it to be recognised as ‘legitimate’ learning activity.

4.3 Semi-structured podcast interviews

As outlined in the methodology I used Braun and Clarke’s (2008) model for conducting thematic analysis. The following themes were identified as the most significant from this data set; framing of dialogues, organisational strategy, and credentialing. Although not as significant in terms of discussion levels one other notable theme emerged and that is FOAMed as a discourse. The rest of this section will analyse each of these thematic groupings. Across these themes perhaps the key idea to emerge is that FOAMed as both dialogic pedagogical process and re-usable and sharable educational ‘product’ could have enormous benefits for how the College formulates its educational strategy, how the curriculum is formed and how the specialty defines itself in uncertain times. Indeed the repercussions for how the RCEM incorporates FOAMed into its educational strategy rivals pedagogical concerns for importance, especially in terms of how they impact learners’ educational development and CPD. The emphasis on strategy and organisational change parallels recent literature on OERs and mirrors the cultural imperative to embrace openness in order to stay relevant (Allen et al 2015, Sclater 2009, Albright 2006, Weller 2010, Wiley and Hilton 2010).

i) Framing of dialogues

This was a recurring and complex theme. ‘Framing’ refers to how the interviewees responsible for determining educational policy for the RCEM – along with those who produce and consume FOAMed resources – play a significant role in formulating dialogues about education. In turn this framing can be distilled into three subthemes; personal professional development, implications for the development of the specialty and broader abstract questions about what FOAMed can contribute to the changing nature of medical education.

There was a palpable sense of how involvement in the FOAMed community underpins a sense of individual professional development for the interviewees. Andy Neill (AN) claimed that it’s ‘good for me, good for my education, good for my performance’ and that it helps ‘prevent burnout’ (AN 12:26)[2].  Damian Roland suggests that FOAMed provides learners with a degree of flexibility as it ‘allows you to select the way you want to get your education’ (DR 14:56), which is significant as it means learning resources align with the demanding professional contexts the majority of learners operate in. FOAMed’s extensive use of social media enables it to support dialogic modes of education as activities such as live Tweeting from conferences allows individuals to reflect and ‘comment on other peoples’ reflection’ (DR 21:20).

The interviewees – but specifically Jason Long – agreed that one of FOAMed’s strengths is promoting self-directed learning as you ‘only get out of FOAMed what you put it’ (JL 4:40). This contrasts with more didactic methods found on conventional VLEs and reinforces that FOAMed can be used to start a review of ‘my way of doing things’ (JL 16:40). Simon Laing argued that it also helps maintain motivation to learn ‘as I feel like I’m not being pushed to do so. Because [it’s] something I want to do rather than something I have to do’ (SL 4:40). The relatively unstructured nature of FOAMed resources also ‘opens up other avenues of learning’ (SL 5:30) which appeals to learners who thrive on autonomously searching for educational resources.

Although a somewhat nebulous concept it also emerged that involvement with the FOAMed movement helps members of the community define new standards for professional and clinical excellence. Dr Neill captured this by stating that ‘FOAMed is a reflection of a desire not [just] to be a competent EP [Emergency Physician] but to be brilliant,’ and involvement with the movement shifts the ‘competency bar’ (20:13, 22:22). Its flexibility is once again important here as FOAMed ‘allows people to choose the methodology to reach a standard to practice’ (DR 16:29), something which might be more difficult to accomplish in more didactic modes of online learning.

Another sub-theme to emerge under professional development was that FOAMed can contribute to the development of the speciality’s educational provision. Dialogic themes were again significant here as FOAMed can act as a ‘commentary on established practice.’ A charge occasionally levelled against FOAMed is that it is inherently iconoclastic and only seeks to challenge rather than endorse established practice, but Dr Neill claimed that ‘a lot of time [FOAMed argues that] established practice is quite good’ (AN 14:49). However the spirit of critique that imbues the FOAMed movement also guards against complacency creeping in as it is ‘good at questioning dogma’ (JL 17:00). Individual word choices – such as ‘commentary’ and ‘questioning’ – are significant here as they reveal how FOAMed is underpinned by a dialogic impulse.

Dialogic themes were also evident in replies to questions about FOAMed’s potential contribution to curriculum formation as it can ‘raise questions about what should be in it…[it’s] a forum for discussion’ (AN 16:03). Staying attuned to the conversations taking place on FOAMed could help the RCEM keep the curriculum relevant to real-life issues (Cullen, Harris, Hill 2012, Johnson et al 2012). The interviews also helped me realise that my choice to include ‘legitimise’ in question one was problematic; inclusion in the curriculum wouldn’t automatically legitimise FOAMed resources, and discussion of the curriculum in the learning resources themselves does not concomitantly legitimise the curriculum, but they do ‘help you reflect [on] things that are currently being talked about’ (SL 9:00, AN 29:01). In times of information overload the RCFN can ease this burden by signposting people to material that will ‘enable them to become better doctors’ (JL 11:01). Curation of material is beneficial as ‘there’s too much stuff out there for me to read’ (DR 31:27) and FOAMed can help signpost in this regard.

Although the extent to which FOAMed is changing long-established practices is yet to be determined it’s certainly true that it’s part of a broader change in how knowledge is produced and disseminated. Specifically it offers a ‘commentary on…traditional textbooks,’ and it’s adept at picking ‘out things (i.e. topics) that don’t necessarily get covered by textbooks’ and ‘that’s what FOAM does par excellence’ (AN 29:22, DR 15:30, 16:05). Aside from knowledge production it also helps reinforce the importance of reflection which is an increasingly significant feature of ongoing medical education. Other leading UK-based FOAMed sites such as St Emlyns also contribute to this as ‘if FOAM does one thing very well is that it pushes that reflection’ (DR 19:39). Dr Laing observed that ‘reflection is a part of producing FOAM’ (13:06) which helps to embed the importance of reflection into changing cultures within medical education. Finally FOAMed resources parallel with the increased emphasis on efficiency in NHS systems as ‘from a time efficiency point of view FOAMed is absolutely brilliant. It might take ten hours to produce a podcast but you could educate thousands of people’ (SL 16:00). Crucially FOAMed can help inert content become active and inclusive conversation (Moran et al 2011).

ii) Organisational strategy

It was somewhat surprising how significant this theme became in the course of the interviews. On reflection it is fair to say that organisational strategic responses to FOAMed rivals pedagogical questions in terms of importance. There was consensus amongst the interviewees that it can only be a positive move that the RCEM has decided to embrace the movement and educate members via FOAMed (AN 5:31), especially as a core tenet of a medical college should be its educational philosophy and portfolio (AN 5:40). Although he was ultimately proved ‘comprehensively wrong’ Dr Roland admitted to some serious reservations when the RCFN was launched as he believed ‘it kills the spirit’ the moment an institution gets involved in FOAMed (DR 9:40). Yet this comment illustrated an underlying issue with ongoing institutional support for the movement as the spirit could still be ‘killed’ if an appropriate strategy to support reflection and recognition of learning undertaken via FOAMed is not devised, or if that strategy is too formalised.

The comments of Dr Long as RCEM Dean were significant in this regard as he plays a pivotal role in formulating the organisation’s educational strategies. He acknowledged that the RCEM needs to deliver ‘a mix of learning styles and teaching styles [that] fits into peoples’ lives’ and that we could be left behind if the movement was ignored (JL 2:49). Interestingly he also claimed that FOAMed helps to encourage other types of college activity – such as attending study days and conferences – as the network’s coverage of these events raises awareness about the standard of educational and clinical presentations delivered at them (JL 3:30/11:00).

A further tension identified in devising appropriate strategic responses rests with the fact that the FOAMed movement develops rapidly whereas the regulation and bureaucratic administration of medical education moves very slowly. As an educational movement FOAMed is ‘responding to the governance needs of someone else’s objectives’ (DR 24:45), but this is compounded as ‘everything with an RCEM badge’ must go through ‘some kind of standard setting’ to safeguard reputational factors (JL 5:31, 10:49). The organisation’s conception of what constitutes CPD and lifelong learning must keep pace with the changing nature of consultanthood, especially as new cohorts of members will progress through training with an expectation that FOAMed-style resources are an accepted form of practice (JL 26:38, DR 30:40). The need to formulate relevant strategic and regulatory responses is becoming more pressing as FOAMed’s educational worth is becoming acknowledged and it’s reaching a point where clinicians ‘can’t imagine many EDs in 5 or 10 years time not having their own FOAMed site’ (SL 29:52). It was refreshing that user or stakeholder experience was a central concern of the interviewees as that can often be overlooked when formulating strategic responses to OERs (Hilton et al 2013).

iii) Credentialing

Discussions about credentialing illustrated the complexity of issues around quality assurance, recognition and regulation. Of course there’s a duality to the credentialing debate as it involves looking at how institutions approve or acknowledge FOAMed, and how individuals are able to receive appropriate recognition for engaging with it. One engrained cultural issue related to quality assurance concerns how more traditional educational resources were peer reviewed as ‘no one ever mandated to see if a textbook was quality assured. It’s about clinicians [in their appraisal portfolios] being honest about the work they’ve done and what they’ve reflected on (DR 35:00). This is compounded as FOAMed is part of the evolving debate about what constitutes valid CPD activities (AN 31:40); furthermore it is also challenging what passes as ‘knowledge’ and how you stay up to date in rapidly changing environments (Robinson et al 2014). Reflection will help to embed FOAMed into appraisal systems, and FOAM has contributed to making reflection respectable, but reflection becomes ‘meaningless when you feel like you’ve got to do it’ (DR, 38:00, 22:38). The organisation has a responsibility to ‘reflect how people now learn’ but incorporating these changes into official documentation is incredibly difficult when dealing with complex regulatory organisations such as the GMC who approve significant changes to curricula (JL 14:08).

There is a danger of succumbing to the pressure to ensure tangible provability is available for all learning resources. Dr Laing warned against the danger of over-credentialing which he experienced from other FOAMed sites he managed, where assessable CPD-approved components added unnecessary complexity for authors and users to the extent they were ultimately removed (SL 23:52, 18:35). The RCEM is also responsible for creating ‘a culture in which people feel they need to provide tangible proof of learning’ (SL 4:12), and yet Dr Roland made the insightful point that it’s time for learners to initiate a discussion ‘about how we assess and test our learning…[We need] more mature engagement between people who are asking for assessment and people who are producing the learning (DR 26:00). This captures another of the dialogic spaces FOAMed is opening up as conversations need to be ongoing between learners and their assessment and regulatory bodies.

iv) FOAMed as discourse

An admirable characteristic of the FOAMed movement is that it represents a ‘sharing of mutual enthusiasm (AN 02:52). However a potential danger is that this enthusiasm supersedes the objective evidence-based foundations that FOAMed is predicated on and produces a discourse which enforces new orthodoxies as much as it challenges existing ones. A consequence of this is that ‘people are seeing FOAM as a concept which people who actually delivered it don’t agree with’ (DR 8:05). The term has become ‘synonymous with a certain ethos or type of individual’ but there is a danger this ethos and construction of personalities or subjects via social media or educational resources could alienate as many people as it engages (DR 17:23). Interviewees are cognisant of this issue as the ‘nature of being involved in it means we’re one end of the spectrum and are keen learners. [The] difficulty is bringing out the mean’ (SL 18:13), but the way FOAMed constructs identities and its own discourse could complicate this. FOAMed may well represent a burgeoning ‘community of practice’ but we must be aware at an organisational level of the members who feel excluded from this community.

4.4 Summary

Although data sets for both methods of collection were modest they generated some valuable findings. The questionnaire revealed an inherent tension evident in learners’ motivation and engagement with RCFN resources and the demands for tangible proof of learning which permeate the educational cultures they operate in. The interviews underlined the importance of reflection to FOAMed but also acknowledged that over-credentialing the movement potentially represents one of the biggest threats to its ethos. There was consensus amongst the interviewees that the pedagogical merits of FOAMed are now recognised as valid, but work needs to be done to devise appropriate organisational strategies to accommodate FOAMed. These include mechanisms to embed reflective practice in an agile way and to ensure it engages with users that its emerging discourse could otherwise alienate.

[1] References for quotes from the free text question of the SAQ will cite the date and time the comment was entered on SurveryMonkey. This will enable learners to look for them in the full record of the SAQ available in the Appendix

[2] References for quotations used from the interviews will provide the interviewee’s initials and the time in the podcast the comments can be found. This will enable readers to navigate straight to the relevant section of the podcast.

Abstract
Abstract
Acknowledgements
Acknowledgements
List of illustrations, tables, figures & abbreviations
List of illustrations, tables, figures & abbreviations
Introduction
Introduction
Review of the literature
Review of the literature
Research design & method
Research design & method
Presentation and discussion of findings
Presentation and discussion of findings
Conclusions
Conclusions
References
References
Appendix: Questionnaire responses
Appendix: Questionnaire responses