INTRODUCTION
The introduction of problem-based learning (PBL) into higher education curricula comes from the medical education discipline, to help prepare students for solving patient problems in clinical settings during the 1960s (Barrett, 2006). The medical school at McMaster University in Canada is largely credited with pioneering this practice (De Graaff & Kolmos, 2007), followed by other institutions, often in their infancy of setting-up programmes, their immaturity providing agility. The term problem-based learning was originally coined by Donald Woods from McMaster with subsequent work carried out by Harold Barrows, both of whom have helped perpetuate the value and acceptance of what at the time was an emerging field. It is defined as:
“PBL can be defined best as the learning that results from the process of working toward the understanding or resolution of problem. The problem is encountered first in the learning process and serves as a focus or stimulus for the application of problem-solving or reasoning skills, as well as for the search for or study of information or knowledge needed to understand the mechanisms responsible for the problem and how it might be resolved” (Barrows & Tamblyn, 1980)
There is broad agreement in the definition of PBL, the most notable distinction between this and other forms, such as enquiry-based learning (Kahn & O’Rourke, 2005) is the order in which the problem is presented: explicitly, this is first, therefore coming before actions taken by students to gain knowledge, skills and understanding. The student-centred nature of PBL is a critical feature of this philosophy which is not as evident in other methods of teaching such as problem-solving learning (Savin-Baden, 2000). Some argue that PBL is an approach rather than a method and can be used for best effect at a programme level:
“PBL is an approach to structuring the curriculum which involves confronting students with problems from practice which provide a stimulus for learning.” (Boud & Feletti, 1998)
Arguably, one of the most important characteristics is that PBL is related to practice. It is and should be closely related to professional competences and those capabilities that are required of the graduate of the discipline (see for example Son, Lee, and Park (2016)). In medical education, this involves making a correct diagnosis and recommending treatment and for other professions, it includes preparing them for situations or scenarios that they will likely encounter, the specifics of which are unknown. Learning for a future context has been thoroughly analysed and approaches recommended that include the idea that university education should prepare students for an unknown future and to deal with situations that are new or are yet to be experienced (Bowden & Marton, 2003).
Critics of PBL argue that minimally-guided instruction produces inferior results when compared to guided instruction (Kirschner, Sweller, & Clark, 2006). Meta-analysis of outcomes of PBL and non-PBL methods based on literature conducted by Albanese and Mitchell (1993) found mixed results on a range of criteria, considering aspects including performance, cost and enjoyment of staff to teach. The authors concluded that caution should be exercised when making comprehensive, curricula-wide conversions to PBL (Albanese & Mitchell, 1993). Much of the evaluation of PBL has been conducted in the ‘home’ of PBL: medical education (Lohfeld, Neville, and Norman (2005), Hartling, Spooner, Tjosvold, and Oswald (2010), Marchais (1999)) and is therefore knowledge of its effectiveness is constrained by disciplinary boundaries.
PBL is credited with achieving wide-ranging pedagogical outcomes and changes to the ways in which students learn, with authors claiming it to be the most innovative instructional method conceived in the history of education (Hung, Jonassen, & Liu, 2008) and the most significant innovation in education for the professions for many years (Boud & Feletti, 1998). PBL is also credited with supporting the development of generic attributes (Wood, 2003). Whilst criticism exists as to the implications and outcomes of PBL, it is clear that when used as part of a wider approach to learning, with appropriate support, there is strong evidence of the benefits of PBL.
Engineering and pbl
As a discipline which is application-focused, Engineering is well-suited to achieving pedagogic benefits through PBL in course design. With Engineering, there can be confusion around the abbreviation because ‘PBL’ can refer to ‘project-based learning’, which often involves similar practices to problem-based learning. For example, it is reported that “what one institution practises as problem-based learning may look very much like what another institution practises as project work.” (Kolmos, 1996). A recent comprehensive review of PBL and comparison of different models established in different parts of the world has shown there is confusion in the field (Servant-Miklos, 2020). Engineering education utilises both project-based and problem-based work, alongside design-based learning (Barak, 2020) so one has to proceed with caution to ensure terms are not misconstrued or misinterpreted. For the avoidance of doubt, when we refer to PBL in this paper, we are doing so as problem-based learning, as defined above.
PBL has been used effectively in university Engineering programmes for more than thirty years, (Woods, 1996) with some evaluating that its use is better-suited earlier in programmes with project work occurring latterly (Perrenet, Bouhuijs, & Smits, 2000). Whilst consensus exists as to the merits and usefulness of PBL, some argue that it should form only part of a programme of Engineering education at degree level (Mills & Treagust, 2003). Due to the applied-nature of Engineering education, problems that are authentic in nature have considerable scope to elicit various types of learning such as application of theory, acquisition of new knowledge and the development of a range of transferable skills including problem-solving, teamwork, communication; contributing to potential professional practice. In similar contexts, longitudinal evaluation has shown three main advantages of PBL: 1) it facilitates training in technical, personal and contextual competences 2) real problems in the professional sphere are dealt with and 3) collaborative learning is facilitated through the integration of teaching and research (Ríos, I.D.L., Cazorla, Díaz-Puente, & Yagüe, 2010).
A key issue relates to authenticity or how “real” a problem in PBL is. Authenticity has been shown to have positive results in other disciplines for example through the use of clinical anecdotes in mental health education (Treolar, McMillan, & Stone, 2015). From the work undertaken by Ríos, I.D.L. and colleagues, it is clear that the methodology used within Engineering curricula to devise projects that had relevance to local populations was an important factor in the design of the course (Ríos, I.D.L. et al., 2010). The proximity of professional practice to PBL, sometimes referred to as authentic PBL (Barrows, 2000) or aPBL provides advantages over traditional methods of instruction, as evaluated by Woods (2012): troubleshooting skills are better; meaning-focused learning is promoted instead of surface approaches to learning; surveys of graduates and alumni are positive and student motivation is higher. In addition, the following careers skills are developed: problem solving, teamwork, confidence, life-long learning, information gathering, interpersonal relations, and communication (Woods, 2012).
How might one achieve aPBL in Engineering to help elucidate such benefits? Problems originating from, defined and supported by industry offer a means of achieving this. The use of consultancy projects with businesses is a commonly adopted approach within university management courses (Annavarjula & Trifts, 2012). For many years, the Engineering Department at Lancaster University in the UK has used such a method as part of accredited four-year long Master of Engineering (MEng) integrated undergraduate degrees. This paper is a case study that uses student feedback data to stimulate reflection by the authors, including those responsible for convening this part of the course. It therefore draws on the experiences and perceptions of both the students and teaching staff to recommend enhancements to the course and ultimately to support the ongoing development of high quality aPBL. The next section provides further detail on the practical aspects of how the module has been managed. We use student feedback over four years (2016-2020) to draw on reflections which we believe will be of interest to people involved in establishing and/or co-ordinating aPBL activities within higher education allied to Engineering and physical science disciplines.
METHOD
ENGR 445 is a 15-credit core module for all integrated masters (MEng) degree courses at Lancaster University, which has five undergraduate programmes: Mechanical Engineering, Chemical Engineering, Electronic and Electrical Engineering, Mechatronic Engineering and Nuclear Engineering. The module is split into three discrete sections: a taught component lasting ten weeks, delivered in Lent term of Year Three; a two-week industry-led PBL project in Summer term of Year Three and a two-week industry-led PBL project in Lent term of Year Four. Historically, the module had been delivered with a high degree of separation between the taught component and the two industry-led PBL projects. Assessment is typically two essays for the taught component (each worth 25%) and two group projects (worth 25%) for each industry-led PBL project. The taught component is intended to introduce students to a range of innovation, business and leadership concepts. Content includes intellectual property; strategic development; new product development; lean manufacturing; financial performance; human resource management; marketing and new venture creation.
The recruitment of industry-led PBL projects is co-ordinated and largely achieved through the dedicated industry engagement team for the Engineering Department. Through its professional networks, business support initiatives and reputation, it is generally able to recruit more projects than required. The aim is to secure sufficient projects for the numbers of students (generally teams of five) and relevant to discipline areas; see above for the course disciplines. Any organisaiton wishing to collaborate on such a project is required to complete an expression of interest (EoI) constituting company background, project brief and intended outcomes. A panel made up of academic and professional staff select the projects to be put forward to the students. Students are presented with the projects during a lecture, provided with the briefs and asked to submit their top four preferences. The convenor then uses this data to assign each student into a team. An academic is asked to supervise the team and mark the report, providing feedback. This process is the same in Year Four as in Year Three. The team make contact with the company, arrange a site visit if appropriate and an introduction (or ‘kick off’) meeting. The team have two full weeks to complete the project and submit their group report. Measures such as peer observation aim to assist with any unequal efforts from team members.
The industry-led PBL projects are preceded by some preparatory lectures which provide students with information on professionalism (including content such as meeting conduct, non-disclosure, communication), project management (building on content elsewhere in the curriculum) and practical/logistical arrangements (such as travel, booking meeting rooms etc). From the point at which the team are assigned their industry-led PBL project, a large emphasis is placed on the students having ownership of their project and hence responsibility for the success of their collective endeavour lies with them. The way in which students go about their projects generally reflects teaching of project management found elsewhere in the curriculum and includes the use of tools and techniques such as risk registers, work breakdown structures, Gantt charts and Pugh matrices. There is naturally an element of idiosyncrasy between groups, reflecting the roles, styles and approach of individuals within the team.
Depending on the nature of the project, student teams may choose to structure their work in a way that best supports the achievement of their objectives, and may include time in computer labs (for example with projects that have a large design, simulation or modelling aspect) or in laboratories (for example where samples are required to be prepared and tested) or in workshops (for example where prototypes are required to be built and tested) or a combination of settings. Some students may also avail themselves of seminar rooms, using white boards for idea generation. Students are encouraged to write their report as they go although in some cases there is evidence this is left to the end. Marking criteria for each group report is: management of project (20%); technical content (30%); professional skills (30%) and report (20%).
To scaffold learning, each team has access to the module convenor for any practical, logistic or non-technical matters via an open-door policy, encouraging students to discuss challenges and help identify solutions. For technical expertise, the students also have access to a dedicated academic supervisor who provides appropriate direction, guidance and advice on any methodological approaches being used. Access to facilities for the purposes of fabrication and testing is often enabled by technical staff who are generally willing to provide additional guidance on particular processes and equipment. Following the submission of their reports, students are encouraged to complete a short, anonymous evaluation questionnaire using the university’s virtual learning environment. This comprises seven closed and two open questions:
1. ‘How did you rate the industrial project as a whole?’
2. ‘How did you rate the support from Lancaster Product Development Unit during and pre-project?’
3. ‘How did you rate the relevance of the industry project to your degree?’
4. ‘How do you rate the experience and knowledge gained during the industrial project?’
5. ‘How did you rate the commitment and contribution from the industry partner?’
6. ‘To what extent would you agree with the following statement: the project was well organised.’
7. ‘To what extent would you agree with the following statement: I felt sufficiently prepared for the project.’
Responses for the first five use a five-point Likert scale: very poor, poor, satisfactory, good and very good. Responses for the final two closed questions use a five-point Likert scale: strongly disagree, disagree, indifferent, agree and strongly agree. The two open questions were: ‘what were the most valuable aspects to the Industrial Project?’ and ‘how could the industrial project element of the module be improved?’ It is fair to point out that the questions are themselves relatively limiting and will be improved in future to elicit feedback related to pedagogic quality. Notwithstanding these shortfalls, the analysis of data, particularly related to the open questions is of use in the design and execution of authentic PBL in higher education.
Data were collected from June 2016 to March 2020, providing eight data sets across four cohorts of students. Response rates varied from 14.8% to 40.4% (Figure 1). From 409 students participating in projects, a total of 110 responses were received over this time period; it is these data that provide the basis for the results. Note that students are asked for feedback for each project completed (IE one in Year Three and one in Year Four). Mean values were calculated for aggregated responses to questions 1 to 7. For the open questions, manual coding was undertaken based on twenty categories, ten for negative and ten for positive (Table 1). Example responses (Table 2) show how the coding methodology was applied to responses to questions 8 and 9.
RESULTS AND DISCUSSION
Importance of internal organisation
One of the most apparent observations from the results is the importance of internal organisation for students. It is clear from the responses received both in the closed questions (Q2 and Q6) related to internal organisation (Figure 2) and the open questions (Figure 3), that this area is significantly important and should not be overlooked if student satisfaction is to be maintained or improved. The two highest reported negative categories of ‘internal organisation’ and ‘timing’ constitute 37.1% of total negative responses after coding, across all years. Further exploration of the data shows that this has changed over time with a peak during the centre of the data collection period which tapered either side of this increase.
Mixed support from external partners in pedagogic development
Across most categories for the closed questions, there is a general uptick shown in the last two data collection points, which is a positive direction of travel. This is affirmed when analysing total positive and total negative responses to the open questions over time, with the last two data points showing more positive than negative. Some recent changes have been introduced which have aimed to better bridge the formal taught component with the industry group projects. The major exception to the rising trend reported above is the commitment from industry partners (question 5), which has appeared to flatline (Figure 5). Evidence from the open questions (Figures 3, 4) suggests this is an issue of a two-ended spectrum, as the negative commitment from industry category (n = 13) is slightly higher than the positive commitment from industry category (n = 10), ordered sixth and fifth, respectively. This supports anecdotal feedback, suggesting some industry partners are extremely committed to both the endeavour and the educational experience it provides, whilst some are ambivalent to the point of nonchalance, displaying little regard for students’ development.
Students failing to connect previous experience as preparation
Evidence suggests that students who have taken part and completed the feedback do not necessarily connect previous experiences of similar activity with providing preparation for similar future activities. Figure 6 shows results to Q7 (‘to what extent did you feel prepared for the project?’) and bar the last two data collection points, all reported either the same or a reduction in feeling sufficiently prepared between third year and fourth year. We would have naturally expected students to have considered the first project as a means of preparing them for future similar situations and therefore this would have increased between years three and four. Given this did not happen, save for the last two data collection points this suggests more should be done on providing students with material related to PBL and the opportunity to formally reflect after their Year Three project, which is seen as an integral part of the learning experience:
“these experiential opportunities require careful planning and time for reflection must be built in if they are to be an effective way of providing university students with relevant employment skills, knowledge and awareness of employer culture.” (Lowden, Hall, Elliot, & Lewin, 2011, p. vii).
The omission of such a vital part of the learning experience provides an easy opportunity for the current convenor to make improvements and provides affirmation to others of the validity of reflection.
Apparent lack of connecting aPBL with employability
One surprising outcome from the data analysis was that students appear not to be making explicit connections between these authentic PBL experiences and their employability. This is evident in the frequency of the category ‘employability’ in the open questions which came eighth out of the ten used for positive, with six mentions across the eight data collection points (Figure 4). One explanation of this might be that students perceive their employability as being part of a richer picture that includes academic performance, voluntary work and paid employment, which all contribute to their skills development. However, it appears students who are failing to make connections between these experiences with employability risk being unable to use them when seeking employment. Given the range of transferable skills that can be demonstrated through these experiences with employers, this is something that requires more overt explanation with the students.
The following three results have been obtained from the experience in delivering and co-ordinating the activity described yet are not patent in the data collected. To exclude these from our reflections would have provided limited results to the reader, which we were keen to avoid, given our aspirations that our work be used for others to learn from. We have therefore included these as being supplementary to the data presented and contributing to a fuller description of our experiences.
Caution for control beyond the project team
A recurring point that we feel warrants more than a fleeting mention is accommodating what may be referred to as the thirst to prototype. This appears to be especially common amongst Engineering and new product development projects. Experience shows us that any short timeframe project, such as these that require the sourcing and supply of parts from third parties is a highly risky endeavour and is likely to have a substantive bearing on the achievement of deliverables put forth by the external partner. For example, a particular component from an electronics supplier may be the difference between a working prototype or not. The absence of a simple measurement device may mean key results cannot be obtained. Any short-term project which is contingent on the provision of supplies by a third party should, from our experience be discouraged. There have been several cases where external partners have promised to provide materials (data, parts, specifications, models, drawings or material samples), which have not been provided at an acceptable point in time for work to commence. This has been wholly discouraging for the students who feel unable to pursue the aims of the project as they were initially described.
Maintaining authenticity supports real-world preparation
There is a tangential but very important point to be made here. The hurdles and challenges faced in these projects are very authentic as they mirror some of those barriers that all professionals deal with on a regular basis. A clear example is that referred to above where delays from other organisations have a direct impact on workflow and hampers the ability of the team to progress the project. This proximity to the experiences of the “real world” can be advantageous as it helps students prepare for imperfection through responding to adversity. This can lead students to consider the value of being flexible, adaptable and solving unforeseen problems. However, this does not form a reliable grounding on which to base good quality learning experiences. Sufficient scaffolding needs to be in place to provide students with the necessary support for responding to the inevitable nuances created in this authentic environment. This resonates strongly with the concepts discussed earlier of preparing students for an unknown future (Bowden & Marton, 2003) to be an important responsibility of education providers.
Balancing student development with business support
Moving a step further away from the data collected and analysed here is a point that rests at the intersection of business engagement/impact and quality of learning. To take the first of these, there may exist within the reader’s institution, department or other function a priority to work with external stakeholders as part of a growing prominence of the impact agenda in HE (McCowan, 2018). This may inadvertently or even surreptitiously cause a shift of focus into the reasons for aPBL towards the benefits to the business (be that design, fabrication, testing, simulating etc) to the point where this may overshadow other priorities. One has to be careful in the management of such teaching-industry partnerships that the focus is not one solely based on the interests of the external organisation. Whilst this of course is an important feature that helps the authenticity of aPBL as discussed earlier, it needs to be attended to in a way that is balanced and complementary to the quality of learning experienced by the students. Having individuals in bridging roles that can simultaneously broker and exploit the impact agenda whilst never losing sight of high-quality learning ought to be a key feature of symbiosis in practice.
Limitations
There are a number of limitations to this case study that readers should be made aware of. First, the data collection method has been solely from feedback questionnaires with no opportunity to formally follow this up with students. Data has been therefore limited to only those respondents that elected to take part in the post-project evaluation. The questions in this evaluation are themselves not a good representation of how to seek valuable feedback and more consideration will be given to the modes of student feedback in the future. The effectiveness of these collaborations could further be better interpreted by having a similarly uniform means of collecting feedback from the collaborating businesses. The content of such data would be a helpful insight into how businesses view the projects and how improvements might be made from their perspective. Ultimately, this paper reports on one module and the feedback of students from one UK university department over a relatively short timeframe.
CONCLUSIONS
An important element to the success of authentic PBL projects is that sufficient resourcing is required for developing and securing authentic problems from industry, a notion reflected by those evaluating curriculum design and implementation for health professional programmes (McMillan, Little, Conway, & Solman, 2019). This is ideally suited to individuals traditionally associated with technology transfer or knowledge exchange offices. At Lancaster University, these people form a growing cross-campus team of business partnership professionals, some in the centre, some embedded within departments or faculties. It is important that these people who may be developing projects have sufficient knowledge of the discipline content of the courses and any facilities that may be available to the students. The resourcing for internal organisaiton should not be underestimated too; this has powerful effects on student satisfaction. Given that these projects are rich, student-centred learning experiences, it is of no surprise that far more time and energy is required than for other (more traditional) teaching, such as lectures.
The recruitment, organisation and co-ordination of these activities require rich external networks alongside knowledge of teaching, the curricula and academic regulations and practice. This is rarely achieved by a single individual working alone; our experience shows that whomever takes a leading role, close collaboration between academic and professional staff will be required for success. This means professional services and academic colleagues working closely together, with mutually agreed responsibilities. The optimum mix will be dependent on circumstances at each department, university and the individuals involved.
University staff members who are proposing or who are doing aPBL should be satisfied that their endeavours will reap considerable benefits to their students and to external partners. The authenticity of working with businesses with problems they face brings about a range of experiences; many are positive yet some of these can be problematic if promises of material is delayed. These can have consequences for the success of the project and motivation of students. However, exposure to the realities of working in industry is an important part of these experiences too and will no doubt prove to be reflective of the working lives students will go on to lead. Teaching staff should recognise and support this through appropriate scaffolding as attempting to eliminate it altogether would surely reduce the benefits to students.
Providing students with the formal opportunity to reflect using an appropriate structure such as the Gibbs reflective cycle (Gibbs & Andrew, 2001) is paramount if the students are to benefit from these experiential learning activities as much as possible. This will help them prepare for future unknown situations. Furthermore, making explicit connections between PBL and employability helps students recognise and articulate the skills they have had the opportunity to demonstrate. The value of students establishing their professional networks should also be a feature of any links university teachers are making.
We believe that businesses which bring the all-important “authentic” to aPBL should be better prepared to understand the educational importance of these activities for the students. We are not suggesting any transference of responsibility from university teaching staff to businesses; just that the latter simply has an increased awareness of pedagogy and how the best educational experiences can be achieved. This may even have an overflow effect and such appreciation may subsequently be absorbed into the organsiaiton, beyond the project. This would be a powerful side effect and presents a compelling argument for making material accessible and useful.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Print
