30 July 2020
COVID-19 had an unprecedented impact on how the IFoA ran our examinations, so the feedback of our member's experience is vital in understanding how the IFoA can improve upon the delivery of these exams in the future.
Following the April 2020 examinations, over 2,600 students completed our post-examination survey. This was the largest number of students ever to complete an IFoA post-examination survey. Students gave feedback on a range of topics, including; COVID-19 communication, examination bookings, the online examination platform, paper difficulty and format, and provided suggestions for improvements. We also asked for feedback from candidates who cancelled their exam place, in order for the IFoA to better understand how to help them access exams in the future.
We would like to thank everyone who took the time to complete the survey and fed back through Student Consultative Forum (SCF), your input has helped inform and improve our communications and processes ahead of the September examination session.
What you told us in numbers
The April 2020 Post Examination Survey results showed that students found the online examination platform easy to test and use, showing an increase of 12% satisfaction compared to those sitting online exams in September 2019. Students did report some issues around uploading paper and improvements to be made in how we communicate about IFoA exams.
- 99% of the candidates were able to test the platform.
- 97% of candidates reported the online platform was easy to use.
- 95% of candidates reported downloading their exam paper without issue.
- 85% of candidates uploaded their paper without issue.
- 93% of candidates understood the technical requirements for the exams.
- 83% of candidates said the FAQ’s provided them with the information they needed.
- 75% of candidates felt the IFoA kept them informed about how COVID-19 was affecting their exam sitting.
Read the full results from our April post-examination survey
Student consultative forums feedback
After each exam session, we also hold Student Consultative Forum (SCF) meetings, where our student members discuss their experiences of recent exams, as well as the broader student experience.
We use feedback from the post-examination surveys and the Student Consultative Forum meetings to improve our processes and communications. The meeting notes and student feedback received is shared with and considered by our various IFoA committees and decision-makers.
Student feedback themes from our April 2020 examinations
We have summarised some of the key discussion points from our recent SCF meetings, along with common themes from the April post-examination survey, as well as the actions we have subsequently taken to address these issues and improve our students’ experience.
Our students told us that the time taken by the IFoA to announce details of the April 2020 session had made them feel frustrated and anxious
We understand that students would have wished to have had this information sooner. Like many other international actuarial associations, we had to carefully balance the wellbeing of our candidates and their families, with the desire of many student members to complete their current studies, during unprecedented global circumstances.
While the majority of other actuarial Associations and most Professional Bodies chose to cancel their examinations, the IFoA chose to investigate whether we could run our examinations in an online format.
Moving the exams online involved our Examiners evaluating each examination paper to establish whether it was suitable to move to an online format; rewriting our Assessment regulations including technical requirements and supporting documents, and ensuring our online exam platform provider could handle the uploading and downloading of over 7,500 scripts. As soon as we were able to confirm that we could move the majority of our exams online we, communicated all the essential details to our student members. All this was achieved within a period of nine working days from the announcement the UK decision to lockdown due to COVID-19.
While we understand that our students would have appreciated an announcement about examinations going ahead being made sooner, we hope students recognise why our announcements may have taken longer than they expected.
Improvements we have made as a result of your feedback
- We have already published most of the guidance for the September 2020 examinations. This includes new FAQ’s and specific emails to members with updates and instructions.
- Students have said they would prefer direct emails in relations to exam updates. We have started to send email updates to all students whenever new FAQs or guidance is published.
- We have recently announced to students that exams we are planning for the exams in 2021 to continue in an online format. We hope our members can now plan their study journey for the future.
Students wanted a more precise definition of the new examination formats
In response to student queries and feedback, we have provided further detail around the examination formats at the earliest possible opportunity, recognising this is a new approach to studying compared to previous exams.
As part of the development of our online examinations, we have decided that the September assessments will be 'fully open book'. During these examinations, you will be entitled to use any reference materials available to you – hardcopy, electronic, or online.
Improvements we have made as a result of your feedback
- We have provided updated guidance on the format of the CS and CM papers. In the upcoming weeks, we will be publishing some specimen questions to help students with their study preparation. We have published a further clarification on open-book exam formats which hopefully will assist students in preparing for their exams in September. For the avoidance of doubt: the materials you use are for reference purposes only. You should not copy either by pasting in material or typing it in manually word for word any content into your answer script. You can consider the material and express it in your own words. If you do copy, then this will be regarded as a breach of the Assessment Regulations.
- The IFoA will be running a webinar in late Summer on approaches to studying that will cover how to approach open book study.
Students wanted to know how pass marks are going to be set, taking COVID-19 into consideration
There are many factors that the Board of Examiners will consider when setting pass marks. They will discuss the reported difficulty of the paper, how candidates answered the questions, and if the available marks per question are appropriate. Other factors considered when pass marks are set include; student feedback on the paper difficulty and time to complete, alongside external factors such as how the change in the format of the exam sitting may have impacted candidates’ performance.
Explanations including the marking scheme, pass rate, what pass marks were agreed, and any factors that have been taken into consideration are included in the relevant examiners' report.
Students in cohort 1 of CP3 could not upload their examination paper
Of the 17 assessments we ran, we experienced only one significant upload issue, which was specific to CP3. The issue only affected candidates in cohort 1 and a small number of students in cohort 2. We were able to resolve the matter in time for the majority of candidates in cohort 2 and 3 to successfully upload their papers.
The upload problem resulted from a server issue rather than an issue with the online examination platform and we apologise to all those students in Cohort 1 and those in Cohort 2 who were affected by it. The IFoA did accept scripts via email submission for candidates affected.
Improvements we have made as a result of your feedback
- We have now have put measures in place to ensure that the issue will not happen again in future examination sittings.
Students would like email confirmation their paper has been successfully uploaded to the online examination platform.
A recurring piece of feedback from SCF and GSCF members is the request to have email confirmation once a candidate has successfully uploaded their paper. At present, our online platform only displays a visual confirmation.
Improvements we have made as a result of your feedback
- We are aiming to have this functionality available for the September 2020 examinations, which will allow students to obtain proof of successful upload. You will be able to print, screengrab or save the receipt page as evidence of successful upload. If your upload process does not complete, you will have no receipt. Further details will be announced later this Summer.
Students reported issues in the download and uploading of their exam papers
Only students 77 out of 10,005 who downloaded their exam papers (excluding CP3 cohort 1) were unable to upload their examination script. Of those 77 who were unable to upload their paper, 35 did not test the upload/download ahead of their exam. The pre-exam testing is part of sitting an online IFoA exam and is going to be increasingly important in future sessions as we develop the online technology.
We recognise that a small number of candidates experienced upload issues, which may have been caused by their employers' security measures on company-owned hardware. Our updated policy for the September exams will provide instructions of what to do should you experience such issues.
We have also received useful feedback from SCF and GSCF members that should a student experience issues downloading or uploading their paper to clear their internet browser ‘Cache’ and retry the download/upload. We will provide this feedback in our future examination instructions.
Improvements we have made as a result of your feedback
- We have taken this feedback on board and will be providing you with instructions of what to do in the event you cannot upload your paper. Now that we have telephone capability restored for the September session, we have revised our procedures about allowing final scripts to be emailed to us.
- If you have completed all the pre-examination checks including upload and download of test papers, and you then experience difficulties with uploading the document during the 15-minute window either at the end of the exam or if you finish early during the exam period you can call the Examinations team where guidance and further instructions will be given.
During August we will be testing some additional improvements to our examination platform. Provided this is successful you will see the following enhancements for September:
- A new authentication process will involve Two Factor Authentication, where a code will be sent via SMS (Short Message Service) to your mobile phone. You will need to enter this code when logging in to the IFoA examination platform.
- When you receive your joining instruction it is important that you access the platform using the link provided. This will allow you to set up your password and verify or change your mobile phone number. Once this is completed you will receive an SMS with a code to allow you to view the welcome page and check the exam(s) you are sitting in the session are correct. You will then need to complete the necessary pre-checks which will be detailed on the platform.
- Confirmation of successful upload of your exam answer script. This includes providing a confirmation or receipt page once the upload is successful. This receipt page will consist of ARN, Filename, Filesize and a Unique ID which would be generated at upload time and kept in the database for validation purposes.
Many students living outside the UK have asked whether we can run exams at more appropriate local times?
Our examinations in September 2020 will have start times between 08:00 and 10:00 UK time. We appreciate that this will be disappointing to candidates who may find themselves sitting the exams at inconvenient times of their day.
Unfortunately, we do not have enough exam papers to run an exam multiple times over the day, and to run a single online-paper at local times slots would compromise integrity of our exams. We also rely on a team of IFoA staff based in the UK to monitor the exam platform and be available if issues arise.
However, you told us that for future online examinations you would like to select your preferred time slots from those offered, and that this is especially important to candidates outside UK time zones.
Improvements made as a result of your feedback
- We have introduced functionality on our booking system for some of our September examinations that allows candidates to select their preferred time slot, as long as there is availability. Unfortunately, to take advantage of this, you will have to book and pay for each examination separately. We acknowledge this may be inconvenient if you are planning to sit for several examinations. We are looking to rectify this for future sessions and to have a solution for the April 2021 session.
Useful information in regards to the September 2020 Examinations