Maintaining assessment integrity in a digital format presents considerable challenges.
This article shares how we have adapted IWCF processes to ensure the assessment process remains valid, reliable and fair in an online environment.
Our recent newsletter highlighted a core value that underpins our work: our unwavering commitment to maintaining the highest standards of integrity. This commitment is critical as we adapt and strengthen our assessment processes in an increasingly digital world.
The COVID-19 pandemic accelerated the transition from paper-based to digital formats. Post- pandemic, many institutions, including IWCF, have embraced online training and assessment due to its flexibility and accessibility.
With this shift comes a significant concern: assessment security. Assessment and awarding bodies now need to consider new types of misconduct and negative behaviours that may not have previously been considered.
Safeguarding high-stakes, summative assessment
Assessment can generally be categorised into two types.
Formative assessment is carried out by the instructor, throughout the learning process. It is often low-stakes (informal, no serious outcomes) and can include quizzes, classroom discussions, exercises and other activities designed to check understanding and provide feedback.
Summative Assessment is often high-stakes, as the outcome determines whether someone is qualified or competent for a role. Created by the assessment/awarding body, it evaluates whether a candidate has achieved the required standard of knowledge and competence.
IWCF delivers high-stakes, summative assessment. Negative online behaviours such as cheating and dishonestly can have catastrophic consequences for the safety our industry. It is critical that our assessments must remain trusted and effective, ensuring those certified are skilled, competent and capable of managing the risks involved with well control operations.
No single tool or feature can fully safeguard the assessment. That is why we apply multiple layers to create a robust authentication and security framework.
Virtual invigilation
Our invigilation partner, VICTVS, plays a key role in overseeing the assessment process. Invigilators (also called proctors) support candidates, monitor the test and help to prevent malpractice.
- ID checks: Invigilators verify each candidate’s ID to confirm they are the authorised test taker.
- Room checks: Candidates must use their webcam to do a 360-degree scan of their room before starting. This confirms they are alone and have no unauthorised materials. The invigilator also ensures the candidate’s video, sound, lighting, and workspace meet IWCF standards.
- Secure PINS: Candidates receive a one-time secure PIN to access their assessment.
- Multi-mode monitoring: Invigilators monitor candidates from multiple angles during their assessment. This includes a mobile device behind them, a webcam on their assessment device and live screen sharing. This approach reduces cheating and ensures candidates follow requirements throughout.
Assessment Design
The online framework also contributes to the security and integrity of the assessment process.
- Randomised question selection: sections of the assessment are randomised to ensure each candidate receives a unique sequence.
- Question banks: the system selects questions from a varied pool to ensure differences between candidates’ assessments.
- Answer order randomisation: Shuffling the answer choices encourages critical thinking and reduces cheating.
- Level-appropriate questions: Including scenario-based questions that reflect real-world well control challenges tests problem-solving and critical thinking rather than memory recall.
Data monitoring
A major advantage of online assessments is the wealth of data they generate. Unlike paper-based assessments, digital platforms provide instant insights into pass rates and performance trends across our global network of accredited training centres.
Detecting integrity issues
- Unusual patterns such as sudden spikes in pass rates in certain regions could highlight compromised security.
- Technical issues: We can identify lags or connection problems and provide support.
- Global comparison: Analysing performance across regions helps us to ensure fairness in assessment conditions.
Data monitoring also plays a significant role in ensuring the validity and reliability of our assessment questions. We will explore this further in a future post.
Commitment to integrity
With the right measures in place, assessment bodies can strengthen the overall assessment process, addressing concerns while providing a secure testing environment.
IWCF will continue to be vigilant and adaptable as learning and assessment becomes increasingly digital and new challenges present themselves.
We are grateful to our team, instructors and members who work daily to protect the integrity of IWCF assessments. We share a responsibility to ensure that our programmes remain not only effective, but fair and secure.
Many of the points mentioned in this post are brief summaries of more in-depth, detailed IWCF procedures. If you have any questions related to our online assessment process, please contact information@iwcf.org.
