Maintaining assessment integrity in a digital format presents considerable challenges. 

This article shares how we have adapted IWCF processes to ensure the assessment process remains valid, reliable and fair in an online environment.

Our recent newsletter highlighted a core value that underpins our work: our unwavering commitment to maintaining the highest standards of integrity. This commitment is critical as we adapt and strengthen our assessment processes in an increasingly digital world.  

The COVID-19 pandemic accelerated the transition from paper-based to digital formats. Post- pandemic, many institutions, including IWCF, have embraced online training and assessment due to its flexibility and accessibility.  

With this shift comes a significant concern: assessment security. Assessment and awarding bodies now need to consider new types of misconduct and negative behaviours that may not have previously been considered. 

Safeguarding high-stakes, summative assessment  

Assessment can generally be categorised into two types.  

Formative assessment is carried out by the instructor, throughout the learning process.  It is often low-stakes (informal, no serious outcomes) and can include quizzes, classroom discussions, exercises and other activities designed to check understanding and provide feedback. 

Summative Assessment is often high-stakes, as the outcome determines whether someone is qualified or competent for a role. Created by the assessment/awarding body, it evaluates whether a candidate has achieved the required standard of knowledge and competence.  

IWCF delivers high-stakes, summative assessment. Negative online behaviours such as cheating and dishonestly can have catastrophic consequences for the safety our industry. It is critical that our assessments must remain trusted and effective, ensuring those certified are skilled, competent and capable of managing the risks involved with well control operations.  

No single tool or feature can fully safeguard the assessment. That is why we apply multiple layers to create a robust authentication and security framework.  

Virtual invigilation  

Our invigilation partner, VICTVS, plays a key role in overseeing the assessment process. Invigilators (also called proctors) support candidates, monitor the test and help to prevent malpractice.  

• ID checks: Invigilators verify each candidate’s ID to confirm they are the authorised test taker. 

• Room checks: Candidates must use their webcam to do a 360-degree scan of their room before starting. This confirms they are alone and have no unauthorised materials. The invigilator also ensures the candidate’s video, sound, lighting, and workspace meet IWCF standards. 

• Secure PINS: Candidates receive a one-time secure PIN to access their assessment. 

• Multi-mode monitoring: Invigilators monitor candidates from multiple angles during their assessment. This includes a mobile device behind them, a webcam on their assessment device and live screen sharing. This approach reduces cheating and ensures candidates follow requirements throughout.  

Assessment Design  

The online framework also contributes to the security and integrity of the assessment process. 

• Randomised question selection: sections of the assessment are randomised to ensure each candidate receives a unique sequence.  

• Question banks: the system selects questions from a varied pool to ensure differences between candidates’ assessments. 

• Answer order randomisation: Shuffling the answer choices encourages critical thinking and reduces cheating.  

• Level-appropriate questions: Including scenario-based questions that reflect real-world well control challenges tests problem-solving and critical thinking rather than memory recall.  

Data monitoring  

A major advantage of online assessments is the wealth of data they generate. Unlike paper-based assessments, digital platforms provide instant insights into pass rates and performance trends across our global network of accredited training centres.  

Detecting integrity issues  

• Unusual patterns such as sudden spikes in pass rates in certain regions could highlight compromised security.  

• Technical issues: We can identify lags or connection problems and provide support.  

• Global comparison: Analysing performance across regions helps us to ensure fairness in assessment conditions. 

Data monitoring also plays a significant role in ensuring the validity and reliability of our assessment questions. We will explore this further in a future post. 

Commitment to integrity  

With the right measures in place, assessment bodies can strengthen the overall assessment process, addressing concerns while providing a secure testing environment. 

IWCF will continue to be vigilant and adaptable as learning and assessment becomes increasingly digital and new challenges present themselves. 

We are grateful to our team, instructors and members who work daily to protect the integrity of IWCF assessments. We share a responsibility to ensure that our programmes remain not only effective, but fair and secure. 

Many of the points mentioned in this post are brief summaries of more in-depth, detailed IWCF procedures. If you have any questions related to our online assessment process, please contact information@iwcf.org. 

Virtual Train the Trainer Blog

COVID-19 changed how we operated, with the delivery of our Train the Trainer and Assessor Training courses impacted. We could no longer hold these courses face to face in global locations due to the global travel restrictions imposed from March 2020.

We had to quickly diversify and adapt our delivery methods to ensure we could continue to effectively deliver these courses and maintain competency standards for new instructor and assessors seeking accreditation.

Tracey Jayne Little who delivers our Train the Trainer course, has written a short blog explaining her experience of adapting the Train the Trainer course for the virtual world and the impact this has had on her training of instructors.

“I have been involved in the development and delivery of e-learning for the past 15 years; the events of the last 12 months have really pushed companies to look at alternatives to classroom-based learning.

Our Train the Trainer programme is very well-suited to virtual classroom (Zoom). Breakout rooms facilitate discussion and activities in small groups as learners are very much in control of their own learning. This is then strengthened by them writing their answers on a virtual whiteboard in breakout rooms and then sharing it in main classroom. This type of hands-on interactive activity embeds the learning. I have actually found it generates more discussion than in the real-life classroom, plus learners’ love being in control of the sharing!

I also have found the ‘chat’ function in Zoom is a huge teaching aid. Not only does it encourage quieter learners to ask questions privately, but I can also use it to privately message learners (when you see eyes down and you know the mobile phone has made an appearance!). Also, for those learners who may experience some difficulty in understanding real-time speed English but can read perfectly, I continually put key points in the ‘chat’ and write key points of every topic on a whiteboard, sharing it on chat-file share. I feel this really adds to the learning. The use of emojis echoes Smartphone technology – learners love them, and it lightens the lesson. I also use a WhatsApp group to support learners – easily accessible technology.

I absolutely love teaching by virtual classroom. I believe it can be a dynamic, inspirational, supportive and inspiring learning solution. The key is using all its functionality so that it truly stays student-centred and keeps the learners engaged. You have to make it mirror the real-life classroom such as possible with pre-class “virtual coffees” and putting learners into breakout rooms in breaks so they can build relationships.

I feel, based on learner feedback and results, that virtual classroom has enhanced the whole Train the Trainer learner experience. In line with good practice, the course was extended from three real-life classroom days to four on guidance from ILM (part of the City & Guilds Group) who also endorse this qualification globally as TJS International is an ILM global Recognised Provider. This creates more course time, shorter days with frequent breaks and increased learner support. This embeds the learning more fully as seen in the results since the use of virtual classroom. Learners do take advantage of the support time and stay behind at the end of class for 1-1 time in breakout rooms whereas in the real-life classroom, everyone gets their coat on!

I feel that Train the Trainer by virtual classroom offers a highly accessible, engaging learner journey with some added benefits for global training.”

Places are available on our Train the Trainer courses in August, September and October 2021. Click here for more information and the upcoming course dates.

The IWCF translation team is responsible for coordinating and overseeing the translation of IWCF exam content into 16 languages. The team are working to embed a new translation process which aims to improve quality. This includes the development of an expanded translation team, the use of new technology to improve workflows and developing linguistic resources to assist translators and reviewers. One of our areas of responsibility is to help build and maintain glossaries for the most commonly used languages in IWCF centres. While not the only factor in maintaining quality, these glossaries are very important for preserving consistency across a complex field, which make them valuable resources for IWCF and our stakeholders.

The field is vast, but the glossaries are a select choice of terms which act as a point of reference, and they are focused on the terms that appear most often in assessments. These help the translators and reviewers make sure that the correct technical terms are used, especially when there is more than one possible option. If translators did not have access to them, it would be like a reconnaissance team trying to complete a mission in thick fog with no GPS or map – some would go north, others would go south, and every so often someone might get stuck in a bog or fall off a cliff – in the worst-case scenario of course!

We don’t expect IWCF translators to over-rely on a glossary in place of their own knowledge and experience. However, the terms have been specifically selected for inclusion to help maintain consistency and quality, therefore our job is to ensure adherence.

To date we have piloted the use of glossaries in 5 languages (French, German, Italian, Norwegian and Russian) and are working with the branches to continuously improve their content. However, there are a few other languages where it has made sense to draft glossaries based on the work taking place. For example, as part of the process to translate the drilling question bank into Azeri we have recorded how technical terms are used and these inform the translators as they work through the project.

The IWCF translation team wants to make sure that the glossary and its equivalent translated versions contain the most up-to-date and accurate terminology for each language. This can be a challenge, but by working with our branches we can harness the expert technical knowledge of our members to aid us in the task. We aim to extend this process to cover all languages we offer.

You are welcome to discuss the importance of glossaries at Branch level and can contact the Translation Team at Translation@iwcf.org for more information.