AI chatbots in unsupervised assessment

TU Delft Assessment Taskforce has created practical guidelines on how lecturers can deal with the influence of AI chatbots on unsupervised assessments and what mitigating measures can be taken. In June 2023, the Special Interest Group AI & Assessment updated the guidelines.

The possibilities of AI chatbots in assessment require further investigation. A group of AI experts carried out a risk analysis of AI chatbots on assessment (Special Interest Group AI & Assessment, SIG AI&A). This page was last updated on 13 June 2023, based on input of the SIG AI&A.

About AI tools

AI tools like the chatbot ‘chatGPT’ can produce convincing answers to certain questions, depending on their nature. However, their output is not always reliable: outputs can contain convincingly presented factual errors (so-called hallucinations). Furthermore, their training data can be outdated. For example, ChatGPT currently uses training data up to September 2021, whereas others (such as Bing) do have access to recent data. It is also important to note that most chatbots do not list their resources (Bing Chat does, though). 

On the positive side, chatbots can help with: 

  1. Checking grammar, spelling, and references in a text
  2. Generating ideas by listing information from different sources in an accessible way
  3. Giving feedback
  4. Summarising or expanding texts and concepts
  5. Coding in a wide variety of computer languages

Use by lecturers for assessments: AI chatbots can help lecturers in creating assessments (including different versions of an assignment), answer models, and rubrics. 

How to assess assignments and projects

Invigilated exams versus assignments and projects

During classical written exams and digital exams in the TUD secure exam environment, students do not have access to the internet, and therefore your students cannot access online AI tools. The same holds for oral exams that are held on campus.

On the other hand, if students work on assignments (exam-like or other) outside an exam hall and without invigilators (Dutch: surveillanten), the use of AI tools cannot be prevented.

Advice for fraud prevention in (non-invigilated) assessment