Implementation of the assessment tool Ans
Calvin Rans, associate professor at the Faculty of Aerospace Engineering teaches a number of large courses in the bachelor and smaller courses in the master. He has been using the assessment tool Ans almost exclusively for the past six years. He utilises Ans and it’s feedback mechanisms as an instrumental tool in helping students learn from their mistakes by providing efficient feedback. In this interview you read his experiences.
Which testing systems do you have experience with?
I have used Möbius, Maple TA and a little bit of Grasple in the past. But I have been using Ans almost exclusively for the past six years.
What are the advantages of Ans?
The biggest advantages are the level of feedback that students can see and the logistics surrounding the grading process. After I’ve checked them with the rubrics and have them released, students can immediately exactly see online what they have and have not earned marks for. Once the exams are scanned I do not have to worry about and validate it, or losing them. I can have teaching assistants (TA) mark them and that helps a lot. I also like the anonymity of it. You can set up the grading in a way that you can grade a question without seeing the results that the student got on the rest of the exam. That helps you remain unbiased in your grading.
What can you say about Ans compared to the other systems?
Ans was designed to scan in paper based exams, grade them online in an anonymous way and share your feedback more efficiently. It does that so much better than any other tool I have seen. Although the implementation is often times quite different from purpose built digital examination tools, it has good digital exam and assignment functionalities as well. For example, making a drawing and uploading is much easier to do in Ans than it is in, let’s say, Möbius. From a completely automated digital assignment only to a final answer, it is less good.
What is a special or new functionality of Ans that you would like to share with your fellow teachers?
Very useful for students is the built-in tool for in an open response question. Students can take a quick snapshot with their webcam of something they wrote on paper. The image will be pulled and stuck automatically from their webcam in the solution.
What do you expect Ans to bring you?
It brings me insight into the grading and assessment process. Ans has very useful dashboards for seeing the statistics on how well students did on questions and gives me learning analytics about it. It provides me information in terms of, do I have TA’s that are being more tough on grading than others, to help identify the issues and the grading process. So for me, especially for very large classes where I cannot grade all of the assignments myself and I have to rely on a team of graders, it provides an excellent platform for managing that team, grading effort and ensuring the most fairness and transparency in the grading process.
Do you have any tips for fellow teachers on how Ans could help them in making better tests than before?
Using Ans forces you to really critically think about your assessment criteria and assessment as feedback, because it's all open for the student to see. You cannot reuse summative tests because too much is communicated about it. So you have to get the right balance of mixing formative and summative. The good thing I have experienced is that students appreciate the formative feedback from Ans assignments because they are getting so much feedback in the assessment. Even though they don't get a grade for it, there's a high participation rate which is more significant than other formative assignments I have done in the past.
What are useful life hacks that save time?
Ans allows you to make assignment templates with the written instructions and in certain settings already made. There are also question libraries with a basic set of assessment criteria which helps me build variety into assessments while maintaining the goals of the assessment.
What are the soon to be called quality characteristics of tests, also about inclusive test?
It has a decent implementation for dealing with things like extra time. And I know they have been adding some accessibility things in terms of fonts. On digital tests, it has ability for students to change the font size to make things readable.
What would you recommend teachers when making assessments for the first time in Ans?
Take a past assignment, try and build it within Ans and test it with a TA just to see the difference in implementation. This way you can see if you encounter any of those limitations that you previously had to work around, or identify if Ans provides opportunities that did not exist before.
What would you recommend when switching to Ans from another tool like Möbius?
I would recommend hiring a TA to switch your exam into Ans and have them teach you how to maneuver the differences. You can also think about attending the trainings and workshops to learn more about Ans. They will be organised in the beginning of next year.
How long does it take to master Ans?
I would say relative to Möbius, Ans is much more intuitive. Most people that I have talked to that moved from Möbius to Ans appreciate the level of organisation that intuitive nature of it. I guess it depends also a little bit on what you want to do. If you are going to do a written exam that is just graded digitally, it is extremely intuitive. If you are going to start doing full digital tests with programming and variation, it is relatively straight forward and easy if you know Python. The scripting for having variability within a test is all Python based.
More information
The project team is working hard on getting all preconditions fixed so Ans will be used as standard assessment tool at TU Delft. On this intranet page you will find more information about the project, the approach and planning and answers to frequently asked questions and ways to get in touch.