Transform your education with Immersive technologies: here are some examples of the NewMedia Centre

Immersive technology allows the visualising of any piece of information, providing the opportunity to transform the most boring data into an exciting practical learning experience. We have explained the benefits and best applications of Educational Virtual Reality, Augmented Reality and Holograms in the previous article, and now it’s time to share what TU Delft has been up to in this area. We’ll be covering 4 different immersive projects of the NewMedia Centre of TU Delft, sharing a glimpse of how they work and touching upon their advantages over traditional education. Whether VR provides a better perspective of unreachable spaces, or an ability to interact through a hologram, or practice risky assignments in a virtual and safe environment: all of these benefits contribute to shaping the next generation University.  

Footage from Mars rovers and images of vehicles have been out there for years, but getting the feel of the scale of these objects in comparison to a human has always been difficult. Sebastiaan de Vet, lecturer of Astrodynamics & Space Missions, TU Delft Aerospace Engineering, wanted to show the scale differences between Mars vehicles created over time by NASA. Many people have seen footage from Mars rovers but they usually lack the feeling of scale as there are usually no people in the scene for reference. 

To solve this issue the NewMedia Centre of TU Delft has created a virtual production studio that enables teachers and researchers to blend into their 3D virtual content thus creating a realtime augmented video. 

Sebastiaan had the idea to walk past a chronological lineup of all the vehicles, where he would stop to talk about each vehicle and then move on to the next one with a final shot showing all vehicles in one scene. 

Thanks to this studio, the dynamic content can be made on an affordable scale for education. It will enable teachers and researchers to step into their 3D virtual content, thus creating a realtime augmented video. 

How does it work? 

The students are able to see the professor on the video as if he is a part of a preset environment: in this case he is walking on Mars' surface. This project requires preparation, however. The virtual studio features free-moving cameras where the foreground and background will move accordingly in scale, position, rotation and focus with the moving cameras. A powerful computer renders the dynamic foreground and background in real time.  

The Mars landscape was created from scratch using Pixel Mega scans. All was put together and after a bit of tweaking a realistic Mars landscape with rover vehicles was created: The 3D Models from the rovers are provided by NASA but  needed to be optimized by the NMC before they could be used in VR. 

“We used three types of cameras: a static camera, a camera on a slider and a free moving cameraman. We also wanted to shoot close-up as well as a wide scenery. The virtual studio allows rebuilding shoots as you like and it’s relatively easy to sync the fore and background on the fly,” says Roland van Roijen, Coördinator Media Lab | Media / XR / 3D Designer. Video

Learning some procedures required for working on a ship wharf is usually a difficult and costly process due to limited access to an actual location and lots of risk involved. Still, students of the faculty of 3mE (Mechanical, Maritime and Materials Engineering) have to practice some assembly and logistics ship operations.  

In order to help students learn easily and safely, the NewMedia Centre created a multiplayer VR application where they can learn multiple disciplines on a ship wharf in a virtual environment. Once in VR, the students perform different tasks from identifying and locating the required parts of the ship to transporting them and assembling the hull of the ship with a crane. During the whole experience they work in a team and perform these practical tasks while learning to navigate through the ship together. All the team members communicate through virtual walkie talkies, created specifically to increase the realism of their communication in VR.

How does it work?

Students use VR headsets which provide full immersion into the environment. For this project a ship and a ship wharf have been created in 3D, using Unreal Engine, and optimised for VR. The application features a multiplayer environment. 

At some point the students have to carry a pipe through the ship. As VR does not allow us to replicate the weight of the object, we have added a real physical pipe with a length of about 150cm with weights attached to it. The object is fitted with a VR puck that translates the object to VR so students can see and feel the object in VR and learn to communicate and navigate in safety while physically carrying a weight. This enables a more realistic perception of the virtual experience and results in more effective learning.  

The maritime project has been run successfully with students from the faculty of 3mE and is scheduled to be expanded over time by adding more functionality and tasks. 

Along with many advantages, the online format has brought a lot of challenges to modern education. The limited access to social interactions, face-to-face support and real-life learning leads to a lack of energy and motivation in students, which, correspondingly also affects educators. Videoconferencing formats still do not provide enough realism and immersiveness.  

To improve and stimulate online education, the NewMedia Center together with the TU Delft student council and LDE Centre for Education and Learning have initiated Project HoloLearn. This initiative, spearheaded by Postdoctoral Researcher at TU Delft Bibeg Limbu and LDE CEL, aims to use holograms to stimulate richer social interactions in online/distance education between students and teachers at TU Delft. 

Educational holograms have a lot of potential, but there is still a lot to explore in this area. One of the goals of TU Delft is to evaluate the possibilities of seamless hologram integration in distance teaching practices and explore its impact on social interactions and the outcome of education compared to traditional video formats. 

How does it work? 

The project Hololearn is exploring two different ways to apply this technology at TU Delft. The first project concerns the creation of a virtual classroom, which may be joined by students and professors at the same time, even if they are at different locations. The students can see the virtual classroom and a “hologram” of the professor via their AR device such as HoloLens glasses. The students get a really immersive class experience from their homes: a teacher appears in 3D, captured by a special camera on his computer. At the same time a teacher, whether he is based at home or in the classroom, can also see the students in a 3D environment, seeing one student at a time or switching between them when needed.  

The second type of project is inspired by Monterrey Institute of technology. The Monterrey project requires students to be in the classroom, while the professor may be presenting from anywhere in the world as a hologram. This project involves a Holo Screen (a special screen that transmits an image of the professor) and a transmission room in which a teacher presents his material). The NewMedia Centre aims to add more capabilities to this application.  

“The project aims to establish a web application that streams immersive classroom interactions over the web. This involves the volumetric capture of the lecturer while also returning the students response and interaction between students amongst each other”, - states Roland van Roijen, 

 Coördinator Media Lab | Media / XR / 3D Designer 

The availability of the physical University Labs may be limited due to its capacity or other factors, like the mandatory requirement to work from home during the pandemic peaks. TU Delft had a goal to increase the capacity and extend the availability of the physical Photovoltaic Lab.  

To still provide sustainable and uninterrupted access to learning, the NewMedia Centre together with a research group Photovoltaic Materials and Devices (PVMD) have created the Virtual PVLAB.  

The Virtual PVLAB is the digital twin of the on-campus PV Laboratory. Each task and each piece of equipment is simulated in a 3D environment that resembles the actual PV Laboratory. The didactical approach pursued in Virtual PVLAB is the same as in its on-campus version: students access the laboratory, preemptively study from the guide and execute a certain task according to a schedule. In the virtual lab students gain practical experience, conducting experiments with light, solar cells, modules, batteries, power electronic components, and system design. They also test the impact of various realistic situations and configurations on the performance of PV systems and all their components.  

Instead of analysing the aforementioned experiments through a mathematical representation in theory, the students can actively manipulate objects in a virtual, live setting and challenge themselves by arranging the measurement setup in a virtual lab. Through practical work the students get a "hands on" experience with solar equipment, thus gaining a more pragmatic understanding of all the processes. 

The opportunity to follow this course in a virtual format also enables students to do it at any time, facilitating their study progress through the MSc programme. 

How does it work?

Students can access the virtual lab through their laptops. They do not need any VR headsets, as the Web Graphics Library provides access to a 3D environment through a web browser. In order to create the web application, the NewMedia Centre has used Unreal Engine to create the 3D environment and equipment, simulate the physics behind the interaction of all objects; and used WebGL to deploy the tasks on all platforms on all major browsers. 

Besides virtual reality, 3D visualisation and holograms, there are lots of new media tools that can bring us to the frontier of science. You can learn more about these tools and other University projects by joining the Virtual Playground, the XR Community of TU Delft. By organising regular meet-ups with experts from academia and business, the Community facilitates knowledge exchange and encourages both students and professors to explore new ways of learning.  

Do you want to work with immersive technologies in your education?  

Join the Virtual Playground Community by the NewMedia Centre.