Out-of-bounds: an outside view of computer memory vulnerabilities
Out-of-bounds writes, you may not have heard of them, but they play an important role in the hacking of programs. From changing account balances in banking software to breaking into critical systems at hospitals, there are many malicious applications. Linus Hafkemeyer developed a tool making it much easier to assess the severity of such vulnerabilities, speeding up the process of fixing them and, ultimately, leading to a more secure digital society.
Linus Hafkemeyer certainly isn’t the first who, as a teenager, tried to circumvent the restrictions on his internet use set by his parents and their router. But where others may grow bored with such tinkering, he became ever more fascinated by cybersecurity, ultimately leading up to a master’s thesis on one of the most hazardous types of memory corruption vulnerabilities.
‘They are called out-of-bounds write vulnerabilities, and they can be used to trick a program into accessing memory at certain locations where that should not be possible,’ Hafkemeyer says. ‘If such a write ends up in unused memory, which will never be accessed again, it will not have any consequences. In many other cases, it will just make the program crash. But in certain cases, it could allow a hacker to, for example, modify critical data or even hijack an entire program.’
Tricking a program into accessing memory at an unintended location is one of the most dangerous types of software security vulnerabilities
Abundant and reliable
What makes out-of-bounds writes so very dangerous is that they are both abundant and reliable. Once someone with bad intentions has developed a method to exploit such a vulnerability, it is very likely to succeed any time the program is being executed. Hafkemeyer: ‘I really got to know these vulnerabilities early on during my master’s, when participating in so-called Capture the Flag challenges – an information security competition in which you, for example, must exploit a vulnerability on a server to obtain a secret and score points.’
Dead set on his research topic, and with it hardly being taught and researched at TU Delft, he organised external supervision by Professor Andrea Continella from the University of Twente. Hafkemeyer: ‘He suggested some research directions, which I then narrowed down until I found a sub-problem that was not yet solved while still being feasible as a master’s thesis project.’
Fuzzing
The current state of the art in discovering out-of-bounds write vulnerabilities is a process called fuzzing, followed by human intuition to find the root cause. ‘In fuzzing, one tries to reveal weird behaviour of a program by throwing a lot of random input at it really fast,’ Hafkemeyer explains. ‘Potential vulnerabilities are exposed by observing certain indicators, such as memory access to unexpected locations.’ The next step, still prior to root cause analysis, is to estimate the severity of each of these vulnerabilities. That is called triaging – a process much like in an emergency room where somebody determines which patients need immediate medical attention, and which ones can wait. ‘I wanted to develop a tool that helps expedite this triaging process.’
From the outside looking in
‘The problem with existing out-of-bounds write detection solutions is that they are invasive; they need to add some code to a program to facilitate detection,’ Hafkemeyer says. ‘But then, when observing program behaviour with the added code, it will differ from how the original, unaltered program would perform, making any output worthless for triaging.’ In an emergency room setting, it would be like triaging patients by having them randomly point at a list of physical complaints.
‘I designed an approach in which we run the unaltered program in a virtual machine, feeding it the input previously found through fuzzing that we suspect of causing an out-of-bound write,’ Hafkemeyer says. ‘This way, we can observe its behaviour from an outsider perspective and see what is going on without having to modify the program.’ His tool returns all the source code-level objects affected by such unintended writes, providing human experts precious information for triaging and even root-cause analysis.
De crux is het monitoren van een computerprogramma zonder die eerst aan te hoeven aanpassen
Insanely complex
‘Looking back, I am amazed by the twists and turns this project took from the original idea to the finished thesis,’ Hafkemeyer says. ‘The more I got into the topic, the more I realized how insanely complex it is.’ Central to his difficulties was compilation – the process of translating the legible source code into an executable program. ‘This process is generally accompanied by a loss of most high-level semantic program information that is necessary for detecting out-of-bounds writes. I had to delve deeply into the inner workings of the compiler and develop ways to pull out the necessary information before it will be lost.’
Computer programmers often talk about “being in the zone” – a hyper-focused and very productive state-of-mind. Next to this zone, Hafkemeyer at times also felt like being in his own tunnel, unsure if what he did made sense. ‘What helped me at those moments was to have a sanity check, presenting what I did to my friends and my supervisor,’ he says. ‘During my thesis work, I also got into historical audiobooks. They allowed me to dive into another reality, even if just for half an hour. And then return to the task at hand with renewed energy and insights.’
Cybersecurity companies
His supervisor, professor Continella, is very impressed by his thesis work, calling it a novel and important approach that has very practical implications in the real world. ‘It will speed up the process of fixing these out-of-bound writes security vulnerabilities, leading to more secure programs,’ he says. ‘It is also very expensive to have highly skilled people triage them, so any timesaving is much appreciated. We are now preparing to submit the approach for publication at a well-known cybersecurity conference, expecting quite some interest from security companies and software developers.’ While writing that paper, Hafkemeyer is enjoying a long vacation, backpacking in Europe, writing some new memories himself.
My tool will speed up the process of fixing security vulnerabilities, leading to more secure programs.
