Interview with Arnas Staude (Part 1): BEAST makes it easy to recognise connections

We use static and dynamic analysis methods as a matter of principle in the fight against malware. Static methods analyse the properties of suspicious files such as the hash, character strings, code fragments, file size and header properties. The good thing about this is that we do not have to execute the malware during the analysis. With dynamic methods, however, we execute the potentially harmful file and observe its behaviour. The difference between the two methods is that, in static analysis, we block the malware before we execute it. With dynamic methods, we detect the malicious behaviour downstream and then stop the malware. With static methods, we have to create a separate detection rule for each malware family. With behaviour-based rules, we can define what is harmful in a more general and long-lasting way.

The behaviour of malware, on the other hand, is much more difficult to change than the program code. The cyber criminals cannot and do not want to do away with the malicious behaviour. If they did, they would no longer be able to use the operating system to read passwords or encrypt important data, for example. In short, they can no longer cause any damage.

Behavioural analysis has been an integral part of G DATA's security solutions for years. Our Behaviour Blocker has done some sound work in this respect. This involves very stable behavioural analysis, which has matured over the years with very good results. Nevertheless, we have noticed that the potential for behaviour-based recognition is far from exhausted. The previous approach did not offer the potential to expand this further. In addition, it was sometimes difficult to understand how individual results came about. This is because the previous approach is score-based and therefore calculates a numerical value between zero and one for each program. If a pre-defined threshold value is exceeded, the program is then considered malicious. This value is calculated using a lengthy mathematical formula. However, this also means that the outcome is not always immediately comprehensible to us, and this aspect led to a file being classified as malicious. This makes case-by-case consideration complex. Dealing with false positive detections is therefore particularly difficult and requires a different approach. But it is difficult to adapt the formula. Unfortunately, we cannot always foresee how a small change in the calculation may have a knock-on effect on other programs later on.

Therefore, we have decided to develop a new approach that is more future-proof. The name BEAST stands for BEhAviour STorage. One aim was to create a comprehensible decision-making process.

We wanted to know and understand what happens in detail. At the same time, we wanted to be able to combine existing technologies such as Bank Guard or Exploit Protection with the new behaviour analysis. These detect anomalies, but do not collect information on how these anomalies occur.

The great advantage of BEAST is its traceability, which puts us in a position to develop a very precise understanding of how  a detection is generated. This is because the basis of BEAST is a powerful graph database which we can use to visualise all the processes. This enables us to detect harmful behaviour in a targeted manner so that no false positives occur. That in turn means that we can now modify the rules much more easily. Basically, we had a lengthy mathematical rule with Behaviour Blocker, so it was unclear what effect changes would have on the overall system. So before we essentially had one big and rather unwieldy rule, now we have numerous small rules that are much easier to modify. In addition, with BEAST, we can carry out many more tests.

For instance, we can also scan offline with the graph database. For the tests, we have built up a large set of known harmful behaviour patterns and can test our rules on it to optimise our detection. At the same time, we also have benign graphs which we use to ensure that no detection takes place there. Maintaining this database ensures good detection and prevents false positives. Once we have classified a behaviour as a false positive, we add it to our database, so BEAST never makes the same mistake twice.

Retrospective here refers to the time component. We look back at what has happened in the past. We can now take into consideration processes that have not existed for a long time as well. Indicators of compromise always exist in the course of infections, be it a server that’s been visited or a file that’s been opened. However, until these indicators are identified as malicious, they often no longer exist on the affected system because, for example, the malware has deleted the files associated with the beginning of the infection chain.

With BEAST, these are still present in the graph, so that the entire infection chain is available and can be reversed.

This information is retained in the graph and any remaining artefacts in the system can still be tracked down. Retrospective Removal enables the original parent process to be identified, starting from a node that has already been recognised as being harmful. It can then be examined more closely. Has it downloaded suspicious files and launched other programs? This then gives you enough information to completely clean up the infection. Such complex interrelationships remain hidden from ordinary behaviour-based technologies. This is particularly useful when malware is “sleeping” and inactive at the start of the infection, or waiting for a command from the botnet operator, but has not yet performed any harmful activities. We would then detect the harmful activities with BEAST.

BEAST is quite unobtrusive. We receive very few support queries, which is a pretty good sign. This means that we are hardly ever interrupting our customers with false positives. We say the more inconspicuous, the better. In addition, we track how BEAST is working very closely via our detection telemetry. This enables us to see that BEAST is successfully protecting a large number of customers every day. External tests have already confirmed that BEAST does its job reliably - detecting malware.

Both technologies are very different, making a direct comparison difficult. After all, we replaced our former Behaviour Blocker with BEAST. What we are already seeing is good interaction with other components such as DeepRay. This means that we have fulfilled one of the key requirements for BEAST: BEAST supports the existing technologies in their work and improves them. In addition, we also use BEAST in the back end and generate additional data there by analysing malware. In this way, we optimise the backend processes and ensure even faster detection and better blocklisting. This enables us to transfer the results of dynamic detection directly to static detection. So we prevent already known malware from being executed again.

At the same time we can adapt our protection rules much better. With Behaviour Blocker, this was hard to do due to the complexity of the algorithm, and this is reflected in much higher responsiveness in our daily work.

The biggest hurdle was definitely the performance. At the beginning, we hadn't set any hard criteria. Over time, however, the issue of performance has become more important, because BEAST was expected to store and process a lot of data to make the information available. Especially when compared to the old approach, it quickly became clear that the new technology was more computationally expensive (meaning it requires more computing power) and memory intensive. BEAST continuously checks current application behaviour for known patterns and therefore needs to constantly process a wide variety of data. The old approach doesn’t have to remember anything. It aggregates everything into a numerical score instead, so that not as much CPU load is generated.

So we had to invest a lot of work in high-performance processes. We estimate that we used half the development time for this, because we tested and compared whether or not there was any improvement after each change. In the end, only the test shows whether our assumptions were correct. Sometimes, promising changes had little impact on performance. On the other hand, smaller and, at first glance, insignificant corrections had a much greater impact on performance than expected. It definitely took a long time until we finally achieved the result we were after.

However you look at it, it was a hell of a lot of work. After all, it took five years to develop BEAST. It was particularly tough during the initial phase, because I was working on my own at first and sometimes I couldn't see the wood for the trees because of the lack of exchange between different development  teams. Things have improved immensely over time, mainly because I was able to get some additional staff on board as well. There were numerous teams involved in the project, not just the core development, but right across the company. This cooperation worked very well. Looking back, everything went very smoothly. The effort was worth it and we can be proud of the result.

In the second part of the interview, Arnas gives more fascinating insights into the development of our new BEAST behaviour monitoring system.