January 7, 2019

Elementary: The Future of the AI Bug-Detectives

Can AI detect bugs and errors, and prevent security breaches?

Shields down

The Internet of Things is rapidly creating a global ecosystem of computer-human interdependence. While this synergy is driving innovation and advancement in almost every facet of our lives, it exposes us to new and challenging vulnerabilities that must be met by new and fulsome countermeasures.


Security vulnerabilities are growing in lockstep with accelerated software development and application complexity. There is an ever-increasing onus on developers to ensure that they create robust programs that can withstand the advancing threat. With billions of lines of code written every year it is currently impossible to ensure complete infallibility of code — while developers are, by nature a highly focused and meticulous bunch, to err is human and they are still just humans!

Program manager at US military Defence Advanced Research Projects Agency (DARPA), Sandeep Neema, says “What’s concerning and challenging is that the bugs in software are not decreasing,” which is why DARPA spends millions of dollars funding the development of Artificial Intelligence (AI) systems that can detect software flaws. In an era where even the simplest of programming errors can throw open the doors to malicious intrusion, it is not surprising that many businesses are looking closely at the role artificial intelligence (AI) and machine learning (ML) could play in reinforcing cyber defenses.  

Beyond the old reliables

The number of cyber attacks is rising. Security researchers regularly discover new malware as well as advanced malware variants, such as Mylobot. Traditional and legacy antivirus solutions simply cannot compete with advanced threats, such as the recent WannaCry ransomware virus. It is estimated that 50–75% of development time is spent testing, with many errors detected due to firewalls, assertions, code reviews, IDE warnings, varying compilers for different OSes, working on different hardware, and so on. It’s still common for developers to review each other’s code and run tests before launching new programs. Despite the enormous commitment to preventing, detecting, and triaging faulty code, errors still account for 9 out of every 10 instances of cybercrime. With a 2017 Enterprise Risk Index report claiming that only 50% of file-based attacks were submitted to malware repositories, it is clear that the hackers have the upperhand. Using polymorphism and obfuscation, targeted attacks evading overloaded security teams and automation-to-scale, attackers are making it nigh on impossible for traditional solutions to keep pace. It is clearly time to up the ante.


In 2017 Microsoft announced the roll out of a new error and virus detection tool designed to meet the requirements of the current threat landscape. The Microsoft Security Risk Detection tool, formerly Project Springfield, is an advanced, cloud-based fuzzing program that uses AI to root out risks before a program is generally available. John Heasman, senior director of software security at DocuSign where the tool was trialed, lauded its effectiveness saying, “It’s rare that these solutions have such a low rate of false positives,” which traditionally pose a huge problem, taking so long to investigate that security experts risk missing the real bugs as they sort through false ones.

Microsoft’s lead researcher for the project,  David Molnar, says:

We use AI to automate the same reasoning process that you or I would use to find a bug, and we scale it out with the power of the cloud

The Microsoft Security Risk Detection tool is an additional layer of security that supports the work of developers; however, it is not yet a replacement for all other systems of threat management.

Social Media giant Facebook is getting in on the act. Facebook’s Artificial Intelligence Research (FAIR) team have rolled out the automated software testing tool, Sapienz. Sapienz, in conjunction with their Infer static analysis program, uses AI to pinpoint the point of weakness in code before passing the information to SapFix, their new AI-hybrid automatic fix generator.


SapFix can run independently of Sapienz. SapFix reduces debugging hours, speeding up development and roll-out time. So committed is Facebook to advancing their research into AI and ML solutions, they are opening a new AI lab in AI lab in Paris to supplement similar facilities in New York and Silicon Valley.

AI bug-detection continues its march into gaming territory

In March 2018, Ubisoft announced their new Commit Assistant tool, which uses AI to flag potentially faulty code before it’s implemented (maybe even before it is written). The developers of Commit Assistant trained their model on almost 10 years of code from their own software libraries, learning from past errors so as to flag them should they reappear. They claim that almost 70% of their annual budget is consumed attending to threats to their programs, therefore this new investment in AI bug-detection could have huge consequences for their bottom line.


A Chinese-American research group based at the University of Texas has developed an AI bug-detection system trained to prevent Zero-Day attacks. This tool was tested on four widely used commercial software programs and uncovered 10 previously undetected flaws.

As previously mentioned, DARPA is already heavily invested in AI bug-detection. Suresh Jagannathan, DARPA program manager, says their AI Mining and Understanding Software Enclaves (MUSE) program is:

...aiming to treat programs—more precisely, facts about programs—as data, discovering new relationships (enclaves) among this 'big code' to build better, more robust software.

Central to the MUSE project is the creation of a continuously operational specification-mining engine, which leverages deep program analysis and big data analytics to develop a database of inferences about properties, behaviors, and vulnerabilities within programs. The desired outcome of MUSE is to flip the development process on its head, eliminating or at least vastly reducing the possibility of error in the first instance.

Shields up?

Big tech companies are gambling big on AI, but traditional companies like healthcare, retail, and telecoms remain hesitant, with very few incorporating AI or ML into their value chains at scale. It seems that despite all the recent investment, the scope of AI deployment is still relatively limited. A recent study of more than 3,000 businesses around the world found that many business leaders are uncertain about return-on-investment from AI expenditure.


For all the advantages of AI-driven virus detection solutions, they do not come without risk. AI and ML models require large quantities of data to learn from. This is expensive, and with so few experts in this burgeoning field, could make adoption slow. There is also the worry that advanced AI and ML models could fall into the wrong hands and be used to attack the defenses they were designed to defend. Worst-case scenario stuff, yes, but something to consider.


Current security systems cannot keep pace with intense and frequently automated attacks, like the WannaCry virus, which affected more than 200,000 machines in a matter of hours. Hackers have the advantage of knowing that many of the most widely used security tools, such as AV and Intrusion Detection Systems, are flawed and they know just how to evade them.


Ultimately AI has the potential to make code vetting less labor-intensive and more accurate. ML and AI are making larger inroads in cybersecurity defense systems, but their current prominence is more of a buzzword than a blueprint for effective bug defenses. Widespread adoption of AI-driven virus detection doesn't look likely anytime soon, but it doesn't hurt to dream of a virus-free future!