Security Experts:

Web Application Scanners Challenged By Modern Web Technologies

Researchers Find Nine Application Technologies Overlooked By Most Web Application Scanners

Security teams are having trouble with the fact that Web application scanners generally cannot detect vulnerabilities in applications built on top of dynamic technologies such as HTML5 and AJAX, according to a recent study.

XSS Vulnerabilities in Hotmail

Web scanners can generally scan classic HTML and JavaScript sites, but are unable to translate pages built using modern Web technologies, NT Objectives said on Thursday. A new generation of mobile and Web applications are being built using JSON, REST, HTML5, and AJAX deliver Rich Internet Applications, mobile apps, and Web services. The "scanner coverage gap" means security teams have to manually test these types of applications to discover vulnerabilities, NT Objectives said.

Along with the report, NT Objectives announced a beta release of its NTOSpider 6, a dynamic application security testing platform which uses a proprietary Universal Translator technology to automatically crawl, detect, and attack vulnerabilities in highly complex and dynamic applications. NTOSpider 6 is designed to find vulnerabilities that were previously only discoverable manually, NT Objectives said.

"The spread of mobile applications, web services and complex Rich Internet Applications (RIA) has made a bad situation worse for security professionals, who are constantly playing catch up to stay ahead of vulnerabilities and frantically defending against persistent hackers," said Dan Kuykendall, co-CEO and CTO of NT Objectives.

NTO's Universal Translator has a broad coverage of the technologies used to build complex, modern applications. Its capabilities include the ability to simulate attacks against Web and mobile backend services by detecting rich client traffic, and to decode and attack popular formats such as JSON, REST, Flash Remoting (AMF), SOAP, and XML. It can crawl and attack rich client traffic including AJAX, JQuery, and GWT. It can also test features such as shopping card and application workflows. NTOSpider performs XSRF token detection to collect and use valid tokens during an attack.

NTOSpider offers repeatable, rapid, and comprehensive automated application security testing. The automated process results in lower risk for the organization and frees up penetration testers to look at aspects of the application that have to be tested manually, such as business logic, NT Objectives said.

In "The Widening Web Application Security Scanner Coverage Gap in RIA, Mobile and Web Services: Is Your Scanner like the Emperor's New Clothes," NT Objectives identified several common underlying Web application technologies that are commonly overlooked by Web scanners when examining RIA, Mobile applications, Web services, and other application workflows.

The complete list includes JSON (such as JQuery), REST, and Google WebTookit in AJAX applications, Flash Remoting (AMF) and HTML5, as well as mobile apps and Web Services using JSON and REST. XML-RPC and SOAP technologies used in Web services, and complex workflows such as shopping cart, and XSRF/CSRF tokens were also listed.

For example, in AJAX applications, deep links, JSON, and the document object model make it difficult for Web scanners, the report said. Many web scanners can handle the first instance of an AJAX page, such as the Inbox view in Google's Gmail, and the second instance, the page when the user opens an email address. But scanners have a progressively harder time going deeper in the application, according to NT Objectives.

Also, when a vulnerability is discovered in a classic web application, scanners would reference the page and parameter where the issue was found. That really isn't possible in an AJAX application, as it is everything is often presented as a single page with many possible user events. The vulnerability may rely on a certain combination of steps to occur before it exists, which makes automated scanning a challenge.

While scanners have never and will never cover an entire web application, they should cover as much as possible, NT Objectives said in its report. "Unfortunately, the coverage gap has widened in recent years placing even more responsibility on manual testing," according to the report.

Related Reading: Top 10 Security Threats for HTML5

view counter
Fahmida Y. Rashid is a Senior Contributing Writer for SecurityWeek. She has experience writing and reviewing security, core Internet infrastructure, open source, networking, and storage. Before setting out her journalism shingle, she spent nine years as a help-desk technician, software and Web application developer, network administrator, and technology consultant.