Palo Alto Networks Appliance Vulnerable To Evasion, Was Tested Thoroughly, Says NSS Labs

A weakness in Palo Alto Networks' Next Generation Firewall, if deployed using the default configuration, could enable attackers to easily bypass the appliance's detection capabilities, according to independent testing firm NSS Labs.

Austin, Texas-based NSS Labs issued a statement on its blog Wednesday calling Palo Alto Networks' criticism about the firm's testing methodology unfounded. Version 6.03 of PAN-OS proved to be susceptible to multiple evasion techniques, said Bob Walder, founder and chief research officer of NSS Labs. The firm said all appliances it tested were placed in a "predefined vendor-recommended setting."

"Palo Alto Networks NGFW misses several critical evasions that leave its customers at risk," Walder wrote. "Palo Alto Networks was treated exactly the same as every other vendor in this test. NSS tests all NGFW products with the predefined vendor-recommended settings."

[Related: Palo Alto Networks Earns Poor Results, Caution Rating In NGFW Test]

Palo Alto Networks has been shipping version 6.03 of PAN-OS since January, according to a Palo Alto Networks spokesperson, who said the company was not commenting further on the NSS Labs test.

The inability to detect common evasion techniques used by attackers caused Palo Alto Networks to be given the only "caution" rating from NSS Labs and a security effectiveness score far below its chief competitors in the next-generation firewall market. Walder's comments were in response to Lee Klarich, Palo Alto Networks' senior vice president of product management, who said the company did not participate in the NSS Labs testing and did not provide guidance on the proper configuration of the appliance for the test.

"No tuning is permitted," Walder said of NSS Labs' testing methodology. "When it comes to NGFW, NSS research shows that most customers deploy these devices with the default/recommended configuration out of the box. This, therefore, is how we deploy NGFWs in our test harness. To reiterate, no tuning is permitted."

The testing firm's study pit Palo Alto against appliances from Barracuda, Check Point Software Technologies, Cisco Systems, Cyberroam, Dell, Fortinet, Intel Security (formerly McAfee) and WatchGuard. NSS Labs gave Palo Alto's PA-3020 appliance a 60.9 percent average security effectiveness score. The test, conducted earlier this year, also found the appliance fell short of the company's claimed 1-Gbps throughput. Appliances from Cisco and Cisco-Sourcefire earned the highest security effectiveness scores followed by Dell SonicWall Supermassive, and WatchGuard's XTM 525 appliance.

Solution providers that sell network security gear from Palo Alto Networks and many of its competitors said tests are used by sales teams and in marketing material, but no customer can ultimately know how well an appliance performs until it is fully deployed and configured.

Some customers have unique rule sets and other requirements that ultimately have an impact on every appliance's performance, said J.D. Butt, vice president of solutions at Chicago-based solution provider Nexum, a Palo Alto Networks partner that also specializes in selling and deploying a variety of networking security vendor appliances. Butt said none of his clients roll out their Palo Alto Networks appliances in default configuration.

"Performance is a paramount thing for us," Butt said. "There's nothing worse than a customer coming back to you three or six months after a purchase saying the product is not meeting their performance requirements."

NEXT: Check Point Could Gain In Customer Evaluations, Say Partners

Some solution providers say the NSS Labs test results may have a significant impact on customer evaluations and point to Palo Alto Networks' closest rival, Check Point Software Technologies, as the company that could benefit most. The Israeli company is continually spending on research and development and building out refined management capabilities, they say. Check Point, which has been called a more expensive option, is increasingly beating competitors on feature availability and on entry price, according to some solution providers.

"They are in every conversation that we have been in," said a Palo Alto Networks partner who requested not to be identified. "Check Point is retooling a lot of its product line to be more standards-based with more integration points from an API perspective."

Palo Alto Networks has been extremely aggressive over the past four or five years capturing market share from Check Point, Juniper and Fortinet, said Stuart Maskell of San Diego-based managed services provider NWTech. Maskell said part of Palo Alto Networks' secret sauce is its throughput. McAfee, which has integrated its acquisition of Stonesoft, also is a strong appliance, especially for McAfee's customer base, which can integrate it into the rest of its product portfolio, Maskell said.

Next-generation firewall appliances continue to experience strong sales, but the market for networking gear is changing rapidly, said Terry Kurzynski, a senior partner at Chicago-based solution provider Halock Security Labs. Palo Alto Networks and others are catching up and adding components similar to FireEye's virtual sandbox, which detonates and examines suspicious files, Kurzynski said.

The time is now for solution providers to examine complementary solutions as the industry shifts from network security appliances to visibility and control over the endpoint, Kurzynski said.

"Clients are experiencing a lot of challenges with systems infected with malware and FireEye and others have made the problem much more visible," Kurzynski said.

A number of vendors have criticized NSS Labs following poor performance ratings in previous studies. FireEye heavily criticized the legitimacy of a test of breach detection vendors in March. In a similar next-generation firewall study conducted by NSS Labs last year, Watchguard performed poorly and criticized the study. NSS Labs executives said its researchers perform tests on behalf of its end clients and buys equipment when vendors choose not to participate in competitive studies.

PUBLISHED OCT. 2, 2014