Zafran Researchers Uncover Widespread WAF Bypass Technique Impacting JPMorganChase, Visa, Intel and Nearly 40% of Fortune 100 companies
The misconfiguration exposes web applications to direct attacks over the Internet which can lead to full compromise, ransomware attacks, or trivial denial-of-service attacks.
What Was Discovered
Zafran’s Research Team discovered a pervasive misconfiguration bug in popular web application firewall (WAF) services by Akamai, Cloudflare, Fastly, Imperva and others. The impacted WAF vendors are responsible for 90% of protected web applications worldwide.
Zafran confirmed the misconfiguration impacts web applications owned by more than 120 companies, including giants such as JPMorganChase, Visa, Intel, Berkshire Hathaway, and UnitedHealth. In fact, the primary website of JPMorganChase (chase.com) was found to be impacted, and JPMC & Zafran worked quickly to remediate the exposure.
The discovered misconfiguration can allow threat actors to bypass WAF protections and directly target web applications and load balancers. In this way, attackers may perform distributed denial-of-service (DDoS) attacks on these exposed web applications, or exploit vulnerabilities in the apps themselves that would have otherwise been identified or blocked by the WAF.
Through a combination of Internet-wide scans and novel fingerprinting techniques, our research team has successfully identified more than 140K domains owned by Fortune 1000 companies which were designed to be protected by CDN/WAF providers. Among them, the research mapped 8K domains to 36K backend servers which are directly accessible over the Internet and consequently exposed to a wide range of attacks.
For a deeper dive on the technical challenges of uncovering the above findings, please refer to our technical blog.
Root Cause Analysis
The misconfiguration stems from the fact that modern WAF providers are also acting as CDN (content delivery network) providers, designed to provide network reliability and caching for web applications. This dual functionality is at the heart of this widespread architectural blindspot of CDN/WAF providers.
Unlike traditional WAFs that are deployed as physical or virtual applications on the customer’s premises, CDNs are designed as a large network of Internet servers that handle web traffic close to the edge (to the end user), and ultimately route traffic to the backend web application over the Internet. This means that when a CDN service is used as a WAF, the web application it protects is open to Internet traffic, and is expected to validate that it responds only to web traffic that originates from and by the CDN service.
When this validation is lacking, backend applications can easily be directly accessed over the Internet. An attacker looking to exploit this, would need to map the external domains to the backend IP addresses (also known as ‘origin servers’). This mapping is thus considered a secret in the design principles of CDN/WAF services, but as our research reveals - it is a secret that can be relatively easily obtained by attackers.
Zafran’s research team was able to demonstrate the impact of this attack through a 20 second DoS of a domain owned by BHHC (a subsidiary of Warren Buffet’s Berkshire Hathaway with over $10B in assets). In the video below we show the DoS in action (the WAF bypass was reported to BHHC as part of the security disclosure).
Why Is This Important
The prevalence of the discovered WAF bypass indicates a systemic architectural blindspot of CDN/WAF providers, and the companies that implement their solutions. The impact of a denial-of-service attack on a business-critical web application can be financially devastating, or worse yet, the WAF bypass can lead to a compromise of sensitive data through this initial attack vector. The CapitalOne breach that led to one of the largest data breaches in history, actually began by leveraging a WAF bypass. More recently, APT41 and its affiliates were observed compromising publicly-facing web applications which lack protections, and exfiltrating sensitive data. Just last month, attackers were also observed launching Cloud ransomware attacks against exposed web applications that would have been blocked by a properly configured WAF.
In general, when it comes to public-facing web applications (that are by design much more vulnerable and prone to attack), the protective layer of a web application firewall is by far the most important (and in many cases - the only) defense mechanism they have.
This research unveils that in this current day and age, where the majority of business is operated over the world-wide-web, the largest companies in the world (20% of the Fortune 1000 companies), are extremely exposed to potentially crippling attacks over their business-critical web applications. While these companies are purchasing high-end web-application protection solutions, many of them are ‘walking naked and exposed’, due to poor implementation of the most important protective fabric of modern web applications.
To illustrate the point above:
One of the most serious CDN/WAF bypasses we identified was in the primary web application of JP Morgan & Chase (chase.com). Luckily, JPMC have quickly responded to our vulnerability disclosure report and have fixed the issue. The potential impact to the largest bank in the world of an attacker taking down its primary website could have been catastrophic, let alone the potential of a full compromise.
Recent estimates show that an average DDoS attack lasts 68 minutes and costs the victim around $6K per minute, or a total of $408K. This figure grows to $1.8 million when the victim is a financial organization.
Substantial losses due to DDoS attacks can also impact other verticals. Take for example a major pizza chain that sells an average of 35 pizzas every second, and was found vulnerable by our research. With each pizza costing $15, just one hour of downtime would result in losses of around $1.9 million.
Mitigation Measures
To prevent attackers from directly reaching out to origin servers over the Internet and bypassing the protections of the CDN/WAF, origin servers are instructed to limit access to their web applications via one or more methods:
- IP whitelisting (Origin IP ACL) – limit access to the origin server for incoming connections from the IP addresses of your CDN provider. This method is the simplest one that exists, but it is not a bulletproof solution. Prior research has found ways to circumvent this protection.
- Pre-shared secret in a custom HTTP header - limiting access to the origin server, by having it validate a pre-shared secret that is set in a custom HTTP header by the CDN. This method is a great short-term mitigation to the bypass, but it requires manually rotating the shared secret from time to time.
- Client certificate (a.k.a mTLS) - The most secure method, is to use mTLS - where both the client (the CDN) validates a publicly signed certificate of the origin server, and the server validates a client-certificate that is used by the CDN. While this method is the most secure, most popular load-balancer solutions don’t support mTLS, and this method may require custom tooling.
Each of the popular CDN providers offer guides for properly protecting origin servers:
See more details of the mitigations in our technical blog, or register for the webinar below.
Disclosure Process
Given this vulnerability’s widespread impact, we have chosen to disclose it via a coordinated 90-day security disclosure to a handful of companies detailed below. The full list of more than 120 impacted companies will remain unpublished at this time, to prevent abuse by bad actors.
On August 23, 2024, we reported the issue to: Visa, Intel, JPMorganChase, BHHC (a subsidiary of Berkshire Hathaway), and UnitedHealth Group.
Of these companies above, JPMorganChase, and UnitedHealth Group have resolved the issue.
BHHC is now engaged with Zafran and actively working to resolve the issue.
How Zafran Customers Detect Control Gaps
Zafran is a Threat Exposure Management platform that uses an agentless approach to assess the technical configurations of your compensating controls to reveal and mitigate your true risk exposure to vulnerabilities and control gaps.
Most Zafran customers have integrated their WAF controls and will automatically see their exposure to this bypass technique using our Control Gaps module (seen below). The specific exposures are ranked on criticality, prioritizing the WAF misconfigurations that affect assets containing critical vulnerabilities.
The in-product recommended response workflows include fixing the control gaps that are most likely to be exploited, as well as specific steps to mitigate the WAF Bypass control misconfiguration.
If you are not a Zafran customer but would like to assess your organization's exposure to this misconfiguration, a simple API-based integration with a WAF provider can provide powerful insights within minutes.
