Best practices
The following sections cover typical rate limiting configurations for common use cases. You can combine the provided example rules and adjust them to your own scenario.
The main use cases for rate limiting are the following:
- Enforce granular access control to resources. Includes access control based on criteria such as user agent, IP address, referrer, host, country, and world region.
- Protect against credential stuffing and account takeover attacks.
- Limit the number of operations performed by individual clients. Includes preventing scraping by bots, accessing sensitive data, bulk creation of new accounts, and programmatic buying in ecommerce platforms.
- Protect REST APIs from resource exhaustion (targeted DDoS attacks) and resources from abuse in general.
- Protect GraphQL APIs by preventing server overload and limiting the number of operations.
A common use case is to limit the rate of requests performed by individual user agents. The following example rule allows a mobile app to perform a maximum of 100 requests in 10 minutes. You could also create a separate rule limiting the rate for desktop browsers.
Setting | Value |
---|---|
Matching criteria | User Agent equals MobileApp |
Expression | http.user_agent eq "MobileApp" |
Counting characteristics | IP |
Rate (Requests / Period) | 100 requests / 10 minutes |
Action | Managed Challenge |
Another use case when controlling access to resources is to exclude or include IP addresses or Autonomous System Numbers (ASNs) from a rate limiting rule.
The following example rule allows up to 10 requests per minute from the same IP address doing a GET
request for /status
, as long as the visitor’s IP address is not included in the partner_ips
IP list.
Setting | Value |
---|---|
Matching criteria | URI Path equals /status and Request Method equals GET and IP Source Address is not in list partner_ips |
Expression | http.request.uri.path eq "/status" and http.request.method eq "GET" and not ip.src in $partner_ips |
Counting characteristics | IP |
Rate (Requests / Period) | 10 requests / 1 minute |
Action | Managed Challenge |
Some applications receive requests originated by other sources (for example, used by advertisements linking to third-party pages). You may wish to limit the number of requests generated by individual referrer pages to manage quotas or avoid indirect DDoS attacks.
Setting | Value |
---|---|
Matching criteria | URI Path equals /status and Request Method equals GET |
Expression | http.request.uri.path eq "/status" and http.request.method eq "GET" |
Counting characteristics | Header (Referer ) 1 |
Rate (Requests / Period) | 100 requests / 10 minutes |
Action | Block |
This example rule requires Advanced Rate Limiting.
SaaS applications or customers using Cloudflare SSL for SaaS might have thousands of hosts under the same zone, which makes creating individual rules per host impractical. To overcome this, you can create a rate limiting rule that uses the host as a counting characteristic.
The following example rule will track the rate of requests to the /login
endpoint for each host:
Setting | Value |
---|---|
Matching criteria | URI Path equals /login and Request Method equals GET |
Expression | http.request.uri.path eq "/login" and http.request.method eq "GET" |
Counting characteristics | IP and Host |
Rate (Requests / Period) | 10 requests / 10 minutes |
Action | Block |
This example rule requires Advanced Rate Limiting.
A typical use case of rate limiting is to protect a login endpoint. The following example contains three different rate limiting rules with increasing penalties to manage clients making too many requests.
Rule #1
Setting | Value |
---|---|
Matching criteria | Hostname equals example.com and URI Path equals /login and Request Method equals POST |
Expression | http.host eq "example.com" and http.request.uri.path eq "/login" and http.request.method eq "POST" |
Counting characteristics | IP |
Counting criteria | URI Path equals /login and Method equals POST and Response code is in (401, 403) |
Counting expression | http.request.uri.path eq "/login" and http.request.method eq "POST" and http.response.code in {401 403} |
Rate (Requests / Period) | 4 requests / 1 minute |
Action | Managed Challenge |
Rule #2
Setting | Value |
---|---|
Matching criteria | Hostname equals example.com and URI Path equals /login and Request Method equals POST |
Expression | http.host eq "example.com" and http.request.uri.path eq "/login" and http.request.method eq "POST" |
Counting characteristics | IP |
Counting criteria | URI Path equals /login and Request Method equals POST and Response Status Code is in (401, 403) |
Counting expression | http.request.uri.path eq "/login" and http.request.method eq "POST" and http.response.code in {401 403} |
Rate (Requests / Period) | 10 requests / 10 minutes |
Action | Managed Challenge |
Rule #3
Setting | Value |
---|---|
Matching criteria | Host equals example.com |
Expression | http.host eq "example.com" |
Counting characteristics | IP |
Counting criteria | URI Path equals /login and Request Method equals POST and Response Status Code is in (401, 403) |
Counting expression | http.request.uri.path eq "/login" and http.request.method eq "POST" and http.response.code in {401 403} |
Rate (Requests / Period) | 20 requests / 1 hour |
Action | Block for 1 day |
Rule #1 allows up to four requests per minute, after which a Managed Challenge is triggered. This configuration allows legitimate customers a few attempts to remember their password. If an automated actor makes several requests, that client will likely be blocked by an unsolved Managed Challenge. On the other hand, if a human gets and passes the challenge when reaching rule #1’s rate limit, rule #2 will provide the next level of protection, allowing for up to 10 requests over the next 10 minutes. For clients exceeding this second threshold, rule #3 (the most severe) will apply, blocking the client for one day.
These three rules have a counting expression separate from the rule expression (also known as mitigation expression). When you configure a separate counting expression, the matching criteria will only be used when an action is triggered. In the counting expression you can include conditions based on the HTTP response status code and HTTP response headers, therefore integrating rate limiting with your backend logic.
You can also decide to have two different expressions — a counting expression and a rule/mitigation expression — to define:
- The requests used to compute the rate.
- The requests actually acted upon.
For example, rule #3 computes the rate considering POST
requests to /login
that returned a 401
or 403
HTTP status code. However, when the rate limit is exceeded, Cloudflare blocks every request to the example.com
host generated by the same IP. For more information on counting expressions, refer to How Cloudflare determines the request rate.
You can use rate limiting to limit the number of operations performed by a client. The exact rule providing this protection will depend on your application. The following examples address content scraping via query string parameters or JSON body.
In this example, clients perform operations (such as looking up prices and adding to basket) on an ecommerce website using different query string parameters. For example, a typical request sent by a client could be similar to the following:
GET https://store.com/merchant?action=lookup_price&product_id=215Cookie: session_id=12345
Your security team might want to consider setting up a limit on the number of times a client can lookup prices to prevent bots — which may have eluded Cloudflare Bot Management — from scraping the store’s entire catalog.
Rule #1
Setting | Value |
---|---|
Matching criteria | URI Path equals /merchant and URI Query String contains action=lookup_price |
Expression | http.request.uri.path eq "/merchant" and http.request.uri.query contains "action=lookup_price" |
Counting characteristics | IP |
Rate (Requests / Period) | 10 requests / 2 minutes |
Action | Managed Challenge |
Rule #2
Setting | Value |
---|---|
Matching criteria | URI Path equals /merchant and URI Query String contains action=lookup_price |
Expression | http.request.uri.path eq "/merchant" and http.request.uri.query contains "action=lookup_price" |
Counting characteristics | IP |
Rate (Requests / Period) | 20 requests / 5 minute |
Action | Block |
These two rate limiting rules match requests performing a selected action (look up price, in this example) and use IP
as the counting characteristic. Similarly to the previous /login
example, the two rules will help reduce false positives in case of persistent (but legitimate) visitors.
To limit the lookup of a specific product_id
via query string parameter, you could add that specific query parameter as a counting characteristic, so that the rate is calculated based on all the requests, regardless of the client. The following example rule limits the number of lookups for each product_id
to 50 requests in 10 seconds.
Setting | Value |
---|---|
Matching criteria | URI Path equals /merchant |
Expression | http.request.uri.path eq "/merchant" |
Counting characteristics | Query (product_id ) |
Rate (Requests / Period) | 50 requests / 10 seconds |
Action | Block |
This example rule requires Advanced Rate Limiting.
You could follow the same pattern of rate limiting rules to protect applications handling reservations and bookings.
Consider an application that handles the operation and its parameters through the request body in JSON format. For example, the lookup_price
operation could look like the following:
POST https://api.store.com/merchantCookie: session_id=12345
Body:{ "action": "lookup_price", "product_id": 215}
In this scenario, you could write a rule to limit the number of actions from individual sessions:
Setting | Value |
---|---|
Matching criteria | URI Path equals /merchant and JSON String action equals lookup_price |
Expression | http.request.uri.path eq "/merchant" and lookup_json_string(http.request.body.raw, "action") eq "lookup_price" |
Counting characteristics | Cookie (session_id ) |
Rate (Requests / Period) | 10 requests / 2 minutes |
Action | Managed Challenge |
This example rule requires Advanced Rate Limiting and payload inspection.
You could also limit the number of lookups of each product_id
regardless of the client making the requests by deploying a rule like the following:
Setting | Value |
---|---|
Matching criteria | URI Path equals /merchant and JSON field action equals lookup_price |
Expression | http.request.uri.path eq "/merchant" and lookup_json_string(http.request.body.raw, "action") eq "lookup_price" |
Counting characteristics | JSON field (product_id ) |
Rate (Requests / Period) | 50 requests / 10 seconds |
Action | Block |
This example rule requires Advanced Rate Limiting and payload inspection.
A general approach to identify traffic from bots is to rate limit requests that trigger a large volume of 403
or 404
response status codes from the origin server. This usually indicates automated activity from scraping applications.
In this situation, you could configure a rule similar to the following:
Setting | Value |
---|---|
Matching criteria | Hostname equals example.com |
Expression | http.host eq "example.com" |
Counting characteristics | IP |
Increment counter when | Response Status Code is in (401, 403) |
Counting expression | http.response.code in {401 403} |
Rate (Requests / Period) | 5 requests / 3 minutes |
Action | Managed Challenge |
To control the rate of actions performed by automated sources, consider use rate limiting rules together with Cloudflare Bot Management. With Bot Management, you can use the bot score as part of the matching criteria to apply the rule only to automated or likely automated traffic. For example, you can use a maximum score (or threshold) of 30
for likely automated traffic and 10
for automated traffic.
If your application tracks sessions using a cookie, you can use the cookie to set the rate limiting context (that is, use it as a counting characteristic). By setting the rate limiting characteristic to Cookie, the rule will group together requests from different IP addresses but belonging to the same session, which is a common scenario when dealing with a bot network performing a distributed attack.
Rule #1
Setting | Value |
---|---|
Matching criteria | Bot Score less than 30 and URI Query String contains action=delete |
Expression | cf.bot_management.score lt 30 and http.request.uri.query contains "action=delete" |
Counting characteristics | Cookie (session_id ) |
Rate (Requests / Period) | 10 requests / 1 minute |
Action | Managed Challenge |
Rule #2
Setting | Value |
---|---|
Matching criteria | Bot Score less than 10 and URI Query String contains action=delete |
Expression | cf.bot_management.score lt 10 and http.request.uri.query contains "action=delete" |
Counting characteristics | Cookie (session_id ) |
Rate (Requests / Period) | 20 requests / 5 minute |
Action | Block |
These example rules require Advanced Rate Limiting and Bot Management.
If the application does not use a session cookie, you can use JA3 fingerprints to identify individual clients. A JA3 fingerprint is a unique identifier, available to customers with Bot Management, that allows Cloudflare to identify requests coming from the same client. All clients have an associated fingerprint, whether they are automated or not.
Setting | Value |
---|---|
Matching criteria | URI Path equals /merchant and Bot Score less than 10 |
Expression | http.request.uri.path eq "/merchant" and cf.bot_management.score lt 10 |
Counting characteristics | JA3 Fingerprint |
Rate (Requests / Period) | 10 requests / 1 minute |
Action | Managed Challenge |
This example rule requires Advanced Rate Limiting and Bot Management.
APIs can put significant strain on the application backend because API requests can be expensive to compute or serve. These requests may also require complex operations (such as data processing and large data lookups) that, if abused, can eventually bring down an origin server.
Advanced Rate Limiting can mitigate many types of volumetric attacks, like DDoS attacks, mass assignment, and data exfiltration.
A common concern is to limit POST
actions. For authenticated traffic, you can use API Discovery to identify a suitable rate of request per endpoint, and then create a rate limiting rule like the following:
Setting | Value |
---|---|
Matching criteria | URI Path equals /endpoint1 and Request Method equals POST |
Expression | http.request.uri.path eq "/endpoint1" and http.request.method eq "POST" |
Counting characteristics | Header (x-api-key ) |
Rate (Requests / Period) | As suggested by API Discovery or assessed by analyzing past traffic. |
Action | Block |
This example rule requires Advanced Rate Limiting. API Discovery requires an additional license.
The counting characteristic can be any header, key, token, cookie, query parameter, or even JSON body field, since some APIs include a session ID or user ID as part of the JSON body. Refer to the following sections for additional information:
- If your unique identifier is in the URI path, refer to Protect resources.
- If your unique identifier is in the JSON body, refer to Prevent content scraping (via body).
GET
requests can also create excessive strain on an application or have an impact on costly resources, such as bandwidth. For example, consider an application with a large amount of stored files (such as images) where clients can download a file by accessing their specific URL:
GET https://api.store.com/files/<FILE_ID>Header: x-api-key=9375
You probably wish to limit the number of downloads to avoid abuse, but you do not want to write individual rules for each file, given the size of the data storage. In this case, you could write a rule such as the following:
Setting | Value |
---|---|
Matching criteria | Hostname equals api.example.com and Request Method equals GET |
Expression | http.host eq "api.example.com" and http.request.method eq "GET" |
Counting characteristics | Path |
Rate (Requests / Period) | As suggested by API Discovery or assessed by analyzing past traffic. |
Action | Block |
This example rule requires Advanced Rate Limiting.
The rule defines a limit of 10 downloads in 10 minutes for every file under https://api.store.com/files/*
. By using Path as the rule characteristic, you avoid having to write a new rule every time there is a new uploaded file with a different <FILE_ID>
. With this rule, the rate is computed on every request, regardless of their source IP or session identifier.
You could also combine Path with the x-api-key
header (or IP, if you do not have a key or token) to set the maximum number of downloads that a specific client, as identified by x-api-key
, can make of a given file:
Setting | Value |
---|---|
Matching criteria | Hostname equals api.store.com and Request Method equals GET |
Expression | http.host eq "api.example.com" and http.request.method eq "GET" |
Counting characteristics | Path and Header (x-api-key ) |
Rate (Requests / Period) | As suggested by API Discovery or assessed by analyzing past traffic. |
Action | Block |
This example rule requires Advanced Rate Limiting.
One of the biggest challenges posed by applications built on GraphQL is that a single path manages all queries to the server, and every request is usually a POST
operation.
The purpose of the request is usually embedded in the body, which has information on what data the client wants to fetch or mutate (according to GraphQL’s terminology for server-side data modification), along with any additional data required to carry out the action.
The following examples are based on an application that accepts reviews for movies. A GraphQL request could look like the following:
POST https://moviereviews.example.com/graphqlCookie: session_id=12345
Body:{ "data": { "createReview": { "stars": 5, "commentary": "This is a great movie!" } }}
To limit the rate of actions, you could use the following rule:
Setting | Value |
---|---|
Matching criteria | URI Path equals /graphql and Body contains createReview |
Expression | http.request.uri.path eq "/graphql" and http.request.body.raw contains "createReview" |
Counting characteristics | Cookie (session_id ) |
Rate (Requests / Period) | 5 requests / 1 hour |
Action | Block |
This example rule requires Advanced Rate Limiting and payload inspection.
The complexity necessary to handle a GraphQL request can vary significantly. Since the API uses a single endpoint, it is difficult to figure out the complexity of each request before it has been served.
To protect the origin server from resource exhaustion, rather than limiting the number of requests you need to limit the amount of complexity necessary to handle a single client over a period of time. Cloudflare Rate Limiting allows you to create rules that track complexity over time and block subsequent requests after reaching a complexity budget or limit.
This type of rate limiting requires that the server scores every served request according to the request’s complexity. Additionally, the server must add this score to the response as an HTTP header. Then, the rate limiting mechanism will use this information to update the budget for that specific client.
For example, the following rule defines a total complexity budget of 1,000 per hour:
Setting | Value |
---|---|
Matching criteria | URI Path contains /graphql |
Expression | http.request.uri.path eq "/graphql" |
Counting characteristics | Cookie (session_id ) |
Score rate (score / period) | 1,000 / 1 hour |
Score location | Response header (score ) |
Action | Block |
This example rule requires Advanced Rate Limiting and payload inspection.
When the origin server processes a request, it adds a score
HTTP header to the response with a value representing how much work the origin has performed to handle it — for example, 400
. In the next hour, the same client can perform requests up to an additional budget of 600
. As soon as this budget is exceeded, later requests will be blocked until the timeout expires.
-
The HTTP header name uses a misspelling of “referrer”. ↩