Skip to content

Toro Cloud Dev Center


Throttling rules

Martini uses throttling rules so it knows which HTTP endpoints to throttle, and the permissible amount of requests the server will process.

Properties

Throttling rules are defined through the following configurable properties:

Name Required Default value Description
Name true "name" A unique identifier for the rule.
Description false A description of what the rule is for.
Enabled false false Indicates whether the rule is active or inactive; no checks would be done for inactive rules.
Priority true 0 Influences the order by which Martini checks matches for rules; the lower the number, the higher the priority. An invocation may only match one rule, and in the event that many rules are applicable, Martini will choose the rule with the highest priority.
Callback Service false The service to be executed when a rule is matched.
URL Patterns false The mappings of HTTP endpoints to throttle in Ant path notation. An empty path list means the rule is applicable to all requests, regardless of path.
Date Type true Rolling Used to define define the permissible amount of requests per time frame. Options are: Rolling and Calendar.
Users true Everyone Group of users the rule is applicable to. Options are: Everyone, Anonymous Users Only, Martini Users Only1. Selecting Everyone means even requests from Marketplace users would be throttled.
Packages false Martini packages the rule is applicable to. An empty package list means the rule is applicable to all packages.
Services false Services the rule is applicable to. An empty service list means the rule is applicable to all services.

Rolling date type

Throttling rules allow you to specify the maximum number of requests the server could process for a given period of time. For rules with rolling date types, this period of time "rolls" with whatever the current time is. Consider the following configuration:

Unit Unit Amount Limits
Days 2 100

This tells Martini to allow only 100 invokes every two days. Martini will subtract exactly two days from the current request time, and then count the number of requests from that point to present to see how many times a service has already been invoked. If the limit has been exceeded, then an error will be thrown.

This means if the first request request was received on Saturday, 04/07/2015 05:43:42 (dd/MM/yyyy' 'HH:mm:ss)2; then Martini will count all invokes from 02/07/2015 05:43:42 to 04/07/2015 05:43:42.

You can configure invocation limits for the following units of time: milliseconds, seconds, minutes, hours, days, weeks, months, and years.

Calendar date type

A calendar date type would prompt Martini to count the number of requests from the starting minute/hour/day/week/month to the present time based on the Gregorian calendar.

For example, a request was received on Saturday, 04/07/2015 05:43:42 (dd/MM/yyyy' 'HH:mm:ss)2. Assuming that the first day of the week is Sunday, the expected starting times when counting service invokes are as follows:

Expected value
Starting minute 04/07/2015 05:43:00
Starting hour 04/07/2015 05:00:00
Starting day 04/07/2015 00:00:00
Starting week 28/06/2015 00:00:00
Starting month 01/07/2015 00:00:00

If a rule with per-minute throttling was matched in this scenario, Martini would count all invocations that have occurred from 04/07/2015 05:43:00 to 04/07/2015 05:43:59. And then it will decide if the invocation limit has been surpassed. In contrast, a rolling date type would have made Martini count invocations from 04/07/2015 05:42:42 to 04/07/2015 05:43:42 instead.

Rate limiting with calendar date types would be more efficient performance-wise. However, throttling may not be as accurate; sometimes Martini would allow twice the number of permissible invokes per minute. For example, if a rule was configured to impose a maximum of five invokes per minute and a user executed five invokes exactly 05:43:59, five more invokes can be made at 05:44:00 because a new counter begins at the start of each minute.

Callback service

A callback service gets invoked when a matching rule is found. It runs before the requested service (that which matched the rule), and can determine whether or not the requested service can be invoked.

The callback service's return value is used in determining whether or not to allow a service invocation. To help implement your logic, you can use the parameters/inputs Martini will pre-populate for the callback service (like what it does for [endpoint][endpoint] invocations).

The requested service will get invoked only when:

  • the returned value of the callback service is true; or
  • the configured return type is void.

The requested service will NOT get invoked when:

  • the returned value of the callback service is false; or
  • an exception is thrown by the callback service.

Exception on callback service during HTTP invocation

When an exception is thrown on a callback service triggered by an HTTP invocation, Martini will send the exception back as the server's HTTP response.

Parameters

Name Type Description
ruleEngine io.toro.martini.rengine.RuleEngine The engine that was used for rule evaluation.
request javax.servlet.http.HttpServletRequest The request involved in invoking the target service.
response javax.servlet.http.HttpServletResponse The response involved in invoking the target service.
ruleMetadata io.toro.martini.throttling.ThrottlingRuleMetadata The rule metadata that was matched.
packageServiceMeta io.toro.martini.service.PackageServiceMeta Martini Package metadata involved in invoking the target service.
martiniPackage io.toro.martini.ipackage.MartiniPackage The Martini Package involved in invoking the target service.
fact io.toro.martini.throttling.ThrottlingFact The input of the rule engine.
packageName java.lang.String The Martini Package name involved in invoking the target service.
user java.security.Principal Authenticated user used for invoking the target service.
serviceName java.lang.String The target service that will be invoked.
userGroup java.util.Set<java.lang.String> The user group(s) the authenticated user belongs to.
invokeKind java.lang.String The type of invocation. Value is always http.
tokenToConsume java.lang.Integer The amount of token(s) to consume. The value is always 1 as every service invocation is only counted as 1 token.

Viewing throttling rules

To view an instance's rules, open the Throttling & Monitoring Rules dialog. To open said dialog, right click on your instance and then click Throttling and Monetization from the context menu. Select the Throttling Rules tab to view existing throttling rules.

Viewing throttling rules

Viewing throttling rules

Creating a new throttling rule

To create a new throttling rule using the Throttling & Monitoring Rules dialog:

Creating a new throttling rule

Creating a new throttling rule

  1. Go to the Throttling Rules tab.
  2. Click the green, '+' labled, circular button.
  3. Populate the appearing form with the configuration you want for your rule.
  4. Click Save.

Updating a throttling rule

To update an existing throttling rule using the Throttling & Monitoring Rules dialog:

Updating an existing throttling rule

Updating an existing throttling rule

  1. Go to the Throttling Rules tab.
  2. Select the rule you want to update from the list of existing rules.
  3. Click the edit button located between the add and delete button.
  4. Populate the appearing form with the configuration you want for your rule.
  5. Click Save.

Enabling or disabling a throttling rule

Aside from toggling the Enabled checkbox in the form, you can also use the context menu to enable or disable a throttling rule. To do this, right click the throttling rule you want to enable or disable, and then select Enable or Disable from the appearing context menu.

Enabling a throttling rule

Enabling a throttling rule

Removing a throttling rule

To delete an existing throttling rule using the Throttling & Monitoring Rules dialog:

Deleting a throttling rule

Deleting a throttling rule

  1. Go to the Throttling Rules tab.
  2. Select the rule you want to update from the list of existing rules.
  3. Click the delete button located below the edit button.
  4. Confirm your action.

  1. Not to be confused with Marketplace users. 

  2. See patterns in SimpleDateFormat