API Management: Performance is a Feature

[eric.wittmann]

I think a key focus of the API Management project in Overlord must be reducing the overhead of the Gateway.  The benefits of introducing the apiman layer into a SOA architecture are obvious:  governance (including authentication, authorization, throttling, metering, etc...).  However, the impact of the apiman layer is clearly performance overhead introduced into every single API call.

In this discussion, I would like to try to capture/anticipate as many aspects of performance as possible.

To start, here is some information provided recently by a stakeholder:

1. Ensure RESTful / HTTP caching continues to work as expected for applications - e.g. Cache-Control, ETags, etc. You want browsing and mid-tier caching to continue to work very, very effectively and to eliminate service calls altogether whenever possible.
2. If a call is transiting your service, ensure there isn't always a database lookup involved in that request's path to being serviced.  Ideally policies should be published once into the proxy from their data repository and stateful access tokens kept in an in-memory datagrid.
3. Any caching done in the Gateway should be able to be invalidated.  Caching is critical to supporting acceptable Gateway performance, but it should be flexible enough to support common SLA use-cases (such as disabling accounts, logging out, etc).

What are some other performance related considerations?

[objectiser]

In terms of performance, we should also look at the how the 'gateway' is integrated into the environment.

There will be cases where the services being represented need to be invoked remotely, and therefore we can't get away from the fact that the requests/responses are going to be marshalled again in this extra step.

However we should ensure that where appropriate the 'gateway' can directly invoke service implementations (i.e. with remoting) in a co-located environment - so in this configuration the gateway is essentially replacing the transport layer for the service. This should be possible with switchyard.

Another configuration we may want to support is providing the gateway functionality as an embedded capability within interceptors/handlers - so in this case the 'gateway' may not be dealing with routing, but may apply authentication and other policies (e.g. throttling, service suspension, etc).

[eric.wittmann]

Good points - we need integration points on both ends of the Gateway (manifested endpoints and proxies to backend services) so that those layers can be customized.  I'm definitely envisioning a connector layer between the Gateway's Policy Chain and the backend service. I hadn't thought as much about the options for endpoint manifestation.  In situations where the endpoints are manifested by some other technology, then the Policy Chain simply needs to be injected into the appropriate place(s) in that technology.  This is excellent to consider when designing/implementing the policy layer.

[eric.wittmann]

Another aspect of performance to consider is how metrics are recorded.  Data collection should be configurable, but when it is enabled we need to be mindful of its impact on performance.  I suspect we will want to do any recording of metrics asynchronously.  One question is how transactional that recording needs to be - it may be more performant to simply asyncronously log the data, but with the understanding that the occasional entry will be lost under extreme situations (e.g. server crash).  Perhaps both must be supported.