API Management? Did you say “API Management?”
Software application development models are evolutionary things. New technologies are always being created and require new approaches. It’s frequently the case today, that a service oriented architecture (SOA) model is used and that the end product is a software service that can be used by applications. The explosion in growth of mobile devices has only accelerated this trend. Every new mobile phone sold is another platform onto which applications are deployed. These applications are often built from services provided from multiple sources. The applications often consume these services through their APIs.
OK, that’s all interesting, but why does this matter?
Here’s why:
If you are providing a service, you’d probably like to receive payment when it’s used by an application. For example, let’s say that you’ve spent months creating a new service that provides incredibly accurate and timely driving directions. You can imagine every mobile phone GPS app making use of your service someday. That is, however, assuming that you can find a way to enforce a contract on consumers of the API and provide them with a service level agreement (SLA). Also, you have to find a way to actually track consumers’ use of the API so that you can actually enforce that SLA. Finally, you have to have the means to update a service and publish new versions of services.
Likewise, if you are consuming a service, for example, if you want to build the killer app that will use that cool new mapping service, you have to have the means to find the API, identify the API’s endpoint, and register your usage of the API with its provider.
The approach that is followed to fulfill both service providers’ and consumers’ needs is...API Management.
JBoss apiman 1.0
apiman is JBoss’ open source API Management system. apiman fulfills service API providers’ and consumers’ needs by implementing:
API Manager - The API Manager provides an easy way for API/service providers to use a web UI to define service contracts for their APIs, apply these contracts across multiple APIs, and control role-based user access and API versioning. These contracts can govern access to services and limits on the rate at which consumers can access services. The same UI enables API consumers to easily locate and access APIs.
API Gateway - The gateway applies the service contract policies of API Management by enforcing at runtime the rules defined in the contracts and tracking the service API consumers’ use of the APIs for every request made to the services. The way that the API Gateway works is that the consumer of the service accesses the service through a URL that designates the API Gateway as a proxy for the service. If the policies defined to govern access to the service (see a later section in this post for a discussion of apiman polices), the API Gateway then proxies requests to the service’s backend API implementation.
The best way to understand API Management with apiman is to see it in action. In this post, we’ll install apiman 1.0, configure an API with contracts through the API Manager, and watch the API Gateway control access to the API and track its use.
Prerequisites
We don’t need very much to run apiman out of the box. Before we install apiman, you’ll have to have Java (version 1.7 or newer) installed on your system. You’ll also need to git and maven installed to be able to build the example service that we’ll use.
A note on software versions: In this post we’ll use the latest available version of apiman as of December 2014. As if this writing, version 1.0 of apiman was just released (December 2014). Depending on the versions of software that you use, some screen displays may look a bit different.
Getting apiman
Like all JBoss software, installation of apiman is simple. First, you will need an application server on which to install and run apiman. We’ll use the open source JBoss WildFly server release 8.2 (http://www.wildfly.org/). To make things easier, apiman includes a pointer to JBoss WildFly on its download page here: http://www.apiman.io/latest/download.html
To install WildFly, simply download http://download.jboss.org/wildfly/8.2.0.Final/wildfly-8.2.0.Final.zip and unzip the file into the directory in which you want to run the sever.
Then, download the apiman 1.0 WildFly overlay zip file inside the directory that was created when you un-zipped the WildFly download. The apiman 1.0 WildFly overlay zip file is available here: http://downloads.jboss.org/overlord/apiman/1.0.0.Final/apiman-distro-wildfly8-1.0.0.Final-overlay.zip
The commands that you will execute will look something like this:
mkdir apiman
cd apiman
unzip wildfly-8.2.0.Final.zip
unzip -o apiman-distro-wildfly8-1.0.0.Final-overlay.zip -d wildfly-8.2.0.Final
Then, to start the server, execute these commands:
cd wildfly-8.2.0.Final
./bin/standalone.sh -c standalone-apiman.xml
The server will write logging messages to the screen. When you see some messages that look like this, you’ll know that the server is up and running with apiman installed:
13:57:03,229 INFO [org.jboss.as.server] (ServerService Thread Pool -- 29) JBAS018559: Deployed "apiman-ds.xml" (runtime-name : "apiman-ds.xml")
13:57:03,261 INFO [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on http://127.0.0.1:9990/management
13:57:03,262 INFO [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:9990
13:57:03,262 INFO [org.jboss.as] (Controller Boot Thread) JBAS015874: WildFly 8.2.0.Final "Tweek" started in 5518ms - Started 754 of 858 services (171 services are lazy, passive or on-demand)
If this were a production server, the first thing that we’d do is to change the OOTB default admin username and/or password. apiman is configured by default to use JBoss KeyCloak (http://keycloak.jboss.org/) for password security. Also, the default database used by apiman to store contract and service information is the H2 database. For a production server, you’d want to reconfigure this to use a production database. Note: apiman includes DDLs for both MySQL and PostgreSQL.
For the purposes of our demo, we’ll keep things simple and use the default configuration.
To access apiman’s API Manager UI, go to: http://localhost:8080/apiman-manager, and log in. The admin user account that we’ll use has a username of “admin” and a password of “admin123!”
You should see a screen that looks like this:
Before we start using apiman, let’s take a look at how apiman defines how services and the meta data on which they depend are organized.
Policies, Plans, and Organizations
apiman uses a hierarchical data model that consists of these elements: Polices, Plans, and Organizations:
Policies
Policies are at the lowest level of the data model, and they are the basis on which the higher level elements of the data model are built. A policy defines an action that is performed by the API Gateway at runtime. Everything defined in the API Manager UI is there to enable apiman to apply policies to requests made to services.
When a request to a service is made, apiman creates a chain of policies to be applied to that request. apiman policy chains define a specific sequence order in which the policies defined in the API Manager UI are applied to service requests.
The sequence in which incoming service requests have policies applied is:
First, at the application level. In apiman, an application is contracted to use one or more services.
Second, at the plan level. In apiman, policies are organized into groups called plans. (We’ll discuss plans in the next section of this post.)
Third, at the individual service level.
What happens is that when a service request is received by the API Gateway at runtime, the policy chain is applied in the order of application, plan, and service. If no failures, such as a rate counter being exceeded, occur, the API Gateway sends the request to the service’s backend API implementation. As we mentioned earlier in this post, the API Gateway acts as a proxy for the service:
Next, when the API Gateway receives a response from the service’s backend implementation, the policy chain is applied again, but this time in the reverse order. The service policies are applied first, then the plan policies, and finally the application policies. If no failures occur, then the service response is sent back to the consumer of the service.
By applying the policy chain twice, both for the originating incoming request and the resulting response, apiman allows policy implementations two opportunities to provide management functionality during the lifecycle. The following diagram illustrates this two-way approach to applying policies:
Plans
In apiman, a “plan” is a set policies that together define the level of service that apiman provides for service. Plans enable apiman users to define multiple different levels of service for their APIs, based on policies. It’s common to define different plans for the same service, where the differences depend on configuration options. For example, a group or company may offer both a “gold” and “silver” plan for the same service. The gold plan may be more expensive than the silver plan, but it may offer a higher level of service requests in a given (and configurable) time period.
Organizations
The “organization” is at top level of the apiman data model.
An organization contains and manages all elements used by a company, university, group inside a company, etc. for API management with apiman. All plans, services, applications, and users for a group are defined in an apiman organization. In this way, an organization acts as a container of other elements. Users must be associated with an organization before they can use apiman to create or consume services. apiman implements role-based access controls for users. The role assigned to a user defines the actions that a user can perform and the elements that a user can manage.
Before we can define a service, the policies that govern how it is accessed, the users who will be able to access, and the organizations that will create and consume it, we need a service and a client to access that service. Luckily, creating the service and deploying it to our WildFly server, and accessing it through a client are easy.
Getting and Building and Deploying the Example Service
The source code for the example service is contained in a git repo (http://git-scm.com) hosted at github (https://github.com/apiman). To download a copy of the example service, navigate to the directory in which you want to build the service and execute this git command:
git clone git@github.com:apiman/apiman-quickstarts.git
As the source code is downloading, you'll see output that looks like this:
git clone git@github.com:apiman/apiman-quickstarts.git
Initialized empty Git repository in /tmp/tmp/apiman-quickstarts/.git/
remote: Counting objects: 104, done.
remote: Total 104 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (104/104), 18.16 KiB, done.
Resolving deltas: 100% (40/40), done.
And, after the download is complete, you'll see a populated directory tree that looks like this:
└── apiman-quickstarts
├── echo-service
│ ├── pom.xml
│ ├── README.md
│ └── src
│ └── main
│ ├── java
│ │ └── io
│ │ └── apiman
│ │ └── quickstarts
│ │ └── echo
│ │ ├── EchoResponse.java
│ │ └── EchoServlet.java
│ └── webapp
│ └── WEB-INF
│ ├── jboss-web.xml
│ └── web.xml
├── LICENSE
├── pom.xml
├── README.md
├── release.sh
└── src
└── main
└── assembly
└── dist.xml
As we mentioned earlier in the post, the example service is very simple. The only action that the service performs is to echo back in responses the meta data in the REST (http://en.wikipedia.org/wiki/Representational_state_transfer) requests that it receives.
Maven is used to build the service. To build the service into a deployable .war file, navigate to the directory into which you downloaded the service example:
cd apiman-quickstarts/echo-service
And then execute this maven command:
mvn package
As the service is being built into a .war file, you'll see output that looks like this:
[INFO] Scanning for projects...
[INFO]
[INFO] Using the builder org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder with a thread count of 1
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building apiman-quickstarts-echo-service 1.0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ apiman-quickstarts-echo-service ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /jboss/local/redhat_git/apiman-quickstarts/echo-service/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ apiman-quickstarts-echo-service ---
[INFO] Compiling 2 source files to /jboss/local/redhat_git/apiman-quickstarts/echo-service/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ apiman-quickstarts-echo-service ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /jboss/local/redhat_git/apiman-quickstarts/echo-service/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ apiman-quickstarts-echo-service ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ apiman-quickstarts-echo-service ---
[INFO] No tests to run.
[INFO]
[INFO] --- maven-war-plugin:2.2:war (default-war) @ apiman-quickstarts-echo-service ---
[INFO] Packaging webapp
[INFO] Assembling webapp [apiman-quickstarts-echo-service] in [/jboss/local/redhat_git/apiman-quickstarts/echo-service/target/apiman-quickstarts-echo-service-1.0.1-SNAPSHOT]
[INFO] Processing war project
[INFO] Copying webapp resources [/jboss/local/redhat_git/apiman-quickstarts/echo-service/src/main/webapp]
[INFO] Webapp assembled in [23 msecs]
[INFO] Building war: /jboss/local/redhat_git/apiman-quickstarts/echo-service/target/apiman-quickstarts-echo-service-1.0.1-SNAPSHOT.war
[INFO] WEB-INF/web.xml already added, skipping
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.184 s
[INFO] Finished at: 2014-12-26T16:11:19-05:00
[INFO] Final Memory: 14M/295M
[INFO] ------------------------------------------------------------------------
If you look closely, near the end of the output, you'll see the location of the .war file:
/jboss/local/redhat_git/apiman-quickstarts/echo-service/target/apiman-quickstarts-echo-service-1.0.1-SNAPSHOT.war
To deploy the service, we can copy the .war file to our WildFly server's "deployments" directory. After you copy the service's .war file to the deployments directory, you'll see output like this generated by the WildFly server:
16:54:44,313 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015876: Starting deployment of "apiman-quickstarts-echo-service-1.0.1-SNAPSHOT.war" (runtime-name: "apiman-quickstarts-echo-service-1.0.1-SNAPSHOT.war")
16:54:44,397 INFO [org.wildfly.extension.undertow] (MSC service thread 1-16) JBAS017534: Registered web context: /apiman-echo
16:54:44,455 INFO [org.jboss.as.server] (DeploymentScanner-threads - 1) JBAS018559: Deployed "apiman-quickstarts-echo-service-1.0.1-SNAPSHOT.war" (runtime-name : "apiman-quickstarts-echo-service-1.0.1-SNAPSHOT.war")
Make special note of this line of output:
16:54:44,397 INFO [org.wildfly.extension.undertow] (MSC service thread 1-16) JBAS017534: Registered web context: /apiman-echo
This output indicates that the URL of the deployed example service is:
http://localhost:8080/apiman-echo
Remember, however, that this is the URL of the deployed example service if we access it directly. We'll refer to this as the "unmanaged service" as we are able to connect to the service directly, without going through the API Gateway. The URL to access the service through the API Gateway ("the managed service") at runtime will be different.
Now that our example service is installed, it’s time to install and configure our client to access the server.
Accessing the Example Service Through a Client
There are a lot of options available when it comes to what we can use for a client to access our service. We’ll keep the client simple so that we can keep our focus on apiman and simply install a REST client into the FireFox browser. The REST Client FireFox add-on (http://restclient.net/) is available here: https://addons.mozilla.org/en-US/firefox/addon/restclient/versions/2.0.3
After you install the client into FireFox, you can access the deployed service using the URL that we just defined. If you execute a GET command, you’ll see output that looks like this:
Now that our example service is built, deployed and running, it’s time to create the organizations for the service provider and the service consumer. The differences between the requirements of the two organizations will be evident in their apiman configuration properties.
Creating Users for the Service Provider and Consumer
Before we create the organizations, we have to create a user for each organization. We'll start by creating the service provider user. To do this, logout from the admin account in the API Manager UI. The login dialog will then be displayed.
Select the "New user" Option and register the service provider user:
Then, logout and repeat the process to register a new application developer user too:
Now that the new users are registered we can create the organizations.
Creating the Service Producer Organization
To create the service producer organization, log back into the API Manager UI as the servprov user and select “Create a new Organization”:
Select a name and description for the organization, and press “Create Organization”:
And, here’s our organization:
Note that in a production environment, users would request membership in an organization. The approval process for accepting new members into an organization would follow the organization's workflow, but this would be handled outside of the API Manager. For the purposes of our demonstration, we'll keep things simple.
Configuring the Service, its Policies, and Plans
To configure the service, we’ll first create a plan to contain the policies that we want applied by the API Gateway at runtime when requests to the service are made. To create a new plan, select the “Plans” tab. We’ll create a “gold” plan:
Once the plan is created, we will add policies to it:
apiman provides several OOTB policies. Since we want to be able to demonstrate a policy being applied, we’ll select a Rate Limiting Policy, and set its limit to a very low level. If our service receives more than 10 requests in a day, the policy should block all subsequent requests. So much for a “gold” level of service!
After we create the policy and add it to the plan, we have to lock the plan:
And, here is the finished, and locked plan:
At this point, additional plans can be defined for the service. We’ll also create a “silver” plan, that will offer a lower level of service (i.e., a request rate limit lower than 10 per day) than the gold plan. Since the process to create this silver plan is identical to that of the gold plan, we’ll skip the screenshots.
Now that the two plans are complete and locked, it’s time to define the service.
We’ll give the service an appropriate name, so that providers and consumers alike will be able to run a query in the API Manager to find it.
After the service is defined, we have to define its implementation. In the context of the API Manager, the API Endpoint is the service’s direct URL. Remember that the API Gateway will act as a proxy for the service, so it must know the service’s actual URL. In the case of our example service, the URL is: http://localhost:8080/apiman-echo
The plans tab shows which plans are available to be applied to the service:
Let’s make our service more secure by adding an authentication policy that will require users to login before they can access the service. Select the Policies tab, and then define a simple authentication policy. Remember the user name and password that you define here as we’ll need them later on when send requests to the service.
After the authentication policy is added, we can publish the service to the API Gateway:
And, here it is, the published service:
OK, that finishes the definition of the service provider organization and the publication of the service.
Next, we'll switch over to the service consumer side and create the service consumer organization and register an application to connect to the managed service through the proxy of the API Gateway.
The Service Consumer Organization
We'll repeat the process that we used to create the application development organization. Log in to the API Manager UI as the “appdev” user and create the organization:
Unlike the process we used when we created the elements used by the service provider, the first step that we’ll take is to create a new application and then search for the service to be used by the application:
Searching for the service is easy, as we were careful to set the service name to something memorable:
Select the service name, and then specify the plan to be used. We’ll splurge and use the gold plan:
Next, select “create contract” for the plan:
Then, agree to the contract terms (which seem to be written in a strange form of Latin in the apiman 1.0 release):
The last step is to register the application with the API Gateway so that the gateway can act as a proxy for the service:
Congratulations! All the steps necessary to provide and consume the service are complete!
There’s just one more step that we have to take in order for clients to be able access the service through the API Gateway.
Remember the URL that we used to access the unmanaged service directly? Well, forget it. In order to access the managed service through the API Gateway acting as a proxy for other service we have to obtain the managed service's URL. In the API Manager UI, head on over to the "APIs" tab for the application, click on the the “>” character to the left of the service name. This will expose the API Key and the service’s HTTP endpoint in the API Gateway:
In order to be able access the service through the API Gateway, we have to provide the API Key with each request. The API Key can be provided either through an HTTP Header (X-API-Key) or a URL query parameter. Luckily, the API Manager UI does the latter for us. Select the icon to the right of the HTTP Endpoint and this dialog is displayed:
Copy the URL into the clipboard. We’ll need to enter this into the client in a bit. The combined API Key and HTTP endpoint should look something like this:
Accessing the Managed Service Through the apiman API Gateway, Watching the Policies at Runtime
Thanks for hanging in there! The set up is done. Now, we can fire up the client and watch the policies in action as they are applied at runtime by the API Gateway, for example:
Open the client, and enter the URL for the managed service (http://localhost:8080/apiman-gateway/ACMEServices/echo/1.0?apikey=c374c202-d4b3-4442-b9e4-c6654f406e3d)
What happens first is that the authentication policy is applied and a login dialog is then displayed:
Enter the username and password (user1/password) that we defined when we created the authentication policy to access the service. The fact that you are seeing this dialog confirms that you are accessing the managed service and are not accessing the service directly.
When you send a GET request to the service, you should see a successful response:
So far so good. Now, send 10 more requests and you will see a response that looks like this as the gold plan rate limit is exceeded:
And there it is. Your gold plan has been exceeded. Maybe next time you’ll spend a little more and get the platinum plan! ;-)
Wrap-up
Let’s recap what we just accomplished in this demo:
We installed apiman 1.0 onto a WildFly server instance.
We used git to download and maven to build a sample REST client.
As a service provider, we created an organization, defined policies based on service use limit rates and user authentication, and a plan, and assigned them to a service.
As a service consumer, we searched for and found that service, and assigned it to an application.
As a client, we accessed the service and observed how the API Gateway managed the service.
And, if you note, in the process of doing all this, the only code that we had to write or build was for the client. We were able to fully configure the service, policies, plans, and the application in the API Manager UI.
What’s Next?
In this post, we’ve only scratched the surface of API Management with apiman. To learn more about apiman, you can explore its website here: http://www.apiman.io/
Join the project mailing list here: https://lists.jboss.org/mailman/listinfo/apiman-user
And, better still, get involved! Contribute bug reports or feature requests. Write about your own experiences with apiman. Download the apiman source code, take a look around, and contribute your own additions. apiman 1.0 was just released, there’s no better time to join in and contribute!
Acknowledgements
The author would like to acknowledge Eric Wittmann for his (never impatient) review comments and suggestions on writing this post!
Downloads Used in this Article
REST Client (http://restclient.net/) FireFox Add-On - https://addons.mozilla.org/en-US/firefox/addon/restclient/versions/2.0.3
Echo service source code - https://github.com/EricWittmann/apiman-quickstarts
apiman 1.0 - http://downloads.jboss.org/overlord/apiman/1.0.0.Final/apiman-distro-wildfly8-1.0.0.Final-overlay.zip
WildFly 8.2.0 - http://download.jboss.org/wildfly/8.2.0.Final/wildfly-8.2.0.Final.zip
Git - http://git-scm.com
Maven - http://maven.apache.org
References
apiman tutorial videos - http://vimeo.com/user34396826
Comments