I am having so much fun developing with OpenShift (specifically minishift). It is so cool to be able to change my code, build and deploy with one simple command.

$ oc start-build MyProject --from-dir=local/git/project.git

This command will pull my local changes, build them and deploy to OpenShift allowing for immediate testing using s2i. If you aren't familiar with s2i, it is a tool that will take your source code, create a docker image automagically. think about all the time I wasted over the years writing code, compiling and building a deployable artifact (like a WAR), and re-deploying to my server (sometimes this even required a server restart).

Upgrade your development experience with minishift! I am even helping a coding instructor set this up for his coding class to allow his student to have a better learning experience. They will spend their time more affectively learning how to code instead of spending wasted days setting up and debugging issues.

Have fun being productive!

Overview

This demo illustrates a retail store scenario using a mobile app to track customer movements while shopping at a store. There are many parts to the demo and I will step through each one in detail. The demo is hosted on github so you can download and try for yourself. Also, a short video of the demo can be found here.

The Demo Components and Roles

Beacon Data

The beacon data is generated by a python script to emulate customers moving in a store. The script will send MQTT messages to Red Hat JBoss A-MQ, Red Hat's messaging platform reflecting customer movements (Enter, Move, Exit).

There are two subscribers to the topics on the message broker, a web UI running a node.js MQTT client and a Red Hat JBoss BRMS instance. BRMS is Red Hat's Business Rules Management system. It serves as a decision point in our demo and has rules defined to determine if customers are "focused" (in the same department for x number of moves) or "roaming" (moving through the store). Both of these scenarios indicate that the customer may need assistance and a message is added to A-MQ indicating the customer's id and location in the store. This message is intended to provide an alert to sales personnel in the store.

Web UI

The web UI is listening for both customer movements and sales alerts and uses this data to show customer movements in the store and display the alerts.

SAP SQL Anywhere

In addition to categorizing customers and adding alerts to A-MQ, BRMS is also responsible for inserting data into SQL Anywhere. The SQL Anywhere instances provides web services that are called by BRMS to insert the data. When network bandwidth or connectivity is available, the data is then pushed to the HANA Cloud Platform. This is the beauty of the Intelligent Edge. The data will continued to be gather in the gateway and made available to the data center when possible.

HANA

The HANA instance in our architecture also pulls data from a second HANA database using HANA Smart Data Access (SDA) that contains the inventory of all stores in our fictitious corporation. All stores are also kept in this second instance.

Red Hat JBoss Data Virtualization

The data virtualization layer federates all related but disparate sources to provide a central connection to the native Android app. This app has been downloaded by the stores customer's and allows us to track their movements in the remote store. We can use this information along with the data in HANA, Salesforce customer data, postgreSQL customer sales history and promotional data from Amazon Redshift to target the precise promotional notification to send to the customer's mobile app. We can tell what department the customer is currently in and what their previous purchases were to match a promotion for the customer that will most likely result in a sale. This is in addition to all of the other data we can use in our app using the unified view of all of our enterprise and beacon data.

Reference Architecture

Summary

Please check out our github repository and try the demo out for yourself! As usual, please don't hesitate to ping me with any questions you may have!

Connecting to OLAP-based data sources is something Red Hat JBoss Data Virtualization (JDV) has supported for some time. The use of the invokeMdx() function of JDV allows a user to connect to and execute Cubes or BEx queries defined in SAP BW. The following steps show you how to create a connection, generate your MDX query and model your JDV procedure to pull back your data for SAP BW. This examples pulls data from a BEx query, but of course you can execute directly against a Cube if required. All you need is the MDX for the Cube.

Setting Up Your Test Environment

1. Creating an InfoCube and BEx query in SAP BW.

If you don't already have an InfoCube and BEx query, here are some ways to create sample data. I created a cube with 1,000,000 rows and add a BEx query on top of it using these steps.

a. Create InfoCube. I created a Cube based on the pre-requisite specified InfoCube in step b. Basically, it required an InfoCube that looks like this:

b. Generate test data. Using CUBE_SAMPLE_CREATE in the SAP UI, I was able to generate a cube with 1,000,000 rows. You can specify whatever number of rows you would like though.

2. Create BEx query. This is a really nice video that walks you through creating a BEx query.

3. Get the MDX for your BEx query. You can do this using SAP transaction MDXTEST or RSCRM_BAPI. For RSCRM_BAPI, you need to enable debug. To do that, go to transaction RSCRMDEBUG. Enter your SAP username and the Debugging Options as mdx_gen. Choose Insert and execute. Then exit this screen and go to transaction RSCRM_BAPI.

Accessing from Red Hat JBoss Data Virtualization

1. Setup a JDV server and connect from JBDS.
2. Add the driver directly into the server configuration file with an entry as shown below:

OR Run the following CLI command in $JDV_HOME/bin folder:

3. Import the source connection for JBDS (Import->Teiid Connection->Source Model)
   • Add the source connection information
     • JDBC URL: jdbc:xmla:Server=http://{bw_host}:{bw_port}/sap/bw/xml/soap/xmla
     • Credentials: sap_user/sap_password
4. Create the view model and add your virtual table.

• The format for the JDV virtual query is:
  • SELECT x.column1 AS column1, x.column2 AS column2, x.column3 AS column3 FROM (EXEC {source_model}.invokeMDX('{MDX_FOR_CUBE_OR_BEx_REPORT}')) AS x, ARRAYTABLE(w.tuple COLUMNS column1 type, column2 type, column3 type) as x

• If you are using my example, your SQL will look something like the following. Notice the root hierarchy value has been changed to "Measures". This is because the enterprise id used in SAP for the root are not resolvable from JDV. Also note that your other enterprise ids used in your SAP system will be different:

SELECT x.zcustid AS cust_id, x.zprdid AS prod_id, x.price AS price, x.quantity AS   quantity

FROM(EXEC SAP_BW_SOURCE.invokeMDX('SELECT

{[Measures].[058YFS98ML8W8QUZ11JQ6UHM1],[Measures].[058YFS98ML8W8QUZ11JQ6UNXL]}  ON COLUMNS,
  NON EMPTY CROSSJOIN({[ZCUSTID].LEVELS(01).MEMBERS},{[ZPRDID].LEVELS(01).MEMBERS})  ON ROWS
FROM [ZIC_TEST/Z_CUST_ORDER_LIST]')) AS w, ARRAYTABLE(w.tuple COLUMNS zcustid string, zprdid string, price decimal, quantity decimal) AS x

5. Save, right-click on your table and preview data.

That's it! You have successfully consumed SAP BW data in JDV!

What It Is

With the release of Teiid 8.11.Final, we have introduced support for SAP HANA as a data source. Teiid is the project used for creating the Red Hat JBoss Data Virtualization (JDV) product and Teiid 8.11.Final will be part of the upcoming Data Virtualization 6.2 release. SAP HANA is an in-memory data platform that is deployable as an on-premise appliance, or in the cloud.  It is a revolutionary platform that’s best suited for performing real-time analytics, and developing and deploying real-time applications. At the core of this real-time data platform is the SAP HANA database which is fundamentally different than any other database engine in the market today.

Image reference from SAP Community Network

JDV overview

How the SAP HANA Translator Works

Using the SAP HANA JDBC driver with Teiid, you can connect to SAP HANA, compose your views, and consume data from SAP HANA and other disparate data sources to create a single view of all of your SAP-related information. JDV can expose this single source via JDBC, ODBC, REST, OData or SOAP services. This allows for consumption of SAP HANA data as well as web services, relational databases, application systems, cloud, etc that can be utilized by enterprise applications, mobile, reporting or whatever client needs access. In addition, with JDV you can create custom views for each end user of the data with full CRUD capabilities, custom security and caching for a secure and performant solution to data integration with SAP.

A Typical Use Case

While there are many possibilities for solving SAP HANA data integration challenges using JDV, some scenarios are more common across all enterprises. A somewhat common problem users encounter is getting disparate data from SAP and other systems or web services and easily consuming that federated data in the field within mobile applications. JDV can easily accomplish this using the SAP HANA translator as well as its full suite of translators and connectors to create a single view and point of connection for your mobile applications. Furthermore, since the data from JDV can be exposed as web services (OData, REST, or SOAP), JDBC, or ODBC, the implementation can be tailored to be application-specific. The following diagram shows what an environment like this might look like, integrating SAP HANA with a SQL Anywhere data source and exposing as an OData service from a JDV instance running in the cloud.

Try it out... for free!

You can download Teiid 8.11.Final now which includes the SAP HANA translator. You will also need the Teiid Designer to create your models and views. Feedback is always welcome and please let me know if you have any questions!

First Impressions

I have attended multiple SAP techEd and d-code conferences over the years, but this was my first Sapphire. As anticipated, it was a different perspective in terms of audience. At techEd and d-code, most are developers with an in-depth knowledge SAP ABAP, development, data integration, open standards and other technical topics. At Sapphire, the average attendee was more business-minded, but many came from a technical background. There were many architects and product managers for example.

My previous webinars and talks on SAP integration with Red Hat JBoss Middleware were all heavily slanted towards the technical details. This time around I had talks discussing our Data Virtualization integration with SAP HANA. The fact that my session was devoid of the technical overtones from previous presentations, gave me the chance to appeal to the business case for our Data Virtualization solution for SAP HANA. I was able to take the time to discuss the issues with enterprise data management, the weaknesses of existing integration technologies, and where Red Hat JBoss Data Virtualization fills the gaps. I then followed the discussion with a drag-and-drop demo that federated data from SAP HANA and a MySql instance in Teiid Designer, the tooling for Data Virtualization. The talk and demo was well-received. I think explaining enterprise data needs, why current solutions fall short and how data virtualization can do what no other technology can to fulfill real-time data federation requirements was a huge eye-opener. Some expressed how they had no idea this technology was even available and the many man-hours it will save. This conference has given me a new perspective. I may re-purpose this presentation for a future, more technical conference. Developers will certainly appreciate the business value as well as the technical value of Data Virtualization.

Takeways

Every time I attend a conference and pull booth-duty, I learn so much from my colleagues regarding other Red Hat products. This time I got a full serving of GSS (Global Support Services) knowledge and fully understand why we have the best customer support organization in the industry. We never rest on our laurels and are always looking at ways to improve and grow. Not only are we the most compelling and cost-effective technical option with our full stack of solutions, we ice the cake with our best-in-class services and support.

Not to be lost in all my Red Hat cheerleading (sorry.. just calling it like I see it), is the fact that our partnership with SAP is stronger than it has ever been. I have been working with my SAP colleagues for so long, it feels like we are one big company. The value of an SAP plus Red Hat integration strategy in the enterprise is as compelling as it is due to a level of cooperation and synergy seldom seen in technology today. The benefactors are our customers whom are reaping the rewards through cost-savings, efficiencies, and productivity never realized before.

Please feel free to reach out to me if you have any questions about our SAP integration solutions with Red Hat JBoss Middleware and follow me on twitter @jonested. I look forward to SAP d-code in November where we have big plans for unveiling more collaboration between SAP and Red Hat. Hope to see you there!

Red Hat JBoss Data Virtualization allows you to expose multiple data sources as a single virtual database. Imagine pulling in all of your various sources together into a single, updatable view or series of views using a single point of connection. Now imagine doing that in the cloud using OpenShift! This article will walk through the steps to accomplish just that. The only prerequisite is that you have an SAP Gateway instance available. The demo uses the SAP Gateway demo system described here: http://scn.sap.com/docs/DOC-40986.

Here are the steps:

1. Setting Up a Data Virtualization instance on OpenShift.

2. Adding a SQL Anywhere instance in OpenShift.

3. Importing from SQL Anywhere and SAP Gateway.

4. Federating those SAP source into a single virtualized view that will be consumed by an Android application in JBoss Developer Studio (JBDS).

5. Run the Android application.

Here is a diagram that illustrates the architecture of this example:

1. Setting up Data Virtualization in OpenShift

The first step will be to provision Data Virtualization on OpenShift and install JBDS with Teiid Designer, the design tool for Data Virtualization. See this article for details: Provision Data Virtualization on OpenShift and Connect from Teiid Designer.

We will be working with an Android application, so you will need to install the Android plugin into JBDS. See http://developer.android.com/sdk/installing/installing-adt.html. Also, you will need to clone the Android application and artifacts from tejones/jboss-sap-integration · GitHub.

2. Adding a SQL Anywhere Instance in OpenShift

Now that we have our tooling and our OpenShift Data Virtualization instance running, we are now ready to create our SQL Anywhere database on OpenShift.

From a command prompt, type:

# rhc cartridge add https://github.com/githubuser8/openshift-origin-cartridge-sybase-sa/raw/master/metadata/manifest.yml -a <your app>

After adding the cartridge, you will have to accept the license agreement. There are instructions generated by the installation. After starting the ASA instance, refresh the OpenShift instance in JBDS to pickup the ASA and you can begin port forwarding in JBDS.

We will need to add our data the SQL Anywhere instance. Any JDBC capable SQL tool will suffice, but I use and love SQuirreL http://squirrel-sql.sourceforge.net/.

The DDL is located in {local_git_repo_location}/jboss-sap-integration/android-dv-sap/demo resources/ddl/flight_sqlana.ddl. I've also included a SQL Anywhere driver for use in the demo located here: {local_git_repo_location}/jboss-sap-integration/android-dv-sap/demo resources/driver/jconn4-26502.jar. You will need that to create the driver setup and connection in SQuirreL. You will use your local host IP in the connection URL since you are using port forwarding. The default credentials are username=dba and password=sql. Here is what your connection in SQuirreL should look like:

You are now ready to connect to the SQL Anywhere source in OpenShift and execute the DDL to create your objects and source your data. Note that the demo data is specific to the data coming from the SAP Gateway demo system. You may need to modify the information in the liveFlightFeed to match up with the test data coming from the Gateway FlightCollection. I am targeting Nov 26, 2014 for this demo. You can change the data to whatever you want. The liveFlightFeed is really meant to simulate a real-time flight data source such as FlightStats.

3. Importing from SQL Anywhere and SAP Gateway

As I have stated, I will connect to the SAP Gateway Demo system and use the Flight data for this example, (https://sapes1.sapdevcenter.com/sap/opu/odata/IWFND/RMTSAMPLEFLIGHT/). To access this data, you need an account. To create relational objects that map to the collections and functions in the SAP Gateway Service, we need to create a data source for the service and then import into Teiid Designer using the Teiid Connection import option. We will do the same for the SQL Anywhere instance so let's begin there.

1. Launch JBoss Developer Studio and switch to the Teiid Designer perspective.  The perspective options are shown in the upper right corner of JBDS.

2. Connect to the Data Virtualization instance in Teiid Designer as demonstrated in step 1.

3. Create a Model Project (or you can use any existing project).

• In ModelExplorer, Right-click > New > Teiid Model Project
• In the Model Project wizard, enter Demo for the Project name.  Click Finish to create the empty project.

4. Select the project, then Right-click > Import... You will see the import dialog (shown below).  Select Teiid Connection >> Source Model then click Next.  We are using the Teiid Connection Importer since we cannot directly connect to the DB port on OpenShift.  This importer allows us retrieve the DB schema through Teiid.

5. On the first page of the import wizard, click the New button in the DataSource section.  Click Add Driver to select the jconn4-26502.jar file from the driver directory of your local git project. Name the datasource Flight_Sybase_ASA and enter jdbc:sybase:Tds:{IP for OpenShift ASA instance}:2638/demo?ServiceName=demo. The username and password are dba and sql. Click Apply and then click OK. Click Next on the importer wizard.

6. On the next page of the wizard, set the Translator type to Sybase and add an optional source import property with the name importer.tableTypes and the value TABLE. This will avoid importing system table for the SQL Anywhere instance. Click Next.

7. On the next page, select a project and give the model name myflight. Click Next. This will deploy a temporary dynamic VDB that will enable the metadata mapping from SQL Anywhere objects the Data Virtualization relational objects by creating DDL.

8. You will now see the DDL that will be used to create the Data Virtualization relational object, click Next to see the objects that will be created and then click Finish.

This will create the objects for us in the ModelExplorer.

Creating the SAP Gateway Relational Objects

The process for importing the SAP Gateway source is a lot like importing SQL Anywhere except the connection information is different and we will use a different translator.

1. Repeat step 4 from above. When creating the datasource for SAP Gateway as in step 5 above, name the datasource SAPGatewayFlight and select the webservice driver. Enter https://sapes1.sapdevcenter.com/sap/opu/odata/IWFND/RMTSAMPLEFLIGHT for the URL, End Point and HTTPBasic for security. The username and password are your credentials for the SAP Gateway service you are using. Click Apply and then click OK. Click Next on the importer wizard.

2. On the next page, select the sap-nw-gateway translator and click Next. Enter SAPGatewayFlight for the model name and click Next.

3. Complete step 8 above and the relational objects that map to the SAP Gateway service will be generated for you in JBDS.

Note: With the current version of the Data Virtualization cartridge for OpenShift, there is an error in the mapping of the "byte" data type. The flightDetails_period field of the FlightCollection table should be change to "short". This will be corrected in the next release of the cartridge. To change the type, select the field and click on the ellipsis of the "datatype" property in the properties panel below. A data type dialog will popup allowing you to select the "short" type.

4. Federating SAP Sources into a Single Virtualized View

Now that you have imported the SQL Anywhere source from OpenShift and the SAP Gateway OData service, you are ready to join the data into a single view. To start, we will need to create a virtual model and table that will be use to federate the data from our SAP Gateway Flight service and our SQL Anywhere databse running in OpenShift.

1. Right-click on the Demo project and select New->Teiid Metadata Model. For Model Name enter AllFlightDataModel. Set the Model Type as View Model and click Finish.

2. Right-click on the new model and select New Child->Table. Enter AllFlightDataTable for the Name and click OK. Double-click on the generated model to open the modeling diagram.

3. Now expand the SAPGatewayFlight model and drag the FlightCollection table to the arrow with the T in the diagram. This will add the FlightCollection table to the transformation. Now do the same with the liveFlightFeed table form the myflight model to complete the transformation. Save your changes.

4. In order to expose the view table as an OData service in Data Virtualization, we need to add a primary key. Right-click on the AllFlightDataTable and select New Child->Primary Key. Name the new Primary Key allFlightData_PK and click the Ellipsis for the Columns property in the properties tab below the Model Explorer. Select the airline_iata, depart_time, arriveTime, flightDate, fromAirport and toAirport columns and click OK Save your changes.

We are now ready to create and deploy our virtual database.

1. Right-click on the Demo model project and click New->Teiid VDB and add your models.

Click Finish.

2. To deploy your new VDB, right-click on the Flight VDB in the Model Explorer and select Modeling->Deploy. A new datasource will be created for you and your Flight VDB will now be available in OpenShift as an OData service. You could also create a JDBC connection to the VDB if you wanted to, but we will use the OData capabilities of Data Virtualization for our Android application.

Testing Your New Service

Let's test our new service running in OpenShift. The URL for an OData service running in Data Virtualization is http://{host}:{port}/{vdbName}/{modelName.tableName} so we will use http://localhost:8080/odata/flight/AllFlightDataModel.AllFlightDataTable?$format=json to see our federated SAP data.

Note: Make sure you have port forwarding running in order to execute the service with the above URL.

5. Run the Android application

Running the Android app in JBDS using the Android emulator, you can access your data in DV on OpenShift using OData urls.

There are so many things to love about Red Hat Summit.. learning about new features and technologies, meeting current and prospective customers, seeing colleagues face-to-face after only knowing them virtually for years, the keynotes (especially the middleware keynote and demo), and presenting topics I am passionate about to people that are eager to listen and learn. That last one is my favorite. This year I spent a lot a time at the Red Hat JBoss Integration pod. There I could educate people on the power and value of Data Virtualization where we had looping demos on Data Virtualization (DV) with big data and SAP NetWeaver Gateway. Attendees were drawn by the lure of big data and intrigued by versatility of Data Virtualization that could not only consume Hadoop and other big data sources, but also integrate them with other sources for a single centralized view of all their analytical data.

The last few years, I have been involved with SAP integration with JBoss Middleware. Summit gave me a platform to show-off consuming SAP NetWeaver Gateway services into DV and exposing as a relational source for federation with other disparate but related data. Not only does this demo well, it illustrates the ability to easily consume SAP data and combine with other important data to create a single unified view of data that can be consumed by enterprise applications via JDBC, ODBC, SOAP, REST or OData. Here is a recording of the demo:

I also gave a joint talk with my Red Hat colleagues Kenny Peeples and William Collins that covered all connectivity options for SAP with Red Hat JBoss Middleware. I gave a brief history of solutions, starting with SAP Enterprise Services Registry which lead to the introduction of SAP NetWeaver Gateway in October of 2011. I also talked about OData which is now an OASIS spec with the recent approval of v4 and showed a demo of Data Virtualization with SAP. Kenny covered JBoss Fuse with the new SAP NetWeaver Camel component and William discussed and demo'd his new JCo Camel component. There was a lot of interest in the talk and many great questions. I have attached the slide deck from the presentation to this blog.

I'm looking forward to next June in Boston for the next Summit! Hope to see you there!