Of all the open source projects that have impressed me over the years Vert.x comes close to the top - probably at the top if you catch me on a good day. It's relative simplicity belies a combinatorial complexity that allows developers to go from zero to enterprise ready in small steps or giant leaps. The fact it has been imitated in recent years in other languages is just another clear indication of the power of the project. And yes, it clearly takes a leaf or two from Node.js and others but as has been said before: "Good Artists Copy; Great Artists Steal."
I've mentioned a few times about reactive, asynchronous patterns and how they're appropriate for things such as microservices. Vert.x is already a great way of developing microservices from scratch but I also believe that the concepts of the polyglot, non-blocking, event bus is ideal for bridging and integrating with existing applications or services. There are going to be a variety of ways for creating (micro) services and applications, but I believe that Vert.x offers some building blocks they all need. I'm hoping to see us use it much more in the coming months.
For many many years the Java world has popularised frameworks for developers. Most developers understand that frameworks sit on top of stacks and typically act as a way of simplifying the interaction with the complexities that exist within. Experienced developers have long been able to build complex systems without frameworks but it has often been at the expense of time and reliability (some definition of efficiency). The best frameworks are extremely prescriptive, taking the plethora of configurable options which are presented to the developer by the combination of software components that make up the stack and narrowing them down to a very specific subset. They take the complexity and flexibility and condense it down to something which more than 80% of developers will find useful or easy to get to grips with.
Now that's not to suggest that this complexity and flexibility is bad. Far from it: one size rarely fits all and whilst one developer may not understand why it's even necessary to have an option to tweak Widget XYZ, another developer would find it impossible for that to not be available. Component developers have always tried to provide 1001 options for the wide range of users they can perceive and that's the right thing for them to do. However, many component developers are unable to consider how their implementations may be best tailored for the majority: in fact it may be anathema to them to do so. In some ways I speak from experience with our experiences around Arjuna! It's only in recent years, with the STM implementation, that I think we've managed to provide a framework that hits most of the user needs.
But I digress. This is not meant to pit framework developers (or users) against component/stack developers (or users). As software developers we need them both. However, in some areas, and it seems specifically in the Java space, we often hear about the frameworks more than the software entities upon which they rely. I suppose it's similar to movies: we hear more about the actors on screen than the plethora of people behind the camera who actually make the films possible. Another analogy might be the iceberg, where the majority lies below the ocean surface! This is a problem I've run into many times over the years and written about a few times. I'd put myself more firmly into the component/stack category of developers, though I've tried to learn from some of the more successful framework implementations and developers when coming up with the Narayana STM solution. Having many of those framework developers on my staff has helped, so it's probably an unfair advantage I have over others! However, my main point in writing this entry isn't about me or what I've done, it's about how frameworks and components/stacks exist in a symbiotic relationship: frameworks need the stacks on which they are deployed, or they have nothing to make them work, and likewise the stacks need frameworks if their capabilities are to be made readily available to a wider range or developers.
What "gets my goat", as we're used to saying in the UK, is that some framework developers are often less than up front with the fact that their offerings are critically dependant on the stacks "below". Don't get me wrong: I'm not making any judgement about the relative importance of the framework versus the stack in the final solution. However, I do believe in credit where credit is due. Now the users of frameworks are excused from knowing too much about what is happening under the covers: it's like driving a performance car and not knowing the details of who designed and built the engine; but at some point you do need to know that what makes a car perform is at least as important as how it looks! Despite the fact I work for Red Hat and am JBoss CTO I'm trying to be objective here. We've made some really good frameworks that build on our stacks. We've also been less than successful on others. Likewise some of our partners or competitors have made some great frameworks that also rely on JBoss components. The right framework can make you incredibly productive. But the right framework also needs the right support from the components and stack upon which it is deployed. If you don't have a good framework but do have a good stack then a good developer can still be productive but not necessarily efficient. If you have a good framework but a poor stack (e.g., not reliable or scalable) then a good developer is going to realise quickly that all that glisters is not gold!
OK so you may well ask what's the conclusion? First of all I'm not sure there's just one conclusion. If you're a developer then of course you're going to look for the framework that makes you the most productive (definition: get done what's needed in the shortest period of time, but ensuring when you move on to your next company or project you don't leave a steaming pile of poop for someone else to pick up). These days we're a wider Java community that focusses a lot on the implementation language and the framework, since they typically go hand in hand. Projects often have tight deadlines and aggressive milestones making it easy to justify selecting your approach based on the experiences of others who may also have been subject to tight deadlines. But ultimately we need to take the time to investigate the entire solution not just the framework because one really can't succeed without the other.
I had a great time at JavaOne this year and Red Hat had a fantastic showing, with many sessions at the event, many more booth sessions (which were packed out as usual) and of course I made a brief appearance during the kickoff keynote. There had been a lot of interest in this event due to some recent events and rumours about what Oracle would or wouldn't say, specifically about the future of Java EE. This was even a topic of conversation during the JCP EC face-to-face meeting preceding JavaOne. But nothing much really happened and life, it seemed, would go on as usual. However, there was still a lot of discussion during several sessions and through the usual chatter on the floors or parties about the relevancy of Java EE these days.
But let's face it, this isn't a question that is only being asked in 2015; I've been involved in enterprise Java since before it was called J2EE and within a few years of bring created people were asking the same question. As I've mentioned many times over the past few years, people have asked the question when Web Services came along, then REST, PaaS/Cloud, mobile and now IoT. Each time the concept of middleware has evolved and Java EE, and its associated implementations, with it. People get hung up on the concept of Java EE and application servers as bloated, monolithic entities without actually looking at the reality of today's implementations (well, at least one). However, I'm not here to repeat what I've said time and time again (remember this from 2011?), most recently at HPTS 2015. Approaches such as WildFly Swarm and the massive interest it has seen, show that there's still a lot of interest. And I've already suggested how Java EE can fit into the next generation platforms.
Rather than revisit this question, which I think I've answered sufficiently, I wanted to link to Ian's entry on the JavaOne session he did, which I attended. It's always good to hear from someone else on the topic, though I'm sure some will argue he's not objective about this and neither am I. If you're still not convinced, think of the core capabilities within any Java EE application server, such as transactions, messaging, security, cacheing etc; services/capabilities that are needed way beyond Java and Java EE, pre-dating Java by decades and used together or independently in many applications. You can consider Java EE as a way of packaging these things together into a convenient bundle, where they are guaranteed to work well together or independently, and in most cases you probably don't even know they're there. Over the years, before Java was created and way beyond when it is just a mention in history books, that packaging will change but you'll still have something recognisable within middleware implementations, perhaps not co-locating services, perhaps not all implemented in the same language etc. And your future applications will probably not be able to tell the difference or know they're there ... again.
So whilst I think the original question is an important one to ask, I think a much better question is "Where should I use the core capabilities within Java EE?" And if you've got a suitably flexible and agile application server implementation, your application won't need to care that Java EE is, or is not, under the covers providing the desired dependability and reliability.
I've written before about the way I think the next generation platform will evolve. Here I want to take a moment to suggest how this might be approached in the future within Red Hat middleware, using JBoss, Fuse and other technologies including Vert.x. I suppose if I were working for any other company than Red Hat I might have to say that there are some disclaimers about my opinions being my own etc. However, whilst they certainly are to a degree, because we're doing this in open source you can see what we're doing and even get involved yourself!
Of course microservices are the future. Ok, maybe there was a hint of sarcasm in that last sentences! Microservices have a role to play, just as SOA does (yes, I still believe the two are closely tied). There is some truth in there though: more streamlined, agile and dedicated services will be the basis of future application development, whether using (immutable) containers such as Docker or just the standard JVM, perhaps with fatjars. However, anyone who believes that the future of software (middleware) will appear instantaneously has obviously not looked back at other transitions, such as bespoke-to-CORBA or CORBA-to-J2EE. These things take time and evolution rather than revolution is the natural approach. Even if you've not been involved in middleware there are similar examples elsewhere in our industry: COBOL really is still in deployment today! Look at the interest we have around Blacktie!
Therefore, the future will evolve. Yes people will want to develop new applications (so called greenfield sites) using the latest and greatest framework or stack. But they'll also want to integrate with existing business logic and services written in a variety of present day technologies. So there'll still be Java EE application servers (e.g., EAP) with business logic within them, some of it legacy, some written from scratch today and into the future, despite what some may believe.
I believe that due to the fact Java EE has been the dominant non-Microsoft development and deployment platform for well over a decade, there are so many developers out there who are comfortable with it. Yes some may complain about the apparent bloat of implementations, but the reality is that it's still very easy to develop against and use from a variety of different programming languages. That's why the evolution towards any new paradigm is going to be heavily influenced by it, if not driven directly by those developers. So yes, I believe that microservices and Java EE go hand in hand for a large percentage of developers. Approaches such as WildFly-Swarm offer precisely what I'm describing: a comfortable entry point for developers and even existing applications, yet the power to move to a more flexible DevOps driven paradigm. WildFly, when used correctly, offers a mature and easy to use platform that has a minimal footprint and faster boot times than the most popular web servers around! And don't forget that Swarm builds on WildFly so we immediately get the maturity of implementations(s) from it.
However, this mad rush towards microservices, trimming of application servers, creation of applications from fatjars etc. needs to be approached with caution. Our industry is renowned for offering panaceas to problems that require throwing away all we've done before and relearning all of those hard earned lesson! We've got to break that cycle and in Red Hat we've got the pieces to do so: with so many open source projects out there purporting to be right for enterprise applications and new ones springing into life almost on a daily basis, it's easy to understand why people believe every new project is good enough for their requirements. Although I believe open source is a superior development methodology, it takes time and effort to build enterprise-ready components such as transaction servers, messaging brokers, etc. They don't just spring into life ready formed and fully capable. "Good enough" is rarely sufficient for enterprises. It's the edge cases like management, reliability, recoverability, scalability and bullet proof security that are hard to do and get right, yet it's precisely those edge cases that matter time and again. Through core development or acquisition, we've built up a stack that is mature and capable. Whether deployed as a stack or individual pieces, it's what we should be building the next generation of middleware solutions upon. A strong base exists today and we need to reuse as much of that as possible rather than rewrite from scratch in some new popular programming language.
Now as I hinted above, maybe we don't package our future stack or platform in quite the same way as we do today. Microservices offers an approach that is in line with the kinds of trimming we've been seeing anyway. Bundling individual components from the application server as easily deployed (container based) services that can then be exposed to other programming languages, frameworks, solutions etc. is definitely part of the overall solution space. Those core services, such as transactions or storage, could be deployed as individual services or, as is more typical with something like Swarm, deployed with the business logic that uses them. I keep coming back to the JBossEverywhere initiative we had a few years ago - ahead of its time!
Ok so we've looked to microservices and how Java EE fits in. But that can't be the entire answer - and it isn't. As I've mentioned before, at least for a very important set of applications and use cases the future is reactive and event driven. Now that could mean Node.js but just as likely in the Java world it probably means Vert.x. Note, given that many of the Java EE APIs aren't reactive or asynchronous in nature then we'll need to evolve them if we wish to tie them in to Vert.x and I do think we need to do that. Whilst some people will want to develop their applications and microservices in Vert.x from scratch, others will want to tie in legacy systems or have access to some of the core services I mention earlier. I see Vert.x as the ideal backbone or glue that brings all of these things together. The mature core services that we've got are precisely the sorts of things that enterprise developers will need for their applications as they grow in complexity - and let's face it, eventually many applications are going to need security, transactions, high performance messaging etc.
In the Java world the unit of containerisation is essentially the JVM. However, most Java developers realise that unless you ship a farjar, which contains everything you need to run your application/service, it's often typical to find that changes in third-party jars downloaded at deployment time can result in the application or service failing to run first time. This is where operating system containers, such as Docker, really come into their own. The ability to create a deployment unit which runs first time and every time is crucial! Container orchestration technologies, such as Kubernetes, are likewise important if you want to deploy services (via Containers) which are highly available, load balanced etc. Therefore, hopefully it's not too difficult to see where Containers will fit into the future architecture - not mandatory by any means, but definitely a piece of the puzzle which should be considered from the outset.
The combination of OpenShift for Container deployment and management, with Fabric8 for developer experience with CI and CD, provides a compelling hybrid cloud environment, especially once you consider all of the JBoss/Fuse middleware integrated, i.e., xPaaS. As I've mentioned before though, xPaaS isn't about simply adding the middleware products to OpenShift; we're also going to make them much more cloud-aware/cloud-enabled. This has a number of implications, but the one I want to mention specifically is that the core capabilities will be made available to developers in a more cloud-natural manner, e.g., users who want reliable messaging won't need to understand the various intricacies of JMS to use A-MQ and in fact won't even have to know A-MQ is working under the covers. And yes, for those of you still paying attention, those core capabilities I mentioned are precisely the same core services we covered earlier on. See the connection?
Up to this point we've really been playing in the traditional enterprise deployment arena: clients, middle tiers and servers. The cloud comes into play here at the back tier (servers), but what about mobile, IoT and ubiquitous computing? I've discussed this a few times before so won't repeat much here except that I think everything we've discussed so far has immediate applicability for ubiquitous computing and that mobile, as well as IoT, are just limited aspects of it. In fact as I showed separately, if the cloud is to scale, mobile/IoT needs to take on a more fat-client approach - anyone remember what I wrote about Shannon's Limit over 4 years ago? Mobile, which really means developing applications for phones that tend to rely upon backend services, is a specific implementation of IoT, which really means developing applications for a range of devices that tend to rely upon backend services (ok, with some gateway technologies in there for good measure.) See what I mean?
If you follow that assertion that everything we need to do going forward is some aspect of ubiquitous computing, then it follows from what we discussed earlier that the new stack approach of core services, Containers, management etc. all come into play and across a variety of different languages and frameworks. Whether you're developing enterprise applications for mobile devices, clouds, involving sensors, or traditional mainframes, you need a stack that is mature, rich, scalable, reliable, trustworthy and open. The Red Hat stack, which has evolved over the last decade and is continuing to evolve, is the only one that matches all of the requirements!
Below is a hand-drawn outline of where I see these things going. Apologies that it's not a nice block diagram and for my handwriting
I'm just back from TechExchange 2015 which was held in Ho Chi Min City, Vietnam. I could wax lyrically about how much I enjoyed the conference (I gave a keynote and co-presented a session on WildFly-Swarm), the sessions and the people but I'm not going to - it was a Red Hat conference so of course I'm going to enjoy it and it's going to be great! No, this entry is about Vietnam. I loved it. The city, the people, the food, the culture were all so far beyond what I had expected it's really hard for me to put into words. Yes the roads were crazy with mopeds (on pavements), cars going in all directions down one way streets, people walking down the middle of roads, and even more stuff that surprised the $*£# out of me (last time I had that much adrenaline flowing through my system was on a rollercoaster!) but it was fascinating and fun at the same time!
I loved the vibrancy of the city. Despite it being so old it felt new. The buzz was great and the weather was ... Hot and humid with a bit of monsoon rain. Despite the fact I was there for only 5 days it felt longer and yet at the same time not long enough. I loved it. If I get the opportunity I will definitely go back!
Only a few years after the rise of public Cloud (essentially after a certain online bookshop offered its services to the public) I recall a number of people talking about how open source would no longer matter. The argument, which from 39000 feet made some sense, went something like this: people use open source because closed source equivalents are too expensive, often too difficult to use and generally just a way of accomplishing something "good enough" in a cheaper way. Because cloud offered capabilities, such as web server, messaging, storage etc. to developers on a pay-as-you-go basis, what did it matter if under the covers the implementations were based on closed source or open source? Therefore, people wouldn't approach developing solutions in the cloud by even considering the provenance of the underlying components.
Now of course even at that time you could argue that the argument didn't hold water at all levels. Specifically the operating system back then was predominately Linux based, which last time I checked was most definitely open source. In the intervening years we've seen Microsoft enter the space and whilst Linux is no longer the only operating system available in the clouds it's definitely still the dominant one. But that's not the only thing to change which drives a tank-sized hole in the original position. Today in all public or private clouds you don't have to look hard to find open source making it a reality. Whether it's OpenStack as the infrastructure, databases such as MySQL or MongoDB providing persistence, A-MQ or RabbitMQ for messaging, EAP/WildFly or GlassFish or Tomcat hosting some of the business logic, open source is integral to the success of public, private and hybrid cloud. Obviously there's a chance a developer may not know or even care that open source is responsible for the success of their application or business, but that doesn't negate the fact that cloud is successful today primarily because of open source.
There's also one important aspect that the original argument failed to take into account: users of open source are just one part of the overall value proposition; contributors/developers of open source are just as important. When you look at all of the examples of open source used in cloud I mentioned above, or go further and look at all of the others, you'll see that as well a individuals helping to develop these critical software pieces you've got pretty much all of the significant software vendors of our age as well as many of the most significant software users (verticals such as finance and healthcare, not to mention NASA).
Not exactly related to cloud, but when you get a company such as Microsoft making such an about turn on open source as they've done in the past year or so then it's probably a good bet that open source isn't dropping down the list of priorities for developers or companies. Therefore, to paraphrase and misquote Mark Twain, I think it's safe to say that reports of its death have been greatly exaggerated.
So I mentioned in a separate entry that I have problems with the term "web-scale". I get what the comment author is saying in that entry and I agree with several of the sentiments that are made but it did raise a few interesting thoughts in my head. Some of them I thought I'd addressed over the years but perhaps it is time to revisit. Or simply reference again in case your favourite search engine isn't working.
I've definitely addressed the issue of monolithic application servers in the past, whether through articles like this one or presentations. Monolithic application servers do exist, but there are application servers (Java and Java EE based) that simply don't match those characteristics. Whilst an application server may not be right for your use cases it's far better to rule it out based upon objective analysis than some (almost) subjective definition. I said a few months back that containers aren't all bad! Even as early as last year there were some good discussions about monoliths, microservices and balls of mud that would be well worth spending a few minutes reading and thinking about, because a knee-jerk reaction to a so-called monolithic architecture isn't necessarily going to give you the benefits you're after.
It's also interesting to think that the move we're seeing in the industry towards separate services is very much in line with what we had back in the CORBA days (something I've repeated a few times.) No, I'm not suggesting CORBA got it all right! Something else that was very much at the heart of CORBA was asynchronous invocations. This wasn't just something that was an add-on/bolt-on to synchronous APIs. Now in the transition from CORBA to Java EE we lost some of this work in favour of the more traditional (synchronous) RPC approach. If you've done your homework (or were around back then) you'll understand why RPC was (and still is) used so much in distributed systems: most of the languages we use(d) are procedural and it's natural to want to extend that paradigm to distributed programming and make remote invocations opaque. Now of course there are well know issues with this approach but that's not really the subject of this article.
The point I'm trying to make is that asynchronous programming isn't the domain of new languages and frameworks. Whilst I do agree that many of the APIs we have grown used to today are synchronous in nature and retro-fitting asynchronous approaches on top would be hard or impossible, it's worth remembering that there's often a good separation between interface and implementation in many mature implementations. For instance, some component implementations within various application servers date back to the CORBA days (or even further), and may have even been designed with asynchronous in mind. So whilst the APIs of some Java EE components may not easily be used asynchronously it's worth remembering that there is a large body of research and development which will help that rapidly evolving front of our industry build on much of what is there today, assuming of course that some implementations aren't there already. As I've said many times before, I hate reinventing the wheel.
OK it's Saturday night (my time) and I've spent the last week travelling across the Atlantic for meetings so have had a bit of time to think on a few things. One of them is how reactive platforms and microservices fit together. I took the time to write down some of what I've been thinking. I think frameworks such as Vert.x and Node.js have a lot to offer microservices and other use cases, but as Robert Frost once said we still have "miles to go before I sleep". That is good because it means there's a lot of exciting research and development still to be done, building on the work of others such as Fischer, Lynch and Paterson.
I've had a few conversations with different people recently on the subject of microservices and (distributed) transactions. One of my biggest worries about microservices in general is that we end up repeating the same conversations and disagreements that we had with SOA, REST and other related technologies. And one of the longest running debates we had during those specific waves was around transactions and their applicability, or not. I continue to believe that transactions are useful for some SOA-based services, though that doesn't mean they have to be atomic transactions. Therefore, I feel that they also have a role to play for microservices and took the opportunity to write down some of my thoughts on the topic.
In one of my previous entries I may have inadvertently given the impression that in order to do microservices you need Linux containers, such as Docker. I want to clarify that so have written a new article which hopefully clarifies where those containers matter, especially in the Java world where we already have a container in the form of the JVM.
After writing my previous entry of how a container image, such as Docker, makes a natural unit of failure for an individual microservice or groups of related services, I wanted to follow up on one of the things I didn't get into detail on: state. Where does the state of the microservice reside? Is it within the container image, which may seem intuitively correct, or is it outside of the image? I've been thinking about state and replication for many years (well over 20) and was revisiting one of my earlier papers recently and got to thinking that it was relevant to microservices, or at least containers (since of course a container doesn't need to be tied to only being useful for microservices!) With that in mind, I found a few hours today to write up some thoughts on state and containers in general, but specifically microservices, Hopefully it makes some sense.
Another mini-vacation so another few hours to sit and write down some things that have been rumbling around in my head for a while. This time it's about Docker (container) images, microservices and how they can be combined to provide a natural unit of failure when building multi-service applications or composite services. Check it out on my personal blog, but hopefully the conclusion should be obvious:
"If you are building multiple microservices, or using them from other groups and organisations, within your applications or composite service(s), then do some thinking about how they are related and if they should fail as a unit then pull them together into a single image."
It's a Saturday evening and I've finally had time to put down further thoughts on container-less development. OK, ignore the fact that I'm at home on a Saturday night instead of out "on the town" - but work is still my hobby (or is it vice versa?) and I'm enjoying it!
I've finally made time to write down some of my thoughts on microservices and how they should relate to SOA and other approaches which have gone before. I've cross-linked the article here for those who may be interested. As I mentioned in the InfoQ interview, I think some of the recent advances in development tools, such as docker-based containers, should make service development easier. However, if we'd had them 10 years ago they'd have fallen under the banner of SOA and I don't think we'd have coined a new term to describe a specific implementation. Whether it's SOA, microservices or some other term, I do think that in the last year or so there have been some interesting developments around frameworks, languages, containers etc. that do make a difference for distributed service-based applications. We just need to join them together with all of the good things we did over the past decade or so and called SOA.