Calling all integration experts!

Remember the old Universal Translator as modeled here by the late Mr. Spock? One of the first (or perhaps future?) examples of integration solutions, and certainly one of the most fondly rememberehttp://zagg-blog.s3.amazonaws.com/community/blog/wp-content/uploads/2012/03/12581.jpgd! But at its heart, it is also an almost perfect representation of the integration challenges today. Many years ago, there was EAI (Enterprise Application Integration) which was all about integrating homegrown applications with purchased package applications and/or alien applications brought in from Mergers and Acquisitions activity. The challenge was to find a way to make these applications from different planets communicate with one another to increase return on assets and provide a complete view of enterprise activity. EAI tools appeared from vendors such as TIBCO, SeeBeyond, IBM, Vitria, Progress Software, Software AG and webMethods to mention just a few.

Then there came the SOA initiative. By building computer systems with applications in the form of reusable chunks of business functionality (called services) the integration challenge could be met by enabling different applications to share common services.

Now the eternal wheel is turning once again, with the integration challenge clothed in yet another disguise. This time it is all about integrating systems with completely different usage a resource characteristics such as mobile devices, IoT components and traditional servers, but also applications of completely new types such as mobile apps and cloud-based SaaS solutions. In an echo of the past, lines of business are increasingly going out and buying cloud-based services to solve their immediate business needs, or paying a third-party developer to create the App they want, only to then turn to IT to get them to integrate the new solutions with the corporate systems of record.

Once again the vendors will respond to these user needs, probably extending and redeveloping their existing integration solutions or maybe adding new pieces where required. But as you look for potential partners to help you with this next wave of integration challenges, it is worth keeping in mind possibly the most important fact of all; a fact that has been evident throughout the decades of integration challenges to date. Every single time the integration challenge has surged to the top of the priority list, the key differentiator contributing to eventual success is not the smarts built into the tools and software / appliances on offer. Rather it is all about the advice and guidance you can get from people with extensive experience in integration challenges. Whether from vendors or service providers, these skills are absolutely essential. When it comes down to it, the technical challenges of integration are just the tip of the iceberg; all the real challenges are how you plan what you are going to do and how you work across disciplines and departments to ensure the solution is right for your company. You don’t have the time to learn this – find a partner who has spent years steeped in integration and listen to what they have to say!

Why enterprise mobile computing needs an mBroker – part 1

mobilephonesMobile computing is all the rage, with employees, consumers and customers all wanting to use their mobile devices to transact business. But how should an enterprise approach mobile computing without getting into a world of trouble? How can the enterprise future-proof itself so that as mobile enterprise access explodes the risks are mitigated?

mBrokers are emerging as the preferred method of building a sustainable, governable and effective enterprise mobile computing architecture. The mBroker brings together ESB, integration broker, service gateway, API management and mobile access technology to provide the glue necessary to bring the mobile and corporate worlds together effectively and efficiently; for a summary of mBroker functionality see this free Lustratus report. In this first post in a series looking at mBrokers, we will look at the fundamental drivers for the basic broking functionality offered by mBrokers.

Integration brokers have been around for many years now. The principle is that when trying to integrate different applications or computing environments, some form of ‘universal translator’ is needed. One application may expect data in one format while another expects a different format for example. A trivial example might be an intenrational application where some components expect mm/dd/yy while others want dd/mm/yy. The broker handles these transformation needs. But it plays another very important role apart from translating between different applications; it provides a logical separation between application components, so that requestors can request services and suppliers can supply services without either knowing anything about each other’s location/environment/technology. In order to achieve this, it provides other functionality such as intelligent routing to find the right service and execution location, once again without the service requestor having to know anything about it.

Enterprise mobile applications face a lot of the same challenges. When crossing from the mobile device end to the corporate business services end, the same problems must be addressed. For example, mobile applications often rely on JSON for format notation and use RESTful invocation mechanisms to drive services. But many corporate networks employ an SOA model based around XML data and SOAP-based invocations of services.  In addition, the same sort of abstraction layer offered by integration brokers is beneficial to avoid the mobile device needing to know about locations of back end applications. It is therefore not surprising to find that integration broker technology is one source for mBroker technology.

 

New Lustratus Research Report – A Competitive Review of SOA Appliances

Just a short note to say that I’ve uploaded a new report to our web store at lustratus.com.

The report, entitled A Competitive Review of SOA Appliances focuses on Intel’s SOA Expressway, IBM’s DataPower range and Layer7’s SecureSpan SOA Appliance. In the report I compare and contrast the technical and strategic approaches each vendor takes to addressing the task of creating, managing, accelerating and securing a service oriented architecture using appliances.

The report can be found here.

Steve Craggs

SOA / ESB confusion

confusionI recently commented on a query in another blog about ESBs, and what they are in relation to SOA.

Since this is a subject that I continue to get asked every now and then, even though I have blogged on it a number of times before, I thought I would reproduce the response I gave.

It starts right at the ESB beginning, and concludes with a few old Lustratus paper references that I believe are still relevant in introducing the ESB and SOA concepts.

When the Enterprise Service Bus concept started off life in the mid 90s it was as an extension of a messaging pipe – that is, message based communications. Prior to the ESB, messages were sent with tools such as IBM’s MQSeries (now WebSphereMQ), Progress Software’s Sonic MQ and a range of others, particularly JMS (Java Messaging Service) implementations. Users quickly realized that more was needed than just the ability to send a message from A to B – value add capabilities were needed such as data transformation, message enrichment and dynamic routing of messages based on content or other circumstances. This resulted in the emergence of ‘message brokers’ – pieces of code that acted as a ‘hub’, where any required actions could be taken on in-flight messages. This is where the ‘hub and spoke’ concept that was the basis of the EAI market came from. Messages from A to B would go via the hub so that the required intelligence could be applied, with the A and B endpoints requiring little intelligence at all.

However, two things happened that caused the ESB concepts to emerge around 1996-97. Standards activity in the integration marketplace increased and took root, and users wanted to find ways to lower the entry price for integration – having to buy a hub was very expensive, particularly when connections were few in the early stages of integration development. There were also fairly groundless concerns about availability with the hub and spoke model due to the perceived single point of failure. As a result, the ESB emerged.


The ESB did two important things – it leveraged major standards, in particular the web services standards offering a standard way to invoke the messaging capabilities of the bus, and it adopted a ‘bus’ rather than ‘hub and spoke’ architecture which resulted in a much lower deployment cost, at least in the early stages of integration development. The bus concept involves placing more intelligence at each node, so that messages can flow A to B without going through the hub. In flight message processing happens at the individual nodes rather than at the hub.


So, an ESB is actually just a ‘smart’ communications pipe, providing not just a way to transfer the messages from A to B but also the in-flight capabilities (often called mediation services) required. In addition, this is all available under a layering of standards. This is why typically ESBs are used with web services invocations, and often utilize JMS servers for the actual transfer mechanism.

SOA is something much bigger. It is a way of architecting your IT programs around a service-oriented concept. The absolute key to this is that an SOA service relates to a BUSINESS piece of functionality as opposed to some programming activity. So, you do not have an SOA service to read a record from a file, but rather an SOA service to ‘Get Customer Details’ which internally will end up reading customer information from files and so on. The secondary characteristic is that this service must be able to be invoked from anywhere, with no requirement to know where the target service will actually run. Therefore, it is clear to see that SOA requires some sort of communications capability, and while this does not have to be an ESB, the ESB fits the role very well particularly with its affinity to standards such as web services.

There are a number of free white papers at lustratus.com that discuss this topic in more detail, particularly ‘SOAs and ESBs’, ‘What is an SOA Service’ and ‘The Year of ESB-ability’.

Steve

Does Microsoft ESB Guidance have a future?

As one might have expected, Microsoft tried to ignore the Enterprise Service Bus (ESB) movement for a long time, but eventually it had to do something to answer demands of its customer base looking for SOA support.

Its response was Microsoft ESB Guidance, a package of

architectural guidance, patterns, practices, and a set of BizTalk Server and .NET components to simplify the development of an Enterprise Service Bus (ESB) on the Microsoft platform

Let’s be honest. This is a typical Microsoft ‘fudge’. Microsoft ESB Guidance is not a Supported Product, but is instead a set of guidelines and one or two components. It is a Microsoft Patterns and Practices offering – in other words, you are on your own. This may be fine if you are a Microsoft development shop, but far more worrying if you are real business user with extensive Microsoft presence. It has a lot of the disadvantages of Open Source, but you still have to pay for Bizt Talk etc..

So what does the future hold? Will trying to bring the Microsoft server world into the SOA domain always be a matter of risk and going it alone? Will Microsoft productize Microsoft ESB Guidance? Are there any alternatives other than just consigning the Microsoft platform to run in isolation on the fringes of the enterprise?

Fortunately, the Microsoft model may actually be working here. I do not believe Microsoft will ever productize ESB Guidance – after all, they have had two years and are still maintaining there are no plans to do this. However, what this position does do is it encourages opportunists to jump in and develop products based around the Microsoft technology and guidance materials. An example is Neuron-ESB, from Microsoft specialists Neudesic.

So, while Lustratus strongly cautions users about the effort, cost and risk of using Microsoft’s own ESB Guidance package, the idea of utilizing a Microsoft-based supported ESB product from a specialist vendor is much more attractive. Of course, whether these new Microsoft-based ESBs are any good is yet to be seen….

Steve

Ultramatics works with IBM to defuse SOA security threat

Ultramatics has just announced SOA SafeGuard product, which is designed to shut one of the major SOA security holes – the opportunity to inject virus and other malware threats through XML file sharing.

This is good news for SOA implementers, but also introduces an interesting new stress point for IBM. Back in 2007 I was on a podcast where I identified the five SOA security traps, one of which was the XML problem. To summarize, most virus and other threat detection solutions look at the datastreams coming into the system and identify threat signatures that indicate the presence of some noxious code, but unfortunately they cannot see inside the XML wrapper, so to all intents and purposes the contents of any attached XML file are invisible. This offers the opportunity for malicious agencies to pop in some nasty code into the XML content and smuggle it through the security gates to the enterprise. Of course, it is not immediately obvious how this would help, in that getting this code executed might not be so easy, but hackers are smart….therefore it is best to close this exposure.

One way to close the window is simply to forbid any XML file sharing, but since industries such as healthcare now more or less rely on this to conform to industry standards and regulations, this is not really practical. The new Ultramatics product claims to be able to protect from these types of intruders. It runs on the IBM DataPower XI50 Integration Appliance, providing a hardware-based shield that can see into the XML files and weed out anything unpleasant. This solution will be very valuable to many SOA companies worried about security.

But there is something else interesting in the product details. The datasheet for the product says it can be used (in conjunction with IBM’s MQSeries) to:

Create a SOA ESB that can perform routing, transformation and protocol mediation functions

This is intriguing. Of course, the idea of an ESB appliance is not new, but the interesting point is that IBM is supplying this capability through the Ultramatics product…..I wonder if the other IBM ESBs, WebSphere ESB and WebSphere Message Broker, see this is encroachment?

Steve

Vendors like to back standards – as long is it is in their interests!

I was reading Danny Goodall’s post on his strategic marketing blog about standards-based marketing…

….and it brilliantly illustrated a point that I think is often experienced in the software marketplace – some vendors rush to back standards and push them, but only to the point that they fit with their own goals.

The example Danny discusses is Sonic Software, part of software vendor Progress. Sonic is well known as the first software vendor to use the ESB acronym (Enterprise Service Bus), and did indeed peddle the standards message hard asDanny, the marketing guru behind Sonic’s early success, remembers:

All the while I was creating marketing programs that stressed Sonic’s commitment to standards and, by implication, I was de-positioning other vendors’ technologies as being the Devil’s spawn due to their reliance on proprietary features. “How,” we asked “would organisations ensure interoperability between their, and their trading partners’ infrastructures if they didn’t conform to the emerging standards?

Of course the standards message is very attractive to users. Buyers are keen to be able to ensure that not only can components interoperate without a lot of extra work, but also that vendor lock-in is weakened through the ability to substitute components from different suppliers, bringing prices down and reducing risk. Therefore, vendors that preach standards may come across initially as ‘good guys’. However, it pays to look more closely to find out how serious the vendor REALLY is about standards. In the Sonic case, while it talked a great story, the mystery was that its own ESB product was unable to run over any standard JMS-based messaging pipe for years. Instead, it used a proprietary interface that ensured Sonic ESB would only work over SonicMQ, the Sonic messaging pipe. This was a real problem for many prospects, because IBM’s WebSphereMQowns around 80% of the messaging pipe business and therefore prospects interested in an ESB were frequently looking to run it over their existing software. This restriction was arguably one of the key reasons Sonic lost its leadership position in the ESB market.

So why did Sonic take this line? Obviously, only Sonic knows, but a cynic would argue that it consistently refused to support the JMS standard in the early years to ensure that it could force the sale of its own messaging pipe. No matter that this meant the user often had to buy another one on top of the incumbent solution.

I am not picking on Sonic here – this is only one of many examples where vendors claim to be standards-based while not shrinking from proprietary solutions when in their own interests. And of course, it is entirely understandable – after all, software vendors are businesses too. To me, the important thing is that users keep away from the rose-colored spectacles. Standards are valuable, and vendors do provide important support, but there will always be compromises.

Steve

Will Swordfish make its point?

The ECLIPSE organization has finally made its announcement of the first release ofSwordfish, the open source ESB (Enterprise Service Bus) framework.

A lot of the work for Swordfish has come from Sopera, a German open source company that has developed an offering around the DeutschePost service bus development. Sopera offers a valid and competent framework for service integration, and therefore it is assumed that Swordfish might also work.

So, will Swordfish make a successful strike at the ESB market? So far, open source ESB projects have not had a great deal of success, and as far as 2009 goes Lustratus has forecast that open source projects will suffer due to the lack of the necessary people resources to turn open source frameworks into a useful user implementation. However, Swordfish has the backing of the influential ECLIPSE organization, which has done a lot to standardize the look and feel of many software infrastructure tools.

Looking at the initial marketing thrust for Swordfish, things don’t look to good. From the announcement letter, the top functional bullet reads

Support for distributed deployment, which results in more scalable and reliable application deployments by removing a central coordinating server.

Well – duh! This is not new – it is part of the basic definition of what an ESB does! However, this initiative is still worth watching, despite the ill-fated marketing attempts so far. ECLIPSE has significant industry backing for its GUI look-and-feel stuff, and indeed most of the big industry names like IBM, Oracle and SAP are involved in the running of ECLIPSE, and provide a lot of the financial backing.

It is this that might be the source of most excitement with Swordfish. Oracle and IBM both actively market and sell their own ESBs, and SAP offers its own equivalent functionality as part of its NetWeaver set of offerings. I wonder how they feel about ECLIPSE driving an open-source ESB version that competes on functionality and is free? I would love to be a fly on the wall in internal ECLIPSE meetings about the future of Swordfish.

Steve

Is the time right for Progress Software to be bought?

In the course of my ongoing analysis of software infrastructure vendors I was intrigued by the recent earnings release from Progress Software…

…and it caused me to dig a bit deeper. Basically, Progress is holding its revenue stream although not growing it, and I guess in today’s environment that is OK. But when the performance of the company over the last few years is considered, a different picture starts to build up.

Basically, Progress made a lot of money from its OpenEdge database product, and this business is still providing a rich ‘cash-cow’ revenue stream. However, not only has this stagnated but it is starting to decay, with Q109 showing a sharp drop. Admittedly this is probably in part due to currency movements, but the trend is clear – this is not a growing business ans the writing is on the wall, at least in the longer term. Progress knows this, and so over the past few years it has been on the acquisition trail, trying desperately to find a new business that can grow sufficiently to become the new OpenEdge. It has tried the area of Data, with its DataDirect division growing through acquisition, but this business has reached a steady state with little or no growth. It tried the area of messaging, being the company that brought the term ESB (Enterprise Service Bus) to the world through its SONIC line of business, but having got a great mindshare and market position it lost focus and this business is now fatally damaged, with others such as IBM, Oracle andMicrosoft taking up the mantle. Recently it acquired the APAMA complex event processing business, Actional (SOA management) and IONA (a datedintegration business based in Ireland). It has since found some success with the excellent APAMA offering in the heartland of financial market data processing, but has struggled to replicate this success in other industries and use cases. Actional has also had some success but it is immutably tied to the SOA star which is having its own problems. And IONA, similarly to Progress, has a nice legacy integration business based around Orbix but has failed utterly over the years to create anything else worthwhile.

The result is that although the IONA purchase has increased revenues in the Progress ‘integration infrastructure’ business unit, this is likely to be a one-off improvement and once again Progress is going to be stuck with an aging cash-cow and no clear rising star to take over responsibility for driving growth.

This might seem a recipe for Progress itself to be acquired. Up to now, this has been unattractive due to the share price, but in thecurrent climate the acquisition looks a lot more interesting. My view is that there are probably two strong candidate acquirers for Progress:

  • Companies looking for attractive maintenance businesses where profit can be maximized by cutting expenses and taking the money until the product line sunsets
  • Companies not currently in the integration space but wanting to get into this lucrative area and looking for a ready-made product set (perhaps to underpin a professional services business)

Who knows what will happen in the current turmoil? I may be way off the mark, but if I was a company fitting either of these two categories, and I had the money, I think now would be a good time to strike. After so many false dawns, I suspect the Progress management team might not resist too hard….

Steve

    Don’t be afraid to ask for SOA help

    While the number of SOA success stories grow, there are a lot of companies that are finding SOA a struggle.

    As often happens when something gets heavily hyped, managers are almost embarrassed to admit that they are having trouble. But the truth is that for many, getting outside help may be the best way forward and end up giving great returns.

    There seem to be three common SOA ‘failure’ scenarios.

    • This SOA-based project is blowing its schedule/budget/SLAs
    • We are diligently implementing SOA, but we just aren’t getting the returns we expected
    • Everybody agreed SOA was a great idea, but now nothing is actually happening

    It is easy to feel that these scenarios must reflect badly on management or technical efforts, since other companies seem to have succeeded with no problems. But in fact, it is perfectly natural to find SOA difficult. In essence, SOA is REALLY different – it is a different way of working, the tools are different, programming is different, design is different…..and so on. However, an important corollary of the success of SOA in other companies is that there is a growing pool of knowledge around SOA procedures and best practices. Already, there are some professional services organizations that have embraced all this accumulated knowledge and developed service offerings specifically designed to unblock the SOA logjam – getting projects moving again, finding why the SOA strategy is not delivering, and clearing up any organizational or procedural blockages.

    Companies should not feel bad about asking for help. It really can be worth it, even if there is an initial investment hit. And fortunately, once IT and business professionals get the hang of SOA, they wont need the outside help, so the cost hit does not have to be an ongoing one. The key is to make sure companies choose the right partner. This is a subject that is discussed more in a recent Lustratus Report, “A little help goes a long way”, that can be downloaded for free from the Lustratus web store.

    Steve