Jan Stafford, Executive Editor
In a 2014 survey, readers of SearchCloudApplications and its sister sites identified application and data integration as the biggest challenges in software-as-a-service adoption. Industry researchers agree. Gartner analyst Eric Knipp calls app integration the No. 1 barrier to cloud deployment success, and 451 Research ranks it as the second-most-acute cloud pain point, beaten only by security.
Integration pain isn’t a cloud adoption deal breaker for most companies, though. Otherwise, the cloud applications market wouldn’t be so huge—$30 billion in cloud application subscription reviews in 2014—or growing so quickly, to the tune of $67 billion in projected subscriptions in 2018, according to IT researcher Apps Run the World.
Competitive pressure spurs that growth, and integration issues have to be handled. A starting point could be answering the nine questions on cloud application integration in this guide’s first article, by Valerie Silverthorne. Each query comes with an expert how-to or why-to discussion.
An overall cloud app integration strategy should include a unifying plan for adoption projects, writes consultant Tom Nolle in his tip on integrating applications across cloud boundaries. Without a step-by-step integration approach, each cloud app adoption project has to start from scratch. Nolle advises organizations to codify best practices for each application integration component or layer.
Making applications play nicely together is only the means to cloud application integration’s goal, enabling workflow. “The purpose of cloud application integration is to connect workflows through components,” writes Nolle in this guide’s third article. Beware of tool vendors that want to tie integration projects to their cloud services and not the project’s targeted workflow goals, he advised.
Planning provides the best relief from chronic integration pain. If your organization is feeling an ache we haven’t addressed in these pages, our experts can provide suggestions for remedies.
Find out more about application integration and SaaS adoption in our e-handbook.
Jan Stafford, Executive Editor
2015 is—wait for it—the year of platform as a service! Of course, pundits have already given PaaS the keys. But 2015 is when they work. This is largely because Amazon released AWS Lambda PaaS in January, cementing the notion that PaaS will drive growth for infrastructure-as-a-service vendors. Also this year, more new PaaS offerings will be tailored for the Internet of Things, mobile devices and other hot technology areas.
Essentially, a PaaS system provides an integrated cloud hardware and software platform and the tools used for software development and delivery. PaaS breaks the ice for those new to IaaS. Most software architects and developers enter projects via PaaS or a PaaS-like environment and aren’t as familiar to the infrastructure, said Kris Bliesner, CEO of consultancy 2ndWatch, an AWS partner. Overall, Amazon’s potential customers need both infrastructure and a platform, he said.
Building IoT applications has been difficult in the past because few development tools could facilitate machine-to-machine communication. In 2013, this type of open source tool emerged and got the ball rolling. New cloud platform offerings embrace IoT, and there’s a laundry list of features that these PaaS IoT options must have to be viable.
IoT presents a strong use case for investing in PaaS, but it’s only one in a crowd. Businesses can use PaaS for many types of application and cloud integration projects, and future use cases abound. We kick off this handbook by exploring PaaS uses, current and future.
So here’s your passkey to the Year of PaaS.
“Once burnt, twice shy” is a cliché that rings true when software development teams fail to build what their target user really wants. “Too many developers and companies know that pain of a user dropping off and never returning because the value was not there,” said Damion Heredia, IBM vice-president for Cloud Platform Services Product Management. Unfortunately, eliciting users’ true requirements often works against delivering applications as quickly as users demand. Speeding up the requirements phase of development with automated “personalization” tools is one approach to making both practices compatible.
Heredia talks about how developers can use the IBM Watson Personality Insights service in this interview. Personality Insights is available on Bluemix, IBM’s Cloud Foundry-based open cloud development platform. The IBM Personality Insights (PI) cloud service uses linguistic analytics to determine potential software users’ needs, usage patterns and other information.
What are some ways that developers try to speed through the user requirements phase of content, service and software creation?
Heredia: The kitchen sink approach comes to mind; just throwing in lots of features to give the impression of value. On the user end, there’s a frustration point at which they’re getting a lot of content served up to them that’s not relevant, that turns them off very quickly to the product.
How would software developers use Watson Personality Insights?
Heredia: We’re making available for developers an API that can help them access personality data about their users. There’s a lot of documentation on the API and also a sandbox environment that is completely free to use.
So, when building recommendation engines around a particular set of data, a developer can embed our service into their application via the API. They’ll take that data, and they’ll actually map it to the behavioral data that they are collecting through their app. For example, they could see a certain demographic in a geographical location and tie that data to personality data.
Could you give an example of how that user data is gathered?
Heredia: The information is gathered through the Watson visual personality assessment tool. For example, say, we show a user an image with a caption associated with it, and the user has the choice between saying ‘me’ or ‘not me’ to that image, based upon their reaction to it. Behind the scenes, Watson is scoring that user’s responses as related to personality traits. We measure on over 100 personality traits, and then return to the developers the score for that user on each of those 100 plus traits. They can use that for recommendations, personalization, targeted marketing. There are other requirements aspects to explore, of course, but PI gives a view of the user that previously was unavailable so quickly.
How much impact has the rising number of mobile applications user had on the demand for tailored features?
Heredia: It’s been a huge factor, because mobile devices have become the predominant way that people access the Internet, content and applications. Mobile has disrupted the traditional text-based delivery system because mobile is a predominately visual medium. Even this (IBM PI) service is an example, because we had to design it to be used on mobile devices. Try to do a traditional personality assessment, like the Meyers-Briggs survey, on your smart phone. Sure, you could, but you would be pulling your hair out, it would be so frustrating. So, we had to find a way for the developer to collect data in a way that reduces that friction, and makes it fun and enjoyable for the user so everyone wins in the end.
In 2014, news about developing cloud applications using IBM Watson’s cognitive capabilities and mobile-first strategies were top traffic-getters on SearchCloudApplications. An article about developing Internet of Things (IoT) apps and a piece about enterprise architects’ cloud projects were runners up.
Looking at why these four stories attracted over 12,000 readers, I talked with our expert contributor, George Lawton, who wrote all of them. Together, we offered some footnotes to each story.
The year’s top story was “IBM’s Watson supercomputer gives developers access to cognitive cloud.” In it, Lawton describes how software developers are using IBM Watson to build customer-focused cloud-based applications. Essentially, Watson provides the ability to analyze data about customer behavior and suggest — or, even, predict — future behaviors. For example, Watson technology has been used by Fluid Inc.’s retail business customers to increase customer engagement and sales. Also, Welltok Inc. is building a Watson app that provides regular wellness activity guides and rewards for users’ healthy behavior.
IBM’s Watson Group is putting $100 million into fostering work by third-party software developers, one of those being Welltok, according an article by reporter Stephanie Neil about cognitive computing. Talking about that investment, Lawton told me that setting up a new Watson Group headquarters spurred “the single largest movement of IBM research personnel in its history,” he said. He predicts spurt of Watson-based product releases in 2015, particularly in vertical markets such as financial services, retail, healthcare and travel.
Mobile-first report takes second
High interest in SearchCloudApplications’ feature on mobile-first development of enterprise apps shows that “mobile is now a premiere consideration in the rollout of new services,” Lawton told me. Mobile-first design of applications is now a must, because people don’t have to use a computer to access information and make transactions.
The mobile-first trend has far-reaching impact. “It’s helping to redefine the traditional notion of service-oriented architectures to embrace a more flexible infrastructure based on much simpler APIs,” Lawton said. At the same time, the gap between native and hybrid development environments is starting to shrink, which promises to lower the bar in keeping up with the proliferation of iPhone, Android and now Windows devices.
Developers, take note of IoT
The intersection of cloud computing and the Internet of Things (IoT) was a buzzy topic at the 2014 Silicon Valley EclipseCon conference. Developers there posited that the cloud could enable quick tests on new IoT projects, according to SearchCloudApplications’ third most popular news story, which covered the IoT cloud discussion at EclipseCon. Cloud testing will speed development and innovation in IoT projects by reducing the number of failed projects traditionally associated with early-adopter rollouts. “Leveraging the cloud to quickly test out new ideas will allow businesses to weed out bad ideas with less risk,” Lawton said.
Watch SearchCloudApps in 2015 for regular, in-depth coverage of cloud-based IoT development. Meanwhile, catch up on IoT with articles on design issues in mobile and IoT development and how IoT technologies are impacting product development.
An enterprise architect’s cloud apps success
Everyone likes to learn by example, so it’s no surprise that a first-person account of cloud development projects was fourth on our news traffic tally. Exploring enterprise architects’ first cloud projects, George Lawton tapped the experiences of veteran software pros. Check it out to learn about how cloud helped in landing NASA’s Curiosity Rover and onboarding two companies’ sales teams to mobile and cloud apps.
If first-person articles appeal to you, you’ll find a bunch of them in our Change Agents profiles of software developers and architects.
That’s the news round-up for 2014. I can’t wait to see what tickles readers’ fancy in 2015.
Happy New Year!
Based on a list of the most popular tweets by SearchCloudApplications.com this year, twitter users were interested in a variety of cloud application topics, from integration to mobile backend as a service (MBaaS) to automation. Here are five of the most clicked on tweets and the articles they referenced.
The above tweet linked to a feature on how mobile developers view building and buying backends. The tweet was the most popular of 2014, and the link grossed 179 clicks. The quote in this tweet reflects the opinion that investing in a backend service is more cost-effective than building an application’s back end. Companies in certain industries, however, may want to avoid MBaaS for compliance reasons. Check out the story for more specifics from developers who have carefully considered if MBaaS is right for their companies.
When integration platform as a service (iPaaS) began to pop up in the cloud app world, there was some confusion over what the service provided customers. Some of that confusion remains because iPaaS is still nascent. When SearchCloudApplications.com shared our definition of the term, different tweets with a link to the article where clicked on several times and retweeted a handful of times. The definition is a good starting point for those who are unfamiliar with iPaaS, as well as developers in need of a refresher.
Automation is a tricky but appealing part of IT. For companies that are able to automate some of their processes, automation can save time and money. SearchCloudApplications.com’s third most clicked on tweet of 2014 was about how cloud computing company Strategic SaaS adopted automated migration software to upgrade the company’s operation, a good case study for other companies considering automation.
Integration appears for the second time on this list with the above tweet, which was the fourth most popular tweet by SearchCloudApplications.com this year. The tip linked to in this tweet provided readers with information on some common and useful tools for cloud app integration. Covering iPaaS and software as a service vendors, this tip reviews some new products that readers might find useful to their companies.
MBaaS joins integration as the reoccurring topics in the top five tweets of the year. This feature highlighted in the tweet above examined an MBaaS success story. It’s another good read for mobile developers, particularly those who are struggling with handling the back end of their applications.
Follow us on Twitter @SearchCloudApps if you’re interested in receiving tweets like those above on a regular basis.
Integration is one of the key issues that arises for companies considering or using the cloud. Cloud app integration is central to cloud computing because it enables different programs to share data, but it is difficult to do well.
Executive site editor Jan Stafford recently wrote about the mistakes made and strategy needed for dealing with cloud integration. Stafford’s blog post focuses on the highlights from interviews with two Gartner analysts, emphasizing the need for enterprise architects to be well-versed on integration.
If that’s you or you’re looking for more information on application integration and cloud integration, SearchCloudApplications.com has published several articles about integration in the last couple of months. Recent tips and features provide information on integration basics, tools and more.
Part of the difficulty of integration is that the need for it is so pervasive in the cloud world. Integration takes place in the following use cases: integration platform as a service (iPaaS) providers, cloud to cloud, cloud to integrator to cloud, or within a hybrid cloud.
IPaaS, one of the solutions to integration issues, also makes things more complicated. New iPaaS and other integration providers emerge every quarter. With so many integration tools to choose from, IT decision-makers need to invest time to assess the market. Recent tool reviews on SearchCloudApplications.com might be a good starting point for your research.
Integration is hard, but it isn’t impossible. If you’ve been running into issue after issue, don’t despair. Remember that other companies have had success with integration. Sometimes the best advice isn’t from an expert pontificating on a subject but from IT professionals who have toiled with the technology that makes integration possible.
For those who are doing a deep dive on the subject, check out this essential guide on cloud app integration with even more information on basics, tools, strategy and iPaaS. Once you’re done, check out our quiz on cloud integration solutions, and test your knowledge on the subject.
Today’s hybrid clouds don’t live up to the name, in Gartner analyst Eric Knipp’s opinion. Without application portability, hybrid clouds are just a collection of remote and in-house applications that have been tailored for each environment and loosely integrated.
“When I think of hybrid cloud, I’m thinking of a place where I can write an application and then deploy it to any hosting location without changing the application at all,” said Knipp, Gartner managing VP. In a true hybrid cloud, workloads can be easily moved back and forth between private and public clouds to facilitate deployment and efficient scaling. Using either policy or some flick of a switch, a developer could make that same application scale to many other places, which may span private, public or even multiple public providers.
Deploying a single application across many environments with no real limitations is a capability that doesn’t exist today, said Knipp, who is co-author of Gartner’s 2015 Planning Guide for Cloud Computing. “To get there, we have to be architecting portable cloud applications in a way that dimensionally can live up to this hybrid cloud fantasy that we have today,” he explained. “I think it’s going to exist, because I see some building block technologies that are maturing.”
Using a platform as a service (PaaS) framework, like Red Hat OpenShift or Cloud Foundry, is a first step toward such “real” hybrid clouds. PaaS greatly increases application developers’ productivity by outsourcing the “plumbing” required for software development. PaaS provides infrastructure provisioning, development tools and cloud testing so that developers can focus on building the business logic and other important features.
Some say that Docker is the best basic building block for the hybrid cloud, but Knipp doesn’t. Docker is an open source container program in which an application and its dependencies can be bundled. A container is where a program building block, or component, can be run. “Docker is a useful enabling technology, but doesn’t solve most portability challenges,” Knipp said. All Docker enables is embedding software into the Docker image that can be shared. “It doesn’t help you with the external dependencies, and that’s where the developer’s problem really lies.” Even before Docker, Knipp said, it wasn’t that hard to build an application, even as a .jar, that could be deployed across many different instances of Tomcat.
“Backing services is what was hard to solve before Docker and still remains hard to solve after Docker,” Knipp explained. All the components involved — the database, the message queues, the in-memory data grids, the analytics service, and so on – add tremendous complexity. It’s not just what’s in that application that’s difficult to handle, it’s all the backend services, Knipp said. PaaS helps by providing and automating many of those services.
Portability has to be the key non-functional requirement consideration for any new application that’s being built in the cloud. “Just because I can take my application and drop it someplace else doesn’t actually make it portable,” Knipp concluded. “For me, hybrid cloud will only be achieved when we have a much higher degree of application portability.”
Twenty-one percent of respondents to a survey by Gigaom Research said they use the cloud to develop enterprise applications. The technology research company recently published the results of three different surveys in a report by analyst George Crump called, “Shadow IT: data protection and cloud security.”
The low percentage of developers creating applications in the cloud may be due to some of the inherent issues of using cloud. The surveys polled software-application developers and other IT workers and decision makers on a variety of cloud practices and issues, including what their top cloud concerns are. Security, performance, learning cloud-related skills, cost and support were the highest ranked, with security well in the lead.
Incidentally, employees may pose the greatest security risk to cloud adopters. Employees commit more than 70% of unauthorized data access, whether accidentally or intentionally. Employees also knowingly use unauthorized cloud applications. Eighty-one percent said they use SaaS applications without IT approval.
Another key finding of the report was that companies use many cloud instances. Of the companies surveyed, 32% said they have more than 50 cloud machine instances in production. An additional 30% use one to ten clouds, and 23% use 11 to 50.
A recent report by Forrester Consulting confirmed this multi-cloud reality. About 83% of respondents struggle with compiling their disparate cloud services, according to the report, which was commissioned by Quincy, Mass.-based software company Infosys.
The report, “Simplify and innovate the way you consume cloud,” also highlighted user experience as a hurdle to working in the cloud. Between 60% and 70% of survey respondents said they are concerned/very concern about the following: the complexity of managing and governing hybrid cloud, that application performance metrics such as availability and speed will be negatively affected in the cloud, and that integration with other applications will be difficult.
When enterprises adopt cloud computing, many of their legacy methods of software integration are instantly obsolete. Hanging on to old integration methods is like trying to fit square pegs into round holes, according to Eric Knipp, Gartner Inc. managing VP. Integration is the biggest barrier to cloud deployment success and the primary driver of change and new opportunities in developers and enterprise architects’ jobs, he said.
Application integration has been a pain point for organizations a long time, Knipp said. Initial efforts focused on point-to-point integration, which usually resulted in fragile integration points that, when multiplied, were not robust. The integrations would break during software upgrades, causing long release cycles. Over time, some, but not all, enterprises remediated integrations using integration middleware, including enterprise services buses (ESBs) and APIs.
“Yet, when organizations adopt SaaS, I see the same point-to-point integrations happening,” Knipp said. “Haven’t we been here before? Don’t we know how this movie ends? It’s not good.”
Today, cloud integration must be a core competency in the utility belt of the modern application developer in an enterprise. “When you’re talking about the mainstream enterprise that is not in the business of writing software unless they have no choice, integration has to absolutely be the first thing that you focus on,” Knipp said.
Gartner Research Director Kyle Hilgendorf agreed. “I like to refer to integration as kind of a four-headed monster,” he said. “You’ve got network integration, data integration, identity integration and then application or services integration.” Knipp and Hilgendorf are co-authors, along with four other analysts, of Gartner’s recent 2015 Planning Guide for Cloud Computing.
To handle all four types of integration well, enterprise architects must ensure that those integration architectures are “pre-plumbed or pre-connected” prior to adoption of public cloud, Hilgendorf said.
In advance of adopting cloud, build a cloud-friendly enterprise architecture and integration strategy, Hilgendorf advised. “Think about how to bridge your networks between your data center and one or more cloud providers,” he said. “Think about how you will do single sign-on and identity and access management federation or synchronization.” Consider how to synchronize the business’ identities for authentication and authorization requests. “All that work and much more has to be done up front by the enterprise architect and architecture team,” he said.
The integration challenges of cloud adoption alone give architects and developers a once in a lifetime opportunity to retool their skillsets for a long-term, successful career, according to both analysts. With the right skills, they’ll be valued leaders as businesses transition from traditional application architectures, deployment methodologies and sourcing arrangements.
“It is more critical than ever for integration to be a core competency in the utility belt of the modern application developer in an enterprise,” Knipp said.
Denodo Express, a free data virtualization tool with a graphical user interface-based studio, is the latest product offered by Denodo Technologies, Inc. Denodo Express connects with and integrates on-premises, in the cloud, structured, unstructured and big data sources. These sources are then delivered and available to end users, as well as enterprise applications, dashboards, portals, Intranet, search and other tools.
“Denodo Express is designed for those data architects that are tired of being a prisoner to archaic data integration methods or are just generally frustrated at not being able to leverage the true value of their data,” said Suresh Chandrasekaran, senior VP of Denodo.
Denodo Express cuts down on the wait time developers often face when they want to integrate data. Along with integration of disparate sources, Denodo Express also performs abstraction; query optimization; caching; extract, transform and load batch scheduling; and data services publishing.
With this new product, Denodo, which is based in Palo Alto, Calif., seeks to eliminate the need for developers to search for and interpret data on their own. Instead, “Denodo Express [delivers it] to them in the format that they prefer, be that [structured query language] for [business intelligence] and reporting tools, Web services for Web and mobile applications, Web parts for SharePoint integration, etc.” said Chandrasekaran.
Denodo Express is free and available for download on the company’s website. The download gives users access to the product for an annual, renewable term. There are no time limits on using Denodo Express; however, the product is best used for smaller projects, such as personal or departmental projects. Once customers want to use Denodo’s data virtualization on an enterprise level, it is recommended that they upgrade to the Denodo Platform, which is not free. Customers interested in the expanded service should contact Denodo for pricing information.
Customers also have access to free educational support when they download Denodo Express. There are tutorials and videos from Denodo experts, as well as online community-based support. Further training courses are fee-based and available online and offline.
The product was released on September 29, 2014.