That might be changing.  This week, multiple software companies introduced tools aimed at easier management, scalability and integration of Hadoop within the enterprise.

Pervasive Software Inc. rolled out Pervasive Data Integrator v10 – Hadoop Edition, designed to enable users to more easily transfer business data to and from Hadoop-based big data stores. The product addresses the challenging task of flowing non-Hadoop data into Hadoop, a framework often noted for its lack of visual data integration tooling.

Similarly, MapR Technologies Inc. released Version 2.0 of the MapR Distribution, a tool that provides increased monitoring, management, isolation and security for Hadoop. Some of its more notable features include central logging and multi-tenancy support, which both aim to enhance control and visibility of Hadoop data.

Also this week, cloud testing took the fore.

SOASTA announced its CloudTest Mobile platform, providing test automation for iOS, including iOS 6. CloudTest resides inside the app, an approach designed to allow developers to test apps as new mobile operating systems are released.

Also on the testing front, at Cloud Expo in New York, Parasoft Corp. showcased the latest release of Parasoft Virtualize, a service virtualization tool. The product aims to allow development and QA teams easier access to environments needed to test applications.

On another note, UShareSoft announced UForge App Store at the same expo. The platform is a collaborative, multi-cloud app store for enterprise IT applications. With its release, UShareSoft hopes to ease the delivery of software and application to the on-premise cloud. The spread of so-called app stores may betoken a trend of  ‘consumerization’ of IT, as updates are shared using now-familiar Smart phone methods. -Stephanie Mann

The cloud by itself is scalable, says Sven Hammar, CEO for Apica. But applications can run very differently on different cloud vendors’ platforms. And some of the same issues that tracked application hosting providers can surface in certain cloud settings. Take for example the ‘noisy neighbor.’  Cloud is meant to handle spiky phenomena such as Super Bowl marketing campaigns. But if your apps are co-located near that suddenly surging campaign, it can be a problem for your app’s performance.

”Sometimes you have a noisy neighbor. Adequate online performance monitoring can detect if you are located next to, for example, a busy campaign site,” said Hammar.  The questions that need to be answered are not unfamiliar to veterans. How many concurrent users can I handle? Where are my bottlenecks? What happens when you update the cache?

Do you trust outside software developers more than in-house software developers? Such may often be the case according to a recent survey conducted by Forrester Consulting on behalf of software tester Coverity. The news is a bit unsettling.

The Forrester Software Integrity Risk Report found that only 44 percent of companies conduct automated code testing during development for third-party code, compared with 69 percent that use automated code testing for internally developed software.  Over 300 software development influencers took part in the study. The results may have ramifications for Web services testing as well, as third-party services gain wider use.

”The biggest surprise was how much less rigor there is there,” said David Peterson, CMO at Coverity, commenting on the study. Peterson reminds us that it was only a few years ago that mere use of open source was a matter of some controversy. But, due to a slew of new mobile development projects and a general push to move quickly, some quality concerns may have been brushed aside for third-party and open source code.

“There is a big push to move faster. There is pressure from the business side,” said Peterson. The rapid adoption of third-party code, especially open source code, has outpaced testing in some places,” he suggested.

Cloud computing has increasingly been about Web applications, but that move may be slowed more than a bit as enterprises digest the damage of Amazon’s recent multi-day cloud snafu. At this writing the cause and effects are not clear.

An early Amazon alert said a networking event triggered a large amount of re-mirroring of Elastic Block Storage volumes at a data center. This mirroring created a shortage of capacity in a local availability zone that impacted Amazon’s ability to recover EBS volumes. Can you say ”cascade?”

Web application upstarts such as Foursquare, Quora and Big Door were hit hard. But cloud poster child Netflix sailed on. In a New York Times article, Netflix spokesman Steve Swasey said the online movie service had dodged the Amazon black out because it fully used Amazon cloud redundancy features.

So the bullet seems to have been dodgeable. Of course, Netflix is rich enough to afford the premium redundancy level. Moreover, it has a deep strain of technology in its DNA. The history police tell us New Age media mogul and Netflix head Reed Hastings built debuggers for pinpointing memory access errors in his early days at Adaptive Technology and Pure Software. One might guess this would lead him to make sure the right questions are asked when Netflix contracts for a cloud.

In 2003, Humphrey received the National Medal of Technology for his “vision for software engineering” and its impact on the US government and academic communities.

“Watts Humphrey was one of the icons of software engineering—one of a handful of engineers like Barry Boehm, Fred Brooks, and Vic Basili who have helped define this young field,” said SEI director and CEO Dr. Paul Nielsen.  ”Watts brought engineering to software engineering. His work has had immeasurable impact on the global software community, tirelessly urging the community to emphasize quality, measurement, and performance.”

Humphrey graduated from the University of Chicago with a Bachelor of Science in Physics, and later received a Masters degree in Physics from the Illinois Institute of Technology and in Business Administration from the University of Chicago Graduate School of Business.

Humphrey went on to lead the IBM software team in the late 1960s, and was appointed vice president of technical development. After working with IBM for almost 30 years, he joined SEI, where he helped pioneer the development of the Software Capability Maturity Model, which later combined with other developments to become a framework for engineering best practices used around the globe.

Humphrey was dubbed “Father of Software Quality” for his dedication to addressing obstacles in software development such as performance problems, defects, and cost increases.

“As the pioneering innovator behind several important software development processes, Watts more than met his promise to change the world of software engineering. His contributions go well beyond methodology and the many awards and accolades he received. For decades, his work inspired software engineers and his colleagues and friends worldwide. His warmth, energy, great spirit and dedication will be missed by all of us at Carnegie Mellon,” said Jared L. Cohon, president of Carnegie Mellon University.

Humphrey has written 12 books and hundreds of technical articles and technical reports. The Watts Humphrey Software Quality Institute in Chennai, India was named in his honor. [From PR Newswire release.]

Mittal told us that, yes, SOA actually is getting adopted out there, that as a result applications are now very integrated and that the work on these apps is more often distributed across multiple teams.

Most especially, “the expectation of the number of releases per year has gone up greatly,” he said.

Greater componentization has reduced development time, but it is more difficult to reduce test time.  That is where the equal reaction comes in. Development time is down but bugs, test provisioning and deployment time are up.

Mittal not surprisingly sees a stronger play for iTKO’s test simulation software in SOAs. There is much integration testing to do, as moving integration testing earlier in the process remains “easier said than done.” He also sees recent inroads in cloud computing as a way for testers to catch up with developers.

Joining Mittal in conversation was Walter Falk, business development executive with IBM’s corporate strategy team.  Falk said testing and development is “by far the largest workload on the cloud” at this time.  Certainly, more immediate provisioning is part of the appeal there.

Earlier this year, iTKO and IBM announced an effort to combine iTKO  LISA software with IBM Global Business Services consulting to speed application deployment either on-premise or in a cloud. “With what we have done with iTKO we are able to reduce the cycle time, and make [testing] resources available at any point in time,” said Falk.

Test virtualization and validation in the cloud should get a real workout as part of another iTKO alignment. Last week, the company said it would integrate its LISA software within the U.S. Department of Defense’s Forge.MIL software and service provider community sites.

From the Vault: Gomez on Ajax testing

With SOA, unit testing takes on a whole new tenor. Composite applications may always be in flux. The fact that an application integrator is not in charge of all SOA system elements is inescapable. [see  Application performance management today, part 3: SOA performance - SearchSoftwareQuality.com]

With AJAX, unit testing takes a whole new tenor, again. AJAX – often the delivery mechanism for SOA – of course, is about more than JavaScript. Typically modern JavaScript appears as part of a framework. JavaScript is used more in environments such as DOM. Means have arisen to bring object-oriented qualities to work horse JavaScript. Now, mashups are appearing as an ultimate enterprise representation of AJAX. And unit testing of such applications is challenging.

A good look at some of the issues is provided by Nicholas C. Zakas on the Yahoo! User Interface Blog, which is all about tools and tricks from the Yahoo libraries. He discussed JavaScript Unit testing and the DOM. He discusses where functional tests outpace unit tests in AJAX undertakings.

Related SOA test/performance info
Application performance management today, part 3: SOA performance - SearchSoftwareQuality.com
Writing Effective JavaScript Unit Tests with YUI Test – YUIblog

The increasingly popular SOAP UI can wend its way through WSDL styles, according to Cohen. ”There are different styles of WSDL, depending on the tool or platform that generated it, ” he said. ”For example, Microsoft tends to use XML schemas in a certain way. SOAP UI understands that. It understands other different types of styles of WSDL as well. ”

SOAP UI has another special play related to larger development styles too. The headlong rush to Agile programming methods is a associated with a rebirth in unit testing. You may even write your tests before you write your app. But what do you do when you are developing but have no control over a service being developed elsewhere. Well the answer for many for the time being is to use mock objects. SOAP UI will create a mock object. This can serve, like some kinds of love, until the real thing comes along.

Read more about SOAP UI on SearchSOA.com. Presented there is a list of SOAP UI reference data. And for those of you in the U.S., happy day after Thanksgiving! Tryptophan forever!

http://searchsoa.techtarget.com/news/article/0,289142,sid26_gci1340132,00.html

“Try to test for things that you don’t expect to happen, he said in a new SearchSOA podcast. ”That’s particularly important when you’re testing security aspects of an architecture. Make sure what you’re testing is not part of the expected flow.”

Fredell also recommends that architects assume that things will go wrong with their SOA implementation and plan for glitches. 

“Architecting for fault tolerance, you assume what can go wrong will definitely go wrong,” he said. “If you’ve got a component of your architecture servicing requests, when that component fails, what does it fail over to, to insure that the clients of your service are not affected.

SOA requires thinking outside the box because it is different from monolithic or client/server applications in that you may not have control over all of the Web services interacting with your SOA application, the CTO said.

“The beauty and the risk associated with service-oriented architecture is on the one hand you can compose services and meet business needs very rapidly,” Fredell said. “On the other hand, you may only have control over one side of the equation, the side where the service is implemented. That may mean that people do things that you really don’t expect.”

Access an interview with Thomas Fredell on SOA and the Unexpected