DataStax closed out the final day of its ‘Accelerate 2019’ conference by focusing on a selection of platform-level developments including its community development stream.
Nate McCall, project chair, Apache Cassandra spoke openly on the journey to Apache Cassandra and its 4.0 iteration.
“Community is the lifeblood of our ecosystem,” said McCall.
Looking at the history of Cassandra, McCall explained how Cassandra had started as a project that came out of from Facebook with some additional technologies emanating on Google code back in 2009. Looking at what has happened over the last decade, he thinks that the project has grown considerably and now presents itself with ‘well-defined protocols’ and a far higher degree of usability.
Today we know that most contributors are running clusters and most of them are really bug — these are the people that are making commits on the project. This means, logically, that many of the features appearing in the project are being created by users operating at that kind of level.
“The marketing team was not involved [in the roadmap development] because we know real developers are the ones driving Cassandra forward,” said McCall.
Explaining that contributions can take any form – not just code – McCall pointed to some specific companies whose contributions have been mainly focused on documentation, which is just fine i.e. it’s all valuable and all needed.
Cassandra development in the community this year will come together at both Apachecon North America… and at the NGCC i.e. the Next Generation Cassandra Conference. The community is now looking to share out who is doing the test tracking for various aspects of the project based upon what those individuals (or teams) have specific competencies and skills in.
Folks fuel forks
In reality, the community knows that lots of folks are running internal forks… and this may be because users are waiting for Cassandra 4.0 to come forward. McCall says that a big part of the reason the community is being made to wait is that the team know that they have to make the 4.0 really really good.
McCall handed over to DataStax CTO and co-founder Jonathan Ellis.
No vendor does more for Cassandra than DataStax claimed Ellis… and he is referring to all the support and training that the company inputs alongside all the code input that it drives towards the core.
In terms of ease of use, Graph is a powerful way of exploring data and DataStax is bringing Graph techniques to Cassandra. Equally, Ellis claims that DataStax will be giving some of the wide distributed data positives back to Graph in general.
Ellis dug into the Kubernetes Operator that DataStax is developing to automate container provisioning, automate roll out of config changes and a range of other functions. He also spoke about DataStax Desktop (which offers a good deal of configuration shortcuts), which is available on the firm’s DataStax Labs website zone.
The Labs zone itself is typified by its beta-level software… users need to sign to the following agreement if they wish to use these technologies:
“You agree to test DataStax Labs Software and provide DataStax with comments, notes, bug reports and feature comments with sufficient documentation, samples, code error, screen shots, etc., to help DataStax evaluate and improve the DataStax Labs Software.”
Ellis also spent time explaining the workings of DataStax Studio, which is an interactive developer tool for CQL (Cassandra Query Language), Spark SQL and DSE Graph.
Thinking about what DataStax has announced in line with this year’s event itself, Ellis also worked through some of the functions available in DataStax Insights, this is the company’s product devoted to performance management and monitoring.
AppStax.io is the firm’s approach to automating a data layer… so this is DataStax aiming to provide a new layer of automation for developers, rather than operations, sysadmin staff or database administrators of any kind.
DataStax is aiming to bring off a complex feat of engineering and there is abstraction, automation and simplification here at many levels.
DataStax CEO Billy Bosworth started out as a database administrator (DBA), so one would hope that he knows how to build, compile, manage and deploy in all senses of those terms, right?
Bosworth took the stage to keynote the opening session at his firm’s ‘Accelerate 2019’ data-developer conference and exhibition, which was held in Washington DC this May 2019.
His CEO presentation was formally entitled: accelerating development in a cloud world.
Now then, that sounds like marketing, so why did he use that title?
Perhaps it’s because DataStax’s core drive is all about user choice (by which we mean data-centric developer/programmer system and operations engineer, rather than ‘human’ user — although the happiness of the latter depends upon the former).
What does technical user choice mean in this sense? It’s all about options such as scale-up and scale-down functionality on Cassandra clusters with consumption-based pricing. So it’s tuning the data delivery model to more sympathetically suit what the use case demands.
Bosworth suggested that developers today face a special challenge because new tools emerge every day. He also said that technology both ‘enables and restrains’ a developer’s ability to accelerate innovation.
Similarly, he has pointed out that operations and IT teams responsible for the deployment, upkeep and support of new applications are simultaneously challenged and supported by new ways to deploy — whether that be on-premises, in the cloud, or using a hybrid approach.
Newer, easier-to-use tools
What all of this drives us towards is the idea that we should be able to get easier-to-use tools that make it faster to develop apps that perform regardless of data location, size, and format — and this is (perhaps) an encapsulation of the mission that DataStax says it is on.
Bosworth is clearly impressed by the scale and performance of Apache Cassandra, but he thinks that the top adjectives that come to mind when using this platform would NOT include ‘easy’ or ‘simple’.
“We want to take the powerful technology that exists in Apache Cassandra and make it easy and better – I might even say fun,” said Bosworth, in his keynote address. “Because companies are having to expand and work in new markets all the time, business is getting even tougher. But even in a world where so many underlying infrastructures are changing at lower levels, one common thread we see across all business models is the move to cloud.”
The DataStax CEO says that his firm’s technology is ideally (okay, he said uniquely) suited cloud environments because it is essentially distributed and has a ‘masterless’ architecture with no single point of failure… but, at the same time, it offers a single toolset to manage cloud and on-premises environments.
Demo: chaos in the clouds
A live demo 18-minutes into a keynote is probably a good indication of any event being a real ground level data-centric developer-centric gathering. Bosworth was joined on stage by Chelsea Navo in her role as global vanguard lead at DataStax.
Navo demo’d DataStax’s technology which is capable of spanning connections to (in the example shown) three major cloud providers. With a massive number of servers are out there, there are millions of points of potential failure… so apps need to be able to withstand a rough ride and rely upon a database that can handle disruptive events happening across the network. If nodes drop out (or whole clouds drop out) from some servers in some cloud, DataStax will provide enough resilience to be able to deal with ‘chaos in the clouds’ and switch to resources that are live and functional.
This all leads towards the need for a new layer of management intelligence for cloud applications… and this is what DataStax is aiming to provide with its new DataStax Constellation announcement.
Constellation will launch later this year with two cloud services: DataStax Apache Cassandra as a Service and DataStax Insights. DataStax Apache Cassandra as a Service will deliver easy scale-up and scale-down Cassandra clusters, on consumption-based pricing, which is backed by the stability and performance enhancements of DataStax Enterprise.
Follow up presentations subsequent to Bosworth’s CEO address featured an address from Deloitte’s principal for technology services Mark White – and also from Judy Meyer, VP, WW ISV business leader and AI Advocate, Microsoft.
“We know that the world of the developer is changing. I want you to think about the ubiquitous compute power that the cloud gives us… at what is now at zettabytes of data. This means that we need a modern data estate – and this can be done using DataStax running on Microsoft Azure,” said Meyer.
When she talks about data estates, Meyer is referring to data existing on all devices, all computers, all ‘things’ in the Internet of Things, all operational databases, data lakes and all databases themselves.
There’s certainly a theme coming forward here: if we’re in a world with complexity in the cloud and chaos in the cloud (and perhaps even chaotic cloud complexity) then we need management tools and platform intelligence capable of helping us navigate out of those black holes. Crucially, we need to be able to do that fast and do it without necessarily needing a massive amount of network engineering expertise, because so many more technology practitioners are exposed to the engine rooms of the cloud itself. The sheer scale of the complexity and raw power in cloud has the potential to actually slow us down and encumber IT systems because it’s such a massive (virtual) engine to fuel, run and operate… but it can also accelerate our applications and data services if we use the correct tools — and DataStax wants to be that toolset.
NoSQL database company Couchbase has moved off the sofa (the firm is no couch potato, get it?) and come forward with new features aligned to allow ‘deployers’ to build (and scale) applications.
The new version of Couchbase Autonomous Operator is enables Kubernetes-based application deployers to bring in a database ready for flexibility and micro-services.
That was ‘deployers’, not ‘developers’ — right?
So rather than talking about software engineers with a pure-breed approach to programming, languages and the command line… we’re talking about sysadmins, DBAs and upwards to software programme managers (let’s also throw in data-developers and other DevOps types from the Ops team that have a good Dev head on them in that case) who might be tasked with the architectural responsibilities of bringing cloud-based data systems into being and ensuring that the workflows inside them are fed with the appropriate levels of system resources in order for them to function.
What kind of organisation might be focused on this technology?
One that is looking to deploy, manage and scale hundreds of database instances and clusters in an environment where they are able to use hybrid-cloud platforms.
Couchbase Autonomous Operator 1.2 new features include automated upgrade of Couchbase clusters and automated rolling upgrade to Kubernetes clusters. There is also support for Helm Charts — Helm allows users to templatise their Kubernetes manifests and provide a set of configuration parameters that allows users to customize their deployment — to simplify their installation and deployment.
Specifically for developers, Autonomous Operator 1.2 allows programmers to perform automated upgrades of Couchbase Server clusters. Upgrading an entire Couchbase cluster is done by specifying the desired Couchbase version in the configuration and then pushing it to Kubernetes.
With support for automatic rolling Kubernetes upgrades, Autonomous Operator 1.2 clams to be able to lighten the load of upgrading the enterprise Kubernetes or OpenShift cluster, while ensuring that the Couchbase deployment doesn’t experience any downtime or data loss.
Pursuing a panacea
Senior director of engineering at Couchbase Matt Ingenthron thinks that we’re all on a road searching for the perfect, flexible system that makes the promise of hybrid, multi- cloud deployments with both their database and app tier a reality.
“With this release, Couchbase is ahead of other databases and much closer to that panacea. It is now possible for development teams to build the Couchbase Autonomous Operator into their development and continuous integration pipelines on a public cloud like GKE, EKS or AKS and then hand off updates to their DevOps team to deploy to on-premises Red Hat OpenShift infrastructure. This removes friction and time in both development and deployment, getting features in releases much faster than before,” said Ingenthron.
Couchbase has also said that Autonomous Operator 1.2 brings production certification and full production support for cloud-managed Kubernetes services on Amazon (AWS EKS), Azure (Azure AKS) and Google (Google GKE) platforms.
The 1.2 release also refreshes support for Red Hat OpenShift to the latest release.
This cloud-agnostic approach is supposed to avoid vendor lock-in and allows enterprise customers with multi-cloud strategies to deploy, manage and move their applications to and from any cloud, at any time.
Open source software platform company SuSE has announced SUSE Enterprise Storage 6, a software-defined storage solution powered by Ceph technology.
Many would argue that storage on its own is snorage (i.e. enough to send you to sleep), but software -defined storage does at least drive us forward into the realm of the software developer.
By way of a reminder, software -defined storage is a way of managing data storage resources and functionality that is essentially uncoupled from (i.e. has no underlying physical dependencies) the actual hardware resources that offer up the amount of storage being used.
Software-defined storage is even more typified and characterised (than by its hardware dependency decoupling factor) by its ability to be upgraded independently of the hardware that it sits upon — just the kind of control factor that software engineers love.
Ceph on the other hand (yeah we know, it sounds like encephalitis brain inflammation, which may be a deliberate joke on behalf of the engineers who built it) is a free-software storage platform that implements object storage on a single distributed computer cluster and provides interfaces for object-, block- and file-level storage
Another theory suggests that Ceph is called Ceph because it’s short for a cephalopod, as in octopi. “The many tentacles of the octopus are supposed to represent the parallelism of Ceph. Ceph came out of UCSC, whose mascot is Sammy the Slug. Slugs and cephalopod are both part of mollusca phylum,” writes Avylove here, on the Red Hat Community pages.
Sorry… wasn’t this story about SuSE Enterprise Storage?
Yes it was, but we like backgrounders, etymology and appropriate level of contextualised exposition before we get to the main course here – okay?
SuSE says its new offering enables IT organisations to adapt to changing business demands with lower operational expense through new features focused on containerised and cloud workload support, improved integration with public cloud and enhanced data protection capabilities.
“We work with the community to develop innovative open source technology that we then test and harden to ensure it is enterprise ready. We aim to transform enterprise storage infrastructure with a truly open and unified, intelligent software-defined storage solution that reduces costs and alleviates proprietary hardware lock-in for customers. SUSE Enterprise Storage 6 reflects SUSE’s ongoing commitment to making Ceph ‘enterprise consumable’ [everywhere],” said Brent Schroeder, CTO Americas for SuSE.
The University of Maine is using SUSE Enterprise Storage to support an HPC landscape underpinning a number of research programmes.
Based on the Ceph Nautilus release, SUSE Enterprise Storage 6 is being positioned as a technology which will maximising application availability with faster and more granular backups that can now use low-cost public cloud resources for better data protection.
It also helps them reduce operational IT expense by optimising data placement with the ability to automatically move data between tiers of storage based on policy, ensuring access to critical data when customers need it. SuSE has more here.
Oracle recently came to the dinner table with its Oracle GraalVM Enterprise Edition, so what it, why is important to developers and does Larry Ellison still have a beard?
Built on Java SE, GraalVM Enterprise is a multilingual virtual machine for use in cloud and hybrid environments.
This low footprint technology claims to be able to start services up to 100X faster and reduce memory usage up to 5x by compiling programs, including Java applications, ahead-of-time.
GraalVM Enterprise contains optimisation algorithms that seek out opportunities to accelerate application processes by rearranging compiled code.
It also includes security features to address some of the common sources of security application vulnerabilities, including buffer overflows in native code.
Considering those issues, GraalVM Enterprise includes a “safe mode” for native libraries based on Oracle’s LLVM bitcode runtime.
Safe Mode compiles even those portions of the application implemented in C code to use managed memory, garbage collection and bounds-checks, which help protect against common security applications vulnerabilities like buffer overflows.
The multilingual capabilities of GraalVM allow legacy applications to be modernised faster. The open source library ecosystem for all of the supported GraalVM languages is available for developers building any application, not just the one for the language in which the application was initially written.
So that’s what GraalVM is and that’s why it matters… oh and yes, as far as we know, Larry does still sport chinny whiskers.
Cloud-native machine data analytics platform company Sumo Logic has used its time in the ring at DockerCon 2019 to announce support for Docker Enterprise.
The company wants to provide joint customers with application performance monitoring and, building on existing Docker capabilities, the new integration provides joint users with monitoring and troubleshooting of the Docker infrastructure, as well as better correlation of issues between the Docker platform and the application for faster root cause analysis.
The intention is to provide not only a view of the health of a customer’s applications running on the Docker platform, but also the health of the Docker platform itself for more complete visibility.
The company says that as enterprises continue to adopt cloud infrastructures they are also investing in tools and processes including containerisation, orchestration and serverless computing in what is now shaping up to be the modern application stack.
As a Sumo Logic customer, Docker can monitor and troubleshoot Docker Hub infrastructure to deliver release cycles and to address anomalies and remediate potential issues for end users downloading images.
Kal De, CTO and EVP of product development at Docker reminds us that Docker Hub is the world’s largest library of container images for the open source and enterprise community.
“Containers provide more benefits than traditional virtualisation by enabling organisations to build more efficient applications that leverage microservices architecture and can be orchestrated across private and multi-cloud environments. Our goal is to empower the people who power modern business and to have Docker employ Sumo Logic as its machine data analytics platform reinforces the value of machine data,” said Christian Beedgen, co-founder and CTO, Sumo Logic.
With Sumo Logic, Docker quality engineering and customer support teams are given additional tools to improve the product release process and to reduce the time needed to solve customer issues.
Advanced analytics, real-time alerting and customisable dashboards from Sumo Logic are said to make it easy for teams to identify issues and push code to production faster while increasing code quality.
Cloud data integration and data integrity company Talend has now supported OpenAPI specification (OAS) version 3.0 with its Talend Data Fabric product.
As noted here by Swagger, the OpenAPI Specification (OAS) defines a standard, language-agnostic interface to RESTful APIs which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation or through network traffic inspection.
The new API service advancements focus on what is being called ‘full API development lifecycle support’ … so we are really thinking about APIs as a living, developing, changing, augmenting (and by that we often mean scaling) element of the way integrated software applications are being constructed.
VP of product management at Talend Jerome Louvel says that the growth of APIs is being driven by microservices and digital initiatives.
As such, he thinks that support for an environment that increases DevOps teams’ productivity, while enabling collaboration with less technical API stakeholders becomes critically important.
“With this release, we can also ensure that customers have a smooth upgrade path to OAS 3.0 from previous versions or alternative formats,” notes monsieur Louvel.
iPass the polymorphism
Talend Cloud is an integration Platform-as-a-Service (iPaaS) designed to collect, govern, transform and share data.
Highlights of OAS 3.0 support in Talend Cloud include data type composition, e.g. merge different customer information (such as profile, contact and preferences into a unified view of a customer) and data type polymorphism.
These (and other) new features are usable on existing API definitions. Through a quick import, developers can transition to OAS 3.0 from Swagger 2 as well as from RAML.
In addition to new API services, the Spring ’19 release of Talend Data Fabric includes new features for its management console, data prep and stewardship capabilities.
JNBridgePro is a provider of interoperability tools to connect Java and .NET frameworks… the company’s mission has always been to enable developers to connect anything Java together with anything .NET.
This month sees the company proffer forth JNBridgePro 10.
This tenth iteration release is the first to support the brand-new .NET Core 3.0… and so will bring and enable Java-to-.NET Core interoperability.
The hope is that interoperability complexities [for developers and ultimately users] could be a thing of the past for all users of Windows, Linux and soon, macOS.
As Visual Studio magazine has reminded us, .NET Core is an open source, modular, high-performance implementation of .NET for creating Windows, Linux and Mac apps and aims to improve upon the ageing .NET Framework that’s tightly coupled to Windows.
In addition to support for .NET Core 3.0, JNBridgePro 10 also supports the new Visual Studio 2019, which was released at the start of April 2019.
“It’s our mission to provide tools that track continuing evolution of the .NET and Java platforms,” said Wayne Citrin, CTO, JNBridge. “We’ve always been interested in .NET Core and the ability to develop cross-platform solutions that run equally well on Windows, Linux and macOS. .NET Core 3.0 is now a mature platform with sufficient features to support JNBridgePro. With JNBridgePro 10, we deliver a solution that helps developers on the front lines of innovation continue to be productive without worrying about cross-platform interoperability.”
JNBridgePro provides full access to the entire .NET and Java API from the other side, whether or not it is service-enabled.
It exposes any Java and .NET binary, enabling code reuse without touching the original source code. It is also architecture-agnostic and so scales from in-process to cross-network, on premises or in the cloud.
The Computer Weekly Open Source Insider team is running Ubuntu.
More specifically, we’re running Ubuntu 19.04 on an originally Windows-empowered Lenovo laptop as a dual boot.
NOTE: Incidentally, this is an altogether joyous experience… although the BIOS boot screen defaults to Ubuntu startup if you fail to choose between Windows and Ubuntu, which seems ever so slightly cheeky given that Ubuntu was invited in as a guest… put it down to open source fervour… although the boot options are obviously very easy to edit in the BIOS itself.
So what of Ubuntu 19.04 then?
Canonical says this version of the OS is all about open infrastructure, developer desktop and IoT.
“The open-source-first on Ubuntu movement in telco, finance and media has spread to other sectors. From the public cloud to the private data center to the edge appliance or cluster, open source has become the reference for efficiency and innovation. Ubuntu 19.04 includes the leading projects to underpin that transition, and the developer tooling to accelerate the applications for those domains” said Mark Shuttleworth, CEO of Canonical.
Ubuntu 19.04 integrates recent updates from open infrastructure projects including OpenStack, Kubernetes, and Ceph – with advanced life-cycle management for multi-cloud and on-prem operations – from bare metal, VMware and OpenStack to every major public cloud.
Optimised Ubuntu Server 19.04 and Minimal Ubuntu 19.04 images are available on all major public clouds.
The Edge X stack and a range of industrial control capabilities are now available for integration on Ubuntu based devices, with long term security updates.
Microsoft Visual Studio Code joined the list of developer tools published as snaps including IntelliJ, pyCharm, Postman and microK8s. AI, machine learning and data science platforms continue to prefer Ubuntu for rapid adoption and fast project iterations.
In Ubuntu 19.04, multiple instances of the same snap can be installed for CI/CD, testing or phased rollouts. For example, two versions of a database or two versions of the golang compiler can be installed at the same time. Snap epochs control when and how data migration happens between major version upgrades.
More than 2,000 independent software publishers now distribute their applications to Linux desktops, appliances and cloud VMs through the snap store, growing 30% since October 2018.
The Computer Weekly Open Source Insider team spoke to Deepak Giridharagopal this week in his role as CTO at Puppet in an attempt to examine the true nature of open source openness.
As defined in clear terms here, Puppet is an open source systems management tool for centralising and automating configuration management tasks relating to both hardware and software.
Giridharagopal argues that the ‘open’ part of open source doesn’t just refer to making the code physically available to the masses.
After all, anyone can just publish some code online and dump out the results of their project… whether they are amazing innovations, unnecessary forks or flaky half-baked ideas.
What he wants us to remember is that best open source projects are the ones that have communities built around them.
It would appear then, that effectively participating in open source communities requires skills beyond just cranking out code all day long.
“[Factors such as] empathy, diplomacy, conflict resolution, being able to deliver constructive criticism and effective evangelism of your opinions are hugely important. These traits are helpful not only to navigate all the elbowing open source environments might involve; they also foster a welcoming, diverse and collaborative culture. Without the ability to interact and empathise with others, debate in a way that is not abrasive… and effectively advocate for your opinions and ideas (let alone your code), you won’t be able to build a healthy, long-lasting project,” said Giridharagopal.
Recognising that our industry is full of ‘brilliant jerks’ [Ed: I think he meant ‘geeks’ not ‘jerks’ right?], Giridharagopal says these talented developers might produce great code, but are often terrible at working with others.
“They can be hostile to other contributors, mock those who don’t deliver code up to ‘their standards’, or denigrate the value of non-code contributions (e.g. documentation or user experience improvements). Sadly, there are many popular open source projects that gleefully harbour brilliant geeks. However, those projects succeed in spite of, not because of, their tolerance of abrasive behaviour,” said Giridharagopal.
Giridharagopal concludes by saying that open source adoption shows no signs of abating.
Thus, he thinks, collaboration skills are no longer optional for those who wish to build new open source communities, or effectively participate in existing ones.
Better communities, after all, produce better software.