Open source news to end the week looks to the somewhat incongruous connection between Menlo Park California and Helsinki — it can only be MariaDB Corporation with that HQ combo.
Updates to end 2018 see the new availability of the MariaDB Connector for Node.js, giving developers a method to build Node.js applications on top of MariaDB’s enterprise relational database.
When benchmarked against other popular Node.js drivers, the MariaDB Connector performed 1.7 times faster in query mode and 35 times faster in batch mode.
What is a software connector?
If that doesn’t paint enough of a picture to answer the question posed in our headline… then let’s try a definition from O’Reilly as linked here and shown below.
“Simply put, software connectors perform transfer of control and data among components. Connectors can also provide services, such as persistence, invocation, messaging and transactions, that are largely independent of the interacting components’ functionalities. These services are usually considered to be ‘facilities components’ in widely used middleware standards such as CORBA, DCOM and RMI.”
Director of technical marketing at MariaDB Corporation Thomas Boyd says that developers are tired of the limitations of building Node.js applications on top of NoSQL.
“We’re seeing a growing trend of developers using MariaDB to replace the ‘M’ in the MEAN stack to enable a broader view of the data across the business using standard SQL,” said Boyd.
The MEAN stack is MongoDB, Express.js, AngularJS (or Angular) and Node.js.
With the MariaDB Connector for Node.js, developers enjoy better overall application performance through the Connector’s Insert Streaming and Pipelining functions. These capabilities reduce network traffic and enable asynchronous processing of database queries from a Node.js application, making the processing of multiple database queries far more efficient.
The Connector works with MariaDB’s standard SQL layer, providing access to the latest MariaDB database features and innovation, such as temporal processing, advanced data protection and more, as well as ease of access to data for reporting and analytics.
The MariaDB Connector for Node.js is available for download as a Release Candidate now.
As many readers will know, GitHub is a web-based version-control and collaboration platform for software developers. Microsoft, the biggest single contributor to GitHub, initiated an acquisition of GitHub for $7.5 billion in June, 2018.
GitHub’s roots are firmly placed in open source, it was founded on Git, an open source code management system created by Linus Torvalds.
But before we start to look at individual tools and projects, let’s dip more generally into the kinds of activities currently playing out on the site itself.
Freedom of speech
GitHub’s own end of year message notes that developers can now express themselves on the site in their preferred medium: words, code, or tiny cartoon images (emojis, even) if they choose.
In 2016, GitHub released emoji reactions to ‘quiet the noise’ of contentless issue and pull request comments like +1.
The GitHub team wanted to get a sense of how its community expresses themselves with emojis and so it looked at which ones they use in (and in reaction to) issue and pull request comments over the last year.
Expressions of approval and celebration make up the most of the reactions. In fact, users are giving the 👍 and celebrating with a 🎉 more than anything else.
“Looking at projects tagged with a primary programming language, we can see which emoji language communities use most. Comments in Ruby projects had the most ❤️s, and C# users are casting nearly double the 👎s as any other group,” notes the team.
Beyond programming communities, geographic trends demonstrate how far some emoji reach. No matter their location, developers react to build consensus and to say, “job well done”.
Zooming in, Japan spread positivity by reacting with more 👍s and ❤️s per user than any other country. And developers in the Czech Republic have something to celebrate. They reacted with the most 🎉s on average.
“Five emoji could never fully represent the complexity of human emotion. When a 👍 isn’t enthusiastic enough, you often post a 🚀. In the last 12 months, the GitHub community created almost 10,000+ issue comments containing no other content but this tiny craft. That’s a lot of warp-speed shipping,” notes the team.
To learn more about the numbers and methodologies, check out this year’s Octoverse report.
LF Networking (LFN) is the label used by the Linux Foundation to denote the coming together of seven top networking projects.
In other (arguably more straightforward) words, LFN is an open source networking stack.
The openly stated aim of LFN is to increase harmonisation across platforms, communities and ecosystems.
This December 2018 sees new platform releases from ONAP (Casablanca) and OPNFV (Gambia) with additional support for cross-stack deployments across use cases such as 5G, Cross-Carrier VPN (CCVPN), as well as enhancements to cloud-native VPN.
The project’s compliance and verification program has also just announced its expansion into Virtual Network Functions (VNF) testing and is now recruiting beta participants.
General manager for networking at the Linux Foundation Arpit Joshipura suggest that there is ‘growing diversity’ among the of contributors to Open Networking Automation Platform ONAP, a key project within Linux Foundation Networking.
AT&T’s Chris Rice insists that his firm remains committed to actively contributing new code; partnering on new 5G initiatives; orchestrating services across VNFs, PNFs (and soon CNFs (Container Network Functions) as well), plus also developing model driven platform enhancements like the Controller Design Studio.
The team as a whole state that ONAP exists to help end users to automate, design, orchestrate and manage services and virtual functions — they say it has now expanded into a scalable platform with multiple, parallel threads.
Expose & orchestrate
“We view standardisation and openness as critical solution requirements for realization of the dynamic and automated service-driven network, and we are very pleased with the results that ONAP is providing in accelerating speed to market and vendor innovation for our customers”, said Anthony Goonetilleke, group president for entertainment, media and technology at ONAP member Amdocs.
Goonetilleke suggests that he and his team are seeing ‘significant positive momentum’ from customers who are starting to use ONAP to expose and orchestrate their networks-as-a-service.
Also highlighted is OPNFV — a system-level integrations, deployment, testing and feature development project — which is designed to advance the state of NFV around cloud native and moves toward implementing continuous delivery (CD) to support DevOps.
Expansion of the OPNFV Verification Program (OVP) is also underway.
Other ONAP member quotes on the current status of the project are linked here.
It sounds like a brand of orange juice… and its community edition is written in C++, but actually ArangoDB is a native multi-model database.
ArangoDB Community Edition is available under open-source license… but news this week focuses on the release of core version 3.4 as a transactional database for developers.
For developers how?
The software engineering play here is a means of interacting with ‘multiple data models’ by using one technology and one query language.
Enhancements in ArangoDB 3.4 include ArangoSearch, a feature which changes ArangoDB, when combined with traversals or joins in the database’s own query language AQL, from a ‘data’ retrieval to an ‘information’ retrieval tool.
But what is a multi-model database anyway?
As explained by TechTarget, a multi model database is a data processing platform with greater scope to define the parameters for how the information in a database is organised and arranged — being able to incorporate multiple models into a single database lets users meet various application requirements without needing to deploy different database systems.
So… basically, more juice from a single space, right?
ArangoSearch combines Boolean and generalised ranking retrieval models (e.g. vector space model). It consists of two components – a search engine and an integration layer. The former is responsible for managing the index, querying and scoring, whereas the latter provides search capabilities for the end user in a convenient way.
Its makers state that ArangoSearch can be combined with all three data models in ArangoDB. If used in conjunction with graph database capabilities, search results could be used, for example, to enhance fraud protection, individualise recommendations or simplify precision medicine.
According to this software’s release notes, “Search uses a special kind of materialised view to enable full-text search on multiple collections at once. Within the view definition one can specify entire collections or individual fields that should be covered by an inverted index using one or several general text analysers. In search queries expressed with AQL, you can rank the results using multiple scorers (TFIDF and BM25) even combined.”
Users can now perform relevance-based matching, phrase and prefix matching, search with complex Boolean expressions, query time relevance tuning and combine complex traversals, geo-queries, and other access patterns with information retrieval techniques.
ArangoDB 3.4 also includes full support for GeoJSON, an open standard format designed for representing simple geographical features, along with their non-spatial attributes. The support encompasses all geo primitives, including multi-polygons or multi-line strings.
Orange juice silliness aside… it appears from Arango’s livery and logos that there’s more of an avocado association.
Elasticsearch is a Java-developed ‘application search’ engine based on the Lucene library, it is open sourced under the Apache License
It provides a distributed, multi-tenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.
Elasticsearch developers David Harsha, Marshall Scorcio, Brian McGue, Kellen Evan have come together to note that Elasticsearch has now been downloaded over 350,000,000 times.
So we know Elasticsearch then, that’s for searching, logging and data storing at the infrastructure level.
Elastic App Search is also for search, and it is built on and on top of Elasticsearch.
Elasticsearch of course seldom exposed to the public Internet.
The team explains that to allow public access to Elasticsearch data, developers mights put it behind a proxy server, or wrap its APIs in existing middleware.
Programmers would then also need to ponder what to expose, keep it secure, manage authentication for… and when and where to write ‘insulating logic’ to allow clients to reach your data.
“If you just want to provide search within your application — to index your data, then construct an application or product from it — you will be most interested in Elastic App Search. It is the intelligent, optimised, API wrapper in front of Elasticsearch,” notes the team.
During the beta period, developers can spin Elastic App Search up on their own hardware at no charge. To do so, download App Search.
This self-managed release of Elastic App Search requires that you have Java 8 installed and Elasticsearch 6.4+ running.
The original ARM operating system RISC OS has gone open source.
The OS itself was originally developed by Acorn Computers in Cambridge, England and was first released in 1987.
It was initially developed to run the 32-bit Arm processor family and was built using assembly language.
RISC OS is open sourced and available under the Apache 2.0 licence — it was one of the first operating systems to support the Raspberry Pi.
According to Elector, RISC OS was the underpinning OS for many famous software packages, including titles such as the Sibelius music processing family and the Xara range of graphics products.
“It also famously powered the graphics and production of many TV shows during the 1980s, 90s and 2000s including household names such as Who Wants to be a Millionaire,” notes Elector.
RISC OS in its open source 2018 status includes BBC BASIC, which is capable of running out of the box on the OS itself.
Between 1987 and 1998, RISC OS was contained within every ARM-based Acorn computer model. These included the Acorn Archimedes range, Acorn’s R line of computers (with RISC iX as a dual boot option), RiscPC, A7000 and also prototype models such as the Acorn NewsPad and Phoebe computer.
A version of the OS (called NCOS) was also used in Oracle’s Network Computer and compatible systems.
Red Hat appears to be continuing with its normal course of business (and product roll out programme) under the ownership of its new IBM parent — the firm’s OpenStack Platform 14 has been launched this month.
This is Red Hat’s cloud-native (and indeed apps-ready) Infrastructure-as-a-Service (IaaS).
Based on the OpenStack “Rocky” community release, version 14 is said to more tightly integrate with Red Hat OpenShift Container Platform, its Kubernetes platform.
The new version is also shaped for better bare-metal resource consumption and enhanced deployment automation.
The mission statement for this software (if there were an official one) would be Red Hat OpenStack Platform 14 aims to deliver a single infrastructure offering that can act as a foundation for traditional, virtualised and cloud-native workloads.
Red Hat says that as the move to containers and cloud-native applications become prominent pieces of enterprise digital transformation strategies, being able to effectively deploy and scale enterprise-grade Kubernetes on OpenStack becomes a necessity for IT teams.
There is also integrated networkin here, which is meant to enable OpenShift container-based and OpenStack virtual workloads from the same tenant to be connected to the same virtual network (Kuryr), which, in theory at least, should help increase performance of the architecture.
Scale-out & scale-in
Additionally, Red Hat OpenStack Platform 14 bring director-based scale-out and scale-in Red Hat OpenShift nodes, enabling businesses to expand or retract resources as workload requirements change.
The firm insists that running workloads on bare-metal servers offers the ability to fully use cloud-native technologies with unimpeded processing power. To better meet this need, Red Hat offers a consistent management experience between virtualized and bare metal nodes.
Red Hat OpenStack Platform 14 also extends integration with Red Hat Ansible Automation, making the deployment process easier than in previous versions.
IT operations teams can now preview a Red Hat OpenStack Platform before it goes live, helping them to identify and resolve any issues. Additional visibility is provided during the deployment process itself, so it’s faster to identify failure points and remediation, including the capacity to repeat and re-apply isolated deployment steps if needed.
This new release of NativeScript comes as the framework hits a milestone of 3.5 million downloads since its original launch in 2015.
NativeScript was created to attempt to ease the mobile app development process by ‘leveraging’ (Ed – we hesitate to use the L-word, but it does fit in this rare case) developers’ existing web development skills to create native mobile apps for iOS and Android.
According to Todd Anglin, VP of product and developer relations at Progress, this release NativeScript has improved developer productivity through capabilities including code sharing, hot module replacement, preview enhancements etc.
“In 2016, NativeScript 2.0 gave developers the ability to use Angular and NativeScript to create native mobile apps, but we wanted to take it a step further by enabling them to keep their code in one place and share business logic between web, iOS and Android,” said Brad Green, engineering director, Angular Framework, Google Inc.
Green says his team has worked closely with the NativeScript team resulting in the NativeScript-Schematics that integrates with the Angular CLI, making the user experience for developing both web and mobile within a single project completely seamless.
Among those new enhancements, hot module replacement allows for instantaneous, stateful application updates, avoiding full page reload, which in turn is supposed to help with accelerating the app development and debugging experience.
There is also a streamlined “Getting Started” experience – and easier development with ‘playground-compatible’ code samples available in the NativeScript Marketplace.
How does the The Apache Software Foundation work out who are the best people?
Let’s remember that the foundation itself is a non-profit corporation established to support Apache software projects, including the Apache HTTP Server… so anybody joining its ranks has to be of a certain standard, obviously.
The name ‘Apache’ was chosen from respect for the Native American Indian tribe of Apache, well-known for their superior skills in warfare strategy and their inexhaustible endurance. It also makes a cute pun on “a patchy web server” — a server made from a series of patches — but this was not its origin.
So again we ask… how does do the members of the Apache Foundation work out who the best developers are?
Sally Khudairi, founder/CMO of OptDyn and vice president at The Apache Software Foundation says that we need to go beyond the ‘binary’ issue of (just) testing for skills requirements.
She argues that this need to be augmented/replaced by ‘can this candidate think critically/creatively?’ sorts of questions to demonstrate intellectual abilities.
“[In the recruitment process] both our Founder/CEO Alex Karasulu and CTO Niclas Hedhman would review the candidate’s code (on GitHub or wherever it was contributed) as well as their interactions on mailing lists to get a clearer idea on how the individual performs technically an well as socially. There is something to be said about the transparency of open source,” said Khudairi.
Alex Karasulu says he usually breaks candidates into two categories.
Karasulu is founder/CEO of OptDyn and member/project mentor at The Apache Software Foundation.
“The first category are the unknowns. They need to be low balled and put into a 60-90 day trial period. The other category are those who have activity in open source and those I would check what they’ve done code wise and their interactions with others like on mailing lists. You can also check how well they’ve organised their code and how persistent they have been on their projects which shows passion. Passion is a key ingredient: you don’t want to work with people without the passion to innovate. Of course this is if I have time and am involved,” said Karasulu.
Niclas Hedhman, CTO of OptDyn and member/project mentor at The Apache Software Foundation agrees with Karasulu.
Hedhman also says that both Google and Microsoft have research on this topic and have found that they are incapable of predicting the post-hire performance from any test, any questions and any previous field/experience. A reality which he finds quite discouraging.
“When I hired for myself, there was two ways that worked for me: find the person I want by looking at open source work performed — and personal recommendation where the recommender had something to lose (reputation). Everything else was a flip a coin accuracy, mostly negative (don’t meet expectations) outcomes. I agree with that big firms end up asking strange questions on details,” said Hedhman.
Hedhman recounts an experience at JP Morgan where he was given a one hour coding task. Candidates got the test cases to pass for a event receiver limiter in front of a high frequency trading engine. They had 60.0 minutes to complete and any time over the application went straight to dustbin.
In his opinion, this is a mad way to hire.
Lars Bøgild Thomsen is director of infrastructure at OptDyn. Thomsen explains that some years ago, he saw a list of questions used by Google to assess system admins. He was ‘absolutely baffled’ how much it focused on remembering details rather than on understanding how to figure out the details.
“I would assume a lot of developer assessment fall into the same trap – assessing how much the applicants remember rather than how good and fast they are at figuring it out. Hence the ‘broom’ approach – something I really (seriously) would like to see in practice one day: the idea being when new staff walk into the door give them a broom and tell them to wipe the floors UNTIL they themselves find something more important to do,” said Thomsen.
The concept above is, if they are too proud to sweep – kick them out right away – if they still wipe floors after 1 week – kick them out too.
Existing staff have to be in on it and obviously share stuff if the sweepers walk up and ask.
“It’s so bizarre it is funny but it came up based on a discussion that a lot of people seem extremely unwilling or unable to figure out where their effort is most needed. Or where they themselves could make a difference. It wouldn’t work with everybody but I honestly do believe that whoever passed that test would be valuable assets in just about any organisation,” he said.
Techie recruitment continues to be a hard nut to crack, so we have to thank the team at Apache and OptDyn for sharing so openly.
Multi-cloud automation company Mesosphere has now come forward with Mesosphere Kubernetes Engine (MKE), Mesosphere DC/OS 1.12 and the public beta of Mesosphere Jupyter Service (MJS).
Mesosphere Kubernetes Engine is Kubernetes-as-a-Service on multi-cloud and edge… and (as readers will likely know) Kubernetes is an open source container-orchestration system for automating the deployment, scaling and management of containerised applications.
This provides what is known as ‘high-density resource pooling’ for cloud apps, a type of load balancing that provides cloud applications with the processing, storage, networking ability and analytics (resources) they need in environments where data throughput is extreme (high-density)… and it does all this without the need for virtualisation.
MJS is intended to simplify delivery of Jupyter Notebooks, popular with data scientists, to streamline how they build and deliver AI-enabled services. Readers will also note that Project Jupyter exists to develop open source software, open standards and services for interactive computing across dozens of programming languages.
“Companies need to move fast to stay relevant in today’s competitive landscape. To do this, IT teams are leveraging leading tools such as Kubernetes, Jupyter Notebooks, advanced security and software registries to drive software innovation,” said Florian Leibert, Mesosphere CEO.
Leibert insists that by natively integrating Kubernetes and Jupyter into DC/OS, his team is able to deliver fast deployment and centralised management — but still enabling experimentation and providing developer choice.
Mesosphere DC/OS 1.12 is all about giving cloud developers edge and multi-cloud infrastructure from a single control plane. With Mesosphere Kubernetes Engine (MKE), enterprise IT can centralise scattered Kubernetes clusters on multiple cloud providers managed from a single platform.
Where is all this leading us?
We know that many data sets are too large to fit on laptops or individual workstations. This forces data scientists and engineers working with Jupyter Notebooks to repeatedly work with smaller data sets, constraining progress and increasing the risk of data leaks.
It leads us towards what we could call ‘on-demand data science’ – that is, data scientists get instant access to the Jupyter Notebooks computing environment, preconfigured with a good deal of the tools they need.
The move to compartmentalise, package and automate many of these functions is (arguably) very much ‘where cloud-native application is at’ right now… but it’s complex stuff, we need to move carefully.