Coffee Talk: Java, News, Stories and Opinions

Page 1 of 2312345...1020...Last »

July 3, 2017  11:26 AM

Advancing JVM performance with the LLVM compiler

cameronmcnz Cameron McKenzie Profile: cameronmcnz

The following is a transcript of an interview between TheServerSide’s Cameron W. McKenzie and Azul Systems’ CTO Gil Tene.

Cameron McKenzie: I always like talking to Gil Tene, the CTO of Azul Systems.

Before jumping on the phone, PR reps often send me a PowerPoint of what we’re supposed to talk about. But  with Tene, I always figure that if I can jump in with a quick question before he gets into the PowerPoint presentation, I can get him to answer some interesting questions that I want the answers to. He’s a technical guy and he’s prepared to get technical about Java and the JVM.

Now, the reason for our latest talk was Azul Systems’ 17.3 release of Zing, which includes an LLVM-based, code-named Falcon, just-in-time compiler. Apparently, it’s incredibly fast, like all of Azul Systems’ JVMs typically are.

But before we got into discussing Azul Systems’ Falcon just-in-time compiler, I thought I’d do a bit of bear-baiting with Gil and tell him that I was sorry that in this new age of serverless computing and cloud and containers, and a world where nobody actually buys hardware anymore, that it must be difficult flogging a high-performance JVM when nobody’s going to need to download one and to install it locally. Well, anyways, Gil wasn’t having any of it.

Gil Tene: So, the way I look at it is actually we don’t really care because we have a bunch of people running Zing on Amazon, so where the hardware comes from and whether it’s a cloud environment or a public cloud or private cloud, a hybrid cloud, or a data center, whatever you want to call it, as long as people are running Java software, we’ve got places where we can sell our JVM. And that doesn’t seem to be happening less, it seems to be happening more.

Cameron McKenzie: Now, I was really just joking around with that first question, but that brought us into a discussion about using Java and Zing in the cloud. And actually, I’m interested in that. How are people using Java and JVMs they’ve purchased in the cloud? Is it mostly EC2 instances or is there some other unique way that people are using the cloud to leverage high-performance JVMs like Zing?

Gil Tene: It is running on EC2 instances. In practical terms, most of what is being run on Amazon today, it is run as virtual instances running on the public cloud. They end up looking like normal servers running Linux on an x86 somewhere, but they run on Amazon, and they do it very efficiently and very elastically, they are very operationally dynamic. And whether it’s Amazon or Azure or the Google Cloud, we’re seeing all of those happening.

But in many of those cases, that’s just a starting point where instead of getting server or running your own virtualized environment, you just do it on Amazon.

The next step is usually that you operationally adapt to using the model, so people no longer have to plan and know how much hardware they’re going to need in three months time, because they can turn it on anytime they want. So they can empower teams to turn on a hundred machines on the weekend because they think it’s needed, and if they were wrong they’ll turn them off. But that’s no longer some dramatic thing to do. Doing it in a company internal data center? It’s a very different thing from a planning perspective.

But from our point of view, that all looks the same, right? Zing and Zulu run just fine in those environments. And whether people consume them on Amazon or Azure or in their own servers, to us it all looks the same.

Cameron McKenzie: Now, cloud computing and virtualization is all really cool, but we’re here to talk about performance. So what do you see these days in terms of bare iron deployments or bare metal deployments or people actually deploying to bare metal and if so, when are they doing it?

Gil Tene: We do see bare metal deployments. You know, we have a very wide mix of customers, so we have everything from e-commerce and analytics and customers that run their own stuff, to banks obviously, that do a lot of stuff themselves. There is more and more of a move towards virtualization in some sort of cloud, whether it’s internal or external. So I’d say that a lot of what we see today is virtualized, but we do see a bunch of the bare metal in latency-sensitive environments or in dedicated super environments. So for example, a lot of people will run dedicated machines for databases or for low-latency trading or for messaging because they don’t want to take the hit for what the virtualized infrastructure might do to them if they don’t.

But having said that, we’re seeing some really good results from people on consistency and latency and everything else running just on the higher-end Amazon. So for example, Cassandra is one of the workloads that fits very well with Zing and we see a lot of turnkey deployments. If you want Cassandra, you turn Zing on and you’re happy, you don’t look back. In an Amazon, that type of cookie-cutter deployment works very well. We tend to see that the typical instances that people use for Cassandra in Amazon with or without us is they’ll move to the latest greatest things that Amazon offers. I think the i3 class of Amazon instances right now are the most popular for Cassandra.

Cameron McKenzie: Now, I believe that the reason we’re talking today is because there are some big news from Azul. So what is the big news?

Gil Tene: The big news for us was the latest release of Zing. We are introducing a brand-new JIT compiler to the JVM, and it is based on LLVM. The reason this is big news, we think, especially in the JVM community, is that the current JIT compiler that’s in use was first introduced 20 years ago. So it’s aging. And we’ve been working with it and within it for most of that time, so we know it very well. But a few years ago, we decided to make the long-term investment in building a brand-new JIT compiler in order to be able to go beyond what we could before. And we chose to use LLVM as the basis for that compiler.

Java had a very rapid acceleration of performance in the first few years, from the late ’90s to the early 2000s, but it’s been a very flat growth curve since then. Performance has improved year over year, but not by a lot, not by the way that we’d like it to. With LLVM, you have a very mature compiler. C and C++ compilers use it, Swift from Apple is based on its, Objective-C as well, the RAS language from Azul is based on it. And you’ll see a lot of exotic things done with it as well, like database query optimizations and all kinds of interesting analytics. It’s a general compiler and optimization framework that has been built for other people to build things with.

It was built over the last decade, so we were lucky enough that it was mature by the time we were making a choice in how to build a new compiler. It incorporates a tremendous amount of work in terms of optimizations that we probably would have never been able to invest in ourselves.

To give you a concrete example of this, the latest CPUs from Intel, the current ones that run, whether they’re bare metal or powered mostly on Amazon servers today, have some really cool new vector optimization capabilities. There’s new vector registers, new instructions and you could do some really nice things with them. But that’s only useful if you have some optimizer that’s able to make use of those instructions when they know it’s there.

With Falcon, our LLVM-based compiler, you take regular Java loops that would run normally on previous hardware, and when our JVM runs on a new hardware, it recognizes the capabilities and basically produces much better loops that use the vector instructions to run faster. And here, you’re talking about factors that could be, 50%, 100%, or sometimes 2 times or 3 times faster even, because those instructions are that much faster. The cool thing for us is not that we sat there and thought of how to use the latest Broadwell chip instructions, it’s that LLVM does that for us without us having to work hard.

Intel has put work into LLVM over the last two years to make sure that the backend optimizers know how to do the stuff. And we just need to bring the code to the right form and the rest is taken care of by other people’s work. So that’s a concrete example of extreme leverage. As the processor hits the market, we already have the optimizations for it. So it’s a great demonstration of how a runtime like a JVM could run the exact same code and when you put it on a new hardware, it’s not just the better clock speed and not just slightly faster, it can actually use the instructions to literally run the code better, and you don’t have to change anything to do it.

Cameron McKenzie: Now, whenever I talk about high-performance JVM computing, I always feel the need to talk about potential JVM pauses and garbage collection. Is there anything new in terms of JVM garbage collection algorithms with this latest release of Zing?

Gil Tene: Garbage collection is not big news at this point, mostly because we’ve already solved it. To us, garbage collection is simply a solved problem. And I do realize that that often sounds like what marketing people would say, but I’m the CTO, and I stand behind that statement.

With our C4 collector in Zing, we’re basically eliminating all the concerns that people have with garbage collections that are above, say, half a millisecond in size. That pretty much means everybody except low-latency traders simply don’t have to worry about it anymore.

When it comes to low-latency traders, we sometimes have to have some conversations about tuning. But with everybody else, they stop even thinking about the question. Now, that’s been the state of Zing for a while now, but the nice thing for us with Falcon and the LLVM compiler is we get to optimize better. So because we have a lot more freedom to build new optimizations and do them more rapidly, the velocity of the resulting optimizations is higher for us with LLVM.

We’re able to optimize around our garbage collection code better and get even faster code for the Java applications running it. But from a garbage collection perspective, it’s the same as it was in our previous release and the one before that because those were close to as perfect as we could get them.

Cameron McKenzie: Now, one of the complaints people that use JVMs often have is the startup time. So I was wondering if there’s anything that was new in terms of the technologies you put into your JVM to improve JVM startup? And for that matter, I was wondering what you’re thinking about Project Jigsaw and how the new modularity that’s coming in with Java 9 might impact the startup of Java applications.

Gil Tene: So those are two separate questions. And you probably saw in our material that we have a feature called ReadyNow! that deals with the startup issue for Java. It’s something we’ve had for a couple of years now. But, again, with the Falcon release, we’re able to do a much better job. Basically, we have a much better vertical rise right when the JVM starts to speed.

The ReadyNow! feature is focused on applications that basically want to reduce the number of operations that go slow before you get to go fast, whether it’s when you start up a new server in the cluster and you don’t want the first 10,000 database queries to go slow before they go fast, or whether it’s when you roll out new code in a continuous deployment environment where you update your servers 20 times a day so you rollout code continuously and, again, you don’t want the first 10,000 or 20,000 web request for every instance to go slow before they get to go fast. Or the extreme examples of trading where at market open conditions, you don’t want to be running your high volume and most volatile trades in interpreter Java speed before they become optimized.

In all of those cases, ReadyNow! is basically focused on having the JVM hyper-optimize the code right when it starts rather than profile and learn and only optimize after it runs. And we do it with a very simple to explain technique, it’s not that simple to implement, but it’s basically we save previous run profiles and we start a run assuming or learning from the previous run’s behavior rather than having to learn from scratch again for the first thousand operations. And that allows us to run basically fast code, either from the first transaction or the tenth transaction, but not from the ten-thousandth transaction. That’s a feature in Zing we’re very proud of.

To the other part of your question about startup behavior, I think that Java 9 is bringing in some interesting features that could over time affect startup behavior. It’s not just the Jigsaw parts, it’s certainly the idea that you could perform some sort of analysis on code-enclosed modules and try to optimize some of it for startup.

Cameron McKenzie: So, anyways, if you want to find out more about high-performance JVM computing, head over to Azul’s website. And if you want to hear more of Gil’s insights, follow him on Twitter, @giltene.
You can follow Cameron McKenzie on Twitter: @cameronmckenzie

August 16, 2018  1:05 AM

Cybersecurity risk management doesn’t need to be all or nothing

Daisy.McCarty Profile: Daisy.McCarty

Cybersecurity risk management should be a concern for organizations of all sizes. New threats and data breaches make the news every few days. But as vendors and cybersecurity risk management consulting firms can attest, far too many companies still lag behind when it comes to implementing safeguards. In part, this is due to the fragmented nature of the available products and services on the market. Even with options available to solve cybersecurity challenges, however, businesses may not know where to start.

So many options, so many gaps

Tulin Sevgin, cyber risk management specialist with InConsult, has found it difficult to come up with comprehensive protection for her company’s clients. Like most risk management consultancies, InConsult wasn’t looking to become a technology firm. But it could hardly ignore the pressing need for cybersecurity risk management as part of the total picture. The race was on to find a vendor that could best serve its clients. Sevgin took this search seriously.

“Instead of developing our own product from scratch, I went to the market to see what was out there, what our competitors were doing, and what I could do differently to give us an edge,” Sevgin said.

She discovered that there were plenty of vendors in the space, but most were aiming at solving the same handful of problems.

“There are a lot of companies out there that do penetration testing,” Sevgin said. “But there aren’t that many doing things like vulnerability management, cloud scanning, external APIs and website scanning, and then also scanning the internal environment to see where your weaknesses are.”

Cybersecurity risk management choices

Instead of finding three or four vendors who specialized in these different areas, her goal was to get it all in one place. And eventually, they found a company that did it all and teamed up with them.

The selected vendor provided security across all the following areas:

  • Third-party vendors
  • Externally-facing websites and APIs
  • Networks and applications
  • Servers and clouds
  • Personally Identifiable Information (PII) and sensitive business data

That’s quite a lineup. Of course, not every business needs to pay for every possible type of security. However, there could be an advantage to working with a vendor or consulting firm that understands the full scope of what’s available to help determine the right direction. It all begins with an accurate assessment.

Where to start

Start with a plan. Determine the potential risks, the possible fallout, the budget available to shore up security, and the risk tolerance of the organization. For example, a public utility responsible for critical infrastructure requires a high level of cybersecurity, whereas a local business has much more modest needs.

According to Sevgin, companies don’t have to look far—or even pay anything—to get started. Free resources are readily available.

“For best practice purposes, the NIST framework is good to look at,” Sevgin said. “These are great guidelines, not the kind that you need to implement from beginning to end. You can choose what’s most effective to address your weaknesses in a way that fits your organization.”

The National Institute of Standards and Technology (NIST) espouses the well-known five-factor approach to cybersecurity:

  1. Identify: Understand the business context, resources tied to critical functions, and potential scenarios.
  2. Protect: Develop and implement safeguards to ensure delivery of critical services, limiting the impact of a potential incident.
  3. Detect: Ensure the ability to identify cybersecurity events in a timely manner through activities such as continuous monitoring and anomaly detection.
  4. Respond: Determine what will happen in the event of a detected cybersecurity incident, including appropriate technological, business activity, and PR responses
  5. Recover: Put plans in place for resilience and restoration of any capabilities or services impaired by a cybersecurity incident.

NIST recommends mapping the security requirements uncovered by this assessment process with answers already on the market. Interestingly, the institute also recognizes the common difficulty of finding it all in one place. “The objective should be to make the best buying decision among multiple suppliers, given a carefully determined list of cybersecurity requirements. Often, this means some degree of trade-off, comparing multiple products or services with known gaps to the Target Profile.”

Shift attitudes toward cyber risk management

In Sevgin’s experience, there are several misconceptions that hold businesses back from taking adequate steps toward a more secure cyber environment. Companies that have not yet been breached may feel invulnerable.

“They say, ‘Why do we need it? We’re fine, we’ve never been breached,’” Sevgin said. “They see cybersecurity risk management as something complex and technical, like the money spent on it is just going into a black hole. Or they just assume that IT has it all covered.”

But that complacent attitude is beginning to change. Folks from senior management down to the operational level are starting to appreciate cybersecurity.

“When these compliance obligations came in like GDPR, it pushed them to find out what’s going on with their cybersecurity,” Sevgin said. “I think we’ll see a cultural shift in the next year or two causing the business to think about cybersecurity as part of their everyday job rather than just relying on IT to do it.”

An exercise in assessing risk

Sevgin offered key advice for the first cybersecurity exercise companies should go through. It’s an approach that entails exploring the worst-case scenario by putting together a data breach response plan.

“How you deal with a breach is very important because getting it wrong leads to reputation damage internally and externally,” Sevgin said.

So what does the process entail?

“It’s easy to do and doesn’t require a lot of money. Once you start writing that plan — and you can get a template from a consulting company or government website — you see how it fits into your existing business continuity and crisis management plan. It really forces you to think about decisions that need to be made on the spot if there is a data breach. The next step is to do a tabletop exercise to put that plan to the test.”

The data breach response plan determines how you manage the incident, the potential reputation damage, and regulatory compliance. Once businesses start writing a plan, they see how it fits with business continuity as a whole. They may also realize their current precarious risk status and recognize they probably don’t have a handle on all their data.

“They start asking questions,” Sevgin said. “‘What data do we have? How much of it is critical or sensitive?’ That’s the time to do a data mapping project to figure this out and lock it down.”

The greatest obstacles to cybersecurity risk management is still a lack of awareness.

“Stay open-minded and don’t be afraid to educate yourself and ask questions so you can understand,” Sevgin said.

That’s a small price to pay when the risk of doing nothing is so high.

August 16, 2018  12:16 AM

Cloudfare puts early mobile network performance in the hands of devs

George Lawton Profile: George Lawton

Application developers can draw on a number of static code analytics tools such as Checkstyle, PMD and SonarQube to identify various application issues and bugs. But these tend to focus on application logic, not mobile network performance. However, mobile networks often suffer from a number of different types of degradations that affect application code in different ways. To address this gap, Cloudflare, an Internet performance service released the Cloudflare Mobile SDK, as a free tool that helps to shift DevOps left by allowing developers to bake mobile network performance awareness into the development phase.

“Our users never complain about poor connectivity; they complain our app doesn’t work,” said Ben Yule, CTO of Spire. “After integrating the SDK, my engineering team has visibility where previously we had none. Cloudflare’s Mobile SDK delivers the missing link in app experience monitoring.”

Apps can fail or slow for a variety of reasons including app code, network calls to the cloud, and server issues. Tools like
like Crashlytics, Rollbar, Sentry, Instabug, and Bugsnag can help identify problems in the mobile app or the server. The new SDK makes it possible to correlate these with mobile network performance issues. Developers can turn on the tool using a simple check box for iOS and two lines of code for Android. It currently supports Objective-C and Java.

New mobile network performance approach

Cloudflare bought Neumob last year, which had developed a replacement for the TCP stack that used UDP to optimize network communications for mobile. Neumob had also developed a mobile network analytics tool that gave developers better insight into the impact on mobile networks on application performance. This made it easier for developers to ascertain how certain code patterns are caused by mobile networks issues.

This could include slow networks, intermittent connections, or switching between mobile and Wi-Fi. Developer can use these tools to identify precise code blocks associated with a particular kind of mobile degradation, which can help guide better code patterns to ameliorate a specific problem.

Matthew Prince, co-founder and CEO of Cloudflare, said, “We looked at that and thought this was an interesting tool that gives developers a view into how the network affects apps in a way they did not see before. We did not think it was something to be sold. So, we decided we would provide it free of cost to mobile developers.”

Mobile network performance monitors

A side benefit is that it will make it easier to measure actual wireless connectivity around the world. All cellphone providers make boastful claims about raw speeds and publish coverage maps. Theoretical speed sounds good on paper but often do not reflect the reality that service can drop, particularly in urban canyons and during peak hours.

“Every cellphone provider says they are the best, but no one has given the hard data,” Prince said. “Our plan is to get samples from apps and we will publish a report about what providers provide the highest quality of service.”

The SDK runs asynchronously in the background. As a result, Cloudflare claims it does not slow down app performance or cause any significant impact on battery usage. Also, the tool does not ship back IP addresses, IMEI numbers, which might otherwise present privacy concerns. “We think of user data as a toxic asset,” said Prince. “We are not in the business of collecting PII data.”

One of the challenges Neumob had was that enterprises did not know how much of their network performance impacted things like conversion. Prince said, “If we can expose how much network errors lead to slower app performance and conversion that feeds the rest of the business, more companies might consider using alternatives to TCP like Neumob.”

Download the SDK here:

July 19, 2018  6:10 PM

The secret history behind the success of npm and Node

George Lawton Profile: George Lawton

When JavaScript was first introduced, it was a programming toy that sort of worked, but worked differently across browsers. A few pioneers, though, saw the potential this language could have on the back-end as well, where it has transformed the application development landscape. We caught up with CJ Silverio, CTO of npm, who explains why npm and Node have grown incredibly popular. Not only can they build better back-ends but they’ve allowed for a new ecosystem of JavaScript frameworks with React on the front-end as well.

What were the factors that drove the success of NodeJS?

CJ Silverio: Around 2009, our founder Isaac Schlueter was excited about server-side JavaScript. About the same time, Ryan Dahl and Node.js were starting to make an interesting run into this area. The idea was to pull JavaScript out of the browsers and run it on computers outside of the browser context. Dahl wanted to write a fast Web server on JavaScript and ended up creating a useful general platform as a result. Schlueter got excited about this, and he realized was that there needed to be a way to share code.

Ruby has RubyGems, Python had Python eggs and Perl had CPAN. Schlueter started working on a package manager for Node. At the time, it was one of several competing package managers for JavaScript. But it happened to have the right feature set and also was written in Node.

So, for years as Node grew, npm became a way people published JavaScript to a shared repository, the npm registry, to build a program. It described a package using a JSON manifest for pulling the things you wanted from the npm registry as shared code you could use to build software.

What was different about npm compared to other JavaScript package managers?

Silverio: It introduced two innovations around semantic versioning and dependency management.

How did semantic versioning work?

Silverio: Packages all have versions. Semantic versioning is like a pinky swear among programmers about what they intend the version to contain. A package has three numbers. The last is the patch number. You pinky swear, “I think I’m fixing a bug.”

The middle version number implies, “I think I’m adding a feature.”

The first number is a major version update which asserts, “I’m breaking you.”

This is not a panacea, but this is just enough communication among programmers about what their intent is. You break people by accident all the time, but semantic versioning lets you signal it.

This turns out not to be a solution to everyone’s problems, but it is enough grease on the skids that it helps.

How did npm address dependency challenges?

Silverio: Dependency hell is a term people throw around. For example, you are trying to depend on one piece of software and something else you depend on also depends on a different version of the first piece of software. This is a much harder problem in Java, C, and C++. Anything that is statically compiled tends to have this problem, which basically mandates there will only be one version. For example, if I want to use OpenSSL, it’s not possible to have two things that require different versions of it in statically compiled languages.

Node includes a mechanism for working around this by nesting dependencies. It does not matter what version they want. Two modules can call different versions of a package, and it is fine they coexist.

How did these catalyze back-end development?

Silverio: These two things were useful enough for people to begin sharing packages on top of npm, and Node started to succeed. Schlueter was working on both npm and Node at the same time. As Node began succeeding, people started writing more JavaScript and sharing more of it as well.

And then the browser developers, who really write most of the JavaScript in world, found that Node would let them use the right tools in the languages they knew well to write websites better.

That is when npm and Node went kaboom.

JavaScript is a massively popular programming language, because it is in the browser. It is a tool that makes writing things in JavaScript easier and can share the JavaScript you were writing, so web developers started using npm a lot.

Our COO, Laurie Voss is into numbers and enjoys producing graphs of npm’s uptake and use. It is not exponential, but it looks exponential.

npm users downloaded 5 billion packages in the last seven days. Four years ago, it was fifty million in one week. There are a lot of developers that share a lot of JavaScript and have developed habits that assume npm is available as a resource. Anytime you’re developing, doing automated testing, running build environments, they will pull in new packages.

There were 50 billion Tarballs downloaded in the last week. The majority was probably continuous integration testing. When people are actually deploying a new app, they will download it once to make a new build.

How do bundling browser apps and compiling back-end apps compare?

Silverio: Webpack and Browserify are used to bundle packages together for browser apps in a way analogous to compilers for other kinds of apps. All browsers run JavaScript but in a different way than people tend to develop programs. Developers make little modules with lots of little files. When you deploy to a website, you have to shove everything into a single file. People use bundling tools to do this. A program called Browserify was one of the first to do this.

The one everyone uses today is webpack. Another is called Rollup. Bundlers do a function that parallels what compilers do. They take source code in one form that is convenient to developers and turn it into the form best suited for browsers to consume.

Browser makers are doing well right now in terms of compatibility among JavaScript implementations. Webpack does not have to make a build for Microsoft edge then for Google Chrome. This same compiled JavaScript will run on both of those. This is one reason that JavaScript development is flourishing. There really is one JavaScript, and one browser object model they are all using.

It is just that the Node servers are different than browsers. This is okay, because they are different things. You end up wanting to use the conveniences of Node when you are developing.

What is the secret horror of working with NodeJS?

Silverio: The secret horror of NodeJS being server-side JavaScript is that the two languages are the same, but the platforms are different. Running a Node program is almost the same as running JavaScript in the browser. But the environments are different. Node can read and write to a file system. For example, you can have native bindings to Node for a computer vision library. These are things that you cannot do on a browser.

This program might not run in the browser. It might run after you do some transformation to make all the nice things you get from Node available in the browser equivalent. That is what tools like webpack do.

How does npm make money from giving away free access to a package manager?

Silverio: We sell a version of our registry you can run on-premise, and soon we will announce this as an npm registry as a service. We also sell a security product which is another thing that matters to a lot of people. The downside of open source is that it also includes a lot of insecure libraries. npm is a place where people can share JavaScript code. The next question that I as a CTO ask is “where did you get that code from and is it safe to use?”

npm acquired ^Lift Security and these people have been experts behind Node security projects since its inception in 2013. This has grown our expertise on JavaScript security for finding and fixing JavaScript security and making decisions about whether what you are using is safe. We have a free version where every person using npm will get access, and when they install something using npm, it will tell them how to fix it.

Deeper analysis and reporting is available as a paid offering. It can also do more analysis over time across projects. For example, we could tell you when a particular vulnerability entered your application, and what your exposure is over time.

I’ve developed Node apps that ran for years. The fact that people are developing websites and development is constant.

How has the mix of front-end and back-end developers shifted?

Silverio; We have about 11 million developers using npm. Most of those are front-end browser developers using Node to do it. Many are using Node but sharing code that might have nothing to do with Node.js. For example, React is hosted on npm. If you want to get the latest version of React, you will run and install it using npm, Yarn or a CLI command that talks to the npm registry.

I refer to it as success as a catastrophe. You have a startup and now you have succeeded. npm has survived a catastrophe of success, because we have made really good use of our CDN.

Currently, npm usage is about 80:20 front-end to back-end developers. Two years ago, it was about 50:50. Node is critically important and is being widely used to write single page web apps.

npm being where it is and as usable and fast as it is has enabled people to do new methods of web development they could not do before. By allowing the plumbing to be taken for granted by developers, it has allowed people to build things on us that were not thinkable before.

When you got JavaScript by doing Google searches and going to a jQuery webpage, you could not build complex things from many small modules, because there was no easy way to find them and assemble them in a reasonable way. But tools like npm and Node make that not necessarily straightforward, but at least possible.

Semantic versioning is not perfect, but most of the time it works. People can just take the software with confidence and npm retrieves the components for them automatically.

July 17, 2018  6:52 PM

Get started with GoLang maps: A tutorial

WalkerRowe WalkerRowe Profile: WalkerRowe

An important part of programming with GoLang is working with maps. In this GoLang maps tutorial, we’ll demonstrate the value of maps and show you how you can integrate them into your programs.

What are GoLang maps?

A map has a key and a value. The value can be any value, such as another map. For example, here is a map of strings with a key of type int and value string. You use the make command to instantiate the map.

people := make(map[int]string)
people[1] = "Walker"

Here is a complete example. In this case, we’ll declare and then instantiate the GoLang map in two steps. Then, we use the GoLang range operator to loop over each key and retrieve the value by index. Then we print it out.

package main
import "fmt"

func main() {
  var pLangMap map[string]string

  pLangMap = make(map[string]string)

  pLangMap["Go"] = "Easy"
  pLangMap["Python"] = "Easy"
  pLangMap["Scala"] = "Difficult"
  pLangMap["Java"] = "Moderate Difficult"
  pLangMap["C++"] = "Difficult"

  for lang := range pLangMap {
    fmt.Println("How Difficult is", lang, pLangMap[lang])


Here is the output. Notice that the keys are printed out of order. This is because Golang is multithreaded and asynchronous.

How Difficult is Scala Difficult
How Difficult is Java Moderate Difficult
How Difficult is C++ Difficult
How Difficult is Go Easy
How Difficult is Python Easy

Sorting GoLang maps

To sort the map items, we cannot simply use sort.Strings since the key is not a string.


cannot use pLangMap (type map[string]string) as type []string in argument to sort.Strings

So let’s make a new map whose key type is an array of strings []string and has an arbitrary value: we use len(map) which is the number of elements in the map. Then, we plug the value into the key and sort on the key with sort.Strings.

func main() {

  var pLangMap map[string]string

  pLangMap = make(map[string]string)

  pLangMap["Go"] = "Easy"
  pLangMap["Python"] = "Easy"
  pLangMap["Scala"] = "Difficult"
  pLangMap["Java"] = "Moderate Difficult"
  pLangMap["C++"] = "Difficult"

  mk := make([]string, len(pLangMap))
  i := 0
  for k, _ := range pLangMap {
    mk[i] = k
  fmt.Println("How Difficult is", mk)

Now the elements are printed in order.

How Difficult is [C++ Go Java Python Scala]

Retrieve Value

Here is how to retrieve a map value.

pLangMap["Go"] = "Easy"
x := pLangMap["Go"]
fmt.Println("How Difficult is", x)

Test for a Key

To test for a key, use the underscore (_) to retrieve a value without using it. If we put some placeholder there like, for example, v, then Go would complain that we are declaring a value but not using it. If the key is not there, then the boolean ok will be equal to false.

_, ok := pLangMap["Got"]

if ok == false {
  fmt.Println("Got not found")

Map of Structs

Here we create a map of a struct of type students then print each student name:

package main

import "fmt"

type student struct {
  name string
  class string
  grades string

func main() {

  var students map[int]student
  students = make(map[int]student)
  students[1] = student{"Walker", "calculus", "A"}

  for k, v := range students {
    fmt.Println("key=",k, " name=" ,


key= 1 name= Walker

As you can see, GoLang maps are incredibly easy to work with, and it’s a simplicity that extends throughout the language, which is why GoLang is becoming an increasingly popular web programming language.

If you are interested in following along with this GoLang maps tutorial, you can paste the code provided below into the GoLang Playground.

July 6, 2018  8:34 PM

Embed the WebSphere Liberty profile in an executable JAR with Maven

cameronmcnz Cameron McKenzie Profile: cameronmcnz

Perhaps is has something to do with my long history of working with the WebSphere Application Server, but I’ve found myself favoring the WebSphere Liberty profile whenever the opportunity arises to embed a servlet runtime within an executable JAR file. In this WebSphere Liberty profile tutorial, I’d like to demonstrate just how easy it is to take an existing Java web app, deploy it to the WebSphere Liberty server and package the entire thing up in a runnable JAR file.

Eclipse Maven webapp wizard

Figure 1. Create a simple Maven webapp in Eclipse.

The web app to deploy to WebSphere Liberty

The first step in this process is to actually have a Java web app to deploy. I used Eclipse’s New Maven Project wizard to create a simple webapp with nothing more than a humble JSP that spits out the date. This isn’t a complicated RESTful web service, or a Spring Boot MVC application, but since this WebSphere Liberty tutorial is focused on creating an embedded JAR, and not the fundamentals of web application development, it doesn’t have to be.

  <h2>Hello WebSphere Liberty Profile World!</h2>
  If this prints the date, the executable JAR file is working!
  <%= new java.util.Date() %>

The generated application also contained a web.xml file, which isn’t needed in modern Java web apps, but Maven generated a simple one for backwards compatibility.

"-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
"" >
<display-name>Embedded WebSphere Liberty Example</display-name>
Embedded WebSphere Liberty project structure

Structure of the WebSphere Liberty profile example project.

The Open Liberty Maven plugin

With the JSP and deployment descriptor created, the last thing to tackle in the process of embedding WebSphere Liberty inside of an executable JAR is the POM file. The Maven POM has three significant parts to it that are noteworthy:

  1. The preamble generated by Eclipse
  2. Configuration of the maven-war-plugin
  3. Configuration of the liberty-maven-plugin

The Maven preamble

The Eclipse generated part of the Maven POM file specifies that the application will be packaged as a deployable war. This part of the POM also indicates that the artifactId is openliberty and the version is example, which means the web application archive that gets generated will be named openliberty-example.war. The POM file as it was originally generated by the Eclipse Maven tooling is as follows:

<project xmlns=""

  <name>embedded WebSphere liberty example</name>


The Maven WAR plugin

The Maven war plugin is needed package the application.

Historically, WAR files have required a web.xml file, but that necessity expired with the release of the Servlet 3.0 API. However, the Maven WAR plugin barks if the web.xml file does not exist unless the failOnMissingWebXml configuration attributes is stet to false. Since my application does contain a deployment descriptor, I could have left this setting out, but I figured I’d leave it in for the sake of anyone trying to deploy a modern web app on an embedded WebSphere Liberty profile and wasn’t aware of this optional setting. The configuration for the first plugin, which appears in the Maven POM file after the name tag and before the closing project tag is as follows:




Configuring the Open Liberty Maven plugin

Up to this point in the tutorial, I have been talking about WebSphere Liberty. But the plugin that will be used here is the one available for Open Liberty. Open Liberty is the open source version of the WebSphere Liberty server, which can be used without the licensing requirements surrounding the IBM’s flagship WebSphere Liberty profile offering.

The Open Liberty plugin entry, which is placed immediately after the closing tag of the maven-war-plugin, is as follows:


There is a lot of meat to the Open Liberty Maven plugin conguration, but these are the highlights:

  1. Version of the Open Liberty profile will embedded
  2. The executable JAR will be named embedded-liberty.jar
  3. Maven goals executed during the prepare-package and package phases will download the Open Liberty server, deploy the Java web app, and package the server and the deployed app in the executable JAR.

The executable Open Liberty JAR file

With the Maven POM configured, the executable Open Liberty JAR file can be created simply by running a maven package command:

C:\webphsere\embedded\openliberty>mvn clean install
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25.016 s
[INFO] Finished at: 2018-07-06T14:22:21-04:00
[INFO] ------------------------------------------------------------------------

Running the embedded Open Liberty JAR file

To run the executable JAR file with Open Liberty embedded, simply move to the \target folder in which the embedded-liberty.jar file was created and issue a java -jar command.

C:\webphsere\embedded\openliberty>java -jar embedded-liberty.jar
(default_host): http://localhost:9080/openliberty-example/
Application openliberty-example started in 0.458 seconds.
The server installed the following: [jsp-2.3, servlet-3.1, el-3.0].
The server defaultServer is ready to run a smarter planet.

When the embedded Open Liberty server is accessed from a browser through the assigned URL, a web page opens up that displays the current date, along with a Hello World message, as per the JSP that was created as part of the application.

Running the embedded OpenLiberty application

Running the embedded Open Liberty server.

And that’s all there is to it. As long as your POM is configured correctly, and your application is a valid Java EE web application, deploying it to either an embedded WebSphere Liberty or an embedded Open Liberty server is a relatively painless endeavor.

The full POM, along with the code used in this embedded WebSphere Liberty profile example can be found on GitHub.

Complete Maven POM for embedded Liberty server

<project xmlns=""
  <name>embedded WebSphere liberty</name>








July 1, 2018  5:29 PM

Surprise results stem from curious data analytics models

JasonTea Profile: JasonTea

Nobody can argue that data analytics models will change the world on a massive scale, as the solutions they produce have the capacity to impact millions of people on a daily basis. Data analytics models are all about the ability to find a hidden pattern or the desire to follow through on a perceptive hunch, so many projects often start off with just a kernel of an idea. While at Arizona State University (ASU) pursuing an MS in Business Analytics, Neeraj Madan set out to demonstrate how the Urban Planning, Parking and Transit Services, along with the local police department, could use data analytics models to make more effective decisions to curb bike theft.

Real world data analytics models

Madan’s description of how data science should be applied boiled everything down to the basics. “While the data science field is evolving at a rapid pace on the technology front, the whole field can be explained in one word: Patterns. It’s all about learning and making use of analogs (positive patterns) and anti-logs (negative patterns) to enable effective decision making.”

What decisions could be impacted with better insight into bike theft patterns? “If you have to plan a patrol route, we can tell you when and where the crimes are typically happening.” Having a visible and consistent police presence could play an important role in reducing bike thefts in hot spots. In the case of adding more secure options for bike owners, the data could also be used for urban planning. When city officials could clearly visualize the areas of high demand for bike parking and high risk for theft, it was easier to see where the next secure parking solution should be implemented.

As with many college campuses around the country, bike theft was perceived as a persistent issue at ASU. Most of the bicycles were locked with a portable anti-theft device when stolen, so the use of basic locks was obviously not addressing this type of crime. Over $150,000 worth of bikes were taken on an annual basis, and only a fraction of this equipment was ever recovered. This problem wasn’t going to be solved without better information. Madan set about gathering the data to explore the problem in detail.

Evaluation of data analytics models

Key factors for evaluation included geography, time of day, and property being stolen. As he sifted and analyzed the data, it became clear that most of the crime was clustered in a few key areas. In fact, 40% of the overall value of property stolen over a five year period was associated with the areas surrounding just ten buildings. Analysis highlighted peak hours and days when thefts were most likely to happen. This information coupled with additional data gave Neeraj the confidence to propose a potential site for new valet bike parking.

Madan had several objectives when gathering and analyzing data: Identifying bike theft hot spots, measuring the effectiveness of existing valet and secured storage options, and proposing a new valet location for a new walk only zone. He took things even farther by exploring the marketplace for stolen bikes (such as popular online venues where these goods were sold). “We would look at Craigslist phone numbers to find patterns. Then, we handed that information over to the police for investigation.” For example, if the same phone number kept showing up over and over with postings to sell various used bikes across cities, it could indicate that the user was acquiring their inventory in a suspicious manner. Or, if a dealer was also listed as an owner under the same number, that could be a red flag.

In addition to recommending construction of new valet parking, targeted patrols of hot spots during peak times for bike theft, and investigation of popular venues for selling stolen bikes, Madan addressed the issue of student perceptions. He found that a full 60% of students believed their bicycles would be stolen on campus. Addressing this concern would entail actually reducing the incidence of theft, enabling the police to more effectively investigate these crimes, and educating students on smarter bike management.

For Neeraj, this self-initiated project was an opportunity to showcase a very practical side of the discipline of data science. “My stakeholders never knew this could be a data oriented problem. They needed someone to look at it from a different point of view.” Madan received a formal letter of recognition and appreciation for his efforts. The authorities went on to implement his various recommendations. It’s good to know that a few more eyes were opened to the possibilities of data analytics to create real change one bicycle at a time.

June 27, 2018  5:51 PM

Nexus repository manager tutorial: Get started with the OSS Maven tool

cameronmcnz Cameron McKenzie Profile: cameronmcnz

If you’re doing software development, you’re more than likely deploying to a local Maven repository. The Nexus repository manager tends to be one of the more popular options, and with both and open source version (Nexus OSS) and one that comes with paid support (Nexus Pro), there really aren’t any articulable barriers to adoption.

In this Nexus repository manager tutorial, we’ll demonstrate how to download Nexus, install Nexus, start the Nexus administrative console and even upload a Java JAR file into the Nexus releases repository. A video accompanies this Nexus repository tutorial to fill in any gaps this article leaves unfilled. This tutorial focuses on the OSS Nexus repository – version 2.14 to be exact. Both Nexus OSS and Nexus Pro have version 3 releases available, but those will be discussed in a subsequent Nexus repository manager tutorial.

Nexus repository manager tutorial

So what are the steps a Maven user needs to follow to get started with the Nexus repository’s OSS version? The first thing you need to do is download Nexus. Sonatype’s repo-manager page allows you to download Nexus OSS v2 as a zip file for windows machines, and as a tar file for *nix based Nexus installs. This Nexus repository tutorial will cover a Windows based installation.

When extracted, the Nexus download reveals a nexus-2.14.9-01 folder which contains a bin directory. It’s from within that bin directory that the command exists that can be used to start the Nexus repository manager. Run the following command from a DOS prompt to start Nexus.

C:\tutorial\nexus-2\bin\ nexus console start

Once the Nexus repository manager has started, the administrative console can be accesses by pointing a web browser to the following address:


I actually like the fact that the Nexus OSS repository starts on port 8081 and not 8080. It’s like the tool understands that there’s probably a Tomcat server or a Jenkins CI server running on port 8080, so it does a single digit increment to avoid any conflicts. That’s just common decency if you ask me. It should also be noted that you need the /nexus context root. Nexus doesn’t install itself to the root of the embedded Derby application server.

Default Nexus OSS username and password

The default username and password for the Nexus Maven repository is: admin:admin123. When the Nexus repository manager’s administrative console loads, log in with these credentials.

The whole point of having a local Maven repository like Nexus OSS is to able to upload your own deployment artifacts to it. To do so, select the Nexus releases repository on the Repositories listing page and click on the Artifact Upload tab.

Admin console for Nexus tutorial

First load of the Nexus repository manager admin console.

The first thing you need to do is provide the Maven group, artifact, version and packaging attributes for the deployment artifact you are about to upload. After doing so, use the Select Artifacts to Upload button to browse to the JAR file of interest. Click Add Artifact and then click the Upload Artifacts option, and the resource is uploaded to the local Maven repo provided by Nexus OSS.

Add JAR to Nexus repository manager

Adding the Nexus repository tutorial’s JAR artifact to Maven

Completing the Nexus repository tutorial

When the upload completes, click on the Browse Index tab to view the fruits of all of the labor put into following this Nexus repository manager tutorial.

Maven Nexus repository tutorial completed

Final result of the Maven Nexus repository tutorial.

And that’s all there is to it. Of course, there are more advanced options to upload a JAR file to your local Maven repo. There’s a REST API, there’s a Jenkins Nexus integration plugin that can be used to pull directly from Git, and of course it can all be kicked off by editing a POM and running a deploy goal in a Maven build job. All of those will be covered in subsequent Nexus repository tutorials. But when you’re just getting started with Nexus, installing the product and uploading an artifact is a great place to start.

build things with Maven and the Nexus repository manager

June 25, 2018  11:16 PM

An introduction to Scala functional programming

WalkerRowe WalkerRowe Profile: WalkerRowe

Scala is designed to allow the functional programming style. But what does Scala functional programming actually mean? Let’s start with the fact that functional programs are immutable.

Immutable software

For example, in regular programming you initialize a variable to some value:

x = 1
return x * 2

This will return 2. But the value of the program can change when another programmer checks out that code and changes the value of x.

With functional programming, there are no global variables. Everything is a function. So the output is always the same. In other words:

function (x) return x * 2

Functional programs are written like math statements. Since there are no global variables, everything must be a function. And since everything must be a function, you need to check that these are completely implemented and not applied to invalid inputs. For example, there is the concept of domains, partially-applied functions, and pattern matching. All of these check to see that the program can handle whatever value is passed to the program without causing it to crash.

Let’s look at each of these concepts.


In math, the domain of a function is the range of values for which the function is valid. For example, the domain of the function sqrt (square root) is x >= 0 since the square root of negative numbers is not defined (Well, they are. They are called complex numbers, i. ).

We write the domain of a function f as {f(x): x ∈ R, >= 0}. This says the domain of x is any real number that is non-negative.

We can express this in Scala using partial functions.

In the function sr, if we did not use the word PartialFunction, the compiler would complain that we are not handling all values.

Below we defined a function that takes a Double and returns a Double. We check that the input is bigger than or equal to zero. If it is, then we return the square root. The sr.isDefinedAt() checks to see that the value we want to pass it is in the domain of the function, i.e., is applied.

import java.lang.Math

val sr: PartialFunction[Double, Double] = {
  case d: Double if d >= 0 => Math.sqrt(d)

res7: Boolean = false

Higher Order Functions

Higher order functions let you break down a complex function into steps. If you remember your math from college, these are called composite functions:

That is f(g(x)), which is read as f of g of x.

For example, we can break the statement 2x – 2 into two steps:

g(x) = 2x
f(g(x)) = g(x) - 2

We can do that in Scala, because in Scala functional programming, you can pass functions as arguments. In function f below, g is a function that returns an Int.

def g(x: Int): Int = { x * 2}
def f(g: Int=> Int, x: Int) : Int = {g(x) - 2}
Here is the result.
res11: Int = 2

How to curry a function

Currying a function means to apply nested functions instead of applying the same operation over each item in a collection. One obvious candidate for that is addition.

For example, to add two number you would write:

def add1(x:Int, y:Int): Int = x + y

Run that:

res15: Int = 5

But now we write a new function that takes an Int and returns a function that returns another Int. When you run that function, it saves its value. Run it again and it adds that value to whatever you passed it the first time. So x + y keeps a running sum:

def add2(x: Int):Int=>Int = y=> x + y

Run it two times:

res20: Int = 3

This is easier to understand if you run this just one time:

res16: Int => Int = <function1>

So we can call this function as many times we want to make an adding machine.

Recursive functions

Functions can call themselves. Let’s calculate n! (n factorial) which is:

 n * (n -1) * (n - 2) … For example, 3! = 3 * 2 * 1.

We can take a shortcut and help the function by writing, in this case, 3 as List(1,2,3) instead of just 3, thus handing the Scala function each value it needs.

The notation x::tail tells the program to take the first and last elements from the list. It iterates over every item in the list 1,2,3 including the empty list Nil. And since all functions have values, this one keeps a running tally of the value of factorial, which makes it iterative or recursive.

def factorial(ints: List[Int]): Int = ints match { 
  case Nil => 1
  case x :: tail => x * factorial(tail)

Run it:

res30: Int = 6

This Scala code is easier to understand if we use addition:

def sum(ints: List[Int]): Int = ints match { 
  case Nil => 0
  case x :: tail => x + sum(tail)

Results in:

res33: Int = 6

And while there’s certainly more to learn, that is a quick synopsis of how to do Scala functional programming.

June 17, 2018  12:30 AM

Aging successfully in technology: Addressing the graying elephant in the room

BobReselman BobReselman Profile: BobReselman

Consider this: the median age of a Google employee is 29. At Amazon, it’s 32. On an industry-wide scale, the average age for a software developer in the United States is 28, according to Stack Overflow’s 2018 Developer Survey. In Russia, it’s 25. For India, it’s 23. Given these facts along with Mark Zuckerberg’s remark back in 2007 that “young people are just smarter,” you can make a good case that the world of software development really is no country for old men.

Yet, older workers do get hired. Google has been known to hire 40 and 50-years-olds. Still, the number of 50-year-old developers slinging code is minimal compared to younger age groups. This situation has created a legitimate fear. According to a report from Harvey Nash consultants, 60% of respondents in the UK, ages 50-69, feel that age has a negative impact as they move forward in their career. For those over the age of 60, the number climbs to 80%.

Is this feeling a reality? Consider the following scenario.

It’s hard to beat the odds

Imagine a hiring manager who is considering two candidates for an AWS architect position. Both are certified solution architects. Both have over five years of solid production experience. Both are looking for salaries that fall well within range. Both have great references. One is 35. The other is 61.

Now, let’s pause for a moment and consider some hard facts that might seem tangential, but as I’ll explain in a moment, they matter. According to the National Cancer Institute, a 35-year-old has a 5% chance of developing some type of cancer on the day hired. For the 61-year-old, the chances shoot up to 25%. Strokes are also significantly more prevalent among those approaching and over 65. And, a 61-year-old’s stamina is going to be well below that of the 35-year-old, even if the older person has a great exercise regimen.

Given these facts alone, who is the hiring manager most likely going select? Will the manager take a risk and hire the older candidate who, despite having qualifications equal to the younger candidate, is rife with actuarial risk? Or will the hiring manager play it safe and select the younger one?

I’ll leave it to you to answer that question. I will make an assertion, though, based on experience and research. Past the age of 55, the older you get in tech, the harder it is to find a job when you hit the market. Yes, if you have management experience and or are well-versed in a variety of technologies, the odds improve, but not by much.

Let me make another assertion. If you are a technologist over 55 and looking for head-down, boots on the groundwork, don’t waste your valuable time in an extended job search. The odds are not in your favor. You’ll do better to pursue another path.

The path to self-employment

While it’s reasonable to assert that most tech companies really don’t want to make the long-term commitment that goes with full-time employment to someone over 55, a surprising many will hire just about any competent person in the short-term. Thus, for older workers, a viable way to beat the odds is to go down the path of self-employment.

Keep in mind that self-employment is not the same as doing a startup. Startups are incredibly risky. Only about 10% succeed. And, they require a lot of time — time you really don’t want to waste as you approach 60.

Self-employment, on the other hand, will put a process in place that provides ongoing access to enough short-term work to live well. Instead of making the single, big bet with a startup, you make a lot of little bets each month. Some will pay off, some won’t. The trick is to have more payoff than not.

So, what does the path to self-employment look like?

Self-employment has requirements. You *do* have to understand some of the basics required to run a business. This means understanding how cash flow works, how to use banking services to your benefit and how to prospect for customers. You’ll need to brush up on your bookkeeping. Plan to pay your own taxes. Also, you’ll need to learn how to determine fair market rates to charge for the work you do and then schedule your work accordingly to meet customer expectations.

Put aside working capital now

Successful self-employment takes about two years. There will be ups and downs. The income for your first month out on your own will probably be a lot less than what you’ll make 24 months down the line. Thus, you’ll need savings to cover six months of expenses. Working capital provides the cushion you’ll need to get through the light months.

For those younger and currently employed, the time to put aside your working capital is now. Should you decide to work in tech into your later years — given current and projected trends in job turnover — chances are you’ll be on the path to self-employment as you approach 60. So, budget for the working capital now. You’ll be glad you did.

Keep yourself producing, even when sales are down

There’s a saying among caterers: You’re only as good as your last party. This is particularly true for the self-employed in tech. In order to stay viable, learn continuously and produce what you’re good at regularly, even during light months when sales are down. As long as you are making good stuff, paying work will come along. Make something valuable every day, no matter what!

Tend to your network

A good rule of thumb is that you will get five continuous customers for every 1,000 people in your network. Thus, always keep your network active. Go to meetups, be active on LinkedIn and Twitter and keep in touch with just about everyone you know. Keep the spirit of sharing at the forefront of your interactions. The more people you know, the more people you will know.

The real secret sauce for aging successfully in tech

To quote Bill Gates from a 2011 Q&A:

“I can understand about having millions of dollars. There’s meaningful freedom that comes with that, but once you get much beyond that I have to tell you, it’s the same hamburger.”

Here’s the secret sauce. It’s not the number of burgers you have in your life that counts. It’s the quality of the burger in front of you. The key to aging successfully in tech is to do meaningful work and live really well on the money you have. It’s worked for me. I’m 63. I’m healthy. The refrigerator’s full and I’m successfully self-employed.

June 15, 2018  10:51 PM

Introduction to Scala for enterprise Java developers

WalkerRowe WalkerRowe Profile: WalkerRowe

Here we begin a series of articles to look at Scala.

Scala is a programming language that rides atop the Java virtual machine. In compiles to Java bytecode and can be packaged in Jar files. Scala and Java are so closely related that you can use Java packages in Scala programs. But Scala is not nearly as wordy as Java, which is Java’s chief complaint. But is offers more than just brevity.

Some important products were built using Scala, including Apache Spark and Kafka. The latter was written by LinkedIn. Spark has a Scala command line interface. Twitter also use Scala, says the Scala language organization. Netflix uses it too. Netflix has always been a company that gives away a lot of their code. For example, here is the Scala Atlas API on Github which Netflix says is a “backend for managing dimensional time series data.”

Some have said that Scala is difficult to learn because it is a functional programming language. Briefly that means it can look more like mathematical statements than the if-then-else style of programming that most people are used too. There is even a word for making Scala, or any, code smaller and tighter thus functional programming-like: sugar syntax.

But you are not forced to write Scala code that is so terse that other people cannot read in.

Running Scala Code

Scala code can be run two ways. You can create a file and then run the command scala That is because Scala is an interpreted language interface as well as a compiled one. So it also has a command line shell into which you can type commands directly and have them evaluated on the spot, aka a REPL (read, evaluate, print, loop), like Python. But for a program that uses external packages, you would want to use sbt, which is the Scala build tool, to build a Jar file. So sbt is like Maven or Ant.

Basic Examples

Scala has maps, tuples, arrays, sets, lists, and something called options.

Below we declare a List with three elements. Notice that there is no need put end the statement with a semicolon (;), but you can. Also note the use of the word var to declare a variable. If we wanted to declare an immutable variable (i.e., one that cannot change) we would have written val.

In the interpreter, the first line we typed below is the code. The second line is the response, or Scala acknowledging what we just entered.

var n = List(1,2,3)
n: List[Int] = List(1, 2, 3)

Note above the Scala says that n is a List of Int. You can be explicit when you define variables such as:

variable: type


A list is an iterable object, or a collection. That means we can run a map operation over it, meaning perform some operation over each member. Scala facilitates this by letting you implicitly declare variables inline and use an inline function all in one step.

For example, below the x => syntax means create a variable x and pass it into a function. Then we multiply x by x in the function. We picked x here. We would have picked any letter.

var m = x => x*x)
m: List[Int] = List(1, 4, 9)

We can also use the items in the List to make another object, say a tuple.

First we declare a list:

var words = List("apple", "apple", "orange")

Then make tuples (x,1):

var grp = => (x,1))
grp: List[(String, Int)] = List((apple,1), (apple,1), (orange,1))

You add 1 to a tuple when you want to write the HelloWord-type WordCount function.

Introduction to Scala Maps

A map is a (key,value) pair.

var grp:Map[String,Int] = Map(("a",1),("b",2),("c",3))
grp: Map[String,Int] = Map(a -> 1, b -> 2, c -> 3)

Here, for example, are the keys.

res46: Iterable[String] = Set(a, b, c)

Use Java Classes

We can use Java classes with Scala:

import java.lang.Math

res20: Double = 3.141592653589793


Scala functions have the form:

def functionName (parameter: Type) : returnType = { code }

as in:

def square (x : Double) : Double = { return x*x }

You can leave off the word return. Then the last value calculated is what is returned.


Classes are just like Java classes, with member functions and variables. Scala uses Traits, which are like Java interfaces and abstract classes.

class Cat (var breed: String) {

def whatBreed(): String = { return breed }

var alleyCat = new Cat("alley cat")
alleyCat: Cat = Cat@637bf67c


res15: String = alley cat

Notice that functions have value, so if you do not assign one explicitly Scala will do that by itself. Here it as created a new value res15. When we ask Scala is res15 gives than and assigns yet another new value res16.

res16: String = alley cat

Because functions have value, we can pass them to other functions. We will illustrate that in another post.

Introduction to Scala Traits

Traits are like Java interfaces or abstract classes. A class can extend one more multiple traits.

trait Coffee {
def isPricey(price: Float): Boolean = { (price > 100) }

class GoldenBlend extends Coffee {

Now create an instance of Coffee. It inherits the isPricey method:

var coffee = new GoldenBlend()
coffee: GoldenBlend = GoldenBlend@517c6a67

scala> coffee.isPricey(200)
res44: Boolean = true


And that’s a quick introduction to Scala for enterprise Java developers. There is a lot more Scala to learn, with functional programming topping the list. It’s a topic we will broach in the follow up tutorial.

Page 1 of 2312345...1020...Last »

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: