Your organization launched a daring new product. It wants fast feedback from customers, so it signs with a company that does customer satisfaction surveys.
Quick survey: Would you trust the results?
It may seem that now — just weeks after political surveys and resulting analysis largely missed Donald Trump’s victory in the U.S. presidential election — is a bad time to ask. But with more organizations using data analysis to make critical business decisions, I’d argue it’s a great time to ask.
Harvard University political scientist Gary King said companies can learn a valuable lesson from what pollsters got wrong: If they want to buy services from a survey research outfit, they should ask tough questions about how it does its surveying.
“I would ask, ‘How do you know that you’re going to get a right answer?'” said King, who is also the director of Harvard’s Institute for Quantitative Social Science. “‘Give me some evidence that what you’re doing makes sense.'”
Beyond customer satisfaction surveying, survey research services include market research, to gather information about markets or customers, and new-product research, to determine whether a certain product or service will satisfy a certain need.
A serious problem with the political side of survey research today, King said, is this: Pollsters can’t easily get a representative sample of voters — that is, a small group of people who reflect the larger population. Lots of people use mobile phones only, and they often don’t want to respond to polls.
The nice thing about polling during elections is, after pollsters gather results of their survey — which point to one result or another — come Election Day, they learn the truth, King said.
“If the survey went wrong somehow or in some way small or large, then we get to know,” King said. “That’s actually a great thing for the survey research team because they get to learn something.”
Pollsters who’ve been proven wrong after the election — and that’s most of them — now have a great opportunity to improve their data collection and ultimately their forecasting prowess, King said.
Companies doing product or market research will have, he hopes, done the same: stumbled early on and then learned from their mistakes.
“There’s plenty of analytics where nobody learns anything; you just do it. And that’s not very good,” King said.
Rising use of mobile phones means pollsters have to rethink how to survey voters during elections. Read about it in this SearchCIO news story.
Marc Tanowitz, a fan of robotics process automation, would be the first to say the term has become a business buzzword.
“People plunk the word robotics in front of anything that is automated, because it sounds good,” he said in an interview with SearchCIO earlier this year. Indeed, the robotics process automation label has been slapped on technologies ranging from the industrial robots found on factory floors to an app that alerts you when an item on Amazon goes on sale.
As managing director at the IT advisory firm Pace Harmon, however, Tanowitz has made it his business to separate the RPA buzz from the benefits of this new technology. And, having advised clients on use cases for robotics process automation (aka RPA) for the past two years, he has some advice for CIOs: Get on it, already — or be prepared for another tussle over shadow IT.
“What is happening in the enterprise is that it is not IT that is driving automation; it is the business functions,” he said. “Not unlike what we saw with the business functions investigating SaaS and cloud platforms without the knowledge of the CIO, we see the same thing happening with robotics process automation.” Tanowitz advised CIOs to look at RPA as another enterprise collaboration tool that can help the business functions operate more efficiently.
Four tips for deploying robotic process automation
Robotics process automation is software that replicates how humans interact with the user interface of computers. The tools are lightweight, requiring minimal coding and little of the heavy lifting associated with other types of business software used for automating enterprise work — ERP implementations, for example, or business process management (BPM) suites.
Much like cloud was a few years ago, the applications are touted as business friendly but, like enterprise cloud applications and deployments, RPA projects need CIO oversight.
Here are four pointers from the Pace Harmon team to keep in mind as you embark on RPA. Go here for a link to the full article.
- Robotics process automation is not a replacement for traditional IT projects: RPA can be a good solution for automating a manual activity that is rules drive, data intensive, repetitive in nature and crosses multiple systems and decision points.
- Business and IT must collaborate for robotics process automation to succeed: Process owners often underestimate the need for IT involvement in RPA deployments, resulting in data security risks, latency issues and redundancy with other IT apps. IT and the business should develop a two-to-three year road map for RPA implementations to avoid these problems.
- Robotics process automation benefits go beyond direct cost savings: RPA also mitigates risk of human error, improves quality and frees up employees to focus on higher value work, thus improving job satisfaction and the reducing employee attrition.
- Robotics process automation will have an impact on outsourcing strategies: The immediate focus of RPA is to automate high volume, repeatable tasks. Next-generation RPA incorporates machine learning and natural language processing technology that will be used to automate more complex tasks. Outsourcing strategies that have relied on labor arbitrage to deliver business process savings need revising in light of the benefits offered by RPA.
One of the key attributes Curt Carver, vice president and CIO at the University of Alabama at Birmingham, looks for in a candidate is emotional maturity. But for some leaders, determining which candidates have the soft skill — and which don’t — can be illusive.
Carver, who served as an officer in the U.S. Army for more than 25 years, uses a couple of litmus tests to make the determination. The key behind both tactics? “I try to create opportunities for authentic and genuine conversations with the shields down to make sure they’re a good fit for the organization,” he said.
Get out of the office
Carver doesn’t perform interviews around a big table. Instead, he takes advantage of the university setting and invites the prospective hire on a walk around campus. The stroll creates a more casual setting, with Carver giving a tour along the way.
“It tends to lower their guard and you get a better sense of who the person is as we walk and talk about various topics around campus,” he said.
The answer to this question is telling
Carver uses this question at the end of every interview: What was the question you thought I was going to ask that I didn’t ask and what would be your answer to that question. “It throws people off, and the genuine person tends to come through,” he said.
The responses can be surprising. Those with emotional maturity talk about how they built a successful team. They talk about a significant accomplishment and why they’re proud of it — but eventually they turn back to a team endeavor or a moment when they empowered someone else or shattered a glass ceiling. “Those are examples, I think, of folks demonstrating empathy, demonstrating emotional maturity, talking about a difficult scenario or situation and how they handled it,” he said. “I think all of these are the right types of answers.”
Not all candidates respond that way. Some freeze up and don’t know how to answer the question; some don’t even attempt to answer the question, replying, instead that Carver covered everything. “That’s obviously not a good answer in an interview,” he said.
And others will take the opportunity to let their pride and hubris take over. One candidate responded by saying he thought he’d be asked about how great he is and then went on to talk how great he is. “I mean, seriously, that’s been an answer to the question,” he said.
Curt Carver promised to the staff, faculty and students that he would improve their lives in at least 100 ways during his first year as vice president and CIO at the University of Alabama at Birmingham. It’s an ambitious goal he achieved, and this year he plans to do it again.
The list of technology wins isn’t something he puts together. For that, he turns to the people whose lives he promised to make better. One of the ways he collects ideas is through a crowdsourcing site called the SPARK initiative. There, the community can suggest ideas for what’s needed or vote ideas already suggested up or down.
“The best decisions come by getting the community involved, creating a voice for everyone, creating the healthy exchange of ideas so that the best ideas through a meritocracy rise to the top,” he said. “The ideas that have the greatest impact on the community are the ones that we act on first.”
The list includes everything from stronger passwords to a faster network for research. “We’re doing all of this genomics research, and, yet, our high-performance computer was being funded out of IT reserves, which means it was not being funded,” he said. “A year later, we’ve got the fastest high-performance computer in the state. There was a 10- or 11-fold increase, depending on whether you’re looking at computer power or storage power.”
Engagement with the community is a key component for the 100 tech wins in a year. The crowdsourcing site is one avenue, but another is a monthly newsletter, which recipients can opt out of if they choose. There, he keeps the community up to date on the newest wins. “We want to be transparent about the things we’re working on and how they align back to the institution’s strategic plan and our strategic plan,” he said.
The enthusiasm has been infectious. Carver said it has helped connect his IT staff to the greater university, giving them an ability to see how what they do affects how the community functions. And his ideas — the newsletter, the crowdsourcing site — have been replicated by other departments at the university. “Imitation is the highest form of flattery,” he said.
Perhaps most stunning to Carver is the reception he’s had from his colleagues. At the end of his first year on the job, he experienced “something that’s never occurred in my life,” he said. What he thought was a budget meeting with his staff turned out to be a celebration of the 100 wins. “I walked into the room, and there were 300 people with cake and a celebration going on,” he said.
What’s it like to be the chief financial officer at a fast-growing, industry-disrupting startup? How does a CFO deal with the startup mindset?
If Brad Dickerson, CFO at the meal-kit delivery startup Blue Apron, and Chris Nielsen, CFO at Redfin, a web-based residential real estate brokerage, are examples, the experience is exhilarating. But it’s also like walking on eggshells.
“A lot of my job is about how do we keep the great things that made the business what it is today and continue to make processes and tools better and more scalable for the future. And it is an incredibly fine line to walk,” said Nielsen, who before joining Redfin in 2013 was at Amazon for seven years, most recently as CFO of Zappos.com.
“The seemingly simplest change can set off a cultural reaction that is unexpected,” he added.
Dickerson of Blue Apron also zeroed in on the CFO’s responsibility to “architect the processes, reporting structures and teams” that help put a fast-growth startup on sound financial footing. In early stage companies, that job is “a lot of fun and energizing,” said Dickerson, who was CFO and COO at sports apparel maker Under Armour, before jumping to Blue Apron 10 months ago. As the No. 2 executive at Under Armour, Dickerson helped guide the fast-growing startup through a successful IPO.
Blue Apron, founded just four years ago, now ships over 8 million meals a month in the U.S., has 5,000 employees and is still in high-growth mode. Employment “literally changes very day,” Dickerson said.
“The value you can add at that [early] stage is exponential month to month,” he said. “At a certain point, when the organization has more people, it is a little tougher for the CFO to add that kind of value. You go from architect-and-building mode to maintenance mode.”
Learning the startup mindset
Dickerson and Nielsen were featured speakers at the recent 14th annual MIT Sloan CFO Summit in Newton, Mass. This year’s event, “Always on: The Digital CFO,” focused on what CFOs must do to speed up and safeguard their companies’ transition to digital businesses.
Conference speakers discussed how siloed processes, legacy technology systems, outdated compensation structures, change-averse employees and corner-office isolation can conspire against making the leap to digital CFO. Indeed, CIOs would have felt right at home.
In the event’s closing interview with Dickerson and Nielsen, the audience got to hear from two CFOs at startups that were born-digital. Here are two key observations about life in the digital fast lane:
- CFO as data point connector
As an e-commerce, subscription-based business, Blue Apron is in “constant contact with customers,” Dickerson said. Customers not only visit the website to choose their meals but also to notify the service of any changes, such as travel plans that would change delivery. “We are literally interacting with subscribers every single week,” he said.
In its four-year lifespan, the company already has 200 weeks of information on subscribers. All that information, combined with data on customer satisfaction, customer loyalty, and operations is fed into the company’s forecasting. “The interesting thing is that in some way, shape or form all these data points connect. Sometimes the connections “happen naturally” and when they don’t, Dickerson said it is his job to connect the data points.
Redfin’s Nielsen agreed. “I do think the role of the CFO increasingly is to provide that kind of connective tissue for all the data. It is difficult to succeed today as a CFO, if you don’t understand the broader metrics and how they could be better.”
- CFO relationship with CEO
Nielsen, who has an undergraduate degree in industrial engineering from Stanford and an MBA from the MIT Sloan School of Management, said one of his top priorities upon joining Redfin was to figure out the rhythm of the business, and in particular, of his CEO boss:
“It’s important to know what they are interested in, what they are not interested in, what are the pieces they like to do and what are the pieces I am going have to do,” he said. “These are important things to do in any job but are particularly important at founder-led businesses, which tend to have a very strong cultures and viewpoints.”
Blue Apron’s Dickerson said the CFO “balances out the energy and passion” typical of the leadership at fast-growing startups:
“Sometimes in founder-led organizations, people tend to see things in a certain way because of that energy and passion; they are often very optimistic about the business, but there is also paranoia — which I see as a good thing — and hyper awareness of the competition. The CFO has to balance out that energy and make sure we are looking at things that they might be missing.”
Indeed, providing that balance is intrinsic to the CFO role, Dickerson said, whether one is tempering the startup mindset or providing guidance to established companies. “We joke about this sometimes, but the CFO has be the adult in the room.”
Facebook has a new CIO. Atish Banerjea joined the social media company last month.
Banerjea comes to the position after CIO stints at NBCUniversal, a news and entertainment company owned by Comcast Corp., and Pearson Education Corp.
Martha Heller, CIO recruiter and president at Heller Search Associates, described him as “an excellent choice.” Heller, who said Banerjea would have been one of her first calls had she been heading up the search, cited several reasons for her endorsement. First, she described Banerjea as having the ability to strike the right balance between managing internal IT systems and developing customer-facing technology products. “And that’s going to be very important for Facebook,” she said.
Second, Heller pointed to his digital advertising experience at NBCUniversal, which she said will serve the social media company well. “He’s done a lot to create technology platforms that drive ad sales revenue, and that’s one of Facebook’s primary business models,” she said.
Finally, she cited his experience working in industries upended by digital technology. “He has great experience of driving transformation,” she said. “I’m not saying Facebook right now needs to transform, but I bet it will.”
CIO job description
During its search, Facebook made the unusual move of publishing the job description on its site and on LinkedIn, giving CIOs a rare glimpse into what a digitally-born company is looking for in its senior IT leader. According to the job description, Banerjea will oversee “building and running systems that will help scale Facebook globally, enabling the company to continue moving toward its goal of making the world more open and connected.”
Office 365 — Microsoft’s package of cloud-based email, word processing, scheduling and other productivity applications — is used in more than a third of organizations worldwide that use cloud applications, forging ahead over rival Google’s G Suite.
A new report by cloud security software vendor Bitglass, which analyzed the email domains of 120,000 organizations worldwide, found that 59% of organizations using cloud apps have deployed Office 365 or G Suite: Thirty-five percent use Microsoft’s cloud productivity apps, an increase of 10 percentage points over last year’s share. G Suite was favored by 25%. The remaining 41% are primarily using traditional software, which lives on physical servers.
Cloud productivity app gap
Public companies go the Office 365 route over G Suite nearly 2:1. That’s because Microsoft makes it easy for organizations that have traditional software to move to Office 365, said Salim Hafid, product manager at Bitglass. And at a majority of organizations, that software is Microsoft’s.
“[There’s] near feature parity between the premises-based applications and Office 365 now,” Hafid said. “Organizations don’t have to worry about, ‘OK, is this core feature that we use actively within the organization still going to be there?'”
The Microsoft-Google split was more equitable among private companies, with 29% on Office 365 and 25% on G Suite. Private companies are often smaller and more willing to switch from, say, an on-premises Microsoft productivity package to G Suite, Hafid said.
“They don’t have to deal with many of the compliance requirements or concerns or challenges that larger companies have to deal with, so it’s much easier to make a very quick change to something entirely new that maybe isn’t proven out.”
Not a slacker
The Bitglass analysis also looked at usage of the popular messaging app Slack. The portion of organizations that have tried the app is 33%, sizeable for a tool that launched just three years ago, Hafid said.
Organizations’ first taste of Slack often starts with employees, who download the free version of the app without IT’s blessing, and, indeed, the 33% includes sanctioned and unsanctioned use. Bitglass didn’t break that down in the report, Hafid said, because it’s difficult to tell what’s sanctioned and what’s not.
The most recent competition to Slack is Microsoft Teams, launched earlier this month. It could be an attractive option for organizations running Office 365, since it meets a lot of the same security and compliance requirements they already have in place, Hafid said.
But Slack will continue to gain users, he said, as organizations cotton onto under-the-radar use and start paying for and managing it.
“The telltale sign of how a cloud app is doing in an organization is more on the side of, Is the organization paying to use the application, being widely used internally and supported internally by IT?”
Single sign-on status
Cloud security was another area of the analysis. Single sign-on, which lets users log in to several applications with one set of credentials, was in use at 32% of organizations. That’s on the low side, Hafid said.
Single sign-on is a critical security component, he said, because it gives organizations visibility into what’s happening with their data, letting them see “what users are logging in from, what devices they’re logging in from, where they’re logging in from.”
Single sign-on was more in use among Office 365 subscribers, at 26%, versus just 6% of G Suite users.
Right now, Hafid said, organizations in regulated industries like healthcare and financial services are using single sign-on. “But over time these other industries will no doubt start to see the value in this.”
Is your business digitizing? Or is it digitalizing?
If you answer the question with another — Does it matter? — Gartner analyst Jan-Martin Lowendahl has something to tell you about his homeland.
“In Sweden, we don’t have different words for efficiency and effectiveness. You do,” he said to a group of IT leaders at the recent Gartner Symposium/ITxpo in Orlando, Fla. “You have different words for digitization and digitalization. Please use them.”
Digitization is converting information into digital form. Digitalization, which Gartner uses a lot, involves revamping a business model and providing new revenue-generating opportunities. Or as Lowendahl put it, “using those digitized assets and doing something completely new.”
As an example, Lowendahl started with a pre-digital product, a vinyl record. Digitizing the analog album gave us compact discs.
“But please remember that CDs have the same business model as long-playing records: Two good songs, 10 crappy ones,” Lowendahl said.
In the early days of the internet, music service Napster figured out that people wanted individual songs, Lowendahl said, and let people share digital audio files. Apple improved on Napster’s innovation by introducing the concept of micropayments, very small financial transactions made online.
“Of course nowadays there’s a totally new business model. Nobody really buys the songs anymore; we rent them by the millions,” Lowendahl said, referring to streaming music services Spotify and Pandora.
Digitalizing higher ed
Lowendahl was presenting on technology trends that affect IT in higher education, so he gave a higher-ed example. Massive open online courses, or MOOCs, which offer an unlimited number of students an education over the web, are digitized. MOOCs that incorporate machine learning, a branch of artificial intelligence that enables computers to self-teach by ingesting and analyzing massive amounts of data, are digitalized.
Some can even grade students’ essays — “Not as good as your Columbia writing professor,” Lowendahl said. But a student can turn in a first draft at, say, 2 a.m., get notified of principal issues and then rewrite and submit the revision to a human grader.
“Then my professor can really deal with the higher-level points rather than the more mundane issues,” Lowendahl said.
Machine learning tools will play a huge part in research in the future — virtual assistants that read what we read and can recommend what to read next. And then there’s supercomputer Watson, which read 70,000 research papers on a tumor suppressor protein and came up with six new candidate medications in two weeks, Lowendahl said. The previous record is about one a year.
Digitalization is about creating content, Lowendahl said, “and that content is now code. Machine learning is going to help us do a lot of things better.”
Much of the current discussion on the evolution of artificial intelligence — an umbrella term to mean everything from speech recognition software to robotics — centers on speed. Machines can digest, consume and analyze more data at a faster rate than ever before, but does that make a machine intelligent?
It’s a hard question to answer (even the experts don’t quite agree), and that’s what made Dileep George’s EmTech presentation so interesting. Titled “Artificial Intelligence at Work,” George, who works for the AI research project Vicarious, started his presentation with an observation: Hardware, motors and plastic are cheap, “but we still don’t have a Rosie the Robot.”
Why isn’t a Rosie the Robot, the beloved family housekeeper in the animated series The Jetsons, here to tidy up your dirty dishes and sweep your floors? Because of a software problem — what George (that’s Dileep George, not George Jetson) called “the common sense problem.”
He illustrated what he meant in two sentences: John pounded a nail on the wall; Sally pounded a nail on the floor. If George were to ask in which case was the nail horizontal, EmTech’s attendees could easily answer that question, he said. We instantly imagine the scenario — John, the wall, maybe even a hammer.
“You have a pretty rich model of the world in your head, and I was poking that knowledge using a few sentences. And you ran a rich simulation in your head to answer the question,” he said.
For machines, not so much. They can’t imagine the situation because they lack what George called “a model of the world.” “To build systems that have common sense, you have to build systems that build models of the world,” he said. “In fact, I would say building a model of the world and being able to act on it is the crux of intelligence.”
But here’s where it gets confusing: There are systems that can successfully function in the world without having a model of it. “Our world is filled with creatures that can do this,” he said. Dinosaurs, fish, amphibians, reptiles are all examples. On the other hand, the Roomba, a robot vacuum, can get stuck cleaning one corner of the room because it doesn’t have a model of its surroundings. That’s true even of the DeepMind Technologies’ artificial intelligence that has learned to play Atari. Change the brightness of the screen, and DeepMind’s AI gets tripped up.
These systems are all using what George called “old brain.” For AI, that translates into using a lot of training data and labels to create a desired response. “Once you train the system, and these are humans training the system, they do produce the response given the test data,” he said. But it’s still no rich model.
So, how can researchers help AI evolve from old brain to new brain, an intelligence which can imagine and reason about the world? George said there are two steps to take:
- Take neuroscience and cognitive science seriously. “We need to understand what is happening in the new brain rather than building very specific circuits for very specific applications,” he said.
- Focus on data efficiency and task generality. Rather than throw a lot of training data at machines and test the machine on limited, narrow data sets, George argues that needs to be flipped. “You have to keep the training data small, much like a child would experience, and we have to test it on a huge data set,” he said. “We have to test for robustness.”
“If you execute on those principles, you can [do things like] break captcha,” a test made up of numbers and letters that’s used to ensure the user of the site is human. In fact, George and his colleagues have done just that — with a small amount of training data and a large test set. “It’s an amazing feat of generalization by humans,” he said. “You did not get training data on any of these captchas, and you can solve these things. A child would be able to solve many of these without training examples.”
The system George and his colleagues are building was trained on 260 single character images – not even full strings from captchas. A deep neural network requires half a million actual captcha sequences to get a comparable performance.
You don’t need a doctorate in computer science to download an app.
That’s a good thing for people working in healthcare, finance, education or pretty much every industry there is. Technology today is people-friendly. But to IT folks like Larry Gagnon, that poses a big challenge.
The senior IT project manager at the University System of New Hampshire said people in different parts of the two colleges and two universities he services surf the web, find an intriguing piece of emerging higher-ed tech – software that, say, professors can share with students or can use to manage faculty — and then download it.
“No longer do you need IT,” said Gagnon at the recent Gartner Symposium/ITxpo in Orlando, Fla. “And the challenge becomes all of sudden they do that and then all of sudden they install it on a server and then all of sudden they’re calling my support line saying, ‘Hey, I need help with this.'”
The new college try
And higher-ed tech today is pretty tempting. Adaptive learning, for example, turns computers into interactive teaching devices that can offer lessons tailored toward particular learning styles. And some predictive analytics applications tap data from a number of sources to forecast how a student might perform in the future. They can be used to help struggling students improve — and stay in school.
They’re nifty tools — just the types of things professors, personnel officers and folks in finance and sales would love to get their hands on. That’s why CIOs need to develop and maintain all kinds of relationships, Gagnon said. One is a partnership with a CXO, or chief experience officer. This is the executive concerned with user experience, or how users react to and feel about a product or service. Conversations with CXOs could give CIOs a heads-up on what users are exposed to.
“Because if they’re coming in after the fact, that’s the challenge,” Gagnon said.
One higher-ed tech leader Gagnon spoke to at the Gartner conference had formed a strong relationship with someone in the procurement office. It works because procurement is responsible for signing contracts. If something technology-related doesn’t trace back to IT, an official can alert his or her tech contact.
The important thing, Gagnon said, is to make those connections — in the personnel department, finance or, specifically for higher education, academic or leadership groups.
“You try to get a seat at the table,” he said. “If you don’t have a seat at the table, how are you getting your information? How are you communicating with the president and the provost? What’s your trust level there?”