Should I take up the new offer I have? It’s offering over 50% rise and a larger responsibility. The industry is the same and the company which is a recent entrant is growing in leaps and bounds. I have been in my current company and role for over 4 years now and it has a great work-life balance. So moving to the new assignment would entail moving to a new city, unknown people, unknown culture, starting afresh building credibility and gaining acceptance from the team as well as peers which would take a year or so.
The CIO was past his mid-life crisis and leaning towards his sunset years with about a decade of active working life remaining based on generally accepted and regulatory definition of retirement age. He was doing well in his current company having spent a decent amount of time managing process, systems and expectations. He had been successful though not overtly so, but consistent in his ability to deliver to promise and keep progressing. He was been headhunted for the first time and was excited by the prospects of the new role.
He was also heavily into work-life balance and an example for some of us on keeping the balance tilted towards life more often than work. While his demeanor suggested a laid back person, he was effective in managing tasks, projects, budgets, suppliers and customers with ease. Financially stable and well off, he was not driven by monetary incentives nor materialistically inclined. Thus I was a little taken aback with his meanderings considering that he was of a sure mind and rarely hesitated or consulted anyone in such matters.
Challenging him to listen to his heart rather than the mind if they were in conflict, I asked him what caused the dilemma? Was he unsure of the new organizations ability to succeed in the chosen industry or was it the comfort zone that caused the see-saw decision? It’s not like that he had spent a lifetime in the current company; why was he feeling discomfort by the thought of change? Continuous prodding finally brought out some interesting tenets for his inability to take a decision and it was not an easy one to solve.
Growth in current company would come over time; it was a profitable and stable entity where he had the freedom to operate. He knew the system and how it works; he had a good team and they delivered consistently. His family was well settled in the current city; wife working, kid entering last year before college, other kid getting to his first job in another city. He had just bought a house with a mortgage and done it up with a lot of care. Workplace being a stone’s throw, he enjoyed life to the fullest even after putting in the required work hours.
The new entity had expanded fast and planned high growth in an industry which had a lot of promise but not too many made money. There was a lot to do for the next couple of years and he would hopefully grow with the company. The new company had a frugal mindset and culture vis-à-vis his current one. Work-life balance would become a memory with pressures in the company, a new entrant in an industry requiring deep pockets. Success would come but require far more effort; comfort versus challenge!
Would you sacrifice work-life balance and family over career? How do you prioritize and determine where you should focus? At which life stage should life takes precedence over career or career over life? When do you reach a point that you get off the treadmill? Many have given up careers to pursue their hearts desire or become entrepreneurs on reaching financial goals; some remain hesitant with fear of the unknown. It is easy to stay in the comfort zone until life gives you a push to get out and start afresh.
I believe there is no right or wrong, no good or bad decision. The inflection point varied by individual; open a restaurant, write a book, teach at B-schools or colleges, the calling has varied for some of my friends. My recommendation to my friend was to take a few days off and introspect for the future and find what keeps him going every day. If you can dream it, you can also make it possible. I for one have been conscious of comfort zones as they give me discomfort. What would be your advice or what would you do?
The flood of resumes was overwhelming; I was surprised by the numbers wondering if there was a crisis out there with people wanting to leave. Maybe it was the company and its reputation that created a pull of sorts that the applications found exciting. Or it could be that the economic situation has resulted in uncertainty in their current positions and thus they sought a comparably stable environment. Anyway the problem of plenty was a good problem to solve which gave us the option to choose from the best and the brightest.
Sifting through the lot it was difficult to shortlist probable candidates; everyone appeared to have been there done that, a menu card of technologies that they professed to know and work experience that would make you want to hire them right away bypassing the process. Discounting spelling errors in favor of experience, the final list of interviewees was drawn up. The list was not as short as expected, but then we did not want to miss out on deserving candidates just because they had turned off spellcheck or had bad grammatical mistakes.
Not having conducted so many interviews at a go, they had to be spread over 3 days with almost 20 candidates. In tow a HR colleague and the functional lead who wanted to evaluate technical skills, armed with a formal assessment sheet to capture impressions, we started the process; questions were divided to suit our respective functions. HR would break the proverbial ice, settle down the person, my teammate tossed difficult technical questions and I looked at attitude and confidence. The days passed by in a whirl and we had a winner!
We have been taught to keep our demeanor friendly and suppress emotions while conducting job interviews; it was difficult to hold on to sanity and control laughter in few occasions. The journey to the end was excruciatingly painful and frustrating; the candidate with the perfect resume turned out to be a disaster. She had put in all the right keywords, technologies and projects; she had been in projects with the technologies in a role that barely gave her basic understanding. Scratching below the surface revealed no substance.
Another one believed that he scored 10/10 on every skill mentioned in his resume and that he had reached the pinnacle of learning. He was stumped on most of the questions which led to his quick exit. One candidate kept repeating that the information about the projects he had worked on was classified and that he was under NDA and thus could not talk. One person narrated the long story of her life for about 30 minutes immune to any attempts at interruption. We thanked her for the enlightening moments and heaved a sigh!
The quality of most discussions made us wonder about current state of knowledge and expertise; or is it that the unwanted and incapable having realized their shaky existence decided to seek newer pastures. Are these the types who find themselves on the left side of the traditional HR performance bell curve? My HR colleague mentioned something about finding the right talent through references and not an open process. Junior and mid-level hiring filtering done by executive search companies and headhunters is typically based on keywords.
The selected candidate did not have all the skills required for the position; he was short on qualifications though the experience was relevant. Knowledge on technology was above average with understanding of his own limitations. He demonstrated how he stayed abreast of current trends and could articulate how he worked in teams. His enthusiasm and candid responses had all of us liking him; his can do attitude clinched it in his favor. We exchanged notes and everyone was in agreement; so we made him an offer.
We were pleased with our find and wanted him to join us expeditiously. He accepted our offer and promised to revert on when he could be part of our company. Weeks passed and HR kept following up on his joining date. One day I receive an email: “I thank you for your offer and opportunity to work for your company; I have carefully considered it and after much deliberation and discussion, I have decided to stay back with my current employer. I wish you all the best and hope our paths will cross again in the future.”
Life is tough!
It’s more than 20 years now from the first collective industry recognition (Energy Star ratings) and the desire to do something about reducing power consumption by IT equipment. The cycles of heightened awareness and hype coincide with economic cycles or at times a paradigm change brought about by new technology or use of existing technology like virtualization or cloud computing. So when one of the organizers asked me to talk about Green IT, I was perplexed on the unexpected resurgence; or was it a red herring or just a slot filler?
Researching the market beyond personal experience to understand the current traction and prominence of Green IT, I started talking to CIOs to understand their journey or milestones on the subject; I was hoping to use some of the insights in my presentation to an audience of CIOs. Thus over the next few weeks the data points that I gathered did not focus on public or private clouds, but to understand if Green was still a discussion in different industries and industry leaders or has Green fatigue set in?
Less than a decade ago carbon credits were a boardroom discussion and some people made a lot of money trading them; then they just vanished. Data Centers started touting Power Usage Effectiveness (PUE) or Data Center Infrastructure Efficiency to demonstrate how Green they were. Overhead non-compute power usage for cooling and other uses ranged from typical 80% to highly efficient 10%. Power used by servers and computers for the same task has been reducing almost linearly over time challenging PUE.
CIOs finished fixing the data center quite early and also created efficiencies for end compute devices with power management policies and improved awareness. From there on they focused energies on office infrastructure, ambient lighting and temperatures, electronic documentation, communication and as a result saved trees. A few also took control of energy usage monitoring with embedded and targeted sensors linked to BMS (Building Management Systems) and reduced power consumption in good measure.
The question is, with so much getting done and almost every enterprise reaping the benefits over the last decade, is Green still a relevant discussion? Who if anyone is tasked with overall enterprise Green initiatives? Is it only about controlling power used or creating a movement to conserve natural resources? Is Green an integral part of decision making explicitly mentioned in evaluation parameters or implicitly used in every decision making? Have people stopped flying around for meetings and embraced Video Conferencing? The answer is yes and no.
I don’t believe that CIOs are measuring and reporting metrics around Green. None of the CIOs I met did, but there is a discussion around power efficiency and overall cost of running BAU or keeping the lights on. CXOs are not enamored by earlier conventional solutions which are now basic hygiene as are new LEED certified buildings. Reality is that for technology and compute power, the bar keeps shifting every 4-5 years; what was green in 2000 or for that matter even 5 years back appears archaic when compared to currently available hardware.
At the conference the audience engaged in a discussion and debate on the relevance, challenges and opportunities; one of the participants sought suggestions to break the cultural vice around personal printers which gave his global centurion company a person to printer ratio of close to one. Someone shared extreme automation to switch off lights and air-conditioners when there was no movement resulting in hilarious and occasionally dark and sweaty situations while people were in the room working!
Green is here to stay and awareness of the new generation brings new opportunities. For CIOs it is important to seek avenues beyond the conventional interventions of the past. Cloud computing is shrinking the data center and mobility is driving the workforce out of offices. Power generation is focusing on renewable energy sources which reduce the global carbon footprint. Some of the power hungry enterprises are going to embrace these as the world works towards creating a better tomorrow.
The role you play in this evolution is up to you!
“You can’t just ask customers what they want and then try to give that to them. By the time you get it built, they’ll want something new.” So said the most iconic leader in the IT industry and stuck to this philosophy as his company built some of the most vied for products; the success that followed remains unparalleled though the bull run has slowed down a bit. Everyone wondered exasperatedly on how they can replicate this model as it is contrary to conventional wisdom and what they were taught in B-school.
Until not too long ago the software industry churned out products with features and functionality based on internal discussions on what the customer may need or in some cases based on what their initial customers asked for. With generic solutions not fulfilling the expanding needs, over time they started hiring industry domain experts to create vertically aligned solutions. This did address the gap partly for a while and then customers started demanding better aligned solutions for their specific problems and opportunities.
Some companies recognized the need early and started creating Customer Advisory Boards (CAB) with CIOs of their large customers to participate in the product roadmap. This was extended to include some of the innovative adopters of their solutions though they may not have been high revenue customers but brought value to the discussion. The ensuing engagement, discussion and debate influenced the prioritization of new features and in some cases the positioning of their solutions resulting in a win-win situation.
Some of the services vendors took the cue and hired from the industry to strengthen their industry practices; consulting companies followed suite thereby changing the discussion with their customers. They determined that the need was to embed the resources internally and not limit to an advisory role. Now the software industry is going through a transition with even mid-sized companies thinking of CAB to gain the benefit of customer connect and better alignment of their product features and evolution to what the industry wants.
Interestingly hardware manufacturers have remained disconnected; they continue to launch products with the philosophy of the icon attributed with the famous quote. Past practice of customer focus groups has largely been discarded by marketing teams. Faster processors, bigger, brighter and higher resolution screens; consumers love it and they do more of the same. Then they have attempted to push the same products to enterprise customers and wondered why it is not gaining traction the way consumers are lapping them up.
CIOs are not excited; what else do you want has been the lament? Over the years the clear message from many CIOs to the IT industry enamored by all things mobile (phones, tablets, and applications) has been that the faster, better, cheaper does not connect with enterprise use cases. Enterprises need manageability, serviceability backed by service levels, and reasonable (measured in years not months) longevity. Consumer devices require additional investments to make them work in our environments.
Consumer applications and games are great; couple of apps on the app store for some customers or pilots on industry specific use cases does not make you an enterprise ready development partner. We don’t want to explain everything from the basics to your team; how are you going to fill in the gap between what we say and what your team understands. Do some homework and more than anything else listen before you start crafting solutions; you have an advantage over your big competitors, use it well.
The question is then, is CAB the way to go for companies who want better traction of their solutions or services in the enterprise space? It is a model that may work for the larger IT companies; how does a smaller outfit get the benefit of the experience? In “Scaling Startups” (http://cio-inverted.blogspot.in/2014/03/scaling-startups.html) I had referred to a mentoring model and role that CIOs can play; maybe it is time for IT companies to embrace CIOs to help them forge ahead. What is important is the change in mindset and philosophy with internal agreement on the new way of working. I hope IT companies understand this sooner than later.
As a CIO, are you up to the game?
The country is going through the most complex exercise of voting for and electing new representatives to the government. Selecting amongst the candidates is difficult; some of them are easy to disregard as they have no visible credible experience to stake a claim to the seat. Handfuls have relevant experience and on paper they look like good options. References to past work demonstrate their ability to deliver and execute; a couple have the backing of their respective political parties who lend the promise of a collective manifesto.
Every 5 years this process repeats itself, sometimes a little earlier if the incumbent government is unable to serve out a full term. Soon we will have a new government, a new head, a set of ministers who will vie for the most visible and high profile ministries. The correlation between portfolios and core competencies is always a good thing to do; however many a times that does not happen. Bureaucrats and the staff within the ministries does not change much, they follow the new directions set by the ministers irrespective of expertise.
Running an enterprise is very much like running the country, especially a large diversified group with interests in varied businesses. Each company and function has a head that is selected from outside more often than inside. The selection is most of the time purportedly on core competency and merit. The difference between a government and an enterprise is normally the available options from which a candidate is chosen. In a government, the candidate is from the elected party, in the other case almost everyone can apply.
Government portfolios are presumably distributed by the head of the winning party and head of the country collaboratively. Most often decisions are based on seniority, past experience, credibility and interest. Cross functional movements are the norm and it is expected that the person would do equally well in the new function too. The rationale here is that a leader need not be a functional expert; the team has adequate skills to advise the leader on the best options when taking a decision. We know how well this process works.
In the corporate world the skew is more towards functional expertise while selecting a person. Cross movements do happen at beginning and mid-careers; moving up the ladder, these are rare. Leaders at the top take on additional responsibilities at times; lateral moves occur but are not frequent. Success rates are higher in comparison as everything is expected to be time bound. It is not a sure shot recipe for success; we do observe failures across the board, many attributable to the leader not being effective or fitting in.
It is not necessarily the interviewer or selectors inability to assess that result in a bad selection. Drawing parallels, it is evident that the best person does not always get selected for the job; favoritism and at times other factors like past workplace association, belonging to same religious sect or geographical area, been to the same school/college, having the same ideologies, result in suboptimal choices. While not always avoidable, enterprises do watch out for such hires critically lest they end up with unwanted baggage.
It is relatively easier to replace a person within an enterprise as compared to the government; performance appraisals even though many a times skewed do elevate non-performance. On the other hand, recently observed citizen activism has its place in creating change. Cabinet reshuffles however move the problem from one area to another; this is rare in the enterprise though not unheard of. Are there learning that can be applied to enterprises to not follow the same path that ails many parts of the government?
I believe that when we choose an elected representative or a new hire, in both cases rigorous due diligence is essential. Our choices can come back and haunt us not just in the short-term but also in the long-run. Both impact our lives and future; we tend to spend more effort in our workplace due to impact proximity and blame bad decisions to the ill choice of others. If we want control of our future and destiny, we have to exercise our rights and influence outcomes. Can we afford not to?
You decide, you have a choice!
Not too long ago one of the consulting companies’ classified people by their digital proficiency and created three groups; digital dinosaurs, digital migrants, and digital natives. This was picked up by many people who used this classification to target their solutions or services to their respective benefit. The resultant divide did not matter to the bourgeois or the elite, they anyway continued to demonstrate behaviors that they did; the world continued to evolve, and gave way to a new species, which is gaining ground.
Characteristics of the first three categories were quite easy to comprehend and evident with their names. The Dinosaurs are the people born before the advent of technology (today’s 60+ generation) who use basic technology to stay connected. Migrants (born before 1990 or thereabouts) embraced the new technology wave sometimes a bit uncomfortable, but more or less adept. The Natives are born into the new world and do not know of a life without social media, mobility, ubiquitous connectivity and instant gratification.
As is with all kinds of evolution, there are genetic mutations, exceptions and the differently abled who do not partake in the normal. They have all the means, the environment, the facilities, the stimulus, the desire; they show a lot of promise and demonstrate a stray spark of brilliance. These individuals are mostly found in the cusp of the natives or wannabe migrants (migrants who believe they are better natives than the natives themselves). It does not take too much of an effort to spot them, they are visibly obvious.
In a digital conference with average participant age less than 24, we tried spotting the pseudo natives; one of the sparsely haired speakers in mid-40s was desperately trying to impress the audience with his knowledge and investments into digital startups. A mobile cloud, take it anywhere you go; make 2G work at 3G speeds; family only social media, were some of the references to his invested concepts which he believed were ahead of the evolution curve. No one had heard of his ventures but the crowd humored him to get his wallet share.
A corporate 20-something CDO (Chief Digital Officer) reveled in the fact that he was ahead of his peers having already achieved the pinnacle position in an industry which was always in the forefront of digital technology. With an MBA from a prestigious institute, he strutted around to the envy of others who wanted to get there. He was on stage being interviewed on his vision and predictions of where the digital world was headed. Not that he wanted to entertain the audience; his responses had most people snickering.
His stated digital strategy for the company revolved around leveraging an existing portal which had lost traction with customers. Is mobility part of your strategy? Off course, we are deploying apps! How do you plan to differentiate? We are looking at global trends! Do more of the same, do it a little differently; will it change the world or bring around a revolution? As the Q&A progressed, the audience became restless wanting to be rid of the listless conversation. One anonymous listener shouted across the room, “Get off the stage, you…!”
Titles aside the general enthusiasm around digital everything ranged from wearable technology to esoteric business cases; the general feeling was that if you are a migrant, there’s a generation gap, if you are older, why are you here? You don’t understand our language; you don’t belong to this new wave. Don’t slow us down; we know where we are going. While some oldies tried to moderate the irrational exuberance reminiscent of the dotcom era, the young believers retaliated with cries of frustration with the dinosaurs.
I think that giving some latitude to sprouting innovation connect to real life use cases is the need of the hour. It would serve the well-wishers to espouse the cause while staying out of the way of the emerging digital tsunami. The velocity and variety of change challenges legacy thinking; to understand and appreciate the thinking process of the new generation entrepreneurs requires unlearning and new mindset failing which the older generation and some from the current too run the risk of being alienated or being labeled Digital Morons.
In the beginning of the year CIO surveys depicted an upbeat mood with redefined priorities, business bouncing back, economic situation getting better and last but not the least IT budgets going up. This was the global optimistic view portrayed and shared by many CIOs that I spoke to also; and everyone wanted to break into a spring dance and celebrate the return of the good old days. Few CIOs enthused about significant increases in their budgets not betraying the fact that they had the benefit of a low base; 100% increase in budget sounds better!
Every organization big or small goes through an annual operating planning of budgeting revenue and expenses. All CXOs play the game with their promoters, headquarters, Board and whosoever is the negotiating and approving authority. Revenues are understated, expenses inflated and the commentary is all about how tough the environment is while we need to invest for the future. Projects get labeled strategic in their quest for approval; expenses become unavoidable, while market conditions constraint growth which is linked to past mediocre performance.
The situation predictably repeats itself annually like clockwork with an element of distrust on either side built out of past experiences. There is an air of wasteful irresponsible spending that needs parental control which needs to be exercised by the approvers. Chastising the minions, the numbers are adjusted amidst protests to reluctant acceptance. If the normalization has been prudent, life releases the brakes and moves the organization into top gear; when the negotiation is unrealistic, then starts the frustrating process of out of budget approvals.
So when I met a large number of CIOs on the unveiling of one such report, I tried to validate if budgets had really gone up; majority in the room had participated in the survey which brought exuberance to the sponsors and vendors in the room as the details unfolded. The dipstick brought in mixed results, the percentage was lower but there was indeed a group which had seen an increase in their budget. The quantum of increase was also a bit lower than illustrated in the report with a higher inclination towards variability.
Deeper analysis revealed increases factored in inflation apart from business expansion or higher levels of dependence on IT with newer technologies taking up a lions’ share. Business As usual (BAU) spends is under pressure and requires rethinking; there is an expectation of lean thinking but willingness to spend for innovation and quantifiable business value. CIOs are engaging the rest of the company in prioritizing the allocation of funds and challenging status quo. The number of non-participative CIOs is dwindling and that is good news.
I did not hear much about the earlier big discussion on open source towards cost reduction; open source is now a viable alternative for some technology stacks. Expectations of free software reducing costs have withered away with experience of engaging teams to sustain such solutions which require a little more effort, specialized skills and lenient service level agreements. In specific segments the uptake was large and benefit quite visible. The push towards open source personal productivity tools has taken a back seat.
Everyone likes good news! And budgets going up after a while is indeed good news for everyone. The moot question is how much of this will be discretionary to the CIO, or will the strings be pulled by the business? The shift of project budgets to business has been gradual but consistent; the perception of lack of control has created many conversations fueling the insecurity of some CIOs. Though rarely observed now, it is also a check on some not to run away with technology ignoring the best interests of business.
CIOs with strong business connect will continue to innovate and create enterprise value with whatever budgets get thrown at them as they have already aligned the business to what is required and in almost all cases they do end up getting what they wanted. CIOs with patronage of a board member may in the short-term get endorsement, but will be under pressure to deliver more than the first set which is business aligned. So if you have an increase, either way, live with the good fortune of funds availability until the mirage lasts.
As a kid I used to watch Star Trek and marvel about Mr. Spock and the technology some of which is reality now. I also devoured (Isaac) Asimov, Arthur C Clarke and other science fiction fantasizing about the future possibilities that will change the way we live. When the internet made its presence felt and mobile technology started gaining traction, the realms of reality stretched imagination. Moving from the humble personal computer to the smartphone, the new world is impacting us in many ways directly and indirectly.
The last few years have seen a lot of technology wanting to attach to human body; fortunately it is still outside the body and not getting inside as yet. Fitness bands gave way to smart watches which doubled up to also show you the time while telling you about who is calling or the latest trends on social media of choice. Glasses anyone? And there was a queue of people wanting to cover their eyes! Directions from your favorite map service on your windscreen? Why not also notify you about what your “friends” are excited about or what is trending?
Is that a distraction to driving? Let’s make cars that can drive themselves while you immerse yourself in the pervasive world of noise from all over. Traveling somewhere? Write reviews and tell your friends, too bad some burglars also took notice that you are away from home. Stay connected wherever you are on Wi-Fi, 3G, 4G, LTE, on your tablet, phablet, phone or at least the watch? Your health parameters are being monitored and stored by some app. You can use it later to determine your health; so can someone else who you don’t know!
Will you scan before you eat? Supermarkets would like to help you create your personalized basket of products which they can remind you to buy. All the packaged food companies would also like to know the products you consume so they know more about your eating habits. Next time you renew your health insurance, don’t be surprised by a markup or discount; the health insurance company also has vested interest in this data! The situation is becoming Orwellian with Big Brother watching every move you make using Big Data.
The question that bothers me is why are we willingly obliging everyone? What makes us want to obsessively stay connected and share the important to obscure to inane details of our lives? Why are we allowing personal data to be used in a way that influences our actions and our behaviors? This was not anyone’s favorite part in any futuristic idea or science fiction? Digital addiction especially for the new Digital Native generation is rewiring responses; the thinking process is changing, our dependence on technology complete.
People have reported withdrawal symptoms when they are unconnected; the thumb is getting used beyond its evolved state leading to new diseases. Accidents are increasing with distracted mind unable to focus on walking or driving. Eyesight is adversely impacted while our hearing ability is impaired due to constant high volume sound from earphones. Over and above all of this, the expectation to respond to any stimulus (email, message, alert) instantly is putting psychological pressure on the connected generation.
Enterprises want productivity with the connectivity; corporate apps hijacking the personal phone, information driven process velocity is the new normal stretching the already challenged balance between work and life. The lines are blurring and both have intertwined into our awake hours in a way that does not leave any personal space or time. The resultant pressure and stress is increasing rate of heart ailments and lifestyle diseases in 20 something youth that were associated with 60+ in the past.
Some companies and governments have begun to take cognizance of this and are enforcing work hours in an attempt to wean off the addiction. How can parents do the same with the upcoming generation which is born to technology? I believe that they will have to lead by example for the young ones and also at work for their teams. Wearable technology is here and urging you to participate; I know what you did last summer is now more like I know what you just did and what you are likely to do!
Digital free day anyone?
I have been sick for most of the week; on Monday there was a niggling pain which I hoped would go away like it did most times. By mid-week it had aggravated considering I ignored not just the pain but also the cause which I was not consciously aware of. Once the connection was made I aggressively tried to remedy the situation, except that it was out of control now and needed expert attention. On Thursday the Doctor looked gravely, a long prescription and pronounced that it required a specialist to treat it.
The specialist sympathized and made it look innocuously simple to fix; for him it was a routine escalation to manage. On Friday he did what he was good at and fixed the root cause with knowledge that the rest will fix itself if I followed the defined SOP. The prescription was not too much, the lingering pain and inaction that restricted me to a bed with limited ambulation was. Daily checkup visits add to the agony as I am now in a state of mind, when will this ordeal end and I declared fit and healthy again to get back to work.
When you have a lot of time to do nothing, run out of music to listen or have no inclination to read with half the senses dulled due to heavy dose of pain killers and antibiotics, you start thinking; Doctors also refer to some of the extreme thoughts as hallucinations. To me my chain of thought was lucid and it created many correlations in the swarm of random disconnected millions of thoughts. Analogous to associations created in a data warehouse by a skilled analyst, I picked associated groups which made a lot of sense.
Early pain = project not on track; you know something is not working, you think it will get better, it doesn’t. Falling sick = project misses milestone; you get the vendor consultant to help, he refers to subject matter experts who have been there done that. SME educates on root cause, defines road ahead and KPIs to keep the project healthy. Recovery = always slow and painful; getting back on track takes a lot of effort, following the prescription, no shortcuts. Most of the time you do get back on track with no further slips; you have lost time, money and momentum.
I realize how we can create correlations between totally disconnected facts and make them look like similar data sets or for that matter draw analogies that sound quite logical. The event graph appears to follow a perfectly aligned path drawn by the same artist. Retail has been doing this with disjointed sets of data and have hit upon success many times; we don’t know what happened to the ones that did not work though. But then maybe life does have predictability that it wants us to find and we are getting better at it.
But I am digressing now, rambling about febrile correlations. IT gets sick quite often; whether it is business as usual or new initiatives, they do face challenges and require fixes from specialists and experts depending on the nature of ailment. We have prophylactic technology to keep things going while the next piece of hardware finds itself being resilient or more reliable than a decade back and networks become self-healing and storage can survive failure of a disk or two; Software still requires human intervention.
Human life expectancy has in the same vein gone up as we find better medicines for micro classified diseases. Our way of treating different patient types has been evolving rapidly with Internet of things allowing embedded Nano sensors connected to Big Data repositories analyzing symptoms as they happen and trigger corrective actions almost instantly with novel drug delivery systems. Okay, maybe the entire chain is not yet feasible, but getting there. Affordable access to such innovation would definitely be a paradigm shift.
As IT gets better, projects get more manageable, technology commoditization makes itself ubiquitous, IT wouldn’t matter! What would matter is how we apply it to real life and help solve problems that have eluded solutions thus far. As more solutions go open source or relinquish patents for global availability, there would be a new world order where healthy humans will score over sick IT. Some of us will be part of this evolution if it happens within this generation. I hope our contributions would have made some difference.
The other day I met a CIO friend who wanted to discuss a tricky situation in which he had landed; he worked in an industry which was in the thick of being projected as one of the industries that will benefit from investments in Big Data. His CEO wanted him to build a data warehouse to rival some of their global competitors, at least one of which was prominently talked about as the poster boy of Big Data analytics. He was thus under pressure to invest while the rest of his IT budget was under pressure.
Having a keen understanding of technology, his company and the industry, he was a non-believer in the Big Data story; according to him the hype around some of the Big Data insights were not commensurate to the investments made in the overall project. And there was nothing new since the first story broke out of one of the companies having found a use case that conventional technologies would not have delivered. He had many data warehouses and Business Intelligence successes in the past for which he was well known too.
By definition Big Data was all about big data sets that earlier available technologies could not bind together within tolerated elapsed time and budgets. Volume, Variety and Velocity defined Big Data; (Business) Value was added later. The availability of high compute resources and ability to store large volumes of data had made solving some problems easier, faster and cheaper; that is not necessarily success from the capitalized Big Data. It is just that larger data sets were analyzed as compared to the past.
The question at hand that needed an answer was whether he should let go and invest as directed by his CEO or he should help the business with a scalable data warehouse which would deliver immediate value. Is it possible to get started small with Big Data (an oxymoron if there was one) and then work with the business to find the needle (if they wanted to find the needle or a pin) in the haystack; after all Big Data is expected to throw up unknown possibilities by random correlations that human minds are not able to pick.
Big data works on “found” data, i.e. data that you have and complex algorithms which can provide some statistical probabilities. Analysts predict the value that different industries can gain from investments; no one is talking about the real value derived. Governments have been making investments with equal zeal as are large enterprises; the providers and consultants are happy to make hay not just while the sun shines but until by accident they discover a needle in the haystack and make a case study out of it putting pressure on the rest of the gold diggers.
What about the data that you don’t have? Can you draw negative inferences from Big Data? For that you have to know what you don’t have! Can what you have tell you what you don’t? The answer to that is still to be found; available data in a Big Data repository cannot indicate to what is missing. The concept of “found” data predicates that available data set is the whole universe from which correlations are to be created. And that is where many Big Data implementations are unable to deliver any meaningful insights.
The veracity (the 5th V) of information in a Big Data store can throw up many false positives which have been the bane of many projects. Data will never be clean unlike conventional data warehouses and the velocity will keep you challenged to move with agility. The ability to come out of the clean and complete data mindset is the beginning of what Big Data may enable. From here to get to Value is a long journey with no near-term goals; if you hit something, consider yourself lucky and celebrate.
My suggestion to my friend was to get started the way he believed he will be able to deliver what the business wanted. Forget the discussion on technology and focus on what matters, insights driven by data. If he can get traction from some CXOs based on the results, no one will grudge whether they came from Big Data or Small Data. The business leader in him understood while the technologist wanted to fight; for his benefit, I hope the business guy prevails.