By James Kobielus (@jameskobielus)
Logs of all sorts–web logs, application logs, database logs, system logs, etc.–are fundamental to the promise of the Internet of Things (IoT).
Without continuous logging of relevant events, the IoT can’t fulfill its core role as the real-time event-notification bus of the online world. Machine-readable event logging is fundamental to all the core applications of IoT, including real-time sensor grids, remote telemetry, self-healing network computing, medical monitoring, traffic management, emergency response, and security incident and event monitoring. Ubiquitous IoT will depend on the ability to support continuous real-time ingest, analysis, correlation, handling, and any-to-any routing of machine-generated information.
IoT’s development depends on implementation of a ubiquitous, general-purpose event-logging infrastructure. This global logging infrastructure must be able to support disparate relational and non relational logged data types; execution of advanced analytics against myriad logged data objects; agility to work in batch and streaming environments; scalability to support growing volumes of in-flight log data replication without choking or slowing down. Individual event logs need not be peta-scale; in fact, most IoT devices will support local logs that are constrained to their increasingly tight storage constraints and disparate form factors.
I recently came across a great article on the untapped potential for general-purpose logging infrastructure in the IoT age. Though the author, LinkedIn software engineer Jay Kreps, doesn’t specifically connect his discussion to IoT, the affinity is obvious. The two trends that he highlights as increasing the need for distributed data logging–“event data firehose” and “explosion of specialized data systems”–are at the very heart of the IoT revolution.
Kreps lays out a real-time pub-sub architecture reminiscent of the time-honored concept of an “enterprise service bus” (ESB). “[M]any of the things we were building,” he says, “had a very simple concept at their heart: the log. Sometimes called write-ahead logs or commit logs or transaction logs, logs have been around almost as long as computers and are at the heart of many distributed data systems and real-time application architectures.”
To the extent that we intend for IoT to evolve into an ESB-like infrastructure for big data applications, we must grapple with the central role of distributed logs and with the protocols that support that support distributed log-data consistency, replication, and concurrency.
It seems to me that this loosely-coupled ESB-like approach for data integration is the best infrastructure for truly flexible, increasingly heterogeneous big data, IoT, and cloud infrastructures. The log will be the common denominator data-storage and integration abstraction.
Target image via Shutterstock
Between CES 2014 and Target’s data breach mess, there was plenty to talk about in the IT industry last week. And, as always, the TechTarget writers covered it all in this week’s roundup.
1. New Samsung enterprise mobility deal may give Knox a kick start – Jake O’Donnell (SearchConsumerization)
Samsung’s new deal with the U.S. government MDM partner DMI has analysts thinking the company’s enterprise offerings could get the needed jolt it needs in the New Year.
2. Target breach update: Information on up to 70 million customers stolen – Brandan Blevins (SearchSecurity)
More bad news for Target as the company has disclosed that over 70 million customers were affected in it’s recent credit card data breach, a much bigger estimate of the previous 40 million.
3. For IBM Watson, no easy answers on commercial cognitive computing – Jack Vaughan (SearchDataManagement)
When IBM’s Watson analytics system bested ‘Jeopardy!’ champs in 2011, all of the world cheered. Now the company must up its game to make Watson a commercial success.
4. Fun tech finds of CES 2014 – Michael Epstein (NotebookReview)
During CES 2014, many quirky gadgets and devices were unveiled including a gravity-defying window washer and edible (yes I said edible) 3D-printed art. What was your favorite gadget from CES 2014?
5. CES to CIOs: Watch out for wearable technology – Karen Goulart (SearchCIO)
Speaking of CES 2014, the big story throughout the conference was “wearables” as wearable technology is here to stay.
Technology image via Shutterstock
With the New Year’s celebrations behind us, we should start looking ahead right? Well, the TechTarget writers don’t see it that way as they are still uncovering top stories from 2013 as well as looking ahead to 2014.
1. DaaS will grow when Microsoft blows up Windows licensing rules – Alyssa Wood (SearchVirtualDesktop)
One DaaS supporter, tuCloud Inc’s CEO Guise Bule, sounds off on unfair Microsoft Windows licensing rules for cloud service providers and the cloud desktop industry in 2014.
2. SDN, hyperscale data center technologies to impact strategies in 2014 – SearchDataCenter Editorial Staff (SearchDataCenter)
With many IT professionals spending a lot of time fighting fires and focusing on strategic improvements, software-defined everything, real-world implementations of converged infrastructure and a hybrid cloud strategy will be on their agenda in 2014.
3. The great eight: Popular enterprise technology stories in 2013 – Sonia Groff (SearchNetworking)
From faster switching to ambitious wireless initiatives, the networking industry went under major changes in all facets. To keep you caught up, here are some of the most popular stories that appeared on SearchNetworking.
4. HTC posts quarterly losses for the ninth time running – Brighthand Editorial Staff (Brighthand)
Even with several successful innovations lately (particularly the HTC One Max), the company has suffered another decline in sales after posting its fourth quarter earnings.
5. Searchlight list of lists: The best, the worst and what’s next in tech – Karen Goulart (SearchCIO)
Was 2013 another flop for the technology industry? Well to end this year’s Searchlight articles, SearchCIO took a look at the best and worst of technology in 2013. What should we look forward to in 2014?
Happy Friday! Now before you go off enjoying the weekend, we thought to bring you a little “IT humor” (courtesy of Imgur). Enjoy!
Christmas gift image via Shutterstock
Are you feeling the after Christmas blues? Perk up as the TechTarget writers continue to share their top stories from the past year. There may also be a few gift ideas in there too.
1. Top 10 data center trends that shaped 2013 – Sharon Zaharoff (SearchDataCenter)
Whether it has the evolution of servers, storage and mainframes or more affordable and accessible technology, the SearchDataCenter’s editors share their top 10 data center stories of 2013.
2. Businesses must cut through the big data hype in 2014 – Ed Burns (SearchBusinessAnalytics)
While there’s a ton of hype surrounding analytics and big data these days, it’s very unlikely that every new technology or strategy will pan out. Businesses will need to be able to differentiate the trends in order to be successful in 2014.
3. After Christmas gift ideas – Jerry Jackson (NotebookReview)
If you’re still trying to figure out what you should get with your new gift cards (maybe a new laptop or some nice accessories), here are some great-after Christmas presents for the technology lover.
4. The cloud looms large over IT career paths – Christina Torode (SearchCIO-Midmarket)
In the recent TechTarget “IT Salary and Careers Survey”, IT professionals believe their profession is on the cusp of significant change due to the much-hyped cloud computing industry.
5. Top 10 UK cyber security stories of 2013 – Warwick Ashford (ComputerWeekly)
As cyber security has moved up the national agenda in the past year, with the UK government paying increasing attention to bolstering UK cyber defenses, here are the top 10 UK cyber security stories of 2013.
2013 image via Shutterstock
As we prepare for 2014, it’s always great to take a look back at some of the most popular stories, quotes and news from the past year. And that’s exactly what the TechTarget writers did in this week’s roundup. What was your favorite story / quote from the IT industry in 2013?
1. SDN market predictions for New Year: NFV, OpenFlow, Open vSwitch boom – Shamus McGillicuddy (SearchSDN)
While the SDN market remained more hype than reality in 2013, SDN technology will take a big step forward in 2014 as overlays will rule the early part of the New Year, but OpenFlow and other hardware-based SDN will start to gain ground.
2. Massive Target data breach: Retailer says 40 million cards compromised – Brandan Blevins (SearchSecurity)
Target Corp. confirmed last week that its aware of a data breach impacting approximately 40 million credit and data debit cards, making it one of the largest breaches of 2013 and one of the largest retail breaches in history.
3. They said what?! This year’s most notable cloud computing quotes – Adam Hughes (SearchCloudComputing)
“Everyone needs a backup plan.” “The Achilles’ heel for Amazon is the noisy neighbor.” These were just some of the best cloud computing quotes that stuck out in our minds in 2013.
4. The top Exchange Server content of the year – SearchExchange Editorial Staff (SearchExchange)
Take a look at why these five pieces of Exchange Server content struck a chord with our readers this year.
5. Top 10 VDI news stories of 2013 – Margaret Jones (SearchVirtualDesktop)
As we saw big VDI news around popular products from VMware, Citrix and Amazon, here are some of the biggest stories from the VDI landscape in 2013.
2014 image via Shutterstock
2014 is upon us! With the new year almost here, the TechTarget writers discussed what to expect for 2014 and looked back at 2013 in this week’s roundup. What are you most looking forward to in the new year?
1. Server virtualization trends to expect in 2014 – SearchServerVirtualization Advisory Board (SearchServerVirtualization)
2013 didn’t disappoint when it came to new virtualization technology. Vendors released many important products and features over the last year, but what’s in store for server virtualization trends in 2014?
2. What’s to come for enterprise mobility in 2014? IT experts weigh in – Jake O’Donnell (SearchConsumerization)
Enterprise mobility had its ups and downs in 2013 including shake-ups at a major mobile hardware vendor and big names trying out new mobile management products. What should we expect in 2014?
3. Server managers sound off on data center duties for 2014 – Stephen J. Bigelow (SearchDataCenter)
In a recent TechTarget survey, server managers detail their data center plans for 2014 and offer insights into the systems management and career challenges they face.
4. iPad Air on plenty of Holiday wish lists – Hannah Becker (TabletPCReview)
If you believe the most recent ChangeWave survey, all indications are that Apple’s iPad Air is on the top of many Christmas wish lists and will be a top seller in the Holiday season.
5. Answers to your top UC questions of 2013 – Katherine Finnell (SearchUnifiedCommunications)
Throughout 2013, many enterprises had questions when it came to adopting and implementing new unified communications technologies, from conducting hands-free VoIP calls and the advantages of using PSTN.
New Year’s Resolutions image via Shutterstock
It’s almost 2014…you know what that means? New Year’s resolutions! Share your 2014 IT resolutions with SearchCIO next week and more in this week’s roundup.
1. What are your top New Year’s IT resolutions? – Emily McLaughlin (SearchCIO)
It’s that time of year again – time to reflect on 2013. Did you move too much — or not enough — to the cloud in 2013? Well, SearchCIO wants to hear your top New Year’s IT resolutions for 2014 during it’s tweet jam on December 18th.
2. PaaS adoption upswing predicted for 2014; will it last? – Beth Pariseau (SearchCloudComputing)
While “Platform as a Service” has generated quite a buzz this year and adoption may grow in 2014, some experts think the category will become increasingly difficult to define as the lines between cloud service types blur.
3. IT priorities in 2014 move beyond virtualization to cloud, automation – Stephen Bigelow (SearchDataCenter)
In a recent TechTarget survey, over 4,100 IT professionals shared their IT priorities for 2014 and changes coming on the horizon for data centers as well.
4. The President may be ready to give up on BlackBerry, but iPhone still too insecure – J.R. Nelson (Brighthand)
It appears President Barack Obama may be sticking with BlackBerry a little while longer as he announced he’s not allowed to use Apple’s iPhone, citing “security reasons”.
5. Ten SDN blogs worth following – Michelle McNickle (SearchSDN)
In 2013, SDN become a major factor in the networking industry, with different vendors launching new products and users beginning to evaluate and implement test beds. To keep up with the all the latest buzz on SDN, here are the top 10 SDN blogs to follow.
Holiday calendar image via Shutterstock
With the Holiday season upon us, there are still plenty of great IT conferences to choose from during the month of December. Come take a look at our calendar to see if you can attend.
1. TechTarget: Modern Infrastructure (December 3, Toronto, Canada)
2. Gartner Application Architecture, Development & Integration Summit (December 3-5, Las Vegas, Nevada)
3. Code Mesh 2013: The Alternative Programming Conference (December 3-5, London, England)
4. The 46th Annual IEEE/ACM International Symposium on Microarchitecture, 2013 (December 7-11, Davis, California)
5. 2013 HealthSummit (December 8-11, Washington D.C)
6. Raymond James Systems, Semiconductors, Software & Supply Chain Conference (December 9-10, New York, New York)
7. Gartner Data Center Conference (December 9-12, Las Vegas, Nevada)
8. TechTarget: End Users vs. Your Data Center (December 10, Denver, Colorado)
9. TechTarget: Storage for Virtual Servers and Desktops (December 10, Los Angeles, California)
10. Dell World 2013 (December 11-13, Austin, Texas)
11. TechTarget: Capitalizing on the Value of SAP HANA (December 12, Chicago, Illinois)
12. Cloud and Big Data Symposium (December 21, Herndon, Virginia)
Data science image via Shutterstock
By James Kobielus (@jameskobielus)
When you think of teams, the word “chemistry” is always lurking in your thoughts. Chemistry is what distinguishes top-performing teams from the dysfunctional ones.
Personalities are one thing, and productivity is quite another. Where data scientist teams are concerned, you are quite likely to find one dominant personality type: people who have ample curiosity, intellectual agility, statistical fluency, analytical acuity, research stamina, and scientific discipline.
However, where productivity is concerned, data scientists vary widely. Some work best in blissful isolation whereas others thrive in collaborative environments. Likewise, some data scientists are awesome polymaths who have mastered a wide range of skills, while others are strict specialists. Some are closer to the statistical analyst end of the skills spectrum, whereas others take pride in being the subject-matter expert that all the data scientists runs to when the question turns to marketing, finance, and what have you.
The productivity of the entire data-scientist team depends on being able to balance this mix of people, aptitudes, skills, and roles. It also depends on being able to incorporate new roles into the team as the nature of big data and data science initiatives evolves. For example, the notion of a “customer experience modeler” is of fairly recent vintage. It may be someone with a degree in the humanities, not mathematics and statistics.
The blend of skills in data-science teams is changing, and the chemistry among new and established disciplines will grow trickier as this trend intensifies. This new reality is the focus of a recent InformationWeek article, “How To Build An Analytics A-Team.” The piece discusses a study by Blue Hill Research in which that firm outlines several important roles within data-science organizations:
• Data visualizer: visual orientation, focusing on innovative ways of presenting data-driven insights within “instinctual graphics”
• Data custodian: quality orientation, focusing on data cleansing and master data management
• Data evangelist: application orientation, focusing on identifying new uses for big data analytics
• Contextual analyst: narrative orientation, focusing on interpreting quantitative insights within the larger business context
• Neuro-analyst: cognition orientation, focusing on how humans can best interact with data-driven analytics to drive comprehension and exploration
If you’ve already included all or many of these as distinct jobs in your initiative, you should consider creating a “center of excellence” supervisor whose job it is to build up the environment where cross-role chemistry takes hold.