I recently met Mark Rodbert, CEO of Idax Software, who has an interesting theory on statistics. We often see the ‘Normal’ bell-shaped distribution – where the top of the bell represents the most likely outcomes, and the left and right tips (outliers) are rare events. Rodberts believes real world events are more likely to follow a Poisson distribution – and this has implications for IT. In this guest blog, Rodbert explains the theory:
At idax we spend a lot of time demonstrating that maths really can help describe the real world. As idax uses mathematics to identify individuals with unusual access it’s pretty important that our clients share our understanding.
Of course, people are used to getting on planes, making a phone call or using Amazon, all of which require pretty sophisticated analytics, but in the realms of big data some things are still counter intuitive. If we got two sales leads last week and 1 the week before we’re on an upward trend, if my train was late twice last week, it will be late this week, and most importantly for us, if I find several people with a high risk profile in their access then someone must be someones fault.
London 2012 – Mo Farah (Photo credit: garda)
But how likely really are these events. Well it turns out that what we need is not someone to blame, but the Poisson distribution. The Poisson is a very versatile statistical tool rather like a lopsided normal distribution, that is good for estimating event frequency, especially if the events are rare. And my all time Poisson concerns the distribution of gold medals for Team GB at the London Olympics. It seems strange to remember that at the start of the games we went a whole three days without a British gold medal. As the press shrieked that we were heading for disaster, unable to meet our targets despite massive investment, the nation held its breath. So what really were Mo Farah’s chances?
Well, as we all now know, actually pretty good. Of course only an idiot would assume that winning 29 medals over 16 days should equate to 2 every day with Sundays off, but how likely was a medal-less day. Well if you assume a Poisson distribution and take an average of 1.8 a day, the chance of a day with no medals is 16%. The chances of a super Saturday with 6 medals were actually 7%.
The bad news is that, as you can see from the chart above the Poisson doesn’t quite fit what actually happened. The good news is that a day without any golds was actually more likely at 38% of all days. The least likely (below 5) was a single gold day, which only happened once. The last day of the boxing, since you ask. So why does any of this matter? Because it shows that human beings are very bad at estimating how frequently things are likely to happen. We assume that events are evenly distributed and get confused when they’re not. Not much of a problem with gold medals; quite a big problem when you’re tying to detect fraud, rogue trading and high levels of access risk. We assume that because unusual failures are, well, unusual they are also uniformly infrequent.
So when it comes to Access and Identity Management its clear that an approach that defines cumulative controls by exception management, otherwise known as “my boss checks my access” – will perform well with the frequent but not so bad but does nothing to stop the infrequent but high risk. So the good news is that if you ask your staff why they have access to something you’ll probably remove a few copies of Visio, but you’re unlikely to spot the guy with access to the general ledger and the payments system who’s ripping the company off. Which just goes to show that what companies need is real analytical capability, and of course a bit of mathematics.
Mark Rodbert is CEO of Idax Software, the identity analytics software provider
English: A Netflix envelope picture taken by BlueMint. (Photo credit: Wikipedia)
Looking at web companies such as Amazon, eBay, Facebook or NetFlix, it is hard not to be impressed by the scale of the operations they have achieved.
Few organisations can boast the size of their customer base. The speed with which they have gone from zero to world domination has to be admired.
While the industry has done very well in finding existing business problems and building so-called solutions, it has been wholly inadequate at addressing the future problems people haven’t thought about yet.
And by doing business in ways no-one had previously considered, these web giants soar above those whose business models are limited by traditional thinking.
Unlike Nicholas Carr’s controversial Harvard Business Review essay – which questioned the relevancy of IT in business – we can learn from Amazon et al and accept that technology can make a difference. In fact, the way some organisations do IT is revolutionising the industry, to the extent that some of their innovations – such as MapReduce, the NoSQL technology originally invented by Google – is now accepted as a way of solving certain IT problems.
E-commerce, social media and smart data place demands on businesses that relational databases cannot support at speed or reasonable cost. In this week’s issue, Computer Weekly looks at how NoSQL is revolutionising data access.
The Gartner report, Capacity and performance management form the basis of web-scale IT which I covered in the Computer Weekly article on web-scale IT infrastructure- describes how eBay, Facebook and NetFlix achieve web scale computing using home-grown and open source tools. While many of the IT operations of these web giants are unfamiliar to most in mainstream IT, they give us an insight into how such organisations drive the business through their pioneering efforts.
English: eBay Logo (Photo credit: Wikipedia)
What can we learn from companies such as eBay and Amazon? These internet businesses are at the cutting edge of technology.
The recent Gartner CRM summit gave delegates an understanding of what CRM means to a web-only retailer. The processing eBay conducts to understand customers better, for example, is eye-watering. The web gives retailers incredible insights into customer service. It is not only possible to track a customer’s identity but, thanks to smart web analytics, eBay can follow the buyer’s journey.
David Stephenson, head of global business analytics at eBay, says it’s a bit like strapping a video camera to a customer’s head. Recording every interaction a customer makes means the auction site collects millions of hours of web analytics. Making sense of it all is a big data problem. In fact, eBay produces 50TB of machine-generated data daily. It also needs to process 100PB of data every day to understand what its customers are doing. Sampling this data may have worked in the past, but this only gives a statistical snapshot.
The company needs to work with suppliers to build bespoke hardware and software for its requirements, because using a traditional data warehouse would be too slow and prohibitively expensive to scale. But even a custom data processing engine cannot comprise the whole answer.
The firm uses three systems: a traditional data warehouse appliance, a NoSQL database and the custom appliance to analyse its customers’ journeys. So while it makes perfect sense for businesses of all sizes to use web analytics to understand customer interaction, an immense amount of technical investment and expertise is required to do so effectively.
During the Build developer’s conference in April, Microsoft is expected to reveal more details of a future version of Windows codenamed Threshold.
Rumours on the web suggest that Threshold could become Windows 9. The OS is set to bring together Windows Phone, Windows 8 and the xBox One operating systems.
Microsoft’s previous attempts at simplifying its various operating systems have had varying degrees of success.
Windows 2000 Workstation and Windows Millenium merged into Windows XP with a single kernel for home and professional users.
On the mobile side, Microsoft attempted to provide a common look and feel with Windows CE, and cross-platform development, but at the time, a desktop-like GUI on a smartphone did not gain acceptance.
With the evolution of the Windows Phone OS, Microsoft introduced a touch UI with tiles, that has made its way onto the Windows 8 OS. This time, however, the touchscreen UI, has not sat very well in the corporate market and home users have generally preferred cheaper Android-powered tablets over Windows 8 powered tablets.
The fact that Microsoft is looking to rebrand Windows RT, its low-end ARM-powered Windows operating system, suggests the company is moving towards a single Windows OS across all devices. Interestingly, the xBox One also runs a version of Windows .
A single core OS would greatly simplify application development and integration. For Microsoft, it would mean core services like Skype, Windows Live and Office 365 would work seamlessly between the xBox One, Windows-powered tablets and smartphones and traditional PCs.
The wider MS ecosystem would benefit – so, in theory, B2C companies could develop services once and target customers across all three platforms.
All will be revealed at Build 2014, but Microsoft has some big changes to make this year, not least, hiring a new CEO to take over from Steve Ballmer.
It has been a while since my last post. Today I spent the last few hours updating Windows 8 and I thought I’d share my experiences.
It’s a 3 GB download and will reboot your PC a few times and configure the system before booting the new OS. A network connection is required after the download for user authentication.
On my configuration I needed to enter my Windows Live login and a code that Microsoft texted to my registered mobile phone.
Finally, the machine boots up and….there is a Start button. Not quite,…my machine still boots into the Windows 8 Start screen with the sliding tiles of apps. The desktop tile does indeed bring up a Start button, but don’t hold your breath. It’s no Windows 7 start menu. It is a button that will bring you back to the Start screen. Amazing.
For anyone like me, who really wants the Start menu, download Classic Start menu. It’s free and still works on Windows 8.1.
Earlier this week Ubuntu’s CEO, mark Shuttleworth unveiled a crowdfunding model to get people to buy into his concept of a converged PC/smartphone device.
The appeal hit its first million within five hours and became the fastest crowdfunding campaign ever to reach $2 million, hitting this milestone in 7 hours, 59 minutes and 58 seconds. It has already also beaten Indiegogo’s previous highest ever campaign, which stood at $1,665,380.
In fact, at the time of writing (3pm GMT, July 24 2013), all 5000 of the $500 device have been “pre-ordered”. I’m being quite deliberate with the words I have used here: yes, I think Shuttleworth’s Ubuntu Edge campaign is a bit like pre-ordering a new brand product on Amazon, that isn’t shipping yet (like the xBox One) or Playstation 4).
OK, both these products do exist – unlike Ubuntu Edge. But, Shuttleworth is not really selling a product. He is selling an idea, allowing people to get in at an early stage and become early adopters.
The campaign aims to raise $32 million over 30 days, for a limited production run of 40,000 devices.
Raspberry Pi (Photo credit: CesarCardoso)
It’s been a while since I raved about losing a weekend getting to grips with Linux on a home project to build a music server.
It took a weekend to get the open source Squeezebox music server software to run on Ubuntu – that was two years ago.
Now thanks to the Raspberry Pi, it is entirely possible to get going in under an hour – if you can Google instructions and are comfortable copying and pasting bits of code.
All you need is a £30 Raspberry Pi (the Model B with 512 MB of memory), a micro USB power supplier (standard Android phone charger should work), a 4 GB or bigger SD card (I used a £12 8 GB class 10 card from Maplins), and a network cable to plug the Pi into your router.
There’s plenty of places to find out how to get started. I’ll link to the stuff I found useful.You’ll need to decide which Rasberry Pi Linux distribution to use, download it from a standard PC then copy the “image” file onto the SD card.
You will need a special program to copy an image of the Raspberry Pi Linux distribution to the SD card, as standard Windows file copy will not work. Again a Google search will show a number of applications that will work: I used Win32 Disk Imager.
Unplug the SD card and plug it into your Pi, connect the network cable to your router and boot up!
This gets you going, the Pi will light up and you’re up and running but you won’t be able to see anything as no monitor or keyboard is connected.
My preferred way to work with a Linux system is by remote access using the excellent Putty tool to connect using SSH.
To use Putty you will need to find the IP address of your Pi. This is found though your router management console. On my D-Link router I log into http://192.168.1.1 and click on the Network tab, which lists all the devices connected to my home network. It detected the Raspberry Pi – mine had a machine name of RaspPi, and I was able to reserve the IP address.This way, I can use Putty with the same IP address to log into the Pi.
Once Putty is running, you will need to log in using the user name (pi) and password (raspberry).
The Linux distribution I chose was SqueezePlug. This allows you to set up a SqueezeBox music server and/or a player to access your existing server. The player can be controlled wirelessly from an Android or iOS device (such as the free Logitech Squeezebox controller in the Google Play store).
That’s it – and it can be installed and running in under an hour.
Going forward, try the Raspbmc Linux distribution, which allows you to run a media centre on the Raspberry Pi. I have been able to use the Pi to stream Freeview, BBC iPlayer and ITV Player to my Nexus 7 Android tablet using the TVHGuide client app. it is also possible to access these streams from any device running an XBMC client.
This time, you need to plug the Pi into a display, you’ll also need a powered USB hub, a compatible USB digital TV receiver and a Wi-Fi adapter. There’s quite a few steps:
I recently spoke to Kim Stevenson, CIO at Intel. When she started in IT, people used to try to issue one device per employee. She says Intel has been running BYOD for three years. “People like choice and they pick devices for the work they wish to accomplish.” At Intel, this means becoming a more productive employee. So as an IT professional, she says it is important to understand this driver, rather than try to resist the change and loss of control that occurs through BYOD.
She says: “We are in an era of business productivity. It is perfectly reasonable for employees to have seven devices. Through our BYOD programme we have documented a gain of 57 minutes in productivity per employee.”
For IT professionals, she says the Number One issue they face is “velocity.” Business unit managers can buy a service directly from a service provider. the consumer It experience is better than enterprise IT. She says IT must address how to deliver the consumer IT experience within the confines of the enterprise.
In this guest post Martin Prendergast, CEO and co-founder, Concorde Solutions and board Member, Cloud Industry Forum, writes about issues to consider when licensing IBM software.
Enterprise software can represent as much as 30% of an organisation’s IT spend, so at a time when budgets are still being squeezed like never before, CIOs are understandably being careful to ensure that their investment in software represents value for money.
However, software licensing costs can be a real bugbear for CIOs, with the potential to quickly ratchet up the overall price through painful non-compliance fines, unwittingly incurred as a result of software vendors’ complex and convoluted terms.
The challenge is exacerbated as each software vendor has its very own unique brand of complexity, which makes the jobs of the IT Asset Manager, the CIO and the CFO even more taxing. In this article, we examine some of the key challenges and solutions for dealing with IBM’s software licensing.The problematic portfolio position.
IBM has over 1,500 products on offer available on around 30 licensing metrics; each metric may differ only very slightly, but can still have a significant impact on licensing requirements and position. Historically, the picture has been further complicated with IBM through its well-known practice of acquisitions, expanding the product portfolio and licensing metrics even further. IBM may choose to retain the licensing metrics of the company they acquire, and sometimes may choose not to.
For customers this can be incredibly difficult to track; and without careful management and analysis of their IT estate, businesses can find themselves operating under altered metrics and contracts without realising. It goes without saying that non-compliance fines can often be the result of this – and large software vendors, as we know, have found a lucrative income stream in such levies.It’s relatively widely known that IBM has a tendency to be one of the most aggressive vendors on the market when it comes to non-compliance. IBM’s fines, which can include a two year back penalty on maintenance clauses in addition to the costs of ‘missing’ licenses, are considered harsh even in comparison to other large vendors.
Indeed, just a few years ago, IBM sought to audit all of its corporate customers without warning and with huge audit teams, which netted them a considerable amount of income. Of course, IBM isn’t the only vendor that is a fan of the surprise audit and there are a couple of things that businesses can do to ensure that if an audit arrives, they’re not caught unaware.
1. Preparation is the first line of defence – ideally businesses should seek independent third-party confirmation of their licensing position both pre and post audit.
2. IBM has now lengthened the list of its products that are eligible for its sub-capacity licensing.
3. Dealing with sub-capacity licensing – irrespective of how enterprises partition their machine, without a sub-capacity license in place, they may still get charged for full-capacity.
Denali is the holding company through which Michael Dell hopes to reinvent Dell. A US securities and exchange filing at the end of March shows the company will actively move away from the PC and high volume servers.
- Denali hired John Swainson, former CEO at CA to run the software business. The company will look at expanding its business into areas like BI and storage software, presents a huge opportunities, software as a service will eat away revenue.
- While Denali has benefited from the trend to migrate workloads from expensive Unix systems to commodity x86 servers, Gartner notes that this potentially short term. According to Gartner, the move to virtualisation and server consolidation will enable businesses to defer server purchases
- A section of the SEC filing prepared by J P Morgan reflects this challenge. The investment bank highlighted Denali’s management plans around reducing margins from end user computing devices, servers and storage to reflect increasingly aggressive competition and buyers spending less.
- The acquisition of Force 10 in 2011 will help drive networking sales. IDC expects the networking business to grow 7.3%. Gartner expects sits software revenue to grow 7.7%, due to the acquisition of Quest. Storage is expected to suffer as a result of the decline in its long-standing relationship with EMC.
- On the services side, J P Morgan notes that Denali should see modest growth in its PC maintenance business, but competitive pricing will put pressure on traditional outsourcing.