Consumer Reports has unleashed a firestorm on Microsoft after suggesting its Microsoft Surface line suffered from unreliability.1 The Microsoft Surface series of devices were launched partly in response to declining PC sales. Competitors such as Google Android started selling devices directly on Google Play. This posed a significant threat to the Microsoft Store, which lacked any reasonable competing device. Having good reason to make certain that it can handle the competition, the Microsoft Surface for Windows RT was launched. In the five years since then, Microsoft has succeeded in establishing the Microsoft Surface as a premium line. It has proven capable of competing with laptops, tablets, and even 2-in-1s. Make no mistake- The Surface Pro, the Surface Laptop, and Surface Book, are the hardware crown jewels of the company. Just look at the past third quarter to see how much of an impact the Surface devices have on the company. For lower revenue and a dip in stocks, Microsoft blamed more competition in the 2-in-1 segment and lower Surface device sales.2 Rising concerns on issues with two of its main devices does not bode well for Microsoft.
The news of Microsoft’s struggle to maintain a functional flagship broke out with a survey from Consumer Reports that suggested about a quarter of Microsoft Surface devices sold would break within the first two years. This affects the new Surface Pro, the Surface Laptops, and the Surface Book. On the surface (no pun intended), this doesn’t sound like a big deal. However, many regard Consumer Reports as a good indicator of what users think about products. The method CR used to make this dire prediction on reliability uses a combination of lab testing and user reports. The findings show that statistically Microsoft Surface devices are more likely to encounter problems than other competitive brands. Nevertheless, it also notes that individually some of these devices under lab testing do perform exceptionally well. So, what’s going on here?
Digging Deeper To Find The Issue
Microsoft responded defensively stating, “we don’t believe these findings accurately reflect Surface owners’ true experiences or capture the performance and reliability improvements made with every Surface generation.” Despite Microsoft’s attempt to discount the quality of these findings, we managed to find other clues that point out to a systemic problem. But first, it’s important to analyze the data that suggested the Microsoft Surface devices were not reliable. CR uses a sample of 90,741 subscribers that had bought brand new devices from the year 2014. In comparison, data suggests that 1.6 million Surface devices were sold just in the year 2015 3. The sample size, while small and prone to bias, is not necessarily inaccurate. In that sample, it appears to be that the majority of the complaints are related to startup, shutdown, and touchscreen responsiveness. Once again, we’ll put aside looking into the actual issue to instead confirm that there is a true statistical problem.
A Story of High Returns and Internal Company Memos
For that, look no further than a graphic that shows the moving average of Surface device returns according to Paul Thurrott 4.
The graph (which looks like it has gone through jpg hell before coming back to earth) suggests a high amount of returns have plagued the Surface devices. Thurott makes several bold claims about the situation. He suggests that Microsoft had knew about driver issues and related it to Intel’s Skylake. But when Microsoft asked Lenovo engineers how they were able to circumvent these driver issues, Lenovo’s engineers replied that they were not experiencing similar issues. From that point on, according to Thurott’s version of events, Microsoft has been trying to solve driver issues that emanate from Microsoft’s custom firmware and not from Intel. Thurott doesn’t stop there, but the story only gets more bizarre. For example, he also suggests that internal Microsoft memos suggest Windows 10’s push for ARM was based on the company’s inability to fix its custom firmware and that AMD “wasn’t up to the task” of “counter[ing] to [Intel’s] dominance” in late 2016. While many started reporting these as reliable news, it’s clear that no else but Paul Thurott has seen these supposed memos. A quick fact-check was needed. It turns out that Microsoft has been speaking about ARM architecture for a while. Popular in mobile devices, it was needed to secure Windows 10’s stance as an Operating System against mobile-popular systems such as Google Android. There is also little evidence that Microsoft was upset with any chipset producer.
Who’s Right About the Surface?
The talk about Microsoft memos are overblown. But here’s what we were able to find. Lenovo’s 2-in-1s and tablets do not appear to suffer from the exact same issues as the Microsoft Surface. The Lenovo Yoga series tablets did experience a bout of touchscreen issues, but those were related to display manufacturers instead of driver issues. And what about the high return rate graphic, you might ask? The return rate was and still is high for an average, but it also does show significant and more importantly, consistent, improvement.
There are signs that driver issues are being resolved. On July 14, Microsoft responded to users’ complaints about random shut downs and hibernation issues with a fix 5. As a result, it seems that Microsoft does have a point when it told Consumer Reports that the Surface line has been improving. Unfortunately, as of August 14th, 2017, a fix for touchscreen issues has not been issued. Two possibilities remain for that issue considering that the majority of users claim that the issue wasn’t present when they bought the device. The first possibility, if true, would be terrible for Microsoft. That is, it may be possible that the touchscreen is rapidly degrading due to inherent technical issues with the way it is manufactured. It would be the worst case scenario as it cements Consumer Reports’ claim that Surface products are unreliable. Older Lenovo products have met similar demises. The second possibility, is that a firmware update is causing these issues. That still begs the question of why Microsoft hasn’t resolved it yet, and how it didn’t affect all users uniformly.
Should You Buy A Surface Product?
In spite of these issues, it is clear that Microsoft is aware of issues with its custom firmware and has been working on it for months if not years. Various claims that Microsoft has been ignoring its custom firmware issues by pushing the blame towards other companies is far too overblown. Instead, it appears to be that Microsoft is slowly correcting these issues. Hopefully, it hasn’t hit a road block due to irredeemable hardware issues (referring to the display in particular). There is only one question left to answer now. Should you buy a Microsoft Surface product? It’s up to you on how much reliability matters, but the answer is, probably not. The Microsoft Surface represents a premium line and comes with a premium price. The Surface Laptops start at $999 and the Surface Book starts at $1,349. And that’s including a $150 discount! The cheaper Surface Pro starts at $799. If you’re a premium shopper, you most likely expect reliable productivity. Thus, we can’t say that the Surface Pro, Surface Laptop, or the Surface Book has met the expectations for its target audience. Maybe, shoppers will have better luck next time with the next Surface device refresh?