Warning: Cannot use a scalar value as an array in /httpdocs/libraries/src/Document/Renderer/Html/ScriptsRenderer.php on line 69
NitroWare.net - Crucial MX500 1TB SATA Solid State Drive Review
Warning: Cannot use a scalar value as an array in /httpdocs/libraries/src/Document/Document.php on line 514

Warning: Cannot use a scalar value as an array in /httpdocs/libraries/src/Document/Document.php on line 518

Explaining SSD reviews by the Tech Media

The short version is some reviews of Solid State Drives online can be poor and uninformative. They relay benchmark results generated by sythetic tests that provide the end user no or little indication as to what improvement in user experience and responsiveness they would get from a SSD upgrade. While HDD to SSD can be demonstrated relatively easily in practice, SSD to SSD is much harder for a practical demonstration of user benefit. Worse so, the typical consumer expects 'more space, more gigabytes' on their next new PC purchase to store bulk data such as photos, video, music and iPhone backups and would be dismayed if presented with a storage solution (SSD) that is much smaller as it is perceived as a downgrade regardless of the massive speed increase.

Synthetic tests such as CrystalDiskMark, ATTO, AS SSD and Anvil provide metrics applicable to power users and not the general public, even to some IT professionals. What message does drive A doing 540MB/s versus drive B doing 535MB/s say to anyone? the real difference is so small it would not be noticed in real world unless a massive transfer occurs where this difference can save many minutes. 

The PC hardware media has been reviewing client SSDs for almost ten years now, and the first PCs from OEMs to offer SSDs as a build to order option are now almost 10 yrs old, dating from the very late 2000s. Some of these may even be in landfill now. What has been forgotten already that were attempts to market SSDs on the PATA Interface, also known as IDE. We reported on this drives here. At the time the ATTO Disk benchmark was used to demonstrate those drives to us and is still used to this day. ATTO is and was a smaller vendor of RAID controller cards and enclosures and has relied on their free benchmark for continuous publicity of their brand.

ATTO is a synthetic disk benchmark that uses the standard windows file APIs to read/write a test file to an already formatted and mounted disk partition, emulating the behaviour of any typical windows application accessing a disk.

It is one thing for a vendor to select a particular benchmark to demonstrate a drive in person or in a document for marketing purposes, it is another to base the drives specifications on the results achieved by ‘popular’ benchmarks used by media reviewers and end users.

The specifications you see that are listed on the SSD’s box, datasheet, marketing flyer or spec sheet are actually derived from common benchmark programs. This is the case for the vast majority of vendors in the industry. Tools used to determine these scores include the now old PCMark Vantage from Futuremark, ATTO Disk Benchmark, and the open source IO meter created and formerly curated by Intel and FIO tool for Linux which is typically used for enterprise drives.

Now guess what, typical hardware reviewers (including myself, I am happy to admit) will use these tools plus CrystalDiskMark, AS SSD and Anvil Disk Benchmark to benchmark drives they have for review, and what will the result will be especially if they follow a review guide provided by the vendor? The result will replicate the advertised manufacturer specs. The reviewer has now followed an infinite loop of proving a vendor claim without adding extra value to the information he/she is presenting to their readership.

A competent and respected reviewer should be able to add value to the commentary about a product or service they are providing to their readership and the industry as a whole. To answer questions to their readership 'why is this better' or 'how is this better' not 'it just is better'.

If vendor X clearly states in the footnotes of their datasheet they achieved their specs using IO meter and a reviewer replicates this, then so what? That’s not much different to someone checking if a 17” wheel is actually 17 inches. Verifying a vendors claims only goes so far and is only one part or a component of a thorough product review which provides value, direction and a message to an end user.

There are many tech review websites with hundreds of thousands of followers on their socials and native readership and many of them are very cluey in their chosen field, so why do don’t they do SSDs differently, well threes several reasons:

  • Technical knowledge - not all ‘hardware reviewers’ have professional IT or electronic engineering knowledge.
  • Variability in testing – depending on scope of review outlet, may vary between a simple mention in a roundup to a very detailed technical analysis
  • Time – turnaround time for reviews is constantly getting shorter. It is often excepted a product review is done within a matter of a few days and it’s not uncommon for only a few hours to be spent on something.
  • Interest – such reporters who have a specialization or interest in Business IT may not care about commodity client hardware

Depending on the reviewer and outlet, these caveats may be acceptable as different media outlets have different areas of focus and specialization. For instance outlets like CNet, ZDnet and Gizmodo have a generalist approach to consumer or business tech and would present an overview as to why an SSD is beneficial over a hard drive and what brands are available.

Whereas sites like Anandtech, PC Perspective, Toms Hardware, The Tech Report, The SSD Review and number of specialist European sites will go right through down to the component level, talk about how one drive has a small power advantage, use custom hand written software to test drives or even perform yearlong torture tests to determine the true endurance of SSDs versus the manufacture claim.

Speaking for myself. I want to tell the big picture and answer specific questions about the product I am writing about such as Is this device faster, why is it better, how is it different, what engineering has gone into the device to make it different or better to other or older devices. Speeds and Feeds can be obtained from the vendor. Theres not much point for us to copy and paste the vendors spec sheet to pad an article.

For SATA drives, even with custom written software there is only so much story to tell. Using the operating system’s own file functions we will typically see trends between the different benchmarks

What media can do better is come up with different scenarios to test drive endurance as well as demonstrate the overall experience difference between different model drives. How does a drive compare in game loading workloads, what about Adobe Creative Suite/Professional Content Creation, disk imaging or benchmarks like BAPCO’S SYSmark, a professional test which tests a number of typical software scenarios using real copies of these typical software in a simulated environment and gives a reference score. Now SYsmark does have some criticism in the industry but so does everything, mainly from vendors who do not want to pay membership fees to be involved in the development of such software.

PCMark also provides such trace based testing, but manufacturers rarely spec their drives with the newer versions as they no longer solely test storage.

One question I hear alot in the community is the affect of an SSD on game load times.

There is also a role for manual file copy tests. This is simply a timed test manually using a watch or helper utility to assert the total time taken to copy files to/from the SSD or to even compress files via ZIP/RAR. The issue with this type of testing is the wide variety and variability of windows systems, with the specific environment and hardware configuration affecting the result. Whereas with a specific benchmark utility there is more control over the end result especially with the tools from Bapco and Futuremark. The copy test answers a very important question, how well does my SSD copy files of varying sizes in the real world. Entry level SSDs such as Crucial’s BX200 and OCZ’s Trion and several others which used Planar TLC NAND and a weaker controller and cache setup were caught out by such tests, where certain large files such as a typical several gigabyte video file overwhelmed the SSD. Even on the other end of the spectrum, Samsung’s enthusiast/performance oriented 960 EVO ‘only’ has a 48 GB cache for the 1TB model, meaning extremely large files will be transferred at lower than advertised speeds.

In their recent tests, Anandtech have resorted to measuring power consumption for some specific file I/O workloads. I am not sure this is relevant or that people care what the power consumption of a SATA drive is. For a NVME drive in the M.2 or PCI-E Add in Card Format this is more relevant as heat generation is a product of power and a cooler running controller means better performance from less throttling. For a NVME drive this does make significant differences. Smaller, less power consuming silicon can also be cheaper to make and sell. On an engineering curiosity level this information is interesting and useful but the power consumption of SSDs generally is 'resonable' especially given the small PCBs many use, their cost and performance.

Allyn Malventano at PCPER wrote his own custom test software which is going beyond the call of duty to ensure repeatability between drives and to test different weighted workloads. But again this approach has it's limitations. It is proprietary and does not answer questions about experience with real world applications and scenarios.

PC hardware reviews need to strike a balance of being concise, delivering scientific test results and relaying the right message to their audience. The majority of publications get one of more of these right but rarely does one get all of these right. It is all of our responsively to strive to be better but unfortunately some more mainstream household name 'technology' blogs (Not PC hardware) don't take this as seriously as they see their role in the grand scheme of things as news reporters not testers or analysts. The manufacturer is always right (!) and they think it is their job to be completely objective from the product they are writing about. Diving into the depths of a product is not seen as objective to some.

I am constantly researching and testing different methods of testing SSDs and this takes alot of time. I have to present something in the meantime therefore there is a typical mix of benchmarks in this review. More applicable real world use cases not only provide more relevance and meaning to what I am reviewing, but may even be easier to test on a repeatability point of view. 

End users want questions answered such as 'will my game load faster on a SSD', indeed some even swap in a SSD for their consoles. It is the job of a professional review to answer this question or at least to best of ability.  This review would even have been more accurate if I had compared the same capacity for all drives tested but that simply is not possible due to what drives we have on hand or have been sampled by the manufacturers.

2018-09-21 | NVIDIA QUADRO RTX 6000 In action - Photoshop, Porsche RTX, Kubernetes, CLARA and Metropolis

2018-09-20 | NVIDIA RTX Live Demo - Ray Traced Porsche on Turing GPU

2018-09-21 | EPSON goes bright with laser projection and 4K

2018-09-15 | 8 NVIDIA TESLA V100 VOLTA GPU server from HP

2017-12-31 | Future of Networking with Extreme Networks - Eric Broockman CTO Interview

2018-02-16 | NVIDIA’S VOLTA is real, it works and it is finally here.

2018-02-07 | motorola harbourside chat and interview | moto x4, g5s plus, 360 camera & gamepad

2018-02-07 | Inwin X-Frame product tour and interview

2018-02-07 | ASUS Zenbook 3 Deluxe UX490 Ultrabook

2017-11-20 | VICHYPER - Australian Hyperloop Interview and Tour

2017-11-17 | Eugene Kaspersky & Kaspersky Lab Press Conference Sydney 2017

2017-11-17 | Eugene Kaspersky Keynote Sydney 2017

2017-05-21 | AMD RYZEN - Australian launch and Q&A Session | BIOS Update Detailed

2016-11-21 | Motorola Mobility VP Engineering Interview | Moto Z and Moto Mods Developer Kit for Makers & Modders

2016-11-30 | EPSON Tech Interview | 2016 Business and Professional Projectors

2016-11-30 | EPSON Tech Interview | 2016 Home Theatre Projectors 6000, 8000 & 9000 Series

2016-11-22 | EPSON HDR Home Theatre Projector Demo | EPSON TW9300W

2016-11-21 | HDR video Demo - AMD Radeon 'Polaris' RX480 with LG OLED TV

2016-11-30 | Intel Game Chamber 2016 Tour and Interviews | VR, Games, PC Mods, Overclocking and FUN

2016-11-30 | Intel Core i7-6950X Broadwell-E @ 5.62 GHz Live Overclock | Intel Game Chamber 2016

2016-08-29 | LG puts OLED in all the things - Integrate/CEDIA AUS 2016 Booth Tour

2016-08-12 | What is Software Defined Networking with Doug Murray, CEO Big Switch Networks.

2016-06-17 | NVIDIA DGX-1 launches in Australia - NVIDIA ANZ Interviewed

2016-05-01 | Build a kickass Dolby ATMOS Home Theatre - The Pros show us how

2016-04-01 | The Panasonic Toughbook CF-20 Story

2016-03-23 | Seagate 8TB Family, 10TB Helium & Mobile HDD 2016 Hard Disk Update

2016-03-21 | Panasonic Toughbook 20 : Fully Rugged detachable Laptop launch - Water, Drop & Weight Tested

2016-03-23 | ASUS ROG Gaming House Tour - Sydney, Australia

2016-03-01 | ASUS ROG GX700 Liquid Cooled Notebook Preview & Interview

2016-03-23 | League of Legends Legacy eSports Team Interview

2015-12-09 | Microsoft Store SYDNEY - Grand Opening Press Conference

2015-11-11 | HP 6th Gen Core Intel Skylake PC & Jet Intelligence Printing Launch - Melbourne Experience Centre

2015-11-11 | HP Experience Centre Melbourne Tour -Part 2

2015-11-11 | HP Experience Centre Melbourne Tour -Part 1

2015-08-28 | Parrot Interviewed - Flower Power Wireless Plant Monitor

2015-08-28 | Parrot Interviewed - ZiK 2.0 Headphones

2015-08-28 | Parrot Interviewed - AR. Drone, MiniDrones, Bebop Drone

2015-08-23 | MOTO G 3rd Gen IPX7 Water Proofing Demo & Moto X Style/Play/G Q&A

2015-08-21 | Motorola Mobility Interviewed - 3rd Gen Moto G, Moto X Play, Moto X Style

2015-08-20 | NVIDIA GEFORCE GTX 950 Launch Ep.1 | Gamestream Co-Op - A New way to share games

2015-08-20 | NVIDIA GEFORCE GTX 950 Launch Ep.2 | MOBA/DOTA 2 Latency and Smoothness

2015-08-08 | Intel SKYLAKE i5-6600K LN2 overclock at 6.33 GHz with ASUS Z170 ROG Hero

2015-08-07 | Intel Skylake i7-6700K @ 4.8 GHz & DDR4 @ 3.5GHz on ASUS Z170 Deluxe Overclocking Guide

2015-08-05 | ASUS Z170 Deluxe Motherboard for Intel Skylake 6th Gen Core CPU Overview

2015-08-05 | Review - ASUS Installation Tool for Intel SKYLAKE 6th Gen Core CPU

2015-08-02 | HP Pavilion and Envy 2015 Consumer Notebook First Look

2015-07-25 | HP Envy Phoenix 2015 Overclockable Gaming Tower First Look

2015-07-22 | Ashes Of The Singularity - DirectX 12 Hands-On Demo with AMD FURY X

2015-07-14 | Media Q&A with AMD and Richard Huddy - AMD Fury X ‘Fiji’ Graphics comes to Australia

2015-07-14 | MSI Interview - AMD Radeon R9 300 Gaming, Armour and Lightning GPUs