Are modern drives able to be low-level reformatted by an end user or is this strictly a factory OEM process

firmwareformattinghard drive

As is well known, modern HDDs (think 4 to 8TB SAS/SATA) are low level formatted by the manufacturer before sale, ready to hold data. "Reformatting" usually just means clearing the user data content within the existing tracks/sectors (this includes wiping track 0, or "quick formatting").

Manufacturers allow download of tools for their drives (such as Seagate's "seatools"), which can do something described as "low level formatting", but its not at all clear to me if this is simply a "lower level reformat" or a true low-level format.

One reason for my doubt is that with modern drives, I'd be surprised if the original track write wasn't part of a larger process, where the data obtained is then fed into calibration and control parameters governing the details of head control for each platter and the unit as a whole, so that manufacturing variations in nanometer-scale electronics and head servos etc, or variations in the microcontroller or HDD firmware, result in internal parameters exactly suited to governing that specific drive. Also to improve yields, drives that are slightly sub-tolerance might have slightly different control parameters that don't push them so hard (so they can still be sold as intended). Furthermore because two PCBs for the same model may not be interchangeable so possibly the issues go deeper.

In which case perhaps the platters can be truly low level reformatted (because the physical components and parameters are unchanged once initially assessed/measured)… or perhaps they can't. Maybe rewriting the tracks would only make the drive's parameters suboptimal rather than unusable.

So my question is, can an end user these days truly low-level format a modern HDD? Meaning, rewrite the actual data tracks as if the disk platters were not already formatted and had no tracks or data whatsoever, and it works? I also mean realistically and outside a lab/production line – whether it can be done via software-only in a home or small-scale environment, is necessary software available that an average end-user would need. There's some "get-out" clauses I've got to mention, to make the question more focused:

  1. I'm assuming that any firmware/flasher encryption keys are a non-issue but they might be a serious obstacle in reality. Are they?

  2. I'm also assuming the user can pull off the PCB but doesn't have a clean room, and really doesn't want to dismantle anything if they can format using a software-only approach through usual SAS/SATA ports. The question is really about whether an ordinary person can realistically truly low-level reformat modern HDDs, not whether one can hack their innards to levels worthy of a DefCon paper :)

  3. I'm assuming that the user can (if they wish) control or rework the firmware flasher which means in theory if there's no other way, they can get the controller to do anything it's physically capable of doing, including any data write. (Assumes no insurmountable keying or dismantling issues). But perhaps the controller is physically incapable of reformatting outside OEM situations for some technical reason, or OEMs might burn a link in the chip itself before sale to prevent some actions. So there may be physical limits on some actions, and these might or might not block low level reformat.

  4. It could be the case that low level reformat is possible but not realistic, because you can't do it due to the callibration issue, unless you have some hardware or specialist equipment/parameter calculators that's needed, and which isn't realistic to obtain/access and there isn't a readily downloadable equivalent to reverse-engineer or replicate. In that case some businesses could do it commercially, with bundled control systems available to OEM partners, researchers and data recovery labs, but not really anyone else. That's useful to know but doesn't really say how far a person without those contacts and resources can go. It essentially says "yes, if you're able and willing to replicate the key part of the OEM production line to do it".

  5. Last, one might be unable to low level format as it stands because of the calibration/params issue, but could hack a simplified firmware or homebrew that doesn't rely on such highly tuned params, flash it, and do what they like on the platter. Again useful to know but I'm really thinking about a low level reformat that let's the drive be used afterwards with the standard manufacturer's firmware, ideally without affecting reliability much if at all.

Why might an end-user want to low level format? Most won't. I can think of three reasons, running from possible to esoteric:

  • An old drive is discovered, for some reason it seems to spin up but bit flips and degradation have somehow degraded the manufacturer low level tracks (but magically not the mechanism or EEPROM/NVRAM firmware/params), and the user is curious if it can have its tracks rewritten and if so, if it would then be usable again, as a kind of hobby/experiment.

  • The user wants to do something akin to old 1970-1980 era hobbyist floppy disk systems, where you wrote your own disk access code and stored data on the disk arbitrarily as you chose. The user has figured out the JTAG and microcontroller, or can unlock OEM functions or reprogram the firmware, they'd now like to know if the drive is physically capable of low level format and, if so, if they are stuck with the tracks the OEM laid down (and modifying inaccessible data within that pre-created track/sector structure only), or if they can rewrite low level from scratch.

  • The user is paranoid and/or works in very high sensitivity areas, and wants to low level format before use, in case the NSA has persuaded the major 2-3 drive manufacturers to add code buried in the low level format (skipped sectors, empty spaces etc). (As an aside this can be done, and in any case extended disk firmware, parameters, cache dumps on power loss, etc is routinely held on non-user areas of the drive as part of ordinary operation. So it's not entirely paranoiac). The user can access and check the firmware (see previous example) but wants to low-level reformat drives they buy before use, to ensure they know exactly what's stored rather than taking it on faith.

So – is this technically possible with modern HDD manufacturing approaches (for majors such as WD/Seagate/Toshiba), and to what extent is it possible?

Best Answer

can an end user these days truly low-level format a modern HDD?

No, because there is no ATAPI command in current versions that provide such capability (e.g. there are essentially only sector write and read commands). Also there is no seek command anymore.
Since the actual drive geometry (numbers of cylinders, heads, and sectors per track) is no longer known by the user (or anybody/anything external to the drive), track format and explicit seek operations should not be performed.

For sure, "low-level" format (i.e. write the track with address marks, IDs, and gaps) disappeared when HDDs started using zoned-bit recording. The "low-level" format became an endangered command when IDE drives fixed the sector size to 512 bytes.

Related Question