Server Storage Selection: HDD, SSD & NVMe

Server Storage Selection: HDD, SSD & NVMe

During server selection, CPU and memory get the attention, but storage is usually the real performance bottleneck: the CPU can wait, a slow disk can stall the whole application. This guide covers media, metrics, RAID and capacity planning for a complete storage decision.

Positioning of the three media types

• Sequential throughput suits large file reads and writes (video, backups, logs).
• Random IOPS matters most for workloads with many small requests, such as databases and message queues.
• Latency has the most direct impact on interactive apps (checkout, search).

Measure with commands, not marketing

Instead of trusting spec sheets, benchmark a server's real storage. First, look at the disks and mounts:

lsblk -o NAME,SIZE,ROTA,TYPE,MOUNTPOINT
df -h

In lsblk, ROTA=1 means a spinning disk and ROTA=0 means solid state — one glance tells you whether the vendor swapped in a lower tier. For random IOPS and latency, fio is the standard tool:

fio --name=randread --rw=randread --bs=4k --size=1G \
    --numjobs=8 --runtime=30 --group_reporting

The iops and clat (completion latency) numbers are what a database actually cares about. If 4K random-read IOPS are only in the hundreds, that disk is not for a database — no matter how fast its sequential reads look.

Match disks to workloads

• OS / system disk: start with NVMe; boot and software loading are noticeably faster.
• Databases: NVMe or high-performance SSD, prioritizing random IOPS; with enough memory, database deployment tuning further reduces disk pressure.
• Web static assets: SATA SSD is enough, with a CDN on top (CDN acceleration).
• Backups/archives: HDD offers the best unit cost; see cloud server backup strategy.

How to choose RAID

RAID combines multiple disks for performance and fault tolerance. Common levels:

Decision: use RAID1 for the system and critical data; RAID5/6 when you need capacity with fault tolerance; RAID10 for databases and high-I/O loads; only consider RAID0 for regenerable temporary data. For hardware pairing see dedicated server selection.

On bare metal, creating a RAID array (for example two NVMe drives in RAID1) usually goes through mdadm:

mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/nvme0n1 /dev/nvme1n1
echo 'DEVICE /dev/nvme*' | tee /etc/mdadm.conf
mdadm --detail --scan >> /etc/mdadm.conf

Make sure the disks hold no data before creating the array — the process overwrites the partition table. In the cloud, many vendors emulate RAID semantics with distributed storage, so what matters more is the vendor's replica count and availability SLA rather than local mdadm parameters.

Block, file and object storage

In the cloud, also distinguish storage types:

• Block storage: attached to a single server as a disk; good for databases.
• File storage (NFS): shared across machines; good for shared directories and clusters.
• Object storage (S3-style): massive unstructured data and static assets, low cost and easy to scale; good for backups and media.

Capacity and performance planning

• Size capacity as "current usage x growth factor x redundancy (RAID overhead)", leaving 30-50% headroom.
• Judge performance by P95 random IOPS, not just sequential reads.
• Separate hot data (NVMe) from cold data (HDD) with tiered storage to control cost.
• Snapshot and backup strategy is in cloud server snapshot and backup and website backup strategy.

Do not forget flash endurance (TBW)

SSDs have a write endurance limit, measured in TBW (total bytes written). Two NVMe drives can look similar in price but differ by several times in endurance: enterprise-grade (TLC/MLC, high TBW) vs. consumer-grade (QLC, low TBW). Selection notes:

• For continuously written workloads like databases and logs, always choose enterprise SSDs and check that the TBW covers "daily write volume x 3-5 years".
• For read-mostly web static assets, consumer SSDs are fine.
• For cache disks that get rewritten constantly, prefer a smaller drive with high TBW to avoid early wear-out.

Capacity sizing example

Take an e-commerce site with 10,000 daily active users: product images plus database total about 200GB. With 1.5x yearly growth and 50% RAID1 overhead, starting capacity should be 200GB x 1.5 x 2 ~= 600GB; with 30% headroom, a 1TB NVMe system disk plus a separate data disk is a safe choice.

The example also works in reverse: on a tight budget, move static assets like product images to object storage, give the database its own NVMe data disk, and use a medium NVMe for the system disk — total cost can drop 30%-40% versus "everything on large NVMe" with almost no change in user experience. Storage selection is not a single-choice question; hot, warm, and cold data each have a home, and that is how you spend money where it counts.

Common mistakes

1. Using sequential speed (e.g., "500MB/s read") to judge database experience -- databases care about random IOPS.
2. Ignoring RAID fault tolerance until data is lost.
3. Putting everything on NVMe without planning, mixing hot and cold, and losing control of cost.
4. Looking only at capacity and ignoring TBW, so log-heavy workloads wear out flash early.

FAQ

Should I prioritize IOPS or capacity when buying? Depends on the workload: databases want IOPS, backups and archives want capacity. Should system and data disks be separate? Yes — if the system disk fails, your data survives and a reinstall is faster. RAID5 or RAID10? For high-I/O loads like databases, RAID10 is safer; for capacity-first general file storage, RAID5/6 is more economical. Can SSDs fail suddenly? They expose health metrics (for example Wear_Leveling_Count in smartctl -a), so periodic checks catch wear-out risk early.

References: RAID (Wikipedia) https://en.wikipedia.org/wiki/RAID; NVMe specifications https://nvmexpress.org/; fio documentation https://fio.readthedocs.io/; mdadm manual https://raid.wiki.kernel.org/