M1 NVMe compatibility reports

[mctom]

I see there is a list on M1 wiki with compatible NVMe drives, so I assume more entries would be welcome.
I just ordered a super cheap generic OEM NVMe, we'll see how it goes. I made sure I can return it this time.

EDIT: Newcomers, please see the proposed "report format" in the @odroid's post below.
The optional benchmark instructions are available on the wiki:
https://wiki.odroid.com/odroid-m1/hardware/nvme

[odroid]

Good idea.
Once you post a test result, I'll make this thread STICKY.

I hope you can fill the following items.

1. Brand :
2. Model :
3. Capacity :
4. NVMe controller and firmware version : (if available)
5. Additional comments : performance (fio and iozone3 benchmark) or any other notes
6. Price and where to buy :

[istanbulls]

It was the first thing that came to my mind when I saw the M.2 NVMe list on the wiki.
I would like to join the party with Leven JP-600 256 GB M.2 NVMe.

I think about $27 can be considered cheap.
https://www.amazon.com.tr/gp/product/B0 ... UTF8&psc=1


1. Brand : LEVEN
2. Model : JP600
3. Capacity : 256GB
4. NVMe controller and firmware version : SM2263XT
NAND type flash memory: 3D TLC (Tri-Level Cell)
6. Price and where to buy : 394TL (27$) Amazon.com.tr
5. Additional comments : performance (fio and iozone3 benchmark) or any other notes VERY SOON!

[mctom]

I can now confirm my cheap drive works perfect.

Brand : Phison? (It's an OEM drive taken from a new laptop)
Model: E12C-128G-PHISON-SSD-B3
Capacity: 128GB
NVMe controller and firmware version: ECFM12.3
Price and where to buy: 79 PLN ($18.5) allegro.pl (auction site)

I ran iozone -a and got a wall of numbers.. If we want to collect benchmarks from end-users, I think a command should be specified so everyone posts results in a compact and comparable format.

[tobetter]

I can now confirm my cheap drive works perfect.

Brand : Phison? (It's an OEM drive taken from a new laptop)
Model: E12C-128G-PHISON-SSD-B3
Capacity: 128GB
NVMe controller and firmware version: ECFM12.3
Price and where to buy: 79 PLN ($18.5) allegro.pl (auction site)

I ran iozone -a and got a wall of numbers.. If we want to collect benchmarks from end-users, I think a command should be specified so everyone posts results in a compact and comparable format.

Can you try run these two comand in the diretory where the NVMe is mounted and share the figures?

Code: Select all

sudo fio --filename=/dev/nvme0n1 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2

[tkaiser]

If we want to collect benchmarks from end-users

...a few general prerequisits have to be met

• The M.2 slot is Gen3 x2 so any x4 SSD will be bottlenecked significantly, especially Gen2 x4 SSDs

• Users might struggle with cleaning contacts and correctly inserting the SSD into the M.2 slot

• Depending on settings SSDs might enter a deep sleep state or optimize their behaviour based on access type. Testing with small write patterns might result in very low transfer speeds

• Especially cheap SSDs are only able to write at high transfer rates for a very short time (or amount of data) until an internal cache is full

• identical environment on all participating M1 systems: e.g. same ext4 settings, enough time between mkfs.ext4 and benchmark execution, performance cpufreq governor and so on

To address problems N°1 and 2 it's necessary to check lspci -v and dmesg output directly after each benchmark run (which speed and link width has been established, did any problems occur at the physical layer?)

To address problem N° 3 write sizes need to be tested (100M, 500M, 1000M) to get the minimum size of data until write performance stabilizes.

To address problem N° 4 again write sizes need to be tested to determine if there's a write cache present and how the SSD behaves when data still fits into the cache or if the cache is full.

As an example: If you test with iozone -e -I -a -s 100M -r 4k -r 16384k -i 0 -i 1 write performance might be reported as low as 600 MB/s at 16M blocksize. When testing with -s 500M with 16M blocksize the test might report 1500 MB/s. And once you test with -s 1000M and the SSD has a write cache just 1G in size the 16M blocksize will show horribly low values.

So good luck defining the one iozone/fio command line that does not just generate numbers without meaning with cheap junk SSDs

[mctom]

Can you try run these two comand in the diretory where the NVMe is mounted and share the figures?

Code: Select all

sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2

It's mounted at / so odroid's home will do, right?
The first didn't want to work, neither in ~, nor in /tmp.

Code: Select all

odroid@gnome-desktop:~$ sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
fio: refusing extend of file due to read-only
fio: refusing extend of file due to read-only
fio: refusing extend of file due to read-only
fio: refusing extend of file due to read-only
odroid@gnome-desktop:~$ sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Iozone: Performance Test of File I/O
                Version $Revision: 3.489 $
                Compiled for 64 bit mode.
                Build: linux 

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                     Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                     Vangel Bojaxhi, Ben England, Vikentsi Lapa,
                     Alexey Skidanov, Sudhir Kumar.

        Run began: Tue Apr  5 18:38:04 2022

        Include fsync in write timing
        O_DIRECT feature enabled
        Auto Mode
        File size set to 102400 kB
        Record Size 4 kB
        Record Size 16 kB
        Record Size 512 kB
        Record Size 1024 kB
        Record Size 16384 kB
        Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    36288    85802    97865    97764    34297    85084                                                                
          102400      16   152476   299172   330765   269799   102622   295968                                                                
          102400     512   495564   507405   806451   795190   787934   511202                                                                
          102400    1024   512225   511457   919059   912427   974837   496028                                                                
          102400   16384   504314   508588   859608   854846   938632   509143                                                                

iozone test complete.

[domih]

Samsung model: SSD 970 PRO 1TB is AOK.

See iozone3 testing there: viewtopic.php?p=346475#p346475.

[tobetter]

Samsung model: SSD 970 PRO 1TB is AOK.

See iozone3 testing there: viewtopic.php?p=346475#p346475.

Thanks, added to Wiki.

[tobetter]

I can now confirm my cheap drive works perfect.

Brand : Phison? (It's an OEM drive taken from a new laptop)
Model: E12C-128G-PHISON-SSD-B3
Capacity: 128GB
NVMe controller and firmware version: ECFM12.3
Price and where to buy: 79 PLN ($18.5) allegro.pl (auction site)

I ran iozone -a and got a wall of numbers.. If we want to collect benchmarks from end-users, I think a command should be specified so everyone posts results in a compact and comparable format.

Thanks, added to Wiki.

[domih]

Samsung model: SSD 970 EVO Plus 500GB is AOK.

See iozone3 testing there: viewtopic.php?p=346494#p346494

[domih]

NOT NVMe but SATA III

HGST (Hitachi) model: HUH721010ALE604 is AOK.

WARNING: the M1 12V rail does not seem to have enough oomph to start the drive. So I went with a separate 12V source. See attached photo.

CONCLUSION: If you want to use a 10TB spinning rust drive(*), better use an SFX PSU + breakout board + custom jack cable to also power the M1 with the SFX PSU.

(*) Or any spinning rust drive that the M1 12V rail can't power.

See iozone3 testing there: viewtopic.php?p=346494#p346494

[mad_ady]

WARNING: the M1 12V rail does not seem to have enough oomph to start the drive. So I went with a separate 12V source. See attached photo.

In the product description it's mentioned that the SATA power port doesn't have 12V, so 3.5" drives won't work out of the box.

[domih]

WARNING: the M1 12V rail does not seem to have enough oomph to start the drive. So I went with a separate 12V source. See attached photo.

In the product description it's mentioned that the SATA power port doesn't have 12V, so 3.5" drives won't work out of the box.

OK, found it:

<<...However, since the SATA power port doesn’t have a 12Volt power rail, a separated external 12Volt SATA power supply is required to use a 3.5inch HDD...>>

I might be obtuse but does this suggests that it only provides 5V and SSD are OK with 5V and don't need 12V?

[odroid]

Most of 2.5" SSD and HDD devices don't need a 12V power rail.

[tobetter]

Can you try run these two comand in the diretory where the NVMe is mounted and share the figures?

Code: Select all

sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2

It's mounted at / so odroid's home will do, right?
The first didn't want to work, neither in ~, nor in /tmp.

Code: Select all

odroid@gnome-desktop:~$ sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
fio: refusing extend of file due to read-only
fio: refusing extend of file due to read-only
fio: refusing extend of file due to read-only
fio: refusing extend of file due to read-only
odroid@gnome-desktop:~$ sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Iozone: Performance Test of File I/O
                Version $Revision: 3.489 $
                Compiled for 64 bit mode.
                Build: linux 

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                     Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                     Vangel Bojaxhi, Ben England, Vikentsi Lapa,
                     Alexey Skidanov, Sudhir Kumar.

        Run began: Tue Apr  5 18:38:04 2022

        Include fsync in write timing
        O_DIRECT feature enabled
        Auto Mode
        File size set to 102400 kB
        Record Size 4 kB
        Record Size 16 kB
        Record Size 512 kB
        Record Size 1024 kB
        Record Size 16384 kB
        Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    36288    85802    97865    97764    34297    85084                                                                
          102400      16   152476   299172   330765   269799   102622   295968                                                                
          102400     512   495564   507405   806451   795190   787934   511202                                                                
          102400    1024   512225   511457   919059   912427   974837   496028                                                                
          102400   16384   504314   508588   859608   854846   938632   509143                                                                

iozone test complete.

Sorry. the command in my post was wrong so it's fixed. Please add --filename=/dev/nvme0n1 to the fio command.

[tkaiser]

Most of 2.5" SSD and HDD devices don't need a 12V power rail.

It's save to assume that consumer 2.5" SATA drives (be it spinning rust or flash) can cope with a stable 5V relying on the 5V rail only.

Known exceptions (but these are expensive server parts):

• Intel SSD 730: can use either 5V or 12V rail preferring the latter

• WD Velociraptor 15k HDDs that need both rails and are sold in an 'IcePack' adapter frame to fit into 3.5" slots only. While the drive itself is 2.5" (as almost all server HDDs back then since less diameter resulted in lower seek times -> better random I/O) this adapter frame made it obvious that 12V are needed too

[tkaiser]

@odroid and @tobetter: basing on the insights here viewtopic.php?p=346720#p346720 maybe it's a good idea to add a small performance tip to the NVMe SSD section of the wiki?

Gen3 SSDs are needed to get the best performance with M1 due to RK3568's PCIe limitation being Gen3 x2 only. As such every Gen2 x4 SSD will only be able to establish a Gen2 x2 connection halfing its sequential transfer speeds as well as harming random I/O performance at least with larger block sizes (1K, 2K or even 4K accesses might not be affected since SSD itself be the bottleneck especially if it's cheap or older or both).

For consumers it's important to check the PCIe link speed the SSD is capable of (unfortunately all the time wrongly declared as protocol revision: 'PCIe 2.1' for example) since even if a Gen3 SSD and a Gen2 SSD show same performance in an advertisement (e.g. '1500 MB/s') the SSD that is only capable of Gen2 speeds will end up in M1 being limited to 800 MB/s max and very likely showing lower random I/O IOPS as well.

[odroid]

@odroid and @tobetter: basing on the insights here viewtopic.php?p=346720#p346720 maybe it's a good idea to add a small performance tip to the NVMe SSD section of the wiki?

Fully agreed. Thanks,
I've added a note in Bold fonts.
https://wiki.odroid.com/odroid-m1/hardw ... le_m2_nvme

[egil]

Brand : Kingston
Model : SNVS500G
Capacity :500.11GB

Performance:

Code: Select all

fio: READ: bw=1507MiB/s (1580MB/s), 1507MiB/s-1507MiB/s (1580MB/s-1580MB/s), io=14.8GiB (15.9GB), run=10043-10043msec
iozone:
        Iozone: Performance Test of File I/O
                Version $Revision: 3.489 $
                Compiled for 64 bit mode.
                Build: linux

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                     Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                     Vangel Bojaxhi, Ben England, Vikentsi Lapa,
                     Alexey Skidanov, Sudhir Kumar.

        Run began: Thu Apr  7 07:38:51 2022

        Include fsync in write timing
        O_DIRECT feature enabled
        Auto Mode
        File size set to 102400 kB
        Record Size 4 kB
        Record Size 16 kB
        Record Size 512 kB
        Record Size 1024 kB
        Record Size 16384 kB
        Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                   
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    38874    89785    99432   102062    39802    91113
          102400      16   144607   275226   291702   288610   102913   238335
          102400     512   938555   990854   992056  1018663   789734  1019625
          102400    1024  1052577  1121665  1084297  1112888   994947  1066922
          102400   16384  1286838  1330574  1323620  1351797  1331433  1227587

[domih]

Western Digital WDS100T1X0E-00AFY0, aka SN850 1TB PCIe Gen 4 x4 is AOK.

See iozone3 and fio testing there: viewtopic.php?p=346818#p346818

I believe you should add the following note:

There is ZERO technical reason to spend $$$ on a PCIe Gen 4 x4 NVMe to use on the M1 which PCIe Gen 3 x2. It would be a waste of money. We just report the compatibility because one of our users tested it. The only valid reason to use such an NVMe would be you're stranded on an island and this is the only available model

You will get the best NVMe results with a PCIe Gen 3 x4 NVMe, refer to viewtopic.php?p=346721#p346721 for details. Thanks to Mr tkaiser.

[domih]

Western Digital model: WDS500G3X0C-00SJG0, aka SN750 500GB PCIe Gen 3 x4 is AOK.

See iozone3 and fio testing there: viewtopic.php?p=346820#p346820

[domih]

Western Digital model: WDBNCE2500PNC, aka WD Blue 250GB SATA III SSD is AOK.

Powered by the M1 SATA III power plug. NO separate power was necessary.

See iozone3 testing there: viewtopic.php?p=346822#p346822

[domih]

Crucial model: CT1000P1SSD8, aka Crucial P1 1TB PCIe Gen 3 x4 is AOK.

See iozone3 and fio testing there: viewtopic.php?p=346826#p346826

[domih]

Crucial model: CT1000MX500SSD1, aka Crucial MX500 1TB SATA III SSD is AOK.

Powered by the M1 SATA III power plug. NO separate power was necessary.

See iozone3 testing there: viewtopic.php?p=346828#p346828

[superpowter77]

Acer AT500M 256GB and 500GB NVMe's are AOK. Those old models were sold by Allnetchina and recommended by Radxa for booting from NVMe on their RockPi4 boards
Shown 256GB model:

Code: Select all

sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly --filename=/dev/nvme0n1 | grep READ
[sudo] password for odroid: 
   READ: bw=1221MiB/s (1281MB/s), 1221MiB/s-1221MiB/s (1281MB/s-1281MB/s), io=11.0GiB (12.9GB), run=10053-10053msec
   

Code: Select all

iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    37107    86580    99801   105849    36012    83188                                                                
          102400      16   137582   264952   305907   298887    97352   264294                                                                
          102400     512   830123   971499  1031603  1029519   760660   903699                                                                
          102400    1024   968377   945940  1096360  1118306   946745  1028639                                                                
          102400   16384   915545   999258  1323994  1357319  1332302   904209                                                                

iozone test complete.

[superpowter77]

Samsung Evo 970 Plus 256GB is AOK.

Code: Select all

sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly --filename=/dev/nvme0n1 | grep READ
   READ: bw=1533MiB/s (1608MB/s), 1533MiB/s-1533MiB/s (1608MB/s-1608MB/s), io=15.2GiB (16.3GB), run=10163-10163msec

Code: Select all

Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    38592    89110   105402   104638    43558    89299                                                                
          102400      16   132413   268497   294148   297841   157821   268510                                                                
          102400     512   904640   926094   986988  1015813   908236  1083450                                                                
          102400    1024  1009044  1021434  1022008  1077248  1011916  1100602                                                                
          102400   16384  1333224  1370033  1341243  1391656  1382672  1382946                                                                

iozone test complete.

[domih]

Samsung model: SSD 970 EVO Plus 1TB is AOK.

See iozone3 and fio testing there: viewtopic.php?p=346997#p346997

[istanbulls]

1. Brand : LEVEN
2. Model : JP600
3. Capacity : 256GB
4. NVMe controller and firmware version : SM2263XT
NAND type flash memory: 3D TLC (Tri-Level Cell)
6. Price and where to buy : 394TL (27$) Amazon.com.tr
5. Additional comments :

Code: Select all

odroid@gnome-desktop:~$ sudo fio --filename=/dev/nvme0n1 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
   READ: bw=1560MiB/s (1636MB/s), 1560MiB/s-1560MiB/s (1636MB/s-1636MB/s), io=15.3GiB (16.4GB), run=10042-10042msec

Code: Select all

odroid@gnome-desktop:/media/odroid/c25e452d-7e76-42f6-9c5f-5a772cfa923d$ sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Fri Apr  8 19:33:52 2022

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    38507    91087   108795   108157    40856    89640                                                                
          102400      16   139110   279204   302773   303820   106233   263009                                                                
          102400     512   910018   924305  1058518  1079553   814520  1049642                                                                
          102400    1024   877688  1063200  1163333  1186255   981285   927905                                                                
          102400   16384  1071329   924293  1382909  1411319  1378466  1102160

Fio tests are always the same, iozone tests are constantly different

[istanbulls]

Code: Select all

odroid@gnome-desktop:~$ sudo fio --filename=/dev/nvme0n1 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
   READ: bw=1579MiB/s (1656MB/s), 1579MiB/s-1579MiB/s (1656MB/s-1656MB/s), io=15.5GiB (16.6GB), run=10041-10041msec
odroid@gnome-desktop:~$ cd  /media/odroid/c25e452d-7e76-42f6-9c5f-5a772cfa923d
bash: cd: /media/odroid/c25e452d-7e76-42f6-9c5f-5a772cfa923d: No such file or directory
odroid@gnome-desktop:~$ cd  /media/odroid/c25e452d-7e76-42f6-9c5f-5a772cfa923d
odroid@gnome-desktop:/media/odroid/c25e452d-7e76-42f6-9c5f-5a772cfa923d$ sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Fri Apr  8 19:46:49 2022

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    41807    85405   105099   109425    42196    90367                                                                
          102400      16   144136   260959   286611   316219   106632   273332                                                                
          102400     512   949246   985932  1061933  1079930   828332  1066445                                                                
          102400    1024   933922  1131893  1165568  1185502   983073  1019892                                                                
          102400   16384  1121536  1022700  1384483  1405848  1375091  1139916                                                                

iozone test complete.

[domih]

.../...
Fio tests are always the same, iozone tests are constantly different

Yes, @tkaiser came up with the explanation for the fio identical results: with good enough PCIe Gen 3 x4 NVMe, the bottleneck of the PCIe Gen 3 x2 available lanes is always reached with fio. I witnessed it my self with 6 different models so far.

See above post: viewtopic.php?p=346721#p346721 which refers to the discussion there viewtopic.php?p=346720#p346720.

For iozone 3, the sources of the variations I can come up with are:
- Any same SSD will not behave exactly the same way for a same operation because internally the writes and rewrites do not happen at the same places.
- Unless you turn off most of the timers, services (others?) the OS may or may not execute some background operations from one test to another (*)

(*) in the other thread, @tkaiser referred me to this interesting doc: https://www.brendangregg.com/blog/2017- ... rages.html.

Hope this helps. If so thank @tkaiser, I'm just an operator in this story, @tkaiser came up with the conclusions

[tkaiser]

Fio tests are always the same, iozone tests are constantly different

@tobetter's command line for both tools differs especially with regard to the data size tested. With fio it's many GB while with iozone it's only 100 MB per block size which will automatically result with many SSDs in fluctuating values since tested data size too small.

Simply increasing the data size (e.g. ten times more: -s 1000M) will result in problems with SSDs that feature some sort of SLC write cache (e.g. Samsung's TurboWrite) which handles only a specific amount of sustained writes at high speeds to slow down often significantly once the write cache size has exceeded. The iozone command line testing through 5 different block sizes in 3 different modes results in 1.5G written when testing with 100M (-s 100M)

Most probably you'll get more consistent iozone results with your Silicon Motion SSD (brand / 'model name' irrelevant, all SM2263XT SSDs are without DRAM cache and should perform identical when limited to a Gen3 x2 connection) by executing this (only testing with 16M block size but 1G data size):

Code: Select all

sudo iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1 -i 2

[istanbulls]

For a very cheap SSD, I think the results are better than I thought.
I was ready for worse performance.

Code: Select all

odroid@gnome-desktop:~$ sudo iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Sat Apr  9 01:25:34 2022

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 1024000 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
         1024000   16384  1007513  1002277  1398281  1386437  1374284   996885                                                                

iozone test complete.

Code: Select all

odroid@gnome-desktop:~$ sudo iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Sat Apr  9 01:24:22 2022

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 1024000 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
         1024000   16384  1051232  1043481  1385510  1387092  1363435  1034329

[tkaiser]

For a very cheap SSD, I think the results are better than I thought.

Yeah, but the collected results are more or less 'numbers without meaning' anyway. Which use case is represented by sequential transfers (MB/s)? What's the practical difference between 1000 MB/s and 1350 MB/s other than 'almost none'?

Usually random I/O and there the IOPS are more important if OS or databases reside on the SSD. And here the various tested SSDs should differ a lot based on benchmark results known. But they don't:

Code: Select all

                                                              random    random
SSD               reclen    write  rewrite    read    reread    read     write
Phison E12 128GB       4    36288    85802    97865    97764    34297    85084
Samsung EVO 970+ 256GB 4    38592    89110   105402   104638    43558    89299
Samsung EVO 970+ 500GB 4    40085    97122   108213   108358    52545    96300
Samsung 970 PRO 1TB    4    38447    84289   105673   106033    39541    81234
Kingston SNVS500G      4    38874    89785    99432   102062    39802    91113
WD SN750 500GB         4    38412   100701    91714    89517    38313    98472
WD SN850 1TB           4    41002   103532    93196    96135    56928   100934
Crucial P1 1TB         4    39476    93073   105124   106824    40309    98040
Acer AT500M 256GB      4    37107    86580    99801   105849    36012    83188
SM2263XT (JP600) 256GB 4    41807    85405   105099   109425    42196    90367

iozone lists all numbers as KB/s so one would need to translate that to 4K IOPS for the two last columns but why? Since the results seem to be all flawed: the 'write' column is always way lower than the 'rewrite' column which strongly indicates 'benchmarking gone wrong'. And this (gone wrong) is the normal. Benchmarking is part of my day job and usually I have to throw results away +95% of times since 'something went wrong'.

Which isn't that surprising since fire-and-forget passive benchmarking usually doesn't work [1]. The iozone command line proposed by tobetter originates from SD card testing where it was confirmed to work (SD cards also being flash storage but way slower interface and really primitive controllers).

When I started 6 years ago over at Armbian with testing for SD cards showing high random I/O performance I used exactly that iozone3 call... for the simple reason another guy called Jeff Geerling already used the very same command line in his prior SD card testing so I decided to go with same tool and only slightly different parameters to get something comparable which worked well with SD cards and even the amazingly fast eMMC Hardkernel used in the meantime. But with modern NVMe SSDs the same methodology seems to fail.

Though doing these kitchen-sink benchmarks is not a real mistake since by carefully looking at the numbers other problems could be spotted (like e.g. PCIe Link training ending up establishing only an x1 link or Gen2 or even Gen1 speeds which could happen for example with dusty M.2 contacts)

[1] https://www.brendangregg.com/activebenchmarking.html –- see there the nice bonnie++ example

[domih]

.../...
Which isn't that surprising since fire-and-forget passive benchmarking usually doesn't work
.../...

To which one must add that you are talking about the rarefied stratospheric world of people/experts doing benchmarking using a scientific methodology.

For the populace, benchmarking means "...let's read press articles that are going to reinforce my good feeling of having bought the best of the best brand...and mine is better than yours...and I'm so smart, I bought the right stuff..." The main stream computer press use that for attracting eyeballs to their sites coz' the more Ad impressions the better. As a result you end up with endless comment threads which make you despair about humanity Finally, benchmarking with an agenda is practiced by most major brands. IMHO INTEL is the best (and worst) at that game.

Back to the humble activity of testing the M1: eliminating all the variables, side effects and knowing what is actually measured is an expert job. Add the common pitfalls of doing basic statistics.

I (should say we) enjoy reading your analysis. Thanks!

[tkaiser]

To which one must add that you are talking about the rarefied stratospheric world of people/experts doing benchmarking using a scientific methodology.

It was more about making the point 'Attention: that command doesn't work (any more). We need a new methodology'. I could imagine a few fio calls being able to test properly for both random I/O and sequential transfer speeds resulting in the community spending time on testing providing more meaningful numbers.

With this sort of storage testing it's the same as with CPU performance testing. Many years ago using 'sysbench cpu' was perfectly fine to get a rough CPU performance assessment for any specific SBC even if it just calculated prime numbers and this inside CPU caches (RAM bandwidth/latency doesn't matter at all unlike most real-world tasks).

And then ODROID-C2 arrived and changed everything since we all had to learn that 'sysbench cpu' is also just another compiler benchmark and when building it for ARMv8 it will use the divide instruction not available with older ARM architectures which makes this benchmark score ~15 times better compared to running a version built for ARMv7 or earlier on the same hardware. As such 'sysbench cpu' had to be buried and I guess now it's time for this iozone call...

[superpowter77]

WD SN550 NVMe 500GB is AOK.

Code: Select all

sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly --filename=/dev/nvme0n1 | grep READ
[sudo] password for odroid: 
   READ: bw=1562MiB/s (1638MB/s), 1562MiB/s-1562MiB/s (1638MB/s-1638MB/s), io=15.4GiB (16.5GB), run=10084-10084msec

Code: Select all

Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    37175    89203    87527    91357    36839    80316                                                                
          102400      16   136650   268217   277776   267635   133343   272452                                                                
          102400     512   833977  1000605   926094   948701   842750  1055440                                                                
          102400    1024   878473  1095035   928300   955758   945878  1124052                                                                
          102400   16384  1256439  1306541  1326460  1347811  1336106  1331816                                                                

iozone test complete.

[superpowter77]

Samsung 980 PRO 256GB is AOK
(This NVMe m.2 drive is PCIe4.0)

Code: Select all

sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly --filename=/dev/nvme0n1 | grep READ
[sudo] password for odroid: 
   READ: bw=1570MiB/s (1646MB/s), 1570MiB/s-1570MiB/s (1646MB/s-1646MB/s), io=15.5GiB (16.6GB), run=10082-10082msec

Code: Select all

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    35457    78068    92327    94254    53384    78517                                                                
          102400      16   120549   239495   280720   288222   150973   254836                                                                
          102400     512   775028   939993   869131   937264   886587   989907                                                                
          102400    1024   842916  1074322   943248  1045910  1016659  1081755                                                                
          102400   16384  1264035  1353732  1274708  1374706  1363157  1329240                                                                

iozone test complete.

[tobetter]

Got a new NVMe that is not incompatible without changing the kernel parameter pci=nomsi, reported by @crashoverride. I will test it and sort out if this NVMe also can be supported in the same OS when I have a time slot.

IMG_20220413_1345487.jpg (74.16 KiB) Viewed 20021 times

[hominoid]

1. Brand : Mushkin
2. Model : Tempest MKNSSDTS256GB-D8
3. Capacity : 256GB
4. NVMe controller and firmware version : InnoGrit IG5216
5. Additional comments : PCIe Gen3 x4 NVMe 1.4
6. Price and where to buy : $35 USD @ Newegg.com (free shipping)

Code: Select all

sudo fio --filename=/dev/nvme0n1 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
   READ: bw=1528MiB/s (1602MB/s), 1528MiB/s-1528MiB/s (1602MB/s-1602MB/s), io=14.0GiB (16.1GB), run=10022-10022msec

Code: Select all

sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Sat Apr 16 20:51:22 2022

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    35781   100819    38632    38538    48884    86912                                                                
          102400      16   142499   198689   107765   153246   107600   194268                                                                
          102400     512  1067364  1151182   820336   894613   914979  1091903                                                                
          102400    1024  1198935  1213528   989095  1080430  1092278  1152009                                                                
          102400   16384  1370147  1297606  1267427  1411503  1415462  1217945                                                                

iozone test complete.

[hominoid]

1. Brand : Samsung
2. Model : 970 EVO PLUS MZ-V7S500B/AM
3. Capacity : 500GB
4. NVMe controller and firmware version : Samsung Phoenix
5. Additional comments : PCIe Gen3 x4 NVMe 1.3, Cache Samsung 512MB Low Power DDR4 SDRAM
6. Price and where to buy : $62 @ Newegg.com (free shipping)

Code: Select all

sudo fio --filename=/dev/nvme0n1 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
   READ: bw=1571MiB/s (1648MB/s), 1571MiB/s-1571MiB/s (1648MB/s-1648MB/s), io=15.4GiB (16.5GB), run=10040-10040msec

Code: Select all

sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Sat Apr 16 21:15:17 2022

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    41791    93346    97195    99128    43692    87740                                                                
          102400      16   142246   281527   294350   296553   180215   279487                                                                
          102400     512   989658  1049834   962604   987844   923238  1043927                                                                
          102400    1024  1086852  1129719  1044288  1071163  1025240  1131267                                                                
          102400   16384  1378562  1397239  1347199  1399644  1397116  1389456                                                                

iozone test complete.

[multicast]

1. Brand : ADATA
2. Model : ADATA XPG SX8200 Pro 1TB 3D NAND NVMe Gen3x4
3. Capacity : 1TB
4. NVMe controller and firmware version :
5. Additional comments :
6. Price and where to buy : $88 @ amazon

Code: Select all

root@gnome-desktop:~# fio --filename=/dev/nvme0n1 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
   READ: bw=1553MiB/s (1628MB/s), 1553MiB/s-1553MiB/s (1628MB/s-1628MB/s), io=15.2GiB (16.4GB), run=10043-10043msec

root@gnome-desktop:/mnt/nvme# iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Iozone: Performance Test of File I/O
                Version $Revision: 3.489 $
                Compiled for 64 bit mode.
                Build: linux 

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                     Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                     Vangel Bojaxhi, Ben England, Vikentsi Lapa,
                     Alexey Skidanov, Sudhir Kumar.

        Run began: Tue Apr 19 14:58:19 2022

        Include fsync in write timing
        O_DIRECT feature enabled
        Auto Mode
        File size set to 102400 kB
        Record Size 4 kB
        Record Size 16 kB
        Record Size 512 kB
        Record Size 1024 kB
        Record Size 16384 kB
        Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    37128    86270    99017   105135    42701    84368                                                                
          102400      16   113853   281548   289946   290371   139486   265674                                                                
          102400     512  1021568  1083988  1025230  1011778   899198  1020733                                                                
          102400    1024  1156420  1218873  1153594  1156016  1062714  1217553                                                                
          102400   16384  1419673  1338744  1364136  1374958  1363969  1428794                                                                

iozone test complete.

[lxxl]

1. Brand : Samsung
2. Model : MZVLB1T0HBLR-00000 PCIe® Gen3 x4 PM981a
3. Capacity : 1TB
4. NVMe controller and firmware version :
5. Additional comments :

Code: Select all

fio --filename=/dev/nvme0n1p2 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
   READ: bw=1566MiB/s (1642MB/s), 1566MiB/s-1566MiB/s (1642MB/s-1642MB/s), io=15.4GiB (16.5GB), run=10041-10041msec

Code: Select all

iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Iozone: Performance Test of File I/O
                Version $Revision: 3.489 $
                Compiled for 64 bit mode.
                Build: linux

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                     Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                     Vangel Bojaxhi, Ben England, Vikentsi Lapa,
                     Alexey Skidanov, Sudhir Kumar.

        Run began: Mon Apr 25 17:42:51 2022

        Include fsync in write timing
        O_DIRECT feature enabled
        Auto Mode
        File size set to 102400 kB
        Record Size 4 kB
        Record Size 16 kB
        Record Size 512 kB
        Record Size 1024 kB
        Record Size 16384 kB
        Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    31336    87418    96895    80358    49335    73971
          102400      16   145517   269858   283416   285686   173267   267330
          102400     512   966758  1045803   961718   997313   924804  1043641
          102400    1024  1108393  1170071  1089629  1125014  1071845  1158370
          102400   16384  1347331  1364823  1334057  1389967  1384116  1358787

iozone test complete.

[lxxl]

In additional here is SSD Sata speed with is also look goood ! full vendor spec speed.
1. Brand : Samsung
2. Model : MZ7KH480 - SM883
3. Capacity : 400 GB
4. NVMe controller and firmware version :
5. Additional comments :

Code: Select all

fio --filename=/dev/sda --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
   READ: bw=540MiB/s (567MB/s), 540MiB/s-540MiB/s (567MB/s-567MB/s), io=5472MiB (5738MB), run=10124-10124msec

[glenno]

1. Brand : Kingston
2. Model : A2000 250GB (SA2000M8/250G) M.2(2280) PCIe NVMe SSD Solid State Drive
3. Capacity : 250GB
4. NVMe controller and firmware version :
5. Additional comments :
6. Price and where to buy : $49 AUD @ msy.com.au

lspci -vv

Code: Select all

0002:21:00.0 Non-Volatile memory controller: Kingston Technologies Device 2263 (rev 03) (prog-if 02 [NVM Express])
	Subsystem: Kingston Technologies Device 2263
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 99
	Region 0: Memory at 380900000 (64-bit, non-prefetchable) [size=16K]
	Capabilities: [40] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [50] MSI: Enable- Count=1/8 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [70] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
			ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
		DevCtl:	Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+
			RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-
			MaxPayload 128 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <1us, L1 <8us
			ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 8GT/s, Width x2, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Not Supported
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+, OBFF Disabled
		LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+, EqualizationPhase1+
			 EqualizationPhase2+, EqualizationPhase3+, LinkEqualizationRequest-
	Capabilities: [b0] MSI-X: Enable- Count=16 Masked-
		Vector table: BAR=0 offset=00002000
		PBA: BAR=0 offset=00002100
	Capabilities: [100 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+
		AERCap:	First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn-
	Capabilities: [158 v1] #19
	Capabilities: [178 v1] Latency Tolerance Reporting
		Max snoop latency: 0ns
		Max no snoop latency: 0ns
	Capabilities: [180 v1] L1 PM Substates
		L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
			  PortCommonModeRestoreTime=10us PortTPowerOnTime=10us
		L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
			   T_CommonMode=0us LTR1.2_Threshold=0ns
		L1SubCtl2: T_PwrOn=10us
	Kernel driver in use: nvme

[tenkawa]

1. Brand : Western Digital
2. Model : 500GB Blue SN550 NVMe M.2 Internal SSD PCIe 3.0 x4
3. Capacity : 500GB
4. NVMe controller and firmware version :
5. Additional comments :
6. Price and where to buy : $50 USD @ https://www.bhphotovideo.com/c/product/ ... _nvme.html

Code: Select all

sudo fio --filename=/dev/nvme0n1 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
READ: bw=1563MiB/s (1639MB/s), 1563MiB/s-1563MiB/s (1639MB/s-1639MB/s), io=15.4GiB (16.5GB), run=10078-10078msec
sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Tue May  3 10:27:23 2022

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 102400 kB
	Record Size 4 kB
	Record Size 16 kB
	Record Size 512 kB
	Record Size 1024 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    48439   120915   110510    96271    40765   110139                                                                
          102400      16   149625   267223   309276   327407   132853   349761                                                                
          102400     512   907639  1035798   977006   994445   896595   764738                                                                
          102400    1024  1087843  1128586   953541   966384   958379   882050                                                                
          102400   16384  1351128  1363892  1389260  1390675  1392445  1354835

[oliverlj]

1. Brand : Samsung
2. Model : SSD 980 1TB
3. Capacity : 1TB
4. NVMe controller and firmware version : 2B4QFXO7
5. Additional comments :

Code: Select all

odroid@server:~$ sudo smartctl -a /dev/nvme0n1
smartctl 7.1 2019-12-30 r5022 [aarch64-linux-4.19.219-odroid-arm64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       Samsung SSD 980 1TB
Serial Number:                      000000000000
Firmware Version:                   2B4QFXO7
PCI Vendor/Subsystem ID:            0x144d
IEEE OUI Identifier:                0x002538
Total NVM Capacity:                 1,000,204,886,016 [1.00 TB]
Unallocated NVM Capacity:           0
Controller ID:                      5
Number of Namespaces:               1
Namespace 1 Size/Capacity:          1,000,204,886,016 [1.00 TB]
Namespace 1 Utilization:            15,112,359,936 [15.1 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            002538 d321c082e0
Local Time is:                      Wed May  4 18:04:21 2022 UTC
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x0055):     Comp DS_Mngmt Sav/Sel_Feat Timestmp
Maximum Data Transfer Size:         512 Pages
Warning  Comp. Temp. Threshold:     82 Celsius
Critical Comp. Temp. Threshold:     85 Celsius
Namespace 1 Features (0x10):        *Other*

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +     5.24W       -        -    0  0  0  0        0       0
 1 +     4.49W       -        -    1  1  1  1        0       0
 2 +     2.19W       -        -    2  2  2  2        0     500
 3 -   0.0500W       -        -    3  3  3  3      210    1200
 4 -   0.0050W       -        -    4  4  4  4     1000    9000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        33 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    0%
Data Units Read:                    17,580 [9.00 GB]
Data Units Written:                 30,118 [15.4 GB]
Host Read Commands:                 32,746
Host Write Commands:                47,739
Controller Busy Time:               1
Power Cycles:                       6
Power On Hours:                     0
Unsafe Shutdowns:                   0
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               33 Celsius
Temperature Sensor 2:               37 Celsius

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

Code: Select all

odroid@server:~$ sudo fio --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly --filename=/dev/nvme0n1 | grep READ
   READ: bw=785MiB/s (823MB/s), 785MiB/s-785MiB/s (823MB/s-823MB/s), io=8102MiB (8496MB), run=10323-10323msec

Code: Select all

odroid@server:~$ sudo iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1 -i 2
	Iozone: Performance Test of File I/O
	        Version $Revision: 3.489 $
		Compiled for 64 bit mode.
		Build: linux 

	Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
	             Al Slater, Scott Rhine, Mike Wisner, Ken Goss
	             Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
	             Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
	             Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
	             Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
	             Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
	             Vangel Bojaxhi, Ben England, Vikentsi Lapa,
	             Alexey Skidanov, Sudhir Kumar.

	Run began: Wed May  4 18:13:46 2022

	Include fsync in write timing
	O_DIRECT feature enabled
	Auto Mode
	File size set to 1024000 kB
	Record Size 16384 kB
	Command line used: iozone -e -I -a -s 1000M -r 16384k -i 0 -i 1 -i 2
	Output is in kBytes/sec
	Time Resolution = 0.000001 seconds.
	Processor cache size set to 1024 kBytes.
	Processor cache line size set to 32 bytes.
	File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride                                    
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
         1024000   16384   754401   759190   734630   735244   735121   757224                                                                

iozone test complete.

Strange is below 1024mib/sec ?

[tkaiser]

Strange is below 1024mib/sec ?

Well, this whole benchmarking party is an excercise in ignorance... see lspci and dmesg being mentioned above: viewtopic.php?p=346446#p346446

M1 is Gen3 x2, almost all those 'benchmarks' above show similar sequential scores as a result of the SoC's PCIe implementation bottlenecking the SSDs. Your test with halved scores hints at either Gen3 x1 or Gen2 x2 and lspci and/or dmesg will tell the whole story (whether only a single lane is connected or PCIe link training ended up with Gen2 speeds).

[oliverlj]

just retry and two lanes are activated

[aodv]

1. Brand : SK Hynix
2. Model : HFM256GDHTNI-87A0B
3. Capacity : 256GB (238.5GB formatted)
4. NVMe controller and firmware version : 11010C00
5. Additional comments : PCIe Gen3 x4 NVMe

Code: Select all

sudo fio --filename=/dev/nvme0n1 --direct=1 --rw=read --bs=1024k --ioengine=libaio --iodepth=64 --size=4G --runtime=10 --numjobs=4 --group_reporting --name=fio-rand-read-sequential --eta-newline=1 --readonly | grep READ
   READ: bw=1543MiB/s (1617MB/s), 1543MiB/s-1543MiB/s (1617MB/s-1617MB/s), io=15.1GiB (16.3GB), run=10056-10056msec

Code: Select all

sudo iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Iozone: Performance Test of File I/O
                Version $Revision: 3.489 $
                Compiled for 64 bit mode.
                Build: linux

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                     Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                     Vangel Bojaxhi, Ben England, Vikentsi Lapa,
                     Alexey Skidanov, Sudhir Kumar.

        Run began: Thu May  5 15:24:41 2022

        Include fsync in write timing
        O_DIRECT feature enabled
        Auto Mode
        File size set to 102400 kB
        Record Size 4 kB
        Record Size 16 kB
        Record Size 512 kB
        Record Size 1024 kB
        Record Size 16384 kB
        Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
          102400       4    36984    66171    64097    58857    26034    63358
          102400      16   126834   152692   174738   148908    74815   126439
          102400     512   789835   810094   746899   764520   646853   802345
          102400    1024   850744   876382   805734   813854   854030   873392
          102400   16384   951560   979934  1319321  1330647  1339400   977177

iozone test complete.

Code: Select all

smartctl 7.1 2019-12-30 r5022 [aarch64-linux-4.19.219-odroid-arm64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:                       SKHynix_HFM256GDHTNI-87A0B
Serial Number:                      CD07N88481120815V
Firmware Version:                   11010C00
PCI Vendor/Subsystem ID:            0x1c5c
IEEE OUI Identifier:                0xace42e
Controller ID:                      1
Number of Namespaces:               1
Namespace 1 Size/Capacity:          256,060,514,304 [256 GB]
Namespace 1 Formatted LBA Size:     512
Namespace 1 IEEE EUI-64:            ace42e 0005c56a60
Local Time is:                      Thu May  5 15:52:01 2022 CDT
Firmware Updates (0x16):            3 Slots, no Reset required
Optional Admin Commands (0x0017):   Security Format Frmw_DL Self_Test
Optional NVM Commands (0x0056):     Wr_Unc DS_Mngmt Sav/Sel_Feat Timestmp
Maximum Data Transfer Size:         64 Pages
Warning  Comp. Temp. Threshold:     88 Celsius
Critical Comp. Temp. Threshold:     89 Celsius
Namespace 1 Features (0x02):        NA_Fields

Supported Power States
St Op     Max   Active     Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +   3.5000W       -        -    0  0  0  0        5       5
 1 +   2.4000W       -        -    1  1  1  1       30      30
 2 +   1.9000W       -        -    2  2  2  2      100     100
 3 -   0.0350W       -        -    3  3  3  3     1000    1000
 4 -   0.0050W       -        -    3  3  3  3     5000    5000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +     512       0         0
 1 -    4096       0         0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02)
Critical Warning:                   0x00
Temperature:                        45 Celsius
Available Spare:                    100%
Available Spare Threshold:          10%
Percentage Used:                    1%
Data Units Read:                    17,943,803 [9.18 TB]
Data Units Written:                 16,875,403 [8.64 TB]
Host Read Commands:                 327,608,310
Host Write Commands:                328,735,229
Controller Busy Time:               1,667
Power Cycles:                       1,021
Power On Hours:                     782
Unsafe Shutdowns:                   52
Media and Data Integrity Errors:    0
Error Information Log Entries:      0
Warning  Comp. Temperature Time:    0
Critical Comp. Temperature Time:    0
Temperature Sensor 1:               45 Celsius
Temperature Sensor 2:               54 Celsius

Error Information (NVMe Log 0x01, max 256 entries)
No Errors Logged