[Solved] Low Samba performance?

Post Reply
alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

[Solved] Low Samba performance?

Post by alpe »

Server: Ubuntu minimal.

I can get about 30MB/s to my usb 2.0 attached hdd, which is expected.
I already ordered an USB 3.0 adapter.
In the meantime I ran the Helios LanTest to a tmpfs folder.

I got 48MB/s write and 38MB/s read.
Using the Performance governor: 64MB/s write and 49MB/s read.
Shouldn't this be at least 80MB/s?
This is what I can expect to get when using USB 3.0 then?

I'm using samba 4.12.3 now. But it was the same when I used the one in the repository (4.7.6).

Thanks.

PS: Windows 10 client.

dd to tmpfs:

Code: Select all

$ dd if=/dev/zero of=/media/tmpfs/teste bs=280M count=1 oflag=dsync
1+0 records in
1+0 records out
293601280 bytes (294 MB, 280 MiB) copied, 0.414932 s, 708 MB/s
smb.conf:

Code: Select all

[global]
        deadtime = 15
        dns proxy = No
        domain master = No
        load printers = No
        log file = /var/log/samba/samba.log
        map to guest = Bad User
        min receivefile size = 16384
        name resolve order = bcast
        ntlm auth = ntlmv1-permitted
        preferred master = No
        printcap name = cups
        remote announce = 192.168.1.255
        remote browse sync = 192.168.1.255
        security = USER
        server min protocol = NT1
        server signing = if_required
        server string = Samba file server
        template shell = /dev/null
        unix extensions = No
        username level = 6
        username map = /etc/samba/smbusers
        username map cache time = 90
        winbind nested groups = No
        winbind nss info = no
        winbind use default domain = Yes
        idmap config * : backend = tdb
        access based share enum = Yes
        acl allow execute always = Yes
        cups options = raw
        hosts allow = 127. 192.168.1.
        strict locking = No
        use sendfile = Yes

[media]
        create mask = 0664
        directory mask = 02775
        path = /media/alpe
        valid users = alpe
        write list = alpe
Last edited by alpe on Sat Aug 08, 2020 9:47 pm, edited 3 times in total.

crashoverride
Posts: 5521
Joined: Tue Dec 30, 2014 8:42 pm
languages_spoken: english
ODROIDs: C1
Has thanked: 0
Been thanked: 490 times
Contact:

Re: Low Samba performance?

Post by crashoverride »

Make sure you are mounting the SMB share using the command line 'mount' command and not through a GUI. There is a long standing bug (a decade or more?) with GVFS that limits SMB transfer rate. An example of one the many ignored bug reports is here:
https://bugzilla.redhat.com/show_bug.cgi?id=985823

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: Low Samba performance?

Post by alpe »

crashoverride wrote:
Tue Jun 09, 2020 2:53 am
Make sure you are mounting the SMB share using the command line 'mount' command and not through a GUI. There is a long standing bug (a decade or more?) with GVFS that limits SMB transfer rate. An example of one the many ignored bug reports is here:
https://bugzilla.redhat.com/show_bug.cgi?id=985823
Thanks, but it's a Windows 10 client.

User avatar
mad_ady
Posts: 10155
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 632 times
Been thanked: 817 times
Contact:

Re: Low Samba performance?

Post by mad_ady »

Try these global options:

Code: Select all

socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536
    write cache size = 524288
    getwd cache = yes
    use sendfile = yes
    min receivefile size = 16384

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: Low Samba performance?

Post by alpe »

mad_ady wrote:
Tue Jun 09, 2020 3:15 am
Try these global options:

Code: Select all

socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536
    write cache size = 524288
    getwd cache = yes
    use sendfile = yes
    min receivefile size = 16384
I disabled 'Interrupt Moderation' on my PC NIC and got better results with my settings(but still not what I believe the N2 can do):
Read: 56MB/s
Write: 75MB/s

Using only those settings that you send on [global]:
Read: 55MB/s
Write: 57MB/s

Using these settings together with mine:
Read: 56MB/s
Write: 59MB/s

:cry:

PS: Performance governor.

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: Low Samba performance?

Post by alpe »

Another speed up by locking SMBD to CPU2 and CPU3. But still has a lot of room for improvement.
Read: 65MB/s
Write: 75MB/s

It's weird that changing this improved the speed. On CPU0 and CPU1 it was far from 100% cpu usage. Possibly more L2 cache on the BIG cores helped.

User avatar
odroid
Site Admin
Posts: 38556
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English, Korean
ODROIDs: ODROID
Has thanked: 2219 times
Been thanked: 1298 times
Contact:

Re: Low Samba performance?

Post by odroid »

What is the current kernel version?

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: Low Samba performance?

Post by alpe »

odroid wrote:
Tue Jun 09, 2020 10:07 am
What is the current kernel version?

Code: Select all

$ uname -a
Linux odroid 4.9.219-72 #1 SMP PREEMPT Mon Apr 13 13:55:42 -03 2020 aarch64 aarch64 aarch64 GNU/Linux

User avatar
odroid
Site Admin
Posts: 38556
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English, Korean
ODROIDs: ODROID
Has thanked: 2219 times
Been thanked: 1298 times
Contact:

Re: Low Samba performance?

Post by odroid »

It looks fine.
As far as I remember, we could get over 80MB/s of SMB transfer speed with performance governor easily.

Can you try iperf test between your Windows PC and N2 to narrow down root causes?

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: Low Samba performance?

Post by alpe »

odroid wrote:
Tue Jun 09, 2020 10:41 am
It looks fine.
As far as I remember, we could get over 80MB/s of SMB transfer speed with performance governor easily.

Can you try iperf test between your Windows PC and N2 to narrow down root causes?
N2 -> PC

Code: Select all

[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   103 MBytes   866 Mbits/sec
[  5]   1.00-2.00   sec   111 MBytes   935 Mbits/sec
[  5]   2.00-3.00   sec   110 MBytes   923 Mbits/sec
[  5]   3.00-4.00   sec   111 MBytes   927 Mbits/sec
[  5]   4.00-5.00   sec   110 MBytes   924 Mbits/sec
[  5]   5.00-6.00   sec   111 MBytes   933 Mbits/sec
[  5]   6.00-7.00   sec   112 MBytes   938 Mbits/sec
[  5]   7.00-8.00   sec   110 MBytes   926 Mbits/sec
[  5]   8.00-9.00   sec   110 MBytes   921 Mbits/sec
[  5]   9.00-10.00  sec   111 MBytes   934 Mbits/sec
[  5]  10.00-10.07  sec  7.56 MBytes   923 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.07  sec  0.00 Bytes  0.00 bits/sec                  sender
[  5]   0.00-10.07  sec  1.08 GBytes   923 Mbits/sec                  receiver
PC -> N2

Code: Select all

[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  5]   0.00-1.00   sec   111 MBytes   926 Mbits/sec    0    432 KBytes
[  5]   1.00-2.00   sec   111 MBytes   938 Mbits/sec    0    432 KBytes
[  5]   2.00-3.00   sec   113 MBytes   947 Mbits/sec    0    432 KBytes
[  5]   3.00-4.00   sec   113 MBytes   946 Mbits/sec    0    432 KBytes
[  5]   4.00-5.00   sec   113 MBytes   949 Mbits/sec    0    432 KBytes
[  5]   5.00-6.00   sec   112 MBytes   940 Mbits/sec    0    432 KBytes
[  5]   6.00-7.00   sec   114 MBytes   952 Mbits/sec    0    432 KBytes
[  5]   7.00-8.00   sec   113 MBytes   948 Mbits/sec    0    432 KBytes
[  5]   8.00-9.00   sec   113 MBytes   944 Mbits/sec    0    432 KBytes
[  5]   9.00-10.00  sec   112 MBytes   942 Mbits/sec    0    432 KBytes
[  5]  10.00-10.03  sec  3.22 MBytes   867 Mbits/sec    0    432 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  5]   0.00-10.03  sec  1.10 GBytes   943 Mbits/sec    0             sender
[  5]   0.00-10.03  sec  0.00 Bytes  0.00 bits/sec                  receiver

User avatar
odroid
Site Admin
Posts: 38556
Joined: Fri Feb 22, 2013 11:14 pm
languages_spoken: English, Korean
ODROIDs: ODROID
Has thanked: 2219 times
Been thanked: 1298 times
Contact:

Re: Low Samba performance?

Post by odroid »

The network connection is not a root cause at least.

User avatar
mad_ady
Posts: 10155
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 632 times
Been thanked: 817 times
Contact:

Re: Low Samba performance?

Post by mad_ady »

What if you try locally - mount your samba share locally and transfer something? It should bypass any windows/network bottleneck and just test samba + storage.
Regarding storage - are you still testing with a ramdisk? If not what filesystem is exposed?

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: Low Samba performance?

Post by alpe »

mad_ady wrote:
Tue Jun 09, 2020 3:47 pm
What if you try locally - mount your samba share locally and transfer something? It should bypass any windows/network bottleneck and just test samba + storage.
That's interesting:

Code: Select all

304087040 bytes (304 MB, 290 MiB) copied, 1.78982 s, 170 MB/s
I'm going to search for some windows 10 tweaks then. Any tips are welcome.
mad_ady wrote:
Tue Jun 09, 2020 3:47 pm
Regarding storage - are you still testing with a ramdisk? If not what filesystem is exposed?
Yes, still ramdisk. I bough the USB3 adapters from overseas, so it's going to arrive in about 2-3 months. :/

User avatar
mad_ady
Posts: 10155
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 632 times
Been thanked: 817 times
Contact:

Re: [windows] Low Samba performance?

Post by mad_ady »

It could be tcp window size. If you do a packet capture with wireshark check available window size. If it reaches zero it will stall the transfer.

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: [windows] Low Samba performance?

Post by alpe »

The USB3 adapter arrived.
And I reached the same speed that when using a ramdisk.
Write: 60MB/s
Read: 65MB/s
:(

User avatar
mad_ady
Posts: 10155
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 632 times
Been thanked: 817 times
Contact:

Re: [windows] Low Samba performance?

Post by mad_ady »

Are you transferring a big file, or lots of small files?

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: [windows] Low Samba performance?

Post by alpe »

mad_ady wrote:
Fri Aug 07, 2020 3:25 am
Are you transferring a big file, or lots of small files?
Using Helios LanTest.
Image

Just tested transfering a big file from Odroid to my PC. And it was even worse than Helios.
Image

From my PC to odroid:
Image

User avatar
mad_ady
Posts: 10155
Joined: Wed Jul 15, 2015 5:00 pm
languages_spoken: english
ODROIDs: XU4, C1+, C2, C4, N1, N2, H2, Go, Go Advance
Location: Bucharest, Romania
Has thanked: 632 times
Been thanked: 817 times
Contact:

Re: [windows] Low Samba performance?

Post by mad_ady »

I'd recommend doing a packet capture and looking for retransmits and tcp window 0 messages.

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: [windows] Low Samba performance?

Post by alpe »

mad_ady wrote:
Fri Aug 07, 2020 5:21 am
I'd recommend doing a packet capture and looking for retransmits and tcp window 0 messages.
There are a few retransmits at the start of transfer. No window 0.
Last edited by alpe on Fri Aug 07, 2020 6:48 am, edited 1 time in total.

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: [windows] Low Samba performance?

Post by alpe »

Some interesting results:

Reading the same mp4 file:
With dd (on odroid): 158MB/s
With gnome on a linux pc: 41MB/s
Windows: 100+MB/s ???

I confirmed using the Task Manager. It was transfering at about 900mbps. Maybe because the previous tests cached it on the odroid ram? At least this proves that SAMBA can push this speed to my windows pc.

Edit: Looking at ifconfig on odroid cached files transfer at almos 110MB/s.
Any idea what the problem could be?

alpe
Posts: 142
Joined: Sat Apr 12, 2014 1:21 pm
languages_spoken: portuguese, english
ODROIDs: C1, N2 4GB
Location: Rio de Janeiro, Brazil
Has thanked: 48 times
Been thanked: 9 times
Contact:

Re: [windows] Low Samba performance?

Post by alpe »

aio read size = 0 apeears to have solved the read speed.
I tested (and still testing) a lot of files. Most reads are at 95-105MB/s. Some are 80-95MB/s.

With aio write size = 0 the write speed is anywere between 25MB/s and 90MB/s. Most often in the lower end. But that's enough for write speed.

I will keep testing to make sure this persist. If so, I'll consider this issue as fixed.

Edit: Read is indeed fixed. Write fluctuates a lot:
Image
These users thanked the author alpe for the post:
odroid (Fri Aug 07, 2020 1:21 pm)

rockla
Posts: 313
Joined: Sat Dec 27, 2014 6:50 pm
languages_spoken: English
ODROIDs: C1,C2,N2+
Has thanked: 2 times
Been thanked: 1 time
Contact:

Re: [Solved] Low Samba performance?

Post by rockla »

Just commenting as aio read size = 0 helped resolved and maximizing full 1 Gbps now. My smb.conf below

Code: Select all

[global]
  server string = N2
  workgroup = N2
  netbios name = N22
  interfaces = 192.168.1.1/255.255.255.0
  encrypt passwords = true
  socket options = IPTOS_LOWDELAY TCP_NODELAY SO_KEEPALIVE
  log level = 0
  max log size = 100
  load printers = No
  dns proxy = No
  remote announce = 192.168.1.255
  min receivefile size = 16384
  use sendfile = yes
  aio read size = 0
  security = user
  map to guest = Bad User
  public = yes
[N2]
  path = /media/usbhd
  guest ok = yes
  writeable = yes
  force user = root
  public = yes


Post Reply

Return to “General Topics”

Who is online

Users browsing this forum: No registered users and 4 guests