[Samba] Poor performance on open/copy/close/rename file operations via remote/VPN connection

[gianfranco pra floriani]

Hello,
I have Samba version 3.0.24 running on a 2.6.14-gentoo-r5 x86 kernel
(xeon 3ghz, 1gb ram raid 5).
All clients accessing samba shares via LAN have no problems. Samba
server works perfectly and fast.
We are instead experiencing serious performance issues when accessing
samba shares from remote clients (WAN), via VPN.
Simple operations like "open a file", "copy & paste a file", "save a
file" from Windows XP SP2 clients are incredibly slow. It may take 10
seconds to open a "save as" dialog box, and maybe 15 more seconds to
save a "hello world" txt file from Notepad.
Other services using the VPN such as SCP, SSH, HTTP, FTP work very good
on the same connection, with no slow issues at all. I tried 2 kinds of
VPN connections (OpenVPN and a router-proprietary VPN
gateway-to-client), and both have the same issue, both only with Samba.
I wonder if there is something I'm missing in client or server
configuration that makes Samba talking very slow when connections are
not coming from the LAN. The file transfer process works fine: once the
"saving file" or "copying file" process has begun, it takes the same
amount of time needed by a SCP or a FTP transfer command using the same
VPN connection. I tried to copy a 2MB file from client to server and the
time needed using SCP and using SAMBA (once the copy process was
started) was the same.
I tried to add some "socket options = TCP_NODELAY SO_SNDBUF=8192
SO_RCVBUF=8192" in smb.conf with no results.
The problem is the same using "explorer", command prompt, or any program
in the client. We currently use all XP SP2 clients.
It looks like the initial and final talking acknowledgement between
client and server for any kind of operation is unacceptably slow, while
the file transfer process seems not to be involved in this problem.
Here is my smb.conf global parameters section:
# Global parameters
[global]
workgroup = MYGROUP
netbios name = MYSERVER
server string = %h server (Samba %v)
encrypt passwords = Yes
obey pam restrictions = Yes
passwd program = /usr/bin/passwd %u
passwd chat = *Enter\snew\sUNIX\spassword:* %n\n
*Retype\snew\sUNIX\spassword:* %n\n .
syslog = 0
log file = /var/log/samba/log.%m
log level = 2
max log size = 1000
invalid users = root
create mask = 0664
directory mask = 0775


[homes]
comment = Home Directories
create mask = 0600
directory mask = 0700
browseable = No


[pdf]
path = /tmp
guest ok = Yes
printable = Yes
print command = /usr/bin/printpdf %s %u
lpq command =
lprm command =

[... shares defitinions omitted..]

Any kind of help will be really appreciated.
Thank you

Gianfranco Pra Floriani
Nexta Media - Italy

[Dave Kempe]

Post by gianfranco pra floriani
Other services using the VPN such as SCP, SSH, HTTP, FTP work very
good on the same connection, with no slow issues at all. I tried 2
kinds of VPN connections (OpenVPN and a router-proprietary VPN
gateway-to-client), and both have the same issue, both only with Samba.
I wonder if there is something I'm missing in client or server
configuration that makes Samba talking very slow when connections are
not coming from the LAN. The file transfer process works fine: once
the "saving file" or "copying file" process has begun, it takes the
same amount of time needed by a SCP or a FTP transfer command using
the same VPN connection. I tried to copy a 2MB file from client to
server and the time needed using SCP and using SAMBA (once the copy
process was started) was the same.

I have only got this to work nicely by using a WINS server pushed to
OpenVPN clients.
add something like to your openvpn config:
push "dhcp-option WINS 192.168.0.64"

and setup the samba server to be the wins server, and make sure everyone
on the lan points to it too (dhcp can do this)

then name resolution seems to work better, the client avoids broadcasts,
which don't flow over the VPN, and everthing is much faster.

dave

[gianfranco pra floriani]

Hello Dave,
Thank you for your reply.
I added the following parameters to the global section:
wins support = yes
name resolve order = wins lmhosts hosts bcast
local master = yes
preferred master = yes
domain master = yes
os level = 65

and set VPN config and DHCP options to use the samba server as WINS
server, and restarted all clients.
I cannot say that the response time to any operation has been improved.
I checked the logs created by Samba for each client: as an example when
trying to "CTRL-C CTRL-V" a 1KB file from explorer nothing in the log
happens for about 17-18 seconds, and only after that time something is
shown:

[2008/03/25 15:17:56, 2] smbd/open.c:open_file(352)
gian opened file docs/divertimento/frame.html read=Yes write=No
(numopen=3)
[2008/03/25 15:17:57, 2] smbd/close.c:close_normal_file(344)
gian closed file docs/divertimento/frame.html (numopen=2)
[2008/03/25 15:17:57, 2] smbd/open.c:open_file(352)
gian opened file docs/divertimento/frame.html read=Yes write=No
(numopen=3)
[2008/03/25 15:18:01, 2] smbd/open.c:open_file(352)
gian opened file docs/divertimento/Copia di frame.html read=No
write=Yes (numopen=4)
[2008/03/25 15:18:01, 2] smbd/close.c:close_normal_file(344)
gian closed file docs/divertimento/Copia di frame.html (numopen=3)
[2008/03/25 15:18:17, 2] smbd/close.c:close_normal_file(344)
gian closed file docs/divertimento/frame.html (numopen=2)

As you can see, it took about 18 seconds to start the operation (no
logging info), and then 21 more seconds from the first "open" to the
last "close". This is quite annoying.
Did I set something wrong?
Thank you all for any help.
Sincerely
gian

Post by Dave Kempe

Post by gianfranco pra floriani
Other services using the VPN such as SCP, SSH, HTTP, FTP work very
good on the same connection, with no slow issues at all. I tried 2
kinds of VPN connections (OpenVPN and a router-proprietary VPN
gateway-to-client), and both have the same issue, both only with Samba.
I wonder if there is something I'm missing in client or server
configuration that makes Samba talking very slow when connections are
not coming from the LAN. The file transfer process works fine: once
the "saving file" or "copying file" process has begun, it takes the
same amount of time needed by a SCP or a FTP transfer command using
the same VPN connection. I tried to copy a 2MB file from client to
server and the time needed using SCP and using SAMBA (once the copy
process was started) was the same.

I have only got this to work nicely by using a WINS server pushed to
OpenVPN clients.
push "dhcp-option WINS 192.168.0.64"
and setup the samba server to be the wins server, and make sure
everyone on the lan points to it too (dhcp can do this)
then name resolution seems to work better, the client avoids
broadcasts, which don't flow over the VPN, and everthing is much faster.
dave

[Dave Kempe]

Post by gianfranco pra floriani
As you can see, it took about 18 seconds to start the operation (no
logging info), and then 21 more seconds from the first "open" to the
last "close". This is quite annoying.
Did I set something wrong?
Thank you all for any help.
Sincerely
gian

Can you make sure the wins.dat file is populated with the client and server?
Also, is it faster if you browse to the IP from the client?
ie: \\192.168.0.64\ or something?

dave

[gianfranco pra floriani]

Hello Dave,
wins.dat is populated (57 entries) with rows like:
"NBGIANNINO#00" 1206781498 10.0.0.190 64R
"BLADE#00" 1206741105 10.0.0.7 66R
"BLADE#03" 1206741105 10.0.0.7 66R
"BLADE#20" 1206741105 10.0.0.7 66R
(blade is the server, nbgiannino is my pc, just as example)

using ip address (\\10.0.0.7) does not change anything in response time.

I tried to stop samba and remove wins.dat and wins.tdb to see if they
were created again once samba was restarted. This is exactly what
happened. Nothing changed in delays though.
This is really weird.

gian

Post by Dave Kempe

Post by gianfranco pra floriani
As you can see, it took about 18 seconds to start the operation (no
logging info), and then 21 more seconds from the first "open" to the
last "close". This is quite annoying.
Did I set something wrong?
Thank you all for any help.
Sincerely
gian

Can you make sure the wins.dat file is populated with the client and server?
Also, is it faster if you browse to the IP from the client?
ie: \\192.168.0.64\ or something?
dave

[Dave Kempe]

Post by gianfranco pra floriani
using ip address (\\10.0.0.7) does not change anything in response time.

then it might not be wins/name resolution at all.
Perhaps a packet sniffer might shed some light on it?
run tethereal Or tshark as its not called on the tun interface on the
server when you are attempting to get the file.

ie, tethereal -i tun0

dave

[Scott Lovenberg]

Post by Dave Kempe

Post by gianfranco pra floriani
using ip address (\\10.0.0.7) does not change anything in response time.

then it might not be wins/name resolution at all.
Perhaps a packet sniffer might shed some light on it?
run tethereal Or tshark as its not called on the tun interface on the
server when you are attempting to get the file.
ie, tethereal -i tun0
dave

What is your resolve order? Are you using DNS or broadcasts before wins?

[gianfranco pra floriani]

Hello Scott,

[global]
[...]
name resolve order = wins lmhosts hosts bcast
[...]
see my previous emails for a full list of global
thank you
gian

Post by Scott Lovenberg

Post by Dave Kempe

Post by gianfranco pra floriani
using ip address (\\10.0.0.7) does not change anything in response time.

then it might not be wins/name resolution at all.
Perhaps a packet sniffer might shed some light on it?
run tethereal Or tshark as its not called on the tun interface on the
server when you are attempting to get the file.
ie, tethereal -i tun0
dave

What is your resolve order? Are you using DNS or broadcasts before wins?

[Udo Rader]

Post by gianfranco pra floriani
Hello,
I have Samba version 3.0.24 running on a 2.6.14-gentoo-r5 x86 kernel
(xeon 3ghz, 1gb ram raid 5).
All clients accessing samba shares via LAN have no problems. Samba
server works perfectly and fast.
We are instead experiencing serious performance issues when accessing
samba shares from remote clients (WAN), via VPN.
Simple operations like "open a file", "copy & paste a file", "save a
file" from Windows XP SP2 clients are incredibly slow. It may take 10
seconds to open a "save as" dialog box, and maybe 15 more seconds to
save a "hello world" txt file from Notepad.
Other services using the VPN such as SCP, SSH, HTTP, FTP work very good
on the same connection, with no slow issues at all. I tried 2 kinds of
VPN connections (OpenVPN and a router-proprietary VPN
gateway-to-client), and both have the same issue, both only with Samba.
I wonder if there is something I'm missing in client or server
configuration that makes Samba talking very slow when connections are
not coming from the LAN. The file transfer process works fine: once the
"saving file" or "copying file" process has begun, it takes the same
amount of time needed by a SCP or a FTP transfer command using the same
VPN connection. I tried to copy a 2MB file from client to server and the
time needed using SCP and using SAMBA (once the copy process was
started) was the same.
I tried to add some "socket options = TCP_NODELAY SO_SNDBUF=8192
SO_RCVBUF=8192" in smb.conf with no results.
The problem is the same using "explorer", command prompt, or any program
in the client. We currently use all XP SP2 clients.
It looks like the initial and final talking acknowledgement between
client and server for any kind of operation is unacceptably slow,
while the file transfer process seems not to be involved in this
problem.

This is quite common with VPN connections. What response time do you get
from a ping (LAN vs. VPN)?

A major network performance for VPN clients is the correct configuration
of various networking parameters (such as MTU, window size, etc. - all
depending on the type of internet connection you have).

And finally, what type of VPN are you using?

Very probably you cannot do much about it from the samba side.
--
?Udo Rader

bestsolution.at EDV Systemhaus GmbH
http://www.bestsolution.at

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba/attachments/20080326/87a4d5b7/attachment.bin

[gianfranco pra floriani]

Post by Udo Rader

Post by gianfranco pra floriani
Hello,
I have Samba version 3.0.24 running on a 2.6.14-gentoo-r5 x86 kernel
(xeon 3ghz, 1gb ram raid 5).
All clients accessing samba shares via LAN have no problems. Samba
server works perfectly and fast.
We are instead experiencing serious performance issues when accessing
samba shares from remote clients (WAN), via VPN.
Simple operations like "open a file", "copy & paste a file", "save a
file" from Windows XP SP2 clients are incredibly slow. It may take 10
seconds to open a "save as" dialog box, and maybe 15 more seconds to
save a "hello world" txt file from Notepad.
Other services using the VPN such as SCP, SSH, HTTP, FTP work very good
on the same connection, with no slow issues at all. I tried 2 kinds of
VPN connections (OpenVPN and a router-proprietary VPN
gateway-to-client), and both have the same issue, both only with Samba.
I wonder if there is something I'm missing in client or server
configuration that makes Samba talking very slow when connections are
not coming from the LAN. The file transfer process works fine: once the
"saving file" or "copying file" process has begun, it takes the same
amount of time needed by a SCP or a FTP transfer command using the same
VPN connection. I tried to copy a 2MB file from client to server and the
time needed using SCP and using SAMBA (once the copy process was
started) was the same.
I tried to add some "socket options = TCP_NODELAY SO_SNDBUF=8192
SO_RCVBUF=8192" in smb.conf with no results.
The problem is the same using "explorer", command prompt, or any program
in the client. We currently use all XP SP2 clients.
It looks like the initial and final talking acknowledgement between
client and server for any kind of operation is unacceptably slow,
while the file transfer process seems not to be involved in this
problem.

This is quite common with VPN connections. What response time do you get
from a ping (LAN vs. VPN)?

Hello Udo,
this is a ping from the server to a client:
PING 10.0.0.190 (10.0.0.190) 56(84) bytes of data.
64 bytes from 10.0.0.190: icmp_seq=1 ttl=128 time=52.7 ms
64 bytes from 10.0.0.190: icmp_seq=2 ttl=128 time=48.9 ms
64 bytes from 10.0.0.190: icmp_seq=3 ttl=128 time=49.2 ms
from client to server the ping time is the same.

Post by Udo Rader
A major network performance for VPN clients is the correct configuration
of various networking parameters (such as MTU, window size, etc. - all
depending on the type of internet connection you have).
And finally, what type of VPN are you using?

we have ssh, scp, ftp and http services running on the same VPN
(OpenVPN 2.0.6 i686-pc-linux-gnu), and all services are running fine, no
delays, no bottlenecks. Samba is the only service having problems.

Post by Udo Rader
Very probably you cannot do much about it from the samba side.

[Alex Harrington]

Post by gianfranco pra floriani
All clients accessing samba shares via LAN have no problems. Samba
server works perfectly and fast.
We are instead experiencing serious performance issues when accessing
samba shares from remote clients (WAN), via VPN.

I've had this before and never solved it. I always put it down to
Windows Explorer "touching" the files for thumbnails etc.

Try mapping a drive from the VPN client to the Samba box, then drop to a
command prompt and try copy/deleting files from there. Is the speed any
better?

Alex

[Udo Rader]

Post by gianfranco pra floriani

Post by Udo Rader

Post by gianfranco pra floriani
Hello,
I have Samba version 3.0.24 running on a 2.6.14-gentoo-r5 x86 kernel
(xeon 3ghz, 1gb ram raid 5).
All clients accessing samba shares via LAN have no problems. Samba
server works perfectly and fast.
We are instead experiencing serious performance issues when accessing
samba shares from remote clients (WAN), via VPN.
Simple operations like "open a file", "copy & paste a file", "save a
file" from Windows XP SP2 clients are incredibly slow. It may take 10
seconds to open a "save as" dialog box, and maybe 15 more seconds to
save a "hello world" txt file from Notepad.
Other services using the VPN such as SCP, SSH, HTTP, FTP work very good
on the same connection, with no slow issues at all. I tried 2 kinds of
VPN connections (OpenVPN and a router-proprietary VPN
gateway-to-client), and both have the same issue, both only with Samba.
I wonder if there is something I'm missing in client or server
configuration that makes Samba talking very slow when connections are
not coming from the LAN. The file transfer process works fine: once the
"saving file" or "copying file" process has begun, it takes the same
amount of time needed by a SCP or a FTP transfer command using the same
VPN connection. I tried to copy a 2MB file from client to server and the
time needed using SCP and using SAMBA (once the copy process was
started) was the same.
I tried to add some "socket options = TCP_NODELAY SO_SNDBUF=8192
SO_RCVBUF=8192" in smb.conf with no results.
The problem is the same using "explorer", command prompt, or any program
in the client. We currently use all XP SP2 clients.
It looks like the initial and final talking acknowledgement between
client and server for any kind of operation is unacceptably slow,
while the file transfer process seems not to be involved in this
problem.

This is quite common with VPN connections. What response time do you get
from a ping (LAN vs. VPN)?

Hello Udo,
PING 10.0.0.190 (10.0.0.190) 56(84) bytes of data.
64 bytes from 10.0.0.190: icmp_seq=1 ttl=128 time=52.7 ms
64 bytes from 10.0.0.190: icmp_seq=2 ttl=128 time=48.9 ms
64 bytes from 10.0.0.190: icmp_seq=3 ttl=128 time=49.2 ms
from client to server the ping time is the same.

doesn't look too bad.

Post by gianfranco pra floriani

Post by Udo Rader
A major network performance for VPN clients is the correct configuration
of various networking parameters (such as MTU, window size, etc. - all
depending on the type of internet connection you have).
And finally, what type of VPN are you using?

we have ssh, scp, ftp and http services running on the same VPN
(OpenVPN 2.0.6 i686-pc-linux-gnu), and all services are running fine, no
delays, no bottlenecks. Samba is the only service having problems.

Are you sure that it is a samba problem? Try to create a share on a WXP
LAN box and try to access it from a remote box.

Your problem is very likely a SMB (and not samba) problem.

And what type of OpenVPN adapter do you use? tun or tap?
--
?Udo Rader

bestsolution.at EDV Systemhaus GmbH
http://www.bestsolution.at

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba/attachments/20080326/6fe2aa93/attachment.bin

[gianfranco pra floriani]

Hello Udo,

Post by Udo Rader

Post by gianfranco pra floriani

Post by Udo Rader

Post by gianfranco pra floriani
Hello,
I have Samba version 3.0.24 running on a 2.6.14-gentoo-r5 x86 kernel
(xeon 3ghz, 1gb ram raid 5).
All clients accessing samba shares via LAN have no problems. Samba
server works perfectly and fast.
We are instead experiencing serious performance issues when accessing
samba shares from remote clients (WAN), via VPN.
Simple operations like "open a file", "copy & paste a file", "save a
file" from Windows XP SP2 clients are incredibly slow. It may take 10
seconds to open a "save as" dialog box, and maybe 15 more seconds to
save a "hello world" txt file from Notepad.
Other services using the VPN such as SCP, SSH, HTTP, FTP work very good
on the same connection, with no slow issues at all. I tried 2 kinds of
VPN connections (OpenVPN and a router-proprietary VPN
gateway-to-client), and both have the same issue, both only with Samba.
I wonder if there is something I'm missing in client or server
configuration that makes Samba talking very slow when connections are
not coming from the LAN. The file transfer process works fine: once the
"saving file" or "copying file" process has begun, it takes the same
amount of time needed by a SCP or a FTP transfer command using the same
VPN connection. I tried to copy a 2MB file from client to server and the
time needed using SCP and using SAMBA (once the copy process was
started) was the same.
I tried to add some "socket options = TCP_NODELAY SO_SNDBUF=8192
SO_RCVBUF=8192" in smb.conf with no results.
The problem is the same using "explorer", command prompt, or any program
in the client. We currently use all XP SP2 clients.
It looks like the initial and final talking acknowledgement between
client and server for any kind of operation is unacceptably slow,
while the file transfer process seems not to be involved in this
problem.

This is quite common with VPN connections. What response time do you get
from a ping (LAN vs. VPN)?

Hello Udo,
PING 10.0.0.190 (10.0.0.190) 56(84) bytes of data.
64 bytes from 10.0.0.190: icmp_seq=1 ttl=128 time=52.7 ms
64 bytes from 10.0.0.190: icmp_seq=2 ttl=128 time=48.9 ms
64 bytes from 10.0.0.190: icmp_seq=3 ttl=128 time=49.2 ms
from client to server the ping time is the same.

doesn't look too bad.

Post by gianfranco pra floriani

Post by Udo Rader
A major network performance for VPN clients is the correct configuration
of various networking parameters (such as MTU, window size, etc. - all
depending on the type of internet connection you have).
And finally, what type of VPN are you using?

we have ssh, scp, ftp and http services running on the same VPN
(OpenVPN 2.0.6 i686-pc-linux-gnu), and all services are running fine, no
delays, no bottlenecks. Samba is the only service having problems.

Are you sure that it is a samba problem? Try to create a share on a WXP
LAN box and try to access it from a remote box.
Your problem is very likely a SMB (and not samba) problem.

Ok, I created a WXP LAN box share and accessed from a remote box: Same
old story: SLOW!

Post by Udo Rader
And what type of OpenVPN adapter do you use? tun or tap?

dev tap0
proto udp

As you point out, it looks more a SMB (and not samba) problem.

I will now try to discover the problems and let you know.
Any hints will be really appreciated.

Thank you very much
gian


(PS: To make a further test, I installed webdrive, it maps a FTP
connection to a network drive letter.. a true beautiful speed, almost as
fast as being in LAN)

[Dave Kempe]

Post by gianfranco pra floriani
dev tap0
proto udp
As you point out, it looks more a SMB (and not samba) problem.

I would seriously suggest you try dev tun0.
make sure the routes work fine, then try again.
I think that you have some sort of networking issue where broadcasts are
getting confused.
perhaps the machines think they are on the same subnet, but they are not
on the same 'wire'

dave

[gianfranco pra floriani]

hello Dave,
I'm still trying to understand how to check all the traffic with
ethereal (now wireshark) as you wrote a few days ago.
I had some troubles installing wirshark server side so now i'm doing the
tests from a client (XP).
Now, I will also check out your suggestion to use tun0.
If I remember, I chose tap because I'm using server-bridge (same subnet)
and not routing for VPN.
However, I had the same performance issues when using the router client
VPN connecting to the router's (hardware config) VPN.
I will let you know.
Thank you so much for your advices.
gian

Post by Dave Kempe

Post by gianfranco pra floriani
dev tap0
proto udp
As you point out, it looks more a SMB (and not samba) problem.

I would seriously suggest you try dev tun0.
make sure the routes work fine, then try again.
I think that you have some sort of networking issue where broadcasts
are getting confused.
perhaps the machines think they are on the same subnet, but they are
not on the same 'wire'
dave

[ziotegolina]

Problem solved romoving openvpn and installing PPPTP.
That software is free and REALLY better than openvpn.

Hallo,
I've got the same problem using openvpn & samba server.
Did you fix it? could you help me telling me how to solve it?
thanks,
Dario

Post by gianfranco pra floriani
hello Dave,
I'm still trying to understand how to check all the traffic with
ethereal (now wireshark) as you wrote a few days ago.
I had some troubles installing wirshark server side so now i'm doing the
tests from a client (XP).
Now, I will also check out your suggestion to use tun0.
If I remember, I chose tap because I'm using server-bridge (same subnet)
and not routing for VPN.
However, I had the same performance issues when using the router client
VPN connecting to the router's (hardware config) VPN.
I will let you know.
Thank you so much for your advices.
gian

Post by Dave Kempe

Post by gianfranco pra floriani
dev tap0
proto udp
As you point out, it looks more a SMB (and not samba) problem.

I would seriously suggest you try dev tun0.
make sure the routes work fine, then try again.
I think that you have some sort of networking issue where broadcasts
are getting confused.
perhaps the machines think they are on the same subnet, but they are
not on the same 'wire'
dave

--
To unsubscribe from this list go to the following URL and read the
instructions: https://lists.samba.org/mailman/listinfo/samba

%-|

[stoffell]

On Mon, Mar 24, 2008 at 10:30 PM, gianfranco pra floriani

Post by gianfranco pra floriani
Hello,
I have Samba version 3.0.24 running on a 2.6.14-gentoo-r5 x86 kernel
(xeon 3ghz, 1gb ram raid 5).
All clients accessing samba shares via LAN have no problems. Samba
server works perfectly and fast.

I have about the same "problem" with samba/openvpn but only with Vista
clients. Also see my previous posts on this subject:
http://lists.samba.org/archive/samba/2007-December/136814.html

And a specific but a bit different issue with Vista (non-SP1), even on
local LAN:
http://lists.samba.org/archive/samba/2008-March/139268.html

Just wanted to let you know, maybe it's related..

cheers,
stoffell