[Samba] Very slow initial opening MS-Word and MS-Excel files from Samba

[Berend Tober]

The first time a Word or Excel file is opened, i.e., when either Word or
Excel have not been actively running "recently" (like, say for several
minutes or more), the time it takes to start the application and load
the file seem inordinately long.

Once the first evolution is complete, files open more-or-less instantly
if I close the first document but leave either app running (but void of
any open documents). This pertains to the same file opened a second
time, or different files -- once Word/Excel has done whatever it is
thats takes so long for initial start up, then load performance is
acceptable (in fact it is very impressive!).

And it is not merely the loading of the app, because it happens both
when I start the apps implicitly by double-clicking on a document in
Windows Explorer, but also if I first start Word/Excel explicitly from
the START menu without a specific document and then open one from the
Samba share using File|Open.

Envirnoment:

Server is RHEL 5 with Samba version samba-3.0.23c-2.el5.2.0.2

Desktops are XP Pro SP2 on some, and XP Home SP2 on others.

MS Office Suite is XP Pro (i.e., Word/Excel 2002)

Desktops also run Symantec Corporate Edition 8.1 (although same problem
continues on a machine from which I uninstalled that as part of my
debugging.)

I initially had no explicit settings regarding opportunistic locking in
smb.conf, but I've tried server different configurations without
improvement. What appears below is the current configuration and the
problem still pertains.


smb.conf:

[global]
workgroup = mygroup
server string = Samba Server
hosts allow = 192.168.123. 127.
cups options = raw
log file = /var/log/samba/%m.log
max log size = 50
dns proxy = no
username map = /etc/samba/smbusers
case sensitive = no
kernel oplocks = no


[homes]
comment =
browseable = no
writeable = yes

[acct]
path = /usr/local/var/samba/acct
writeable = yes
guest ok = yes
browseable = no
create mask = 0765
directory mask = 0775
oplocks = False
level2 oplocks = False


[engr]
path = /usr/local/var/samba/engr
writeable = yes
browseable = no
guest ok = yes
create mask = 0765
directory mask = 0775
oplocks = False
level2 oplocks = False


Active Directory is not used. All workstations (about 25) simply map
drive letters locally.

This performance problem appeared only after we began using Samba for
file server. Never saw this in close to 15 years of using Netware -- so
I suspect it is not purely an MS-Office problem, but something to do
with the interaction between MS-Office and Samba. I eagerly await your help!

[Alex Crow]

Berend,

This wasn't a migration from an NT domain was it? We had the problem
after a migration that starting up Office programs was incredibly slow -
it turned out there were a load of Office registry entries pointing to
UNC paths on the old Windows PDC.

Just an idea...

Cheers

Alex

[Berend Tober]

Post by Alex Crow
This wasn't a migration from an NT domain was it? We had the problem
after a migration that starting up Office programs was incredibly slow -
it turned out there were a load of Office registry entries pointing to
UNC paths on the old Windows PDC.

No. Not NT. Previous file server was Netware 5.1 (and 4.2, 4.1, and
4.0... before that). Never had the problem on those machines. (And as a
salute to Novell, it is pretty impressive to note that the machine we
retired was a 1.5Ghz P4 w/ 512MB RAM and it was readily able to keep up
with two dozen desktop clients that universally had better CPU and RAM
specs! The new box is substantially more capable with 16x the RAM, for
starters).

Under Netware the local workstation users did not have to explicitly map
drive letters, because Netware had a login script utility that handled
that. In the new environment we just established each client with
"reconnect at login" drive letter mappings to the corresponding Samba
shares to where all the files had been copied to from the legacy Netware
server -- so everyone that had made desktop shortcuts into often-used
sub-directories on the old server were still able to use them on the new
server when the showed up on Monday morning. Those desktop shortcuts are
certainly not the source of the problem, though, because I'm debugging
by navigating to the file location directly in Windows Explorer, or
using the File|Open dialog from the main menu in either app.

[Alex Crow]

Post by Berend Tober

Post by Alex Crow
This wasn't a migration from an NT domain was it? We had the problem
after a migration that starting up Office programs was incredibly slow -
it turned out there were a load of Office registry entries pointing to
UNC paths on the old Windows PDC.

No. Not NT. Previous file server was Netware 5.1 (and 4.2, 4.1, and
4.0... before that). Never had the problem on those machines. (And as a
salute to Novell, it is pretty impressive to note that the machine we
retired was a 1.5Ghz P4 w/ 512MB RAM and it was readily able to keep up
with two dozen desktop clients that universally had better CPU and RAM
specs! The new box is substantially more capable with 16x the RAM, for
starters).
Under Netware the local workstation users did not have to explicitly map
drive letters, because Netware had a login script utility that handled
that. In the new environment we just established each client with
"reconnect at login" drive letter mappings to the corresponding Samba
shares to where all the files had been copied to from the legacy Netware
server -- so everyone that had made desktop shortcuts into often-used
sub-directories on the old server were still able to use them on the new
server when the showed up on Monday morning. Those desktop shortcuts are
certainly not the source of the problem, though, because I'm debugging
by navigating to the file location directly in Windows Explorer, or
using the File|Open dialog from the main menu in either app.

Just an idea - we found things that were completely unrelated to how we
thought the previous setup would be. It still might be worth scanning
the registry on a couple of client machines though.

Cheers

Alex
--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.

"Transact" is operated by Integrated Financial Arrangements plc
Domain House, 5-7 Singer Street, London EC2A 4BQ
Tel: [phone] Fax: [phone]
(Registered office: as above; Registered in England and Wales under
number: 3727592)
Authorised and regulated by the Financial Services Authority (entered on
the FSA Register; number: 190856)

[Aaron Kincer]

My first question would be does this happen with other applications or
strictly Office? Do you get the same behavior if you attempt to open a
.doc file with Open Office?

Second, have you watched your samba logs in real time (example: tail -f
/var/log/samba/<your_pc_log>) as you try to open a file to see what's
happening behind the scenes?

Lastly, I'm not sure what happens when you use true or false as a value
for oplocks and level2 oplocks, but I thought the values were either yes
or no.

Post by Berend Tober
The first time a Word or Excel file is opened, i.e., when either Word
or Excel have not been actively running "recently" (like, say for
several minutes or more), the time it takes to start the application
and load the file seem inordinately long.
Once the first evolution is complete, files open more-or-less
instantly if I close the first document but leave either app running
(but void of any open documents). This pertains to the same file
opened a second time, or different files -- once Word/Excel has done
whatever it is thats takes so long for initial start up, then load
performance is acceptable (in fact it is very impressive!).
And it is not merely the loading of the app, because it happens both
when I start the apps implicitly by double-clicking on a document in
Windows Explorer, but also if I first start Word/Excel explicitly from
the START menu without a specific document and then open one from the
Samba share using File|Open.
Server is RHEL 5 with Samba version samba-3.0.23c-2.el5.2.0.2
Desktops are XP Pro SP2 on some, and XP Home SP2 on others.
MS Office Suite is XP Pro (i.e., Word/Excel 2002)
Desktops also run Symantec Corporate Edition 8.1 (although same
problem continues on a machine from which I uninstalled that as part
of my debugging.)
I initially had no explicit settings regarding opportunistic locking
in smb.conf, but I've tried server different configurations without
improvement. What appears below is the current configuration and the
problem still pertains.
[global]
workgroup = mygroup
server string = Samba Server
hosts allow = 192.168.123. 127.
cups options = raw
log file = /var/log/samba/%m.log
max log size = 50
dns proxy = no
username map = /etc/samba/smbusers
case sensitive = no
kernel oplocks = no
[homes]
comment =
browseable = no
writeable = yes
[acct]
path = /usr/local/var/samba/acct
writeable = yes
guest ok = yes
browseable = no
create mask = 0765
directory mask = 0775
oplocks = False
level2 oplocks = False
[engr]
path = /usr/local/var/samba/engr
writeable = yes
browseable = no
guest ok = yes
create mask = 0765
directory mask = 0775
oplocks = False
level2 oplocks = False
Active Directory is not used. All workstations (about 25) simply map
drive letters locally.
This performance problem appeared only after we began using Samba for
file server. Never saw this in close to 15 years of using Netware --
so I suspect it is not purely an MS-Office problem, but something to
do with the interaction between MS-Office and Samba. I eagerly await
your help!

[Berend Tober]

Post by Aaron Kincer
My first question would be does this happen with other applications or
strictly Office?

No one has complained about other apps, which in our case the next most
heavily used is AutoCAD. I think I would have heard by now if that were
a problem.

Post by Aaron Kincer
Do you get the same behavior if you attempt to open a
.doc file with Open Office?

The answer is yes to that, but I would estimate that it is a little less
noticeable.

Post by Aaron Kincer
Second, have you watched your samba logs in real time (example: tail -f
/var/log/samba/<your_pc_log>) as you try to open a file to see what's
happening behind the scenes?

I have, and with log level = 5, but I don't really get a lot out that
data. Do you think posting a sample would be beneficial?

Post by Aaron Kincer
Lastly, I'm not sure what happens when you use true or false as a value
for oplocks and level2 oplocks, but I thought the values were either yes
or no.

I've seen both posted in different articles my buddy Google and I turned
up, and testparm reported no problems, but I'll switch false to no.

[Steve Romanow]

How about the temp directory in Office Preferences. Is it on the local
workstation, or does it default to the dir where the file is opened. I
always thought that could slow things down. (i.e. the creation of the
~foo.doc file).

HTH,
Steve

[Berend Tober]

Post by Berend Tober

...
Do you get the same behavior if you attempt to open a .doc file
with Open Office?

The answer is yes to that, but I would estimate that it is a little
less noticeable.

Let me correct that. It is a lot less noticable. Maybe even it doesn't
happen and I'm seeing is the difference in load time for the OO
software. Hard to tell for sure. Definitely it is not as bad with OO.

Post by Berend Tober
Lastly, I'm not sure what happens when you use true or false as a value
for oplocks and level2 oplocks, but I thought the values were either yes
or no.

Using "no" in place of "false" made not difference.

[Aaron Kincer]

Is there any difference in behavior when you open the files from within
the applications themselves so that you can remove the application load
time from the equation?

Just a little FYI--MS Office is a strange animal in how it and Samba
play together. It is not uncommon to see behavior there that you don't
see anywhere else. For example--I'd pay good money to see a Samba
configuration where MS Word (2003) sets the archive bit properly after a
file is edited.

Also, as others have mentioned, Windows and its applications can have
long memories about servers contacted in the past. For example, the list
of recently opened files.

I doubt this would have an effect, but just for giggles you could flush
the dns cache (ipconfig /flushdns)

Post by Berend Tober

Post by Berend Tober

...
Do you get the same behavior if you attempt to open a .doc file
with Open Office?

The answer is yes to that, but I would estimate that it is a little
less noticeable.

Let me correct that. It is a lot less noticable. Maybe even it doesn't
happen and I'm seeing is the difference in load time for the OO
software. Hard to tell for sure. Definitely it is not as bad with OO.

Post by Berend Tober
Lastly, I'm not sure what happens when you use true or false as a
value for oplocks and level2 oplocks, but I thought the values were
either yes or no.

Using "no" in place of "false" made not difference.

[Jonathan Johnson]

Please review the Samba HOWTO, chapter 10, "Common Errors" where it
discusses this issue.

http://us4.samba.org/samba/docs/man/Samba-HOWTO-Collection/NetworkBrowsing.html#id350945

Jonathan Johnson
Sutinen Consulting, Inc.
www.sutinen.com

Post by Aaron Kincer
Also, as others have mentioned, Windows and its applications can have
long memories about servers contacted in the past. For example, the
list of recently opened files.

[Aaron Kincer]

I fixed this by using the following in login scripts for every share
each domain account was supposed to have:

net use driveletter: /delete
net use driverletter: \\newserver\newsharelocation

This fixed 99% of the invalid shares we had. There were a few people
that had manually mapped drives. I just removed their mappings and added
their mapped drives to their login scripts to set it correctly and that
solved the issue.

I eventually turned this off after I was satisfied all bad mappings had
been resolved. Of course, it was the complaint of a user who uses
offline files heavily that really helped nudge me into turning it off.
The delete command was mucking up his sync process when he logged in
first thing in the morning. Offline files sync as soon as they possibly
can. Login scripts continue run if they are taking a while to complete
despite syncing. The whole delete and recreating of a share seems to
take just long enough so that a share can be delete while syncing is
going on. The sync process gets very moody when you take away its mapped
drive during sync.

Post by Jonathan Johnson
Please review the Samba HOWTO, chapter 10, "Common Errors" where it
discusses this issue.
http://us4.samba.org/samba/docs/man/Samba-HOWTO-Collection/NetworkBrowsing.html#id350945
Jonathan Johnson
Sutinen Consulting, Inc.
www.sutinen.com

Post by Aaron Kincer
Also, as others have mentioned, Windows and its applications can have
long memories about servers contacted in the past. For example, the
list of recently opened files.

[Marcello Romani]

Post by Berend Tober
The first time a Word or Excel file is opened, i.e., when either Word or
Excel have not been actively running "recently" (like, say for several
minutes or more), the time it takes to start the application and load
the file seem inordinately long.
Once the first evolution is complete, files open more-or-less instantly
if I close the first document but leave either app running (but void of
any open documents). This pertains to the same file opened a second
time, or different files -- once Word/Excel has done whatever it is
thats takes so long for initial start up, then load performance is
acceptable (in fact it is very impressive!).
And it is not merely the loading of the app, because it happens both
when I start the apps implicitly by double-clicking on a document in
Windows Explorer, but also if I first start Word/Excel explicitly from
the START menu without a specific document and then open one from the
Samba share using File|Open.
Server is RHEL 5 with Samba version samba-3.0.23c-2.el5.2.0.2
Desktops are XP Pro SP2 on some, and XP Home SP2 on others.
MS Office Suite is XP Pro (i.e., Word/Excel 2002)
Desktops also run Symantec Corporate Edition 8.1 (although same problem
continues on a machine from which I uninstalled that as part of my
debugging.)
I initially had no explicit settings regarding opportunistic locking in
smb.conf, but I've tried server different configurations without
improvement. What appears below is the current configuration and the
problem still pertains.
[global]
workgroup = mygroup
server string = Samba Server
hosts allow = 192.168.123. 127.
cups options = raw
log file = /var/log/samba/%m.log
max log size = 50
dns proxy = no
username map = /etc/samba/smbusers
case sensitive = no
kernel oplocks = no
[homes]
comment =
browseable = no
writeable = yes
[acct]
path = /usr/local/var/samba/acct
writeable = yes
guest ok = yes
browseable = no
create mask = 0765
directory mask = 0775
oplocks = False
level2 oplocks = False
[engr]
path = /usr/local/var/samba/engr
writeable = yes
browseable = no
guest ok = yes
create mask = 0765
directory mask = 0775
oplocks = False
level2 oplocks = False
Active Directory is not used. All workstations (about 25) simply map
drive letters locally.
This performance problem appeared only after we began using Samba for
file server. Never saw this in close to 15 years of using Netware -- so
I suspect it is not purely an MS-Office problem, but something to do
with the interaction between MS-Office and Samba. I eagerly await your help!

Have you tried copying (tens of MBs) files or even those same office
files to/from the share ? What is the performance there ?

I had a similar performance problem which showed up in Office, but I
found out it was not limited to "office" files, but affected the
transfer speed between the client and the server in general.

I have found a solution and posted it to this list, search for the
thread "Vista client / Linux server - hight browsing latency" (or I will
send you the e-mail if you so desire).

HTH

[Berend Tober]

Post by Berend Tober
The first time a Word or Excel file is opened, i.e., when either Word
or Excel have not been actively running "recently" (like, say for
several minutes or more), the time it takes to start the application
and load the file seem inordinately long.
Once the first evolution is complete, files open more-or-less
instantly ...

Part of this implementation involved "hedging my bets", meaning I left
the Novell Netware Client software installed on all the desktop
computers in case there was a major, unanticipated disaster and we had
to resume legacy server operations. Once that client software is
removed, the initial load performance problem ceases.