[Samba] vfat / ftruncate problem

[Tom Schaefer]

I recently added a 60Gig HD to my little linux server at home with the intention of sharing it via Samba to all my PCs which run Windows 98SE. I really want to use vfat file system on it.

The server is running Mandrake 8.1, so kernel 2.4.something, and the Samba that came with it.

Much to my disappointment I quickly discovered poor performance copying files to it, the copy is like slow to get going, and trying to copy big files to it (like 750Meg is the size of the file I keep testing with) just fails outright.

I tried it on a different system on my Lan, also running Mandrake 8.1, same story copying to vfat from Win98SE.

I downloaded, compiled, and installed Samba 2.2.8a, same story.

I installed Mandrake 9.1, also kernel 2.4.something and used the samba it came with it, 2.2.7 I think, same story.

It all works completely fine copying to ext2 or ext3 partitions from Windows Explorer.

Here's a real kicker, it even works completely fine with no delays from Win98SE copying to a Samba shared vfat partition, IF, I just open a DOS command prompt and do the copy from there. But if I just exit the command prompt and attempt the exact same copy in Windows Explorer, it will fail.

I messed with a multitude of samba options and mount options for the vfat partition all to no avail. I finally think I've homed in on the problem when I set Samba to log level 10, (log level 9 was not sufficient) at which point I discovered its hanging up on a ftruncate command.

When I attempt to copy my 750Meg file to the samba shared vfat partition, the last couple things logged are an fallocate IIRC, and then an ftruncate, the server just sits there 90 seconds or so grinding and grinding then I guess something times out and either Samba drops the connection to the Win98SE client or Win98SE drops it itself, Win98SE gives an error for the copy that the resource is no longer available, but the connection restablishes itself right away then, but the copy attempt has been aborted.

I think its the same type of deal copying smaller files to the Samba shared vfat partition that does work except there is a long pause before the copy really gets going, I think Samba is doing the fallocate deal and it just takes quite a while which is where the pause comes in before the copying gets going, but doesn't take so long that the connection times out.

I imagine this could be easily be reproduced by anyone willing to try as I reproduced it on two very different machines running samba (one is a Pentium II system, the other an AMD Duron) two different versions of Mandrake and 3 different versions of Samba.

I did a bunch of googling and saw that in Samba 2.2.1a some kind of fix was added in for the vfat/fallocate bug, but the impression I got from an archived message on some Samba developers list was that what Samba actually does is go ahead and attempt the fallocate on vfat partitions and if it fails, which apparently it always will on vfat, then Samba does something else instead. More of a workaround than a fix. Maybe thats the best Samba can do and this is really something that should be handled by the kernel/file system people. My hunch is that the problem with the workaround Samba has implemented is that with great big files the time it takes for fallocate to fail on a vfat partition is just to long so the Samba workaround doesn't even kick in, something times out before fallocate can return the failure code and the workaround can kick in.

Comments anyone?

Thankyou,
Tom Schaefer

[Tom Schaefer]

I kept fighting with it and came up with a solution. (I think)

My analysis in the previous posting was somewhat flawed. It isn't that the ftruncate command to the vfat partition is taking to long to fail. Its that after its failed the workaround implemented by Samba is taking to long.

I modified the the source code of vfs-wrap.c, in particular this section:

result = sys_ftruncate(fd, len);
if (result == 0)
goto done;

to

/*result = sys_ftruncate(fd, len);
if (result == 0)*/
result = 0;
goto done;

recompiled smbd, put the new binary in place, restarted samba and lo and behold it worked! Now I can copy my 750Meg test file from Windows98SE Windows Explorer to a samba shared vfat partition and the copy starts immediately and completes successfully after a minute or two.

So, now my question is - What kind of problem(s) if any am I setting myself up for by doing this?

Thankyou,
Tom Schaefer

[Uli Luckas]

Post by Tom Schaefer
result = sys_ftruncate(fd, len);
if (result == 0)
goto done;
to
/*result = sys_ftruncate(fd, len);
if (result == 0)*/
result = 0;
goto done;
recompiled smbd, put the new binary in place, restarted samba and lo and behold it worked! Now I can copy my 750Meg test file from Windows98SE Windows Explorer to a samba shared vfat partition and the copy starts immediately and completes successfully after a minute or two.
So, now my question is - What kind of problem(s) if any am I setting myself up for by doing this?

So here you skip all truncate requests where sys_ftruncate does not
succeede. This does not look like a good idea. I guess there usually is
a reason for a function to be called in the first place.

But let's look at your specific situation. There are two kind's of
truncates. An expandig and a shrinking. From the comments in the source
I take it that VFAT truncates work for shrinking but not for expanding.
So at least you can still shrink files.

When Windows tries to copy a file it seems to expand the destination to
it's final size first. I guess this is just to avoid fragmentation and
to avoid writing 90% of a huge file before failing. So you seem to be
safe in this situation.

An application on the client could choose to ftruncate-expand an empty
file to 4k just to be abel to read 4096 zeros out of it :-) This is
absolutely legal and would fail, as the file would never be
ftruncate-expanded with your modifications.
Honestly, databases do this kind of stuff. And I would not be surprised
if MS-Office apps would use this feature. I can't come up with an
example where bytes beyond the last written byte are ever being read,
but I would not rely on it to never happen.

A much safer Idea would be to skip truncating, returning an error. Maybe
Windows can deal with failing truncates.
Try:

result = sys_ftruncate(fd, len);
/* if (result == 0) */
goto done;

Regards
Uli

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://lists.samba.org/archive/samba/attachments/20030411/35d9eac7/attachment.bin

[Tom Schaefer]

On 11 Apr 2003 17:51:38 +0200

Post by Uli Luckas
A much safer Idea would be to skip truncating, returning an error. Maybe
Windows can deal with failing truncates.
result = sys_ftruncate(fd, len);
/* if (result == 0) */
goto done;

I greatly appreciate your response Uli. Thankyou for taking the time to understand my question and write back with your very meaningful comments.

I would have wrote back to you sooner but I wanted to try out your suggestion first. Your way does work and following your reasoning it does sound like a better work around to my problem so I'm sticking with your way.

Yes you seem to understand the problem completely. But in summary, for anyone else who might ever be interested in this thread:

When Windows 98SE Windows Explorer copies a file to a Samba shared vfat partition it first, creates the file (so I guess a size of 0 bytes at this point), then extends it the same size as the source file, THEN starts copying the source file to it.

vfat on Linux doesn't support extending a file so Samba uses its own alternate method of just writing bytes of zeros one by one to the newly created file until it is the right size and then begins the copying. The problem is, with very large files (hundreds of megabytes) this Samba work around of extending the file can take so long that something (Samba or Windows I'm not sure which side) times out before the actual copying begins and just results in an error on the Win 98SE box.

So, what I've done is circumvented the part where Samba extends the file. I messed with it quite a bit this weekend and haven't noticed this causing any problems.

Perhaps some database application might choke with my workaround in place, but its unlikely I'll ever be accessing a database stored on that Samba share so I can live with that possibility.

Thanks again Uli.

Tom Schaefer

[Gerald (Jerry) Carter]

Post by Tom Schaefer
I kept fighting with it and came up with a solution. (I think)
My analysis in the previous posting was somewhat flawed. It isn't that
the ftruncate command to the vfat partition is taking to long to fail.
Its that after its failed the workaround implemented by Samba is taking
to long.
I modified the the source code of vfs-wrap.c, in particular this
result = sys_ftruncate(fd, len);
if (result == 0)
goto done;
to
/*result = sys_ftruncate(fd, len);
if (result == 0)*/
result = 0;
goto done;
recompiled smbd, put the new binary in place, restarted samba and lo and
behold it worked! Now I can copy my 750Meg test file from Windows98SE
Windows Explorer to a samba shared vfat partition and the copy starts
immediately and completes successfully after a minute or two.
So, now my question is - What kind of problem(s) if any am I setting
myself up for by doing this?

Probably a lot of headaches. You have crippled the all ftruncate()
support in smbd. If a client tries to set the size on a file, it
will fail. Whether the client will care of it, is another matter.
Just be prepared for strange errors.




cheers, jerry
----------------------------------------------------------------------
Hewlett-Packard ------------------------- http://www.hp.com
SAMBA Team ---------------------- http://www.samba.org
GnuPG Key ---- http://www.plainjoe.org/gpg_public.asc
"You can never go home again, Oatman, but I guess you can shop there."
--John Cusack - "Grosse Point Blank" (1997)