develooper Front page | perl.perl5.porters | Postings from October 2005

Re: Inconsistent behaviour when removing files on Cygwin

Thread Previous | Thread Next
Yitzchak Scott-Thoennes
October 27, 2005 01:20
Re: Inconsistent behaviour when removing files on Cygwin
Message ID:
On Thu, Oct 27, 2005 at 09:19:32AM +0200, demerphq wrote:
> On 10/27/05, Yitzchak Scott-Thoennes <> wrote:
> > On Wed, Oct 26, 2005 at 05:28:30PM +0200, S?bastien Aperghis-Tramoni wrote:
> > > Hello,
> > >
> > > While correcting a bug in Net::Pcap Makefile.PL with a Windows developer,
> > > we stumbled upon the strange behaviour of the unlink() function under
> > > Cygwin. It looks like it tries to emulates the Unix behaviour but fails
> > > to achieve it.
> >  ...
> > > Ok, the equivalent program in C produces the same results, so the error is
> > > likely to be inside Cygwin API, but shouldn't this be documented somewhere?
> > > Maybe in perlport/"DOS and Derivatives"?
> >
> > perlport says it short and sweet: (implied: For portable code:) "Don't
> > C<unlink> or C<rename> an open file."  I don't think repeating that in
> > a dosish section would be more helpful.
> >
> > cygwin may defer unlinks until the file is closed or until the process
> > ends.  Exactly what it does depends on the exact flavor of windows and
> > how reliable (cygwin thinks) the relevant win32 api calls are there.
> Actually, afaiui its not a "may" its a "must". The Win32 API doesnt
> support unlink directly.

I'd heard a rumor there was a way of opening a file that allowed
it to be unlinked before close, hence the may.

> The way you delete a file in Win32 is you use CreateFile with the
> DELETE_ON_CLOSE flag. When you close the file it is either marked as
> "Pending Delete" (if other processes also have the file open) or it is
> outright deleted (if no other processes have the file open).
> (See references below, this is better explained in the last posting on
> the last url in my reference list)
> The Win32 C Libraries provide _unlink and _wunlink for compatibility
> purposes, but they just do what is described above.
> Note however this applies ONLY to files opened with FILE_SHARE_DELETE.
> If a file is not opened with this sharing flag then nothing can delete
> or rename the file while it is open.
> Anyway, as i said before the reason the Cygwin and the Win32 version
> behave differently is because Cygwin obviously opens files with
> FILE_SHARE_DELETE allowing the OS to mark the handle as
> "DELETE_ON_CLOSE" via unlink. Whereas Win32 Perl does NOT open files
> with the FILE_SHARE_DELETE flag set in normal circumstances (it does
> internally for such things as stat()).
> The end result is that Cygwin has decided to go with the unixy
> behaviour of "you can delete open files"

To the extent possible, obviously.

> and Win32 Perl has decided to
> go with the Win32 behaviour of "by default you cannot delete open
> files".
> But the truth is that you CAN'T delete open files at all on Win32
> systems, you can only ask the OS to delete them when the refcount on
> the handle falls to zero.
> Im guessing that the bit you say about "until the process ends"
> implies that on Win95/98/ME cygwin somehow simulates the delete on
> close behaviour. Im not sure how as those versions of windows dont
> support FILE_SHARE_DELETE so if some process had the file open there
> is no way to delete the file until it is closed again.

It just uses an atexit-like handler (except it's invoked even on _exit)
so if a different process had the file open, or the file ended up being
deleted and recreated, it doesn't exactly work ideally; there's a comment
in the code that does this:
> Also, there is some further insanity in this picture: if two process
> have a file open with FILE_SHARE_DELETE is possible for one of them to
> delete the file, (and thus have it marked DELETE_PENDING) and then
> have the other one change the flag so it is not DELETE_PENDING so the
> file never actually gets deleted.  The first process thinks the file
> has been deleted correctly, but the second overrides it. There are
> some other traps and zaps here too from what i have seen, so the
> default behaviour of not using FILE_SHARE_DELETE is maybe actually the
> least surprising approach.

Reminds me of the Win32/cygwin difference in handling perl -i with
no suffix specified; cygwin makes a shot at allowing at unix-assuming
code to run, while Win32 forces a change, but has no surprising edge

The cygwin unlink code (for those who wish to delve) is:

extern "C" int
unlink (const char *ourname)
  int res = -1;
  DWORD devn;

  path_conv win32_name (ourname, PC_SYM_NOFOLLOW);

  if (win32_name.error)
      set_errno (win32_name.error);
      goto done;

  if ((devn = win32_name.get_devn ()) == FH_PROC || devn == FH_REGISTRY
      || devn == FH_PROCESS)
      set_errno (EROFS);
      goto done;

  syscall_printf ("_unlink (%s)", win32_name.get_win32 ());

  if (!win32_name.exists ())
      syscall_printf ("unlinking a nonexistent file");
      set_errno (ENOENT);
      goto done;
  else if (win32_name.isdir ())
      syscall_printf ("unlinking a directory");
      set_errno (EPERM);
      goto done;

  bool setattrs;
    setattrs = false;
      /* Allow us to delete even if read-only */
      setattrs = SetFileAttributes (win32_name,
                                    (DWORD) win32_name
                                    & ~(FILE_ATTRIBUTE_READONLY
                                        | FILE_ATTRIBUTE_SYSTEM));
  /* Attempt to use "delete on close" semantics to handle removing
     a file which may be open.

     CV 2004-09-17: Not if the file is on a remote share.  If two processes
     have open handles on a file and one of them calls unlink, then it
     happens that the file is remove from the remote share even though the
     other process still has an open handle.  This other process than gets
     Win32 error 59, ERROR_UNEXP_NET_ERR when trying to access the file.

     For some reason, that does not happen when using DeleteFile, which
     nicely succeeds but still, the file is available for the other process.
     To reproduce, mount /tmp on a remote share and call

       bash -c "cat << EOF"

     Microsoft KB 837665 describes this problem as a bug in 2K3, but I have
     reproduced it on shares on Samba 2.2.8, Samba 3.0.2, NT4SP6, XP64SP1 and
     2K3 and in all cases, DeleteFile works, "delete on close" does not. */
  if (!win32_name.isremote () && wincap.has_delete_on_close ())
      HANDLE h;
      h = CreateFile (win32_name, 0, FILE_SHARE_READ, &sec_none_nih,
                      OPEN_EXISTING, FILE_FLAG_DELETE_ON_CLOSE, 0);
      if (h != INVALID_HANDLE_VALUE)
          if (wincap.has_hard_links () && setattrs)
            SetFileAttributes (win32_name, (DWORD) win32_name);
          BOOL res = CloseHandle (h);
          syscall_printf ("%d = CloseHandle (%p)", res, h);
          if (GetFileAttributes (win32_name) == INVALID_FILE_ATTRIBUTES
              || !win32_name.isremote ())
              syscall_printf ("CreateFile (FILE_FLAG_DELETE_ON_CLOSE) succeeded");
              goto ok;
              syscall_printf ("CreateFile (FILE_FLAG_DELETE_ON_CLOSE) failed");
              if (setattrs)
                SetFileAttributes (win32_name, (DWORD) win32_name & ~(FILE_ATTRIBUTE_READONLY | FILE_ATTRIBUTE_SYSTEM));

  /* Try a delete with attributes reset */
  if (DeleteFile (win32_name))
      syscall_printf ("DeleteFile after CreateFile/CloseHandle succeeded");
      goto ok;

  DWORD lasterr;
  lasterr = GetLastError ();

  SetFileAttributes (win32_name, (DWORD) win32_name);

  /* Windows 9x seems to report ERROR_ACCESS_DENIED rather than sharing
     violation.  So, set lasterr to ERROR_SHARING_VIOLATION in this case
     to simplify tests. */
  if (wincap.access_denied_on_delete () && lasterr == ERROR_ACCESS_DENIED
      && !win32_name.isremote ())

  /* FILE_FLAGS_DELETE_ON_CLOSE was a bust.  If this is a sharing
     violation, then queue the file for deletion when the process
     exits.  Otherwise, punt. */
    goto err;

  syscall_printf ("couldn't delete file, err %d", lasterr);

  /* Add file to the "to be deleted" queue. */
  user_shared->delqueue.queue_file (win32_name);

 /* Success condition. */
  res = 0;
  goto done;

 /* Error condition. */
  __seterrno ();
  res = -1;

  syscall_printf ("%d = unlink (%s)", res, ourname);
  return res;

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About