rm -rf doesn't work with current installwatch on XFS (Bug #94)

Added by Vlad Glagolev almost 2 years ago. Updated almost 2 years ago.

Status:Closed Start date:08/26/2011
Priority:High Due date:
Assignee:- % Done:

100%
Category:-
Target version:-
Grimoire:

Description

This might be related to #37,

but 'rm -rf ...' breaks if no castfs is installed.
for example mysql spell is broken when tests are disabled:

if [[ $MYSQL_TESTS == n ]]; then
rm -rf "${INSTALL_ROOT}/usr/share/mysql/mysql-test"
fi

fails with:

make[1]: Leaving directory `/usr/src/mysql-5.1.58/win'
rm: cannot remove `/usr/share/mysql/mysql-test/lib/My/SafeProcess': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/lib/My/File': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/lib/v1/My': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/std_data/parts': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/std_data/ndb_backup50': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/std_data/ndb_backup51': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/std_data/ndb_backup51_data_be': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/std_data/ndb_backup51_data_le': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/std_data/funcs_1': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/collections': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/extra': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/binlog': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/federated': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_1/bitdata': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_1/include': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_1/lib': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_1/views': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_1/cursors': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_1/datadict': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_1/storedproc': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_1/triggers': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_2/charset': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_2/data': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_2/include': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_2/lib': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_2/r': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/funcs_2/t': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/jp/t': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/jp/r': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/jp/std_data': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/jp/include': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/manual': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/ndb_team': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/rpl': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/stress': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/ndb': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/rpl_ndb': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/parts': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/innodb': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/innodb_plugin': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/engines/funcs': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/engines/iuds': Directory not empty
rm: cannot remove `/usr/share/mysql/mysql-test/suite/engines/rr_trx': Directory not empty
 ! Problem Detected ! 
INSTALL failed!
Creating compile log /var/log/sorcery/compile/mysql-5.1.58.bz2

smgl-test.tgz - SMGL test cases spell (645 Bytes) Vlad Glagolev, 08/28/2011 08:17 am

strace.castfs.log.gz - strace log of rm with castfs installed (9.6 kB) Vlad Glagolev, 08/28/2011 08:17 am

strace.installwatch.log.gz - strace log of rm without castfs installed (20.8 kB) Vlad Glagolev, 08/28/2011 08:17 am

log.log.gz (194.9 kB) Vlad Glagolev, 08/28/2011 09:30 am

Associated revisions

Revision 2f8dc141
Added by Remko van der Vossen almost 2 years ago

installwatch: patch a memory handling bug

Relative path handling in installwatch contained a bug in which the
directory size was used in a malloc for the expanded path string,
instead of a saner PATH_MAX value. (fixes #94)

Revision 9dee415c
Added by Remko van der Vossen almost 2 years ago

installwatch: patch a memory handling bug

Relative path handling in installwatch contained a bug in which the
directory size was used in a malloc for the expanded path string,
instead of a saner PATH_MAX value. (fixes #94)
(cherry picked from commit 2f8dc141adb19a4e27e4c6dd9227ac4a2e89b6c2)

History

#1
Updated by Vlad Glagolev almost 2 years ago

  • Priority changed from Normal to Urgent
  • Description changed from This might be related to #37, but 'rm -rf ...' breaks if no castfs i... to This might be related to #37, but 'rm -rf ...' breaks if no castfs i... More

#2
Updated by Vlad Glagolev almost 2 years ago

The problem appeared when we updated installwatch to the current (git-based) version from 0.6.3 (which was forced by recent coreutils).

a little history:
http://bugs.sourcemage.org/show_bug.cgi?id=13873
http://bugs.sourcemage.org/show_bug.cgi?id=15559

  • Description changed from This might be related to #37, but 'rm -rf ...' breaks if no castfs i... to This might be related to #37, but 'rm -rf ...' breaks if no castfs i... More

#3
Updated by Jaka Kranjc almost 2 years ago

works for me. It's the first install of mysql, test grimoire chroot without castfs.

#4
Updated by Vlad Glagolev almost 2 years ago

what's your kernel version and filesystem? maybe it's some extra attrs issue?

#5
Updated by Jaka Kranjc almost 2 years ago

2.6.38 and ext4.

#6
Updated by Vlad Glagolev almost 2 years ago

.27, .35 and xfs everywhere where I met the issue, hmm..

#7
Updated by Vlad Glagolev almost 2 years ago

added strace logs, and spell mould to test.

it simply copies /var/log to /usr/test and then removes sorcery dir in it.

with no castfs it simply fails with this:

rm: cannot remove `/usr/test/log/sorcery/install': Directory not empty
rm: cannot remove `/usr/test/log/sorcery/md5sum': Directory not empty
rm: cannot remove `/usr/test/log/sorcery/queue': Directory not empty
rm: cannot remove `/usr/test/log/sorcery/possessed': Directory not empty
 ! Problem Detected ! 
INSTALL failed!

#8
Updated by Vlad Glagolev almost 2 years ago

by the way, I did a few casts, and every time it left just for 1 file in every subdir:

:/usr/test/log/sorcery# find . -type f
./possessed/shadow-4.1.4.3
./install/g++-4.6.1
./md5sum/g++-4.6.1
./queue/install

so rmdir of course is unable to remove non-empty dirs.

#9
Updated by Vlad Glagolev almost 2 years ago

looks like a weird memory bug..

from strace.installwatch.log:

stat("/usr/test/log/sorcery/posse/shadow-4.1.4.3", 0x7fff916ffa30) = -1 ENOENT (No such file or directory)
lstat("/usr/test/log/sorcery/posse/shadow-4.1.4.3", 0x206cea0) = -1 ENOENT (No such file or directory)
unlink("/usr/test/log/sorcery/posse/shadow-4.1.4.3") = -1 ENOENT (No such file or directory)
stat("/usr/test/log/sorcery/g++-4.6.1", 0x7fff916ffa30) = -1 ENOENT (No such file or directory)
lstat("/usr/test/log/sorcery/g++-4.6.1", 0x2074000) = -1 ENOENT (No such file or directory)
unlink("/usr/test/log/sorcery/g++-4.6.1") = -1 ENOENT (No such file or directory)
stat("/usr/test/log/sorcery/g++-4.6.1", 0x7fff916ffa30) = -1 ENOENT (No such file or directory)
lstat("/usr/test/log/sorcery/g++-4.6.1", 0x2073dc0) = -1 ENOENT (No such file or directory)
unlink("/usr/test/log/sorcery/g++-4.6.1") = -1 ENOENT (No such file or directory)

#10
Updated by Vlad Glagolev almost 2 years ago

even more.

once you set installwatch debugging features with:

export INSTW_DBGLVL=5
export INSTW_DBGFILE=/tmp/log.log

you'll see that broken paths start with instw_setpath(), at least for shadow log file:

__fxstatat(5,shadow-4.1.4.3,0x14dbea0,0400)
instw_setpathrel(0x7fff7b05e190,5,shadow-4.1.4.3)
instw_setpath(0x7fff7b05e190,/usr/test/log/sorcery/posse/shadow-4.1.4.3)
instw(0x7fff7b05e190) {
        gstatus     : 3
        error       : 0
        status      : 0
        dbglvl      : 4
        pid         : 19719
        root        : "(null)" 
        backup      : "(null)" 
        transl      : "(null)" 
        meta        : "(null)" 
        mtransl     : "(null)" 
        mdirls      : "(null)" 
        path        : "/usr/test/log/sorcery/posse/shadow-4.1.4.3" 
        reslvpath   : "/usr/test/log/sorcery/posse/shadow-4.1.4.3" 
        truepath    : "/usr/test/log/sorcery/posse/shadow-4.1.4.3" 
        translpath  : "/usr/test/log/sorcery/posse/shadow-4.1.4.3" 
        mtranslpath : "" 
        mdirlspath  : "" 
}

full installwatch debug log is attached.

#11
Updated by Vlad Glagolev almost 2 years ago

btw, for the clear tests, installwatch must be compiled against glibc >2.10

#12
Updated by Vlad Glagolev almost 2 years ago

so now it's either an XFS bug (which, I'm sure, it's not) or installwatch bug about some specific FS' options/configuration.

I tried this:

  1. dd if=/dev/zero of=disk-image count=409600
  2. mkfs -t ext3 -q disk-image
  3. mount -o loop=/dev/loop0 disk-image /usr/test

and voi la! It worked flawlessly..

this is all tested on 64bit box.

#13
Updated by Vlad Glagolev almost 2 years ago

same problem on 32-bit box.

installwatch just randomly cuts the paths:

stat64("/usr/test/log/sorcery/posse/openssl-0.9.8r", 0xbffe8f1c) = -1 ENOENT (No such file or directory)
lstat64("/usr/test/log/sorcery/posse/openssl-0.9.8r", 0x805bed0) = -1 ENOENT (No such file or directory)

#14
Updated by Vlad Glagolev almost 2 years ago

I'm sure there was no such problem with previous release (0.6.3), but heh:

current installwatch.c ~6 times bigger than 0.6.3 one..

  • Priority changed from Urgent to High
  • Subject changed from rm -rf doesn't work with current installwatch to rm -rf doesn't work with current installwatch on XFS
  • Status changed from New to In Progress

#15
Updated by Remko van der Vossen almost 2 years ago

  • % Done changed from 0 to 100
  • Status changed from In Progress to Resolved

#16
Updated by Vlad Glagolev almost 2 years ago

wich: "directory entry files are bigger on other file systems".

hehe, my fix appeared more memory-effective, but finally we considered it as suspicious and more dangerous cause "rm -rf uses unlinkat so that it can handle deep paths, including those which are actually longer than PATH_MAX, this expanding of paths by installwatch ruins that" © wich

thanks a lot for fixing this.

#17
Updated by Vlad Glagolev almost 2 years ago

closing the bug.

  • Status changed from Resolved to Closed

Also available in: Atom PDF