how to determine if files are on same or different file systems

[Kamus of Kadizhar]

Thanks to everyone on this list. I now have a functioning piece of
python code!! )

Now I'm trying to clean it up.

I have the same (or similar) lines repeated several times:

shutil.copy2(newArrivalsDir+'/'+movie,archivesDir)
thumb = string.replace(movie,'.avi','.jpg')
shutil.copy2(newArrivalsDir+'/tn/'+thumb,archivesDir+'/tn/')

or

os.rename(otherFavDir+'/'+movie,dir+'/'+movie)
thumb = string.replace(movie,'.avi','.jpg')
os.rename(otherFavDir+'/tn/'+thumb,dir+'/tn'+thumb)

what varies is the name of the function (shutil.copy2 or os.rename
depending on if I am renaming or copying) and the names of the source
and dest directories. This particular snippet is repeated about a
half-dozen times.

It would be nice if I could write a function that would determine if the
source and destination are on the same file system or not, and thus use
rename or copy appropriately, or if there is already such a built-in
function.

TIA,

-Kamus

 

[Serge Orlov]

It would be nice if I could write a function that would determine if the
source and destination are on the same file system or not, and thus use
rename or copy appropriately, or if there is already such a built-in
function.

shutil.move ? (Introduced in Python 2.3)

 

[Skip Montanaro]

Kamus> It would be nice if I could write a function that would determine
Kamus> if the source and destination are on the same file system or not,
Kamus> and thus use rename or copy appropriately, or if there is already
Kamus> such a built-in function.

You might want to check out os.path.ismount(). Using it, you can walk up
the two paths toward the root until you hit elements of bot paths where
os.path.ismount() returns True. If the two mount points are the same, the
two original paths are on the same file system. Here's some sample code.
It doesn't help much on my Mac because it essentially has just a single disk
partition. On my web server (Linux w/ multiple partitions), it seems to
work okay:

% df
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hda1 248847 93239 142758 40% /
/dev/hda6 14333167 9393563 4194190 70% /backup
none 385572 0 385572 0% /dev/shm
/dev/hda5 9867356 7702853 1652433 83% /home
/dev/hda7 1981000 1568913 309675 84% /usr
/dev/hda8 248847 120886 115111 52% /var
% python samefs.py
('/', '/usr') 0
('/bin', '/etc') 1
('/etc/passwd', '/var/log') 0
('/Applications', '/Volumes Kisses In The Rain') 0

Skip

import os

def samefs(path1, path2):
if not (os.path.exists(path1) and os.path.exists(path2)):
return False

while path1 != os.path.dirname(path1):
if os.path.ismount(path1):
break
path1 = os.path.dirname(path1)

while path2 != os.path.dirname(path2):
if os.path.ismount(path2):
break
path2 = os.path.dirname(path2)

return path1 == path2

print ("/", "/usr"), samefs("/", "/usr")
print ("/bin", "/etc"), samefs("/bin", "/etc")
print ("/etc/passwd", "/var/log"), samefs("/etc/passwd", "/var/log")
print ("/Applications", "/Volumes Kisses In The Rain"), \
samefs("/Applications", "/Volumes Kisses In The Rain")

 

[Peter Otten]

Thanks to everyone on this list. I now have a functioning piece of
python code!! )

Now I'm trying to clean it up.

I have the same (or similar) lines repeated several times:

shutil.copy2(newArrivalsDir+'/'+movie,archivesDir)
thumb = string.replace(movie,'.avi','.jpg')
shutil.copy2(newArrivalsDir+'/tn/'+thumb,archivesDir+'/tn/')

or

os.rename(otherFavDir+'/'+movie,dir+'/'+movie)
thumb = string.replace(movie,'.avi','.jpg')
os.rename(otherFavDir+'/tn/'+thumb,dir+'/tn'+thumb)

what varies is the name of the function (shutil.copy2 or os.rename
depending on if I am renaming or copying) and the names of the source
and dest directories. This particular snippet is repeated about a
half-dozen times.

Probably time to turn it into a function

def moveMovieAndThumb(fromDir, toDir, movieName):
# your code

or similar.

It would be nice if I could write a function that would determine if the
source and destination are on the same file system or not, and thus use
rename or copy appropriately, or if there is already such a built-in
function.

No need to determine it beforehand, just try (untested):

def movefile(src, dst):
try:
os.rename(src, dst)
except OSError:
shutil.copy2(src, dst)
os.remove(src)


Of course, if you are using Python 2.3 you should use shutil.move() as
pointed out by Serge Orlov; the above was mostly posted to illustrate the
popular concept "It's easier to ask forgiveness than permission", i. e.
with Python's powerful exception handling mechanism you need not fear the
failure of a particular code snippet, as long as you provide the
appropriate error handling.

Random remarks:
- You might take a look at os.path.join()
- Ok, this is paranoia, but I would ensure that ".avi" is only at the end of
the string
- Use methods of the str object rather than functions in the string module,
e. g. "abc".replace("a", "d") rather than string.replace("abc", "a", "d")

Peter

 

[Kamus of Kadizhar]

No need to determine it beforehand, just try (untested):

def movefile(src, dst):
try:
os.rename(src, dst)
except OSError:
shutil.copy2(src, dst)
os.remove(src)


Of course, if you are using Python 2.3 you should use shutil.move() as
pointed out by Serge Orlov; the above was mostly posted to illustrate the
popular concept "It's easier to ask forgiveness than permission", i. e.
with Python's powerful exception handling mechanism you need not fear the
failure of a particular code snippet, as long as you provide the
appropriate error handling.

Peter:

I am copying 600mb - 1.2gb files over slow wireless to an NFS mount -
not exactly the model of reliability. It takes 40 minutes to an hour to
copy one of these. I am paranoid of failures - I don't want to lose movies.

So, I don't want to rely on innate error handling, because the failure
could come from wireless failure, and the copy could fail as well.
That's why I'm looking for a way to tell if a particular partition is
network mounted.

The basic idea is that movies get moved on the same partition, but
copied (and not deleted) when moving between network mounted partitions
and local partitions. Deletes will be subject to manual review until I
get a comfort factor.

Random remarks:
- You might take a look at os.path.join()
- Ok, this is paranoia, but I would ensure that ".avi" is only at the end of
the string

".avi$" ? I still haven't figured out Python's string handling really
well. Or use the length of the string and count backwards? I admit, I
got lazy on this one.

- Use methods of the str object rather than functions in the string module,
e. g. "abc".replace("a", "d") rather than string.replace("abc", "a", "d")

OK, I'll try that.

-Kamus

 

[Peter Otten]

Peter:

I am copying 600mb - 1.2gb files over slow wireless to an NFS mount -
not exactly the model of reliability. It takes 40 minutes to an hour to
copy one of these. I am paranoid of failures - I don't want to lose
movies.

So, I don't want to rely on innate error handling, because the failure
could come from wireless failure, and the copy could fail as well.
That's why I'm looking for a way to tell if a particular partition is
network mounted.

The basic idea is that movies get moved on the same partition, but
copied (and not deleted) when moving between network mounted partitions
and local partitions. Deletes will be subject to manual review until I
get a comfort factor.

I still think that EAFP

try:
os.rename(src, dst)
except OSError:
# any error here will still result in a traceback
# if not handled in client code
shutil.copy2(src, dst)
# os.remove() ommitted intentionally

is superior style to LBYL (look before you leap)

if isSamePartition(src, dst):
os.rename(src, dst)
else:
shutil.copy2(src, dst)

and fail to see the increased danger for your data. Plus, it saves you from
writing the nonexistent isSamePartition()
Maybe you could replace the copy2() call with the invocation of some utility
that reliably copies over an unreliable connection (maybe can someone else
come up with a name), or you take the extra amount of work/time and compare
the two files after you have copied them.

".avi$" ? I still haven't figured out Python's string handling really
well. Or use the length of the string and count backwards? I admit, I
got lazy on this one.

Use regular expressions only as a last resort.
('/dir/name', '.ext')

Putting it into a function:
.... return os.path.splitext(path)[0] + newext
....'/sowhat/movie.jpg'

I you want to do it only with string methods:

path = "/one/two.three"
path[ath.rindex(".")] + ".four"

'/one/two.four'

However, you would have to ensure that the dot position is after the last
slash.

For a quick overview what is available for strings:
['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__',
'__ge__', '__getattribute__', '__getitem__', '__getnewargs__',
'__getslice__', '__gt__', '__hash__', '__init__', '__le__', '__len__',
'__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__',
'__str__', 'capitalize', 'center', 'count', 'decode', 'encode', 'endswith',
'expandtabs', 'find', 'index', 'isalnum', 'isalpha', 'isdigit', 'islower',
'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip',
'replace', 'rfind', 'rindex', 'rjust', 'rstrip', 'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

Peter

 

[Jeff Epler]

You can find the device a file is on using the .st_dev field of its stat
information:
>>> os.stat("/").st_dev
see the stat manpage for more details.

You can find the magic number for a particular filesystem using statfs(2),
but as far as I can tell this is not exposed in Python. If it were,
you'd write something like
>>> os.statfs("/").f_type
to get the type of the filesystem where / resides. Some f_type values
are mentioned in my statfs(2) manpage, including
NFS_SUPER_MAGIC 0x6969
os.statvfs() seems to be unrelated, though my system lacks a manpage for
it.

Jeff

 

[Christos TZOTZIOY Georgiou]

It would be nice if I could write a function that would determine if the
source and destination are on the same file system or not, and thus use
rename or copy appropriately, or if there is already such a built-in
function.

The os.stat call result has a st_dev (the device) field that could be
very useful to you. It works for Windows too, where you can use extra
calls from win32api or win32file, can't remember which to know the file
system type too.