in need of some sorting help

[=?iso-8859-1?B?aWFuYXLp?=]

Hey all,

if i use a os.walk() to append files to a list like so...

files = []
root = self.path.GetValue() # wx.TextCtrl input
filter = self.fileType.GetValue().lower() # wx.TextCtrl input
not_type = self.not_type.GetValue() # wx.CheckBox input

for base, dirs, walk_files in os.walk(root):
main.Update()
# i only need the part of the filename after the
user selected path:
base = base.replace(root,"")

for entry in walk_files:
entry = os.path.join(base,entry)
if filter != "":
if filter in entry.lower() and not
not_type:
files.append(entry)
if filter not in entry.lower() and
not_type:
files.append(entry)
else:
files.append(entry)

.... will it sort properly on mac and *nix? if not, is there a tried an
true sorting method someone could graciously let me know of?
oh by sort properly i mean:
file1.ext
file2.ext
file3.ext
file4.ext
zzfile.ext
folder1\file1.ext
folder1\file2.ext
folder1\file3.ext
folder2\file1.ext
folder2\file2.ext
folder2\file3.ext


something tells me it's probably better to do my own sorting, just in
case, so i tried:

files.sort(key=lambda x: x.lower())

but that didn't work, made them out of order.

TIA

 

[Kent Johnson]

Hey all,

if i use a os.walk() to append files to a list like so...

files = []
root = self.path.GetValue() # wx.TextCtrl input
filter = self.fileType.GetValue().lower() # wx.TextCtrl input
not_type = self.not_type.GetValue() # wx.CheckBox input

for base, dirs, walk_files in os.walk(root):
main.Update()
# i only need the part of the filename after the
user selected path:
base = base.replace(root,"")

for entry in walk_files:
entry = os.path.join(base,entry)
if filter != "":
if filter in entry.lower() and not
not_type:
files.append(entry)
if filter not in entry.lower() and
not_type:
files.append(entry)
else:
files.append(entry)

... will it sort properly on mac and *nix? if not, is there a tried an
true sorting method someone could graciously let me know of?

The lists of files and directories yielded by os.walk() will be in the
order that the OS returns them from os.listdir(). According to the docs
this list is in "arbitrary" order.

You can sort the lists yourself. If you modify dirs in place it will
affect the subsequent walk. So you could use
for base, dirs, walk_files in os.walk(root):
dirs.sort(key=str.lower) # note no need for lambda
walk_files.sort(key=str.lower)
# etc

Kent

 

[Peter Otten]

dirs.sort(key=str.lower) # note no need for lambda

However, this will raise a TypeError for unicode directory names while the
lambda continues to work.

Peter

 

[=?iso-8859-1?B?aWFuYXLp?=]

thank you for the help, i didn't know you could sort in place like
that. definitly will come in handy.
However, i need the sorting done after the walk, due to the way the
application works... should have specified that, sorry.

TIA

 

[Ben Cartwright]

However, i need the sorting done after the walk, due to the way the
application works... should have specified that, sorry.

If your desired output is just a sorted list of files, there is no good
reason that you shouldn't be able sort in place. Unless your app is
doing something extremely funky, in which case this should do it:

root = self.path.GetValue() # wx.TextCtrl input
filter = self.fileType.GetValue().lower() # wx.TextCtrl input
not_type = self.not_type.GetValue() # wx.CheckBox input

matched_paths = {}
for base, dirs, walk_files in os.walk(root):
main.Update()
# i only need the part of the filename after the
# user selected path:
base = base.replace(root, '')

matched_paths[base] = []
for entry in walk_files:
entry = os.path.join(base, entry)
if not filter:
match = True
else:
match = filter in entry.lower()
if not_type:
match = not match
if match:
matched_paths[base].append(entry)

def tolower(x): return x.lower()
files = []
# Combine into flat list, first sorting on base path, then full
path
for base in sorted(matched_paths, key=tolower):
files.extend(sorted(matched_paths[base], key=tolower))

--Ben

 

[=?iso-8859-1?B?aWFuYXLp?=]

arrrg i did it again, not enough explanation... new to asking for
programing help online.
anyway the reason is that the list can be rearanged later in the
program by another function, and i need a way to put it in order
again.. so yes it is doing some pretty funky stuff, lol.
so although your method works well, it would have been problematic for
me to implement a two list system in the app for various reasons. but
you did make me understand a way to sort this thing finally: sort by
base path then by full path, which is how i came up with:

files.sort(key=lambda x: x.lower())
files.sort(key=lambda x: os.path.dirname(x))

well actually i am sorting first by full name, then by base, taking
advantage of python2.4's stable sort().

If you can think of a more efficient (faster) way of doing this please
let me know. i'm not crazy about having to sort this list twice, it can
get pretty big (50,000 entries)

anyway thanks all, this thing had me stuck for over a month !!

 

[Kent Johnson]

base path then by full path, which is how i came up with:

files.sort(key=lambda x: x.lower())
files.sort(key=lambda x: os.path.dirname(x))

well actually i am sorting first by full name, then by base, taking
advantage of python2.4's stable sort().

If you can think of a more efficient (faster) way of doing this please
let me know. i'm not crazy about having to sort this list twice, it can
get pretty big (50,000 entries)

Use a key function that returns a tuple of the two values you want to
sort on:
def make_key(f):
return (os.path.dirname(f), f.lower())

files.sort(key=make_key)

Kent

 

[=?iso-8859-1?B?aWFuYXLp?=]

sweet that works great!
thanks again for all the help.

 

[Scott David Daniels]

....
files.sort(key=lambda x: x.lower())
files.sort(key=lambda x: os.path.dirname(x))

This is exactly why some of us hate lambda. It encourages
long-way-around thinking.
files.sort(key=lambda x: os.path.dirname(x))

is better written as:
files.sort(key=os.path.dirname)

Learn to have your skin itch when you see:

... lambda name:<expr>(name)

And yes, I concede that it _is_ useful in your .lower expression.
It's not that lambda is bad, rather that it seems to encourages
this 'lambda x:f(x)' stuff.

--Scott David Daniels
(e-mail address removed)