[Windows 7, Python 2.6] Can't write to a directory made w/ os.makedirs

O

OlyDLG

Hi! I'm working on a script utilizing os.makedirs to make directories
to which I'm then trying to write files created by exe's spawned w/
subprocess.call; I'm developing in Stani's Python Editor, debugging
using Winpdb. I've gotten to the point where
subprocess.Popen._execute_child is raising a WindowsError(5,'Access is
denied') exception. I've tried: setting the mode in the makedirs
call; using os.chmod(<dir>, stat.S_IWRITE); and even setting a
breakpoint before the subprocess.call and unsetting the read-only
attribute of the pertinent directory tree using the Windows directory
Properties dialog--which doesn't "take," if that's a clue--all to no
avail. I suspect that maybe it has to do w/ not running Winpdb as
administrator, but I'm not sure how to do that when starting Winpdb
from w/in SPE, and in any event, it doesn't seem reasonable that
there's no way to get this to work w/in the script itself. Please
help!
 
T

Tim Golden

Hi! I'm working on a script utilizing os.makedirs to make directories
to which I'm then trying to write files created by exe's spawned w/
subprocess.call; I'm developing in Stani's Python Editor, debugging
using Winpdb. I've gotten to the point where
subprocess.Popen._execute_child is raising a WindowsError(5,'Access is
denied') exception. I've tried: setting the mode in the makedirs
call; using os.chmod(<dir>, stat.S_IWRITE); and even setting a
breakpoint before the subprocess.call and unsetting the read-only
attribute of the pertinent directory tree using the Windows directory
Properties dialog--which doesn't "take," if that's a clue--all to no
avail.
Click to expand...

Whatever you're trying to do, resetting the read-only bit
on the Windows directory is not the answer. It has, bizarrely,
nothing to do with the read/write-ness of a directory; rather,
it reflects the system-ness of a folder.

There's no general answer I can see to give, without
knowing what the security is on your folders or what
other thing might impede your processes from writing
files in them. Is it definite that the ERROR_ACCESS_DENIED
is in fact the result of attempting to write a file into
one of the newly-created directories?

Can you do something like this (adapt for your own environment):

<code>
import os, sys
import subprocess

os.makedirs ("c:/temp/a/b/c")

with open ("c:/temp/a/b/c/test1.txt", "w"):
pass

subprocess.call ([
sys.executable,
"-c",
"open ('c:/temp/a/b/c/test2.txt', 'w').close ()"
])

</code>

ie can the Python process creating the directories,
and a subprocess called from it create a simple file?

Depending on where you are in the filesystem, it may indeed
be necessary to be running as administrator. But don't try
to crack every security nut with an elevated sledgehammer.


TJG
 
D

David Goldsmith

On 31/12/2011 22:13, OlyDLG wrote:
 > Hi!  I'm working on a script utilizing os.makedirs to make directories
 > to which I'm then trying to write files created by exe's spawned w/
 > subprocess.call; I'm developing in Stani's Python Editor, debugging
 > using Winpdb.  I've gotten to the point where
 > subprocess.Popen._execute_child is raising a WindowsError(5,'Access is
 > denied') exception.  I've tried: setting the mode in the makedirs
 > call; using os.chmod(<dir>, stat.S_IWRITE); and even setting a
 > breakpoint before the subprocess.call and unsetting the read-only
 > attribute of the pertinent directory tree using the Windows directory
 > Properties dialog--which doesn't "take," if that's a clue--all to no
 > avail.

Whatever you're trying to do, resetting the read-only bit
on the Windows directory is not the answer. It has, bizarrely,
nothing to do with the read/write-ness of a directory; rather,
it reflects the system-ness of a folder.

There's no general answer I can see to give, without
knowing what the security is on your folders or what
other thing might impede your processes from writing
files in them. Is it definite that the ERROR_ACCESS_DENIED
is in fact the result of attempting to write a file into
one of the newly-created directories?

Can you do something like this (adapt for your own environment):

<code>
import os, sys
import subprocess

os.makedirs ("c:/temp/a/b/c")

with open ("c:/temp/a/b/c/test1.txt", "w"):
   pass

subprocess.call ([
   sys.executable,
   "-c",
   "open ('c:/temp/a/b/c/test2.txt', 'w').close ()"
])

</code>

ie can the Python process creating the directories,
Yes.

and a subprocess called from it create a simple file?
No.

Depending on where you are in the filesystem, it may indeed
be necessary to be running as administrator. But don't try
to crack every security nut with an elevated sledgehammer.
Click to expand...

If you mean running as admin., those were my sentiments exactly. So,
there isn't something specific I should be doing to assure that my
subproceses can write to directories?
 
T

Tim Golden

If you mean running as admin., those were my sentiments exactly. So,
there isn't something specific I should be doing to assure that my
subproceses can write to directories?
Click to expand...


In the general case, no. By default, a subprocess will have
the same security context as its parent. The exception is
where the parent (the Python processing invoking subprocess.call
in this example) is already impersonating a different user;
in that case, the subprocess will inherit its grandparent's
context.

But unless you're doing something very deliberate here then
I doubt if that's biting you.

Can I ask: are you absolutely certain that the processes
you're calling are doing what you think they are and failing
where you think they're failing?

TJG
 
D

David Goldsmith

On 01/01/2012 12:05, David Goldsmith wrote:
 >> ie can the Python process creating the directories,
 >
 > Yes.
 >
 >> and a subprocess called from it create a simple file?
 >
 > No.
 >
 >> Depending on where you are in the filesystem, it may indeed
 >> be necessary to be running as administrator. But don't try
 >> to crack every security nut with an elevated sledgehammer.
 >
 > If you mean running as admin., those were my sentiments exactly.  So,
 > there isn't something specific I should be doing to assure that my
 > subproceses can write to directories?

In the general case, no. By default, a subprocess will have
the same security context as its parent. The exception is
where the parent (the Python processing invoking subprocess.call
in this example) is already impersonating a different user;
in that case, the subprocess will inherit its grandparent's
context.

But unless you're doing something very deliberate here then
I doubt if that's biting you.

Can I ask: are you absolutely certain that the processes
you're calling are doing what you think they are and failing
where you think they're failing?

TJG
Click to expand...

I'm a mathematician: the only thing I'm absolutely certain of is
nothing.

Here's my script, in case that helps:

import os
import sys
import stat
import os.path as op
import subprocess as sub
from os import remove
from os import listdir as ls
from os import makedirs as mkdir

def doFlac2Mp3(arg, d, fl):
if '.flac' in [f[-5:] for f in fl]:
newD = d.replace('FLACS', 'MP3s')
mkdir(newD)
for f in fl:
if f[-5:]=='.flac':
root = f.replace('.flac', '')
cmd = ['"C:\\Program Files (x86)\\aTunes\\win_tools\
\flac.exe" -d ' +
'--output-prefix=' + newD + '\\', f]
res = sub.call(cmd)#, env={'PATH': os.defpath})
if not res:
cmd = ['"C:\\Program Files (x86)\\aTunes\\win_tools
\\lame.exe" -h',
newD + root + '.wav', newD + root +
'.mp3']
res = sub.call(cmd)#, env={'PATH': os.defpath})
if not res:
rf = newD + root + '.wav'
remove(rf)

top=sys.argv[1]
op.walk(top, doFlac2Mp3, None)
 
M

MRAB

In the general case, no. By default, a subprocess will have
the same security context as its parent. The exception is
where the parent (the Python processing invoking subprocess.call
in this example) is already impersonating a different user;
in that case, the subprocess will inherit its grandparent's
context.

But unless you're doing something very deliberate here then
I doubt if that's biting you.

Can I ask: are you absolutely certain that the processes
you're calling are doing what you think they are and failing
where you think they're failing?

TJG
Click to expand...

I'm a mathematician: the only thing I'm absolutely certain of is
nothing.

Here's my script, in case that helps:

import os
import sys
import stat
import os.path as op
import subprocess as sub
from os import remove
from os import listdir as ls
from os import makedirs as mkdir

def doFlac2Mp3(arg, d, fl):
if '.flac' in [f[-5:] for f in fl]:
newD = d.replace('FLACS', 'MP3s')
mkdir(newD)
for f in fl:
if f[-5:]=='.flac':
root = f.replace('.flac', '')
cmd = ['"C:\\Program Files (x86)\\aTunes\\win_tools\
\flac.exe" -d ' +
'--output-prefix=' + newD + '\\', f]
res = sub.call(cmd)#, env={'PATH': os.defpath})
if not res:
cmd = ['"C:\\Program Files (x86)\\aTunes\\win_tools
\\lame.exe" -h',
newD + root + '.wav', newD + root +
'.mp3']
res = sub.call(cmd)#, env={'PATH': os.defpath})
if not res:
rf = newD + root + '.wav'
remove(rf)

top=sys.argv[1]
op.walk(top, doFlac2Mp3, None)
Click to expand...

I think that if the command line should be something like:

"C:\Program Files (x86)\aTunes\win_tools\flac.exe" -d
--output-prefix=FOO\ BAR

then the cmd should be something like:

cmd = ['C:\\Program Files (x86)\\aTunes\\win_tools\\flac.exe', '-d',
'--output-prefix="' + newD + '\\"', f]
 
T

Tim Golden

Here's my script, in case that helps:
Click to expand...

It certainly does. A few things occur to me.

First, you
shouldn't need to double-quote the path; the subprocess.call
should do that for you as long as you're using the list
version of the param -- which you are.

Second, you almost certainly don't want to be using the
env param, at least not in the way you are. Depending on
the way in which your app runs, either pass the appropriate
directory as the cwd= param, or copy and override the
current environ dict, ie either do this:

subprocess.call (
['c:/program files/somewhere/app.exe', 'blah1', 'blah2'],
cwd="c:/somewhere/else"
)

or this:

env = dict (os.environ)
env['PATH'] = "c:/somewhere/else"
# or env['PATH'] += ";c:/somewhere/else"
subprocess.call (
['c:/program files/somewhere/app.exe', 'blah1', 'blah2'],
env=env
)


See if any of that helps

TJG
 
D

Dave Angel

In the general case, no. By default, a subprocess will have
the same security context as its parent. The exception is
where the parent (the Python processing invoking subprocess.call
in this example) is already impersonating a different user;
in that case, the subprocess will inherit its grandparent's
context.

But unless you're doing something very deliberate here then
I doubt if that's biting you.

Can I ask: are you absolutely certain that the processes
you're calling are doing what you think they are and failing
where you think they're failing?

TJG
Click to expand...
I'm a mathematician: the only thing I'm absolutely certain of is
nothing.

Here's my script, in case that helps:

import os
import sys
import stat
import os.path as op
import subprocess as sub
from os import remove
from os import listdir as ls
from os import makedirs as mkdir

def doFlac2Mp3(arg, d, fl):
if '.flac' in [f[-5:] for f in fl]:
newD = d.replace('FLACS', 'MP3s')
mkdir(newD)
for f in fl:
if f[-5:]=='.flac':
root = f.replace('.flac', '')
cmd = ['"C:\\Program Files (x86)\\aTunes\\win_tools\
\flac.exe" -d ' +
'--output-prefix=' + newD + '\\', f]
res = sub.call(cmd)#, env={'PATH': os.defpath})
if not res:
cmd = ['"C:\\Program Files (x86)\\aTunes\\win_tools
\\lame.exe" -h',
newD + root + '.wav', newD + root +
'.mp3']
res = sub.call(cmd)#, env={'PATH': os.defpath})
if not res:
rf = newD + root + '.wav'
remove(rf)

top=sys.argv[1]
op.walk(top, doFlac2Mp3, None)
Click to expand...
The line cmd= is bogus. You're trying to run a program with a -h after
the filename. The reason you're passing a list to sub.call is to
separate the parameters from the program name, not to mention with
quotes in its name. So the -h has to be a separate list item.
(Although Windows will probably handle it correctly if you combine the
-h with the FOLLOWING argument, it's still bad practice]

cmd = [ "c:\\program files .... \\lame.exe", "-h", newD + root + ....."
 
D

David Goldsmith

I'm a mathematician: the only thing I'm absolutely certain of is
nothing.
Click to expand...
Here's my script, in case that helps:
Click to expand...
import os
import sys
import stat
import os.path as op
import subprocess as sub
from os import remove
from os import listdir as ls
from os import makedirs as mkdir
Click to expand...
def doFlac2Mp3(arg, d, fl):
     if '.flac' in [f[-5:] for f in fl]:
         newD = d.replace('FLACS', 'MP3s')
         mkdir(newD)
         for f in fl:
             if f[-5:]=='.flac':
                 root = f.replace('.flac', '')
                 cmd = ['"C:\\Program Files (x86)\\aTunes\\win_tools\
\flac.exe" -d ' +
                        '--output-prefix=' + newD + '\\', f]
                 res = sub.call(cmd)#, env={'PATH': os.defpath})
                 if not res:
                     cmd = ['"C:\\Program Files(x86)\\aTunes\\win_tools
\\lame.exe" -h',
                             newD + root + '.wav',  newD + root +
'.mp3']
                     res = sub.call(cmd)#, env={'PATH': os.defpath})
                     if not res:
                         rf = newD + root +'.wav'
                         remove(rf)
Click to expand...
top=sys.argv[1]
op.walk(top, doFlac2Mp3, None)
Click to expand...

The line cmd= is bogus.  You're trying to run a program with a -h after
the filename.  The reason you're passing a list to sub.call is to
separate the parameters from the program name, not to mention with
quotes in its name.  So the -h has to be  a separate list item.
(Although Windows will probably handle it correctly if you combine the
-h with the FOLLOWING argument, it's still bad practice]

cmd = [ "c:\\program files .... \\lame.exe", "-h", newD + root + ....."
Click to expand...

Thanks, this is the kind of feedback I was hoping for--the
documentation for subprocess.call is notably "sparing" to put it
politely. However, having not yet tried it, I'm skeptical, as what
doc there is says:

"subprocess.call(args, *, stdin=None, stdout=None, stderr=None,
shell=False)

Run the command described by args. Wait for command to complete,
then return the returncode attribute.

The arguments shown above are merely the most common ones,
described below in Frequently Used Arguments (hence the slightly odd
notation in the abbreviated signature). The full function signature is
the same as that of the Popen constructor - this functions passes all
supplied arguments directly through to that interface.
:
:
"class subprocess.Popen(args, bufsize=0, executable=None, stdin=None,
stdout=None, stderr=None, preexec_fn=None, close_fds=False,
shell=False, cwd=None, env=None, universal_newlines=False,
startupinfo=None, creationflags=0)

Arguments are:

args should be a string, or a sequence of program arguments...On
Windows: the Popen class uses CreateProcess() to execute the child
child program, which operates on strings. If args is a sequence, it
will be converted to a string in a manner described in Converting an
argument sequence to a string on Windows." Thus I didn't really see
the point of separating out the -h string--which tells lame to use
high quality encoding--into another sequence element, but as per your
advice, I'll try it (along with Tim's and MRAB's suggestions; I'll
report back what works).
 
D

David Goldsmith

I'm a mathematician: the only thing I'm absolutely certain of is
nothing.
Click to expand...
Here's my script, in case that helps:
Click to expand...
import os
import sys
import stat
import os.path as op
import subprocess as sub
from os import remove
from os import listdir as ls
from os import makedirs as mkdir
Click to expand...
def doFlac2Mp3(arg, d, fl):
     if '.flac' in [f[-5:] for f in fl]:
         newD = d.replace('FLACS', 'MP3s')
         mkdir(newD)
         for f in fl:
             if f[-5:]=='.flac':
                 root = f.replace('.flac', '')
                 cmd = ['"C:\\Program Files (x86)\\aTunes\\win_tools\
\flac.exe" -d ' +
                        '--output-prefix=' + newD + '\\', f]
                 res = sub.call(cmd)#, env={'PATH': os.defpath})
                 if not res:
                     cmd = ['"C:\\Program Files(x86)\\aTunes\\win_tools
\\lame.exe" -h',
                             newD + root + '.wav',  newD + root +
'.mp3']
                     res = sub.call(cmd)#, env={'PATH': os.defpath})
                     if not res:
                         rf = newD + root +'.wav'
                         remove(rf)
Click to expand...
top=sys.argv[1]
op.walk(top, doFlac2Mp3, None)
Click to expand...

I think that if the command line should be something like:

"C:\Program Files (x86)\aTunes\win_tools\flac.exe" -d
--output-prefix=FOO\ BAR

then the cmd should be something like:

cmd = ['C:\\Program Files (x86)\\aTunes\\win_tools\\flac.exe', '-d',
'--output-prefix="' + newD + '\\"', f]
Click to expand...

Thanks again all! MRAB: your advice was sufficient to get me over the
hump, so to speak, thanks for chiming in!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Is it possible to set the date/time of a directory in windows with Python? If so how? 2

Members online

Total: 51 (members: 3, guests: 48)
Robots: 424

Forum statistics

Threads
473,968
Messages
2,570,153
Members
46,699
Latest member
AnneRosen

Latest Threads

Top