How to store a variable when a script is executing for next time execution?

[Avnesh Shakya]

hi,
I am running a python script and it will create a file name like filename0.0.0 and If I run it again then new file will create one more like filename0.0.1...... my code is-

i = 0
for i in range(1000):
try:
with open('filename%d.%d.%d.json'%(0,0,i,)): pass
continue
except IOError:
dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
break
But It will take more time after creating many files, So i want to store value of last var "i" in a variable so that when i run my script again then I can use it. for example-
my last created file is filename0.0.27 then it should store 27 in a variable and when i run again then new file should be created 0.0.28 according to last value "27", so that i could save time and it can create file fast..

Please give me suggestion for it.. How is it possible?
Thanks

 

[Cameron Simpson]

| hi,
| I am running a python script and it will create a file name like filename0.0.0 and If I run it again then new file will create one more like filename0.0.1...... my code is-
|
| i = 0
| for i in range(1000):
| try:
| with open('filename%d.%d.%d.json'%(0,0,i,)): pass
| continue
| except IOError:
| dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
| break
| But It will take more time after creating many files, So i want to store value of last var "i" in a variable so that when i run my script again then I can use it. for example-
| my last created file is filename0.0.27 then it should store 27 in a variable and when i run again then new file should be created 0.0.28 according to last value "27", so that i could save time and it can create file fast..
|
| Please give me suggestion for it.. How is it possible?

Write it to a file? Read the file next time the script runs?

BTW, trying to open zillions of files is slow.
But using listdir to read the directory you can see all the names.
Pick the next free one (and then test anyway).
--
Cameron Simpson <[email protected]>

The mark must be robust enough to survive MP3 transmission over the Internet,
but remain inaudible when played on the yet to be launched DVD-Audio players.
- the SDMI audio watermarkers literally ask for the impossible, since all
audio compressors aim to pass _only_ human perceptible data
http://www.newscientist.com/news/news.jsp?id=ns224836

 

[Dave Angel]

hi,
I am running a python script and it will create a file name like filename0.0.0 and If I run it again then new file will create one more like filename0.0.1...... my code is-

i = 0

Redundant initialization of i.

for i in range(1000):
try:
with open('filename%d.%d.%d.json'%(0,0,i,)): pass
continue
except IOError:
dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
break
But It will take more time after creating many files, So i want to store value of last var "i" in a variable

There are no variables once the program ends. You mean you want to
store it in the file. That's known as persistent storage, and in the
general case you could use pickle or something like that. But in your
simple case, the easiest thing would be to simply write the last value
of i out to a file in the same directory.

Then when your program starts, it opens that extra file and reads in the
value of i. And uses that for the starting value in the loop.

so that when i run my script again then I can use it. for example-

my last created file is filename0.0.27 then it should store 27 in a variable and when i run again then new file should be created 0.0.28 according to last value "27", so that i could save time and it can create file fast..

Please give me suggestion for it.. How is it possible?
Thanks

Incidentally, instead of opening each one, why not check its existence?
Should be quicker, and definitely clearer.

Entirely separate suggestion, since I dislike having extra housekeeping
files that aren't logically necessary, and that might become out of synch :

If you're planning on having the files densely populated (meaning no
gaps in the numbering), then you could use a binary search to find the
last one. Standard algorithm would converge with 10 existence checks if
you have a limit of 1000 files.

 

[Jussi Piitulainen]

I am running a python script and it will create a file name like
filename0.0.0 and If I run it again then new file will create one
more like filename0.0.1...... my code is-

i = 0
for i in range(1000):
try:
with open('filename%d.%d.%d.json'%(0,0,i,)): pass
continue
except IOError:
dataFile = file('filename%d.%d.%d.json'%(0,0,i,), 'a+')
break

But It will take more time after creating many files, So i want to
store value of last var "i" in a variable so that when i run my
script again then I can use it. for example- my last created file is
filename0.0.27 then it should store 27 in a variable and when i run
again then new file should be created 0.0.28 according to last value
"27", so that i could save time and it can create file fast..

You could get a list of all filenames that match the pattern. Extract
the last components as numbers, and add 1 to the maximum.

i = 1 + max(int(name.split('.')[-1])
for name in glob.glob('filename.0.0.*))

That assumes that there already is at least one such file and all such
files have a last component that can be parsed as an int. Take an
appropriate amount of care.

Or you could also create a file, say lastname.0.0.31, to track the
name, and when you find it there, create filename.0.0.32 and replace
lastname.0.0.32; panic if there is more than one lastname.0.0.*, or
fewer than one.

Or as above but track with nextname.0.0.31 to create filename.0.0.31
and replace the tracking name with nextname.0.0.32 for the next file.

Or save the number somewhere else.

 

[Chris Angelico]

If you're planning on having the files densely populated (meaning no gaps in
the numbering), then you could use a binary search to find the last one.
Standard algorithm would converge with 10 existence checks if you have a
limit of 1000 files.

Or, if you can dedicate a directory to those files, you could go even simpler:

dataFile = open('filename0.0.%d.json'%len(os.listdir()), 'w')

The number of files currently existing equals the number of the next file.

ChrisA

 

[MRAB]

Or, if you can dedicate a directory to those files, you could go even simpler:

dataFile = open('filename0.0.%d.json'%len(os.listdir()), 'w')

The number of files currently existing equals the number of the next file.

Assuming no gaps.

 

[Chris Angelico]

Assuming no gaps.

Which is also a stated assumption of the binary search. The only
additional assumption of the file-count method is that there be no
other files in that directory.

ChrisA