File exists but Python says 'not found'.

[prerit86]

I'm running this code that reads 2 csv files (one of them is train.csv). The code gives an error saying 'file not does not exist'. However, the file does exists in the same location as the .py file. Can someone please help me on this. Thanks!

Code Output-->

Reading dataset...
Traceback (most recent call last):
File "c:\Project_1\regression_2.py", line 163, in <module>
main(**args)
File "c:\Project_1\regression_2.py", line 80, in main
train_data = pd.read_csv(train)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 401, in parser
_f
return _read(filepath_or_buffer, kwds)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 209, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 509, in __init
__
self._make_engine(self.engine)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 611, in _make_
engine
self._engine = CParserWrapper(self.f, **self.options)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 893, in __init
__
self._reader = _parser.TextReader(src, **kwds)
File "parser.pyx", line 312, in pandas._parser.TextReader.__cinit__ (pandas\sr
c\parser.c:2846)
File "parser.pyx", line 512, in pandas._parser.TextReader._setup_parser_source
(pandas\src\parser.c:4893)
IOError: File train.csv does not exist

 

[Robert Kern]

I'm running this code that reads 2 csv files (one of them is train.csv). The code gives an error saying 'file not does not exist'. However, the file does exists in the same location as the .py file. Can someone please help me on this. Thanks!

How are you running the code? If you are doing pandas.read_csv('train.csv'), the
file must be in the current working directory of the running process. The
location of the .py file contain the code is not relevant. For example, if you
are using an IDE to run the script, you may need to configure how it runs the
script to pick a particular directory for it to run in.

Code Output-->

Reading dataset...
Traceback (most recent call last):
File "c:\Project_1\regression_2.py", line 163, in <module>
main(**args)
File "c:\Project_1\regression_2.py", line 80, in main
train_data = pd.read_csv(train)

Since the filename of regression_2.py in the traceback is fully-qualified, I
expect that you are running the program from something other than the
c:\Project_1\ directory.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[prerit86]

The variable 'train' is being called like this ->

def main(train='train.csv', test='test.csv', submit='logistic_pred.csv'):
print "Reading dataset..."
train_data = pd.read_csv(train)
test_data = pd.read_csv(test)

Let me know if I need to post the full code.

 

[Antoon Pardon]

Op 01-07-13 12:47, (e-mail address removed) schreef:

I'm running this code that reads 2 csv files (one of them is train.csv). The code gives an error saying 'file not does not exist'. However, the file does exists in the same location as the .py file. Can someone please help me on this. Thanks!

Code Output-->

Reading dataset...
Traceback (most recent call last):
File "c:\Project_1\regression_2.py", line 163, in <module>
main(**args)
File "c:\Project_1\regression_2.py", line 80, in main
train_data = pd.read_csv(train)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 401, in parser
_f
return _read(filepath_or_buffer, kwds)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 209, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 509, in __init
__
self._make_engine(self.engine)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 611, in _make_
engine
self._engine = CParserWrapper(self.f, **self.options)
File "c:\Python27\lib\site-packages\pandas\io\parsers.py", line 893, in __init
__
self._reader = _parser.TextReader(src, **kwds)
File "parser.pyx", line 312, in pandas._parser.TextReader.__cinit__ (pandas\sr
c\parser.c:2846)
File "parser.pyx", line 512, in pandas._parser.TextReader._setup_parser_source
(pandas\src\parser.c:4893)
IOError: File train.csv does not exist

My guess is that train.csv is a symbolic link that points to a file
that doesn't exist. So looking at your directory you can see an entry
but actualy trying to open it will fail.

 

[prerit86]

I got it. The working directory was different. Sorry, I'm new and didn't the working directory has to be the location of the data. I thought the location of .py file and data file should be same. Thanks! Es. Robert Kern.

 

[Antoon Pardon]

Op 01-07-13 12:57, (e-mail address removed) schreef:

The variable 'train' is being called like this ->

def main(train='train.csv', test='test.csv', submit='logistic_pred.csv'):
print "Reading dataset..."
train_data = pd.read_csv(train)
test_data = pd.read_csv(test)

Let me know if I need to post the full code.

I think Robert wanted to know how you started the program.
What instruction do you use to launch?
In what directory are you launching your program?
Is that the same directory where train.csv is present?

 

[prerit86]

My answers

I think Robert wanted to know how you started the program.
What instruction do you use to launch?
- used command c:\python27\python.exe c:\project_1\code.py

In what directory are you launching your program?
- working directory was c:
- python is in c:\python27
- code was in c:\project_1
- changed the working directory to c:\project_1 now and it worked

Is that the same directory where train.csv is present?
- train.csv is present in c:\project_1

 

[Dave Angel]

I got it. The working directory was different. Sorry, I'm new and didn't the working directory has to be the location of the data. I thought the location of .py file and data file should be same. Thanks! Es. Robert Kern.

Python didn't make that assumption, the author of the script did. By
using a relative name like train.csv, he is implicitly forcing Python to
use the current working directory.

If he had wanted the file to be found in the same directory as one of
the source files, he could have built an absolute path by using the
__file__ attribute of the module.

However, for non-const files, the source directory is generally a bad
place to put them.

Confounding things is the strange habit that Windows has of deciding
that the script location *should* be your current directory. I believe
Explorer does that if you right-click select a script.

 

[prerit86]

I know. Had I written the code, I would have not done this. I just wanted to get some initial results by leveraging this code. I would now build on this to improve my work's accuracy.

Thanks for the inputs!