How to bypass Windows 'cooking' the I/O? (One more time, please)

[norseman]

I know I saw the answer recently, as in since February '08, but I can't
re-find it. I tried the mail archives and such and my own
collections but the piece I saw still eludes me.


Problem: (sos=same old s...) Microsoft insists the world work it's way
even when the Microsoft way was proven wrong decades ago. In this case
it's (still) 'cooking' the writes even with 'rwb' and O_RDWR|O_BINARY in
(proper respective) use.

Specific: python created and inspected binary file ends:
00460: 0D 1A (this is correct)

after a write
os.lseek(target, -1, 2)
os.write(target,record)
the expected result would be:
00460: 0D 20 .....data bytes.... 1A

BUT I get:
00460: 20 .... data bytes... 1A

It is one byte off!!! And the 0D has to be there. Signifies the end of
the header.


Same python program runs as expected in Linux. Maybe because that's
where it was written?!


What I seek is the way to slap Microsoft up side the head and make it
work correctly. OK, well, at least in this situation.


Note: Things like this justify Python implementers bypassing OS calls
(data fetch, data write) and using the BIOS direct. Remember, the CPU
understands bit patterns only. It has no comprehension of 'text',
'program', 'number', 'pointer', blah blah blah.... All that is totally
beyond it's understanding. A given bit pattern means 'do that'. The CPU
is 100% binary. Memory, storage and the rest is just bits-on, bits-off.
Patterns. Proper binary I/O is mandatory for the machine to function.


Anyway - if whoever mentioned the flags and such to 'over ride'
Microsoft's BS would re-send that piece I would be very appreciative.


Steve
(e-mail address removed)

 

[Tim Roberts]

Problem: (sos=same old s...) Microsoft insists the world work it's way
even when the Microsoft way was proven wrong decades ago. In this case
it's (still) 'cooking' the writes even with 'rwb' and O_RDWR|O_BINARY in
(proper respective) use.

No, it doesn't. Where did you get the idea that 'rwb' is a valid fopen
mode? It's not. If you need to read and write an existing file, you want
"r+b".


C:\tmp>python
Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.

f = open('x.bat','r+b')
s = f.read()
s 'sed -e "s/[ \\t]*$//" -e "/^$/d" %1\rhow about that\r\n'
f.seek(-1,2)
f.write('xxx\r\n')
f.close()
f = open('x.bat','rb')
t = f.read()
t 'sed -e "s/[ \\t]*$//" -e "/^$/d" %1\rhow about that\rxxx\r\n'

Same python program runs as expected in Linux. Maybe because that's
where it was written?!

Perhaps your Linux C runtime library accepts the fopen mode 'rwb', but if
it does, it's a non-standard extension.

What I seek is the way to slap Microsoft up side the head and make it
work correctly. OK, well, at least in this situation.

It works correctly if you use it correctly.

 

[Ulrich Eckhardt]

In this case it's [MS Windows] (still) 'cooking' the writes even
with 'rwb' and O_RDWR|O_BINARY in (proper respective) use.

Specific: python created and inspected binary file ends:
00460: 0D 1A (this is correct)

I'm not actually sure what the 0x1a is supposed to do there. The 0x0d is
a '\r' which looks like a MacOS line ending, but I also seem to remember
0x1a being use as EOF signal in some old system's text files. Anyway, you
mentioned some "binary file" (aren't they all?) so this might not matter
after all.

after a write
os.lseek(target, -1, 2)

That should be os.SEEK_END and not 2. In any case, it moves the write
position to the last byte (-1 offset to the end of the file).

os.write(target,record)
the expected result would be:
00460: 0D 20 .....data bytes.... 1A

I'm sorry, but I don't see that. write() will not insert the bytes but first
overwrite till EOF and then append further output, i.e. the 0x1a at the end
is only present if it was present in the record.

BUT I get:
00460: 20 .... data bytes... 1A

It is one byte off!!! And the 0D has to be there. Signifies the end of
the header.

Okay, this looks strange, both the offset and the fact that it seems to
insert bytes. How about providing a minimal example?

Anyway - if whoever mentioned the flags and such to 'over ride'
Microsoft's BS would re-send that piece I would be very appreciative.

You are not one with the Tao, which is why you programs don't run.

Uli