scraping display to obtain all on-screen text using OCR

[Jonathan Epstein]

I would like to perform a more classical type of "screen scraping"
than what most people now associate with this term. I only want to
find all the text on the current screen, and obtain associated screen
coordinates. This probably must be done using OCR.

This need only run on Windows. A fairly-pure Python solution would be
ideal because most of the software which would use this functionality
is also written in Python.

The ideal output would consist of a list of tuples, where each tuple
consists of ("string found", a, b, c, d) where the latter four
constitute a bounding rectangle associated with the text that was
found. It might also be handy to throw in some font information.

Thanks in advance for any pointers.

Jonathan

 

[Roger Binns]

I would like to perform a more classical type of "screen scraping"
than what most people now associate with this term. I only want to
find all the text on the current screen, and obtain associated screen
coordinates. This probably must be done using OCR.

This need only run on Windows.

You can use the accessibility APIs to get that information. Start at
http://weblogs.asp.net/oldnewthing/archive/2004/04/23/118893.aspx

A fairly-pure Python solution would be
ideal because most of the software which would use this functionality
is also written in Python.

You may be able to do it using win32all, ctypes or worst case a
SWIG wrapper.

Roger

 

[Paul Rubin]

I would like to perform a more classical type of "screen scraping"
than what most people now associate with this term. I only want to
find all the text on the current screen, and obtain associated screen
coordinates. This probably must be done using OCR.
This need only run on Windows.

Usually you do that by intercepting the Windows text painting events,
rather than anything as horrendous as OCR'ing.