fetching text from the screen

[Juergen Kareta]

Hello list,

I'm thinking about a python script which fetch some text from the screen
independent of what application provides the text on the screen. In this
regard it should be similar to the babylon software: www.babylon.com


Here my thoughts:

1) getting the mouse position
2) calculate a adequate rectangle around the mouse position
3) fetch the screen content in this rectangle
4) use a OCR library to get the text out of 3)
5) do something usefull with it.

Has someone experiences with this task ? Is this a good roadmap or
could someone think of a better way? Which OCR library shoud I use ?
Found tesseract-ocr (linux, windows) and simpleOCR (bad, only windows
(activeX).

Thanks in advance for your replays.

Jürgen

 

[Gabriel Genellina]

I'm thinking about a python script which fetch some text from the screen
independent of what application provides the text on the screen. In this
regard it should be similar to the babylon software: www.babylon.com


Here my thoughts:

1) getting the mouse position
2) calculate a adequate rectangle around the mouse position
3) fetch the screen content in this rectangle
4) use a OCR library to get the text out of 3)
5) do something usefull with it.

On Windows, I'd try first using WindowFromPoint to get a window handle,
and the sending it a WM_GETTEXT message. This should work for all windowed
controls that contain text of some kind. I'd use your generic approach
when this doesn't work.

 

[Juergen Kareta]

On Windows, I'd try first using WindowFromPoint to get a window handle,
and the sending it a WM_GETTEXT message. This should work for all
windowed controls that contain text of some kind. I'd use your generic
approach when this doesn't work.

Hi Gabriel,

thanks for your interesting suggestions. I'll try that to figure out,
how it works in different situations.

Jürgen