How to extract text from windows.

[Kelvin]

Hi, there:

I have seen some program which extracts text from the computer screen.
For example, my English-Chinese dictionary, which extracts a English word
from the buttons and menus on other applications.

How does this kind of trick work? How do I program them? What are the
keyword to search for information on this topic?

Thanks in advance.

Kelvin

 

[Malcolm Dew-Jones]

Kelvin ([email protected]) wrote:
: Hi, there:

: I have seen some program which extracts text from the computer screen.
: For example, my English-Chinese dictionary, which extracts a English word
: from the buttons and menus on other applications.

: How does this kind of trick work? How do I program them? What are the
: keyword to search for information on this topic?


Look at the windows api. Functions such as GetWindow(), EnumWindows(),
GetWindowText(), are required.

Given a top level window, recursively scan through the children. (Each
control etc is a window.)

All the data about each window is available by examining the various
properties of each window (which are available using functions such as
GetWindowText()).

For the generic classes provided by the windows api, the data has the
"obvious" meaning, i.e. the text for a button window is the text shown on
the button.

The only problem is when the window uses a custom class to generate the
display. Then you have to know how to interpret the data from that
window. Most custom classes are the tools provided by a particular
development environment, (e.g. MFC) so if you have the same environment
then you can interpret the data by simply passing it to the appropriate
api functions of that environment.

Actually, for some things you may have to use other functions,
GetMenuString() comes to mind, perhaps there are others.

 

[Simon]

Your basic technique is as follows:
* Run a timer that regularly gets updates about the window that is
below the cursor.
* At each timer interval you will need to use the APIs WindowFromPoint
and ChildWindowFromPoint. These let you know the window handle below
the cursor.
* From the window handle you can use the GetWindowText to retrieve the
text of the button, edit box, static box or whatever similar item you
desire.
* Do a dictionary matching and translation of whatever text you got
above.

Truth is that most of this is easy to do. See
http://cuinl.tripod.com/Tips/wincaption.htm for an example. The
dictionary part may be a bit more difficult depending on whether you
are doing a simply word-for-word match or you are actually
interpreting a phrase from English to Chinese.

Simon Hayden
www.AutoUpdatePlus.com
Get updates to your clients the quick and easy way.