Python and C

D

diffuser78

I was a C Programmer for a while. Lately started to learn Python for
one small project at school. I joined a small company where they use
C++ for development.

Can we use Python and C together ? I mean create some classes in Python
and some number crunching algorithms coded in C (for speed) and
integreate both of them.

Could somebody just show a real small example or give pointers to the
same.

Thanks
 
D

Diez B. Roggisch

I was a C Programmer for a while. Lately started to learn Python for
one small project at school. I joined a small company where they use
C++ for development.

Can we use Python and C together ? I mean create some classes in Python
and some number crunching algorithms coded in C (for speed) and
integreate both of them.

Could somebody just show a real small example or give pointers to the
same.

There is even a whole section in the documentation geared towards that -
and to spoil the fun of actually reading it: Yes, Python and C go aleong
_very_ well, and for C++ there are several specialized wrapping generators
that e.g. make PyQt go round. Check out SIP or boost.python or SWIG.

http://docs.python.org/api/api.html

Diez
 
A

Alex Martelli

I was a C Programmer for a while. Lately started to learn Python for
one small project at school. I joined a small company where they use
C++ for development.

Can we use Python and C together ? I mean create some classes in Python
and some number crunching algorithms coded in C (for speed) and
integreate both of them.

Sure! It's extremely common practice, known as "extending Python".

Could somebody just show a real small example or give pointers to the
same.

Besides the Extending/Embedding and C API documents which are part of
the standard set of Python docs, to which you've already been pointed in
other responses, you can find tutorials on the net. A tiny one is at
<http://www.developer.com/lang/other/article.php/2191421>, with pointers
to other materials. Also, examples can be found in the Demos directory
of the Python source distribution -- even if you have a binary distro
installed, get the source distro anyway, it's full of goodies and you
can use much of Python itself as an example of Extending (the Modules
and Objects directories of the source distro, in particular, are just
such examples).

C is the lowest, most fundamental level of extension, but there are many
other alternatives -- SWIG to wrap existing libraries, Boost or SCXX or
SIP to wrap specifically C++ with very different philosophies (template
heavy, minimal, Qt-based), pyrex (a Python "dialect" plus C-like
declarations to make it compilable to fast machine code), and others
yet.


Alex
 
D

David Boddie

Alex said:
C is the lowest, most fundamental level of extension, but there are many
other alternatives -- SWIG to wrap existing libraries, Boost or SCXX or
SIP to wrap specifically C++ with very different philosophies (template
heavy, minimal, Qt-based), pyrex (a Python "dialect" plus C-like
declarations to make it compilable to fast machine code), and others
yet.

It is a common misconception that SIP is only really used to wrap
Qt-based libraries, though that may be its main use in many projects:

"SIP is a tool that makes it very easy to create Python bindings
for C and C++ libraries. It was originally developed to create PyQt,
the Python bindings for the Qt toolkit, but can be used to create
bindings for any C or C++ library."

[http://www.riverbankcomputing.co.uk/sip/]

David
 
P

P Boy

I have written some C extension before but it was pretty tedious. I
have recently found another approach by using ctypes
(http://starship.python.net/crew/theller/ctypes/). Which you develop
your C module in dynamic library (DLL in Windows world), the from
Python, you can call the C functions in the DLL using ctypes.

I have a library from Microsoft, ewfapi.dll, ewfapi.lib (not required),
and ewfapi.h (see download) which I want to program the EWF capability
in Windows XP Embedded. I was thinking about writing a Python C
extension for it, but using ctypes is much easier. I only need to
redefine some data structures from the header file, and not worry about
common C tasks such as memory management, reference counting, etc.

Ewfapi files can be downloaded from
http://www.microsoft.com/downloads/...35-ae95-4864-ba3c-d488d3980972&displaylang=en

I have some sample usage in Python shown below, which I can
interactively control the EWF feature using Python shell (in an XP
embedded machine).


# ewf.py

import ctypes
import string

# These are the functions from the EWF API
##EwfMgrGetDriveLetterFromVolumeName(volumeName)
##
##EwfMgrVolumeNameListIsEmpty(volumeEntry)
##
##EwfMgrVolumeNameEntryPop(volumeEntry)
##
##EwfMgrVolumeNameListDelete
##
##EwfMgrOpenProtected
##
##EwfMgrClose = _ewfapiPy.EwfMgrClose
##
##EwfMgrClearCommand = _ewfapiPy.EwfMgrClearCommand
##
##EwfMgrSetPersistentData = _ewfapiPy.EwfMgrSetPersistentData
##
##EwfMgrGetPersistentData = _ewfapiPy.EwfMgrGetPersistentData
##
##EwfMgrCheckpoint = _ewfapiPy.EwfMgrCheckpoint
##
##EwfMgrRestore = _ewfapiPy.EwfMgrRestore
##
##EwfMgrDisable = _ewfapiPy.EwfMgrDisable
##
##EwfMgrEnable = _ewfapiPy.EwfMgrEnable
##
##EwfMgrCommit = _ewfapiPy.EwfMgrCommit
##
##EwfMgrCommitAndDisableLive = _ewfapiPy.EwfMgrCommitAndDisableLive
##
##EwfMgrCommitFile = _ewfapiPy.EwfMgrCommitFile
##
##EwfMgrSetLevel = _ewfapiPy.EwfMgrSetLevel
##
##EwfMgrGetProtectedVolumeConfig =
_ewfapiPy.EwfMgrGetProtectedVolumeConfig
##
##EwfMgrGetProtectedVolumeList = _ewfapiPy.EwfMgrGetProtectedVolumeList
##
##EwfMgrOpenOverlayStore = _ewfapiPy.EwfMgrOpenOverlayStore
##
##EwfMgrGetOverlayStoreConfig = _ewfapiPy.EwfMgrGetOverlayStoreConfig
##
##EwfMgrRegisterLowSpaceNotification =
_ewfapiPy.EwfMgrRegisterLowSpaceNotification

# Short handle to the API
ewfapi = ctypes.windll.ewfapi

# Data structure from EWFAPI.h translated into Python

# Map these enumerations to string
EWF_CMD = { 0 : 'NO Command',
1 : 'Enable',
2 : 'Disable',
3 : 'Set Level',
4 : 'Commit',
}

EWF_STATE = { 0 : 'Enabled',
1 : 'Disabled',
}

EWF_TYPE = { 0 : 'Disk',
1 : 'RAM',
2 : 'RAM Reg',
}

# Forward Declaration
pVolumeNameEntry = ctypes.POINTER('VolumeNameEntry')

# Structure of a linked-list
class VolumeNameEntry(ctypes.Structure):
_fields_ = [('Next', ctypes.POINTER(pVolumeNameEntry)),
('Name', ctypes.c_wchar * 256),
]

# Set the pointer to the structure
ctypes.SetPointerType(pVolumeNameEntry, VolumeNameEntry)


# Volume descriptor
class VolumeDesc(ctypes.Structure):
_fields_ = [('DeviceName', ctypes.c_wchar * 256),
('VolumeID', ctypes.c_ubyte * 16),
]

pVolumeDesc = ctypes.POINTER(VolumeDesc)


# Overlay configuration
class OverlayStoreConfig(ctypes.Structure):
_fields_ = [('FormatVersion', ctypes.c_ulong),
('VolumeSize', ctypes.c_longlong),
('NumSegments', ctypes.c_ulong),
('FreeSegments', ctypes.c_ulong),
('SegmentSize', ctypes.c_ulong),
('MaxVolumes', ctypes.c_ulong),
('NumVolumes', ctypes.c_ulong),
('MaxLevels', ctypes.c_ushort),
('VolumeDescArray', pVolumeDesc),
]

pOverlayStoreConfig = ctypes.POINTER(OverlayStoreConfig)

class Command(ctypes.Structure):
_fields_ = [('Command', ctypes.c_int),
('Param1', ctypes.c_ulong),
('Param2', ctypes.c_ulong),
]

pCommand = ctypes.POINTER(Command)

class FileTime(ctypes.Structure):
_fields_ = [('LowDateTime', ctypes.c_ulong),
('HighDateTime', ctypes.c_ulong),
]

pFileTime = ctypes.POINTER(FileTime)

class LevelDesc(ctypes.Structure):
_fields_ = [('LevelName', ctypes.c_wchar * 64),
('LevelEndTime', FileTime),
('LevelDataSize', ctypes.c_longlong),
]

pLevelDesc = ctypes.POINTER(LevelDesc)


class VolumeConfig(ctypes.Structure):
_fields_ = [('Type', ctypes.c_int),
('State', ctypes.c_int),
('BootCommand', Command),
('PersistentData', ctypes.c_ubyte * 32),
('MaxLevels', ctypes.c_ushort),
('ClumpSize', ctypes.c_ulong),
('CurrentLevel', ctypes.c_ushort),
('DiskMap_RamDataSize', ctypes.c_longlong),
('DiskDataSize', ctypes.c_longlong),
('MemMapSize', ctypes.c_ulong),
('VolumeDesc', VolumeDesc),
('LevelDescArray', pLevelDesc),
]

def formatUnsigned(u):
return '%02X' % u

def unsignedArrayToHex(array):
return string.join(map(formatUnsigned, array)).replace(' ','')

def displayVolumeConfiguration(config, drive):
config_type = EWF_TYPE[config.Type]
print 'Type : %s' % config_type
print 'State : %s' % EWF_STATE[config.State]
print 'Boot Cmd : %s' % EWF_CMD[config.BootCommand.Command]
print 'Param 1 : %u' % config.BootCommand.Param1
print 'Param 2 : %u' % config.BootCommand.Param2
print 'Persistent Data: %s' %
unsignedArrayToHex(config.PersistentData)
print
print 'Maximum Levels : %u' % config.MaxLevels
print 'Clump Size : %u Bytes' % config.ClumpSize
print 'Current Level : %u' % config.CurrentLevel
if config_type == 'Disk':
print 'Disk Map Size : %u Bytes' %
config.DiskMap_RamDataSize
print 'Disk Data Size : %u Bytes' % config.DiskDataSize
else:
print 'RAM Data Size : %u Bytes' %
config.DiskMap_RamDataSize

print 'Memory Map Size : %u Bytes' % config.MemMapSize
print 'Device Name : %s' % config.VolumeDesc.DeviceName
print 'Volume ID : %s' %
unsignedArrayToHex(config.VolumeDesc.VolumeID)
print 'Drive : %s' % drive

def information():
# Get the pointer and convert to linked-list node
volume_list =
VolumeNameEntry.from_address(ewfapi.EwfMgrGetProtectedVolumeList())
while 1:
drive =
chr(ewfapi.EwfMgrGetDriveLetterFromVolumeName(volume_list.Name))
print '%s: = %s' % (drive, volume_list.Name)
handle = ewfapi.EwfMgrOpenProtected(volume_list.Name)
config =
VolumeConfig.from_address(ewfapi.EwfMgrGetProtectedVolumeConfig(handle))
displayVolumeConfiguration(config, drive)
if
ewfapi.EwfMgrVolumeNameListIsEmpty(ctypes.c_void_p.from_address(ctypes.addressof(volume_list))):
break


if __name__ == "__main__ ":
print 'Running'
information()
 
P

P Boy

I have written some C extension before but it was pretty tedious. I
have recently found another approach by using ctypes
(http://starship.python.net/crew/theller/ctypes/). Which you develop
your C module in dynamic library (DLL in Windows world), the from
Python, you can call the C functions in the DLL using ctypes.

I have a library from Microsoft, ewfapi.dll, ewfapi.lib (not required),
and ewfapi.h (from msdn.com) which I want to program the EWF capability
in Windows XP Embedded. I was thinking about writing a Python C
extension for it, but using ctypes is much easier. I only need to
redefine some data structures from the header file, and not worry about
common C tasks such as memory management, reference counting, etc.


I have some sample usage in Python shown below, which I can
interactively control the EWF feature using Python shell (in an XP
embedded machine).


# ewf.py


import ctypes
import string


# These are the functions from the EWF API
##EwfMgrGetDriveLetterFromVolumeName(volumeName)
##
##EwfMgrVolumeNameListIsEmpty(volumeEntry)
##
##EwfMgrVolumeNameEntryPop(volumeEntry)
##
##EwfMgrVolumeNameListDelete
##
##EwfMgrOpenProtected
##
##EwfMgrClose = _ewfapiPy.EwfMgrClose
##
##EwfMgrClearCommand = _ewfapiPy.EwfMgrClearCommand
##
##EwfMgrSetPersistentData = _ewfapiPy.EwfMgrSetPersistentData
##
##EwfMgrGetPersistentData = _ewfapiPy.EwfMgrGetPersistentData
##
##EwfMgrCheckpoint = _ewfapiPy.EwfMgrCheckpoint
##
##EwfMgrRestore = _ewfapiPy.EwfMgrRestore
##
##EwfMgrDisable = _ewfapiPy.EwfMgrDisable
##
##EwfMgrEnable = _ewfapiPy.EwfMgrEnable
##
##EwfMgrCommit = _ewfapiPy.EwfMgrCommit
##
##EwfMgrCommitAndDisableLive = _ewfapiPy.EwfMgrCommitAndDisableLive
##
##EwfMgrCommitFile = _ewfapiPy.EwfMgrCommitFile
##
##EwfMgrSetLevel = _ewfapiPy.EwfMgrSetLevel
##
##EwfMgrGetProtectedVolumeConfig =
_ewfapiPy.EwfMgrGetProtectedVolumeConfig
##
##EwfMgrGetProtectedVolumeList = _ewfapiPy.EwfMgrGetProtectedVolumeList

##
##EwfMgrOpenOverlayStore = _ewfapiPy.EwfMgrOpenOverlayStore
##
##EwfMgrGetOverlayStoreConfig = _ewfapiPy.EwfMgrGetOverlayStoreConfig
##
##EwfMgrRegisterLowSpaceNotification =
_ewfapiPy.EwfMgrRegisterLowSpaceNotification


# Short handle to the API
ewfapi = ctypes.windll.ewfapi


# Data structure from EWFAPI.h translated into Python


# Map these enumerations to string
EWF_CMD = { 0 : 'NO Command',
1 : 'Enable',
2 : 'Disable',
3 : 'Set Level',
4 : 'Commit',
}


EWF_STATE = { 0 : 'Enabled',
1 : 'Disabled',
}


EWF_TYPE = { 0 : 'Disk',
1 : 'RAM',
2 : 'RAM Reg',
}


# Forward Declaration
pVolumeNameEntry = ctypes.POINTER('VolumeNameEntry')


# Structure of a linked-list
class VolumeNameEntry(ctypes.Structure):
_fields_ = [('Next', ctypes.POINTER(pVolumeNameEntry)),
('Name', ctypes.c_wchar * 256),
]


# Set the pointer to the structure
ctypes.SetPointerType(pVolumeNameEntry, VolumeNameEntry)


# Volume descriptor
class VolumeDesc(ctypes.Structure):
_fields_ = [('DeviceName', ctypes.c_wchar * 256),
('VolumeID', ctypes.c_ubyte * 16),
]


pVolumeDesc = ctypes.POINTER(VolumeDesc)


# Overlay configuration
class OverlayStoreConfig(ctypes.Structure):
_fields_ = [('FormatVersion', ctypes.c_ulong),
('VolumeSize', ctypes.c_longlong),
('NumSegments', ctypes.c_ulong),
('FreeSegments', ctypes.c_ulong),
('SegmentSize', ctypes.c_ulong),
('MaxVolumes', ctypes.c_ulong),
('NumVolumes', ctypes.c_ulong),
('MaxLevels', ctypes.c_ushort),
('VolumeDescArray', pVolumeDesc),
]


pOverlayStoreConfig = ctypes.POINTER(OverlayStoreConfig)


class Command(ctypes.Structure):
_fields_ = [('Command', ctypes.c_int),
('Param1', ctypes.c_ulong),
('Param2', ctypes.c_ulong),
]


pCommand = ctypes.POINTER(Command)


class FileTime(ctypes.Structure):
_fields_ = [('LowDateTime', ctypes.c_ulong),
('HighDateTime', ctypes.c_ulong),
]


pFileTime = ctypes.POINTER(FileTime)


class LevelDesc(ctypes.Structure):
_fields_ = [('LevelName', ctypes.c_wchar * 64),
('LevelEndTime', FileTime),
('LevelDataSize', ctypes.c_longlong),
]


pLevelDesc = ctypes.POINTER(LevelDesc)


class VolumeConfig(ctypes.Structure):
_fields_ = [('Type', ctypes.c_int),
('State', ctypes.c_int),
('BootCommand', Command),
('PersistentData', ctypes.c_ubyte * 32),
('MaxLevels', ctypes.c_ushort),
('ClumpSize', ctypes.c_ulong),
('CurrentLevel', ctypes.c_ushort),
('DiskMap_RamDataSize', ctypes.c_longlong),
('DiskDataSize', ctypes.c_longlong),
('MemMapSize', ctypes.c_ulong),
('VolumeDesc', VolumeDesc),
('LevelDescArray', pLevelDesc),
]


def formatUnsigned(u):
return '%02X' % u


def unsignedArrayToHex(array):
return string.join(map(formatUnsigned, array)).replace(' ','')


def displayVolumeConfiguration(config, drive):
config_type = EWF_TYPE[config.Type]
print 'Type : %s' % config_type
print 'State : %s' % EWF_STATE[config.State]
print 'Boot Cmd : %s' % EWF_CMD[config.BootCommand.Command]
print 'Param 1 : %u' % config.BootCommand.Param1
print 'Param 2 : %u' % config.BootCommand.Param2
print 'Persistent Data: %s' %
unsignedArrayToHex(config.PersistentData)
print
print 'Maximum Levels : %u' % config.MaxLevels
print 'Clump Size : %u Bytes' % config.ClumpSize
print 'Current Level : %u' % config.CurrentLevel
if config_type == 'Disk':
print 'Disk Map Size : %u Bytes' %
config.DiskMap_RamDataSize
print 'Disk Data Size : %u Bytes' % config.DiskDataSize
else:
print 'RAM Data Size : %u Bytes' %
config.DiskMap_RamDataSize


print 'Memory Map Size : %u Bytes' % config.MemMapSize
print 'Device Name : %s' % config.VolumeDesc.DeviceName
print 'Volume ID : %s' %
unsignedArrayToHex(config.VolumeDesc.VolumeID)
print 'Drive : %s' % drive


def information():
# Get the pointer and convert to linked-list node
volume_list =
VolumeNameEntry.from_address(ewfapi.EwfMgrGetProtectedVolumeList())
while 1:
drive =
chr(ewfapi.EwfMgrGetDriveLetterFromVolumeName(volume_list.Name))
print '%s: = %s' % (drive, volume_list.Name)
handle = ewfapi.EwfMgrOpenProtected(volume_list.Name)
config =
VolumeConfig.from_address(ewfapi.EwfMgrGetProtectedVolumeConfig(handle))

displayVolumeConfiguration(config, drive)
if
ewfapi.EwfMgrVolumeNameListIsEmpty(ctypes.c_void_p.from_address(ctypes.addr­essof(volume_list))):

break


if __name__ == "__main__ ":
print 'Running'
information()
 
T

Terry Reedy

P Boy said:
I have recently found another approach by using ctypes
(http://starship.python.net/crew/theller/ctypes/).

Ctypes is now a 2.5 standard lib module and in being tested as such on
multiple systems.
I have a library from Microsoft, ewfapi.dll, ewfapi.lib (not required),
and ewfapi.h (from msdn.com)

Has anyone yet written a program to grab C struct declaration from the .h
to produce code like

# Overlay configuration
class OverlayStoreConfig(ctypes.Structure):
_fields_ = [('FormatVersion', ctypes.c_ulong),
('VolumeSize', ctypes.c_longlong),
('NumSegments', ctypes.c_ulong),
('FreeSegments', ctypes.c_ulong),
('SegmentSize', ctypes.c_ulong),
('MaxVolumes', ctypes.c_ulong),
('NumVolumes', ctypes.c_ulong),
('MaxLevels', ctypes.c_ushort),
('VolumeDescArray', pVolumeDesc),
]
?
Would save tedium and errors and make ctypes even more helpful.

Terry Jan Reedy
 
A

Alex Martelli

David Boddie said:
It is a common misconception that SIP is only really used to wrap
Qt-based libraries, though that may be its main use in many projects:

"SIP is a tool that makes it very easy to create Python bindings
for C and C++ libraries. It was originally developed to create PyQt,
the Python bindings for the Qt toolkit, but can be used to create
bindings for any C or C++ library."

Ah, I see that SIP 4 is much better -- I was more experienced with 3.*,
which, what between lack of docs, no support for C, etc, was less
suitable for such a general role. If SIP 4 can be installed quite
independently of Qt, then it's indeed grown to the stature you mention.


Alex
 
P

P Boy

Has anyone yet written a program to grab C struct declaration from the .h
to produce code like

# Overlay configuration
class OverlayStoreConfig(ctypes.Structure):
_fields_ = [('FormatVersion', ctypes.c_ulong),
('VolumeSize', ctypes.c_longlong),
('NumSegments', ctypes.c_ulong),
('FreeSegments', ctypes.c_ulong),
('SegmentSize', ctypes.c_ulong),
('MaxVolumes', ctypes.c_ulong),
('NumVolumes', ctypes.c_ulong),
('MaxLevels', ctypes.c_ushort),
('VolumeDescArray', pVolumeDesc),
]
?

http://starship.python.net/crew/theller/ctypes/codegen.html says it can
be done. However, it requires MSVC 7.1 and I have not yet install it
since I already have MSVC (6,7,8) on my PC.

My header file is relatively small. The manual translation exercise
also allowed me to understand more details about ctypes.
 
T

Terry Reedy

P Boy said:
Has anyone yet written a program to grab C struct declaration from the
.h
to produce code like

# Overlay configuration
class OverlayStoreConfig(ctypes.Structure):
_fields_ = [('FormatVersion', ctypes.c_ulong),
('VolumeSize', ctypes.c_longlong),
etc

http://starship.python.net/crew/theller/ctypes/codegen.html says it can
be done. However, it requires MSVC 7.1 and I have not yet install it
since I already have MSVC (6,7,8) on my PC.

Or vc6. Neither of which most people have, especially those not using Swig
because they don't do C. Ok, we need program entirely in Python. Possible
exercise for someone.
My header file is relatively small. The manual translation exercise
also allowed me to understand more details about ctypes.

Sure, the first time I would do such also.

tjr
 
T

Thomas Heller

Terry said:
P Boy said:
Has anyone yet written a program to grab C struct declaration from the
.h
to produce code like

# Overlay configuration
class OverlayStoreConfig(ctypes.Structure):
_fields_ = [('FormatVersion', ctypes.c_ulong),
('VolumeSize', ctypes.c_longlong),
etc
http://starship.python.net/crew/theller/ctypes/codegen.html says it can
be done. However, it requires MSVC 7.1 and I have not yet install it
since I already have MSVC (6,7,8) on my PC.

Or vc6. Neither of which most people have, especially those not using Swig
because they don't do C. Ok, we need program entirely in Python. Possible
exercise for someone.

Alan Green apparently has got it to work with the free vctoolkit:

http://cardboard.nu/blog/2005_07_14/ctypes_code_generator_for_chea.html


Thomas
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,289
Messages
2,571,448
Members
48,126
Latest member
ToneyChun2

Latest Threads

Top