Incrementally converting a C app to Python

[=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=]

Hello. At work I'm the sole maintainer and developer of a medium sized
~50k line C application. The lion share of this app was written before
my time so it is pretty hard to read and also contain lots of bugs.
I'm contemplating converting the project to Python because I think
that, in the long run, maintenance will become much easier. And also
because is so much more fun than C.

The app is to big to be rewritten from scratch so I'm thinking of a
way to rewrite it "incrementally." Something like one module, or file,
at a time. Using tools like SWIG, Pyrex or embedding Python in C it
seems like this would be possible. My question is what tools should I
apply and what strategy should I use? Would it be best to use a
top-down approach and rewrite the main files first and work my way
down wrapping the remaining C files using SWIG? Or should I convert
the isolated code files to Python and then embedd it in the C code?
Maybe Pyrex help me with this task? Any insight on this matter is
greatly appreciated. Surely someone has done the exact same thing as
me before.

 

[=?iso-8859-1?B?Ry4gTW9uevNu?=]

Hi Bjorn,

I think the best approach is highly-dependent to your current C
application design and "way of doing things". For a type of application
you would take the opposite path than for other I thought.

Yes I did that one time, but I re-coded all from scratch... and since
that I usually do in reverse: code in Python, then optimize if needed
in C via pyrex coded extensions and glue.

If you don't know well the current application internals, both
approaches can be harder: to convert step by step to python or
re-implement from scratch. You need to know well what is doing there
that pretty hard to read code for being sure for success and good
starting guidelines.

I thought If your app doesn't have very critical parts, and if you
really know what the application is doing "inside", I would bet for a
complete reimplementation from scratch. More if you say current C code
has a lots of bugs! And all the code you need to be in C, can be easily
done via pyrex, ctypes, etc. Have in mind I don't know your
application!

Swig is harder, is more suited to export whole libraries to between
languages. Anyway, I'm no expert with Swig!

Regards,
Gonzalo Monzón.

 

[Nick Craig-Wood]

Hello. At work I'm the sole maintainer and developer of a medium sized
~50k line C application. The lion share of this app was written before
my time so it is pretty hard to read and also contain lots of bugs.
I'm contemplating converting the project to Python because I think
that, in the long run, maintenance will become much easier. And also
because is so much more fun than C.

As long as python is fast enough, then go for it!

The app is to big to be rewritten from scratch so I'm thinking of a
way to rewrite it "incrementally." Something like one module, or file,
at a time. Using tools like SWIG, Pyrex or embedding Python in C it
seems like this would be possible. My question is what tools should I
apply and what strategy should I use? Would it be best to use a
top-down approach and rewrite the main files first and work my way
down wrapping the remaining C files using SWIG? Or should I convert
the isolated code files to Python and then embedd it in the C code?
Maybe Pyrex help me with this task? Any insight on this matter is
greatly appreciated. Surely someone has done the exact same thing as
me before.

In my experience it is easier to embed C into python than python into
C. This is because there are lots of tools to help you with embedding
C into python (eg swig or ctypes), but the other way round you are
more on your own.

So I would re-write the main application loop in python, then swig all
the C files and mash them all together. This approach will work well
if the C app is well modularised. If not you'll have to work the
other way round, embedding bits of modularised python into C.

I'd try to convert as much as possible into python in the first pass
though. Anything involving string manipulation, hash lookups, linked
lists etc will be much easier in python and you'll find you'll make 1
line of python for 5-10 lines of C probably - so maybe that 50k LOC
doesn't look so scary as 5k-10k LOC in Python.

I bet your C code doesn't have unit tests either... Make sure you
write those as you are going along in Python. It will make more code,
but it will give you a wonderful feeling of confidence that your code
will actually work, and work the same as the C code.