Distributing methods of a class across multiple files

L

lh

Is this possible please? I have done some searching but it is hard to
narrow down Google searches to this question. What I would like to do
is, for example:
1) define a class Foo in file test.py... give it some methods
2) define a file test2.py which contains a set of methods that are
methods of class Foo defined in test.py. I can import Foo obviously
but it isn't clear to me how to identify the methods in test2.py to be
methods of class Foo defined in test.py (normally I would just indent
them "def"'s under the class but the class isn't textually in
test2.py).

In short I would like to distribute code for one class across multiple
files so a given file doesn't get ridiculously long.

Thank you,
Luke
 
R

Roy Smith

lh said:
Is this possible please? I have done some searching but it is hard to
narrow down Google searches to this question. What I would like to do
is, for example:
1) define a class Foo in file test.py... give it some methods
2) define a file test2.py which contains a set of methods that are
methods of class Foo defined in test.py. I can import Foo obviously
but it isn't clear to me how to identify the methods in test2.py to be
methods of class Foo defined in test.py (normally I would just indent
them "def"'s under the class but the class isn't textually in
test2.py).

In short I would like to distribute code for one class across multiple
files so a given file doesn't get ridiculously long.

The student asks the master, "How long is a file?"

The master replies, "Just long enough to hold its contents".

"But, what if my class is ridiculously long? Won't the file that
contains it also be ridiculously long?"

"Just as one cannot count the grains of sand on a beach, neither can one
count the number of kilobytes one can buy for a few cents at your local
office supply store"

And the student was enlightened and stopped thinking the whole world was
Java or C++.
 
C

Cameron Simpson

| Is this possible please? I have done some searching but it is hard to
| narrow down Google searches to this question. What I would like to do
| is, for example:
| 1) define a class Foo in file test.py... give it some methods
| 2) define a file test2.py which contains a set of methods that are
| methods of class Foo defined in test.py. I can import Foo obviously
| but it isn't clear to me how to identify the methods in test2.py to be
| methods of class Foo defined in test.py (normally I would just indent
| them "def"'s under the class but the class isn't textually in
| test2.py).
|
| In short I would like to distribute code for one class across multiple
| files so a given file doesn't get ridiculously long.

You may need to define "ridiculously long". What's your objecttion to
a long file? What specific difficulties does it cause? I'm not arguing
here that all programs should be in a single monolithic file, just that
breaking things up just on size is a rather arbitrary basis.

If methods supply a particular type of functionality you can write a
mixin class, like this:

from fooey import FooeyClass
from bloohey import BlooheyCLass

class Foo(object, FooeyClass, BlooheyClass):
... core methods of Foo here ...

and then put methods involving Fooeyness in the class definition of
FooeyClass and so forth. But unless you intend to reuse FooeyClass to
augument other classes or FooeyClass really is a well defined standalone
piece of functionality, this is probably more contortation than it is
worth.

Cheers,
--
Cameron Simpson <[email protected]> DoD#743
http://www.cskk.ezoshosting.com/cs/

Hello, my name is Yog-Sothoth, and I'll be your eldritch horror today.
- Heather Keith <[email protected]>
 
F

Frank Millman

lh said:
Is this possible please? I have done some searching but it is hard to
narrow down Google searches to this question. What I would like to do
is, for example:
1) define a class Foo in file test.py... give it some methods
2) define a file test2.py which contains a set of methods that are
methods of class Foo defined in test.py. I can import Foo obviously
but it isn't clear to me how to identify the methods in test2.py to be
methods of class Foo defined in test.py (normally I would just indent
them "def"'s under the class but the class isn't textually in
test2.py).

In short I would like to distribute code for one class across multiple
files so a given file doesn't get ridiculously long.

I take the point of the other responders that it is not a normal thing to
do, but I had a few long but rarely used methods which I wanted to move out
of the main file just to keep the main file tidier. I came up with this
solution, and it seems to work.

In test2.py -

def long_method_1():
pass

def long_method2():
pass

In test.py -

import test2

class Foo:
long_method_1 = test2.long_method_1
long_method_2 = test2.long_method_2

Then in Foo I can refer to self.long_method_1().

HTH

Frank Millman
 
S

Steven D'Aprano

Is this possible please? I have done some searching but it is hard to
narrow down Google searches to this question. What I would like to do
is, for example:
1) define a class Foo in file test.py... give it some methods
2) define a file test2.py which contains a set of methods that are
methods of class Foo defined in test.py.

Technically, yes, this is possible, but you shouldn't need to do it.
Needing to split a single class across multiple files is a sign of bad
design. If the class is that huge, then it probably does too many things
and should be broken up into multiple classes and then reassembled using
composition.



[...]
In short I would like to distribute code for one class across multiple
files so a given file doesn't get ridiculously long.

What do you call "ridiculously long"?

One of the largest modules in the Python standard library is decimal. I
consider decimal to be about as big as a single module should get: over
5000 lines of code. Any larger, and you should consider splitting it into
a package with multiple files.

But note that those 5000 lines include over 500 lines of comments,
details documentation, plenty of blank lines, 14 public classes, 3 public
functions, and at least 17 private functions or classes. The main class,
decimal.Decimal, is about 2800 lines of code.

If your class is smaller than that, I don't think you need to worry about
splitting it.

But let's suppose you really do have a good reason to split the class
into multiple files. And not just "because that's how I'd do it in Java".

Suppose you have a class Spam, and it has two methods, spam() and eggs().
You want to split the methods into different files. (Perhaps you want to
win a bet.) There are three main possibilities:

(1) Inheritance.

(2) Composition or delegation.

(3) Dynamic code injection.


Let's start with inheritance.

In module a.py, create a class:

class EggMixin:
def eggs(self):
c = self.colour
print("%s eggs go well with %s spam" % (c, c))

Notice that as a mixin, EggsMixin isn't required to provide the
self.colour attribute.

Technically it isn't necessary for this to be a mixin, but it is probably
the best design.

Now in module main.py, create the Spam class you really want, using
multiple inheritance to inherit from the "real" superclass and the mixin:

import a
class Spam(SpamParent, a.EggMixin):
def __init__(self, colour='green'):
print("Spam spam spam LOVELY SPAM!!!")
self.colour = colour
def spam(self, n):
return "spam!"*n


By the way, SpamParent is optional. If you don't need it, just leave it
out.


Now, on to composition. First, let's redesign the egg class in a.py:

class Egg:
def __init__(self, owner):
self.owner = owner
def eggs(self):
c = self.owner.colour
print("%s eggs go well with %s spam" % (c, c))


And in main.py:

import a
class Spam(object):
def __init__(self, colour='green'):
print("Spam spam spam LOVELY SPAM!!!")
self.colour = colour
# Create an Egg that has this Spam instance as the owner.
egg = a.Egg(owner=self)
# And store it for later use.
self._egg = egg
def spam(self, n):
return "spam!"*n
def eggs(self):
# Delegate to the saved egg.
return self._egg.eggs()


Last but not least, lets try dynamic code injection. In a.py, create a
function using a placeholder self parameter:


def eggs(self):
c = self.colour
print("%s eggs go well with %s spam" % (c, c))


And here is your main module:


import a
class Spam(object):
def __init__(self, colour='green'):
print("Spam spam spam LOVELY SPAM!!!")
self.colour = colour
def spam(self, n):
return "spam!"*n

# Add the extra method that we want.
Spam.eggs = a.eggs


So now you have three ways of doing something that shouldn't be done :)
 
J

Jean-Michel Pichavant

lh said:
Is this possible please? I have done some searching but it is hard to
narrow down Google searches to this question. What I would like to do
is, for example:
1) define a class Foo in file test.py... give it some methods
2) define a file test2.py which contains a set of methods that are
methods of class Foo defined in test.py. I can import Foo obviously
but it isn't clear to me how to identify the methods in test2.py to be
methods of class Foo defined in test.py (normally I would just indent
them "def"'s under the class but the class isn't textually in
test2.py).

In short I would like to distribute code for one class across multiple
files so a given file doesn't get ridiculously long.

Thank you,
Luke

If the file is ridiculously long, could be that the class has a
ridiculous number of methods. If you spread your class into multiple
files, you will have a ridiculous number of tiny files with ridiculous
names.

My 2 cents : keep 1 class in 1 file, and keep your scopes consistent.

JM
 
L

lh

First, thanks for all the thoughtful replies. I am grateful.
Second, I figured I'd get a lot of judgement about how I really
shouldn't be doing this. Should have pre-empted it :) oh well. There
is a place IMHO for filename as another structuring element to help
humans in search. Also it can be convenient to have two people who are
working on methods for one class that have different directions to
have different files (even if modern tools can handle distinct edits
on the same file by multiple folks).
Third, length. Well 5000 lines eh... I'm nowhere near that guess I can
stick with one file.
Steven thanks especially for your thorough reply about alternatives.
Again I really appreciate the quality replies made here. very
impressed. I had not thought of the mixin approach in particular. I
was just hoping there was something like "extend class Foo" I could
put at the top of the file and then keep writing methods indented
below it. But I will think about some of these alternatives...

Humbly,
Luke
 
N

Neil Cerutti

First, thanks for all the thoughtful replies. I am grateful.
Second, I figured I'd get a lot of judgement about how I really
shouldn't be doing this. Should have pre-empted it :) oh well.
There is a place IMHO for filename as another structuring
element to help humans in search. Also it can be convenient to
have two people who are working on methods for one class that
have different directions to have different files (even if
modern tools can handle distinct edits on the same file by
multiple folks).

Of the three solutions Steven presented, the latter two leave
very strong coupling between the code in your separate files.
This makes working with the files independently impractical.
Stick with mixin classes and pay heed to the Law of Demeter if
you want to de-couple them enough to work on independently.
 
D

Dennis Lee Bieber

Third, length. Well 5000 lines eh... I'm nowhere near that guess I can
stick with one file.

Original Pascal (primarily a teaching language) did not support
separate compilation nor "include" files. Everything was done within one
source file. (Often with a one-pass recursive descent parser)

The old convention I'd learned was to keep functions down to a
(printer) page (classical 6 lines per inch, 11" high, tractor feed -- so
about 60 lines per function -- possibly extend to a second page if
really needed.

I'd think even a large complex class definition could fit each
method into something of that size. Okay, initialization may be long
(especially for GUI layouts), but the operations shouldn't be that
complex (in length; doing an orbit propagation with a J2 geopotential
model may be complex numerically, but as I recall it can fit into less
than two pages of text -- in FORTRAN 77)
 
C

Chris Angelico

Third, length. Well 5000 lines eh... I'm nowhere near that guess I can
stick with one file.

Of all the source files I have at work, the largest is about that, 5K
lines. It gets a little annoying at times (rapid deployment requires
GCC to do its magic on that file), but it's quite manageable. I'd say
you can go even further than that with Python, quite happily.

ChrisA
 
R

Roy Smith

Dennis Lee Bieber said:
The old convention I'd learned was to keep functions down to a
(printer) page (classical 6 lines per inch, 11" high, tractor feed -- so
about 60 lines per function -- possibly extend to a second page if
really needed.

The generalization of that is "keep a function small enough that you can
see it all at once". In the days of line printers and green-bar, that
meant about 60 lines. As we moved to glass ttys, that started to mean
24 lines. These days, with most people having large high-res monitors,
the acceptable limit (by that standard) is growing again. I can easily
get 60 lines on my laptop screen.

On the other hand, back in the neolithic age, when line printers and
mainframes roamed the world, function calls were expensive. People were
encouraged to write larger functions to avoid function calls. Now,
function calls are cheap. Both because optimizing compilers and better
hardware support have made them cheap, and because hardware in general
has gotten so fast nobody cares about it anymore (nor should they).

So, I'd say the driving principle should be that a function should do
one thing. Every function should have an elevator talk. You should be
able to get on an elevator with a function and when you ask it, "So,
what do you do?", it should be able to explain itself before you get to,
"Sorry, that's my floor". If it takes longer than that to explain, then
it's probably several functions fighting to get out.
 
C

Chris Angelico

So, I'd say the driving principle should be that a function should do
one thing.  Every function should have an elevator talk.  You should be
able to get on an elevator with a function and when you ask it, "So,
what do you do?", it should be able to explain itself before you get to,
"Sorry, that's my floor".  If it takes longer than that to explain, then
it's probably several functions fighting to get out.

Agreed, that's definitely a good rule of thumb. But there's still some
value in keeping the code length down too. In a program I maintain at
work (in C++, but the same holds true), some functions have a perfect
elevator talk, yet are quite a few screenfuls of code - this is not
really advisable. For instance, one function is "handle responses to
previously-sent requests". It incorporates all the code for handling
requests of type X, of type Y, and of type Z. If I were writing this
from scratch in Python, I would probably have X, Y, and Z all
separated out, and the checkresponses function would simply be a
dispatcher.

ChrisA
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,989
Messages
2,570,207
Members
46,782
Latest member
ThomasGex

Latest Threads

Top