Integrated Testing - Peppable?

[carl.manaster]

Hi,

Please excuse the intrusion from an admirer, but not a user, of Python.

I've got an idea that I think could improve the language and might be
relatively simple to implement. I've developed a prototype with a toy
language here: <https://sourceforge.net/projects/zbt/>, with screenshot
here: <https://sourceforge.net/project/screenshots.php?group_id=130278>

The idea is that if the language had testing built into it, editors
could take advantage of that to highlight, not by syntax, but by
correctness (as defined by the tests). As a side effect, coverage
coloring is also possible.

So far, I see the need for three keywords: "expect", "returns", and
"archetype". "expect" and "returns" get used together, as an
expression of what the function should return, given certain inputs.
"archetype" would define edit-type instantiable objects that could be
used in the tests. The biggest lack at present is support for void
functions.

So... Should I turn this into a PEP?

Peace,
--Carl

 

[Jeremy Bowers]

So... Should I turn this into a PEP?

I would think a much more productive step one would be to put together the
proposed functionality with unittest and the trace module, and use the
output of your tool to drive some sort of simple output showing what you
are talking about. (Putting together a Tk app to show coverage would be
pretty simple as long as you don't try to get too fancy.)

I'm not too optimistic about ever seeing this in the language, but without
an implementation, I think it's about 0 likelihood. But remember, that's
just my opinion.

You need the experience you'll get from the implementation to write a PEP
that has a prayer of acceptance.

At least with a tool, worst case scenario, your PEP is rejected and you
put together a module to do it anyways. If you get enough people to use
it, you might at least get the module into the standard lib, though that
is also another big step.

No prose can compare to a live, functional demonstration. I, for instance,
am not *certain* this would be useful, but I think it is the sort of thing
that if I got a taste of it I might fall in love with. (Coverage is, of
course, not a sufficient condition to know your testing is complete, but
it is a necessary one, and a graphical readout with uncovered lines
colored bright red could be quite interesting and useful, even if it isn't
initially integrated into anything.)

 

[John Roth]

Hi,

Please excuse the intrusion from an admirer, but not a user, of Python.

I've got an idea that I think could improve the language and might be
relatively simple to implement. I've developed a prototype with a toy
language here: <https://sourceforge.net/projects/zbt/>, with screenshot
here: <https://sourceforge.net/project/screenshots.php?group_id=130278>

The idea is that if the language had testing built into it, editors
could take advantage of that to highlight, not by syntax, but by
correctness (as defined by the tests). As a side effect, coverage
coloring is also possible.

So far, I see the need for three keywords: "expect", "returns", and
"archetype". "expect" and "returns" get used together, as an
expression of what the function should return, given certain inputs.
"archetype" would define edit-type instantiable objects that could be
used in the tests. The biggest lack at present is support for void
functions.

So... Should I turn this into a PEP?

I'd say no, for a couple of reasons. One is that there
are Eclipse plugins that do this for JUnit; they don't
require any changes to Java in order to function. They
just require JUnit, which is pretty ubuquitous.

Second, is that the IDEs aren't part of Python proper.

Outside of that, it's might be quite a good idea to
do something similar with unittest, doctest or py.test.

John Roth

 

[carl.manaster]

Thanks, Jeremy,

No prose can compare to a live, functional demonstration.

I agree; that's what my prototype amounts to:

<http://prdownloads.sourceforge.net/zbt/zbt.zip?download>

(sorry, it's Windows-only)

But I take your larger message, and John's, to heart - this isn't
worthy of making it into Python. OK; thanks anyway for taking the time
to think about it and reply.

Peace,
--Carl

 

[carl.manaster]

Thanks, John,

I think there's a true, powerful, difference between inline tests and
external (albeit adjacent) ones. I think the developer experience
should be superior with inline tests for several reasons:
- during initial development, without any navigation beyond the Enter
key, you flow directly from writing the test to writing the code that
implements it;
- later, on review, the tests have not moved away from the code they
test, so:
----if there are errors, the alarms show up right beside the erroneous
code;
----if you are looking for examples of how to call a routine, the
example is right there.

But I take your point; this doesn't belong in Python. OK. Thanks.

Peace,
--Carl

 

[Jeremy Bowers]

Thanks, Jeremy,


I agree; that's what my prototype amounts to:

<http://prdownloads.sourceforge.net/zbt/zbt.zip?download>

(sorry, it's Windows-only)

But I take your larger message, and John's, to heart - this isn't
worthy of making it into Python. OK; thanks anyway for taking the time
to think about it and reply.

But I do encourage you, if you believe in this, to continue working on it
as a module. Unfortunately, I don't run Windows so if it's Windows only I
can't really look at that. I would certainly be interested in at least
trying it if you ever got a more generally cross-platform release. Be sure
to ANN: ounce it here, if you do.

As I just wrote in another message (although stated another way), "useful"
and "in the standard library" aren't necessarily related!