Does Python really follow its philosophy of "Readability counts"?

[Russ P.]

Also, attribute protection is just a tiny aspect.  The high assurance
community really wants as much static verification as it can possibly
get.  Python doesn't really lend itself to that.

Which is why I was hoping that Python might get enforced data hiding
as well as optional static type declarations that can actually be used
for static verification. But maybe that's all just a pipe dream on my
part -- or "onanism."

 

[Russ P.]

Then -don't- use python. Use some other boring
language. (!@#$!@#)

--JamesMills

You're emailing me again. Please don't do that. It's bad enough to get
death threats in a newsgroup -- I don't need them in my inbox too.
Thanks.

 

[Paul Rubin]

You wouldn't happen to have a source for this information would you ?

Just the url that is there.

And this (if true) hardly suprises me as Ada is one of the most
rigorous and strictest languages I have ever used.

I have no direct experience with it but have read a little about it.
It looks to be a fairly vanilla block structured imperative language
like Pascal with some concurrency stuff added. The sense I have is
that it's not especially harder to program in than C or C++, but the
programs come out much more reliable.

If you want to see something really rigorous and strict, take a look
at ATS: http://ats-lang.org . Here is a certified quicksort in ATS:

http://www.ats-lang.org/EXAMPLE/MISC/listquicksort.dats

The point is that ATS's type system is powerful enough to define a
type for a sorting routine (a function that takes a list as input and
produces a sorted permutation of the input list). If you then write
something that purports to be a sorting routine but fails to produce a
sorted permutation, the compiler will reject the program with a type
error. If your routine compiles at all, it sorts correctly.

 

[alex23]

Looks interesting. If it can somehow be integrated into the language
as full-fledged feature, then I'd say it has potential. As I said
before, I am not looking for a hack or a quick fix. I am interested in
well-engineered data hiding that is fully supported as part of the
language.

I think this is the main issue we disagree on. I'm happier for Python
to remain lightweight where such features can be easily added on
demand through external libraries. I see no reason why a library
couldn't be as "well-engineered" a solution as an extension to the
interpreter, its use being a mandated (and enforced) requirement
within personal projects. The language just has to enable such
solutions, which I feel Python does well.

Perhaps what really concerns me with the inclusion of data hiding into
the language itself is that people would use it I've always enjoyed
the openness of the Python libs, the "consenting adults" approach it
takes, and the ease with which I can override their inherent
functionality when I need to. (But I think we've been over this
before, the whole "the lib developer knows best" vs "I the coder know
best" argument...)

[I am a bit confused though, because I seem to recall that Mr. Banks
claimed earlier in this thread that enforced data hiding is useless.]

Maybe he was just trying to stave off threads like this one

 

[alex23]

Does the metaclass prevent reaching into the __dict__ in an instance?

Yes, if you specify that '__dict__' is one of the private attributes.

Also, attribute protection is just a tiny aspect.  The high assurance
community really wants as much static verification as it can possibly
get.  Python doesn't really lend itself to that.

I can't argue against that. It will be interesting to see what comes
out of the introduction of parameter annotation in 3.0.

 

[Paul Rubin]

Zope is about 375 KLOC[1],

How many LOCS would it require if it was written in ADA ?

No idea. Ada is a lot more verbose than Python, but I'm baffled at
what the heck Zope is actually doing with all that code.

None of which are going to *kill* anyone FWIW. Now how many of these
bugs would have language-enforced access restriction prevented ?

I'm not about to audit 375 KLOC of code to find out. Regardless of
whether access restriction actually prevents bugs though, it certainly
makes auditing easier. And auditing does find bugs.

[2] The Space Shuttle avionics (written in the 1980's!) are 2 MLOC

of a hi-level dynamic language ? Hm, I guess not.

I think written in some forerunner of Ada.

Given the difference in LOCS count between a low-level static language
and a hi-level dynamic language for the implementation of a same given
features set, you cannot just define "large" by the # of LOCS. Not
that I'm going to compare Zope with Space shuttle's stuff.

I don't think static vs dynamic makes much difference in the amount of
LOCS to accomplish some function. In my limited experience with
Haskell (statically typed but very high level), it takes around the
same amount of Haskell code as Python code (sometimes more, sometimes
less) to do a given task. More important for concise code is features
like higher-level functions and types (e.g. it's easy in Python or
Haskell, but a huge pain in Java or C, to write down a value that is a
list of dictionaries whose keys are tuples and whose values are more
lists of dictionaries, to code something list a listcomp, etc).

 

[Russ P.]

I think this is the main issue we disagree on. I'm happier for Python
to remain lightweight where such features can be easily added on
demand through external libraries. I see no reason why a library
couldn't be as "well-engineered" a solution as an extension to the
interpreter, its use being a mandated (and enforced) requirement
within personal projects. The language just has to enable such
solutions, which I feel Python does well.

That could work. As long as it's well-engineered (not a hack), well-
supported (part of the standard library), and does the job, that's
probably all that matters.

 

[Aaron Brady]

Russ: There are SHOULD NOTs and there are MUST NOTs. snip

...     _private = 'spam'
...>>> p = Parrot()

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ProtectionError: attribute is read-only from outside of class Parrot

Would that be so bad? I don't think so.

Sorry, I didn't see the last part originally. I don't think 'outside
of class Parrot' is well-defined in Python. Does '_private' have to
be a member of 'Parrot', an instance of 'Parrot', or the calling
instance of 'Parrot', before entering the calling scope, or at the
time the call is made? Since many of these can change on the fly,
there's more than one consistent interpretation to the syntax.

 

[Bruno Desthuilliers]

Russ P. a écrit :
(snip)

In any case, I have suggested that Python should perhaps get a new
keyword, "private" or "priv".

And quite a few people - most of them using Python daily - answered they
didn't wan't it.

 

[Bruno Desthuilliers]

Russ P. a écrit :
(snip)

Since when is no one is allowed to suggest a potential improvement to
a product unless they are willing to implement it themselves? Imagine
what the world would be like if such a rule applied to all products.

There are two points here. The first one is whether what you suggest is
seen as "a potential improvement", by the language's designers and by
the majority of the language's users. The answer here is obviously "no".
The second point is about your vision of Python as "a product". It is
not - at least, not in the sense of a "commercial product". It's a FOSS
project, and this effectively means that if you really want to get
something done, specially when almost no one seems interested, you *do*
have to DoItYourself(tm). But I *really* doubt it'll make it Python anyway.

IOW : if you want access restrictions in Python, fork the project and do
what you want with your fork (as long as you respect the Python licence
of course).

 

[Bruno Desthuilliers]

Russ P. a écrit :
(snip)

Your mistake for being a moron. But it seems to happen regularly,
doesn't it. How much more of my time are you going to waste, loser?

Calling people names is certainly not the best way to defend your
opinions here. Adios, Mr. P.

 

[Tim Rowe]

I have no direct experience with it but have read a little about it.
It looks to be a fairly vanilla block structured imperative language
like Pascal with some concurrency stuff added. The sense I have is
that it's not especially harder to program in than C or C++, but the
programs come out much more reliable.

It is rather more than that, in that it's particularly strict (and it
has a particularly powerful arithmetic model), but its roots are
certainly there. At a conference on the safe subset, SPARK Ada, one of
the SPARK developers pointed out that the SPARK Ada subset was
effectively Modula2.

Programs done in Ada are, by objective measures, more reliable than
those done in C and C++ (the very best released C++ programs are about
as good as the worst released Ada programs), although I've always
wondered how much of that is because of language differences and how
much is because Ada tends to be used on critical projects that also
tend to get a lot more attention to development standards.

 

[Bruno Desthuilliers]

Paul Rubin a écrit :

Zope is about 375 KLOC[1],

How many LOCS would it require if it was written in ADA ?

No idea. Ada is a lot more verbose than Python, but I'm baffled at
what the heck Zope is actually doing with all that code.

None of which are going to *kill* anyone FWIW. Now how many of these
bugs would have language-enforced access restriction prevented ?

I'm not about to audit 375 KLOC of code to find out. Regardless of
whether access restriction actually prevents bugs though, it certainly
makes auditing easier.

pylint is your friend.

And auditing does find bugs.

[2] The Space Shuttle avionics (written in the 1980's!) are 2 MLOC

of a hi-level dynamic language ? Hm, I guess not.

I think written in some forerunner of Ada.

Yeps, probably something like this. Not specially hi level nor dynamic,
is it ?

I don't think static vs dynamic makes much difference in the amount of
LOCS to accomplish some function.

My own experience is that it does make a huge difference - talking about
imperative languages, of course.

In my limited experience with
Haskell (statically typed but very high level),

"dynamic" and "static" were not meant to concern typing here (or at
least not only typing).

Haskell and MLs are indeed statically typed, but with a powerfull type
inference system, which gives great support for genericity <ot>(hmmm...
is that the appropriate word ?)</ot>

Now these are functional languages, so the notion of "access
restriction" is just moot in this context !-)

it takes around the
same amount of Haskell code as Python code (sometimes more, sometimes
less) to do a given task.

Not exactly my own experience, but I confess I'm much more fluent with
Python than with Haskell !-)

More important for concise code is features
like higher-level functions and types (e.g. it's easy in Python or
Haskell, but a huge pain in Java or C, to write down a value that is a
list of dictionaries whose keys are tuples and whose values are more
lists of dictionaries, to code something list a listcomp, etc).

Yes, indeed. So you agree that Haskell is not exactly a "low level"
language, do you ?-)

Ok, I should probably have made clear I was thinking of a hi-level
dynamic _imperative_ language vs a low-level static _imperative_
language. FP is quite another world.

 

[Mark Wooding]

I'm not sure what you mean by "conflating module and class systems."
Are you referring to the fact that Java requires each class to be in
its own file of the same name (I don't use Java, but that's what I've
heard)? If so, I agree that is a bad idea.

No. I mean that using classes as a unit of access control is wrong. A
class is a unit of behaviour, but that behaviour can (and often should)
come from a number of places.

Common Lisp gets this right. Classes define slots for their instances;
slots are named by symbols. If you can write the symbol, you can access
the slot. But symbols are managed by packages: packages can export some
symbols and keep others internal; and they can import symbols from other
packages. The same mechanism works for functions, variables, classes,
types, macros, and all the other random namespaces that Lisp (like most
languages) has.

Python keeps access control separate from classes. And I think I'd like
it to stay that way.

(Lisp's package system also solves the problem that a class's -- and its
superclasses' -- attributes and methods form a namespace which isn't
well-managed in many languages. Since CL uses symbols for these, and
symbols belong to packages, MDW::MUMBLE isn't the same symbol as
RUSS-P::MUMBLE and so they name different slots.)

-- [mdw]

 

[Mark Wooding]

Which is why I was hoping that Python might get enforced data hiding
as well as optional static type declarations that can actually be used
for static verification. But maybe that's all just a pipe dream on my
part -- or "onanism."

I think that, if you want static assurances, Python -- or any dynamic
language, for that matter -- just isn't for you. Please feel free to
use Java, C#, C++, or more interesting languages like ML variants or
Haskell -- or even Ada dialects, which (I should have mentioned) do seem
to have some very impressive proof-maintenance tools available for them.

-- [mdw]

 

[MRAB]

So anyone thinking beyond an "immediate need" is "engaging in online
onanism"?

That doesn't mean what you think it means.

 

[Luis Zarrabeitia]

That could work. As long as it's well-engineered (not a hack), well-
supported (part of the standard library), and does the job, that's
probably all that matters.

But you keep failing to explay why do you need it to be _part of the standard_
library (or whatever).

If you need it in your project, _use_ it. If you don't, then don't use it. If
_you_ need that thing you call security, just use it already and quit
complaining that we don't use it. Is there a policy in your project that you
can't use any external?

 

[Luis Zarrabeitia]

Oh, I see. Well, that would be ok, except Pylint is not specified as
detecting the types of access that Russ is concerned with. It can't,
for example, flag uses of setattr that might affect a particular
class. That would take something a lot fancier.

True.
And I doubt that the C++ compiler will flag the pointers running wildly
pointing to random memory addresses and accessing the data in there. I doubt
that pointer will 'respect' the 'private' keyword. I also doubt that the C#
or Java _compiler_ will prevent you from using reflection somehow to access
that data.

But somehow the discussion shifted from an optional requirement (giving you
the chance to explicitly use 'from lock import unlock; o = unlock(obj)')
to "it can't be done _ever_" (using setattr/getattr is as explicit as your
analogous 'unlock' function).

Btw, the correctness of a program (on a turing-complete language) cannot be
statically proven. Ask Turing about it.

 

[Paul Rubin]

But somehow the discussion shifted from an optional requirement (giving you
the chance to explicitly use 'from lock import unlock; o = unlock(obj)')
to "it can't be done _ever_" (using setattr/getattr is as explicit as your
analogous 'unlock' function).

The idea was the lock would be on the class, not on instances of the
class. So setattr/getattr would work on instances of unlocked classes
but not locked ones. If you wanted an unlocked instance of a locked
class, you could do it by making an unlocked subclass.

Btw, the correctness of a program (on a turing-complete language) cannot be
statically proven. Ask Turing about it.

That's an irrelevant red herring, even if you're after provability.
It just means that there exist some correct programs that cannot be
proven correct. What you care about is whether there is a provable
program for some particular computing task, which there almost always
is. It doesn't matter if there are also some unprovable ones.

That is just common sense. When you write a program you presumably
write it using some reasoning that lets you convince yourself that the
program should always work, and that's an informal proof. (The idea of static
verification is to find the small gaps of reasoning that humans make
all the time, but once found these gaps are usually easily fixed).

I guess one occasionally writes programs that rely on unproven and
maybe unprovable mathematical conjectures, but that is pretty rare,
especially in traditional programming where you only care about
getting correct values out. Even in areas which do rely on unproven
conjectures (e.g. in cryptography, we rely on the conjecture that
certain functions are difficult to invert), it's usually possible to
explicitly state the conjectures that you depend on, and prove that
"if this conjecture is true then the program is correct".

 

[Luis Zarrabeitia]

The idea was the lock would be on the class, not on instances of the
class.  So setattr/getattr would work on instances of unlocked classes
but not locked ones.  If you wanted an unlocked instance of a locked
class, you could do it by making an unlocked subclass.

Well, then, go change pylint to do that. But don't try to force it on _me_: If
I want to access an attribute for an object on my program (from wherever it
came from), I don't want _you_ or anyone else saying "no, its private".

Even better. Realize that you are trying to use a tool made for debugging,
documenting, and catching some obvious errors (private/public)
for "security". Go fix Bastion, that would be your best bet. It was yanked
out of python because it was insecure, but if someone were to fix it, I'm
sure the changes would be welcomed. And then, it would be my choice whether
to let you lock your instances in _my_ program, and it would be yours to lock
all of mine in yours (and way more).

Btw, when I was programming in .Net (long time ago), I found a little library
that I wanted to use. The catch: it wasn't opensource, it had a gratis
version, and a full, payed version... The difference? The full one had some
attributes public that were private on the free one. Well, I didn't wanted
code on my app from someone who didn't know the meaning of the 'private'
keyword anyway. You (well, Russ more than you) almost seem to be trying
something like that.

That's an irrelevant red herring,

True, my bad.

But I don't think it is _fully_ irrelevant. See, a thread that begun with
saying that a piece of code was hard to read has evolved to changing the
language so we could have tools to statically proving "certain things" with
the code. And each time a "suggestion" appears (from people that, at least at
the time of suggesting it, were genuinely interested on helping) that doesn't
fulfill your [plural, including russ] goal of having static typing and data
hiding in the official python, you shoot it down because some other "it
cannot be proven statically" reason. Sure, it is always a very well
defined "certain" thing and not correctness in general, but that "certain"
thing changes as soon as a suggestion is made to amend it.

I'm obviously growing tired of that pattern.

Have pylint check if someone uses getattr in your code, at all. If pylint
doesn't do it, just grep for it and don't run that code. Simple.

Btw, this may put and end to this discussion:

import gc
gc.get_objects()

No amount of metaclasses, pylints, decorator, etc, will stop someone from
using that to get access to _all_ objects of the program. If you need to keep
a secret, don't put it on the same memory space that the untrusted code in
python (actually, don't do it on any language). If you need static proofs,
use a less dynamic subset of python, and use or develop tools to check if you
tried to get out, ever (because if you get out once, you will not be able to
track the results). Or, don't use python. (and certainly don't use C++, for
that matter, even if C++ has a 'private' keyword).