Python & Go

[sturlamolden]

I'm just learning about Google's latest: the GO (Go?) language.
(e.g.

).
There are some distinctly Pythonoid features to the syntax, such
as "import this_or_that", the absence of parentheses at the top of
flow control constructs, and quite a few statements without a
trailing semicolon.  Then again, there's a lot that looks distinctly
un-Pythonlike, such as the curly brackets all over the place.  And
among the un-Pythonlike stuff there's a lot that looks like nothing
else that I've ever seen...

It seems that the argument for using Go over Python is speed. They
achieve that by static typing. According to Debian benchmarks, LuaJIT
delivers the same performance as Google cleaims for Go (about 20% loss
compared to C), and that is from a completely dynamic language. (Lua
is Python's sibling.) It makes me wonder what Python would be like on
LuaJIT.

Perhaps we should make a Python frontend for the Lua VM and try? It
would be fun

Anyway, the impressive performance of LuaJIT on Debian benchmarks just
tells med that static typing a la Go is not needed for fast execution
of CPU-bound code.

And looking at Go, I cannot understand why Google prefer this over
e.g. Lua.

 

[Michele Simionato]

The two goals of replacing C with "something more modern" and at
the same time have a "nearly zero learning curve" seem to me mutually
negating.  The closer to zero the learning curve is, the closer to
C/C++, and therefore the less modern, that language will be.

Not at all. A language with a small learning curve should be as far as
possible for C++!

Go is in many ways simpler than C (no header files, a simpler
compilation process, no pointer arithmetic, no ternary operator, no
while loop, in a sense no threads, etc) and it has an object
orientation simpler than most languages; actually it looks even
simpler than Python in many respects (no properties, no decorators, no
metaclasses,
a much simpler version of inheritance ...). It has static typing that
makes things a bit more complicated, but also safer in same respect.
It has also reflection and the ability to do a lot of things at
runtime. If people starts writing libraries it has the potential to
cover both C and Python niches at the same time!

 

[John Nagle]

It's interesting. The semantics are closer to Java than any other
mainstream language. While Java usually is run with a "virtual machine",
Go is more like Java hard-compiled (which you can do with GCC.)

It's far less dynamic than Python, which is good for performance.
(That's not an inherent problem with Python. It's a problem with the
CPython implementation. See Shed Skin.)

The declaration syntax is borrowed from the Pascal/Modula/Ada
family of languages, and it's an improvement over the C/C++ approach.
I suspect that Go is LALR(1), which means it can be parsed with a
simple context-independent parser. C and C++ are very difficult to
parse, and modules can't be parsed independently. (Because of the
C/C++ type syntax, you have to look up type names to find out how
to parse declarations. So the include files have to be present
to parse declarations. This is why there aren't many tools that
manipulate C and C++ source code.)

Having concurrency in the language is a win. Syntax for queues
is a minor win. But the language doesn't directly address the issue of "who
locks what". There are shared variables, and mutexes, but the language doesn't
let you talk about which variables are shared. When the language doesn't
know that, you either have to restrict optimization (as in Python) or have
painful mechanisms like "mutable" as in C++.

Leaving out exceptions was a mistake. Exceptions are well understood now,
and they're far better than the usual "ignore errors" approach one sees in lamer
C programs.

The interface mechanism is simple enough. In a static language, you can
convert "duck typing" to inheritance at link time, when you have the chance
to see what can match what. So the implementation doesn't actually have to
search for a match at run time.

John Nagle

 

[Paul Rubin]

And looking at Go, I cannot understand why Google prefer this over
e.g. Lua.

I thought Lua had no type system and no concurrency.

 

[Paul Rubin]

One more thing: I found Rob Pike's mutterings on generics (towards
the end of his rollout video) rather offputting, because he gave
the impression that some important aspects of the language were
not even considered before major decisions for it were set in stone.
It looks like, if they ever get around to supporting generics, it
will be a late-in-the-day hack.

Mark Chu-Carroll has a new post about Go:

http://scienceblogs.com/goodmath/2009/11/the_go_i_forgot_concurrency_an.php

Someone named Mightybyte also has a post about Go, and its comment
thread is similarly pretty good:

http://softwaresimply.blogspot.com/2009/11/standardizing-go.html

One of the commenters wrote something that I felt was similar to my
own impression of the language:

From a PLT perspective Go isn't that interesting. The concurrency and
"goroutines" look promising for possibly being able to do what Erlang
does. The type system is pretty weak. From a practical perspective
people are singing praise for how fast it compiles, but not much
mention so far of how fast the resulting code is. Compare tinycc vs
gcc for speed of compilation vs resulting code performace to see why
I'm skeptical.

 

[sturlamolden]

Syntax for queues is a minor win.

No, that's syntax bloat.

The go keyword could be a problem as well. I suspect it could infringe
on Cilk++ patents. Perhaps Go cannot be used without a licence from
Cilk Arts?

 

[Paul Rubin]

The go keyword could be a problem as well. I suspect it could infringe
on Cilk++ patents. Perhaps Go cannot be used without a licence from
Cilk Arts?

Also as somebody said, if after a while they decide to make a new
version of the language, they'll have to call it Go2, which will
necessarily be considered harmful.

 

[Terry Reedy]

Mark Chu-Carroll has a new post about Go:

http://scienceblogs.com/goodmath/2009/11/the_go_i_forgot_concurrency_an.php

In a couple of minutes, I wrote his toy prime filter example in Python,
mostly from the text rather than the code, which I can barely stand to
read. It ran the first time without error.

def plurals():
i = 2
while True:
yield i
i += 1

def primefilter(src, prime):
for i in src:
if i % prime:
yield i

src = plurals()
while True:
i = next(src)
print(i)
src = primefilter(src, i)

As I commented there
"It stopped at 7877 when it hit the default recursion limit of 1000,
which could easily be increased to get out-of-memory error instead.

I think Google is making a blunder if it moves to another old-fashioned
language whose code is bloated with junky boilerplate that doubles the
size. It would be much better, for instance, to tweak Python, which it
has had great success with, to better run on multiple cores."

Terry Jan Reedy

 

[Terry Reedy]

On Sun, Nov 15, 2009 at 12:10 AM, Terry Reedy <[email protected]

Paul Rubin wrote:

Mark Chu-Carroll has a new post about Go:

http://scienceblogs.com/goodmath/2009/11/the_go_i_forgot_concurrency_an.php


In a couple of minutes, I wrote his toy prime filter example in
Python, mostly from the text rather than the code, which I can
barely stand to read. It ran the first time without error.


Yes, but the cool thing about the Go version is that it does each
generator in a different thread, so in theory it could run twice as fast
on a multi-core machine.

Which is why I added, in my opinion, that "It would be much better, for
instance, to tweak Python, which it has had great success with, to
better run on multiple cores."

For instance, add a new keyword 'go' such that

go def f(): yield 1

runs the generator in a different thread, possibly on a different core.

To go further, restrict Python's dynamism, require 3.x annotations, and
call the result GoPython ;-).

Actually, it is already possible to post-process code objects to, in
effect, remove the effect of many dynamic assumptions
http://code.activestate.com/recipes/277940/

Perhaps, with a different implementation, not even a keyword is needed,
just a built-in decorator:

@go
def f(): yield 1

The go decorator would replace f with a wrapper that runs instances gf
of f in threads or whatever, calls next(gf) immediately for parallel
opereation, and caches the first yielded value until the calling
function calls next(wrapper) to retrieve it.

It seems to me that generators are already 'channels' that connect the
calling code to the __next__ method, a semi-coroutine based on the body
of the generator function. At present, the next method waits until an
object is requested. Then it goes into action, yields an object, and
rests again. For parallel operations, we need eager, anticipatory
evaluation that produces things that *will* be needed rather than lazy
evaluation of things that *are* needed and whose absence is holding up
everything else.

I see no reason why we cannot have that with Python. I not even sure we
cannot have it with CPython, but I am not familiar enough with threads,
processes, and CPython internals.

Terry Jan Reedy

 

[Steven D'Aprano]

In <[email protected]> Paul Rubin


Fast compilation also means that Go can conceivably become an attractive
alternative to interpreted languages, because the compilation stage can
be made as unobtrusive as, say, Python's byte-compilation stage (except
that the Go compiler is generating native code).

Python (like many other languages) already has unobtrusive compilation.
That's why you get .pyc files, and that's what the compile() built-in
function does. It is compilation to byte-code rather than machine-code,
but still.

Psyco does JIT compilation to machine-code for CPython, at the cost of
much extra memory. It's also limited to 32-bit Intel processors. The aim
of the PyPy project is to (eventually) make JIT machine-code compilation
available to any Python, on any machine.

 

[Michele Simionato]

     Leaving out exceptions was a mistake.  Exceptions are well understood now,
and they're far better than the usual "ignore errors" approach one sees in lamer
C programs.

I am also surprised about the lack of exceptions. I could infer that
Rob Pike and Ken Thompson are idiots that lack experience with
languages with exceptions, or I could infer that they have reasons for
doing so. I do not know about exceptions enough to have an opinion
myself. However I will notice that in Python, when using threads,
exceptions do not work so well: if I forget to trap an exception in a
thread I see a traceback on stderr, but the other threads continue to
run, basically ignoring the exception. Probably the language that get
things right is Erlang, which is supervising all crashed process and
it is designed to safely recover for unexpected events.

 

[Michele Simionato]

It seems to me that generators are already 'channels' that connect the
calling code to the __next__ method, a semi-coroutine based on the body
of the generator function. At present, the next method waits until an
object is requested. Then it goes into action, yields an object, and
rests again.

I see no reason why we cannot have that with Python. I not even sure we
cannot have it with CPython, but I am not familiar enough with threads,
processes, and CPython internals.

Of course we can have Go capabilities with Python. We already have
generators and the multiprocessing module. It is just a matter of
performance: I assume the Go implementation is more efficient. It
would be nice to have numbers to quantify this claim. One can still
write prototypes in Python and convert them in Go and the process
looks less cumbersome than converting them in C or C++. I could never
force myself to write C or C++; but I do not have any particular
resistence to coding in Go, should I need a performance-oriented
language.

 

[Michele Simionato]

Let me add a quote from the FAQ:

"""
Why does Go not have exceptions?

Exceptions are a similar story. A number of designs for exceptions
have been proposed but each adds significant complexity to the
language and run-time. By their very nature, exceptions span functions
and perhaps even goroutines; they have wide-ranging implications.
There is also concern about the effect they would have on the
libraries. They are, by definition, exceptional yet experience with
other languages that support them show they have profound effect on
library and interface specification. It would be nice to find a design
that allows them to be truly exceptional without encouraging common
errors to turn into special control flow that requires every
programmer to compensate.

Like generics, exceptions remain an open issue.
"""

 

[Steve Howell]

One more thing: I found Rob Pike's mutterings on generics (towards
the end of his rollout video) rather offputting, because he gave
the impression that some important aspects of the language were
not even considered before major decisions for it were set in stone.
It looks like, if they ever get around to supporting generics, it
will be a late-in-the-day hack.

By "set in stone," do you mean "implemented"? Or have Rob Pike and
friends literally put a redesign freeze on the language that was just
released this month?

 

[sturlamolden]

Psyco does JIT compilation to machine-code for CPython, at the cost of
much extra memory. It's also limited to 32-bit Intel processors. The aim
of the PyPy project is to (eventually) make JIT machine-code compilation
available to any Python, on any machine.

Who wants a JIT when it's orders of magnitude slower than interpreted
Python?

 

[Simon Forman]

In a couple of minutes, I wrote his toy prime filter example in Python,
mostly from the text rather than the code, which I can barely stand to read.
It ran the first time without error.

def plurals():
 i = 2
 while True:
   yield i
   i += 1

def primefilter(src, prime):
 for i in src:
   if i % prime:
     yield i

src = plurals()
while True:
 i = next(src)
 print(i)
 src = primefilter(src, i)

As I commented there
"It stopped at 7877 when it hit the default recursion limit of 1000, which
could easily be increased to get out-of-memory error instead.

I think Google is making a blunder if it moves to another old-fashioned
language whose code is bloated with junky boilerplate that doubles the size.
It would be much better, for instance, to tweak Python, which it has had
great success with, to better run on multiple cores."

Terry Jan Reedy

FWIW,

def plurals():
i = 3
while True:
yield i
i += 2

 

[sturlamolden]

It would be much better, for instance, to tweak Python, which it
has had great success with, to better run on multiple cores."

Python run well on multiple cores, you just have to use processes
instead of threads.

 

[Terry Reedy]

FWIW,

def plurals():
i = 3
while True:
yield i
i += 2

Of course, in fact, I thought of at the time

def plurals():
i = 6
while True:
yield i-1
yield i+1
i += 6

5,7, 11,13, 17,19, 23,(25-first miss), 29,31, (35-2nd miss),37, 41,43,
47,(49 erd)...Reduced the base cases by another 1/3. I *think* without
measuring, that that compensates for the extra -1,+1. But I was
implementing *Mark's* toy example, and first posted it on his blog, so I
copied his naive version.

Even better, to avoid extra -1, +1

def plurals():
yield 2
yield 3
i = 5
while True:
yield i
i += 2
yield i
i += 4

Of course, testing with primes greater that square root(candidate) is
wasteful, and recusion should be converted to iteration to avoid
recursion limit.

Terry Jan Reedy

 

[Terry Reedy]

Python run well on multiple cores, you just have to use processes
instead of threads.

But not so trivially as to add one word to an existing function.
Hence by tweak, I meant, as explained in another post, to add a keyword
or just a decorator that will do what the go keyword does in go.

 

[sturlamolden]

But not so trivially as to add one word to an existing function.
Hence by tweak, I meant, as explained in another post, to add a keyword
or just a decorator that will do what the go keyword does in go.

A decorator function like @go could just call os.fork and run the
function in the child. We already have a between-process Queue in
multiprocessing to use as channels.

Or we could have a context manager that forks in __enter__ and
waitpids or exits in __exit__. The body of the with-statement would
then be executed in the child process, the parent would just raise an
exception to skip directly to __exit__.

Not rocket science as all. Yes that would give us a new isolated
process, but such isolation is present in Erlang as well.

(Windows is more tricky though, as there is no efficent fork.)