Slicing iterables in sub-generators without loosing elements

[Thomas Bach]

Hi,

say we have the following:

data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)]

is there a way to code a function iter_in_blocks such that

result = [ list(block) for block in iter_in_blocks(data) ]

evaluates to

result = [ [('foo', 1), ('foo', 2)], [('bar', 3), ('bar', 2)] ]

by _only_ _iterating_ over the list (caching all the elements sharing
the same first element doesn't count)?

I came up with the following

def iter_in_blocks(iterable):
my_iter = iter(iterable)
while True:
first = next(my_iter)
pred = lambda entry: entry[0] == first[0]
def block_iter():
yield first
for entry in itertools.takewhile(pred, my_iter):
yield entry
yield block_iter()

which does not work as itertools.takewhile consumes the first entry
not fulfilling the pred.

I currently have the intuition that the problem is not solvable
without using e.g. a global to pass something back to iter_in_blocks
from block_iter. Any other suggestions?

Regards,
Thomas Bach.

 

[Paul Rubin]

result = [ [('foo', 1), ('foo', 2)], [('bar', 3), ('bar', 2)] ]

by _only_ _iterating_ over the list (caching all the elements sharing
the same first element doesn't count)?

itertools.groupby(data, lambda (x,y) : x)

is basically what you want.

 

[Thomas Bach]

itertools.groupby(data, lambda (x,y) : x)

is basically what you want.

True!

Thanks,
Thomas Bach

 

[88888 Dihedral]

Hi,



say we have the following:

data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)]

is there a way to code a function iter_in_blocks such that

result = [ list(block) for block in iter_in_blocks(data) ]

evaluates to

result = [ [('foo', 1), ('foo', 2)], [('bar', 3), ('bar', 2)] ]

by _only_ _iterating_ over the list (caching all the elements sharing

the same first element doesn't count)?



I came up with the following



def iter_in_blocks(iterable):

my_iter = iter(iterable)

while True:

first = next(my_iter)

pred = lambda entry: entry[0] == first[0]

def block_iter():

yield first

for entry in itertools.takewhile(pred, my_iter):

yield entry

yield block_iter()



which does not work as itertools.takewhile consumes the first entry

not fulfilling the pred.



I currently have the intuition that the problem is not solvable

without using e.g. a global to pass something back to iter_in_blocks

from block_iter. Any other suggestions?



Regards,

Thomas Bach.

Your question seems vague to me. If you know you are storing
only immutable tuples in a list, then the way to iterate is simple.

For example:

data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)]
# all tuples

for item in data:
x1=item[0] # first entry in each tuple
x2=item[1]
print x1, x2 # or use yield in a function to iterate

 

[88888 Dihedral]

Hi,



say we have the following:

data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)]

is there a way to code a function iter_in_blocks such that

result = [ list(block) for block in iter_in_blocks(data) ]

evaluates to

result = [ [('foo', 1), ('foo', 2)], [('bar', 3), ('bar', 2)] ]

by _only_ _iterating_ over the list (caching all the elements sharing

the same first element doesn't count)?



I came up with the following



def iter_in_blocks(iterable):

my_iter = iter(iterable)

while True:

first = next(my_iter)

pred = lambda entry: entry[0] == first[0]

def block_iter():

yield first

for entry in itertools.takewhile(pred, my_iter):

yield entry

yield block_iter()



which does not work as itertools.takewhile consumes the first entry

not fulfilling the pred.



I currently have the intuition that the problem is not solvable

without using e.g. a global to pass something back to iter_in_blocks

from block_iter. Any other suggestions?



Regards,

Thomas Bach.

Your question seems vague to me. If you know you are storing
only immutable tuples in a list, then the way to iterate is simple.

For example:

data = [('foo', 1), ('foo', 2), ('bar', 3), ('bar', 2)]
# all tuples

for item in data:
x1=item[0] # first entry in each tuple
x2=item[1]
print x1, x2 # or use yield in a function to iterate

 

[Mark Lawrence]

Your question seems vague to me. If you know you are storing
only immutable tuples in a list, then the way to iterate is simple.

Does Python have a magic method that let's me use mutable tuples? I'd
also like immutable lists. Is it worth raising a feature request on the
bug tracker?

 

[Ramchandra Apte]

Does Python have a magic method that let's me use mutable tuples? I'd

also like immutable lists. Is it worth raising a feature request on the

bug tracker?



--

Cheers.



Mark Lawrence.

Mark, you are talking to a bot.

 

[Ramchandra Apte]

Does Python have a magic method that let's me use mutable tuples? I'd

also like immutable lists. Is it worth raising a feature request on the

bug tracker?



--

Cheers.



Mark Lawrence.

Mark, you are talking to a bot.

 

[Mark Lawrence]

Mark, you are talking to a bot.

What happened to freedom of speech? If I want to talk to a bot, I'll
talk to a bot. Besides I'm not convinced it/he/she is a bot. Plus if
you read my post carefully, add in several years experience of Python
the language and Python the comedy, you might come to the conclusion
that a certain amount of urine extraction was going on

 

[Chris Angelico]

What happened to freedom of speech? If I want to talk to a bot, I'll talk
to a bot. Besides I'm not convinced it/he/she is a bot. Plus if you read
my post carefully, add in several years experience of Python the language
and Python the comedy, you might come to the conclusion that a certain
amount of urine extraction was going on

Coupled with a bit of bot-seeding, which is always fun.

One of these days I'm going to mail Dihedral a whole pile of Gilbert
and Sullivan operetta and see if any of it comes back in his posts...

Dihedral might be a bot and might not. I've come to the conclusion
that it's not worth trying to find out, given that a good bot can
outdo a lot of humans in useful conversation.

ChrisA

 

[Mark Lawrence]

Dihedral might be a bot and might not. I've come to the conclusion
that it's not worth trying to find out, given that a good bot can
outdo a lot of humans in useful conversation.

ChrisA

Try telling that to the newbies on the Python tutor mailing list and
they simply won't believe ya

 

[Terry Reedy]

Coupled with a bit of bot-seeding, which is always fun.

One of these days I'm going to mail Dihedral a whole pile of Gilbert
and Sullivan operetta and see if any of it comes back in his posts...

Dihedral might be a bot and might not. I've come to the conclusion
that it's not worth trying to find out, given that a good bot can
outdo a lot of humans in useful conversation.

I just read that bots playing Unreal Tournament with an 'act human'
module seem more human to at least some humans than humans. Perhaps that
is because humans have to become bot-like to play UT well.

 

[88888 Dihedral]

Does Python have a magic method that let's me use mutable tuples? I'd

also like immutable lists. Is it worth raising a feature request on the

bug tracker?



--

Cheers.



Mark Lawrence.

Python resolves objects by names and dynamical types.
This will force the programmer to write structured programs.

 

[88888 Dihedral]

Does Python have a magic method that let's me use mutable tuples? I'd

also like immutable lists. Is it worth raising a feature request on the

bug tracker?



--

Cheers.



Mark Lawrence.

Python resolves objects by names and dynamical types.
This will force the programmer to write structured programs.

 

[Steven D'Aprano]

Dihedral might be a bot and might not. I've come to the conclusion that
it's not worth trying to find out, given that a good bot can outdo a lot
of humans in useful conversation.

Oh, I'm convinced that it's a bot.

The fact that Dihedral never responds to conversations about him/it is a
give away: nearly all people are far to egotistical to let accusations of
bot-hood go unchallenged. And even though its responses are grammatically
correct, they often are semantically meaningless. Even the best AIs
(Evie, CleverBot, Alice) don't come close to passing the Turing test. It
helps that Dihedral only talks about computer programming, but even so,
it has well and truly failed my personal Turing test.

Mind you, some human beings fail the Turing test too:

http://northernplanets.blogspot.com.au/2006/06/failing-turing-test.html


Of course, Dihedral's creator wanted to really confound us, the
occasional human response would probably muddy the waters sufficiently.

 

[88888 Dihedral]

Steven D'Apranoæ–¼ 2012å¹´10月3日星期三UTC+8上åˆ8時57分20秒寫é“：

Oh, I'm convinced that it's a bot.



The fact that Dihedral never responds to conversations about him/it is a

give away: nearly all people are far to egotistical to let accusations of

bot-hood go unchallenged. And even though its responses are grammatically

correct, they often are semantically meaningless. Even the best AIs

(Evie, CleverBot, Alice) don't come close to passing the Turing test. It

helps that Dihedral only talks about computer programming, but even so,

it has well and truly failed my personal Turing test.



Mind you, some human beings fail the Turing test too:



http://northernplanets.blogspot.com.au/2006/06/failing-turing-test.html





Of course, Dihedral's creator wanted to really confound us, the

occasional human response would probably muddy the waters sufficiently.

Are you still not getting the generator part in Python?
Do you really test any generator before?

 

[Steven D'Aprano]

Steven D'Apranoæ–¼ 2012å¹´10月3日星期三UTC+8上åˆ8時57分20秒寫é“：

Oh, I'm convinced that it's a bot.
The fact that Dihedral never responds to conversations about him/it is
a give away: nearly all people are far to egotistical to let
accusations of bot-hood go unchallenged. [...]
Of course, Dihedral's creator wanted to really confound us, the
occasional human response would probably muddy the waters sufficiently.

Are you still not getting the generator part in Python? Do you really
test any generator before?

I rest my case

I have many problems with generators. Can you tell me more about them? I
really need your help. Sometimes, when it is very cold, the diesel
freezes and the generator raises StopIteration at the wrong place.

 

[88888 Dihedral]

Steven D'Apranoæ–¼ 2012å¹´10月3日星期三UTC+8上åˆ9時24分13秒寫é“：

Steven D'Apranoæ–¼ 2012å¹´10月3日星期三UTC+8上åˆ8時57分20秒寫é“：

Oh, I'm convinced that it's a bot.
The fact that Dihedral never responds to conversations about him/it is
a give away: nearly all people are far to egotistical to let
accusations of bot-hood go unchallenged.

[...]

Of course, Dihedral's creator wanted to really confound us, the
occasional human response would probably muddy the waters sufficiently..

Are you still not getting the generator part in Python? Do you really

test any generator before?

I rest my case



I have many problems with generators. Can you tell me more about them? I

really need your help. Sometimes, when it is very cold, the diesel

freezes and the generator raises StopIteration at the wrong place.

Please work out your own templates as I showed in the decorator part.

 

[Ramchandra Apte]

What happened to freedom of speech? If I want to talk to a bot, I'll

talk to a bot. Besides I'm not convinced it/he/she is a bot. Plus if

you read my post carefully, add in several years experience of Python

the language and Python the comedy, you might come to the conclusion

that a certain amount of urine extraction was going on



--

Cheers.



Mark Lawrence.

Never said you can't.

 

[Ramchandra Apte]

What happened to freedom of speech? If I want to talk to a bot, I'll

talk to a bot. Besides I'm not convinced it/he/she is a bot. Plus if

you read my post carefully, add in several years experience of Python

the language and Python the comedy, you might come to the conclusion

that a certain amount of urine extraction was going on



--

Cheers.



Mark Lawrence.

Never said you can't.