Possible improvement to slice opperations.

[Ron Adam]

It's a common misconception that all ideas should be explainable simply.
This is not necessarily the case, of course. When a subject is difficult
then all sorts of people bring their specific misconceptions to the
topic, and suggest that if only a few changes were made the whole thing
would be easier to understand.

What misconception do you think I have?

Unfortunately, while that might make the topic in question easier to
understand for some it would make it difficult, and even
counter-intuitive, for others.

True, and that's why asking and getting opinions on a subject is a good
idea.

As many have said before me, there's a reason why slicing and
indexing are the way they are. The interfaces were designed by people
with long years of programming and teaching experience.

No one has yet explained the reasoning (vs the mechanics) of the
returned value of the following.

L = range(10)
L[3::-1]

So far every attempt to explain it has either quoted the documents which
don't address that particular case, or assumed I'm misunderstanding
something, or implied it isn't neccisary to do.

It's quite easy to get me to change my mind on something, just show me a
convincing explanation and I will.

[...]
You said it quite well yourself:

It's easy to think you understand something when you don't. I spend
quite a while figuring this out, And am sure about how it works. If
there are issues with this, then it will probably be in how I describe
it, what words or terminology is used, and weather or not it's the
proper approach.

There are actually two distinct proposals here, not just one.

1. Fixing negative strides so they return the slice indexed as
you say they should.

2. Using base one negative index's and picking item from the
right of negative index's instead of the right.


They don't both need to implemented, Item 1 could be fixed in 2.5.

Given that Python has a 1's-complement operator already I don;t see why
you can't just leave Python alone and use it, since it seems to keep you
happy. If "fixing" item 1 in 2.5 would induce major code breakage,
there's less than a snowball's chance in hell it will happen.

I doubt fixing item (1) would induce major code breakage. As near as I
can tell, it's rarely, (nearly never), used in forms other than L[::-1].

Item (2) however would require a lot of small changes in index's.
Mostly changing.. L[:-1] to L[:~1] or to L[:-2]. So no, I don't expect
one's based indexing to be added any time soon. It could be useful as a
function or object in it's own.

"Professor Einstein, could you tell our readers how general relativity
works?"

Actually I can, but it would be off topic for this news group.

Cheers,
Ron

 

[Ron Adam]

OTOH, ISTM we must be careful not to label an alternate alpha-version
"way to model the real world" as a "misunderstanding" just because it is alpha,
and bugs are apparent ;-)

Thanks! I couldn't have said this myself.

BTW, it just occurred to me that alternate slice member semantics could be RELATIVE,
EACH depending on sign. I.e.,

x[start:stop:step]

could mean start with x[start] as now as a starting point reference point element,
then let stop be a relative range length in either direction from the
starting point, and step absolute value indicating step size, and its sign
indicating insertion side for zero length slices. In effect, stop would
be a count for the slice range in either direction starting with the starting
element

s = 'abcde'
s[2:2] => 'cd'
s[2:-2] => 'cb'
s[-2:-3] => 'dcb'
s[-2:0] => ''
s[2:0] => ''
s[-2:-3:2] => 'db'
r = range(10)
r[5:0] = 'a'
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 8, 9]
r[-2:0:-1] = 'b'
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 'b', 8, 9]
r[-2:0] = ['c', 'd']
r => [0, 1, 2, 3, 4, 5, 'a', 6, 7, 'b', 8, c, d, 9]

Interesting, so it would be...

slice( position, count, step )

Items are addressed directly so there's no 'gap's to account for.

note that slice assignment would work by iterating through the right hand
sequence, which could do interesting things:

r = range(6)
r[3:-2] = 'ab'
r => [0, 1, 'b', 'a', 4, 5]
but two reverse relative slices match order, so
r = range(10)
r[5:-3] = range(10)[-1:-3] # rh seq is 9, 8
r => [0, 1, 8, 9, 4, 5]

I think this is kind of interesting, and I probably wouldn't have thought of
it if I had not been for Ron's enthusiasm for his "misunderstanding" ;-)

In a way, "misunderstandings" are the mutations of open source evolution of ideas,
most of which die, but some of which mutate again and may occasionally survive.
So we need them ;-)

Here's another possible "misunderstanding".

(or alternative alpha-version) ;-)


Have you thought of using a slice object as a buffer pointer for
indirect operations?

r = range(100)
d = [10:20]
r.range_add(d,5) # Adds 5 to items in range d

d = d[5:] -> [15:20] # relative range modification.
# slice of a slice object

r.range_sub(d,3) # subtract 3 from items in range d

Or more flexible ...
r.range_modify(d, add(), 5)

Using your suggestion that would be...

r = range(100)
d = [10:10]
r.range_add(d,5)

d = d[5:] -> [15:5] # interesting symmetry.
r.range_sub(d,3)

Of course adding and subtracting slice objects could also be possible.

d = [10:20]
e = [15:25]
f = d + e -> [10:25]

or ...

d = [10:10]
e = [15:10]
f = d + e -> [10:15]


Cheers,
Ron

 

[Bengt Richter]

in my post, "the real world" is the existing Python implementation. what
theoretical construct are you discussing?

I guess I was talking about a "real world" of abstractions (if that is not an oxymoron ;-)
re sequences and slice access methods, of which the existing python implementation
provides a concrete example of one "model" and Ron's efforts provide some attempts at
an alternative "model". Of course, as far as the concrete "real world" goes, python does what
it does, and a model of _that_ that doesn't fit is a real misunderstanding ;-)

BTW, how did relative slice semantics strike you? They could live along side normal ones
by prefixing the slice brackets with a '.', like r.[3:-2] for 2 (abs(-2)) elements
leftwards(-2<0) starting with r[3]. The logic for None defaults and non-1 steps and
guaranteeing legal ranges is messy, but otherwise UIAM

r.[start:count] == r[start:start+count:-(count<0) or 1]

allowing signed start and count values. IMO r[start] is easy for newbies
to understand as a starting element, whether from end or beginning, and
an absolute value of count is easy, with the sign saying which direction
to scan from the start element, irrespective of how the latter was specified.

Regards,
Bengt Richter

 

[Terry Reedy]

To repeat, the current reason is compatibility with the original design for
NumPy. Perhaps there is some explanation for that in sliceobject.c (or
whatever the file is called), the CVS checkin messages, or the newsgroup
archives before the first checkin (in 1996, I believe).

Terry J. Reedy

 

[Steve Holden]

Steve Holden wrote:




What misconception do you think I have?

This was not an ad hominem attack but a commentary on many attempts to
"improve" the language.

Unfortunately, while that might make the topic in question easier to
understand for some it would make it difficult, and even
counter-intuitive, for others.

True, and that's why asking and getting opinions on a subject is a good
idea.

As many have said before me, there's a reason why slicing and
indexing are the way they are. The interfaces were designed by people
with long years of programming and teaching experience.

No one has yet explained the reasoning (vs the mechanics) of the
returned value of the following.

L = range(10)
L[3::-1]

So far every attempt to explain it has either quoted the documents which
don't address that particular case, or assumed I'm misunderstanding
something, or implied it isn't neccisary to do.

It's quite easy to get me to change my mind on something, just show me a
convincing explanation and I will.

>>> L[3::-1] [3, 2, 1, 0]
>>> L[3::1] [3, 4, 5, 6, 7, 8, 9]
>>>

I don;t see the problem here. The start specifies which element is the
first in the slice, the stop is the default (end of the sequence) and
the stride is "successive elements to the left" when it's -1 and
"successive elements to the right" when it's 1.

Or perhaps you can tell me what I've missed?

[...]
You said it quite well yourself:

It's easy to think you understand something when you don't. I spend
quite a while figuring this out, And am sure about how it works. If
there are issues with this, then it will probably be in how I describe
it, what words or terminology is used, and weather or not it's the
proper approach.

There are actually two distinct proposals here, not just one.

1. Fixing negative strides so they return the slice indexed as
you say they should.

2. Using base one negative index's and picking item from the
right of negative index's instead of the right.


They don't both need to implemented, Item 1 could be fixed in 2.5.

Given that Python has a 1's-complement operator already I don;t see why
you can't just leave Python alone and use it, since it seems to keep you
happy. If "fixing" item 1 in 2.5 would induce major code breakage,
there's less than a snowball's chance in hell it will happen.

I doubt fixing item (1) would induce major code breakage. As near as I
can tell, it's rarely, (nearly never), used in forms other than L[::-1].

Well, I don't see the problem yet, so I don't actually think that needs
"fixing".

Item (2) however would require a lot of small changes in index's.
Mostly changing.. L[:-1] to L[:~1] or to L[:-2]. So no, I don't expect
one's based indexing to be added any time soon. It could be useful as a
function or object in it's own.

My point was that you can make those changes in your own code, leaving
others to accept the situation as it is.

Actually I can, but it would be off topic for this news group.

Right, I wasn't trying to suggest that you didn't know what you were
talking about - or even that you didn't understand general relativity
(on which my grasp could be said to be tenuous) - merely that some
things are inherently difficult, and no matter how you twist the
implementations about, the difficulties will remain.

regards
Steve

 

[Bengt Richter]

What's the meaning of the 1's complement operator (for example, what
is ~1), when ints and longs are the same?

assert ~x == -1-x # or -1^x

The only problem is seeing the result printed, since people insist
that hex(a) will be '-'[:a<0]+hex(abs(a))

which brings up base-complement representation for signed numbers,
where the first digit is always 0 or base-1 to indicate positive and negative:
(BTW, I found a bug when I dug this up from my disk, so a previous post with this
might have that bug (bad leading digit check wasn't sign-sensitive))
... if not (2 <= B <= len(digits)): raise ValueError('bad base = %r'%B)
... if not x: return digits[0]
... s = []
... while x and x != -1:
... x, r = divmod(x, B)
... s.append(digits[r])
... if not s or s[-1] != (digits[0], digits[B-1])[x<0]:
... s.append(digits[x<0 and B-1 or 0])
... return ''.join(s[::-1])
... ... if s == digits[0]: return 0
... acc = s[0].lower() == digits[B-1] and -B**len(s) or 0
... for i, c in enumerate(s[::-1]):
... acc += digits.index(c)*B**i
... return acc
... '03'

Regards,
Bengt Richter

 

[Bengt Richter]

What's the meaning of the 1's complement operator (for example, what
is ~1), when ints and longs are the same?

Python 2.2.1 (#1, Aug 25 2004, 16:56:05)
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
$ python
Python 2.4.1 (#1, May 27 2005, 18:02:40)
[GCC 3.3.3 (cygwin special)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
What's going to change when ints and longs are finally integrated?

I suspect that Paul may be worrying that bit operations on longs will be
bit operations on signs and positive magnitudes rather than signed numbers
of arbitrary width. This may be promoted by the fact the so far we have no
builting format for showing the bits of negative numbers the way hex used to do.

I scratched the itch this way:
'fe00000000'
oops, precendence ... 'fe00000003'

vs. the useless '-0x1FFFFFFFDL'

(at least for looking at bit operation results ;-)

BTW, note base complementing, with base-1 or 0 as sign:
'0111111111111111111111111111111100'

Regards,
Bengt Richter

 

[Patrick Maupin]

No one has yet explained the reasoning (vs the mechanics) of the

returned value of the following.

L = range(10)
L[3::-1]

So far every attempt to explain it has either quoted the documents which
don't address that particular case, or assumed I'm misunderstanding
something, or implied it isn't neccisary to do.

It's quite easy to get me to change my mind on something, just show me a
convincing explanation and I will.

Once I saw this, I was surprised, because slices currently work exactly
like I would and do expect on this example, so I now have to admit that
I didn't read your original post fully and carefully. I have gone back
to look to figure out what you don't like and what you want, and I am
very surprised.

To me, your way reeks of black magic -- if there is a sign on the
stride, then strange and wondrous transformations happen. I think it
is conceptually (and probably programatically) much simpler the way it
is.

In any case, you asked for a rationale. I'll give you mine:

L = range(10)
L[3:len(L):-1] == [L for i in range(3,len(L),-1)] True

If you manage somehow to hange that 'True' to a 'False', I'll
personally be extremely unhappy.

Regards,
Pat

 

[Patrick Maupin]

I previously wrote (in response to a query from Ron Adam):

In any case, you asked for a rationale. I'll give you mine:

L = range(10)
L[3:len(L):-1] == [L for i in range(3,len(L),-1)] True

After eating supper, I just realized that I could probably make my
point a bit clearer with a slightly longer example:

L = range(10)
for stride in [-3, -2, -1, 1, 2, 3]:

.... for start in range(len(L)):
.... for end in range(len(L)):
.... P = L[start:end:stride]
.... Q = [L for i in range(start, end, stride)]
.... assert P == Q

This should never fail with an assertion error. You will note that it
shows that, for non-negative start and end values, slicing behavior is
_exactly_ like extended range behavior. I cannot imagine that the
behavior of range() could be made any more intuitive than it already
is. I personally feel that your proposed change makes slice() less
intuitive on its own, but even if I did not feel that way, your way
would have to be SIGNIFICANTLY better than the current way to make it
worthwhile to make slice() behavior differ from that of range().

In my initial skimming of your post, I originally thought you were
referring to negative start and end values. Negative start and end
values will sometimes cause issues, but the utility of their current
implementation far outweighs the few corner cases which (as I mentioned
in an earlier post) sometimes need some special case logic to deal
with.

Regards,
Pat

 

[Ron Adam]

To repeat, the current reason is compatibility with the original design for
NumPy. Perhaps there is some explanation for that in sliceobject.c (or
whatever the file is called), the CVS checkin messages, or the newsgroup
archives before the first checkin (in 1996, I believe).

Terry J. Reedy

Thanks, I'll check the cvs, so far I havn't been able to find any
references to it.

 

[Ron Adam]

I previously wrote (in response to a query from Ron Adam):

In any case, you asked for a rationale. I'll give you mine:

L = range(10)
L[3:len(L):-1] == [L for i in range(3,len(L),-1)]

True

After eating supper, I just realized that I could probably make my
point a bit clearer with a slightly longer example:

L = range(10)
for stride in [-3, -2, -1, 1, 2, 3]:

... for start in range(len(L)):
... for end in range(len(L)):
... P = L[start:end:stride]
... #Q = [L for i in range(start, end, stride)]

Q = [L for i in range(start, end)][::stride]

... assert P == Q

With the changed line above it will pass with the example nxlist type.

The result is different because the method is different. So in order
for this test to not give an assert, the same order needs to be used.

This should never fail with an assertion error. You will note that it
shows that, for non-negative start and end values, slicing behavior is
_exactly_ like extended range behavior.

Yes, and it passes for negative start and end values as well.

I cannot imagine that the
behavior of range() could be made any more intuitive than it already
is.

If range were also changed.. (I'm not suggesting it should be)
range(0,10,-1), would count down from 10 to zero. The sign of the step
would determine which side to iterate from.

I think they are both fairly equivalent as far as intuitiveness. But I
think I agree, changing both range and slice is probably out of the
question.

I personally feel that your proposed change makes slice() less
intuitive on its own, but even if I did not feel that way, your way
would have to be SIGNIFICANTLY better than the current way to make it
worthwhile to make slice() behavior differ from that of range().

Well I did post it as a "possible" improvement, meaning I wasn't sure.
And did ask for opinions. This was the type of reply I was looking for.
Thanks for replying, and for taking a serious look.

In my initial skimming of your post, I originally thought you were
referring to negative start and end values. Negative start and end
values will sometimes cause issues, but the utility of their current
implementation far outweighs the few corner cases which (as I mentioned
in an earlier post) sometimes need some special case logic to deal
with.

I was referring to both, it seemed to me that my suggestion had enough
good things in it that it would be worth asking others if it was
feasible, and also if it were desirable. In any case, it's an
interesting topic and looks like it could have been a valid alternative
if it were done this way from the start. Probably isn't good enough to
change now.

Thanks again, this pretty much explains why slices opperate the way they
do. And it explains why the edge case's happen as well I think.

Cheers,
Ron

 

[Ron Adam]

This was not an ad hominem attack but a commentary on many attempts to
"improve" the language.

Ok, No problem. ;-)

No one has yet explained the reasoning (vs the mechanics) of the
returned value of the following.

L = range(10)
L[3::-1]

So far every attempt to explain it has either quoted the documents
which don't address that particular case, or assumed I'm
misunderstanding something, or implied it isn't neccisary to do.

It's quite easy to get me to change my mind on something, just show me
a convincing explanation and I will.

L[3::-1] [3, 2, 1, 0]
L[3::1] [3, 4, 5, 6, 7, 8, 9]

>
I don;t see the problem here. The start specifies which element is the
first in the slice, the stop is the default (end of the sequence) and
the stride is "successive elements to the left" when it's -1 and
"successive elements to the right" when it's 1.

This is how it works. But determining the correct index's to use in
some cases can be tricky.

Or perhaps you can tell me what I've missed?

Ok, lets see... This shows the problem with using the gap indexing model.

L = range(10)

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
0 1 2 3 4 5 6 7 8 9 10 # index's

L[3::1] -> [3, 4, 5, 6, 7, 8, 9] 3rd index to end... ok
L[3:6:1] -> [3, 4, 5] 3rd index to 6th index... ok

L[3::-1] -> [3, 2, 1, 0] 4th index to beginning... umm
L[6:3:-1] -> [6, 5, 4] 7th index to 4th index... ?

So negative strides use a different index position?

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-1 0 1 2 3 4 5 6 7 8 9 # index's

L[3::-1] -> [3, 2, 1, 0] 3rd index to beginning... ok
L[6:3:-1] -> [6, 5, 4] 6th index to 3rd index... ok

To reach the '0' we have to check the index...

if i <= -1:
r = L[i+3::-1]
else:
r = L[i+3:i:-1]


Using negative index's ...

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 # index's

L[-3::1] -> [7, 8, 9] ok
L[-6:-3:1] -> [4, 5, 6] ok

L[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] -2nd to -10th ?
L[-3:-6:-1] -> [7, 6, 5] -2nd index to -5th.. ?


So we have to shift all the index's to the right again.

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 # index's

L[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] -3rd to -11th
L[-3:-6:-1] -> [7, 6, 5] -3rd index to -6th.

I feel it would be nicer if the index's didn't shift for negative strides.

Maybe gap addressing isn't really the best way to explain slicing?

Most of this function has to do with filling in the defaults, but it
seems to work. I'm sure theres a better way to determine the defaults
than this rather awkward if-else tree.

def slc(L, start, stop=None, step=1):
if stop == None:
if start == None:
if step >= 0:
start = 0
stop = len(L)
else:
start = len(L)-1
stop = -1
elif start >=0:
if step>0:
stop = len(L)
else:
stop = -1
else:
if step>0:
stop = 0
else:
stop = -(len(L)+1)
if start == None:
if stop >= 0:
if step>0:
start = 0
else:
start = len(L)-1
else:
if step>0:
start = -1
else:
start = 0
new = []
for i in range(start,stop,step):
new.append(L)
return new

This is more precise, but not neccisarily any easier to understand
without mentally tracing the conditions. But it's clear now why slices
behave the way they do.


Having to reverse the order of the index's along with using -steps is a
bit bothersome. It might be easier if it were of the form...

L[left:right:step]

Then positive and negative index's could be easily be mixed, and the
slice is always the space between. Step determines the output order and
stride.

Thats easy enough to explain even for beginners. But as Patrick pointed
out it's not consistent with range.

This alternate "suggestion" gets the slice first then reverses it. It
has the advantage that the above alternate indexing adjustments go away
also.

And.

L[3:6:1] = L[3:6:-1] reverses a sub sequence.


This disadvantages are: it's different, it's not compatible with current
range. It probably breaks more backward compatibility than I suspect.

My point was that you can make those changes in your own code, leaving
others to accept the situation as it is.

It's only a suggestion and an interesting idea I thought I would share
and see if anyone would like to discuss. I did title this as a 'possible
improvement', and not as proposed improvement.

Right, I wasn't trying to suggest that you didn't know what you were
talking about - or even that you didn't understand general relativity
(on which my grasp could be said to be tenuous) - merely that some
things are inherently difficult, and no matter how you twist the
implementations about, the difficulties will remain.

But shouldn't we try to make things easier when possible?

Cheers,
Ron

 

[Patrick Maupin]

Yes, and it passes for negative start and end values as well.

Umm, no:

..>> for stride in [-3, -2, -1, 1, 2, 3]:
.... for start in range(-1,len(L)):
.... for end in range(-1,len(L)):
.... P = L[start:end:stride]
.... Q = [L for i in range(start, end, stride)]
.... assert P==Q, [start, end, stride, P, Q]
....
Traceback (most recent call last):

Thanks again, this pretty much explains why slices opperate the
way they do. And it explains why the edge case's happen as well I think.

You're welcome.

Regards,
Pat

 

[Magnus Lycka]

Ok, lets see... This shows the problem with using the gap indexing model.

L = range(10)

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
0 1 2 3 4 5 6 7 8 9 10 # index's

L[3::1] -> [3, 4, 5, 6, 7, 8, 9] 3rd index to end... ok
L[3:6:1] -> [3, 4, 5] 3rd index to 6th index... ok

L[3::-1] -> [3, 2, 1, 0] 4th index to beginning... umm
L[6:3:-1] -> [6, 5, 4] 7th index to 4th index... ?

So negative strides use a different index position?

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-1 0 1 2 3 4 5 6 7 8 9 # index's

L[3::-1] -> [3, 2, 1, 0] 3rd index to beginning... ok
L[6:3:-1] -> [6, 5, 4] 6th index to 3rd index... ok

Ok, I see what you mean. The "view slices as indices standing
between the items in the sequence" model actually breaks down
with negative strides.

Then you need to view it more in the mathematical way of
half-open (or half-closed if you prefer) intervals.

[a,b) = { x | a <= x < b }

See http://en.wikipedia.org/wiki/Half-closed_interval

I still think the current sematics are the least surprising
though. For instance, if we want to implement a ring class
looking something like the one below, the logical way to do
that would allow it to be used as if it was a list.

Actually, such a ring class (extented to handle extended slices
correctly) would (I think) solve the tricky cases with things
such as l[0:-0] or l[9:-1:-1].

The complete implementation is left as an exercise to the
reader, although there's probably something like it in the
Python cookbook already.

class Ring(list):
def __init__(self, size):
self.size = size
def __setitem__(self, i,v):
return list.__setitem__(self, i%self.size, v)
def __getitem__(self, i):
return list.__getitem__(self, i%self.size)
def __setslice__(self, i, j, v):
return list.__setslice__(self, i%self.size, j%self.size,v)
def __getslice__(self, i, j):
return list.__getslice__(self, i%self.size, j%self.size)

 

[Ron Adam]

Ron Adam wrote:

Yes, and it passes for negative start and end values as well.

Umm, no:

.>> for stride in [-3, -2, -1, 1, 2, 3]:
... for start in range(-1,len(L)):
... for end in range(-1,len(L)):
... P = L[start:end:stride]
... Q = [L for i in range(start, end, stride)]
... assert P==Q, [start, end, stride, P, Q]
...
Traceback (most recent call last):
File "<stdin>", line 6, in ?
AssertionError: [-1, 0, -3, [9, 6, 3], []]

Ah, Yes... I it was way too late last night and I mistakenly changed the
values of L... which was meaningless.

Range lines in the for statements, need to read..

range(-len(l),0)

But then it doesn't include all the values of L.

 

[Ron Adam]

Ok, lets see... This shows the problem with using the gap indexing
model.

L = range(10)

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
0 1 2 3 4 5 6 7 8 9 10 # index's

L[3::1] -> [3, 4, 5, 6, 7, 8, 9] 3rd index to end... ok
L[3:6:1] -> [3, 4, 5] 3rd index to 6th index... ok

L[3::-1] -> [3, 2, 1, 0] 4th index to beginning... umm
L[6:3:-1] -> [6, 5, 4] 7th index to 4th index... ?

So negative strides use a different index position?

[ 0 , 1 , 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] # elements
-1 0 1 2 3 4 5 6 7 8 9 # index's

L[3::-1] -> [3, 2, 1, 0] 3rd index to beginning... ok
L[6:3:-1] -> [6, 5, 4] 6th index to 3rd index... ok

Ok, I see what you mean. The "view slices as indices standing
between the items in the sequence" model actually breaks down
with negative strides.

Yes, that and the edge case's is why I this topic keeps coming up. Then
there's the 'None' default values that depend on both the stride sign,
and the index signs. These are all features in some context, and can be
annoyances in others. As long as you stick with positive stride values,
it's not much of a problem though, so an alternate solution will have to
be really good.

Then you need to view it more in the mathematical way of
half-open (or half-closed if you prefer) intervals.

[a,b) = { x | a <= x < b }

See http://en.wikipedia.org/wiki/Half-closed_interval

Interesting... Maybe just a different syntax that designates the stop
as being inclusive would work?

[a,b] = { x | a <= x <= b }

L[a;<b] a to b-1, same as L[a:b]
L[a;b] a to b
L[a>;b] a+1 to b
L[a>;<b] a+1 to b-1

I still think the current sematics are the least surprising
though. For instance, if we want to implement a ring class
looking something like the one below, the logical way to do
that would allow it to be used as if it was a list.

Actually, such a ring class (extented to handle extended slices
correctly) would (I think) solve the tricky cases with things
such as l[0:-0] or l[9:-1:-1].

The complete implementation is left as an exercise to the
reader, although there's probably something like it in the
Python cookbook already.

class Ring(list):
def __init__(self, size):
self.size = size
def __setitem__(self, i,v):
return list.__setitem__(self, i%self.size, v)
def __getitem__(self, i):
return list.__getitem__(self, i%self.size)
def __setslice__(self, i, j, v):
return list.__setslice__(self, i%self.size, j%self.size,v)
def __getslice__(self, i, j):
return list.__getslice__(self, i%self.size, j%self.size)

This is nice. I might even find a use for it. ;-)

Cheers,
Ron

 

[Bengt Richter]

Steve Holden wrote: [...]

My point was that you can make those changes in your own code, leaving
others to accept the situation as it is.

It's only a suggestion and an interesting idea I thought I would share
and see if anyone would like to discuss. I did title this as a 'possible
improvement', and not as proposed improvement.

Right, I wasn't trying to suggest that you didn't know what you were
talking about - or even that you didn't understand general relativity
(on which my grasp could be said to be tenuous) - merely that some
things are inherently difficult, and no matter how you twist the
implementations about, the difficulties will remain.

But shouldn't we try to make things easier when possible?

Sure ;-)

It occurs to me that maybe this discussion of slicing has a parallel
in mathematical intervals, and we might usefully check if it makes sense there.

IOW, let's compare [a, b] vs [a, b) vs (a, b] vs (a,b)

ISTM the python range model corresponds to [a, b) in terms of integers. The step thing
modifies that, but leave that aside to concetrate on the main issue.

For math, I think the notation requires a<=b, but for programming, python has a convention
for specifying related intervals and a subsetting function, with similar notation adding step.

Leaving aside abs(step)!=1 which specifies subsetting, we could say that

[a:b:1]
is
[a, b)
and
[a:b,-1]
is
(a, b]

but the latter returned in reverse order.

If we factor out the issues of reversing and subsetting, we just have
the indication of which kind of interval: half-open to the right or left.

That we could do by
[a:b] => [a, b)
and
.[a:b] => (a, b]

Then the question is, do we need sugar for reversed(x.[a:b])
or list(reversed(x.[a:b])) for the right hand side of a statement,
and do we want to to use both kinds of intervals in slice assignment?
(maybe and yes ;-)

The reason for yes is that it solves the which-gap problem in assigning to [a:a]
if you define [a, a) as an empty interval to the left of a and (a, a] as an empty
interval to the right of a, sort of like +0 and -0 in half-open intervals ;-)
Replacing the empty interval does the insertion on the side you want ;-)

I am choosing the convention to stay compatible with python's current behaviour,
even though I'm not sure what the math world says about a<x<=a vs a<=x<a as
different-in-some-sense intervals. I guess the intervals as point sets are the same,
but the interval definitions are different...

Other than the a:a distinction, in terms of integers and UIAM
.[a:b]
is just sugar for
[a+1:b+1]
but the sugar is nice, and the slice assignment to either side is nicer.

I'll deal with the subsetting another time ...

Regards,
Bengt Richter

 

[Bengt Richter]

Then you need to view it more in the mathematical way of
half-open (or half-closed if you prefer) intervals.

[a,b) = { x | a <= x < b }

Funny, I just posted with the same thought (and some additional considerations)
I hadn't read yours yet, or I would have mentioned it ;-)

Regards,
Bengt Richter

 

[Ron Adam]

Then the question is, do we need sugar for reversed(x.[a:b])
or list(reversed(x.[a:b])) for the right hand side of a statement,
and do we want to to use both kinds of intervals in slice assignment?
(maybe and yes ;-)

Yes, I think this is the better way to do it, as this address's the
underlying causes instead of treating the symptoms.

The reason for yes is that it solves the which-gap problem in assigning to [a:a]
if you define [a, a) as an empty interval to the left of a and (a, a] as an empty
interval to the right of a, sort of like +0 and -0 in half-open intervals ;-)
Replacing the empty interval does the insertion on the side you want ;-)
>
I am choosing the convention to stay compatible with python's current behaviour,
even though I'm not sure what the math world says about a<x<=a vs a<=x<a as
different-in-some-sense intervals. I guess the intervals as point sets are the same,
but the interval definitions are different...

Not sure either. I think intervals is an important concept and enabling
python to work with them would be good if it could be done in a simple
and consistent way. This extends a lot further than just slice
operations because of the relationship between ...

for x in range() -> slice(range)


So defining an interval object that can be used as an iterator in a for
loop might be the beginning, and then finally slice with an alternate
syntax to an abbreviated form. So then you would have the relationship
of...

for x in interval() -> slice(interval)

Other than the a:a distinction, in terms of integers and UIAM
.[a:b]
is just sugar for
[a+1:b+1]
but the sugar is nice, and the slice assignment to either side is nicer.

Wouldn't that be [a:b+1] ?

As sugar the index's are translated at compile time, it may run into the
current edge case indexing problems. So it will most likely need to be
an actual interval object, with an alternative syntax to use it.

I'll deal with the subsetting another time ...

No hurry, this isn't a hack it out because "the boss wants it on his
desk Monday" situation. ;-)

Regards,
Bengt Richter

Cheers,
Ron

 

[Michael Hudson]

Ron Adam wrote: [...]

REVERSE ORDER STEPPING
----------------------
When negative steps are used, a slice operation
does the following. (or the equivalent)

1. reverse the list
2. cut the reversed sequence using start and stop
3. iterate forward using the absolute value of step.

I think you are looking at this from the wrong perspective.
Whatever sign c has:
For s[a:b:c], a is the index for the first item to include,
b is the item after the last to include (just like .end() in
C++ iterators for instance), and c describes the step size.

Yes, and that is how it "should" work. But....

With current slicing and a negative step...

[ 1 2 3 4 5 6 7 8 9 ]
-9 -8 -7 -6 -5 -4 -3 -2 -1 -0

r[-3:] -> [7, 8, 9] # as expected
r[-3::-1] -> [7, 6, 5, 4, 3, 2, 1, 0] # surprise

The seven is include in both cases, so it's not a true inverse
selection either.

Did you read what Magnus said: "a is the index for the first item to
include"? How could r[-3::x] for any x not include the 7?

Cheers,
mwh