Bug in slice type

[Dennis Lee Bieber]

So how do I express a -0? Which should point to the gap after the last
item.

Step one: Obtain a processor that uses ones-complement arithmetic.

Step two: Port Python to said processor...

...

 

[Steve Holden]

Step one: Obtain a processor that uses ones-complement arithmetic.

Ah, the good old Univac 418 ... [drools into beard and mumbles]

Step two: Port Python to said processor...

Unfortunately the 418 would probably come in at about a micro-pystone,
so perhaps we should emulate it on something more modern?

regards
Steve

 

[Ron Adam]

You just did ;-) but I probably do not know what you mean.

b[-1:] = ['Z'] # replaces last item
b[-1:-0] = ['Z'] # this doesn't work

If you are using negative index slices, you need to check for end
conditions because you can't address the end of the slice in a
sequential/numerical way.

b = list('abcdefg')
for x in range(-len(b),-1):
print b[x:x+2]

['a', 'b']
['b', 'c']
['c', 'd']
['d', 'e']
['e', 'f']
[]


b = list('abcdefg')
for x in range(-len(b),-1):
if x<-2:
print b[x:x+2]
else:
print b[x:]

['a', 'b']
['b', 'c']
['c', 'd']
['d', 'e']
['e', 'f']
['f', 'g']

The slice index of the gap after the last item is len(seq).



As I posted before (but perhaps it arrived after you sent this), one number
indexing rounds down, introducing a slight asymmetry.

I didn't see that one, but I agree. Single index's are asymmetric,
positive slices with two index's are again symetric, negative slices
with negative strides or steps are again asymmetric.

???

print a[4:1:-1] # 6|g| 5|f| 4|e| 3|d| 2|c| 1|b| 0|a| ?
-> edc

print a[-3:-6:-1] # -1|g|-2|f|-3|e|-4|d|-5|c|-6|b|-7|a|-8|
-> edc

# special case '::'
print a[6::-1] # 6|g| 5|f| 4|e| 3|d| 2|c| 1|b| 0|a| ?
-> gfedcba

print a[-1:-8:-1] # -1|g|-2|f|-3|e|-4|d|-5|c|-6|b|-7|a|-8
-> gfedcba

All of the following get the center 'd' from the string.

a = 'abcdefg'
print a[3] # d 4 gaps from beginning
print a[-4] # d 5 gaps from end

It is 3 and 4 gaps *from* the left and right end to the left side of the
'd'. You can also see the asymmetry as coming from rounding 3.5 and -3.5
down to 3 and down to -4.

Since single indexing only refers to existing items and aren't used to
insert between items, this still works even with the slight asymmetry.

print a[3:4] # d
print a[-4:-3] # d

These are is symmetric, as we claimed.

Yes, no problem here except for addressing the -0th (end gap) position
without special casing to either a positive index or a[-n:].

print a[3:2:-1] # d These are symetric?!
print a[-4:-5:-1] # d
print a[3:-5:-1] # d
print a[-4:2:-1] # d

The pattern seems to be: left-gap-index : farther-to-left-index : -1 is
somehow equivalent to left:right, but I never paid much attention to
strides and don't know the full rule.

a[start:stop:-1]
a[stop:start] # exchange index's
a.reverse() # reverse string

a[4:1:-1] # 6 |g| 5 |f| 4 |e| 3 |d| 2 |c| 1 |b| 0 |a| ?
a[1:4] # ? |a| 0 |b| 1 |c| 2 |d| 3 |e| 4 |f| 5 |g| 6
a.reverse() # -> edc

Notice the index's are 1 less than with positive strides.

Stride slices are really a different subject from two-gap slicing. They
were introduced in the early years of Python specificly and only for
Numerical Python. The rules were those needed specificly for Numerical
Python arrays. They was made valid for general sequence use only a few
years ago. I would say that they are only for careful mid-level to expert
use by those who actually need them for their code.

I'd like to see those use case's. I have a feeling there are probably
better ways to do it now.

Doing a quick search in python24/lib, there are only two places that use
a negative step or stride value to reverse a sequence.

---------- PICKLE.PY
return binary[::-1]
ashex = _binascii.hexlify(data[::-1])

I don't think people would miss negative strides much if they were
removed. Replacing these case's with reverse() methods shouldn't be that
difficult.

Cheers,
Ron

 

[Bengt Richter]

The problem with negative index's are that positive index's are zero
based, but negative index's are 1 based. Which leads to a non
symmetrical situations.

Note that you can insert an item before the first item using slices. But
not after the last item without using len(list) or some value larger
than len(list).

IMO the problem is that the index sign is doing two jobs, which for zero-based
reverse indexing have to be separate: i.e., to show direction _and_ a _signed_
offset which needs to be realtive to the direction and base position.

A list-like class, and an option to use a zero-based reverse index will illustrate:
... def __init__(self, value=0):
... self.value = value
... def __repr__(self): return 'Zbrx(%r)'%self.value
... def __sub__(self, other): return Zbrx(self.value - other)
... def __add__(self, other): return Zbrx(self.value + other)
... ... def normslc(self, slc):
... sss = [slc.start, slc.stop, slc.step]
... for i,s in enumerate(sss):
... if isinstance(s, Zbrx): sss = len(self.value)-1-s.value
... return tuple(sss), slice(*sss)
... def __init__(self, value):
... self.value = value
... def __getitem__(self, i):
... if isinstance(i, int):
... return '[%r]: %r'%(i, self.value)
... elif isinstance(i, Zbrx):
... return '[%r]: %r'%(i, self.value[len(self.value)-1-i.value])
... elif isinstance(i, slice):
... sss, slc = self.normslc(i)
... return '[%r:%r:%r]: %r'%(sss+ (list.__getitem__(self.value, slc),))
... def __setitem__(self, i, v):
... if isinstance(i, int):
... list.__setitem__(self, i, v)
... elif isinstance(i, slice):
... sss, slc = self.normslc(i)
... list.__setitem__(self.value, slc, v)
... def __repr__(self): return 'Zbrxlist(%r)'%self.value
...

>>> zlast = Zbrx(0)
>>> zbr10 = Zbrxlist(range(10))
>>> zbr10[zlast] '[Zbrx(0)]: 9'
>>> zbr10[zlast:] '[9:None:None]: [9]'
>>> zbr10[zlast:zlast] = ['end']
>>> zbr10 Zbrxlist([0, 1, 2, 3, 4, 5, 6, 7, 8, 'end', 9])
>>> ztop = Zbrx(-1)
>>> zbr10[ztop:ztop] = ['final']
>>> zbr10 Zbrxlist([0, 1, 2, 3, 4, 5, 6, 7, 8, 'end', 9, 'final'])
>>> zbr10[zlast:] "[11:None:None]: ['final']"
>>> zbr10[zlast] "[Zbrx(0)]: 'final'"
>>> zbr10[zlast+1] '[Zbrx(1)]: 9'
>>> zbr10[zlast+2]

"[Zbrx(2)]: 'end'"

a = list('abcde')
a[len(a):len(a)] = ['end']
a

['a', 'b', 'c', 'd', 'e', 'end']

a[-1:-1] = ['last']
a

['a', 'b', 'c', 'd', 'e', 'last', 'end'] # Second to last.

a[100:100] = ['final']
a

['a', 'b', 'c', 'd', 'e', 'last', 'end', 'final']

Zbrxlist(['a', 'b', 'c', 'd', 'e'])

Forgot to provide a __len__ method ;-)

>>> a[len(a.value):len(a.value)] = ['end']
>>> a

Zbrxlist(['a', 'b', 'c', 'd', 'e', 'end'])

lastx refers to the last items by zero-based reverse indexing

>>> a[lastx] "[Zbrx(0)]: 'end'"
>>> a[lastx:lastx] = ['last']
>>> a

Zbrxlist(['a', 'b', 'c', 'd', 'e', 'last', 'end'])

As expected, or do you want to define different semantics?
You still need to spell len(a) in the slice somehow to indicate
beond the top. E.g.,

>>> a[lastx-1:lastx-1] = ['final']
>>> a

Zbrxlist(['a', 'b', 'c', 'd', 'e', 'last', 'end', 'final'])

Perhaps you can take the above toy and make something that works
they way you had in mind? Nothing like implementation to give
your ideas reality ;-)

Regards,
Bengt Richter

 

[Terry Reedy]

You just did ;-) but I probably do not know what you mean.

b[-1:] = ['Z'] # replaces last item
b[-1:-0] = ['Z'] # this doesn't work

If you are using negative index slices, you need to check for end
conditions because you can't address the end of the slice in a
sequential/numerical way.

OK, now I understand your question, the answer 'get a one's complement
machine', and your point that without a -0 separate from +0, there is a
subtle asymmetry that is easy to overlook.

I didn't see that one,

Perhaps you did not see my other long post, where I drew ascii pictures?

In brief: if you draw an axis with ticks, label the tick with ints, and put
chars or item references in the space *between* the ticks, then the
'average' coordinate of the stuff between is n.5. The slice interval is
nn+1), but a single int label has to be either n or (n+1), rounding down
or up. And both choices have been made because both have advantages.

In the US (and UK?), the ground level floor of a multifloor building is the
first floor. In continental Europe (all or just some?), the ground floor
is the ground (effectively zeroth) floor while the first floor up is the
first stage (resting place on the stairway).

I don't think people would miss negative strides much if they were
removed. Replacing these case's with reverse() methods shouldn't be that
difficult.

Yes, the introduction of reversed partly obsoleted the use of negative
strides, at least outside of its numerical array origin.

Terry J. Reedy

 

[Scott David Daniels]

Although it is _way_ too late to try something like this, once upon
a time you could have done all of this using the one's complement
operator:
~0 does exist and is distinct from 0.
So you could talk about a slice:
str[4 : ~2]
and so on.

--Scott David Daniels
(e-mail address removed)

 

[Ron Adam]

b[-1:] = ['Z'] # replaces last item
b[-1:-0] = ['Z'] # this doesn't work

If you are using negative index slices, you need to check for end
conditions because you can't address the end of the slice in a
sequential/numerical way.

OK, now I understand your question, the answer 'get a one's complement
machine', and your point that without a -0 separate from +0, there is a
subtle asymmetry that is easy to overlook.

Yes, I don't thinks it falls in the catagory of bugs, probably closer
to a minor wart.

Perhaps you did not see my other long post, where I drew ascii pictures?

Yes, I saw it. I think we are expressing the same things in different ways.

In the US (and UK?), the ground level floor of a multifloor building is the
first floor. In continental Europe (all or just some?), the ground floor
is the ground (effectively zeroth) floor while the first floor up is the
first stage (resting place on the stairway).

In the VA Hospital here in Tampa, the ground floor in the front elevator
is on the same level as the lobby, while the ground floor in the
elevator on the other side of the building is on the floor below. ;-)

 

[Ron Adam]

IMO the problem is that the index sign is doing two jobs, which for zero-based
reverse indexing have to be separate: i.e., to show direction _and_ a _signed_
offset which needs to be realtive to the direction and base position.

Yes, that's definitely part of it.

A list-like class, and an option to use a zero-based reverse index will illustrate:

... def __init__(self, value=0):
... self.value = value
... def __repr__(self): return 'Zbrx(%r)'%self.value
... def __sub__(self, other): return Zbrx(self.value - other)
... def __add__(self, other): return Zbrx(self.value + other)
...... def normslc(self, slc):
... sss = [slc.start, slc.stop, slc.step]
... for i,s in enumerate(sss):
... if isinstance(s, Zbrx): sss = len(self.value)-1-s.value
... return tuple(sss), slice(*sss)
... def __init__(self, value):
... self.value = value
... def __getitem__(self, i):
... if isinstance(i, int):
... return '[%r]: %r'%(i, self.value)
... elif isinstance(i, Zbrx):
... return '[%r]: %r'%(i, self.value[len(self.value)-1-i.value])
... elif isinstance(i, slice):
... sss, slc = self.normslc(i)
... return '[%r:%r:%r]: %r'%(sss+ (list.__getitem__(self.value, slc),))
... def __setitem__(self, i, v):
... if isinstance(i, int):
... list.__setitem__(self, i, v)
... elif isinstance(i, slice):
... sss, slc = self.normslc(i)
... list.__setitem__(self.value, slc, v)
... def __repr__(self): return 'Zbrxlist(%r)'%self.value
...

zlast = Zbrx(0)
zbr10 = Zbrxlist(range(10))
zbr10[zlast] '[Zbrx(0)]: 9'
zbr10[zlast:] '[9:None:None]: [9]'
zbr10[zlast:zlast] = ['end']
zbr10 Zbrxlist([0, 1, 2, 3, 4, 5, 6, 7, 8, 'end', 9])
ztop = Zbrx(-1)
zbr10[ztop:ztop] = ['final']
zbr10 Zbrxlist([0, 1, 2, 3, 4, 5, 6, 7, 8, 'end', 9, 'final'])
zbr10[zlast:] "[11:None:None]: ['final']"
zbr10[zlast] "[Zbrx(0)]: 'final'"
zbr10[zlast+1] '[Zbrx(1)]: 9'
zbr10[zlast+2]

"[Zbrx(2)]: 'end'"
Zbrxlist(['a', 'b', 'c', 'd', 'e'])

Forgot to provide a __len__ method ;-)

a[len(a.value):len(a.value)] = ['end']
a

Zbrxlist(['a', 'b', 'c', 'd', 'e', 'end'])

lastx refers to the last items by zero-based reverse indexing

a[lastx] "[Zbrx(0)]: 'end'"
a[lastx:lastx] = ['last']
a

Zbrxlist(['a', 'b', 'c', 'd', 'e', 'last', 'end'])

As expected, or do you want to define different semantics?
You still need to spell len(a) in the slice somehow to indicate
beond the top. E.g.,

a[lastx-1:lastx-1] = ['final']
a

Zbrxlist(['a', 'b', 'c', 'd', 'e', 'last', 'end', 'final'])

Perhaps you can take the above toy and make something that works
they way you had in mind? Nothing like implementation to give
your ideas reality ;-)

Thanks, I'll play around with it. ;-)

As you stated before the index is doing two jobs, so limiting it in some
way may be what is needed. Here's a few possible (or impossible) options.

(Some of these aren't pretty.)


* Disallow *all* negative values, use values of start/stop to determine
direction. Indexing from far end needs to be explicit (len(n)-x).

a[len(a):0] reverse order
a[len(a):0:2] reveres order, even items

(I was wondering why list's couldn't have len,min, and max attribute
that are updated when ever the list is modified in place of using
len,min, and max functions? Would the overhead be that much?)

a[len.a:0]


* Disallow negative index's, use negative steps to determine indexing
direction. Order of index's to determine output order.

a[len(a):0:-1] forward order, zero based indexing from end.
a[0:len(a):-1] reverse order, zero based from end.
a[0:1:-1] last item

I works, but single a[-1] is used extremely often. I don't think having
to do a[0:1:-1] would be very popular.


* A reverse index symbol/operator could be used ...

a[~0] -> last item, This works BTW. ~0 == -1
a[~1] -> next to last item

(Could this be related to the original intended use?)


a[~0:~0] slice after end ?. Doesn't work correctly.

What is needed here is to index from the left instead of the right.

a[~0] -> item to left of end gap.

*IF* this could be done; I'm sure there's some reason why this won't
work. ;-), then all indexing operations with '~' could be symmetric with
all positive indexing operations. Then in Python 3k true negative
index's could cause an exception... less bugs I bet. And then negative
steps could reverse lists with a lot less confusion, keeping that
functionality as well.

Maybe a[~:~] == a[-len(a):-0]
a[~:] == a[-len(a):len(a)]
a[:~] == a[0:-0]

a[~:~] == a[:]

This looks too nice, has it been suggested before?


Ok... a lot of off the top of my head thinking which probably means
these need to be thought out much better. But that's how a lot of good
ideas start out. ;-)

 

[Terry Reedy]

(I was wondering why list's couldn't have len,min, and max attribute
that are updated when ever the list is modified in place of using
len,min, and max functions?

Python's list and, I believe, other builtin roster objects do have a
C-level length attribute. For mutables, it is updated. __len__ is just a
get of int value the and conversion to int object. Min and max are defined
and sensible for only a subset of lists, and when they are, are very seldom
need repeated updates.

Would the overhead be that much?)

For the fraction of times used, yes.

Terry J. Reedy