sum function

[mike20007]

Hi All,

I am new to python and am getting the data from hbase.
I am trying to do sum on the column as below


scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
total = 0.0
r = client.scannerGet(scanner)
while r:
for k in (r[0].columns):
total += float(r[0].columns[k].value)
r = client.scannerGet(scanner)

print total

Do you know of better (faster) way to do sum?

Any thoughts please?

Thanks

 

[Ian Kelly]

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
total = 0.0
r = client.scannerGet(scanner)
while r:
for k in (r[0].columns):
total += float(r[0].columns[k].value)
r = client.scannerGet(scanner)

print total

Do you know of better (faster) way to do sum?

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
next_r = itertools.partial(client.scannerGet, scanner)
total = sum(float(col.value) for r in iter(next_r, None) for col in
r.itervalues())

 

[Ian Kelly]

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
next_r = itertools.partial(client.scannerGet, scanner)
total = sum(float(col.value) for r in iter(next_r, None) for col in
r.itervalues())

That should be "functools" above, not "itertools".

 

[Mike]

I get below error

NameError: name 'functools' is not defined

Thanks

 

[Mike]

I get below error

NameError: name 'functools' is not defined

Thanks

 

[Mike]

Thanks Ian for the quick reply.

I get the below error.

NameError: name 'itertools' is not defined

Thanks

 

[Mike]

Thanks Ian for the quick reply.

I get the below error.

NameError: name 'itertools' is not defined

Thanks

 

[Chris Angelico]

I get below error

NameError: name 'functools' is not defined

Thanks

functools is a module:

import functools

ChrisA

 

[Dave Angel]

I get below error

NameError: name 'functools' is not defined

functools is a module in the standard library. You need to import it.

import functools

 

[Mike]

functools is a module in the standard library. You need to import it.



import functools







--



DaveA

I imported functools. Now I get the below error please.


Traceback (most recent call last):
File "test.py", line 16, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
File "test.py", line 16, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
AttributeError: 'list' object has no attribute 'itervalues'

Thanks

 

[Mike]

functools is a module in the standard library. You need to import it.



import functools







--



DaveA

I imported functools. Now I get the below error please.


Traceback (most recent call last):
File "test.py", line 16, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
File "test.py", line 16, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r.itervalues())
AttributeError: 'list' object has no attribute 'itervalues'

Thanks

 

[Mike]

I agree with you, Ian. Thanks for all the help. Now I get the below error.

File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

Thanks

 

[Mike]

I agree with you, Ian. Thanks for all the help. Now I get the below error.

File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

Thanks

 

[Ramchandra Apte]

I agree with you, Ian. Thanks for all the help. Now I get the below error.



File "test.py", line 17, in <module>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

File "test.py", line 17, in <genexpr>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())



Thanks

You have missed the last line of the traceback (error)

 

[Ramchandra Apte]

I agree with you, Ian. Thanks for all the help. Now I get the below error.



File "test.py", line 17, in <module>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

File "test.py", line 17, in <genexpr>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())



Thanks

You have missed the last line of the traceback (error)

 

[Mike]

Hi All,



I am new to python and am getting the data from hbase.

I am trying to do sum on the column as below





scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])

total = 0.0

r = client.scannerGet(scanner)

while r:

for k in (r[0].columns):

total += float(r[0].columns[k].value)

r = client.scannerGet(scanner)



print total



Do you know of better (faster) way to do sum?



Any thoughts please?



Thanks

Sorry about that. Here you go

Traceback (most recent call last):
File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
IndexError: list index out of range

 

[Ramchandra Apte]

Hi All,
I am new to python and am getting the data from hbase.
I am trying to do sum on the column as below
scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
total = 0.0
r = client.scannerGet(scanner)
while r:
for k in (r[0].columns):
total += float(r[0].columns[k].value)
r = client.scannerGet(scanner)
print total
Do you know of better (faster) way to do sum?
Any thoughts please?
Thanks

Sorry about that. Here you go



Traceback (most recent call last):

File "test.py", line 17, in <module>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

File "test.py", line 17, in <genexpr>

total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())

IndexError: list index out of range

the variable "r" is an empty list

 

[Mike]

Hi All,

I am new to python and am getting the data from hbase.

I am trying to do sum on the column as below

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])

total = 0.0

r = client.scannerGet(scanner)

while r:

for k in (r[0].columns):

total += float(r[0].columns[k].value)

r = client.scannerGet(scanner)

print total

Do you know of better (faster) way to do sum?

Any thoughts please?

Sorry about that. Here you go
Traceback (most recent call last):
File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
IndexError: list index out of range

the variable "r" is an empty list

Here is the actual code.

scanner = client.scannerOpenWithStop("tab", "10", "1000", ["cf:col1"])
next_r = functools.partial(client.scannerGet, scanner)
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())


Scanner does have rows.

Are we missing something please?

Thanks

 

[Terry Reedy]

If you want as many people as possible to read your posts, stop using a
mail-agent and site that spits in the face of readers by doubling blank
lines each iteration. Alternatives have been discussed previously.

 

[Ian Kelly]

Sorry about that. Here you go

Traceback (most recent call last):
File "test.py", line 17, in <module>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
File "test.py", line 17, in <genexpr>
total = sum(float(col.value) for r in iter(next_r, None) for col in r[0].columns.itervalues())
IndexError: list index out of range

Maybe the sentinel value is not None as I assumed, and it's
overrunning the end of the data? What does
client.scannerGet return when there is no more data?