Accessing Json data (I think I am nearly there) complete beginner

[Andrew Edwards-Adams]

Hey guys
I think its worth stating that I have been trying to code for 1 week.
I am trying to access some Json data. My innitial code was the below:

"import mechanize
import urllib
import re

def getData():
post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/all_time"
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.addheaders = [('User-agent', 'Firefox')]

#These are the parameters you've got from checking with the aforementioned tools
parameters = {'page' : '1',
'rp' : '10',
'sortname' : 'total_pl',
'sortorder' : 'desc'}
#Encode the parameters
data = urllib.urlencode(parameters)
trans_array = browser.open(post_url,data).read().decode('UTF-8')

#print trans_array

myfile = open("test.txt", "w")
myfile.write(trans_array)
myfile.close()

getData()

raw_input("Complete")"

I was recommended to use the following code to access the Json data directly, however I cannot get it to return anything. I think the guy that recommended me this method must have got something wrong? Or perhaps I am simply incompetent:

import mechanize
import urllib
import json
def getData():
post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/current_week"
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.addheaders = [('User-agent', 'Firefox')]

#These are the parameters you've got from checking with the aforementioned tools
parameters = {'page' : '1',
'rp' : '50',
'sortname' : 'total_pl',
'sortorder' : 'desc'
}
#Encode the parameters
data = urllib.urlencode(parameters)
trans_array = browser.open(post_url,data).read().decode('UTF-8')

text1 = json.loads(trans_array)
print text1['rows'][0]['id'] #play around with these values to access different data..

getData()

He told me to "#play around with these values to access different data.." really cant get anything out of this, any ideas?

Many thanks AEA

 

[Andrew Berg]

I was recommended to use the following code to access the Json data directly, however I cannot get it to return anything.

Where exactly is the problem? Do you not get JSON back? Do you get the wrong values? Do you get a KeyError or IndexError trying to get
values from text1? Are there gremlins going around flipping bits in memory? It's good that you posted code, but "really cant get anything
out of this" isn't very useful.

 

[MRAB]

Hey guys
I think its worth stating that I have been trying to code for 1 week.
I am trying to access some Json data. My innitial code was the below:

"import mechanize
import urllib
import re

def getData():
post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/all_time"
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.addheaders = [('User-agent', 'Firefox')]

#These are the parameters you've got from checking with the aforementioned tools
parameters = {'page' : '1',
'rp' : '10',
'sortname' : 'total_pl',
'sortorder' : 'desc'}
#Encode the parameters
data = urllib.urlencode(parameters)
trans_array = browser.open(post_url,data).read().decode('UTF-8')

#print trans_array

myfile = open("test.txt", "w")
myfile.write(trans_array)
myfile.close()

getData()

raw_input("Complete")"

I was recommended to use the following code to access the Json data directly, however I cannot get it to return anything. I think the guy that recommended me this method must have got something wrong? Or perhaps I am simply incompetent:

import mechanize
import urllib
import json
def getData():
post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/current_week"
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.addheaders = [('User-agent', 'Firefox')]

#These are the parameters you've got from checking with the aforementioned tools
parameters = {'page' : '1',
'rp' : '50',
'sortname' : 'total_pl',
'sortorder' : 'desc'
}
#Encode the parameters
data = urllib.urlencode(parameters)
trans_array = browser.open(post_url,data).read().decode('UTF-8')

text1 = json.loads(trans_array)
print text1['rows'][0]['id'] #play around with these values to access different data..

getData()

He told me to "#play around with these values to access different data.." really cant get anything out of this, any ideas?

Many thanks AEA

I've just tried it. It prints "1048".

 

[Andrew Edwards-Adams]

Where exactly is the problem? Do you not get JSON back? Do you get the wrong values? Do you get a KeyError or IndexError trying to get

values from text1? Are there gremlins going around flipping bits in memory? It's good that you posted code, but "really cant get anything

out of this" isn't very useful.

Hi thanks for the reply Andrew, my first bit of code was heading in the right direction I was managing to pull out the usernames from the JSON, using REGEX. I was told that the second piece of code is the more efficient way to pull data from the JSON. When I say it doesnt do anything I can adjust the areas suggested. But it returns nothing, I just get the "= Restart ="line appear in the python shell. If there was a trackback/debug I might know where to look, but its not yielding errors, its simply yielding nothing.What i dont know, is if it is the code that isnt working, or what I am inputting in the " print text1['rows'][0]['id']" that isnt working.


Since this code:

import mechanize
import urllib
import json
def getData():
post_url = "http://www.tweetnaps.co.uk/leaderboards/leaderboard_json/current_week"
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.addheaders = [('User-agent', 'Firefox')]

#These are the parameters you've got from checking with the aforementioned tools
parameters = {'page' : '1',
'rp' : '50',
'sortname' : 'total_pl',
'sortorder' : 'desc'
}
#Encode the parameters
data = urllib.urlencode(parameters)
trans_array = browser.open(post_url,data).read().decode('UTF-8')

appears in both examples and my first attempt is definitely returning datafrom the JSON, I can only assume that the error is in the last two lines of his code, or as stated my imputing incorrect parameters.

:S

 

[Andrew Berg]

If there was a trackback/debug I might know where to look, but its not yielding errors, its simply yielding nothing. What i dont know, is if it is the code that isnt working, or what I am inputting in the " print text1['rows'][0]['id']" that isnt working.

If fed a valid JSON object, json.loads() will return a regular dictionary. You can print (or pretty print with the pprint module) text1 to
see everything. If you're not familiar with dictionaries and lists, thoroughly read the tutorial before writing or modifying any more code.
http://docs.python.org/2/library/json.html#json.loads

 

[Andrew Berg]

Hi thanks for the reply Andrew, my first bit of code was heading in the right direction I was managing to pull out the usernames from the JSON, using REGEX.

It's also worth mentioning that regexes are almost always the wrong tool, especially for standardized formats like JSON. They can be very
useful when nothing saner will get the job done, but are a nasty mess with no real benefit otherwise.

 

[Andrew Edwards-Adams]

It's also worth mentioning that regexes are almost always the wrong tool, especially for standardized formats like JSON. They can be very

useful when nothing saner will get the job done, but are a nasty mess with no real benefit otherwise.

Thanks Andrew, yes I believe this is what the guy who provided me with the code was thinking. I was about to embark on what was definitely going to be an inefficient long regex.

 

[Mark Lawrence]

Thanks Andrew, yes I believe this is what the guy who provided me with the code was thinking. I was about to embark on what was definitely going to be an inefficient long regex.

Funny old world, there doesn't appear to be working code but someone is
already thinking about definite inefficiency. What evidence do you have
to support this claim?

 

[Andrew Edwards-Adams]

Funny old world, there doesn't appear to be working code but someone is

already thinking about definite inefficiency. What evidence do you have

to support this claim?



--

If you're using GoogleCrap™ please read this

http://wiki.python.org/moin/GoogleGroupsPython.



Mark Lawrence

haha true Mark, I have made mistakes like this before become ok at coding VBA to achieve things in the most inefficient manner possible. Something that I will learn from for the current python coding project.