On using Python, the Digg API, and simplejson

Here are some quick notes on using the Digg API with a Python script. Note, there is a Python toolkit for Digg but I just used urllib2 and the Digg API endpoints for the sake of simplicity.

I wanted the output in JSON format so I specified the response type as JSON. To decode JSON directly to a Python data structure, I used simplejson.

Here is a simple example which returns the JSON output for the Digg story Dell vs. Apple: This Time it's Personal which has a "clean title" of Dell_vs_Apple_This_Time_it_s_Personal.

#!/usr/bin/env python

import urllib2

APPKEY = 'http%3A%2F%2Fwww.example.com'
story_clean_title = 'Dell_vs_Apple_This_Time_it_s_Personal'
url = ''.join([
        'http://services.digg.com',
        '/story/%s' % story_clean_title,
        '?appkey=%s' % APPKEY,
        '&type;=json',
        ])
json = urllib2.urlopen(url).read()
print json

Results:

{"timestamp":1219168025,"total":"1","offset":0,"stories":[{"id":"8038250","link":"http:\/\/www.businessweek.com\/magazine\/content\/08_34\/b4097022701166.htm?campaign_id=rss_daily","submit_date":1219047878,"diggs":763,"comments":198,"title":"Dell vs. Apple: This Time it's Personal","description":"Now Bucher is again squaring off against his former company. He's spearheading an ambitious plan at Dell (DELL) to break Apple's dominant hold on the digital entertainment market.","promote_date":1219095692,"status":"popular","media":"news","user":{"name":"msaleem","icon":"http:\/\/digg.com\/users\/msaleem\/l.png","registered":1126518985,"profileviews":136052,"fullname":"Muhammad Saleem"},"topic":{"name":"Apple","short_name":"apple"},"container":{"name":"Technology","short_name":"technology"},"thumbnail":{"originalwidth":370,"originalheight":245,"contentType":"image\/jpeg","src":"http:\/\/digg.com\/apple\/Dell_vs_Apple_This_Time_it_s_Personal\/t.jpg","width":80,"height":80},"href":"http:\/\/digg.com\/apple\/Dell_vs_Apple_This_Time_it_s_Personal"}],"count":1}

Here is a slightly less simple example which returns the comments for the same story above. It uses simplejson to decode the Digg story JSON data and get the story ID which is then used to get the comment data.

#!/usr/bin/env python

import simplejson
import urllib2
from pprint import pprint

APPKEY = 'http%3A%2F%2Fwww.example.com'

def main():
    story_clean_title = 'Dell_vs_Apple_This_Time_it_s_Personal'

    # get story
    json = get_json('/story/%s' % story_clean_title)
    pydata = simplejson.loads(json)
    story_id = pydata['stories'][0]['id']

    # get comments
    json = get_json('/story/%s/comments' % story_id)
    pydata = simplejson.loads(json)
    pprint(pydata)

def get_json(endpoint):
    """ returns json data for requested digg endpoint 
    """
    url = ''.join([
            'http://services.digg.com',
            endpoint,
            '?appkey=%s' % APPKEY,
            '&type;=json',
            ])
    return urllib2.urlopen(url).read()

if __name__ == '__main__':
    main()

Results:

{u'comments': [{u'content': u"For those who are having trouble understanding what this is about.  This is not about creating a competing closed platform.  This is about creating a standard platform for selling music online.  Imagine iTunes but with the ability to add other music stores into it.  So when you want to buy a song, you can browse this music store or that music store.  And all will work with your mp3 player.  Whether it is a basic one that mounts as an external usb drive, or one that is able to sync the songs up the way iTunes does it with the iPod.  And we know it is gonna be good because the guy that is driving it worked at Apple, so he knows what quality is.  I'm sure it will have the ability to import or export your music library to iTunes if you so choose.  But the point is that it is trying to create a standard that any device maker can follow.  And hopefully, it will have no drm.  Otherwise it's as useless as iTunes with its drm.\n\nRemember the same thing happened when windows 3.1 came along and immediately sold as many as 10x the number of pc's compared to mac's at the time.  Anytime there is an open standard, no matter how good the closed standard is, the open one wins because that means lower prices for consumers since any manufacturer can use it.  Apple thrives on closed standards when it comes to selling their products because it enables them to lock their customers in.  And before they know it, they're locked into it and can't get out of it without great expense.\n\nI know that the Apple fanboi's are gonna bury me for this, but I'll say it anyways.  It was nice being on top.  But you can't be on top forever.",
                u'date': 1219163527,
                u'down': 0,
                u'id': 17963749,
                u'level': 0,
                u'replies': 0,
                u'replyto': None,
                u'root': 17963749,
                u'story': 8038250,
                u'up': 1,
                u'user': u'pyrates'},
               {u'content': u'@thinkdifferent: I should of specified its a $1 cheaper when you buy the full album. Single tracks are the same price but albums are usually about $8.99. ',
                u'date': 1219159148,
                u'down': 0,
                u'id': 17961525,
                u'level': 0,
                u'replies': 0,
                u'replyto': None,
                u'root': 17961525,
                u'story': 8038250,
                u'up': 1,
                u'user': u'mrgermy'},
               {u'content': u'Zune XPS w/ Vista Ultimate combo deal.',
                u'date': 1219146354,
                u'down': 0,
                u'id': 17956943,
                u'level': 0,
                u'replies': 0,
                u'replyto': None,
                u'root': 17956943,
                u'story': 8038250,
                u'up': 1,
                u'user': u'hurdboy'},
               {u'content': u'Digg: "Dell vs. Apple: This Time it\'s Personal"\nBusiness Week: "Bucher says his quest to challenge Apple is all business and not personal."\n\nBurried as inacurate.',
                u'date': 1219134278,
                u'down': 0,
                u'id': 17954900,
                u'level': 0,
                u'replies': 0,
                u'replyto': None,
                u'root': 17954900,
                u'story': 8038250,
                u'up': 1,
                u'user': u'KAMiKAZOW'},
               {u'content': u'Dell is junk and Apple is overpriced',
                u'date': 1219117388,
                u'down': 0,
                u'id': 17950148,
                u'level': 0,
                u'replies': 0,
                u'replyto': None,
                u'root': 17950148,
                u'story': 8038250,
                u'up': 1,
                u'user': u'DeuceDiggalow'},
               {u'content': u'Dell, all you have to do to kick apple in the jewels is get your act together with Ubuntu.\n\nAmarok will do the rest.',
                u'date': 1219112434,
                u'down': 2,
                u'id': 17948056,
                u'level': 0,
                u'replies': 4,
                u'replyto': None,
                u'root': 17948056,
                u'story': 8038250,
                u'up': 3,
                u'user': u'ethana2'},
               {u'content': u'Apple = Overpriced, under-featured, but pretty\nDell = priced right, full-featured, and awesome looking',
                u'date': 1219105961,
                u'down': 8,
                u'id': 17945330,
                u'level': 0,
                u'replies': 2,
                u'replyto': None,
                u'root': 17945330,
                u'story': 8038250,
                u'up': 3,
                u'user': u'freesf'},
               {u'content': u'Go Dell! Best of luck to you :)',
                u'date': 1219104145,
                u'down': 0,
                u'id': 17944493,
                u'level': 0,
                u'replies': 0,
                u'replyto': None,
                u'root': 17944493,
                u'story': 8038250,
                u'up': 0,
                u'user': u'Sabre24q7'},
               {u'content': u'Actually the Dell DJ wasnt that bad for its time. Obviously now its looks very dated and far from the best now, but at its time it was a decent player and had pretty good sound quality. It also had more features than the ipod does even today.\n\n Creative was the people who made it and it was a very basic player but it did what it was suppose to and had a lot of room on it for a cheap price. ',
                u'date': 1219104085,
                u'down': 2,
                u'id': 17944474,
                u'level': 0,
                u'replies': 1,
                u'replyto': None,
                u'root': 17944474,
                u'story': 8038250,
                u'up': 1,
                u'user': u'jsc315'},
               {u'content': u"I'm not saying this will or won't succeed. I'm just pointing out something people seem to be missing.\n\nJust because Dell customers aren't pompous and loudmouthed doesn't mean they aren't loyal and happy customers. There are a LOT of Dell users that like the company and what the products they make.",
                u'date': 1219102724,
                u'down': 1,
                u'id': 17943861,
                u'level': 0,
                u'replies': 1,
                u'replyto': None,
                u'root': 17943861,
                u'story': 8038250,
                u'up': 2,
                u'user': u'Urkel'}],
 u'count': 10,
 u'offset': 0,
 u'timestamp': 1219168299,
 u'total': u'55'}