Fastest Way to get list of streams and why some links dont have "url"

Hello, I’m trying to show a list of streamers but what I’m doing is pretty slow. I was wondering if there was a faster way to get a list of streamers.

def getStreamers(self, gameName):
    getSource = requests.get("https://api.twitch.tv/kraken/streams?game=" + gameName + "&limit=15")
    soup = BeautifulSoup(getSource.text)
    loadSource = json.dumps(getSource.text)
    text = json.loads(getSource.text)
    self.listOfStreamers = []
    print gameName
    for item in text["streams"]:
        krakenTwitchURL = item["_links"]["self"]
        if "https" not in krakenTwitchURL:
            krakenTwitchURL = krakenTwitchURL.encode("ascii").replace("http", "https")
        getUrl = requests.get(krakenTwitchURL)
        urlText = json.loads(getUrl.text)
        streamName = urlText["stream"]["channel"]["display_name"] 
        self.listOfStreamers.append(streamName)
        streamURL = urlText["stream"]["channel"]["url"]
        print streamURL

This works half of the time. When it works, its fine but its kinda slow. Is there a more time efficient way? Also, the other half of the time, I keep getting errors like KeyError: ‘stream’, and KeyError: ‘url’. When I tried to print out krakenTwitchURL, sometimes it has “url” and sometimes it doesn’t. I’m using this link as an example: https://api.twitch.tv/kraken/streams/dotapit.
When it doesn’t:

{"_links":{"self":"https://api.twitch.tv/kraken/streams/dotapit","channel":"https://api.twitch.tv/kraken/channels/dotapit"},"stream":{"_id":13224673232,"game":"Dota 2","viewers":12025,"created_at":"2015-02-19T17:03:43Z","_links":{"self":"https://api.twitch.tv/kraken/streams/dotapit"},"preview":{"small":"http://static-cdn.jtvnw.net/previews-ttv/live_user_dotapit-80x45.jpg","medium":"http://static-cdn.jtvnw.net/previews-ttv/live_user_dotapit-320x180.jpg","large":"http://static-cdn.jtvnw.net/previews-ttv/live_user_dotapit-640x360.jpg","template":"http://static-cdn.jtvnw.net/previews-ttv/live_user_dotapit-{width}x{height}.jpg"},"channel":{"_id":59168409,"name":"dotapit","created_at":"2014-03-18T16:16:57Z","updated_at":"2015-02-19T20:25:50Z","_links":{"self":"https://api.twitch.tv/kraken/users/dotapit"},"display_name":"dotapit","logo":"http://static-cdn.jtvnw.net/jtv_user_pictures/dotapit-profile_image-8ef58f9ee0f444ef-300x300.jpeg","bio":"The Dota Pit League is back! We\u2019re happy to announce the second season of our tournament in which the top teams in Europe and North America.","type":"user"}}}

When it does:

{"_links":{"self":"https://api.twitch.tv/kraken/streams/dotapit","channel":"https://api.twitch.tv/kraken/channels/dotapit"},"stream":{"_id":13224673232,"game":"Dota 2","viewers":12957,"created_at":"2015-02-19T17:03:43Z","_links":{"self":"https://api.twitch.tv/kraken/streams/dotapit"},"preview":{"small":"http://static-cdn.jtvnw.net/previews-ttv/live_user_dotapit-80x45.jpg","medium":"http://static-cdn.jtvnw.net/previews-ttv/live_user_dotapit-320x180.jpg","large":"http://static-cdn.jtvnw.net/previews-ttv/live_user_dotapit-640x360.jpg","template":"http://static-cdn.jtvnw.net/previews-ttv/live_user_dotapit-{width}x{height}.jpg"},"channel":{"_links":{"self":"https://api.twitch.tv/kraken/channels/dotapit","follows":"https://api.twitch.tv/kraken/channels/dotapit/follows","commercial":"https://api.twitch.tv/kraken/channels/dotapit/commercial","stream_key":"https://api.twitch.tv/kraken/channels/dotapit/stream_key","chat":"https://api.twitch.tv/kraken/chat/dotapit","features":"https://api.twitch.tv/kraken/channels/dotapit/features","subscriptions":"https://api.twitch.tv/kraken/channels/dotapit/subscriptions","editors":"https://api.twitch.tv/kraken/channels/dotapit/editors","videos":"https://api.twitch.tv/kraken/channels/dotapit/videos","teams":"https://api.twitch.tv/kraken/channels/dotapit/teams"},"background":null,"banner":null,"broadcaster_language":"en","display_name":"dotapit","game":"Dota 2","logo":"http://static-cdn.jtvnw.net/jtv_user_pictures/dotapit-profile_image-8ef58f9ee0f444ef-300x300.jpeg","mature":false,"status":"LIVE: Asus.Polar vs Team Tinker - DotaPit by G2A.com S3 w/ @MautDota & @TralfDota","partner":true,"url":"http://www.twitch.tv/dotapit","video_banner":null,"_id":59168409,"name":"dotapit","created_at":"2014-03-18T16:16:57Z","updated_at":"2015-02-19T20:25:50Z","delay":120,"followers":37492,"profile_banner":null,"profile_banner_background_color":"null","views":5254263,"language":"en"}}}

As you can see, sometimes it has “url” and sometimes it doesn’t. This also happens with “stream” when i call streamURL = urlText[“stream”][“channel”][“url”].

To speed up your script: there is no reason for you to requests.get(item["_links"]["self"]). The content of item and urlText will be identical.

As for item["channel"]["url"] being undefined randomly: I believe there’s currently a bug where we don’t return all fields when we should. We’re looking into it.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.