Opened 4 years ago

Closed 3 years ago

#9003 closed defect (fixed)

Twitter example uses deprecated 1.0 API

Reported by: ovnicraft Owned by: ashishnitinpatil
Priority: Low Milestone:
Component: Core Tor/Stem Version:
Severity: Keywords: website
Cc: ashishnitinpatil@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by atagar)

Our example for reading Twitter feeds uses the 1.0 version of the Twitter API. This API has now been deprecated and queries fail with...

{"errors": [{
  "message": "The Twitter REST API v1 is no longer
              active. Please migrate to API v1.1.
              https://dev.twitter.com/docs/api/1.1/overview.",
  "code": 68
}]}

This causes our example to fail with...

atagar@morrigan:~/Desktop/stem$ python twitter_script 
Traceback (most recent call last):
  File "twitter_script", line 39, in <module>
    print "%i. %s" % (index + 1, tweet["created_at"])
TypeError: string indices must be integers

Patches welcome for migrating the example to the 1.1 API!

Child Tickets

Attachments (1)

0001-Fixed-twitter-example-to-work-with-twitter-API-versi.patch (4.1 KB) - added by sambuddhabasu 3 years ago.

Download all attachments as: .zip

Change History (21)

comment:1 Changed 4 years ago by atagar

Status: newneeds_information

Hi ovnicraft, that tutorial is failing for you because tor is talking a really long time to start up (so stem is timing out). Please try the first client usage tutorial - that will print out the bootstrap messages so we'll get a better idea of what it's getting hung up on.

comment:2 Changed 4 years ago by ovnicraft

Hi, ok following first part, i got it: https://gist.github.com/ovnicraft/5762810 my issue was solved.

So with Read Twitter they has deprecated api v1, now they recommend use 1.1.
I was reading about changes in API and twitter forces auth to all requests maybe tutorial needs some update about it or got how to make requests to twitter w/o authentification.

Cristian,
All the best

comment:3 Changed 4 years ago by atagar

Priority: normalminor
Status: needs_informationaccepted
Summary: Crash stem tutorial in ExitNodesTwitter example uses deprecated 1.0 API

So with Read Twitter they has deprecated api v1, now they recommend use 1.1.

Drats. Thanks for the heads up, swapping this ticket to be about the API change.

I was reading about changes in API and twitter forces auth to all requests maybe tutorial needs some update about it or got how to make requests to twitter w/o authentification.

Looks like we'll want application-only OAuth so the example will work for folks without twitter accounts (... like me).

I'm presently focusing on other stem improvements so I don't plan to invest time here in the near future - patches welcome!

comment:4 Changed 4 years ago by atagar

Description: modified (diff)

comment:5 Changed 4 years ago by atagar

Added a warning to the example citing this ticket (change).

comment:6 Changed 4 years ago by atagar

Keywords: website added

comment:7 Changed 4 years ago by ashishnitinpatil

Cc: ashishnitinpatil@… added
Status: acceptedneeds_review

Sorry Sorry! Been busy with personal projects for quite a while.
TBH, I was lazy when I came up with the API 1, the 1.1 was way above me, so, now that I have used it in my own apps, I quite familiar.
Following does the job, but I would really appreciate if you change the keys & secrets to someone else's account (better if the account is made for stem).
The script works fine without Tor. (Didn't test it with Tor. Sorry. But it will work)
https://pastee.org/tf8s9

comment:8 Changed 4 years ago by atagar

Status: needs_reviewneeds_information

Thanks, a couple questions...

  • Mind a link to the instructions for consumer_key, consumer_secret, and access tokens? I'm not personally a twitter user.
  • Are you sure this works? In particular you have...
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

api = tweepy.API(auth)

... but tweepy is undefined. Does this mean we can't effectively provide a tutorial using urllib any longer?

comment:9 Changed 4 years ago by ashishnitinpatil

Status: needs_informationneeds_review
  1. The keys & secrets can be obtained by signing up on dev.twitter.com (if you don't have an account, you need to create one on twitter first) & then creating an application. After the app creation (filling the form on dev.twitter.com/apps/new),

you'll be redirected to the app settings page which will have the consumer_key & consumer_secret. The access tokens can be generated there itself.

  1. I am sorry, I need to improve on my documentation skills. Tweepy is a python library for creating twitter apps. Source - https://github.com/tweepy/tweepy

Unfortunately, the deprecation of API 1.0 forces us to use OAuth, hence I've used tweepy. I actually skipped everything that was unnecessary so that the example can be kept as simple & readable as possible. Finally, as you might've guessed, I forgot to import tweepy. I need to improve on verifying too.

So, we can not provide a tutorial using urllib anymore.
And, dont forget to "import tweepy".
Sorry for the mess-up. This bug shouldn't have been there in the first place if I had showed more courage on oauth. Won't happen again! :) Learning is fun!

comment:10 in reply to:  9 Changed 4 years ago by ashishnitinpatil

Replying to ashishnitinpatil:

  1. The keys & secrets can be obtained by signing up on dev.twitter.com (if you don't have an account, you need to create one on twitter first) & then creating an application. After the app creation (filling the form on dev.twitter.com/apps/new),

you'll be redirected to the app settings page which will have the consumer_key & consumer_secret. The access tokens can be generated there itself.

Only one twitter account is needed. Same is used to sign in on dev.twitter.

comment:11 Changed 4 years ago by atagar

Status: needs_reviewneeds_information

Tweepy is a python library for creating twitter apps. Source - https://github.com/tweepy/tweepy

Hmmm, in that case it's not a very good example for our purposes. We were configuring urllib to go through a tor socks proxy, if we're using tweepy instead (which might or might not be using urllib) then this is no expanding on the earlier examples for using urllib through tor.

Maybe I should simply remove it from the page. Pity since this had been a nice real-world example.

Thoughts?

comment:12 Changed 4 years ago by ashishnitinpatil

Owner: changed from atagar to ashishnitinpatil
Status: needs_informationaccepted

Looks like I am not thinking straight, or thinking at all.
I will try to work on this, but I can't guarantee any time-limit.
Looks like I'll need to extract all that is needed for the oauth & also the actual tweet reading. Or maybe tweak tweepy to use the given socks proxy.
Well, I don't whether to remove it or not, but it's true that the example is pretty useless now since it won't work.

comment:13 in reply to:  12 ; Changed 4 years ago by ovnicraft

Replying to ashishnitinpatil:

Looks like I am not thinking straight, or thinking at all.
I will try to work on this, but I can't guarantee any time-limit.
Looks like I'll need to extract all that is needed for the oauth & also the actual tweet reading. Or maybe tweak tweepy to use the given socks proxy.
Well, I don't whether to remove it or not, but it's true that the example is pretty useless now since it won't work.

I can talk about with Jake here in Ecuador we'll get a little hackathon here and i will work in this. i hope i can fix it

comment:14 in reply to:  13 ; Changed 4 years ago by ashishnitinpatil

Replying to ovnicraft:

I can talk about with Jake here in Ecuador we'll get a little hackathon here and i will work in this. i hope i can fix it

I am available for all the help you need! Today is the last day of my exams :)

comment:15 in reply to:  14 Changed 4 years ago by ovnicraft

Replying to ashishnitinpatil:

Replying to ovnicraft:

I can talk about with Jake here in Ecuador we'll get a little hackathon here and i will work in this. i hope i can fix it

I am available for all the help you need! Today is the last day of my exams :)

Great for that, we will working in thursday we can chat in IRC with all of them !

comment:16 Changed 3 years ago by sambuddhabasu

Is anyone working on this issue? If no one is working, I am interested in fixing this.

comment:17 Changed 3 years ago by atagar

Hi sambuddhabasu. Nope, nobody's working on this.

comment:18 Changed 3 years ago by atagar

Thanks sambuddhabasu! Does this work as-is or does it require the individual to fill in the constants you defined? I suspect the later unless Twitter allows anonymous read access.

comment:19 Changed 3 years ago by sambuddhabasu

It requires the user to fill in the constants. Should I add a comment in the tutorial which specifies this?

comment:20 Changed 3 years ago by atagar

Resolution: fixed
Status: acceptedclosed

Probably would be useful. I added the link you mentioned on irc and revised the example to be a bit shorter...

https://gitweb.torproject.org/stem.git/commit/?id=d293eb8

Thanks sambuddhabasu! Unfortunately I don't have a twitter account to double check that the new example works so feel free to reopen if I messed something up. ;)

Note: See TracTickets for help on using tickets.