Opened 3 years ago

Closed 3 years ago

#13779 closed enhancement (implemented)

Create script to upload bundles to Google drive

Reported by: ilv Owned by: poly
Priority: High Milestone:
Component: Applications/GetTor Version:
Severity: Keywords: revamp gettor provider
Cc: mrphs, sukhbir, poly@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

A script for uploading bunldes to this cloud service should work as follows:

  1. Read the content of a directory and recognize:
    • The operating systems for which is intended (.tar.gz for Linux, .dmg for OSX, .exe for Windows)
    • The architecture (32/64 bits)
    • The locale. This could be pointed out by two characters (e.g. 'de') or four characters separated in the middle by a dash (e.g. 'es-ES').
    • The sha256 of the file (calculate it).
    • The associated file signature (samename.asc).
  1. Upload everyone of those files as long as they have their corresponding signature (.asc) file.
  1. Get the fingerprint of the PGP key that signed the bundles.
  1. Create the links file and add a link for every bundle uploaded. For that matter you should use the create_links_file and add_link methods available on gettor.core.

It's recommended to study how the script for uploading bundles to Dropbox works.

Note that the codebase for this can be found on https://github.com/ileiva/gettor.git

Child Tickets

Change History (9)

comment:1 Changed 3 years ago by poly

Cc: poly@… added
Owner: changed from sukhbir to poly
Status: newassigned

I've talked to ilv and will be working on this component. I'll probably implement it using the google drive API.

Last edited 3 years ago by poly (previous) (diff)

comment:2 Changed 3 years ago by ilv

Priority: minormajor

Great, if you have any doubts please ping me on #tor-dev.

comment:3 Changed 3 years ago by poly

Status: assignedneeds_review

I have implemented google drive support. This consists of three main files:

  1. Created "UPLOAD-GOOGLE-DRIVE" with detailed instructions on how to setup and use script
  2. Created "drive.cfg" store store creds and general config
  3. Created "bundles2drive.py" the script that actually uploads the files and fetches the links.

Find the commit here:
https://github.com/0xPoly/gettor/commit/1eda9c6ca26623867bc9e49b4218b1b5e4108713

Here is a sample 'drive.links' file generated:
http://pastebin.com/iFz6285y

When the code review is done, please comment here or shoot me an email and I'll send a pull request.

comment:4 Changed 3 years ago by ilv

Hi poly, thanks for this. My comments:

  • I had no problems with steps 1) and 2), all good.
  • When I tried step 3) the script raises an UploadError error which is not defined. For some reason something fails when it tries to do drive_service.files().insert(...). Here is my output:
Uploading 'tor-browser-linux64-3.6.2_en-EN.tar.xz'...
Traceback (most recent call last):
  File "bundles2drive.py", line 291, in <module>
    uploaded_files = upload_files(drive_service, upload_dir)
  File "bundles2drive.py", line 182, in upload_files
    raise UploadError
NameError: global name 'UploadError' is not defined

What could be the reason?

Also, where can I see all the files that I have uploaded?

Thanks again.

comment:5 Changed 3 years ago by poly

Hmmm... I've pushed some code fixes. Find them at https://github.com/0xPoly/gettor/commit/51ca1cdd03377d2a8aa8b203bf2dfac8c6f40680 (was watching for wrong error + some forgotten imports)

It would now print the error message if there is trouble uploading.

Have you created the OAuth2 key as described? Are there any earlier error messages?

You can check the files that you have uploaded through drive.google.com. When the upload is successful, it will print the upload links to the console and into the links file.

comment:6 Changed 3 years ago by poly

Can you please make sure all the following is also done?

  1. Go to https://console.developers.google.com/project and make sure there is a project called 'gettor'
  2. Click on 'gettor' and choose 'APIs' under 'APIs & Auth' in the left sidebar.
  3. Make sure that 'Drive API' has a green ON to the left of it. If not, push the OFF button - it will become green.
  4. Go to 'Credentials' in the left sidebar. Copy the client ID and client secret *only* into drive.cfg

Thanks

comment:7 Changed 3 years ago by ilv

I checked all the steps you mentioned and updated the code but I'm still not able to get it working. Here is the output I receive when I execute it: http://pastebin.com/vkQv8Q23

Any ideas?

P.S.: I had to change 'HttpError' by 'errors.HttpError' in lines 183 and 194.

comment:8 Changed 3 years ago by ilv

I asked a friend to try this and it worked quite well. It must be something related to my Linux configuration or something. I still don't have a reply from sukhe. Let's wait a couple more days and then we merge it.

comment:9 Changed 3 years ago by poly

Resolution: implemented
Status: needs_reviewclosed
Note: See TracTickets for help on using tickets.