#5995 closed project (fixed)

Build Q&A Forum with translations

Reported by: phobos Owned by: phobos
Priority: normal Milestone:
Component: Company Version:
Keywords: SponsorI, SponsorL Cc: tails@…, runa, weasel, mo
Actual Points: Parent ID:
Points:

Description

Build a Question and Answer forum where the community votes on the top answers to a question related to anything Tor. The software should support translations inline or as a separate forum for various languages. Sort of like a stackexchange for Tor. The software should be published as freely licensed, open source software (BSD 3-Clause likely).

Child Tickets

TicketSummaryOwner
#6926Deploy the first version of the community softwarephobos
#7765setup VM for askbot python install

Change History (36)

comment:1 follow-up: Changed 23 months ago by runa

what happens when the top question asks for ways to download copyrighted material? or ways to get to Silk Road? or get to a specific forum with shady images? What are we going to do, if anything?

comment:2 in reply to: ↑ 1 Changed 23 months ago by phobos

Replying to runa:

what happens when the top question asks for ways to download copyrighted material? or ways to get to Silk Road? or get to a specific forum with shady images? What are we going to do, if anything?

Frankly, this is a legal quagmire. I'd love a better answer than our current plan. The current answer to this question is to have moderators either remove the question if it's illegal (in the USA ) or give it infinite negative points to bury it. We'd rather spend our money on improving tor, not fighting silly lawsuits. We already have this situation with rt and our mailing lists (and their public archives).

comment:3 follow-up: Changed 22 months ago by arma

Would reddit accomplish the goal here? If not, what does it lack?

comment:4 in reply to: ↑ 3 ; follow-up: Changed 22 months ago by phobos

Replying to arma:

Would reddit accomplish the goal here? If not, what does it lack?

My initial concerns are:

  1. who owns the data?
  2. how do we get our data out of reddit?
  3. who has control over our content and data?
  4. do we want to promote conde nast via reddit?
  5. how does reddit manage tor exit relays?
  6. will reddit run a hidden service for us?

comment:5 Changed 22 months ago by arma

intrigeri points out the brainstorming page that tails has: https://tails.boum.org/todo/improve_the_forum/#index2h1

comment:6 in reply to: ↑ 4 Changed 22 months ago by runa

Replying to phobos:

Replying to arma:

Would reddit accomplish the goal here? If not, what does it lack?

My initial concerns are:

  1. who owns the data?
  2. how do we get our data out of reddit?
  3. who has control over our content and data?
  4. do we want to promote conde nast via reddit?
  5. how does reddit manage tor exit relays?
  6. will reddit run a hidden service for us?

We can just set up our own Reddit-ish-site with https://github.com/reddit/reddit

comment:7 Changed 21 months ago by karsten

  • Keywords SponsorZ added
  • Milestone Sponsor Z: November 1, 2013 deleted

Switching from using milestones to keywords for sponsor deliverables. See #6365 for details.

comment:8 follow-up: Changed 20 months ago by mikeperry

  • Keywords SponsorZ-large added; SponsorZ removed

As a sponsor task for building our own forum, this will probably be painful to keep up to date and manage if we run it ourselves. However, we could use the deliverable to fund a web service admin of some kind. Ideally someone who could also take care of trac, gettor, bridges, check, etc. Hence I'm tossing SponsorZ-large on it.

Despite this, I'm not opposed to some existing forum-capable integrated support solution, like desk.com or zendesk. Would obviously be cheaper than a whole person worth of admin...

I really don't even think reddit.com itself is that bad an idea to try out until it fails, if we have no $$ to spend on any of this. Will we be able to shut down or at least rename the reddit forum if we end up hating it? If so, seems worth a try? If not, hrmm..

comment:9 in reply to: ↑ 8 Changed 20 months ago by runa

Replying to mikeperry:

I really don't even think reddit.com itself is that bad an idea to try out until it fails, if we have no $$ to spend on any of this. Will we be able to shut down or at least rename the reddit forum if we end up hating it? If so, seems worth a try? If not, hrmm..

We could set up our own Reddit clone. Probably nicer and better and easier than trying to use a subreddit on reddit.com.

comment:10 Changed 19 months ago by phobos

  • Keywords SponsorI added; SponsorZ-large removed

comment:11 Changed 19 months ago by T(A)ILS developers

  • Cc tails@… added

comment:12 Changed 18 months ago by phobos

https://help.openstreetmap.org is an example of a successful osqa deployment.

comment:13 follow-up: Changed 18 months ago by phobos

https://answers.atlassian.com/ just launched the other day, based on osqa.

comment:14 Changed 18 months ago by runa

  • Cc runa added

comment:15 in reply to: ↑ 13 Changed 18 months ago by runa

  • Cc weasel added

Replying to phobos:

https://answers.atlassian.com/ just launched the other day, based on osqa.

The footer says it's powered by OSQA. It might be worth nothing that OSQA is currently not in Debian, meaning we'll have fun setting it up and hosting it.

comment:16 Changed 18 months ago by runa

I suggest we set up ask.torproject.org and get something going. OSQA seems like the best option for a self-hosted solution.

comment:17 Changed 18 months ago by T(A)ILS developers

Please have another look at https://tails.boum.org/todo/improve_the_forum/

OSQA do not really have an upstream any more. That lead us to reconsider it as a viable option.

Askbot and Shapado both have active upstreams which also offer support and development services.

Shapado already supports multiple groups and languages, even if the support could be improved. Such feature could be used to have one installation shared by Tor, Tails and Orbot and we could move questions from one group to another.

If you really wish to go to the OSQA, we have a bunch of patches already regarding at least dependency on Google CDN and making email optional. IIRC, it does not have any export option, so it probably means it will not be possible to move to other software without a substancial amount of work.

comment:18 Changed 17 months ago by phobos

  • Keywords SponsorL added

comment:19 Changed 17 months ago by runa

I think we should stay away from projects which are no longer maintained. We should also avoid paid solutions, such as Askbot and Shapado, when possible. I have sent an email to Stack Exchange to ask how we can go about setting up ask.tpo and get that up and running, who owns the data, how we moderate it, and so on.

comment:20 follow-up: Changed 17 months ago by ague

I don't really understand why you describe Askbot and Shapado as "paid solutions": they are both free software (GPLv3 and AGPLv3, respectively); adding the few missing features could be done by contracting the authors or freely by volunteers; hosting can be done using Tor project infrastructure.

Would Stack Exchange provide the service for free? What is their logging policy? Who would be paying the costs?

comment:21 in reply to: ↑ 20 ; follow-up: Changed 17 months ago by runa

Replying to ague:

I don't really understand why you describe Askbot and Shapado as "paid solutions": they are both free software (GPLv3 and AGPLv3, respectively); adding the few missing features could be done by contracting the authors or freely by volunteers; hosting can be done using Tor project infrastructure.

You're right. My brain ignored the links to the git repositories and only saw the pricing plans available on websites. Have you (read: Tails) decided on a solution yet?

Would Stack Exchange provide the service for free? What is their logging policy? Who would be paying the costs?

Stack Exchange would provide the service for free. I don't know what their logging policy is, but will be sure to ask when they reply to my email.

How about we figure out whether to go for a self-hosted solution vs third-party-hosted solution first?

comment:22 in reply to: ↑ 21 Changed 17 months ago by T(A)ILS developers

We definitely want a self-hosted solution and not a third-party-hosted solution. We want to be able to configure all the privacy features that we want and to have full power on the logs.

We would love to see this solution being shared by Tor, Tails, and possibly other projects.

The conclusion of our benchmarking [1] is that there is no really good solution at the moment to do what we want to do: either only for Tails, either for both Tor and Tails. The best candidates for improvements would be Askbot and Shapado.

[1] https://tails.boum.org/todo/improve_the_forum

But in both cases it would require working with the upstream to get the features we need implemented (either by them, either by someone else and then get merged). Both upstream are ready to be hired to work on custom features, as advertised on their websites. Nobody in our team is really excited about doing this work but some of us might end up doing it if
there is really no other choice.

Tails might be interested in moving its own forum to one of those tools even before all the improvements that are listed here are implemented. We ended up postponing our migration in order to pick a common tools for Tor and Tails, so we want to make sure that we choose the right target for this early migration.

In other words, we would like to first:

  • decide upon a single tool that both Tor and Tails would be happy with
  • make plans to implement the features that are missing (who does the work? who pays? etc.)
  • ensure there is a migration path to the new features

... and once this is all decided and planned, Tails would still be happy to migrate its own forum even it's not on a shared instance for some time.

We would be ready to cooperate with you on this project: choosing between one of those two, coming up with a more concrete list of requirements, dealing with the upstream, etc.

The features we think are missing for a shared instance are:

For both Askbot and Shapado

  • They lack anonymous posting: both requires user account but we want to make the email address field optional. That should be easy.
  • They are offering a commercial hosting of their product and as a side effect setting up a dedicated instance is tricky or buggy. We might need some help or debugging while installing our own.
  • Review the email notifications options and make sure they match our needs (still to be defined). Both already have an email notification mechanisms.
  • Review and complete the translations for the language we want to support the most (list to be defined).
  • When typing a new question, they update on the fly a list of matching questions through Ajax. There should be a similar mechanism to avoid submitting duplicated questions that works with slow connections and without JavaScript.
  • The main features of the application should work with JavaScript disabled. At the moment, both login mechanisms do not work with JavaScript disabled.

Askbot

  • Handle groups (or categories or subforums). People should be able to use a single login for both Tor and Tails but the administration rights, karma, badges, and email notifications should be separate. It should be possible to move questions from one group to the other.
  • Multilingual support. I'm not sure that's a strict requirement but Shapado almost has it: the possibility of having the interface translated in the preferred language of a user, of showing and getting notified only of questions in languages you know, etc.

Shapado

  • Fix the multilingual support (the filtering by language is broken).
  • Add a feature to move questions between groups.
  • Add the possibility to vote for a question.
  • Fix the password size limitation.

Bonus

  • Allow answering to questions by email.

What do you think?

comment:23 Changed 17 months ago by mo

  • Cc mo added

comment:24 Changed 17 months ago by mo

I don't think a hosted Q&A forum is acceptable due to lack of control (IP logs etc). I would be willing to maintain the platform and maybe hack on the code a bit.

Both projects don't see much dev activity. My vote goes for Askbot.

Shapado

  • language selection works for me. it is very slow on the demo site, which might be the reason why you thought it was broken.
  • you can answer tickets via mail, but only if you know the specific address that is at the bottom of each thread. i don't see how this is a very useful feature?
  • rails
  • irc: 3 users, no activity (freenode #shapado)
  • "some bugs weren't fixed in two years from the original setup" http://meta.osqa.net/questions/11809/how-to-migrate-content-from-shapado-to-osqa

Askbot

  • users can ask questions via mail. i find that more useful than being able to answer via email.
  • in my opinion the user interface looks way better than shapado
  • python (django)
  • has unit tests
  • higher number of commits in the last months
  • irc: 2 users, no activity (freenode #askbot)

General

  • it would be nice if one could post both question and answer at the same time. that way, one could post sanitized versions of support tickets quickly. i even see the possible scenario of a closed "forum" that only lives from sanitized ticket answers (as a public searchable database of tickets, which is lacking at the moment).
  • i am contacting the askbot dev to discuss this a bit with him; let's see what he says.

comment:25 Changed 17 months ago by mo

Is it really a hard requirement to have a separate forum for TAILS? Couldn't we just get away with an appropriate tag?

comment:26 Changed 17 months ago by mo

The core dev of Askbot got back to me pretty quickly. Kudos. :)

[12:38:09] <    gamambel> | evgeny: hi! thanks for getting back to me so quickly.
[12:38:32] <    gamambel> | evgeny: what do you mean by "user groups feature"? i didn't see anything resembling "subforums".
[12:39:09] <      evgeny> | sure - groups are like tags that mark people and content
[12:39:21] <    gamambel> | evgeny: also, we require guest postings from non-registered users. this has been a feature suggestion 
          on askbot forums already. what do you think, would it be manageable to hack that into askbot?
[12:39:50] <      evgeny> | the question is - what are you going to do with spam?
[12:39:57] <    gamambel> | evgeny: to be clear, i am not asking you to add all "our" features. we are considering, if you/your 
          time doesn't have time or the willingness to implement it, to maybe do it ourselves.
[12:40:18] <      evgeny> | you're going to get a lot of robot-posted trash
[12:40:38] <    gamambel> | evgeny: CAPTCHA for guest postings? add them to a moderation queue (not visible to the public)? i 
          don't really know.
[12:40:47] <      evgeny> | possible
[12:41:09] <      evgeny> | anonymous user can be emulated by auto-assigning posts to a special user account
[12:41:27] <      evgeny> | some spam will get past captcha too
[12:41:41] <      evgeny> | we have moderation
[12:41:56] <      evgeny> | also - we have "reply by email"
[12:42:04] <    gamambel> | especially since we want to avoid recaptcha because it transmits the user IP to google servers...
[12:42:31] <      evgeny> | I think you will need some form of captcha to deal with spam
[12:42:32] <    gamambel> | can i interact with askbot completely by mail? say, subscribe to a certain tag, get new questions my 
          mail, and then answer via mail?
[12:42:40] <      evgeny> | or some clever system for auto-dedection
[12:42:41] <    gamambel> | how do users "log in" via mail?
[12:42:54] <      evgeny> | "interact completely" - no
[12:43:03] <      evgeny> | but you can ask, answer and comment
[12:43:14] <      evgeny> | users first register on the site
[12:43:20] <      evgeny> | then they get an email
[12:43:24] <    gamambel> | that would be killer :) most tor people don't really like the idea of a web forum, but are used to 
          answering questions on mailinglists.
[12:43:56] <      evgeny> | sure - it could be useful
[12:45:28] <    gamambel> | one last question for now then: you seem to be available for contract work if we wanted a certain 
          feature but couldn't do it ourselves. is that correct?
[12:45:47] <      evgeny> | yes - that's what we do. full time now.
[12:46:23] <    gamambel> | nice! hope it works out for you :)
[12:46:59] <      evgeny> | I'm working on askbot full time for a year now, it's going quite well.
[12:47:14] <      evgeny> | based in Chile now
[12:47:48] <    gamambel> | if i have more questions i will find you here or via email. if you have comments, likewise. are you 
          okay with me posting this IRC log with our trac ticket?
[12:48:00] <    gamambel> | thank you for taking the time.
[12:48:29] <      evgeny> | sure thing
[12:48:48] <    gamambel> | great!

comment:27 Changed 17 months ago by mo

"I've looked through your ticket - we are currently working on multilingual support and have user groups feature. Some multilingual support will be done by early January. One company needs this then so we have a deadline here."

comment:28 Changed 16 months ago by phobos

I think askbot is looking like the leading candidate. If we cannot have an anonymous user, we should have a single user account with public credentials so anyone can login to it (just like our wiki/trac). Another option is to support openid/oauth logins via gmail, facebook, twitter, etc for users that don't want to create yet another account to ask a question.

I think we just set it up, do some basic branding of the site, and start adding content to it from the faq and help desk.

comment:29 Changed 16 months ago by phobos

weasel and I discussed, going with askbot sounds great. We already have a postgres db host, so hosting and maintaining the python askbot front end is up to runa, mo, and I.

comment:30 Changed 16 months ago by runa

Sounds good, who is setting it up where?

comment:31 follow-up: Changed 16 months ago by mo

If I understood weasel correctly, we should avoid using pip/easy_install. It does not have authentication and uses plain http to download packages.

Not all Askbot dependencies are available as Debian package. Only a few are listed on the Askbot website. Not even the Django in Debian satisfies Askbot's requirement (wants >= 1.3.1, debian has 1.2.3).

Here's what "Askbot install" puts into python's site-packages.

akismet-0.2.0-py2.6.egg
askbot-0.7.47-py2.6.egg
beautifulsoup4-4.1.3-py2.6.egg
Coffin-0.3.7-py2.6.egg
distribute-0.6.10-py2.6.egg
Django-1.4.3-py2.6.egg-info
django_celery-2.2.7-py2.6.egg
django_countries-1.0.5-py2.6.egg
django_followit-0.0.3-py2.6.egg
django_keyedcache-1.4_6-py2.6.egg
django_kombu-0.9.2-py2.6.egg
django_recaptcha_works-0.3.4-py2.6.egg
django_robots-0.9.1-py2.6.egg
django_tinymce-1.5.1b4-py2.6.egg
Jinja2-2.6-py2.6.egg
lamson-1.1-py2.6.egg
longerusername-0.4-py2.6.egg
markdown2-2.1.0-py2.6.egg
nose-1.2.1-py2.6.egg
oauth2-1.5.211-py2.6.egg
pip-0.7.2-py2.6.egg
pystache-0.3.1-py2.6.egg
python_daemon-1.6-py2.6.egg
pytz-2012h-py2.6.egg
setuptools.pth
South-0.7.6-py2.6.egg
Unidecode-0.04.10-py2.6.egg

I had to manually add (inside my virtualenv)

pip install psycopg2 # requires apt-get install python-dev
pip install mock
pip install celery==2.2.7
pip install django-picklefield 
pip install httplib2
pip install lockfile

comment:32 Changed 16 months ago by T(A)ILS developers

Glad to see this moving forward. In the end why did you oriented your choice to Askbot rather than Shapado? We had a had time choosing ourselves.

And also, I understand that you want to start with an non-patched version. What is your short-term and long-term plan to get the current problems or desired features solved?

I'm referring mostly to:

  • Multilingual support. That would be the ability for a user to decide in which language to see the interface and maybe filter the content according to a personal list of languages.
  • Usability when JavaScript is disabled. I wrote down that both had problems to login without JavaScript. Would that be a problem with TBB?
  • Anonymous posting. We consider the shared password option as a hack that could have side effects: for example that account would get some karma, badges, etc. We wrote a basic patch to solve the anonymous posting on OSQA. It should be easy to port it to Askbot. Shall we send it to you?
  • Group feature. For Tails to share that tool with Tor, we want people to be able to use a single login for both Tor and Tails but the administration rights, karma, badges, and email notifications should be separate. It should be possible to move questions from one group to the other.

comment:33 Changed 16 months ago by T(A)ILS developers

Once you have this installed, would it be possible to have a "Tails" tag on your OSQA place? That would be a starting point for us to get rid of our old forum and see how do we do with this kind of tools. We also believe that this might also boost our collaboration on it for the future.

That would probably require having us granted some admin credentials. But that can be discussed once you have it installed.

comment:34 in reply to: ↑ 31 Changed 16 months ago by intrigeri

Replying to mo:

Not even the Django in Debian satisfies Askbot's requirement (wants >= 1.3.1, debian has 1.2.3).

For what it's worth, squeeze-backports has 1.4.2.

comment:35 Changed 15 months ago by phobos

See child #6926 for status on this ticket.

comment:36 Changed 13 months ago by phobos

  • Resolution set to fixed
  • Status changed from new to closed

done.

Note: See TracTickets for help on using tickets.