Opened 5 years ago

Closed 22 months ago

#10591 closed task (fixed)

Create a sitemap for current torproject.org website

Reported by: phobos Owned by: isabela
Priority: Medium Milestone: WebsiteV3
Component: Webpages/Website Version:
Severity: Normal Keywords: www-team easy defer-new-website
Cc: Actual Points:
Parent ID: #18243 Points:
Reviewer: Sponsor:

Description (last modified by isabela)

During the Tor Dev Meeting in Berlin 2015, we decided on new information structure for torproject.org website:
https://trac.torproject.org/projects/tor/wiki/Website/MainSiteRedesign

We also had some suggestions on what to do with the content that we were removing from the homepage.

The goal of this task is to map all the current content on torproject.org so we can make sure we are not forgetting any page and we have a plan for everything we are moving out of it.

Child Tickets

Attachments (3)

tor-project-ia-proposed-030110.pdf (112.4 KB) - added by phobos 5 years ago.
proposed IA from 2010 redesign
graph.dot (7.3 KB) - added by jagtalon 3 years ago.
DOT file of the sitemap
graph.pdf (40.1 KB) - added by jagtalon 3 years ago.
PDF file of the sitemap

Download all attachments as: .zip

Change History (18)

Changed 5 years ago by phobos

proposed IA from 2010 redesign

comment:1 Changed 5 years ago by lunar

Here is the list of every HTML files created by a build of the current website (SVN r26526):

index.html.en
getinvolved/relays.html.en
getinvolved/translation-overview.html.en
getinvolved/volunteer.html.en
getinvolved/open-positions.html.en
getinvolved/mirrors.html.en
getinvolved/tshirt.html.en
getinvolved/translation.html.en
docs/tor-manual-dev.html.en
docs/tor-relay-debian.html.en
docs/tor-doc-windows.html.en
docs/running-a-mirror.html.en
docs/documentation.html.en
docs/hidden-services.html.en
docs/manual.html.en
docs/bridges.html.en
docs/pluggable-transports.html.en
docs/tor-manual.html.en
docs/trademark-faq.html.en
docs/android.html.en
docs/tor-doc-relay.html.en
docs/debian.html.en
docs/installguide.html.en
docs/short-user-manual.html.en
docs/tor-doc-unix.html.en
docs/debian-vidalia.html.en
docs/tor-hidden-service.html.en
docs/faq-abuse.html.en
docs/signing-keys.html.en
docs/tor-doc-win32.html.en
docs/faq.html.en
docs/tor-doc-osx.html.en
docs/rpms.html.en
docs/verifying-signatures.html.en
docs/proxychain.html.en
docs/N900.html.en
projects/obfsproxy-debian-instructions.html.en
projects/arm.html.en
projects/onionoo.html.en
projects/torweather.html.en
projects/projects.html.en
projects/tordnsel.html.en
projects/torbrowser-details.html.en
projects/obfsproxy.html.en
projects/obfsproxy-instructions.html.en
projects/gettor.html.en
projects/sampleproject.html.en
projects/torbrowser.html.en
projects/vidalia.html.en
press/inthemedia.html.en
press/2008-12-19-roadmap-press-release.html.en
press/2011-08-28-tor-022-stable.html.en
press/2010-03-25-tor-store-press-release.html.en
press/2010-09-16-ten-things-circumvention-tools.html.en
press/press.html.en
press/2009-03-12-performance-roadmap-press-release.html.en
torbutton/torbutton-faq.html.en
torbutton/index.html.en
torbutton/torbutton-options.html.en
eff/tor-legal-faq.html.en
eff/tor-dmca-response.html.en
download/download-easy.html.en
download/download.html.en
download/download-unix.html.en
about/torusers.html.en
about/sponsors.html.en
about/board.html.en
about/jobs-extdev.html.en
about/contact.html.en
about/gsoc.html.en
about/jobs-browserhacker.html.en
about/jobs.html.en
about/contributors.html.en
about/corepeople.html.en
about/jobs-projectcoordinator.html.en
about/volunteers.html.en
about/overview.html.en
about/financials.html.en
about/jobs-pluggabletransport.html.en
about/translators.html.en
about/jobs-lead-automation.html.en
about/jobs-translatorsupport.html.en
about/jobs-coredev.html.en
donate/donate-hardware.html.en
donate/donate-service.html.en
donate/donate.html.en
donate/become-sponsor.html.en
donate/matching-program.html.en
donate/thankyou.html.en

The following two are HTML pages that are generated from DocBook:

torbutton/en/design/index.html.en
projects/torbrowser/design/index.html

comment:2 Changed 5 years ago by mttp

I used http://www.xml-sitemaps.com/ to automatically generate a sitemap, and I added the xml file to the www root directory. I also added the line

Sitemap: https://www.torproject.org/sitemap.xml

to robots.txt. What else needs to be done now?

Last edited 5 years ago by mttp (previous) (diff)

comment:3 Changed 5 years ago by phobos

We had one of these in the past. We learned the hardway if it's not 100% accurate (updated when pages are updated), the result ranking demotion in google/bing/etc are fairly draconian. How can we automatically keep this sitemap 100% accurate? Preferably, it's created at build time during "publish" of the website.

Last edited 5 years ago by phobos (previous) (diff)

comment:4 Changed 5 years ago by phobos

Please note this is also advertising a 3rd party service, we don't want to do that.

comment:5 Changed 5 years ago by phobos

Also, this has the various apache dir listings in it? Seems silly to include these. See these lines for example:

1463 <url>
1464   <loc>https://www.torproject.org/dist/misc/ttdnsd/?C=M;O=D</loc>
1465   <changefreq>daily</changefreq>
1466   <priority>0.21</priority>
1467 </url>

comment:6 Changed 5 years ago by phobos

The original point of this ticket is not to create a sitemap xml file for search engines, but to look at the information architecture of the site as a whole. We could use this IA to design a newer IA or rework the current IA for easier usability.

comment:7 in reply to:  6 Changed 5 years ago by mttp

Replying to phobos:

The original point of this ticket is not to create a sitemap xml file for search engines, but to look at the information architecture of the site as a whole. We could use this IA to design a newer IA or rework the current IA for easier usability.

I don't want to put the website at risk of being search engine punished. I'll just remove the xml file, as it appears I added it based on misunderstanding.

comment:8 Changed 4 years ago by Sebastian

Keywords: easy added

This should be an easy script to write which traverses the directories, looks for .html.en files, puts them into a data structure and then outputs them to either dot format or html or something.

comment:9 in reply to:  8 Changed 3 years ago by wilbns

Replying to Sebastian:

This should be an easy script to write which traverses the directories, looks for .html.en files, puts them into a data structure and then outputs them to either dot format or html or something.

Hello, could you (or please let me know if there's a better person to ask) give me an example of what the output should look like?

comment:10 Changed 3 years ago by Sebastian

Keywords: defer-new-website added
Severity: Normal

I'm thinking just graphviz DOT files (https://en.wikipedia.org/wiki/DOT_(graph_description_language\))

comment:11 Changed 3 years ago by isabela

Parent ID: #18243

comment:12 Changed 3 years ago by isabela

Owner: set to isabela
Status: newassigned

comment:13 Changed 3 years ago by isabela

Description: modified (diff)

comment:14 Changed 3 years ago by jagtalon

Hello there. I made the dot file of the sitemap and also a pdf of it. The dot file can be viewed in OmniGraffle or KGraphViewer (see attached files)

Changed 3 years ago by jagtalon

Attachment: graph.dot added

DOT file of the sitemap

Changed 3 years ago by jagtalon

Attachment: graph.pdf added

PDF file of the sitemap

comment:15 Changed 22 months ago by isabela

Resolution: fixed
Status: assignedclosed
Note: See TracTickets for help on using tickets.