Merge pull request #3 from tootsuite/master

Updating to current

17 files changed, 865 insertions, 0 deletions

diff --git a/docs/Contributing-to-Mastodon/Sponsors.md b/docs/Contributing-to-Mastodon/Sponsors.md
new file mode 100644
index 000000000..5916ceb45
--- /dev/null
+++ b/docs/Contributing-to-Mastodon/Sponsors.md
@@ -0,0 +1,31 @@
+Sponsors
+========
+
+These people make the development of Mastodon possible through [Patreon](https://www.patreon.com/user?u=619786):
+
+**Extra special Patrons**
+
+- [World'sTallestLadder](https://mastodon.social/users/carcinoGeneticist)
+- [glocal](https://mastodon.social/users/glocal)
+- [Jimmy Tidey](https://mastodon.social/users/jimmytidey)
+- [Kurtis Rainbolt-Greene](https://mastodon.social/users/krainboltgreene)
+- [Kit Redgrave](https://socially.constructed.space/users/KitRedgrave)
+
+**Thank you to the following people**
+
+- [Sophia Park](https://mastodon.social/users/sophia)
+- [WelshPixie](https://mastodon.social/users/WelshPixie)
+- [John Parker](https://mastodon.social/users/Middaparka)
+- [Christina Hendricks](https://mastodon.social/users/clhendricksbc)
+- [Jelle](http://jelv.nl)
+- [Harris Bomberguy](https://mastodon.social/users/Hbomberguy)
+- [Martin Tithonium](https://mastodon.social/users/tithonium)
+- [Edward Saperia](https://nwspk.com)
+- [Yoz Grahame](http://yoz.com/)
+- [Jenn Kaplan](https://gay.crime.team/users/jkap)
+- [Natalie Weizenbaum](https://mastodon.social/users/nex3)
+- [Matteo De Micheli](http://matteodem.ch/)
+- [BirdMachine](https://mastodon.social/users/BirdMachine)
+- [Jessica Hayley](https://mastodon.social/users/jayhay)
+- [Niels Roesen Abildgaard](http://hypesystem.dk/)
+- [Zatnosk](https://github.com/Zatnosk)
diff --git a/docs/Contributing-to-Mastodon/Translating.md b/docs/Contributing-to-Mastodon/Translating.md
new file mode 100644
index 000000000..d47e83e7e
--- /dev/null
+++ b/docs/Contributing-to-Mastodon/Translating.md
@@ -0,0 +1,48 @@
+Translating
+===========
+
+If you want to localise Mastodon into your language, here is how.
+
+There are two parts to Mastodon, the server and the web client. The translations for the web client are in `app/assets/javascripts/components/locales`. For the server-side, the translations live in `config/locales` and are divided into different files. Here are all the files you’ll need to translate:
+
+| Original file (English) | Location | Description |
+|---|---|---|
+| [`en.jsx`](/Gargron/mastodon/tree/master/app/assets/javascripts/components/locales/en.jsx) | `app/assets/javascripts/components/locales/en.jsx` | Strings for the web client |
+| [`en.yml`](/Gargron/mastodon/tree/master/config/locales/en.yml) | `config/locales/en.yml` | Strings for general use |
+| [`simple_form.en.yml`](/Gargron/mastodon/tree/master/config/locales/simple_form.en.yml) | `config/locales/simple_form.en.yml` | Strings for the settings area |
+| [`devise.en.yml`](/Gargron/mastodon/tree/master/config/locales/devise.en.yml) | `config/locales/devise.en.yml` | Generic strings for Devise |
+| [`doorkeeper.en.yml`](/Gargron/mastodon/tree/master/config/locales/doorkeeper.en.yml) | `config/locales/doorkeeper.en.yml` | Generic strings for Doorkeeper |
+
+## Translating
+
+If you use Github, first clone the Mastodon repository to your account.
+
+1. Duplicate the files in their folder and replace `en` in the filenames by your language’s standard two-letters code ([ISO 639-1](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)).
+   For instance `simple_form.en.yml` becomes `simple_form.es.yml` in the Spanish translation.
+2. Also replace the language code in the first lines of all the files, and the last line of the `.jsx` file.
+3. Translate the right-side values from English to your language. Keep the indentation and punctuation.
+
+Since Devise and Doorkeeper are popular libraries, there may already be translation files for your language available on the Internet.
+
+## Declaring the language
+
+The locales are mentioned in several other files. To activate your translation, add your language code to the different lists present in these files:
+
+| File | Location | Comment |
+|---|---|---|
+| [`index.jsx`](/Gargron/mastodon/tree/master/app/assets/javascripts/components/locales/index.jsx) | `app/assets/javascripts/components/locales/index.jsx` | 2 lines to add |
+|[`mastodon.jsx`](/Gargron/mastodon/tree/master/app/assets/javascripts/components/containers/mastodon.jsx) | `app/assets/javascripts/components/containers/mastodon.jsx` | 1 line to add + 1 list to complete |
+| [`settings_helper.rb`](/Gargron/mastodon/tree/master/app/helpers/settings_helper.rb) | `app/helpers/settings_helper.rb` | 1 line to add + your language’s name |
+| [`application.rb`](/Gargron/mastodon/tree/master/config/application.rb) | `config/application.rb` | 1 list to complete |
+
+## Sending the translation
+
+You can then push the files to git and submit a pull request.
+
+## Testing the translation
+
+Once the pull request is accepted, wait for the code to be deployed on a Mastodon instance. Log-in with your account there, and change the locale in the settings. Browse and use the website. See if everything makes sense in context and if anything seems out of place or breaks the layout. Invite other Mastodon users speaking your language to try it and give feedback. Make changes accordingly and update the translation.
+
+## Updating the translation
+
+Keep an eye on the original English files in `app/assets/javascripts/components/locales` and `config/locales`. When they are updated, pass on the changes to your language files. For new strings, add the new lines to the same position and translate them. Once you’re finished with the updates, you can submit a new pull request.
diff --git a/docs/Extensions.md b/docs/Extensions.md
new file mode 100644
index 000000000..a3d64ebf1
--- /dev/null
+++ b/docs/Extensions.md
@@ -0,0 +1,15 @@
+Protocol extensions
+===================
+
+Some functionality in Mastodon required some additions to the protocols to enable seamless federation of those features:
+
+1. ActivityStreams was lacking verbs for block/unblock. Mastodon creates Salmon slaps for block and unblock events, which are not part of a user's public feed, but are nevertheless delivered to the target user. The intent of these Salmon slaps is not to notify the target user, but to notify the target user's server, so that it can perform any number of UX-related tasks such as removing the target user as a follower of the blocker, and/or displaying a message to the target user such as "You can't follow this person because you've been blocked"
+
+  The Salmon slaps have the exact same structure as standard follow/unfollow slaps, the verbs are namespaced:
+
+  - `http://mastodon.social/schema/1.0/block`
+  - `http://mastodon.social/schema/1.0/unblock`
+
+2. Statuses can be marked as containing sensitive (or not safe for work) media. This is symbolized by a `<category term="nsfw" />` on the Atom entry
+
+3. Statuses that are intended to be listed publicly on e.g. "whole known network" or "public" timelines contain a `<link rel="mentioned" href="http://activityschema.org/collection/public" ostatus:object-type="http://activitystrea.ms/schema/1.0/collection"/>`. Conversely, statuses which do not contain that, are intended to be low key, unlisted
diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 000000000..000593761
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,24 @@
+Index
+=====
+
+**Mastodon** is a free, open-source GNU social-compatible social network server. A decentralized alternative to commercial platforms, it avoids the risks of a single company monopolizing your communication. Anyone can run Mastodon and participate in the social network seamlessly.
+
+### Using Mastodon
+- [Frequently Asked Questions](Using-Mastodon/FAQ.md)
+- [List of Mastodon instances](Using-Mastodon/List-of-Mastodon-instances.md)
+- [Apps](Using-Mastodon/Apps.md)
+
+### Using the API
+- [API documentation](Using-the-API/API.md)
+- [Testing the API with cURL](Using-the-API/Testing-with-cURL.md)
+- [OAuth details](Using-the-API/OAuth-details.md)
+- [Tips for app developers](Using-the-API/Tips-for-app-developers.md)
+
+### Running Mastodon
+- [Production guide](Running-Mastodon/Production-guide.md)
+- [Development guide](Running-Mastodon/Development-guide.md)
+
+### Contributing to Mastodon
+- [Sponsors](Contributing-to-Mastodon/Sponsors.md)
+- [Translate Mastodon in your language](Contributing-to-Mastodon/Translating.md)
+- [Report bugs and submit ideas](https://github.com/tootsuite/mastodon/issues)
diff --git a/docs/Running-Mastodon/Administration-guide.md b/docs/Running-Mastodon/Administration-guide.md
new file mode 100644
index 000000000..1b9dc8630
--- /dev/null
+++ b/docs/Running-Mastodon/Administration-guide.md
@@ -0,0 +1,28 @@
+Administration guide
+=================
+
+So, you have a working Mastodon instance... now what?
+
+## Administration web interface
+
+A user that is designated as `admin = TRUE` in the database is able to access a suite of administration tools:
+
+* View, edit, silence, or suspend users - https://yourmastodon.instance/admin/accounts
+* View PubSubHubbub subscriptions - https://yourmastodon.instance/admin/pubsubhubbub
+* View domain blocks - https://yourmastodon.instance/admin/domain_blocks
+* Sidekiq dashboard - https://yourmastodon.instance/sidekiq
+* PGHero dashboard for PostgreSQL - https://yourmastodon.instance/pghero
+* Edit site settings - https://yourmastodon.instance/admin/settings
+
+## Site settings
+
+Your site settings are stored in the `settings` database table, and editable through the admin interface at https://yourmastodon.instance/admin/settings.
+
+You are able to set the following settings:
+
+- Contact username
+- Contact email
+- Site description
+- Site extended description
+
+You may wish to use the extended description (shown at https://yourmastodon.instance/about/more ) to display content guidelines or a user agreement (see https://mastodon.social/about/more for an example).
\ No newline at end of file
diff --git a/docs/Running-Mastodon/Development-guide.md b/docs/Running-Mastodon/Development-guide.md
new file mode 100644
index 000000000..80e6e2f94
--- /dev/null
+++ b/docs/Running-Mastodon/Development-guide.md
@@ -0,0 +1,48 @@
+Development guide
+=================
+
+**Don't use Docker to do development**. It's a quick way to get Mastodon running in production, it's **really really inconvenient for development**. Normally in Rails development environment you get hot reloading of backend code and on-the-fly compilation of assets like JS and CSS, but you lose those benefits by compiling a Docker image. If you want to contribute to Mastodon, it is worth it to simply set up a proper development environment.
+
+In fact, all you need is described in the [production guide](Production-guide.md), **with the following exceptions**. You **don't** need:
+
+- Nginx
+- SystemD
+- An `.env.production` file. If you need to set any environment variables, you can use an `.env` file
+- To prefix any commands with `RAILS_ENV=production` since the default environment is "development" anyway
+- Any cronjobs
+
+The command to install project dependencies does not require any flags, i.e. simply
+
+    bundle install
+
+By default the development environment wants to connect to a `mastodon_development` database on localhost using your user/ident to login to Postgres (i.e. not a md5 password)
+
+You can run Mastodon with:
+
+    rails s
+
+And open `http://localhost:3000` in your browser. Background jobs run inline (aka synchronously) in the development environment, so you don't need to run a Sidekiq process.
+
+You can run tests with:
+
+    rspec
+
+You can check localization status with:
+
+    i18n-tasks health
+
+You can check code quality with:
+
+    rubocop
+
+## Development tips
+
+You can use a localhost->world tunneling service like ngrok if you want to test federation, **however** that should not be your primary mode of operation. If you want to have a permanently federating server, set up a proper instance on a VPS with a domain name, and simply keep it up to date with your own fork of the project while doing development on localhost.
+
+Ngrok and similar services give you a random domain on each start up. This is good enough to test how the code you're working on handles real-world situations. But as soon as your domain changes, for everybody else concerned you're a different instance than before.
+
+Generally, federation bits are tricky to work on for exactly this reason - it's hard to test. And when you are testing with a disposable instance you are polluting the databases of the real servers you're testing against, usually not a big deal but can be annoying. The way I have handled this so far was thus: I have used ngrok for one session, and recorded the exchanges from its web interface to create fixtures and test suites. From then on I've been working with those rather than live servers.
+
+I advise to study the existing code and the RFCs before trying to implement any federation-related changes. It's not *that* difficult, but I think "here be dragons" applies because it's easy to break.
+
+If your development environment is running remotely (e.g. on a VPS or virtual machine), setting the `REMOTE_DEV` environment variable will swap your instance from using "letter opener" (which launches a local browser) to "letter opener web" (which collects emails and displays them at /letter_opener ).
\ No newline at end of file
diff --git a/docs/Running-Mastodon/Heroku-guide.md b/docs/Running-Mastodon/Heroku-guide.md
new file mode 100644
index 000000000..6aa8be774
--- /dev/null
+++ b/docs/Running-Mastodon/Heroku-guide.md
@@ -0,0 +1,13 @@
+Heroku guide
+============
+
+[![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)
+
+Mastodon can theoretically run indefinitely on a free [Heroku](https://heroku.com) app. It should be noted this has limited testing and could have unpredictable results.
+
+1. Click the above button.
+2. Fill in the options requested.
+  * You can use a .herokuapp.com domain, which will be simple to set up, or you can use a custom domain. If you want a custom domain and HTTPS, you will need to upgrade to a paid plan (to use Heroku's SSL features), or set up [CloudFlare](https://cloudflare.com) who offer free "Flexible SSL" (note: CloudFlare have some undefined limits on WebSockets. So far, no one has reported hitting concurrent connection limits).
+  * You will want Amazon S3 for file storage. The only exception is for development purposes, where you may not care if files are not saaved. Follow a guide online for creating a free Amazon S3 bucket and Access Key, then enter the details.
+  * If you want your Mastodon to be able to send emails, configure SMTP settings here (or later). Consider using [Mailgun](https://mailgun.com) or similar, who offer free plans that should suit your interests.
+3. Deploy! The app should be set up, with a working web interface and database. You can change settings and manage versions from the Heroku dashboard.
\ No newline at end of file
diff --git a/docs/Running-Mastodon/Production-guide.md b/docs/Running-Mastodon/Production-guide.md
new file mode 100644
index 000000000..76964d995
--- /dev/null
+++ b/docs/Running-Mastodon/Production-guide.md
@@ -0,0 +1,188 @@
+Production guide
+================
+
+## Nginx
+
+Regardless of whether you go with the Docker approach or not, here is an example Nginx server configuration:
+
+```nginx
+map $http_upgrade $connection_upgrade {
+  default upgrade;
+  ''      close;
+}
+
+server {
+  listen 443 ssl;
+  server_name example.com;
+
+  ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
+  ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
+
+  keepalive_timeout    70;
+  sendfile             on;
+  client_max_body_size 0;
+  gzip off;
+
+  root /home/mastodon/live/public;
+
+  add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
+
+  location / {
+    try_files $uri @proxy;
+  }
+
+  location @proxy {
+    proxy_set_header Host $host;
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_set_header X-Forwarded-Proto https;
+
+    proxy_pass_header Server;
+
+    proxy_pass http://localhost:3000;
+    proxy_buffering off;
+    proxy_redirect off;
+    proxy_http_version 1.1;
+    proxy_set_header Upgrade $http_upgrade;
+    proxy_set_header Connection $connection_upgrade;
+
+    tcp_nodelay on;
+  }
+
+  error_page 500 501 502 503 504 /500.html;
+}
+```
+
+## Running in production without Docker
+
+It is recommended to create a special user for mastodon on the server (you could call the user `mastodon`), though remember to disable outside login for it. You should only be able to get into that user through `sudo su - mastodon`.
+
+## General dependencies
+
+    curl -sL https://deb.nodesource.com/setup_4.x | sudo bash -
+    sudo apt-get install imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev nodejs
+    sudo npm install -g yarn
+
+## Redis
+
+    sudo apt-get install redis-server redis-tools
+
+## Postgres
+
+    sudo apt-get install postgresql postgresql-contrib
+
+## Rbenv
+
+It is recommended to use rbenv (exclusively from the `mastodon` user) to install the desired Ruby version. Follow the guides to [install rbenv][1] and [rbenv-build][2] (I recommend checking the [prerequisites][3] for your system on the rbenv-build project and installing them beforehand, obviously outside the unprivileged `mastodon` user)
+
+[1]: https://github.com/rbenv/rbenv#installation
+[2]: https://github.com/rbenv/ruby-build#installation
+[3]: https://github.com/rbenv/ruby-build/wiki#suggested-build-environment
+
+Then once `rbenv` is ready, run `rbenv install 2.3.1` to install the Ruby version for Mastodon.
+
+## Git
+
+You need the `git-core` package installed on your system. If it is so, from the `mastodon` user:
+
+    cd ~
+    git clone https://github.com/Gargron/mastodon.git live
+    cd live
+
+Then you can proceed to install project dependencies:
+
+    gem install bundler
+    bundle install --deployment --without development test
+    yarn install
+
+## Configuration
+
+Then you have to configure your instance:
+
+    cp .env.production.sample .env.production
+    nano .env.production
+
+Fill in the important data, like host/port of the redis database, host/port/username/password of the postgres database, your domain name, SMTP details (e.g. from Mailgun or equivalent transactional e-mail service, many have free tiers), whether you intend to use SSL, etc. If you need to generate secrets, you can use:
+
+    rake secret
+
+To get a random string.
+
+## Setup
+
+And setup the database for the first time, this will create the tables and basic data:
+
+    RAILS_ENV=production bundle exec rails db:setup
+
+Finally, pre-compile all CSS and JavaScript files:
+
+    RAILS_ENV=production bundle exec rails assets:precompile
+
+## Systemd
+
+Example systemd configuration for the web workers, to be placed in `/etc/systemd/system/mastodon-web.service`:
+
+```systemd
+[Unit]
+Description=mastodon-web
+After=network.target
+
+[Service]
+Type=simple
+User=mastodon
+WorkingDirectory=/home/mastodon/live
+Environment="RAILS_ENV=production"
+Environment="PORT=3000"
+ExecStart=/home/mastodon/.rbenv/shims/bundle exec puma -C config/puma.rb
+TimeoutSec=15
+Restart=always
+
+[Install]
+WantedBy=multi-user.target
+```
+
+Example systemd configuration for the background workers, to be placed in `/etc/systemd/system/mastodon-sidekiq.service`:
+
+```systemd
+[Unit]
+Description=mastodon-sidekiq
+After=network.target
+
+[Service]
+Type=simple
+User=mastodon
+WorkingDirectory=/home/mastodon/live
+Environment="RAILS_ENV=production"
+Environment="DB_POOL=5"
+ExecStart=/home/mastodon/.rbenv/shims/bundle exec sidekiq -c 5 -q default -q mailers -q push
+TimeoutSec=15
+Restart=always
+
+[Install]
+WantedBy=multi-user.target
+```
+
+This allows you to `sudo systemctl enable mastodon-*.service` and `sudo systemctl start mastodon-*.service` to get things going.
+
+## Cronjobs
+
+I recommend creating a couple cronjobs for the following tasks:
+
+- `RAILS_ENV=production bundle exec rake mastodon:media:clear`
+- `RAILS_ENV=production bundle exec rake mastodon:push:refresh`
+- `RAILS_ENV=production bundle exec rake mastodon:feeds:clear`
+
+You may want to run `which bundle` first and copypaste that full path instead of simply `bundle` in the above commands because cronjobs usually don't have all the paths set. The time and intervals of when to run these jobs are up to you, but once every day should be enough for all.
+
+You can edit the cronjob file for the `mastodon` user by running `sudo crontab -e mastodon` (outside of the mastodon user).
+
+## Things to look out for when upgrading Mastodon
+
+You can upgrade Mastodon with a `git pull` from the repository directory. You may need to run:
+
+- `RAILS_ENV=production bundle exec rails db:migrate`
+- `RAILS_ENV=production bundle exec rails assets:precompile`
+
+Depending on which files changed, e.g. if anything in the `/db/` or `/app/assets` directory changed, respectively. Also, Mastodon runs in memory, so you need to restart it before you see any changes. If you're using systemd, that would be:
+
+    sudo systemctl restart mastodon-*.service
diff --git a/docs/Running-Mastodon/Vagrant-guide.md b/docs/Running-Mastodon/Vagrant-guide.md
new file mode 100644
index 000000000..a94478392
--- /dev/null
+++ b/docs/Running-Mastodon/Vagrant-guide.md
@@ -0,0 +1,64 @@
+Vagrant guide
+=============
+
+A quick way to get a development environment up and running is with Vagrant. You will need recent versions of [Vagrant](https://www.vagrantup.com/) and [VirtualBox](https://www.virtualbox.org/) installed.
+
+## Basic setup
+
+Install the latest versions of Vagrant and VirtualBox for your operating systems, and then run:
+
+    vagrant plugin install vagrant-hostsupdater
+
+This is optional, but will update your 'hosts' file when you start the virtual machine, allowing you to access the site at http://mastodon.dev (instead of http://localhost:3000).
+
+To create and provision a new virtual machine for Mastodon development:
+
+    git clone git@github.com:tootsuite/mastodon.git
+    cd mastodon
+    vagrant up
+
+Running `vagrant up` for the first time will run provisioning, which will:
+
+- Download the Ubuntu 14.04 base image, if there isn't already a copy on your machine
+- Create a new VirtualBox virtual machine from that image
+- Run the provisioning script (located inside the Vagrantfile), which installs the system packages, Ruby gems, and JS modules required for Mastodon
+- Run the startup script
+
+## Starting the server
+
+The Vagrant box will automatically start after provisioning. It can be started in future with `vagrant up` from the mastodon directory.
+
+Once the Ubuntu virtual machine has booted, it will run the startup script, which loads the environment variables from `.env.vagrant` and then runs `rails s -d -b 0.0.0.0`. This will start a Rails server. You can then access your development site at http://mastodon.dev (or at http://localhost:3000 if you haven't installed vagrants-hostupdater).
+
+To stop the server, simply run `vagrant halt`.
+
+## Using the server
+
+You should now have a working Mastodon instance, although it will not federate, as it is not publicly accessible. Should you need temporary federation for development and testing, see the Ngrok information in the [Development Guide](Development-guide.md).
+
+By default, your instance's ActionMailer will use "Letter Opener Web" for email. This means that any email that would normally be sent, will instead be stored, and accessible at http://mastodon.dev/letter_opener - you can use this to verify a registered user account.
+
+## Making changes/developing
+
+You are able to set environment variables, which are used for Mastodon configuration, by editing the `.env.vagrant` file. Any changes you make will take effect after a Vagrant restart.
+
+Vagrant has mounted your mastodon folder inside the virtual machine. This means that any change to the files in the folder(e.g. the Rails controllers or the React components in /app) should immediately take effect on the live server. This allows you to make and test changes, and create new commits, without ever needing to access the virtual machine.
+
+Should you need to access the virtual machine (for example, to manually restart the Rails process without restarting the box), run `vagrant ssh` from the mastodon folder. You will now be logged in as the `vagrant` user on the VirtualBox Ubuntu VM. You will want to `cd /vagrant` to see the app folder.
+
+## Debugging
+
+You can find the Rails server logs in in the `log` folder, which will often have the information you need.
+
+If your Mastodon instance or Vagrant box are really not behaving, you can re-run the provisioning process. Stop the box with `vagrant halt`, and then run `vagrant destroy` - this will delete the virtual machine. You may then run `vagrant up` to create a new box, and re-run provisioning.
+
+## Testing
+
+To run the `rspec` tests and `rubocop` style checker, you may either:
+
+* Install the relevant gems locally, or
+* SSH into the virtual machine, `cd /vagrant`, and then run the commands
+
+## Support/help
+
+If you are confused, or having any issues with the above, the Mastodon IRC channel ( irc.freenode.net #mastodon ) is a good place to find assistance.
\ No newline at end of file
diff --git a/docs/Specs-and-RFCs-used.md b/docs/Specs-and-RFCs-used.md
new file mode 100644
index 000000000..9bb1bb622
--- /dev/null
+++ b/docs/Specs-and-RFCs-used.md
@@ -0,0 +1,12 @@
+Specs and RFCs used
+===================
+
+* [OStatus](https://www.w3.org/community/ostatus/wiki/images/9/93/OStatus_1.0_Draft_2.pdf)
+* [Salmon](http://www.salmon-protocol.org/salmon-protocol-summary)
+* [Portable Contacts](https://web.archive.org/web/20160305010620/http://portablecontacts.net/draft-spec.html)
+* [Atom](https://tools.ietf.org/html/rfc4287)
+* [Atom ActivityStreams](http://activitystrea.ms/specs/atom/1.0/)
+* [Atom Threading](https://tools.ietf.org/html/rfc4685)
+* [PubSubHubbub](https://pubsubhubbub.github.io/PubSubHubbub/pubsubhubbub-core-0.4.html)
+* [Webfinger](https://tools.ietf.org/html/rfc7033)
+* [Link-based Resource Descriptor Discovery](https://tools.ietf.org/html/rfc6415)
diff --git a/docs/Using-Mastodon/Apps.md b/docs/Using-Mastodon/Apps.md
new file mode 100644
index 000000000..e0a2730c1
--- /dev/null
+++ b/docs/Using-Mastodon/Apps.md
@@ -0,0 +1,15 @@
+List of apps
+============
+
+Some people have started working on apps for the Mastodon API. Here is a list of them:
+
+|App|Platform|Link|Developer(s)|
+|---|--------|----|------------|
+|Matodor|iOS/Android|<https://github.com/jeroensmeets/mastodon-app>|[@jeroensmeets@mastodon.social](https://mastodon.social/users/jeroensmeets)|
+|Tusky|Android|<https://github.com/Vavassor/Tusky>|[@Vavassor@mastodon.social](https://mastodon.social/users/Vavassor)|
+|Albatross|iOS||[@goldie_ice@mastodon.social](https://mastodon.social/users/goldie_ice)|
+|tootstream|command-line|<https://github.com/magicalraccoon/tootstream>|[@Raccoon@mastodon.social](https://mastodon.social/users/Raccoon)|
+|mastodroid|Android|<https://github.com/alin-rautoiu/mastodroid>||
+|Tooter|Chrome extension|<https://github.com/ineffyble/tooter>|[@effy@mastodon.social](https://mastodon.social/users/effy)|
+
+If you have a project like this, let me know so I can add it to the list!
diff --git a/docs/Using-Mastodon/FAQ.md b/docs/Using-Mastodon/FAQ.md
new file mode 100644
index 000000000..daedcbdd8
--- /dev/null
+++ b/docs/Using-Mastodon/FAQ.md
@@ -0,0 +1,43 @@
+Frequently Asked Questions
+==========================
+
+#### What is a Mastodon?
+
+A prehistoric animal, predecessor of the mammoth.
+
+#### Why the name Mastodon?
+
+There's a progressive metal band with the same name that I'm a fan of that brought the animal to my attention. I thought it's a pretty cool name/animal.
+
+#### How exactly is it decentralized?
+
+There are different ways in which something can be decentralized; in this case, Mastodon is the "federated" kind. Think e-mail, not BitTorrent. There are different servers (instances), users have an account on one of them, but can interact and follow each other regardless of where their account is.
+
+#### Technically, how does the federation work?
+
+We are using the OStatus suite of protocols:
+
+1. Webfinger for user-on-domain lookup
+2. Atom feeds with ActivityStreams, Portable Contacts, Threads extensions for the actual content
+3. PubSubHubbub for subscribing to Atom feeds
+4. Salmon for delivering certain items from the Atom feeds to interested parties such as the mentioned user, author of the status being replied to, person being followed, etc
+
+#### What is mastodon.social?
+
+The "flagship" instance of Mastodon, aka the server I run myself with the latest code. It's not supposed to be the only instance in the end.
+
+#### What else is part of the federated network?
+
+Let's call it the "fediverse". It has existed for a longer while, populated by GNU social servers, Friendica, Hubzilla, Diaspora etc. Not every one of those servers is fully compatible with every other. Mastodon strives to be fully standards-compliant and compatibility with GNU social is higher in priority than the others.
+
+#### I tried logging into a GNU social client app with Mastodon and it didn't work, why?
+
+While Mastodon is compatible with GNU social in terms of server to server communication, the client to server API (aka how you access Mastodon) is different. Therefore, client apps that were made for specifically GNU social will not work with Mastodon. The reason for this is half technical, half ideological.
+
+Because Mastodon has been created from a blank slate, it is much simpler to have the API mirror internal structures as closely as possible, rather than build an emulation layer. Secondly, the GNU social client API is actually a half-way implementation of the legacy Twitter API - that's the reason why it works with some older Twitter client apps. However, many of those apps are not maintained anymore, the GNU social API does not actually keep up with the real Twitter API and never fully implemented all its features; at the same time, the Twitter API was never meant for a federated service and so obscures some of the functionality.
+
+#### How is Mastodon funded?
+
+Development of Mastodon and hosting of mastodon.social is funded through my [Patreon (also BTC/PayPal donations)](https://www.patreon.com/user?u=619786). Beyond that, I am not interested in VC funding, monetizing, advertising, or anything of that sort. I could offer setup/maintenance services on demand.
+
+The software is free and open source and communities should host their own servers if they can, that way the costs are more or less distributed. Obviously it'd be hard for me to pay the bills if literally everyone decided to use the mastodon.social instance only.
\ No newline at end of file
diff --git a/docs/Using-Mastodon/List-of-Mastodon-instances.md b/docs/Using-Mastodon/List-of-Mastodon-instances.md
new file mode 100644
index 000000000..2f15df083
--- /dev/null
+++ b/docs/Using-Mastodon/List-of-Mastodon-instances.md
@@ -0,0 +1,12 @@
+List of Mastodon instances
+==========================
+
+* [mastodon.social](https://mastodon.social)
+* [social.tchncs.de](https://social.tchncs.de)
+* [on.vu](https://on.vu)
+* [animalliberation.social](https://animalliberation.social)
+* [socially.constructed.space](https://socially.constructed.space)
+* [epiktistes.com](https://epiktistes.com)
+* [toot.zone](https://toot.zone)
+
+Let me know if you start running one so I can add it to the list!
diff --git a/docs/Using-the-API/API.md b/docs/Using-the-API/API.md
new file mode 100644
index 000000000..9f5280870
--- /dev/null
+++ b/docs/Using-the-API/API.md
@@ -0,0 +1,280 @@
+API overview
+============
+
+## Contents
+
+- [Available libraries](#available-libraries)
+- [Notes](#notes)
+- [Methods](#methods)
+  - Posting a status
+  - Uploading media
+  - Retrieving a timeline
+  - Retrieving notifications
+  - Following a remote user
+  - Fetching data
+  - Deleting a status
+  - Reblogging a status
+  - Favouriting a status
+  - Threads (status context)
+  - Who reblogged/favourited a status
+  - Following/unfollowing accounts
+  - Blocking/unblocking accounts
+  - Creating OAuth apps
+- [Entities](#entities)
+  - Status
+  - Account
+- [Pagination](#pagination)
+
+## Available libraries
+
+- [For Ruby](https://github.com/tootsuite/mastodon-api)
+- [For Python](https://github.com/halcy/Mastodon.py)
+- [For JavaScript](https://github.com/Zatnosk/libodonjs)
+- [For JavaScript (Node.js)](https://github.com/jessicahayley/node-mastodon)
+
+## Notes
+
+When an array parameter is mentioned, the Rails convention of specifying array parameters in query strings is meant. For example, a ruby array like `foo = [1, 2, 3]` can be encoded in the params as `foo[]=1&foo[]=2&foo[]=3`. Square brackets can be indexed but can also be empty.
+
+When a file parameter is mentioned, a form-encoded upload is expected.
+
+## Methods
+### Posting a new status
+
+**POST /api/v1/statuses**
+
+Form data:
+
+- `status`: The text of the status
+- `in_reply_to_id` (optional): local ID of the status you want to reply to
+- `media_ids` (optional): array of media IDs to attach to the status (maximum 4)
+- `sensitive` (optional): set this to mark the media of the status as NSFW
+- `visibility` (optional): either `private`, `unlisted` or `public`
+
+Returns the new status.
+
+**POST /api/v1/media**
+
+Form data:
+
+- `file`: Image to be uploaded
+
+Returns a media object with an ID that can be attached when creating a status (see above).
+
+### Retrieving a timeline
+
+**GET /api/v1/timelines/home**
+**GET /api/v1/timelines/mentions**
+**GET /api/v1/timelines/public**
+**GET /api/v1/timelines/tag/:hashtag**
+
+Returns statuses, most recent ones first. Home timeline is statuses from people you follow, mentions timeline is all statuses that mention you. Public timeline is "whole known network", and the last is the hashtag timeline.
+
+Query parameters:
+
+- `max_id` (optional): Skip statuses younger than ID (e.g. navigate backwards in time)
+- `since_id` (optional): Skip statuses older than ID (e.g. check for updates)
+
+### Notifications
+
+**GET /api/v1/notifications**
+
+Returns notifications for the authenticated user. Each notification has an `id`, a `type` (mention, reblog, favourite, follow), an `account` which it came *from*, and in case of mention, reblog and favourite also a `status`.
+
+### Following a remote user
+
+**POST /api/v1/follows**
+
+Form data:
+
+- uri: username@domain of the person you want to follow
+
+Returns the local representation of the followed account.
+
+### Fetching data
+
+**GET /api/v1/statuses/:id**
+
+Returns status.
+
+**GET /api/v1/accounts/:id**
+
+Returns account.
+
+**GET /api/v1/accounts/verify_credentials**
+
+Returns authenticated user's account.
+
+**GET /api/v1/accounts/:id/statuses**
+
+Returns statuses by user. Same options as timeline are permitted.
+
+**GET /api/v1/accounts/:id/following**
+
+Returns users the given user is following.
+
+**GET /api/v1/accounts/:id/followers**
+
+Returns users the given user is followed by.
+
+**GET /api/v1/accounts/relationships**
+
+Returns relationships (`following`, `followed_by`, `blocking`) of the current user to a list of given accounts.
+
+Query parameters:
+
+- `id` (can be array): Account IDs
+
+**GET /api/v1/accounts/search**
+
+Returns matching accounts. Will lookup an account remotely if the search term is in the username@domain format and not yet in the database.
+
+Query parameters:
+
+- `q`: what to search for
+- `limit`: maximum number of matching accounts to return
+
+**GET /api/v1/blocks**
+
+Returns accounts blocked by authenticated user.
+
+**GET /api/v1/favourites**
+
+Returns statuses favourited by authenticated user.
+
+### Deleting a status
+
+**DELETE /api/v1/statuses/:id**
+
+Returns an empty object.
+
+### Reblogging a status
+
+**POST /api/v1/statuses/:id/reblog**
+
+Returns a new status that wraps around the reblogged one.
+
+### Unreblogging a status
+
+**POST /api/v1/statuses/:id/unreblog**
+
+Returns the status that used to be reblogged.
+
+### Favouriting a status
+
+**POST /api/v1/statuses/:id/favourite**
+
+Returns the target status.
+
+### Unfavouriting a status
+
+**POST /api/v1/statuses/:id/unfavourite**
+
+Returns the target status.
+
+### Threads
+
+**GET /api/v1/statuses/:id/context**
+
+Returns `ancestors` and `descendants` of the status.
+
+### Who reblogged/favourited a status
+
+**GET /api/v1/statuses/:id/reblogged_by**
+**GET /api/v1/statuses/:id/favourited_by**
+
+Returns list of accounts.
+
+### Following and unfollowing users
+
+**POST /api/v1/accounts/:id/follow**
+**POST /api/v1/accounts/:id/unfollow**
+
+Returns the updated relationship to the user.
+
+### Blocking and unblocking users
+
+**POST /api/v1/accounts/:id/block**
+**POST /api/v1/accounts/:id/unblock**
+
+Returns the updated relationship to the user.
+
+### OAuth apps
+
+**POST /api/v1/apps**
+
+Form data:
+
+- `client_name`: Name of your application
+- `redirect_uris`: Where the user should be redirected after authorization (for no redirect, use `urn:ietf:wg:oauth:2.0:oob`)
+- `scopes`: This can be a space-separated list of the following items: "read", "write" and "follow" (see [this page](OAuth-details.md) for details on what the scopes do)
+- `website`: (optional) URL to the homepage of your app
+
+Creates a new OAuth app. Returns `id`, `client_id` and `client_secret` which can be used with [OAuth authentication in your 3rd party app](Testing-with-cURL.md).
+
+___
+
+## Entities
+
+### Status
+
+| Attribute           | Description |
+|---------------------|-------------|
+| `id`                ||
+| `uri`               | fediverse-unique resource ID |
+| `url`               | URL to the status page (can be remote) |
+| `account`           | Account |
+| `in_reply_to_id`    | null or ID of status it replies to |
+| `reblog`            | null or Status|
+| `content`           | Body of the status. This will contain HTML (remote HTML already sanitized) |
+| `created_at`        ||
+| `reblogs_count`     ||
+| `favourites_count`  ||
+| `reblogged`         | Boolean for authenticated user |
+| `favourited`        | Boolean for authenticated user |
+| `media_attachments` | array of MediaAttachments |
+| `mentions`          | array of Mentions |
+| `application`       | Application from which the status was posted |
+
+Media Attachment:
+
+| Attribute           | Description |
+|---------------------|-------------|
+| `url`               | URL of the original image (can be remote) |
+| `preview_url`       | URL of the preview image |
+| `type`              | Image or video |
+
+Mention:
+
+| Attribute           | Description |
+|---------------------|-------------|
+| `url`               | URL of user's profile (can be remote) |
+| `acct`              | Username for local or username@domain for remote users |
+| `id`                | Account ID |
+
+Application:
+
+| Attribute           | Description |
+|---------------------|-------------|
+| `name`              | Name of the app |
+| `website`           | Homepage URL of the app |
+
+### Account
+
+| Attribute         | Description |
+|-------------------|-------------|
+| `id`              ||
+| `username`        ||
+| `acct`            | Equals username for local users, includes @domain for remote ones |
+| `display_name`    ||
+| `note`            | Biography of user |
+| `url`             | URL of the user's profile page (can be remote) |
+| `avatar`          | URL to the avatar image |
+| `header`          | URL to the header image |
+| `followers_count` ||
+| `following_count` ||
+| `statuses_count`  ||
+
+## Pagination
+
+API methods that return collections of items can return a `Link` header containing URLs for the `next` and `prev` pages. [Link header RFC](https://tools.ietf.org/html/rfc5988)
diff --git a/docs/Using-the-API/OAuth-details.md b/docs/Using-the-API/OAuth-details.md
new file mode 100644
index 000000000..d0b5abd40
--- /dev/null
+++ b/docs/Using-the-API/OAuth-details.md
@@ -0,0 +1,12 @@
+OAuth details
+=============
+
+We use the [Doorkeeper gem for OAuth](https://github.com/doorkeeper-gem/doorkeeper/wiki), so you can refer to their docs on specifics of the end-points.
+
+The API is divided up into access scopes:
+
+- `read`: Read data
+- `write`: Post statuses and upload media for statuses
+- `follow`: Follow, unfollow, block, unblock
+
+Multiple scopes can be requested during the authorization phase with the `scope` query param (space-separate the scopes).
diff --git a/docs/Using-the-API/Testing-with-cURL.md b/docs/Using-the-API/Testing-with-cURL.md
new file mode 100644
index 000000000..977773a08
--- /dev/null
+++ b/docs/Using-the-API/Testing-with-cURL.md
@@ -0,0 +1,16 @@
+Testing the API with cURL
+=========================
+
+Mastodon builds around the idea of being a server first, rather than a client itself. Similarly to how a XMPP chat server communicates with others and with its own clients, Mastodon takes care of federation to other networks, like other Mastodon or GNU Social instances. So Mastodon provides a REST API, and a 3rd-party app system for using it via OAuth2.
+
+You can get a client ID and client secret required for OAuth [via an API end-point](API.md#oauth-apps).
+
+From these two, you will need to acquire an access token. It is possible to do using your account's e-mail and password like this:
+
+    curl -X POST -d "client_id=CLIENT_ID_HERE&client_secret=CLIENT_SECRET_HERE&grant_type=password&username=YOUR_EMAIL&password=YOUR_PASSWORD" -Ss https://mastodon.social/oauth/token
+
+The response will be a JSON object containing the key `access_token`. Use that token in any API requests by setting a header like this:
+
+    curl --header "Authorization: Bearer ACCESS_TOKEN_HERE" -sS https://mastodon.social/api/statuses/home
+
+Please note that the password-based approach is not recommended especially if you're dealing with other user's accounts and not just your own. Usually you would use the authorization grant approach where you redirect the user to a web page on the original site where they can login and authorize the application and are then redirected back to your application with an access code.
diff --git a/docs/Using-the-API/Tips-for-app-developers.md b/docs/Using-the-API/Tips-for-app-developers.md
new file mode 100644
index 000000000..561f1e273
--- /dev/null
+++ b/docs/Using-the-API/Tips-for-app-developers.md
@@ -0,0 +1,16 @@
+Tips for app developers
+=======================
+
+## Authentication
+
+Make sure that you allow your users to specify the domain they want to connect to before login. Use that domain to acquire a client id/secret for OAuth2 and then proceed with normal OAuth2 also using that domain to build the URLs.
+
+In my opinion it is easier for people to understand what is being asked of them if you ask for a `username@domain` type input, since it looks like an e-mail address. Though the username part is not required for anything in the OAuth2 process. Once the user is logged in, you get information about the logged in user from `/api/v1/accounts/verify_credentials`
+
+## Usernames
+
+Make sure that you make it possible to see the `acct` of any user in your app (since it includes the domain part for remote users), people must be able to tell apart users from different domains with the same username.
+
+## Formatting
+
+The API delivers already formatted HTML to your app. This isn't ideal since not all apps are based on HTML, but this is not fixable as its part of the way OStatus federation works. Most importantly, you get some information on linked entities alongside the HTML of the status body. For example, you get a list of mentioned users, and a list of media attachments, and a list of hashtags. It is possible to convert the HTML to whatever you need in your app by parsing the HTML tags and matching their `href`s to the linked entities. If a match cannot be found, the link must stay a clickable link.