Shexia

Congratulations! You’ve started creating a new tool – that’s awesome!

Guide to this tool

You’ll want to replace a lot of the code that this template generated for you, since it’s mainly for demonstration purposes. Here’s what the different parts demonstrate:

• This index page is almost completely static. The only interesting part is how the hyperlinks in this list are generated – check out the template source code (templates/index.html) for that.

• The greeting endpoint issues a simple greeting to an arbitrary person. It demonstrates simple URL routes (you can append any name after /greet/, and it will appear in the page), as well as simple templating (including the name in the page). Try setting the name to something hilarious like <script>alert('pwnd'); to see how the template automatically escapes user input. (The name can’t contain a slash, hence the missing </script>.)

• The praising endpoint serves several functions at once. Its basic function is to heap praise upon the visitor.

  The default praise is a very generic phrase, but the page includes a form which allows users to set a different praise. The praise is stored inside the user’s session, demonstrating Flask’s session handling. The form also includes a “CSRF token”, and if the form is submitted with an incorrect token (or without one), it is rejected and the session is not updated. This isn’t the only way to do CSRF protection, but it’s one fairly simple implementation; you should never ever have forms without any CSRF protection. (If you don’t want to implement CSRF error handling in every form endpoint, take a look at the require_valid_submitted_request function.)

  This page also includes OAuth support. If OAuth has been configured (see below), then the user can optionally log in using the link in the navigation bar, and if they are logged in, the page will praise them by name and gender the default praise correctly according to the gender they set in their MediaWiki preferences. This demonstrates OAuth handling and how to make authenticated MediaWiki API requests.

• test_app.py is not part of the tool itself, but contains several unit tests for it, written using the pytest framework. You can add your own tests here, and run them using the pytest command.

Feel free to rip whatever code you want from these pages and rearrange it as fits your tool, then throw away everything else that you don’t need.

Automated checks for this tool have been set up in the Makefile. You can run them using the command make check (or just make) in the source code directory (where app.py and requirements.txt are).

Next steps

Here are some next steps you might want to do.

Create the tool account on Wikimedia Toolforge

Create a new tool account with toolsadmin, using the name shexia. See the documentation on Wikitech for more information.

Set up the source code repository

Create a new Phabricator repository with toolsadmin, using the default repository name. (Feel free to host your source code elsewhere, if you want, but in that case don’t forget to update the link in the navbar in base.html.)

Next, push the source code from wherever you’re currently running it to Phabricator. (The following commands should be run from the directory where app.py and requirements.txt are.)

git init .
git add .
git commit -m 'Initial commit'
git remote add origin vcs@git-ssh.wikimedia.org:source/tool-shexia.git
git push -u origin master

To automatically run checks on Travis CI, you’ll also want to push your code to GitHub. Create a new repository named tool-shexia, then add it as a second remote and push your code to it:

git remote add github git@github.com:tool-shexia.git
git push github master

Note that you’ll want to push to both remotes, origin and github, after committing code changes. Also, if this is your first time using Travis CI, you have to sign in there first and authorize it to access your GitHub repositories. (You can also follow the Travis CI Tutorial – note that the .travis.yml file it mentions was already created for you by this template.)

Deploy the tool on Wikimedia Toolforge

SSH into Toolforge (login.tools.wmflabs.org; see the documentation on Wikitech for more information), and then run become shexia.

Create the base directory for the tool.

mkdir -p ~/www/python/

Clone the source code repository.

git clone https://phabricator.wikimedia.org/source/tool-shexia.git ~/www/python/src/

Set up the Python virtual environment. We do this from within a Kubernetes shell, to ensure that we’re using the same Python version that the tool will later run under.

webservice --backend=kubernetes python3.5 shell
python3 -m venv ~/www/python/venv
source ~/www/python/venv/bin/activate
pip install --upgrade pip
pip install -r ~/www/python/src/requirements.txt
exit

Start the webservice.

webservice --backend=kubernetes python3.5 start

Your tool should now be running: visit tools.wmflabs.org/shexia to check. Generally, any time you make changes, the update process will consist of the following commands:

git -C ~/www/python/src pull
webservice --backend=kubernetes python3.5 restart

If the tool requires any new dependencies (that is, if the requirements.txt file was changed), you’ll also want to run the second pip install command from above again. Make sure you do that from within the virtual environment (i. e. you’ve run the source command from above first), and ideally inside the Kubernetes shell as well.

Configure the tool

You should also create the config file for the tool. It currently serves three purposes: it configures the secret key with which Flask signs the user session to protect it against manipulation; it sets the path which Flask uses for the session cookie (optional, but recommended for the Toolforge installation); and it optionally contains the OAuth secrets your tool needs. (Of course, you can add other things to the configuration file if you want to.)

To start working with the config file, run the following two commands:

cp config.yaml.example config.yaml
chmod go-rwx config.yaml

It is imperative that you make the file unreadable to anyone else (with the second command) before you start to actually fill it with real values: it contains secrets which no one else should be able to read.

(Where should you run these commands, by the way? Almost certainly on Toolforge, as your tool account, inside the ~/www/python/src/ directory. You can also do it on the local version, if you want.)

After this, you can start editing the configuration file, using your favorite text editor. On Toolforge, this will have to be a terminal-based editor; if you’re not familiar with those yet, nano is a fairly user-friendly option:

nano config.yaml

For the SECRET_KEY option, you should pick some long (about 50 characters), unguessable string. If you want, you can just mash your keyboard for a bit; alternatively, you can run the following command to generate and print a random string:

cat /proc/sys/kernel/random/uuid

The APPLICATION_ROOT option should be uncommented (by removing the # character at the beginning of the line), but only on Toolforge. For the local version, you can just leave this line as it is.

The oauth section is only necessary if you want to set up OAuth (see the next section), otherwise you can remove it completely.

Register an OAuth consumer

You can register an OAuth consumer on this special page. The application name should be your tool name, Shexia; the version can stay at 1.0 for now, and the description you’ll have to write yourself. Do not check the “This consumer is for use only by fnielsen” box.

For the OAuth callback URL, specify https://tools.wmflabs.org/shexia/oauth/callback (assuming you didn’t mess with the route of the oauth_callback function). You can use this consumer for local testing as well, if you want: the login will redirect you to some URL like https://tools.wmflabs.org/shexia/oauth/callback?oauth_verifier=xxx&oauth_token=yyy, which you can manually rewrite to something like http://localhost:5000/oauth/callback?oauth_verifier=xxx&oauth_token=yyy, simply by editing the URL in your browser’s address bar.

If you know for sure that you will only want to access one project (e. g. www.wikidata.org) with this tool, you can select that as the applicable project, otherwise just leave it at the default “all projects”. Note that test.wikidata.org is a separate wiki, so if you want to access both real and test Wikidata with the same tool, you’ll already need “all projects” access.

Select which rights you want access to, if any (otherwise select one of the “user identification only” options). This will highly depend on your tool, but “edit existing pages”, “create, edit and move pages” and “high-volume editing” are probably the most generally useful ones.

Leave the allowed IP ranges and public RSA key at their default value, you won’t use these. (Toolforge’s IP range is probably not guaranteed to be stable, and we’re not using RSA.)

Check the box at the end of the form (after reading it, and only if you actually agree with what it says – you’re not just following this guide blindly, right?), and submit it. The next page will show you two random strings; these will be the consumer_key and consumer_secret in the config.yaml file.

Afterwards, you can post a steward request for someone to approve of your new consumer, by using the Oauthapprequest template on Steward requests/Miscellaneous. (If you don’t do that, someone will eventually get around to looking at your request too, but it might take longer.) Until your consumer has been approved, you can only test it using your own account (i. e., the one that submitted the request), so you’ll still be able to test the tool by yourself, but other people won’t have access to it. (You will get an email notification once the request is accepted, so there’s no need to keep refreshing your watchlist or anything like that.)

Update the README

The default README.md file created by this template contains some general usage instructions for how to run the tool, but nothing specific as to what it does or what it’s good for. You should probably replace at least the first paragraph to have a better description than “this tool does things”.

Write on-wiki documentation

The tool’s navbar and the README.md file already link to the tool’s on-wiki documentation page, so you should probably create that and put end-user documentation for the tool there.

Announce the tool

When everything else is done (or doesn’t apply, e. g. perhaps your tool just doesn’t need OAuth), you’re ready to announce your tool to the world. (You can also do it earlier, if you want, of course, while it’s still work in progress.) Here are some inspirations for where you could do that:

• project chat
• tool list
• talk pages of relevant WikiProjects
• mailing lists
• social media