Frequently Asked Questions¶
- General Information
What is Stem?¶
Does Stem have any dependencies?¶
No. All you need in order to use Stem is Python.
When it is available Stem will use pycrypto to validate descriptor signatures. However, there is no need to install pycrypto unless you need this functionality.
What Python versions is Stem compatible with?¶
Stem works with Python 2.6 and greater, including the Python 3.x series.
Can I interact with Tor's controller interface directly?¶
Yup. You don't need a library to interact with Tor's controller interface, and interacting with it directly is a great way to learn about what it can do. The exact details for how you connect to Tor depend on two things...
- Where is Tor listening for controller connections? This is specified by either the ControlPort or ControlSocket option in your torrc. If you have neither then Tor will not accept controller connections.
- What type of authentication is Tor's controller interface using? This is defined by your CookieAuthentication or HashedControlPassword option. If you have neither then Tor does not restrict access.
We'll tackle each of these scenarios one at a time...
I'm using a ControlPort¶
If you are using a ControlPort then the easiest method of talking with Tor is via telnet. You always need to authenticate after connecting, even if Tor does not restrict access. If your torrc doesn't have a CookieAuthentication or HashedControlPassword then to authenticate you will simply call AUTHENTICATE after connecting without any credentials.
% cat ~/.tor/torrc ControlPort 9051 % telnet localhost 9051 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. AUTHENTICATE 250 OK GETINFO version 250-version=0.2.5.1-alpha-dev (git-245ecfff36c0cecc) 250 OK QUIT 250 closing connection Connection closed by foreign host.
I'm using a ControlSocket¶
A ControlSocket is a file based socket, so we'll use socat to connect to it...
% cat ~/.tor/torrc ControlSocket /home/atagar/.tor/socket % socat UNIX-CONNECT:/home/atagar/.tor/socket STDIN AUTHENTICATE 250 OK GETINFO version 250-version=0.2.5.1-alpha-dev (git-245ecfff36c0cecc) 250 OK QUIT 250 closing connection
I'm using password authentication¶
Tor's other method of authentication is a credential you know. To use it ask Tor to hash your password, then use that in your torrc...
% tor --hash-password "my_password" 16:E600ADC1B52C80BB6022A0E999A7734571A451EB6AE50FED489B72E3DF
Authenticating with this simply involves giving Tor the credential...
% cat ~/.tor/torrc ControlPort 9051 HashedControlPassword 16:E600ADC1B52C80BB6022A0E999A7734571A451EB6AE50FED489B72E3DF % telnet localhost 9051 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. AUTHENTICATE "my_password" 250 OK GETINFO version 250-version=0.2.5.1-alpha-dev (git-245ecfff36c0cecc) 250 OK QUIT 250 closing connection Connection closed by foreign host.
Are there any other controller libraries?¶
The following are the functional controller libraries I'm aware of. Dates are for highly active development. If I missed one then please let me know!
|Stem||Python||October 2011 - Present|
|Txtorcon||Python (Twisted)||February 2012 - Present|
|TorCtl||Python||July 2008 - November 2011|
|TorUtils||PHP||October 2015 - Present|
|PHP TorCtl||PHP||February 2013|
|JTorCtl||Java||June 2005 - May 2009|
|Rust Controller||Rust||May 2016|
How do I connect to Tor?¶
Once you have Tor running and properly configured you have a few ways of connecting to it. The following are the most common methods for getting a Controller instance, from the highest to lowest level...
This function connects and authenticates to the given port or socket, providing you a one-line method of getting a Controller that's ready to use. If Tor requires a password then the user will be prompted for it. When the connection cannot be established this prints a description of the problem to stdout and returns None.
The connection module helpers above are all well and good when you need a quick-and-dirty connection for your commandline script, but they're inflexible. In particular their lack of exceptions and direct use of stdin/stdout make them undesirable for more complicated situations. That's where the Controller's from_port() and from_socket_file() methods come in.
These provide the most flexible method of connecting to Tor, and for sophisticated applications is what you'll want.
For the diehards among us you can skip the conveniences of a high level Controller and work directly with the raw components. At Stem's lowest level your connection with Tor is a ControlSocket subclass. This provides methods to send, receive, disconnect, and reconnect to Tor.
How do I request a new identity from Tor?¶
In Tor your identity is the three-hop circuit over which your traffic travels through the Tor network.
Tor periodically creates new circuits. When a circuit is used it becomes dirty, and after ten minutes new connections will not use it. When all of the connections using an expired circuit are done the circuit is closed.
An important thing to note is that a new circuit does not necessarily mean a new IP address. Paths are randomly selected based on heuristics like speed and stability. There are only so many large exits in the Tor network, so it's not uncommon to reuse an exit you have had previously.
Tor does not have a method for cycling your IP address. This is on purpose, and done for a couple reasons. The first is that this capability is usually requested for not-so-nice reasons such as ban evasion or SEO. Second, repeated circuit creation puts a very high load on the Tor network, so please don't!
With all that out of the way, how do you create a new circuit? You can customize the rate at which Tor cycles circuits with the MaxCircuitDirtiness option in your torrc. Vidalia and arm both provide a method to request a new identity, and you can do so programmatically by sending Tor a NEWNYM signal.
To do this with telnet...
% telnet localhost 9051 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. AUTHENTICATE 250 OK SIGNAL NEWNYM 250 OK
And with Stem...
from stem import Signal from stem.control import Controller with Controller.from_port(port = 9051) as controller: controller.authenticate() controller.signal(Signal.NEWNYM)
How do I reload my torrc?¶
Tor is configured through its torrc. When you edit this file you need to either restart Tor or issue a HUP for the changes to be reflected. To issue a HUP you can either...
- Run pkill -sighup tor.
- Send Tor a HUP signal through its control port...
from stem import Signal from stem.control import Controller with Controller.from_port(port = 9051) as controller: controller.authenticate() controller.signal(Signal.HUP)
How do I read *.tar.xz descriptor archives?¶
Stem's parse_file() and DescriptorReader can read plaintext descriptors and tarballs. However, metrics uses *.xz compression. Python 3.3 adds builtin xz support, but if you're using an earlier version of python you will need to decompress the archives yourself.
With modern versions of tar you can simply decompress archives via tar xf archive.tar.xz, or programmatically using lzma.
What is that 'with' keyword I keep seeing in the tutorials?¶
with Controller.from_port(port = 9051) as controller: # do my stuff
... is equivalent to...
controller = Controller.from_port(port = 9051) try: # do my stuff finally: controller.close()
This helps to make sure that regardless of if your code raises an exception or not the control connection will be cleaned up afterward. Note that this means that if you leave the 'with' scope your Controller will be closed. The following for instance is a bug common when first learning Stem...
class BandwidthReporter(object): def __init__(self, controller): self.controller = controller def print_bandwidth(self): bytes_read = self.controller.get_info("traffic/read") bytes_written = self.controller.get_info("traffic/written") print "My Tor relay has read %s bytes and written %s." % (bytes_read, bytes_written) if __name__ == '__main__': with Controller.from_port(port = 9051) as controller: reporter = BandwidthReporter(controller) # The following line is broken because the 'controller' we initialised # above was disconnected once we left the 'with' scope. To fix this the # print_bandwidth() call should be in the 'with' block. reporter.print_bandwidth()
How do I get started?¶
The best way of getting involved with any project is to jump right in! Our bug tracker lists several development tasks. In particular look for the 'easy' keyword when getting started. If you have any questions then I'm always more than happy to help! I'm atagar on oftc and also available via email.
To start hacking on Stem please do the following and don't hesitate to let me know if you get stuck or would like to discuss anything!
- Clone our git repository: git clone https://git.torproject.org/stem.git
- Get our test dependencies: sudo pip install mock pycodestyle pyflakes.
- Find a bug or feature that sounds interesting.
- When you have something that you would like to contribute back do the following...
- If you don't already have a publicly accessible Stem repository then set one up. GitHub in particular is great for this.
- File a trac ticket, the only fields you'll need are...
- Summary: short description of your change
- Description: longer description and a link to your repository with either the git commits or branch that has your change
- Type: 'defect' if this is a bug fix and 'enhancement' otherwise
- Priority: rough guess at the priority of your change
- Component: Core Tor / Stem
- I'll review the change and give suggestions. When we're both happy with it I'll push your change to the official repository.
How do I run the tests?¶
Stem has three kinds of tests: unit, integration, and static.
Unit tests are our most frequently ran tests. They're quick, they're easy, and provide good test coverage...
~$ cd stem/ ~/stem$ ./run_tests.py --unit
Integration tests start a live Tor instance and test against that. This not only provides additional test coverage, but lets us check our continued interoperability with new releases of Tor. Running these require that you have Tor installed. You can exercise alternate Tor configurations with the --target argument (see run_tests.py --help for a list of its options).
~/stem$ ./run_tests.py --integ ~/stem$ ./run_tests.py --integ --tor /path/to/tor ~/stem$ ./run_tests.py --integ --target RUN_COOKIE
See run_tests.py --help for more usage information.
How can I test compatibility with multiple python versions?¶
Stem supports python versions 2.6 and above, including the 3.x series. You can test all versions of python you currently have installed on your system with tox. If you're using a Debian based system this can be as simple as...
~/stem$ sudo apt-get install python-tox python2.7 python3.3 python-dev python3-dev ~/stem$ tox ... ____ summary _____ py27: commands succeeded py33: commands succeeded
Tox fetches Stem's dependencies for each version of python. One of these dependencies is pycrypto which requires python-dev (or python3-dev if testing with python3).
Tox also allows you to customize the underlying commands and environments. For example...
# run the tests with just python 2.6 ~/stem$ tox -e py26 # pass arguments to 'run_tests.py' ~/stem$ tox -e py26 -- -u --test response.events
How do I build the site?¶
If you have Sphinx version 1.1 or later installed then building our site is as easy as...
~$ cd stem/docs ~/stem/docs$ make html
When it's finished you can direct your browser to the _build directory with a URI similar to...
What is the copyright for patches?¶
Stem is under the LGPLv3 which is a fine license, but poses a bit of a problem for sharing code with our other projects (which are mostly BSD). To share code without needing to hunt down prior contributors we need Tor to have the copyright for the whole Stem codebase. Presently the copyright of Stem is jointly held by its main author (Damian) and the Tor Project.
If you submit a substantial patch I'll ask if you're fine with it being in the public domain. This would mean that there are no legal restrictions for using your contribution, and hence won't pose a problem if we reuse Stem code in other projects.