Stem Docs

East of the Sun & West of the Moon

East of the Sun & West of the Moon

The following is an overview of some of the utilities Stem provides.

Terminal Styling

Know what's better than text? Pretty text!

OSX, Linux, BSD... really, everything except Windows supports terminal formatting through ANSI escape sequences. Doing this yourself is easy, but we also provide a module to make it even easier.


../_images/words_with.png

import itertools
import re

from stem.util import term
from stem.util.term import Attr, Color


def get_words_with(target, attr):
  """
  Provides words with the given substring highlighted within it.

  :param str target: substring to match against
  :param tuple attr: terminal formatting to highlight the match with

  :returns: **iterable** with words containing that substring
  """

  word_matcher = re.compile('(.*)(%s)(.*)' % target, re.I)

  with open('/etc/dictionaries-common/words') as dictionary_file:
    for word in dictionary_file:
      match = word_matcher.match(word)

      if match:
        yield ''.join((
          match.group(1),
          term.format(match.group(2), *attr),
          match.group(3),
        ))


if __name__ == '__main__':
  target = raw_input("What substring would you like to look for? We'll get words containing it: ")
  attr = (Attr.BOLD, Color.YELLOW)

  print("Words with '%s' include...\n" % term.format(target, *attr))

  for words in itertools.izip_longest(*(get_words_with(target, attr),) * 4):
    print('%-30s%-30s%-30s%-30s' % tuple([w if w else '' for w in words]))

Multiprocessing

Python's multiprocessing module gives building blocks to parallelize around the Global Interpreter Lock. However, honestly it's clunky to use.

Ever just wanted to simply turn your threads into subprocesses? We can do that.

Threaded

import threading
import time

def fibonacci(n):
  if n < 2:
    return n
  else:
    return fibonacci(n-2) + fibonacci(n-1)

# calculate fibonacci sequences four times in parallel

start_time, threads = time.time(), []

for i in range(4):
  t = threading.Thread(target = fibonacci, args = (35,))
  t.setDaemon(True)
  t.start()

  threads.append(t)

for t in threads:
  t.join()

print('took %0.1f seconds' % (time.time() - start_time))
% python fibonacci_threaded.py
took 21.1 seconds

Multi-process

import stem.util.system
import time

def fibonacci(n):
  if n < 2:
    return n
  else:
    return fibonacci(n-2) + fibonacci(n-1)

# calculate fibonacci sequences four times in parallel

start_time, threads = time.time(), []

for i in range(4):
  threads.append(stem.util.system.DaemonTask(fibonacci, (35,), start = True))

for t in threads:
  t.join()

print('took %0.1f seconds' % (time.time() - start_time))
% python fibonacci_multiprocessing.py
took 6.2 seconds

Connection Resolution

Connection information is a useful tool for learning more about network applications like Tor. Our stem.util.connection.get_connections() function provides an easy method for accessing this information, with a few caveats...

  • Connection resolvers are platform specific. We support several but not not all, most notably Windows (ticket 9850).
  • By default Tor runs with a feature called DisableDebuggerAttachment. This prevents debugging applications like gdb from analyzing Tor unless it is run as root. Unfortunately this also alters the permissions of the Tor process /proc contents breaking numerous system tools (including our resolvers). To use this function you need to either run as root (discouraged) or add DisableDebuggerAttachment 0 to your torrc.

Please note that if you operate an exit relay it is highly discouraged for you to look at or record this information. Not only is doing so eavesdropping, but likely also a violation of wiretap laws.

With that out of the way, how do you look up this information? Below is a simple script that dumps Tor's present connections.

import sys

from stem.util.connection import get_connections, system_resolvers
from stem.util.system import pid_by_name

resolvers = system_resolvers()

if not resolvers:
  print("Stem doesn't support any connection resolvers on our platform.")
  sys.exit(1)

picked_resolver = resolvers[0]  # lets just opt for the first
print("Our platform supports connection resolution via: %s (picked %s)" % (', '.join(resolvers), picked_resolver))

tor_pids = pid_by_name('tor', multiple = True)

if not tor_pids:
  print("Unable to get tor's pid. Is it running?")
  sys.exit(1)
elif len(tor_pids) > 1:
  print("You're running %i instances of tor, picking the one with pid %i" % (len(tor_pids), tor_pids[0]))
else:
  print("Tor is running with pid %i" % tor_pids[0])

print("\nConnections:\n")

for conn in get_connections(picked_resolver, process_pid = tor_pids[0], process_name = 'tor'):
  print("  %s:%s => %s:%s" % (conn.local_address, conn.local_port, conn.remote_address, conn.remote_port))
% python example.py
Our platform supports connection resolution via: proc, netstat, sockstat, lsof, ss (picked proc)
Tor is running with pid 17303

Connections:

  192.168.0.1:59014 => 38.229.79.2:443
  192.168.0.1:58822 => 68.169.35.102:443