East of the Sun & West of the Moon¶
The following is an overview of some of the utilities Stem provides.
Terminal Styling¶
Know what's better than text? Pretty text!
OSX, Linux, BSD... really, everything except Windows supports terminal formatting through ANSI escape sequences. Doing this yourself is easy, but we also provide a module to make it even easier.
import itertools
import re
from stem.util import term
from stem.util.term import Attr, Color
def get_words_with(target, attr):
"""
Provides words with the given substring highlighted within it.
:param str target: substring to match against
:param tuple attr: terminal formatting to highlight the match with
:returns: **iterable** with words containing that substring
"""
word_matcher = re.compile('(.*)(%s)(.*)' % target, re.I)
with open('/etc/dictionaries-common/words') as dictionary_file:
for word in dictionary_file:
match = word_matcher.match(word)
if match:
yield ''.join((
match.group(1),
term.format(match.group(2), *attr),
match.group(3),
))
if __name__ == '__main__':
target = raw_input("What substring would you like to look for? We'll get words containing it: ")
attr = (Attr.BOLD, Color.YELLOW)
print("Words with '%s' include...\n" % term.format(target, *attr))
for words in itertools.izip_longest(*(get_words_with(target, attr),) * 4):
print('%-30s%-30s%-30s%-30s' % tuple([w if w else '' for w in words]))
Multiprocessing¶
Python's multiprocessing module gives building blocks to parallelize around the Global Interpreter Lock. However, honestly it's clunky to use.
Ever just wanted to simply turn your threads into subprocesses? We can do that.
Threaded
import threading
import time
def fibonacci(n):
if n < 2:
return n
else:
return fibonacci(n-2) + fibonacci(n-1)
# calculate fibonacci sequences four times in parallel
start_time, threads = time.time(), []
for i in range(4):
t = threading.Thread(target = fibonacci, args = (35,))
t.setDaemon(True)
t.start()
threads.append(t)
for t in threads:
t.join()
print('took %0.1f seconds' % (time.time() - start_time))
% python fibonacci_threaded.py
took 21.1 seconds
Multi-process
import stem.util.system
import time
def fibonacci(n):
if n < 2:
return n
else:
return fibonacci(n-2) + fibonacci(n-1)
# calculate fibonacci sequences four times in parallel
start_time, threads = time.time(), []
for i in range(4):
threads.append(stem.util.system.DaemonTask(fibonacci, (35,), start = True))
for t in threads:
t.join()
print('took %0.1f seconds' % (time.time() - start_time))
% python fibonacci_multiprocessing.py
took 6.2 seconds
Connection Resolution¶
Connection information is a useful tool for learning more about network applications like Tor. Our stem.util.connection.get_connections() function provides an easy method for accessing this information, with a few caveats...
- Connection resolvers are platform specific. We support several platforms but not all.
- By default Tor runs with a feature called DisableDebuggerAttachment. This prevents debugging applications like gdb from analyzing Tor unless it is run as root. Unfortunately this also alters the permissions of the Tor process /proc contents breaking numerous system tools (including our resolvers). To use this function you need to either run as root (discouraged) or add DisableDebuggerAttachment 0 to your torrc.
Please note that if you operate an exit relay it is highly discouraged for you to look at or record this information. Not only is doing so eavesdropping, but likely also a violation of wiretap laws.
With that out of the way, how do you look up this information? Below is a simple script that dumps Tor's present connections.
import sys
from stem.util.connection import get_connections, system_resolvers
from stem.util.system import pid_by_name
resolvers = system_resolvers()
if not resolvers:
print("Stem doesn't support any connection resolvers on our platform.")
sys.exit(1)
picked_resolver = resolvers[0] # lets just opt for the first
print("Our platform supports connection resolution via: %s (picked %s)" % (', '.join(resolvers), picked_resolver))
tor_pids = pid_by_name('tor', multiple = True)
if not tor_pids:
print("Unable to get tor's pid. Is it running?")
sys.exit(1)
elif len(tor_pids) > 1:
print("You're running %i instances of tor, picking the one with pid %i" % (len(tor_pids), tor_pids[0]))
else:
print("Tor is running with pid %i" % tor_pids[0])
print("\nConnections:\n")
for conn in get_connections(picked_resolver, process_pid = tor_pids[0], process_name = 'tor'):
print(" %s:%s => %s:%s" % (conn.local_address, conn.local_port, conn.remote_address, conn.remote_port))
% python example.py
Our platform supports connection resolution via: proc, netstat, sockstat, lsof, ss (picked proc)
Tor is running with pid 17303
Connections:
192.168.0.1:59014 => 38.229.79.2:443
192.168.0.1:58822 => 68.169.35.102:443