Stem Docs

System Utilities

System Utilities

Helper functions for working with the underlying system. These are mostly os dependent, only working on linux, osx, and bsd. In almost all cases they're best-effort, providing None if the lookup fails.

Changed in version 1.3.0: Dropped the get_* prefix from several function names. The old names still work, but are deprecated aliases.

Changed in version 1.5.0: Added the SYSTEM_CALL_TIME global, which tracks total time spent making system commands.

Module Overview:

is_windows - checks if we're running on windows
is_mac - checks if we're running on a mac
is_gentoo - checks if we're running on gentoo
is_slackware - checks if we're running on slackware
is_bsd - checks if we're running on the bsd family of operating systems

is_available - determines if a command is available on this system
is_running - determines if a given process is running
size_of - provides the memory usage of an object
call - runs the given system command and provides back the results

name_by_pid - gets the name for a process by the given pid
pid_by_name - gets the pid for a process by the given name
pid_by_port - gets the pid for a process listening to a given port
pid_by_open_file - gets the pid for the process with an open file
pids_by_user - provides processes owned by a user
cwd - provides the current working directory for a given process
user - provides the user a process is running under
start_time - provides the unix timestamp when the process started
tail - provides lines from the end of a file
bsd_jail_id - provides the BSD jail id a given process is running within
bsd_jail_path - provides the path of the given BSD jail

is_tarfile - checks if the given path is a tarball
expand_path - expands relative paths and ~ entries
files_with_suffix - provides files with the given suffix

get_process_name - provides our process' name
set_process_name - changes our process' name
stem.util.system.Status(enum)

State of a subprocess.

New in version 1.6.0.

Status Description
PENDING not yet started
RUNNING currently being performed
DONE completed successfully
FAILED failed with an exception
stem.util.system.argc_t

alias of LP_c_char_p

exception stem.util.system.CallError(msg, command, exit_status, runtime, stdout, stderr)[source]

Bases: exceptions.OSError

Error response when making a system call. This is an OSError subclass with additional information about the process. Depending on the nature of the error not all of these attributes will be available.

Variables:
  • msg (str) -- exception string
  • command (str) -- command that was ran
  • exit_status (int) -- exit code of the process
  • runtime (float) -- time the command took to run
  • stdout (str) -- stdout of the process
  • stderr (str) -- stderr of the process
exception stem.util.system.CallTimeoutError(msg, command, exit_status, runtime, stdout, stderr, timeout)[source]

Bases: stem.util.system.CallError

Error response when making a system call that has timed out.

New in version 1.6.0.

Variables:timeout (float) -- time we waited
class stem.util.system.DaemonTask(runner, args=None, priority=15, start=False)[source]

Bases: object

Invokes the given function in a subprocess, returning the value.

New in version 1.6.0.

Variables:
  • runner (function) -- function to be invoked by the subprocess
  • args (tuple) -- arguments to provide to the subprocess
  • priority (int) -- subprocess nice priority
  • status (stem.util.system.State) -- state of the subprocess
  • runtime (float) -- seconds subprocess took to complete
  • result (object) -- return value of subprocess if successful
  • error (exception) -- exception raised by subprocess if it failed
run()[source]

Invokes the task if it hasn't already been started. If it has this is a no-op.

join()[source]

Provides the result of the daemon task. If still running this blocks until the task is completed.

Returns:response of the function we ran
Raises :exception raised by the function if it failed with one
stem.util.system.is_windows()[source]

Checks if we are running on Windows.

Returns:bool to indicate if we're on Windows
stem.util.system.is_mac()[source]

Checks if we are running on Mac OSX.

Returns:bool to indicate if we're on a Mac
stem.util.system.is_gentoo()[source]

Checks if we're running on Gentoo.

Returns:bool to indicate if we're on Gentoo
stem.util.system.is_slackware()[source]

Checks if we are running on a Slackware system.

Returns:bool to indicate if we're on a Slackware system
stem.util.system.is_bsd()[source]

Checks if we are within the BSD family of operating systems. This currently recognizes Macs, FreeBSD, and OpenBSD but may be expanded later.

Returns:bool to indicate if we're on a BSD OS
stem.util.system.is_available(command, cached=True)[source]

Checks the current PATH to see if a command is available or not. If more than one command is present (for instance "ls -a | grep foo") then this just checks the first.

Note that shell (like cd and ulimit) aren't in the PATH so this lookup will try to assume that it's available. This only happends for recognized shell commands (those in SHELL_COMMANDS).

Parameters:
  • command (str) -- command to search for
  • cached (bool) -- makes use of available cached results if True
Returns:

True if an executable we can use by that name exists in the PATH, False otherwise
stem.util.system.is_running(command)[source]

Checks for if a process with a given name or pid is running.

Changed in version 1.6.0: Added support for list and pid arguments.

Parameters:command (str,list,int) -- process name if a str, multiple process names if a list, or pid if an int to be checked
Returns:True if the process is running, False if it's not among ps results, and None if ps can't be queried
stem.util.system.size_of(obj, exclude=None)[source]

Provides the approximate memory usage of an object. This can recurse tuples, lists, deques, dicts, and sets. To teach this function to inspect additional object types expand SIZE_RECURSES...

stem.util.system.SIZE_RECURSES[SomeClass] = SomeClass.get_elements

New in version 1.6.0.

Parameters:
  • obj (object) -- object to provide the size of
  • exclude (set) -- object ids to exclude from size estimation
Returns:

int with the size of the object in bytes
Raises :

NotImplementedError if using PyPy
stem.util.system.name_by_pid(pid)[source]

Attempts to determine the name a given process is running under (not including arguments). This uses...

1. Information from /proc
2. ps -p <pid> -o command
Parameters:pid (int) -- process id of the process to be queried
Returns:str with the process name, None if it can't be determined
stem.util.system.pid_by_name(process_name, multiple=False)[source]

Attempts to determine the process id for a running process, using...

1. pgrep -x <name>
2. pidof <name>
3. ps -o pid -C <name> (linux)
   ps axc | egrep " <name>$" (bsd)
4. lsof -tc <name>
5. tasklist | str <name>.exe
Parameters:
  • process_name (str) -- process name for which to fetch the pid
  • multiple (bool) -- provides a list of all pids if True, otherwise results with multiple processes are discarded
Returns:

Response depends upon the 'multiple' argument as follows...

  • if False then this provides an int with the process id or None if it can't be determined
  • if True then this provides a list of all int process ids, and an empty list if it can't be determined
stem.util.system.pid_by_port(port)[source]

Attempts to determine the process id for a process with the given port, using...

1. netstat -npltu | grep 127.0.0.1:<port>
2. sockstat -4l -P tcp -p <port>
3. lsof -wnP -iTCP -sTCP:LISTEN | grep ":<port>"

Most queries limit results to listening TCP connections. This function likely won't work on Mac OSX.

Parameters:port (int) -- port where the process we're looking for is listening
Returns:int with the process id, None if it can't be determined
stem.util.system.pid_by_open_file(path)[source]

Attempts to determine the process id for a process with the given open file, using...

lsof -w <path>
Parameters:path (str) -- location of the socket file to query against
Returns:int with the process id, None if it can't be determined
stem.util.system.pids_by_user(user)[source]

Provides processes owned by a given user.

New in version 1.5.0.

Parameters:user (str) -- user to look up processes for
Returns:list with the process ids, None if it can't be determined
stem.util.system.cwd(pid)[source]

Provides the working directory of the given process.

Parameters:pid (int) -- process id of the process to be queried
Returns:str with the absolute path for the process' present working directory, None if it can't be determined
stem.util.system.user(pid)[source]

Provides the user a process is running under.

Parameters:pid (int) -- process id of the process to be queried
Returns:str with the username a process is running under, None if it can't be determined
stem.util.system.start_time(pid)[source]

Provides the unix timestamp when the given process started.

Parameters:pid (int) -- process id of the process to be queried
Returns:float for the unix timestamp when the process began, None if it can't be determined
stem.util.system.tail(target, lines=None)[source]

Provides lines of a file starting with the end. For instance, 'tail -n 50 /tmp/my_log' could be done with...

reversed(list(tail('/tmp/my_log', 50)))
Parameters:
  • target (str,file) -- path or file object to read from
  • lines (int) -- number of lines to read
Returns:

generator that reads lines, starting with the end
Raises :

IOError if unable to read the file
stem.util.system.bsd_jail_id(pid)[source]

Gets the jail id for a process. These seem to only exist for FreeBSD (this style for jails does not exist on Linux, OSX, or OpenBSD).

Parameters:pid (int) -- process id of the jail id to be queried
Returns:int for the jail id, zero if this can't be determined
stem.util.system.bsd_jail_path(jid)[source]

Provides the path of the given FreeBSD jail.

Parameters:jid (int) -- jail id to be queried
Returns:str of the path prefix, None if this can't be determined
stem.util.system.is_tarfile(path)[source]

Returns if the path belongs to a tarfile or not.

New in version 1.2.0.

Parameters:path (str) -- path to be checked
Returns:True if the path belongs to a tarball, False otherwise
stem.util.system.expand_path(path, cwd=None)[source]

Provides an absolute path, expanding tildes with the user's home and appending a current working directory if the path was relative.

Parameters:
  • path (str) -- path to be expanded
  • cwd (str) -- current working directory to expand relative paths with, our process' if this is None
Returns:

str of the path expanded to be an absolute path, never with an ending slash
stem.util.system.files_with_suffix(base_path, suffix)[source]

Iterates over files in a given directory, providing filenames with a certain suffix.

New in version 1.2.0.

Parameters:
  • base_path (str) -- directory to be iterated over
  • suffix (str) -- filename suffix to look for
Returns:

iterator that yields the absolute path for files with the given suffix
stem.util.system.call(command, default = UNDEFINED, ignore_exit_status = False)[source]

Issues a command in a subprocess, blocking until completion and returning the results. This is not actually ran in a shell so pipes and other shell syntax are not permitted.

Changed in version 1.5.0: Providing additional information upon failure by raising a CallError. This is a subclass of OSError, providing backward compatibility.

Changed in version 1.5.0: Added env argument.

Changed in version 1.6.0: Added timeout and cwd arguments.

Parameters:
  • command (str,list) -- command to be issued
  • default (object) -- response if the query fails
  • ignore_exit_status (bool) -- reports failure if our command's exit status was non-zero
  • timeout (float) -- maximum seconds to wait, blocks indefinitely if None
  • env (dict) -- environment variables
Returns:

list with the lines of output from the command
Raises :
  • CallError if this fails and no default was provided
  • CallTimeoutError if the timeout is reached without a default
stem.util.system.get_process_name()[source]

Provides the present name of our process.

Returns:str with the present name of our process
stem.util.system.set_process_name(process_name)[source]

Renames our current process from "python <args>" to a custom name. This is best-effort, not necessarily working on all platforms.

Parameters:process_name (str) -- new name for our process
Raises :IOError if the process cannot be renamed
stem.util.system.get_name_by_pid(pid)

Attempts to determine the name a given process is running under (not including arguments). This uses...

1. Information from /proc
2. ps -p <pid> -o command
Parameters:pid (int) -- process id of the process to be queried
Returns:str with the process name, None if it can't be determined
stem.util.system.get_pid_by_name(process_name, multiple=False)

Attempts to determine the process id for a running process, using...

1. pgrep -x <name>
2. pidof <name>
3. ps -o pid -C <name> (linux)
   ps axc | egrep " <name>$" (bsd)
4. lsof -tc <name>
5. tasklist | str <name>.exe
Parameters:
  • process_name (str) -- process name for which to fetch the pid
  • multiple (bool) -- provides a list of all pids if True, otherwise results with multiple processes are discarded
Returns:

Response depends upon the 'multiple' argument as follows...

  • if False then this provides an int with the process id or None if it can't be determined
  • if True then this provides a list of all int process ids, and an empty list if it can't be determined
stem.util.system.get_pid_by_port(port)

Attempts to determine the process id for a process with the given port, using...

1. netstat -npltu | grep 127.0.0.1:<port>
2. sockstat -4l -P tcp -p <port>
3. lsof -wnP -iTCP -sTCP:LISTEN | grep ":<port>"

Most queries limit results to listening TCP connections. This function likely won't work on Mac OSX.

Parameters:port (int) -- port where the process we're looking for is listening
Returns:int with the process id, None if it can't be determined
stem.util.system.get_pid_by_open_file(path)

Attempts to determine the process id for a process with the given open file, using...

lsof -w <path>
Parameters:path (str) -- location of the socket file to query against
Returns:int with the process id, None if it can't be determined
stem.util.system.get_cwd(pid)

Provides the working directory of the given process.

Parameters:pid (int) -- process id of the process to be queried
Returns:str with the absolute path for the process' present working directory, None if it can't be determined
stem.util.system.get_user(pid)

Provides the user a process is running under.

Parameters:pid (int) -- process id of the process to be queried
Returns:str with the username a process is running under, None if it can't be determined
stem.util.system.get_start_time(pid)

Provides the unix timestamp when the given process started.

Parameters:pid (int) -- process id of the process to be queried
Returns:float for the unix timestamp when the process began, None if it can't be determined
stem.util.system.get_bsd_jail_id(pid)

Gets the jail id for a process. These seem to only exist for FreeBSD (this style for jails does not exist on Linux, OSX, or OpenBSD).

Parameters:pid (int) -- process id of the jail id to be queried
Returns:int for the jail id, zero if this can't be determined
stem.util.system.get_bsd_jail_path(jid)

Provides the path of the given FreeBSD jail.

Parameters:jid (int) -- jail id to be queried
Returns:str of the path prefix, None if this can't be determined