Ellipsis Object
The Ellipsis object is used to indicate the presence of an ellipsis in an index lookup .There is a single object of this type, accessed through the built-in name Ellipsis. It has no attributes and evaluates as True. None of Python's built-in types make use of Ellipsis, but it may be useful if you are trying to build advanced functionality into the indexing operator on your own objects.The following code shows how an Ellipsis gets created and passed into the indexing operator e 3, , 4 Calls...
abc
The abc module defines a metaclass and a pair of decorators for defining new abstract base classes. A metaclass that represents an abstract base class. To define an abstract class, you define a class that uses ABCMeta as a metaclass. For example class Stackable In Python 3, use the syntax metaclass abc.ABCMeta class Stackable metaclass abc.ABCMETA A class created in this manner differs from an ordinary class in a few critical ways First, if the abstract class defines methods or properties that...
Basic Concepts
A running program is called a process. Each process has its own system state, which includes memory, lists of open files, a program counter that keeps track of the instruction being executed, and a call stack used to hold the local variables of functions. Normally, a process executes statements one after the other in a single sequence of control flow, which is sometimes called the main thread of the process. At any given time, the program is only doing one thing. A program can create new...
Description Slg
Maximum length of a filename in a directory. Indicates whether an attempt to create a file with a name longer than pc_name_max for a directory will fail with an Maximum length of a relative path name when the directory fd is the current working directory. Size of the pipe buffer when fd refers to a pipe or FIFO. Indicates whether priority I O can be performed on fd. Indicates whether synchronous I O can be performed on fd. Indicates whether fd allows special-character processing to be disabled....
collections
The collections module contains high-performance implementations of a few useful container types, abstract base classes for various kinds of containers, and a utility function for creating name-tuple objects. Each is described in the sections that follow. Two new containers are defined in the collections module deque and defaultdict. Type representing a double-ended queue deque, pronounced deck object. iterable is an iterable object used to populate the deque.A deque allows items to be inserted...
Handler Objects
The logging module provides a collection of pre-built handlers that can process log messages in various in ways. These handlers are added to Logger objects using their addHandler method. In addition, each handler can be configured with its own filtering and levels. The following handler objects are built-in. Some of these handlers are defined in a submodule logging.handlers, which must be imported specifically if necessary. Sends log messages to a UDP server located on the given host and port....
Unicode String Handling
A common problem associated with I O handling is that of dealing with international characters represented as Unicode. If you have a string s of raw bytes containing an encoded representation of a Unicode string, use the s .decode encoding , errors method to convert it into a proper Unicode string.To convert a Unicode string, u, to an encoded byte string, use the string method u .encode encoding , errors . Both of these conversion operators require the use of a special encoding name that...
Addressing
In order to perform any communication on a socket, you have to specify a destination address.The form of the address depends on the address family of the socket. For Internet applications using IPv4, addresses are specified as a tuple host, port . Here are two examples 'www.python.org', 80 '66.113.130.182', 25 If host is the empty string, it has the same meaning as INADDR_ANY, which means any address. This is typically used by servers when creating sockets that any client can connect to. If...
Customization of Application Servers
Other library modules often use the SocketServer class to implement servers for application-level protocols such as HTTP and XML-RPC. Those servers can also be customized via inheritance and extending the methods defined for basic server operation. For example, here is a forking XML-RPC server that only accepts connections originating on the loopback interface from xmlrpc.server import SimpleXMLRPCServer from socketserver import ForkingMixIn except ImportError from SimpleXMLRPCServer import...
Functions Xgb
The socket module defines the following functions create_connection address , timeout Establishes a SOCK_STREAM connection to address and returns an already connected socket object. address is tuple of the form host, port , and timeout specifies an optional timeout. This function works by first calling getaddrinfo and then trying to connect to each of the tuples that gets returned. fromfd fd, family, socktype , proto Creates a socket object from an integer file descriptor, fd. The address...
Type Conversion from C to Python
The following C function is used to convert the values contained in C variables to a Python object PyObject Py_BuildValue char format, This constructs a Python object from a series of C variables. format is a string describing the desired conversion. The remaining arguments are the values of C variables to be converted. The format specifier is similar to that used with the PyArg_ParseTuple functions, as shown in Table 26.4. Table 26.4 Format Specifiers for Py_BuiidVaiue s String char s String...
Description Pke
Tuple containing names of variables referenced by nested functions. Tuple containing names of free variables used by nested functions. Tuple containing the literals used by the bytecode. Tuple containing names used by the bytecode. Name of the file in which the code was compiled. String encoding bytecode offsets to line numbers. Required stack size including local variables . Integer containing interpreter flags. Bit 2 is set if the function uses a variable number of positional arguments using...
The WSGI Specification
With WSGI, a web application is implemented as a function or callable object webapp environ, start_response that accepts two arguments. environ is a dictionary of environment settings that is minimally required to have the following values which have the same meaning and names as is used in CGI scripting Length of data passed Type of query data MIME types accepted by the client Netscape persistent cookie value Referring URL Client browser Extra path information passed Query string Method 'GET'...
E
Figure 23.1 Possible browser display where the background text is just an ordinary HTML document and the pop-up window is dynamically generated by the popupdata.py script. The rest of this chapter describes built-in modules related to the low-level interface by which Python interfaces with webservers and frameworks.Topics include CGI scripting, a technique used to access Python from third-party web servers and WSGI, a middleware layer used for writing components that integrate with Python's...
Shared Data and Synchronization
Normally, processes are completed isolated from each other with the only means of communication being queues or pipes. However, two objects can be used to represent shared data. Underneath the covers, these objects use shared memory via mmap to make access possible in multiple processes. Value typecode, argl, argN, lock Creates a ctypes object in shared memory. typecode is either a string containing a type code as used by the array module e.g., 'i', 'd', etc. or a type object from the ctypes...
xmlrpc Package
The xmlrpc package contains modules for implement XML-RPC servers and clients. XML-RPC is a remote procedure call mechanism that uses XML for data encoding and HTTP as a transport mechanism. The underlying protocol is not specific to Python so programs using these modules can potentially interact with programs written in other languages. More information about XML-RPC can be obtained at http www. xmlrpc.com. The xmlrpc.client module is used to write XML-RPC clients. In Python 2, this module is...
List of string classifiers
Creating a setup.py file is enough to create a source distribution of your software. Type the following shell command to make a source distribution This creates an archive file such as spam-1.0.tar.gz or spam-1-0.zip in the directory spam dist.This is the file you would give to others to install your software.To install, a user simply unpacks the archive and performs these steps cd spam-1.0 python setup.py install This installs the software into the local Python distribution and makes it...
Object Inspection and dir
The dir function is commonly used to inspect objects. An object can supply the list of names returned by dir by implementing__dir__ self . Defining this makes it easier to hide the internal details of objects that you don't want a user to directly access However, keep in mind that a user can still inspect the underlying__dict__attribute of instances and classes to see everything that is defined. his chapter describes Python's built-in operators, expressions, and evaluation rules. Although much...
Documentation Strings Obd
It is common practice for the first statement of function to be a documentation string describing its usage. For example Computes n factorial. For example The documentation string is stored in the__doc__attribute of the function that is commonly used by IDEs to provide interactive help. If you are using decorators, be aware that wrapping a function with a decorator can break the help features associated with documentation strings. For example, consider this code return func args, kwargs return...
urllib Package
The urllib package provides a high-level interface for writing clients that need to interact with HTTP servers, FTP servers, and local files. Typical applications include scraping data from web pages, automation, proxies, web crawlers, and so forth. This is one of the most highly configurable library modules, so every last detail is not presented here. Instead, the most common uses of the package are described. In Python 2, the urllib functionality is spread across several different library...
Message Formatting
By default, Handler objects emit log messages exactly as they are formatted in logging calls. However, sometimes you want to add additional contextual information to the messages such as timestamps, filenames, line numbers, and so forth. This section describes how this extra information can be automatically added to log messages. To change the log message format, you must first create a Formatter object Formatter fmt , datefmt Creates a new Formatter object. fmt provides a format string for...
boolself Returns False or True for truthvalue testing hashself Computes an
Objects can implement one or more of the relational operators lt , gt , lt , gt , , . Each of these methods takes two arguments and is allowed to return any kind of object, including a Boolean value, a list, or any other Python type. For instance, a numerical package might use this to perform an element-wise comparison of two matrices, returning a matrix with the results. If a comparison can't be made, these functions may also raise an exception.Table 3.14 shows the special methods for...
binascii
The binascii module contains low-level functions for converting data between binary and a variety of ASCII encodings, such as base 64, BinHex, and UUencoding. Converts a line of uuencoded text s to binary and returns a byte string. Lines normally contain 45 binary bytes, except for the last line that may be less. Line data may be followed by whitespace. Converts a string of binary data to a line of uuencoded ASCII characters. The length of data should not be more than 45 bytes. Otherwise, the...
XML Example Document
The following example illustrates a typical XML document, in this case a description of a recipe. lt xml version 1.0 encoding iso-8859-1 gt lt recipe gt lt title gt lt item num 4 gt Large avocados, chopped lt item gt lt item num 1 gt Tomato, chopped lt item gt lt item num 1 2 units C gt White onion, chopped lt item gt lt item num 2 units tbl gt Fresh squeezed lemon juice lt item gt lt item num 1 gt Jalapeno pepper, diced lt item gt lt item num 1 units tbl gt Fresh cilantro, minced lt item gt lt...
Unicode Data Encodings
Table 9.3 lists some of the most commonly used encoders in the codecs module. Table 9.3 Encoders in the codecs Module 'latin-1', 'iso-8859-1' Latin-1 or ISO-8859-1 encoding 'utf-8' 8-bit variable-length encoding 'utf-16' 16-bit variable-length encoding 'utf-16-le' UTF-16, but with explicit little endian encoding 'utf-16-be' UTF-16, but with explicit big endian encoding 'unicode-escape' Same format as u string 'raw-unicode-escape' Same format as ur string The following sections describe each of...
http Package
The http package consists of modules for writing HTTP clients and servers as well as support for state management cookies .The Hypertext Transfer Protocol HTTP is a simple text-based protocol that works as follows 1. A client makes a connection to an HTTP server and sends a request header of the following form GET document.html HTTP 1.0 Connection Keep-Alive User-Agent Mozilla 4.61 en X11 U SunOS 5.6 sun4u Host rustler.cs.uchicago.edu 8000 Accept image gif, image x-xbitmap, image jpeg, image...
cgitb
This module provides an alternative exception handler that displays a detailed report whenever an uncaught exception occurs. The report contains source code, values of parameters, and local variables. Originally, this module was developed to help debug CGI scripts, but it can be used in any application. enable display , logdir , context , format Enables special exception handling. display is a flag that determines whether any information is displayed when an error occurs.The default value is 1....
Making Memory Measurements
The sys module has a function getsizeof that can be used to investigate the memory footprint in bytes of individual Python objects. For example gt gt gt import sys gt gt gt sys.getsizeof 1 gt gt gt sys.getsizeof Hello World 52 gt gt gt sys.getsizeof 1,2,3,4 gt gt gt sum sys.getsizeof x for x in 1,2,3,4 For containers such as lists, tuples, and dictionaries, the size that gets reported is just for the container object itself, not the cumulative size of all objects contained inside of it. For...
Symbols Numbers
debugger command, pdb module, 187 not equal to operator, 66 ' single quotes, 11, 27 ''' triple quotes, 11, 27 double quotes, 11, 27 triple quotes, 11, 27 rewriting on package installation, 153 modulo operator, 65 string formatting operator, 8, 70,162 operator, 75 amp bitwise-and operator, 65 amp set intersection operator, 15, 75 amp operator, 75 function call operator, 76 tuple, 14, 29 keyword only arguments, Python 3, 625 passing sequences as function arguments, 94 sequence replication...
functools
The functools module contains functions and decorators that are useful for creating higher-order functions, functional programming, and decorators. partial function , args , kwargs Creates a function-like object, partial, that when called, calls function with positional arguments args, keyword arguments kwargs, and any additional positional or keyword arguments that are supplied. Additional positional arguments are added to the end of args, and additional keyword arguments are merged into...
Semaphore and Bounded Semaphore
A semaphore is a synchronization primitive based on a counter that's decremented by each acquire call and incremented by each release call. If the counter ever reaches zero, the acquire method blocks until some other thread calls release . Creates a new semaphore. value is the initial value for the counter. If omitted, the counter is set to a value of 1. A Semaphore instance, s, supports the following methods Acquires the semaphore. If the internal counter is larger than zero on entry, this...
Description Sbi
Left shift Right shift Bitwise and Bitwise or Bitwise xor exclusive or Bitwise negation The bitwise operators assume that integers are represented in a 2's complement binary representation and that the sign bit is infinitely extended to the left. Some care is required if you are working with raw bit-patterns that are intended to map to native integers on the hardware. This is because Python does not truncate the bits or allow values to overflow instead, the result will grow arbitrarily large in...
Process Management
The following functions and variables are used to create, destroy, and manage processes abort Generates a SIGABRT signal that's sent to the calling process. Unless the signal is caught with a signal handler, the default is for the process to terminate with an error. This variable contains the default search path used by the exec p functions if the environment doesn't have a 'PATH' variable. Equivalent to execv path, arg0, argl, . execle path, arg0, argl, , env Equivalent to execve path, arg0,...
Address Families
Some of the socket functions require the specification of an address family.The family specifies the network protocol being used. The following constants are defined for this purpose Transparent Inter-Process Communication protocol UNIX domain protocols Of these, AF_INET and AF_INET6 are the most commonly used because they represent standard Internet connections. AF_BLUETOOTH is only available on systems that support it typically embedded systems . AF_NETLINK, AF_PACKET, and AF_TIPC are only...
Defining Customized Servers
Servers often need special configuration to account for different network address families, timeouts, concurrency, and other features. This customization is carried out by inheriting from one of the four basic servers described in the previous section. The following class attributes can be defined to customize basic settings of the underlying network socket The address family used by the server socket.The default value is socket.AF_INET. Use socket.AF_INET6 if you want to use IPv6. A Boolean...
Predefined Exception Classes
The following exceptions are raised by programs Failed attribute reference or assignment. EOFError End of file. Generated by the built-in functions input and raw_input . It should be noted that most other I O operations such as the read and readline methods of files return an empty string to signal EOF instead of raising an exception. Failed floating-point operation. It should be noted that floating-point exception-handling is a tricky problem and only that this exception only gets raised if...
Coroutines
Normally, functions operate on a single set of input arguments. However, a function can also be written to operate as a task that processes a sequence of inputs sent to it.This type of function is known as a coroutine and is created by using the yield statement as an expression yield as shown in this example print Looking for, matchtext while True line yield Get a line of text To use this function, you first call it, advance it to the first yield , and then start sending data to it using send ....
pdetach Executes the program and detaches from it The calling program continues
spawnv is available on Windows and some versions of UNIX. spawnve mode, path, args, env Executes the program path in a new process, passing the arguments specified in args as command-line parameters and the contents of the mapping env as the environment. args can be a list or a tuple. mode has the same meaning as described for spawnv . The same as spawnv except that all the arguments are supplied as extra parameters. spawnle mode, path, argl, , argn, env The same as spawnve except that the...
Distributing Python Programs and Libraries
To distribute Python programs to others, you should use the distutils module. As preparation, you should first cleanly organize your work into a directory that has a README file, supporting documentation, and your source code. Typically, this directory will contain a mix of library modules, packages, and scripts. Modules and packages refer to source files that will be loaded with import statements. Scripts are programs that will run as the main program to the interpreter e.g., running as python...
Gaierror Errno -3 Temporary Failure In Name Resolution
The following exceptions are defined by the socket module. This exception is raised for socket- or address-related errors. It returns a pair errno, mesg with the error returned by the underlying system call. Inherits from IOError. Error raised for address-related errors. Returns a tuple herrno, hmesg containing an error number and error message. Inherits from error. Error raised for address-related errors in the getaddrinfo and getnameinfo func-tions.The error value is a tuple errno, mesg ,...
mimetypes
The mimetypes module is used to guess the MIME type associated with a file, based on its filename extension. It also converts MIME types to their standard filename extensions. MIME types consist of a type subtype pair for example 'text html', 'image png', or 'audio mpeg'. Guesses the MIME type of a file based on its filename or URL. Returns a tuple type, encoding in which type is a string of the form type subtype and encoding is the program used to encode the data for transfer for example,...
Abstract Base Classes Hfl
The collections module defines a series of abstract base classes.The purpose of these classes is to describe programming interfaces on various kinds of containers such as lists, sets, and dictionaries. There are two primary uses of these classes. First, they can be used as a base class for user-defined objects that want to emulate the functionality of built-in container types. Second, they can be used for type checking. For example, if you wanted to check that s worked like a sequence, you...
Generators and yield 1
print Counting down from d n while n gt 0 yield n n - 1 return If you call this function, you will find that none of its code starts executing. For example Instead, a generator object is returned.The generator object, in turn, executes the function whenever next is called or__next__ in Python 3 . Here's an example gt gt gt c.next Use c._next_ in Python 3 When next is invoked, the generator function executes statements until it reaches a yield statement.The yield statement produces a result at...
Logging LevelMethod
CRITICAL log.critical fmt , args , exc_info , extra ERROR log.error fmt , args , exc_info , extra WARNING log.warning fmt , args , exc_info , extra INFO log.info fmt , args , exc_info , extra DEBUG log.debug fmt , args , exc_info , extra The fmt argument is a format string that specifies the format of the log message. Any remaining arguments in args serve as arguments for format specifiers in the format string. The string formatting operator is used to form the resulting message from these...
Managed Objects
Unlike threads, processes do not support shared objects. Although you can create shared values and arrays as shown in the previous section, this doesn't work for more advanced Python objects such as dictionaries, lists, or instances of user-defined classes.The multiprocessing module does, however, provide a way to work with shared objects if they run under the control of a so-called manager.A manager is a separate subprocess where the real objects exist and which operates as a server. Other...
Module Reloading and Unloading
Python provides no real support for reloading or unloading of previously imported modules. Although you can remove a module from sys.modules, this does not generally unload a module from memory.This is because references to the module object may still exist in other program components that used import to load that module. Moreover, if there are instances of classes defined in the module, those instances contain references back to their class object, which in turn holds references to the module...
Zip64 Exceed 2gb
if os.path.basename f.name README data t.extractfile f .read print s f.name The tempfile module is used to generate temporary filenames and files. mkdtemp suffix ,prefix , dir Creates a temporary directory accessible only by the owner of the calling process and returns its absolute pathname. suffix is an optional suffix that will be appended to the directory name, prefix is an optional prefix that will be inserted at the beginning of the directory name, and dir is a directory where the...
Declarative Programming
List comprehensions and generator expressions are strongly tied to operations found in declarative languages. In fact, the origin of these features is loosely derived from ideas in mathematical set theory. For example, when you write a statement such as x x for x in a if x gt 0 , it's somewhat similar to specifying a set such as x2 x a,x gt 0 . Instead of writing programs that manually iterate over data, you can use these declarative features to structure programs as a series of computations...
Basic Interpreter Operation and Setup
The following functions are used to set up the interpreter and to run scripts int PyRun_AnyFile FILE fp, char filename If fp is an interactive device such as tty in UNIX, this function calls PyRun_InteractiveLoop . Otherwise, PyRun_SimpleFile is called. filename is a string that gives a name for the input stream. This name will appear when the interpreter reports errors. If filename is NULL, a default string of is used as the file name. int PyRun_SimpleFile FILE fp, char filename Similar to...