Firststep changes
In order to use the compatibility layer there are still a few changes that need to be made to your code. Many of these changes can be made by running the alter_codel module with your code as input. 1. Importing the alter_codel module handles all these changes a import Numeric - gt import numpy.oldnumeric as Numeric b import Numeric as XX - gt import numpy.oldnumeric as XX c from Numeric import lt name1 gt , lt nameN gt - gt from numpy.oldnumeric import lt name1 gt , , lt nameN gt d from Numeric...
PyArrayMultiIterType
This type provides an iterator that encapsulates the concept of broadcasting. It allows N arrays to be broadcast together so that the loop progresses in C-style contiguous fashion over the broadcasted array. The corresponding C-structure is the PyArrayMultilterObj ect whose memory layout must begin any object, obj, passed in to the PyArray_Broadcast obj function. Broadcasting is performed by adjusting array iterators so that each iterator represents the broadcasted shape and size, but has its...
Registering a ufunc loop
You may also want to register low-level ufunc loops for your data-type so that an ndarray of your data-type can have math applied to it seamlessly. Registering a new loop with exactly the same argjypes signature, silently replaces any previously registered loops for that data-type. Before you can register a 1-d loop for a ufunc, the ufunc must be previously created. Then you call PyUFunc_RegisterLoopForType with the information needed for the loop. The return value of this function is 0 if the...
Array flags Basic Array Flags
An ndarray can have a data segment that is not a simple contiguous chunk of well-behaved memory you can manipulate. It may not be aligned with word boundaries very important on some platforms . It might have its data in a different byteorder than the machine recognizes. It might not be writeable. It might be in Fortan-contiguous order. The array flags are used to indicate what can be said about data associated with an array. NPY_C_CONTIGUOUS The data area is in G-style contiguous order last...
Creating a brandnew ndarray
Quite often new arrays must be created from within extension-module code. Perhaps an output array is needed and you don't want the caller to have to supply it. Perhaps only a temporary array is needed to hold an intermediate calculation. Whatever the need there are simple ways to get an ndarray object of whatever datatype is needed. The most general function for doing this is PyArray_NewFromDescr. All array creation functions go through this heavily re-used code. Because of its flexibility, it...
Inline Ccode
Probably the most widely-used method of employing weave is to in-line C C code into Python in order to speed up a time-critical section of Python code. In this method of using weave, you define a string containing useful C-code and then pass it to the function weave.inline code_string, variables , where codejstring is a string of valid C C code and variables is a list of variables that should be passed in from Python. The C C code should refer to the variables with the same names as they are...
Noncontiguous memory layout
Both of the examples presented above are single-segment arrays where the entire array is visited by sequentially marching through memory one element at a time. When an algorithm in C or Fortran expects an N-dimensional array, this single segment of a certain fundamental type is usually what is expected along with the shape N-tuple. With a single-segment of memory representing the array, the one-dimensional index into computer memory can always be computed from the N-dimensional index. This...
Conversion of src files
NumPy distutils supports automatic conversion of source files named lt some-file gt .src. This facility can be used to maintain very similar code blocks requiring only simple changes between blocks. During the build phase of setup, if a template file named lt somefile gt .src is encountered, a new file named lt somefile gt is constructed from the template and placed in the build directory to be used instead. Two forms of template conversion are supported. The first form occurs for files named...
Advanced Fancy Indexing
The implementation of advanced indexing represents some of the most difficult code to write and explain. In fact, there are two implementations of advanced indexing. The first works only with 1-d arrays and is implemented to handle expressions involving a.flat obj . The second is general-purpose that works for arrays of arbitrary dimension up to a fixed maximum . The one-dimensional indexing approaches were implemented in a rather straightforward fashion, and so it is the general-purpose...
Reduce
lt op gt .reduce array , axis 0, dtype None For each one-dimensional sequence along the axis dimension of the array, return a single number resulting from recursively applying the operation to succesive elements along that dimension. If the input array has N dimensions, then the returned array has N 1 dimensions. This produces the equivalent of the following Python code gt gt gt indx index_exp array.ndim gt gt gt indx axis 0 N array.shape axis gt gt gt result array indx .astype dtype gt gt gt...
Trigonometric functions
All trigonometric functions use radians when an angle is called for. The ratio of degrees to radians is 180 n. sin x , y cos x , y tan x , y The standard trignometric functions. y sin x , y cos x , and y tan x . arcsin x , y arccos x , y arctan x , y The inverse trigonometric functions y sin-1 x , y cos-1 x , y tan-1 x . These return the value of y in radians such that sin y x with y G f cos y x with y 0,7r and tan y x with y f, f , respectively. Returns tan-1 f but takes into account the sign...
Loading the shared library
A simple, but robust way to load the shared library is to get the absolute path name and load it using the cdll object of ctypes. lib ctypes . cdll lt full_path_name gt However, on Windows accessing an attribute of the cdll method will load the first DLL by that name found in the current directory or on the PATH. Loading the absolute path name requires a little finesse for cross-platform work since the extension of shared libraries varies. There is a ctypes .util. f ind_library utility...
Registering coercion rules
By default, all user-defined data-types are not presumed to be safely castable to any builtin data-types. In addition builtin data-types are not presumed to be safely castable to user-defined data-types. This situation limits the ability of user-defined data-types to participate in the coercion system used by ufuncs and other times when automatic coercion takes place in NumPy. This can be changed by registering data-types as safely castable from a particlar data-type object. The function...
Error handling
Universal functions can trip special floating point status registers in your hardware such as divide-by-zero . If available on your platform, these registers will be regularly checked during calculation. The user can determine what should be done if errors are encountered. Error handling is controlled on a per-thread basis. Four errors can be individually configured divide-by-zero, overflow, underflow, and invalid. The errors can each be set to ignore, warn, raise, or call. The easiest way to...
Set Operations
The set operations were kindly contributed by Robert Cimrman. These set operations are based on sorting functions and all expect 1-d sequences with unique elements with the exception of uniqueld and intersect Id _nu which will flatten N-d nested-sequences to 1-d arrays and can handle non-unique elements. Return the unique elements of arr as a 1-d array. If retindx is True, then also return the indices, ind, such that arr.flat ind is the set of unique values. Return the sorted intersection of a1...
Contents
2.1 Data-Type 2.2 Basic indexing 2.3 Memory Layout of 2.3.1 Contiguous Memory 2.3.2 Non-contiguous memory 2.4 Universal Functions for 2.5 Summary of new 2.6 Summary of differences with Numeric 2.6.1 First-step 2.6.2 Second-step 2.6.3 Updating code that uses Numeric using alter_codeN 38 2.6.4 Changes to think 2.7 Summary of differences with Numarray 40 2.7.1 First-step changes 2.7.1.1 Import 2.7.1.2 Attribute and method changes 42 2.7.2 Second-step changes 2.7.3 Additional Extension modules 3.1...
Simplify creation of an extension module
The inline function creates one extension module for each function to-be inlined. It also generates a lot of intermediate code that is duplicated for each extension module. If you have several related codes to execute in C, it would be better to make them all separate functions in a single extension module with multiple functions. You can also use the tools weave provides to produce this larger extension module. In fact, the weave.inline function just uses these more general tools to do its...
Contiguous Memory Layout
There is a fundamental ambiguity in how the mapping to a one-dimensional index can take place which is illustrated for a 2-dimensional array in Figure 2.3. In that figure, each block represents a chunk of memory that is needed for representing the underlying array element. For example, each block could represent the 8 bytes needed to represent a double-precision floating point number. In the figure, two arrays are shown, a 4x3 array and a 3x4 array. Each of these arrays takes 12 blocks of...
zeroslike arr
Syntactic sugar for zeros a.shape, a.dtype, isfortran arr ones shape , dtype int, order 'C' Syntactic sugar for a zeros shape, dtype, order a 1. fromstring string ,dtype int, count -1, sep If sep is , then return a new 1-d array with data-type descriptor given by dtype and with memory initialized copied from the raw binary data in string. If count is non-negative, the new array will have count elements with a ValueError raised if count requires more data than the string offers , otherwise the...
Array conversion tolist
The contents of self as a nested list. gt gt gt a array 1,2,3 , 4,5,6 print a.tolist 1, 2, 3 , 4, 5, 6 If no arguments are passed in, then this method only works for arrays with one element a.size 1 . In this case, it returns a standard Python scalar object if possible copied from the first element of self. When the data type of self is longdouble or clongdouble, this returns a scalar array object because there is no available Python scalar that would not lose information. Void arrays return a...
Other indexing devices
Return a tuple of Python objects that implements the index expression and can be modified and placed in any other index expression. gt gt gt index.exp 2 5, . . . ,4, -1 slice 2, 5, None , Ellipsis, 4, slice None, None, -1 Translate index expressions into the equivalent Python objects. This is similar to index_expression except a tuple is not always returned. For example gt gt gt S_ 1 10 slice 1, 10, None gt gt gt s_ 1 10 , -3 4 0.5 slice 1, 10, None , slice -3, 4, 0.5 This provides a standard...
NumPy Distutils
NumPy provides enhanced distutils functionality to make it easier to build and install sub-packages, auto-generate code, and extension modules that use Fortran-compiled libraries. To use features of numpy distutils use the setup command from numpy.distutils.core. A useful Configuration class is also provided in numpy.distutils.misc.util that can make it easier to construct keyword arguments to pass to the setup function by passing the dictionary obtained from the todict method of the class ....
Fancyindexing implementation
The concept of indexing was also abstracted using the idea of an iterator. If fancy indexing is performed, then a PyArrayMapIterObject is created. This internal object is not exposed to Python. It is created in order to handle the fancy-indexing at a high-level. Both get and set fancy-indexing operations are implemented using this object. Fancy indexing is abstracted into three separate operations 1 creating the PyArrayMapIterObject from the indexing object, 2 binding the PyArrayMapIterObject...
Fancyindexing check
The fancy_indexing_check routine determines whether or not to use standard view-based indexing or new copy-based indexing. If the indexing object is a tuple, then view-based indexing is assumed by default. Only if the tuple contains an array object or a sequence object is fancy-indexing assumed. If the indexing object is an array, then fancy indexing is automatically assumed. If the indexing object is any other kind of sequence, then fancy-indexing is assumed by default. This is over-ridden to...
UnSigned Integer
Unsigned versions of the integers can be defined by pre-pending a 'u' to the front of the integer name. npy_ u byte unsigned char npy_ u short unsigned short npy_ u int unsigned int npy_ u long unsigned long int npy_ u longlong unsigned long long int npy_ u intp unsigned Pyjntptr.t an integer that is the size of a pointer on the platform . 13.2.3.3 Complex Floating point npy_ c float float npy_ c double double npy_ c longdouble long double complex types are structures with .real and .imag...
Shape functions
Force a sequence of arrays including array scalars to each be at least 1-d. Force a sequence of arrays including array scalars to each be at least 2-d. Dimensions of length 1 are pre-pended to reach a two-dimensional array. Force a sequence of arrays including array .scalars to each be at least 3-d. Dimensions of length 1 are pre-pended to reach a two-dimensional array. Return a new array with the contents of arr shifted rolled by the amount given in the integer argument shift along the axis...
Linear Algebra linalg
These functions are in the numpy.linalg sub-package. inv A Table 10.1 Functions in numpy.dual both in NumPy and SciPy Table 10.1 Functions in numpy.dual both in NumPy and SciPy norm, det, inv, pinv, solve, eig, eigh, eigvals, eigvalsh, lstsq, cholesky, svd Return the matrix inverse of the 2-d array A. The result, X, is such that dot A,X is equal to eye A.shape to within machine precision . Find the solution to the linear equation Ax b, where A is a 2-d array and b is a 1-d or 2-d array. Find...
Required subroutine
There is exactly one function that must be defined in your C-code in order for Python to use it as an extension module. The function must be called init lt name gt where lt name gt is the name of the module from Python. This function must be declared so that it is visible to code outside of the routine. Besides adding the methods and constants you desire, this subroutine must also contain calls to import_array and or import_ufunc depending on which C-API is needed. Forgetting to place these...
Dealing with types
13.3.3.1 General check of Python Type Evaluates true if op is a Python object whose type is a sub-type of PyArray_Type. Evaluates true if op is a Python object with type PyArray_Type. If op implements any part of the array interface, then out will contain a new reference to the newly created ndarray using the interface or out will contain NULL if an error during conversion occurs. Otherwise, out will contain a borrowed reference to Py_NotImplemented and no error condition is set....
Basic indexing slicing
Indexing is a powerful tool in Python and NumPy takes full advantage of this power. In fact, some of capabilities of Python's indexing were first established by the needs of Numeric users.2 Indexing is also sometimes called slicing in Python, and slicing for an ndarray works very similarly as it does for other Python sequences. There are three big differences 1 slicing can be done over multiple dimensions, 2 exactly one ellipsis object can be used to indicate several dimensions at once, 3...
Miscellaneous Functions
Some miscellaneous functions are available in NumPy which are included largely for compatibility with MLab of the old Numeric package. One notable difference, however, is that due to a separate implementation of the modified Bessel function, the kaiser window is available without needing a separate library. Compute the sinc function for x which can be a scalar or array. The sinc is defined as y sinc x with the caveat that the limiting value 1.0 of the Modified Bessel function of the first kind...
flatnonzero arr
Return indices that are non-zero in a flattened version of arr. Equivalent to a. ravel . nonzero 0 . for n, obj in enumerate mask.flat if obj The values array is repeated if it is too short. In particular, this means that indexing on the values array is modular it's length, which might be surprising you are expecting putmask to work the same as arr mask values. Return an array of indices similar to argsort except sorting is done using all of the provided keys. First a sort is computed using key...
dsplit
Split a single array into multiple sub-arrays along the third axis depth . Only works on arrays of 3 or more dimensions. apply_along_axis funcld, axis, arr, args Execute funcld arr selJ , args where funcld takes 1-d arrays and arr is an N-d array, where sel j is a selection object sufficient to select a 1-d sub-array along the given axis. The function is executed for all 1-d arrays along axis in arr. For each axis in the axes sequence, call func as res func a, axis . If res is the same shape as...
Accumulate
lt op gt .accumulate array , axis 0, dtype None This method is similar to reduce, except it returns an array of the same shape as the input, and keeps intermediate calculations. The operation is still performed along the access. This method underlies the operations of the cumsum and cumprod methods of arrays. The following Python code implements an equivalent of the accumulate method. gt gt gt il index_exp array.ndim gt gt gt i2 index_exp array.ndim gt gt gt i1 axis 0 N array.shape axis gt gt...
Broadcasting
Each universal function takes array inputs and produces array outputs by performing the core function element-wise on the inputs. The standard broadcasting rules are applied so that inputs without exactly the same shapes can still be usefully operated on. Broadcasting can be understood by four rules 1. All input arrays with ndim smaller than the input array of largest ndim have 1's pre-pended to their shapes. 2. The size in each dimension of the output shape is the maximum of all the input...
Py Array ISBYTESWAPPED m
Evaluates true if the data area of the ndarray m is not in machine byte-order according to the array's data-type descriptor. PyArray_EquivTypes Bool PyArrayJDescr typel, PyArrayJDescr type2 Return NPY_TRUE if typel and type2 actually represent equivalent types for this platform the fortran member of each type is ignored . For example, on 32-bit platforms, NPY_LONG and NPY_INT are equivalent. Otherwise return NPY_FALSE. PyArray_EquivArrTypes Bool PyArrayObj ect al, PyArrayObj ect a2 Return...
median m
Returns the median of m along its first dimension. The list argument is a 1-d integer array. Let r be the returned 1-d array whose length is list.max 1 . If weights is None, then r i is the number of occurrences of i in list. If weight is present, then the th element is Notice that if weights is None, it is equivalent to a weights array of all 1. The length of weights must be the same as the length of list. Return an array of integers the same length as x with values i such that bins i 1 lt x...
Converting an arbitrary sequence object
The main routine for obtaining an array from any Python object that can be converted to an array is PyArray_FromAny. This function is very flexible with many input arguments. Several macros make it easier to use the basic function. PyArray_FROM_OTF is arguably the most useful of these macros for the most common uses. It allows you to convert an arbitrary Python object to an array of a specific builtin data-type e.g. float , while specifying a particular set of requirements e.g. contiguous,...
Array construction using index tricks
The functions and classes in this category make it simpler to construct arrays. This indexing cross function is useful for forming indexing arrays necessary to select out the cross-product of N 1-dimensional arrays. Note that the default indexing does not do a cross-product which might be unexpected for someone coming from other programming environments . The default indexing is more general purpose. Using the ix_ constructor can produce the indexing arrays necessary to select a cross-product....
WARNING Rrb
Data-type objects must be reference counted so be aware of the action on the data-type reference of different G-API calls. The Rule is that when a data-type descriptor object is returned it is a new reference. Functions that take PyArray_Descr objects and return arrays steal references to their inputs unless otherwise noted. Unless you just created the data-type object you must usually increase the reference count of an object passed in describing the data-type. PyArrayDescr_Check int PyObject...
Array item selection and manipulation
For array methods that take an axis keyword, it defaults to None. If axis is None, then the array is treated as a 1-D array. Any other value for axis represents the dimension along which the operation should proceed. take indices , axis None, out None, mode 'raise' The functionality of this method is available using the advanced indexing ability of the ndarray object. However, for doing selection along a single axis it is usually faster to use take. If axis is not None, this method is...
Enumerated Types
There is a list of enumerated types defined providing the basic 21 data types plus some useful generic names. Whenever the code requires a type number, one of these enumerated types is requested. The types are all called NPY_ lt NAME gt where lt NAME gt can be BOOL, BYTE, UBYTE, SHORT, USHORT, INT, UINT, LONG, ULONG, LONGLONG, ULONGLONG, FLOAT, DOUBLE, LONGDOUBLE, CFLOAT, CDOUBLE, CLONG-DOUBLE, OBJECT, STRING, UNICODE, VOID NTYPES, NOTYPE, USERDEF, DEFAULT_TYPE The various character codes...
PyArrayTypeObjectFromType PyObject int type
Returns a scalar type-object from a type-number, type. Equivalent to except for reference counting and error-checking. Returns a new reference to the typeobject on success or NULL on failure. PyArray_ScalarKind NPY.SCALARKIND int typenum, PyArrayObj ect arr Return the kind of scalar represented by typenum and the array in arr if arr is not NULL . The array is assumed to be rank-0 and only used if typenum represents a signed integer. If arr is not NULL and the first element is negative then...
Reduceat
lt op gt .reduceat array , indices , axis 0, dtype None This method is a generalization of both reduce and accumulate. It offers the ability to reduce along an axis but only between certain indices. The indices input must be a one dimensional index sequence. Then, if Ik is the kth element of indices, the reduceat method computes lt op gt .reduce array Ik Ik 1 . This formula assumes Ik 1 gt Ik, and also that Ik 1 is the length of the input array when Ik is the last element. There is no...
PyArrayDescrType
The PyArrayDescr_Type is the built-in type of the data-type-descriptor objects used to describe how the bytes comprising the array are to be interpreted. There are 21 statically-defined PyArray_Descr objects for the built-in data-types. While these participate in reference counting, their reference count should never reach zero. There is also a dynamic table of user-defined PyArray_Descr objects that is also maintained. Once a data-type-descriptor object is registered it should never be...
Continuous Distributions
Continuous random numbers can take on an uncountable number of values. Therefore, the value returned by a continuous distribution is denoted x. Because there is an uncountable number of possibilities for the random number1, a continuous distribution is modeled by a probability density function, f x . To obtain the probability that the random number generated by X is in a certain interval, we integrate this density function I f x dx Probability X lt b . To obtain a probability, we have to...
Creating arrays 1
PyArray_NewFromDescr PyObject PyTypeObj ect subtype, PyArray_Descr descr, int nd, npy_intp dims, npy_intp strides, void data, int flags, PyObject obj This is the main array creation function. Most new arrays are created with this flexible function. The returned object is an object of Python-type subtype, which must be a subtype of PyArray-Type. The array has nd dimensions, described by dims. The data-type descriptor of the new array is descr. If subtype is not amp PyArray_Type e.g. a Python...
miscutil
Configuration package_name None, parent_name None, top_path None, package_path None, attrs Construct a configuration instance for the given package name. If parent_name is not None, then construct the package as a sub-package of the parent_name package. If top_path and package_path are None then they are assumed equal to the path of the file this instance was created in. The setup.py files in the numpy distribution are good examples of how to use the Configuration instance. Return a dictionary...
flipud m
Return the array, m, with columns preserved and rows reversed in the up-down direction. For m.ndim gt 1, this works on the first dimension equivalent to m -1 Rotate the first two dimensions of an array, m, by k 90 degrees in the counterclockwise direction. Must have m.ndim gt 2. Construct an N x M array where all the diagonals starting from the lower left corner up to the kth diagonal are all ones. Return a upper-triangular 2-d array from m with all the elements below the kth diagonal set to 0....
nantonum arr
Returns an array with non-finite numbers changed to finite numbers. The mapping converts nan to 0, inf to the maximum value for the data type and -inf to the minimum value for the data type. Return a real arr if arr is complex with imaginary parts less than some tolerance. If tol gt 1, then it represents a multiplicative factor on the value of epsilon for the data type of arr. Cast obj to an array of the given type. This is equivalent to array obj, copy 0 .astype dtype_or_alias . When one type...