cython

What it is: Cython is a superset of Python—every Python program is a valid cython program. C variable types and other C features can be used to speed up programs, often 50x or more faster than the original Python. Cython programs are converted into C and compiled.

FOSDEM 2018 – Lift your Speed Limits with Cython – Fast native code for Python

The speaker presents a horrible chimera of a programming language, wherein the drawbacks and limitations of Python are augmented by the drawbacks and limitations of C. The result is a language that introduces header files to Python and requires breathtaking amounts of boilerplate. The primary goal of Cython appears to be transforming the programming experience from “implementing a solution to a given problem” to “trying to guess when to turn off exception handling so that your code runs marginally faster.” – n-gate.com

I’m a total beginner. Am just putting here things I’m learning.. I’m used to C++ and recently started programming in Python. Now Cython (from Sage/IPython from the Mac bash shell)

e.g. see my six knights program.

Great article by W Stein on running cython progs from command line

bash$ sage -cython myfile.pyx

(that creates myfile.c)

bash$ sage -sh
sage-sh$ gcc -I$SAGE_ROOT/local/include/python2.7 -shared -fPIC myfile.c -o myfile.so
sage-sh$ exit

(to get back to bash)
(or whatever version of python your Sage came with. The article says 2.6 – I have 2.7)
(that creates myfile.so)
then:

$ sage
sage: import myfile
sage: myfile.myfunc(args)

(or however your program’s functions work)


Size of integer data types
from stdint.pxd:

# 7.18.1.1 Exact-width integer types

ctypedef signed char int8_t
ctypedef signed short int16_t
ctypedef signed int int32_t
ctypedef signed long int64_t
ctypedef unsigned char uint8_t
ctypedef unsigned short uint16_t
ctypedef unsigned int uint32_t
ctypedef unsigned long uint64_t

Global
Error: local variable ‘j’ referenced before assignment
Global variables written to in functions must be declared as ‘global’ in the top of the function – with ‘global var‘ – otherwise it is assumed u are talking about a new local variable with the same name. A Python object. Grr. 😛 This is Python also.

Very informative page: https://github.com/cython/cython/wiki/FAQ

with Sage terminal/IPython, the error messages are much less helpful than in Sage notebook cython. Using the debugger on a compilation error:
%debug – enters the debugger
h – help. lists commands
l – lists the program around where the program stopped.

ARGHH just spent an hour wondering about this error message:

RuntimeError: Error compiling /Users/admin/.sage/sage_notebook.sagenb/home/admin/32/code/sage27.spyx:
running build
running build_ext
..
..
building 0: note: each undeclared identifier is reported only once for each function it appears in
error: command ‘gcc’ failed with exit status 1

etc etc etc. Mysterious. I had two numpy arrays declared identically.. but cython seemed to have a problem with one…. Eventually I realized that the problem was solved by loading a pic into the array. Doing nothing with it causes this very unhelpful error message. And that was from Sage – the error message in IPython was just the same. 😛 I think calling a variable ‘undeclared’ in spite of the line “cdef np.ndarray[np.uint8_t, ndim=3] g” is a bit odd.

Important page on compiling cython. setting compiler directives etc. essential.

****do Section on getting ndarray fully C-ed.

NUMPY + C ARRAYS – COPYING, INITIALIZING ETC
http://stackoverflow.com/questions/29036068/how-to-expose-a-numpy-array-from-c-array-in-cython

CIMPORTING

How do I declare numeric or integer C types?
Answer: In most cases, you don’t need to. For types declared in stdint.h, just cimport them from libc.stdint which comes with Cython, e.g.

from libc.stdint cimport uint32_t, int64_t
cdef int64_t i = 5

How do I declare an object of type bool?
Well, that depends on whether you want the C99/C++ bool or the Python bool. Previously, Cython always defaulted to the Python bool type, which led to hard-to-debug issues when users unsuspectingly used bool in wrapping C++ code. We decided to make the choice explicit — you can import whichever you’d like:
• For the Python type, do from cpython cimport bool.
• For the C++ type, do from libcpp cimport bool.
Note that there is also a type called bint, which is essentially a C int but automatically coerces from and to a Python bool value, i.e. cdef object x = some_c_integer_value gives either True or False.

Since Cython 0.18, you can just use ‘const’ in your code and in your declarations.


To get the yellow-text cython html speed-test file, and .c file, in the current folder from sage terminal:

sage: import subprocess
sage: subprocess.call([“cython”,”-a”,”my-cython-file.pyx”])


If you use the @cython.cdivision(True) decorator, cython will not add exception checking for division by zero, this allows the division statement to be converted to one line of pure C code
( https://ask.sagemath.org/question/8088/convert-from-int-to-double-and-back-in-cython/ )


Use xrange instead of range!! it’s faster because it only makes the list of numbers when needed. (I’m using Python 2.7 – apparently in Python 3, range now is always (the old) xrange, so there’s no xrange command any more.)

==============================================
bint – boolean variable type

use cdef blocks e.g.

cdef:
	int a=1
	int m[3][3]
	bint b

define constants with DEF:
DEF Numb = 17

don’t C-define all types! doesn’t always make it faster. Important in heavily used loops to have typed variables etc.

use %time (for whole programs/functions) or %timeit for testing for speed. %prun tells how long each function call/part takes.

for..in range()” loops are C optimized if the index variable has been declared by cdef. Also iteration over C arrays and sliced pointers:

cdef double* data = ... 
for x in data[:10]: 

undeclared types are assumed to be Python objects.

Cast to a ~Python object like:

<object>100000

there’s no -> operator in Cython. Use p.x instead of p->x

Pointers: instead of *p, use p[0]
&” works as in C.

The name “object” can also be used to explicitly declare something as a Python object. This can be useful if the name being declared would otherwise be taken as the name of a type, for example,:
cdef ftang(object int):

type casting: “<" + ">“:

cdef char *p, float *q
p= <char>q

to get the address, cast to <void*>

• There are three types of function declarations in Cython as the sub-sections show below.
• Only “Python” functions can be called outside a Cython module from Python interpreted code.
2.4.1 Callable from Python (def)
• Are declared with the def statement
• Are called with Python objects
• Return Python objects
• See Parameters for special consideration
2.4.2 Callable from C (cdef)
• Are declared with the cdef statement.
• Are called with either Python objects or C values.
• Can return either Python objects or C values.
2.4.3 Callable from both Python and C (cpdef)
• Are declared with the cpdef statement.
• Can be called from anywhere, because it uses a little Cython magic.
• Uses the faster C calling conventions when being called from other Cython code.

NB!!!!
• Normal Python as well as extension type classes can be defined.
• Extension types:
• Are considered by Python as “built-in” types.
• Can be used to wrap arbitrary C-data structures, and provide a Python-like interface to them from Python.
• Attributes and methods can be called from Python or Cython code
• Are defined by the cdef class statement.

cdef class Shrubbery: 
cdef int width, height 
def __init__(self, w, h): 
self.width = w 
self.height = h 
def describe(self): 
print "This shrubbery is", self.width, "by", self.height, "cubits."