I’ve been working a lot with Cython recently. Cython is a programming language based on Pyrex, which gets translated and optimized into C++. You can use it to create optimized C++ extensions using a Python-like language or to glue existing C/C++ libraries with your Python codebase. It’s possible to expose objects both ways. Cython files are similar to C/C++ files in that there are header files (.pyd) and implementation files (.pxd). These are some collected random notes.

  • All wrappers of C++ classes (using cdef extern from) must be declared in the .pyd files
  • Cython comes with two main commands. cython command takes a .pyx file and produces a C/C++ file, while the cythonize command takes a glob of .pyx files and produces the resulting shared library (.so file)
  • There’s currently no way to tell CLion to use cython from a specific virtualenv. The only way around this is to run cmake on the command line with your virtual environment active - if you then don’t reset the cache you can keep running cmake in CLion as well and it will work.
  • There is a Cython plugin for cmake (used by scikit-build, can be found here). This plugin allows you to create your extensions right from cmake (it calls the cython/cythonize commands under the hood), but it’s often much easier to let distutils/setuptools do it.
  • There is no good way to do incremental compilation of Cython code, because there is no way to compile multiple Cython files and link them into a single module. You can use Cython’s include directive to smoosh the Cython files into one and then build a module from that, but then you lose incremental compilation on the Cython code. If you have C++ code, you can still get incremental compilation on that part, if you use something like cmake. Your pure C++ code gets compiled into a shared library which then is used in your Cython extension. Alternatively, you split up your Cython code into several shared libraries, but then you get some code duplication as each library holds the same Python init code.