Extending Python with Compiled Languages Extending with C C/C++ • Python native libraries • Basically a C shared libraries • Python working perfectly with C/C !++ shared libraries • C functions are needed to be wrapped with python - compatible functions using python C API Code typedef struct { int length; int * values; } Array; Array *SieveOfEratosthenes(int n) Add Python Support • Include Python.h • Wrap functions with python - compatible wrappers • Define array of methods • Define PyModuleDef • Create function PyInit_<name> that returns PyModuleDef Add Python Support #define PY_SSIZE_T_CLEAN #include <Python/Python.h> #if PY_MAJOR_VERSION >= 3 ... ... ... #endif Add Python Support static PyObject *CEratosphenes_SieveOfEratosphenes( PyObject * self, PyObject * args) Add Python Support if (!PyArg_ParseTuple(args, "I", &n)) { return NULL; } Add Python Support PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...); PyAPI_FUNC(int) PyArg_ParseTupleAndKeywords( PyObject *, PyObject *, const char *, char **, ...); https://docs.python.org/3/c-api/arg.html Add Python Support PyObject * result = PyList_New(0); ... PyList_Append(result, PyLong_FromLong(p)); Add Python Support static PyMethodDef Methods[] = { {“sieve_of_eratosphenes", CExtExample_SieveOfEratosphenes, METH_VARARGS, "Return list of prime numbers until given number."}, {NULL, NULL, 0, NULL} "/* Sentinel "*/ }; Add Python Support static PyModuleDef CExtExample = { PyModuleDef_HEAD_INIT, "CExtExample", "Example module written in C", -1, Methods}; Add Python Support PyObject *PyInit_CExtExample(void) { return PyModule_Create(&CExtExample); } Building from setuptools import setup, Extension extension = Extension('CExtExample', sources=['CExtExample.c']) setup(name='CExtExample', version='1.0', description='This is Example module written in C', ext_modules=[extension]) Building $ ~/p/E/C nm CExtExample.cpython-37m - darwin.so 0000000000001050 d _CExtExample 0000000000000dd0 t _CExtExample_SieveOfEratosphenes 00000000000010c0 d _Methods 0000000000000db0 T _PyInit_CExtExample U _PyList_Append U _PyList_New U _PyLong_FromLong U _PyModule_Create2 U %__ PyArg_ParseTuple_SizeT U _free U _malloc U dyld_stub_binder Memory Management • Do not use malloc/free on python objects • Use Py<StructName>_New instead • Python have it’s own allocator and own private heap • You can use malloc/free on c structures Pros & Cons • Freedom of GIL • Manual memory management • High speed Extending with Go Extending with Go Go • Compiling Go code to shared libraries • Using CGo • CGo !!= Go • Use Go shared library in C extension