Embeding Python & Extending Python with FFPython

Introduction

ffpython is a C++ lib, which is to simplify tasks that embed Python and extend Python. As the author, I am a developer for MMO server. Mainly I use C++ to implement part that needs to response user's requests in realtime, while other logic part that needs to be modified frequently is implemented by Python. Python makes it possible to reload our part of the server when the server is running. Python is so easy that even my colleague with no programing skills can implement some npc script. When I was first in charge of integrating Python runtime interpreter to our C++ server, I used Boost.python. Somehow Boost.python helped me solve most problems about Python API especially parts of Python reference. But Boost.python is not perfect.

• When exception happens while invoking python function by boost.python, boost.python doesn't provide interface to fetch traceback information.
• Boost python is a big lib to some extent. Someone will find it much difficult who just wants to try to experience how embedding Python works.
• Boost.pyhon supports badly to extend python by using C++ static function and class in runtime. Boost.python recommends to use dynamic library to extending python.
• Someone who wants to learn example code using Python API will find it difficult to understand well boost.python codes, in my opinion.
• If you have the need to convert data between C++ STL container and python object builtin, you have to implement these codes yourself, while Python reference is so annoying.

FFPython

Firstly, I implemented it just for converting data between C++ STL container and python object. Some of my colleagues have writes a lot of code using Python API directly. But there is some trap in some Python API even engineers who have a lot of experience using Python may have lost. PyDict_SetItem will auto increase reference key and value arguments, but other Python builtin structure API like PyTuple_SetItem will not increase reference the argument. It will cause memory leak. So I wanted to wrap operations about converting data between C++ STL container and Python builtin structure. Finally, I found some elegant ways to wrap embedding Python and extending Python by C++ template skills. So that is how FFPyhton was born.

Embedding Python

Sometimes I think it's easier to show codes to readers here. ^_^. But it will be forbidden by administrator if I post lots of code. O(n_n)O~. So see code files or Github.

When Embedding Python, such functions are most needed.

• Fetch global variable of python script(or module). This happens when you use Python script as config files.
• Call python function in script. That is the most useful interface. Two parts of it are important.
  1. It should be supported to use C++ builtin type and C++ STL container as argument.
  2. It should be supported to convert returned value of Python builtin types (like list, tuple, dict, string...) to C++ builtin types.
• Fetch exception information when exception happens, especially traceback information. Because of feature of dynamic type, exception regularly happens even in online server, let alone debugging time. ffpython will throw std exception when Python exception happens. So it is much easier to print or log traceback info by outputexception.what() .
• ffpython support nine arguments.
• ffpython implemented by C++ template to wrap Python API. It is easy to understand how it works if you see the code.

 printf("sys.version=%s\n", 
ffpython.get_global_var<string>("sys", "version").c_str()); 

int a1 = 100; float a2 = 3.14f; string a3 = "OhWell";
ffpython.call<void>("fftest", "test_base", a1, a2, a3);
vector<int> a1;a1.push_back(100);a1.push_back(200);
list<string> a2; a2.push_back("Oh");a2.push_back("Nice");
vector<list<string> > a3;a3.push_back(a2);
ffpython.call<bool>("fftest", "test_stl", a1, a2, a3);
typedef map<string, list<vector<int> > > ret_t;
ret_t val = ffpython.call<ret_t>("fftest", "test_return_stl");

Extending Python

ffpython recommends to register C++ function/class in runtime. It works to design and develop MMO game server. So that is common use for me. There are some key points when embedding Python.

• ffpython supports to register C++ static function. C++ builtin types and STL container can be as arguments.
• C++ class can be registered to Python.

Register C++ static function, all base type supported. Arg num can be nine.

static int print_val(int a1, float a2, const string& a3, const vector<double>& a4)
{
    printf("%s[%d,%f,%s,%d]\n", __FUNCTION__, a1, a2, a3.c_str(), a4.size());
    return 0;
}

ffpython_t ffpython;//("ext1");
ffpython.reg(&print_val, "print_val");
ffpython.init("ext1"); 

Register C++ class, Python can use it just like builtin types.

class foo_t{

public:
    foo_t(int v_):m_value(v_){
    printf("%s\n", __FUNCTION__);
    }
    virtual ~foo_t(){
        printf("%s\n", __FUNCTION__);
    } 
    int get_value() const { return m_value; }
    void set_value(int v_) { m_value = v_; }
    void test_stl(map<string, list<int> >& v_) 
    {
        printf("%s\n", __FUNCTION__);
    }
    int m_value;
};
class dumy_t: public foo_t
{
public:
    dumy_t(int v_):foo_t(v_)
    {
        printf("%s\n", __FUNCTION__);
    }
    ~dumy_t()
    {
        printf("%s\n", __FUNCTION__);
    }
    void dump() 
    {
        printf("%s\n", __FUNCTION__);
    }
};
static foo_t* obj_test(dumy_t* p)
{
    printf("%s\n", __FUNCTION__);
    return p;
}
void test_register_base_class(ffpython_t& ffpython)
{
    ffpython.reg_class<foo_t, PYCTOR(int)>("foo_t")
            .reg(&foo_t::get_value, "get_value")
            .reg(&foo_t::set_value, "set_value")
            .reg(&foo_t::test_stl, "test_stl")
            .reg_property(&foo_t::m_value, "m_value");
};   

Summary

• ffpython only one implements head file, it is easy to integrate to project.
• ffpython is simply wrap for Python API, so it is efficient.
• github: https://github.com/fanchy/ffpython
• python2.5 python2.6 python2.7, win / linux
• python3.x is being developed, but unfortunately, Python3.x API is so different to python2.x, even different between python3.2 and python3.3, Headache!!