Brainfold On Python, game development and everything

Surprising results

I did some preliminary benchmarking today and got very interesting results. I have only wrapped a 3D vector class so far, so I tested a couple of operations (add, multiply, dot & cross product) against the libs I mentioned in the previous post: NumPy, euclid, vectypes and cgkit. For cgkit, I tested both 1.2.0 and 2.0 alpha 9. All other libs were the latest version. I tested using the Python timeit module with 1,000,000 calls and 3 repeats per test and took the lowest number. Repeated test results generally differed only by milliseconds.

Benchmark results

The immediately obvious surprise is the abysmal performance of NumPy especially in cross products. I don't think NumPy optimizes for fixed-size arrays; I would have been better off with the pure-Python euclid and vectypes modules.

The other surprises were euclid vs vectypes and cgkit1 vs cgkit2. Euclid and vectypes are both by Alex Holkner of Pyglet fame. In both cases, the newer library (vectypes and cgkit2) was also slower.

Of course the most positive surprise for me was the performance of PyEigen. It's only a trivial wrapping, but was about 2-8x faster than the best alternative, cgkit1. I'm very happy with the results and definitely going to continue development of the library / wrapper.

Update: Here's the benchmark code. Also an interesting detail: PyEigen is 136x faster than NumPy with cross products.