Use PEP 757 API for fmpz <-> int conversions

[skirpichev]

Closes #159

This lacks implementation for Python < 3.14. Not sure how to do this better, conditional compilation seems to be deprecated by Cython.

To my surprise, #324 approach seems to be better. Here my benchmarks.

Export

Benchmark	ref	pr324	patch
1<<7	166 ns	169 ns: 1.02x slower	169 ns: 1.02x slower
1<<38	216 ns	not significant	219 ns: 1.01x slower
1<<300	3.06 us	923 ns: 3.31x faster	787 ns: 3.88x faster
1<<3000	15.9 us	2.17 us: 7.34x faster	3.70 us: 4.29x faster
1<<10000	49.4 us	5.93 us: 8.33x faster	11.6 us: 4.27x faster
Geometric mean	(ref)	2.88x faster	2.33x faster

Import

Benchmark	ref	pr324	patch
1<<7	454 ns	451 ns: 1.01x faster	459 ns: 1.01x slower
1<<38	471 ns	466 ns: 1.01x faster	475 ns: 1.01x slower
1<<300	4.80 us	1.91 us: 2.52x faster	1.10 us: 4.37x faster
1<<3000	23.6 us	3.07 us: 7.68x faster	4.17 us: 5.65x faster
1<<10000	73.0 us	6.78 us: 10.76x faster	12.5 us: 5.86x faster
Geometric mean	(ref)	2.92x faster	2.69x faster

Details

# bench-export.py

import os

import pyperf

_T = os.getenv('_T')
if _T == "gmpy2.mpz":
    from gmpy2 import mpz
elif _T == "gmp.mpz":
    from gmp import mpz
else:
    from flint import fmpz as mpz

cases = ['1<<7', '1<<38', '1<<300', '1<<3000', '1<<10000']
runner = pyperf.Runner()
for c in cases:
    i = eval(c)
    m = mpz(i)
    runner.bench_func(c, int, m)

# bench-import.py

import os

import pyperf

_T = os.getenv('_T')
if _T == "gmpy2.mpz":
    from gmpy2 import mpz
elif _T == "gmp.mpz":
    from gmp import mpz
else:
    from flint import fmpz as mpz

cases = ['1<<7', '1<<38', '1<<300', '1<<3000', '1<<10000']
runner = pyperf.Runner()
for c in cases:
    i = eval(c)
    runner.bench_func(c, mpz, i)

[oscarbenjamin]

conditional compilation seems to be deprecated by Cython.

The way to do it is to move the conditional compilation to C e.g.:

python-flint/src/flint/flintlib/types/fmpz_mod_mat_compat.pxd

Lines 4 to 14 in b2b71f4

	cdef extern from *:
	"""
	/*
	* fmpz_mod_mat function signatures were changed in FLINT 3.1.0
	*/
	#if __FLINT_RELEASE >= 30100 /* Flint 3.1.0 or later */
	#define compat_fmpz_mod_mat_init(mat, rows, cols, ctx) fmpz_mod_mat_init(mat, rows, cols, ctx)
	#define compat_fmpz_mod_mat_init_set(mat, src, ctx) fmpz_mod_mat_init_set(mat, src, ctx)
	#define compat_fmpz_mod_mat_clear(mat, ctx) fmpz_mod_mat_clear(mat, ctx)
	#define compat_fmpz_mod_mat_set(A, B, ctx) fmpz_mod_mat_set(A, B, ctx)

[oscarbenjamin]

To my surprise, #324 approach seems to be better.

Is there anything that can improve this either in CPython or GMP?

Or should we just conclude that for python-flint the approach in gh-324 is best?

The approach in gh-324 seems simpler and works for all supported CPython versions without needing any conditional compilation.

What are gmpy2 or python-gmp going to use?

[skirpichev]

The way to do it is to move the conditional compilation to C e.g.

Yes, makes sense. I think it should be possible to expose simple wrappers like gmpy2's mpz_set_PyLong(), which will have all needed fallbacks.

Is there anything that can improve this either in CPython or GMP?

I don't think that something can be impoved on CPython side, at least not for "big enough" integers. The PEP 757 API provides raw access to the "array of digits" view for CPython int's.

Though, maybe mpz_import/export() could be improved.

What are gmpy2 or python-gmp going to use?

Unfortunately, #324 approach can't be translated to C easily for all supported CPython versions. Probably, I'll keep the current solution, but will benchmark also new approach (PyLong_AsNativeBytes/PyLong_FromNativeBytes).

[oscarbenjamin]

maybe mpz_import/export() could be improved.

Without having looked at the code my guess would be that because these functions are quite general they end up being less efficient than PyLong_AsNativeBytes/PyLong_FromNativeBytes when dealing specificially with CPython's 30 bit limb format.

[oscarbenjamin]

Okay thanks @skirpichev for the benchmarks here. This PR was useful but I think what it proves is that we should go with gh-324 instead.

[oscarbenjamin]

This PR was useful but I think what it proves is that we should go with gh-324 instead.

Actually perhaps it suggests that we should not use mpz_import/mpz_export but it might still be faster to use PEP 757 with some custom C code that is specialised for the 30 bit limb case.