ffi: add shared-buffer fast path for numeric and pointer signatures #5890

Job	Run time
Build slim tarball	2m 13s
aarch64-linux: with shared libraries	1h 30m 29s
x86_64-linux: with shared libraries	1h 49m 36s
aarch64-darwin: with shared libraries	2h 17m 53s
x86_64-darwin: with shared libraries	3h 41m 40s
aarch64-darwin: with shared libraries	24m 10s
aarch64-linux: with shared libraries	1h 30m 29s
x86_64-linux: with shared libraries	1h 49m 36s
Build slim tarball	2m 13s
x86_64-darwin: with shared libraries	3h 41m 40s
	16h 49m 59s

Provide feedback