STD lib roadmap

[joeblew99]

Very nice project.

I am a golang programmer and it would be nice to have many of the machine learning python code available to golang programmers.

Is there a CI build system to build and test the pythons libs yet ?

[S-YOU]

I guess the STD lib regarding machine learning you mean is like numpy, scipy, etc.
Actually, most of core part of those are written in C.

[joeblew99]

@S-YOU thanks - thats what i was afraid of.
I guess the upper layer stuff could be ported like the dataframes ? I am thinking that there might be the ability to map it to the FS and DB then. For example boltdb.

[S-YOU]

Well, I haven't looked source codes, but pandas has less C than numpy, so there is more chances to be able to port.
I noticed some projects porting or rewriting numpy to ruby or javascript, but not getting attractions and fading away.
Its part of syntax which python provide that make numpy and pandas popular in my opinion.

[trotterdylan]

Thanks for reporting this! I will get around to publishing a roadmap which may help answer questions like these.

As S-YOU says, many of the libraries you may be interested in are written in C so it's not clear when or if they'll work on Grumpy.

[joeblew99]

i will leave open then for you then, so you can write the roadmap.

i am imagining a one hit port of each lib, and then write golang underneath the ported code where it used c code before.
But this is controversial, because it assumes devs writing code eventually swap over to golang, since its a once hit port.

[aep]

python's data science ecosystem is quite expansive,
and that's the main reason i was hoping grumpy could help translate it to go.

However, for that to make any sense it would really need working c extensions.
I was wondering why grumpy doesn't support that. If its about different internal object representation, what would prevent us from building a compat wrapper for the existing c extensions?

[joeblew99]

you make a good point. There is alot of good code there to be leveraged as a transition. As far as i knwo the c code that python wraps is structured in a way that means it needs to be changed to be wrapped by golang. But eager to hear others opinions here.

…

On Mon, May 22, 2017 at 4:57 PM Arvid E. Picciani ***@***.***> wrote: python's data science ecosystem is quite expansive, and that's the main reason i was hoping grumpy could help translate it to go. However, for that to make any sense it would really need working c extensions. I was wondering why grumpy doesn't support that. If its about different internal object representation, what would prevent us from building a compat wrapper for the existing c extensions? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#112 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALcac7NjcRrRn4P1b3gMPoFWg4WkiLBfks5r8aJUgaJpZM4LjlaJ> .

[trotterdylan]

Conceptually a compatibility adapter seems possible. Prior art includes JyNI for Jython and CPyExt for PyPy. For many CPython extensions this could work well since you would just call the Grumpy equivalent (e.g. PyDict_SetItem -> grumpy.(*Dict)SetItem). I think there are probably APIs that wouldn't map as cleanly. The bigger problem, though, is that many C extensions operate directly on Python object structs assuming the GIL is held, and there's no good way to intercept those kinds of mutations. Worse, many mutations won't have a clean mapping to Grumpy's ABI.

All that said, JyNI and CPyExt may offer some approaches here so I'd love to see somebody dig into what's possible.

[joeblew99]

I hit this today... Needed some hardcode financial Maths.
Python has functionality at:
code: https://github.com/numpy/numpy/blob/master/numpy/lib/financial.py
docs: https://docs.scipy.org/doc/numpy/reference/routines.financial.html
IDE: https://sourcegraph.com/github.com/numpy/numpy/-/blob/numpy/lib/financial.py

Some started a golang port too.
https://github.com/orcaman/financial

Would this be a good candidate for running through grumpy ? I am no python expert to really know if it has dependency issues.

[trotterdylan]

financial.py uses a lot of numpy functionality so it would be challenging to get that working.

orcaman/financial looks like a reasonable alternative. From within a Grumpy program you could do:

from __go__.github.com.orcaman.financial import nper, mirr, ...

[joeblew99]

@trotterdylan thanks ! looks like a viable path