Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This will only help if the workers are separate processes. Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks.


> Thread workers will hold the GIL in Python and prevent network I/O while they are doing CPU bound tasks.

Using cython:

    with nogil:
        # whatever you need to do, as long as it
        # doesn't touch a python object
If you're doing heavy calculations from python you should at least be considering cython.


sure, the pool only cares about concurrent.futures.Executor interface, an implementation could be processes or cloud resources.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: