Importing gensim, numpy+mkl modules TD099

hi,
trying to get gensim module working here.
Ive manged to get it recognized by the python 3.5 interpreter, and I’ve added the 64-bit file path to the preferences, but I get an error for numpy+mkl library.

Any clues?

error below:

python >>> import numpy
python >>> import gensim
Traceback (most recent call last):
  File "<Textport>", line 1, in <module>
  File "C:/Program Files/Derivative/TouchDesigner099/bin/TouchInit.py", line 175, in tdcustomimport
    r = previousimport(*args, **kw)
  File "C:/Program Files/Python35/Lib/site-packages\gensim\__init__.py", line 6, in <module>
    from gensim import parsing, matutils, interfaces, corpora, models, similarities, summarization
  File "C:/Program Files/Derivative/TouchDesigner099/bin/TouchInit.py", line 175, in tdcustomimport
    r = previousimport(*args, **kw)
  File "C:/Program Files/Derivative/TouchDesigner099/bin/TouchInit.py", line 175, in tdcustomimport
    r = previousimport(*args, **kw)
  File "C:/Program Files/Python35/Lib/site-packages\gensim\parsing\__init__.py", line 7, in <module>
    from .preprocessing import *
  File "C:/Program Files/Derivative/TouchDesigner099/bin/TouchInit.py", line 175, in tdcustomimport
    r = previousimport(*args, **kw)
  File "C:/Program Files/Python35/Lib/site-packages\gensim\parsing\preprocessing.py", line 10, in <module>
    from gensim import utils
  File "C:/Program Files/Derivative/TouchDesigner099/bin/TouchInit.py", line 175, in tdcustomimport
    r = previousimport(*args, **kw)
  File "C:/Program Files/Derivative/TouchDesigner099/bin/TouchInit.py", line 175, in tdcustomimport
    r = previousimport(*args, **kw)
  File "C:/Program Files/Python35/Lib/site-packages\gensim\utils.py", line 42, in <module>
    import scipy.sparse
  File "C:/Program Files/Derivative/TouchDesigner099/bin/TouchInit.py", line 175, in tdcustomimport
    r = previousimport(*args, **kw)
  File "C:/Program Files/Derivative/TouchDesigner099/bin/TouchInit.py", line 175, in tdcustomimport
    r = previousimport(*args, **kw)
  File "C:/Program Files/Python35/Lib/site-packages\scipy\__init__.py", line 61, in <module>
    from numpy._distributor_init import NUMPY_MKL  # requires numpy+mkl
ImportError: cannot import name 'NUMPY_MKL'

and here it imports fine in the python shell:

Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import gensim
C:\Program Files\Python35\lib\site-packages\gensim\utils.py:865: UserWarning: detected Windows; aliasing chunkize to chunkize_serial
  warnings.warn("detected Windows; aliasing chunkize to chunkize_serial")
>>>

TD099 comes with numpy but not a version with mkl for scipy etc. It’s best to install a different version of numpy in a virtual env and use the path to that virtual env as your custom modules path in TD. I made a virtual environment with

venv C:\Python35_TD --python=C:\Python35\python.exe

I got the numpy module here
lfd.uci.edu/~gohlke/pythonlibs/#numpy

I activated the virtual environment by running activate.bat in C:\Python35_TD\scripts.
Then in the same command prompt I did

pip install ~whatever-numpy~.whl

(Insert your own wheel and keep installing other wheels)

Now I can use this to import scipy

import sys
packages = "C:/Python35_TD/Lib/site-packages"

# put my virtual env first in path:
# remove it from sys.path and then put it first
if packages in sys.path:
	sys.path.remove(packages)
	sys.path.insert(0, packages)
import numpy as np
print(np.__version__) # checking it's not the one that came with TD
from scipy.spatial import Voronoi

At least that’s what I remember doing. Let me know how it goes…

Thanks for the pointers, and in case anyone else runs into this, here’s how I managed to get it all going.

the virtual environment command that I used was:

python -m venv C:\Python35-TD

cd C:\Python35-TD\Scripts
activate.bat

then downloaded the wheel files from here into the venv folder:
lfd.uci.edu/~gohlke/pythonlibs/#numpy
lfd.uci.edu/~gohlke/pythonlibs/#scipy

and a pip install for each

then

pip install --upgrade gensim

added the venv folder to the system preferences

and in TD followed the wiki from here:
derivative.ca/wiki099/index … n_Tutorial

import sys
sys.path.remove('C:\\Program Files\\Derivative\\TouchDesigner099\\bin\\lib\\site-packages')

import numpy
numpy.version.version
'1.13.1'

import gensim
C:/Program Files/Python35/Lib/site-packages\gensim\utils.py:865: UserWarning: detected Windows; aliasing chunkize to chunkize_serial
  warnings.warn("detected Windows; aliasing chunkize to chunkize_serial")

and now on to the fun part :wink:

Hi,

I am trying to import pytorch in TD, and I am stuck with an issue which seems to be related to numpy+mkl library[import torch: DLL load failed: The operating system cannot run %1. (pip installation) · Issue #8894 · pytorch/pytorch · GitHub]. I followed this discussion, and tried to install compatible versions:
Python: 3.5.4
Numpy+mkl: 17.5 and 16.6
cuda: 9.2
TD: 20140(Python version 3.5.1+)
In a command line window python, I could import and run pytorch properly. However, after putting my local package first in TD textport, I import numpy, and TD crashes. Should I install the numpy+mkl.whl inside an open TD textport instead?
I successfully installed Tensorflow with numpy and scipy before, but I was unaware of an mkl version issue.

Although there is another option, which is to call a subprocess and use my local python in cmd, I still wonder if it is a simple mistake I could fix.

Hopefully, some one could help and give a suggestion. Thank you very much!

Cheers,

Yong