pydev cannot run unittest (it works in spyder)

Question

This is not extremely important since I can make it work in spyder, but I like pydev quite a lot (I'm very used to eclipse) and it is just annoying, I've spent some time on it already and I'd like to fix it.

I've got this unittest (actually it belongs to nltk) and I cannot make it work in pydev. Things that I've tried:

• Run it in spyder => it works fine
• Updated pydev and eclipse (though I'm not yet in eclipse Mars released last week; my versions are, for eclipse 4.5.0.20150603-1639 and for pydev 4.1.0.2015052700003)
• I can run other unit tests perfectly with pydev, with breakpoints and everything, provided they are in projects different to nltk
• Read quite a few SO questions, but didn't find anything that helped much
• I've also tried to copy parts of the content of the test to one of my projects and it works well -which probably means there's something silly I'm doing, but I don't get it. Must be something about the combination of my environment with the fact of running the test from the nltk structure
• Get the same behavior in python 2.7 and 3.4

The error that I get is the following:

Traceback (most recent call last):
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydevd.py", line 13, in <module>
    from pydevd_breakpoints import * #@UnusedWildImport
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydevd_breakpoints.py", line 15, in <module>
    from pydevd_comm import GetGlobalDebugger
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydevd_comm.py", line 79, in <module>
    import _pydev_completer
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/_pydev_completer.py", line 21, in <module>
    import _pydev_imports_tipper
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/_pydev_imports_tipper.py", line 2, in <module>
    import inspect
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/lorenzorubio/tools/anaconda/lib/python2.7/inspect.py", line 39, in <module>
    import tokenize
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/lorenzorubio/git/nltk/nltk/tokenize/__init__.py", line 62, in <module>
    from nltk.data              import load
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/lorenzorubio/git/nltk/nltk/__init__.py", line 115, in <module>
    from nltk.decorators import decorator, memoize
  File "/Users/lorenzorubio/tools/pydev-mars/Eclipse.app/Contents/Eclipse/plugins/org.python.pydev_4.1.0.201505270003/pysrc/pydev_monkey_qt.py", line 71, in patched_import
    return original_import(name, *args, **kwargs)
  File "/Users/lorenzorubio/git/nltk/nltk/decorators.py", line 183, in <module>
    @decorator
  File "/Users/lorenzorubio/git/nltk/nltk/decorators.py", line 161, in decorator
    if inspect.isclass(caller):
AttributeError: 'module' object has no attribute 'isclass'

I was hoping that somebody maybe could give me a hint about what is happening.

The test is in this github link. (In order to run it you should download nltk in the twitter branch, though it should be merged to the main branch soon, but anyway, I don't expect anybody to actually run it, it is already enough to get one person annoyed with this).

I'm using anaconda for dealing with the python installation, and of course I add in the PYTHONPATH the path of the branch I'm using for nltk.

Thank you so much for your reading and maybe your pointers.

EDIT: Thanks @That1Guy for your comment!

It is indeed a conflict in naming, not in inspect but in tokenize. There is a tokenize in python that conflicts with the tokenize in nltk. I did try to change in .../anaconda/lib/python2.7/inspect.py and instead of

import tokenize

I edit to

from tokenize import tokenize

and it works!

(nltk tokenize would be import nltk.tokenize)

Still, it is not clean at all to modify a system file... is there a better solution? And why this happens only in pydev?

Thanks again.