ipython freezes at start if IPYTHONDIR is on an NFS mount #882
Comments
|
Most likely the problem is caused by not having pyzmq (or zeromq itself) installed. Since the ipython won't work without that, the real issue is not handling this situation more gracefully. |
|
Plain terminal IPython should work without ZMQ, and I think we already have an error message when we do need it. I have a feeling it's related to the profile directory being mounted over NFS, but I don't know exactly what the problem is. |
|
@fperez: Any ideas on this one? I seem to remember other people having a problem when IPYTHONDIR is mounted over the network, but I don't know what the problem is. |
|
Running ipython under strace, the following is the part where it appears to freeze: (i.e. the last few lines strace prints out) open("/loc/balin/d1/abeld/.ipython/profile_default/history.sqlite", O_RDWR|O_CREAT, 0644) = 3 (/loc/balin/d1/ is mounted via nfs, the history.sqlite file itself is empty.) |
|
OK, the SQLite FAQ says: "...fcntl() file locking is broken on many NFS implementations. You should avoid putting SQLite database files on NFS if multiple processes might try to access the file at the same time." We do open two connections to the database (we use a separate thread for writing history, for performance reasons), so perhaps this is the problem. I don't know what we can do about this. If it raised an error, we could handle it, but if it just blocks, that doesn't work. Is there some reliable way to tell whether the IPYTHONDIR folder is on an NFS mount? |
|
Like the SQLite docs say, it affects 'many NFS implementations', not all. And I imagine that it can depend on configuration - my NFS environment was configured quite a bit from the defaults, which may matter. |
|
This is nasty: I'm not quite sure how to detect whether a path is on nfs (I can think of several hacks, but I don't know how reliable/portable/robust they would be). In the long run, I think the solution will have to be to allow the history db to be housed at runtime in a temp directory. IPython would have to copy the db out to a temp directory on startup, use it from the temp directory, and copy it back to its normal place in atexit() operations. Some care will be needed to ensure that with multiple concurrent sessions, only the first copies the database file out so later ones don't clobber it. Handling the fact that sessions can die via segfaults without doing atexit cleanups will be very tricky. @abeld, in the meantime, your best bet will be to set the |
|
I don't think it's feasible to detect NFS in particular, move the db to a temp location, and restore it after the session. I think we just have to instruct users on NFS to put their history somewhere local ( What would be helpful is if we can more directly detect a hang in the right place, and then show a message, explaining how to put their history off of the NFS mount. @takluyver - Am I right in assuming that it is the init_db call that would be hanging? Perhaps we can detect a hang there with SIGALRM (though it may be a locked up call that won't respond to the signal). |
|
I think it's the sqlite3.connect call hanging, yes. The trace above shows an open() call at the C level. |
|
Guys, in light of #909 and #940 having been merged, I'm closing this, as I don't think there's anything more we can do. The nfs limitations of sqlite are beyond our power to fix, and now we have alternatives thanks to these two PRs, so there really isn't anything else we can do here. Unfortunately, users whose |
Using ipython 0.11 in virtualenv, I can't start ipython because it freezes, and won't even react to Ctrl-C or 'kill' (has to be killed with 'kill -9')
Running with debug info, I get:
(env)abeld@piko2:0:/l/piko2/d0/abeld/termekhalo_korrelaciok$ ipython --log-level=DEBUG --debug
[TerminalIPythonApp] Config changed:
[TerminalIPythonApp] {'Application': {'log_level': 10}}
[TerminalIPythonApp] Using existing profile dir: u'/loc/remote_computer/d1/abeld/.ipython/profile_default'
[TerminalIPythonApp] Attempting to load config file: ipython_config.py
(/loc/remote_computer/d1 is mounted via NFS but is accessible)
The text was updated successfully, but these errors were encountered: