New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Global variables undefined in interactive use of embedded ipython shell #62
Comments
[ LP comment 1 by: Fernando Perez, on 2010-04-25 23:36:38.673176+00:00 ] OK, I can confirm the problem (even on trunk), but it's hard. I am just for now making sure this bug is confirmed so we keep tracking it, but I'm not sure how to fix it. The issue is that in embedded shells, we try to update the global namespace to use the surrounding scope (that's the point of an embedded shell, to be able to see what's around you). But this then causes python to fail to resolve the ipython interactive namespace when nested things (like local functions) are defined. See the mainloop() method of the embedded shell for details. I need to think a lot more about how to fix this one, any ideas very much welcome. |
Over at launchpad... chairmanK wrote 20 hours ago: Me Too. In addition to functions, this bug also appears in generator expressions. |
The equivalent component in trunk seems to be IPython.frontend.terminal.InteractiveShellEmbed. But that's broken in other ways, and evidently hasn't been tested much. Does anyone know what its future is? |
Could this be the same issue as #136? |
This has been now fixed:
|
Can someone explain a little more what went into the fix of this? I still encounter the issue, but only when I'm one layer removed via a method call, and only if I run from a script, not interactively. python 2.7.2 on OSX 10.6.8, ipython 0.11 and 0.12 both exhibit similar behavior (using 0.12 for this comment) This is a problem with our project which features (prominently) an embedded IPython shell. testembed.py from IPython import embed
def hi():
embed()
if __name__ == '__main__':
#embed()
hi() Run this on the command line with
However, comment out the call to
After poking around, I think it has to do with the Thoughts? |
This is somewhat complicated, but I believe that's a restriction in Python itself. In the failing case you show,
This is probably because nested scopes were added to Python relatively late (they were a future import in 2.1, and always on in 2.2). |
Ok, I think I understand. Looks like we can't really do anything about this then, other than dumping what I would write interactively to a file and then reading that file back in. Probably too complex for what we want to be able to do interactively. Thanks, btw. |
Sorry to keep harping on here, but it just makes interactive sessions feel very clunky: Regular python:
Regular IPython:
Embedded IPython:
I guess there isn't much that can be done, but I don't get why we can create a dynamic closure in regular python/ipython but not the embedded version. |
By the oddities of Python scoping, in standard Python/IPython, we're not actually creating a closure. In contrast, when you make a closure, Python attaches references to each variable over which it has closed, as determined by the compiler - but that only works when the inner and outer functions are compiled at the same time. It looks like this:
This is possibly done to save memory - if the closure carried a reference to the local scope where it was defined, none of the variables from that scope could be freed while the closure was live. |
I'm using Python 2.7.2 and IPython 0.12.1 on OSX 10.7.3 and having this issue still. When I run |
There's not a lot we can directly do about that, but we should probably encourage third parties away from |
@takluyver Do you mean that better to use ipython directly in this case? |
It's possible for Django to start IPython in a way that won't cause this problem, but that's not the way we currently make easy. The problem occurs when IPython starts with separate local and global namespaces. There's no reason that Django requires separate local and global namespaces, but that's what calling For reference, here's the code in Django: |
@takluyver That makes sense, thank! I'm opening a Django ticket for this. |
@takluyver, now that we've merged embed_kernel so all the main pieces are in place, do you want to tackle a bit a cleanup of this to make slightly more fine-tuned uses easier? |
I'll have a look at what interface might make most sense. |
I am still having trouble with this issue. I've tried narrowing down exactly what's causing it, and the best I can determine is that it only happens on Ubuntu 12.04. I've tried all of the latest release versions on other servers, and it works fine. But any time I define a function in ipython or use %cpaste to paste in a function from another file, the inside of that function has no access to the global scope. It makes it basically impossible to do anything useful in terms of writing functions on the fly. |
I am still having this problem with IPython 0.13 when it is called from other tools (e.g. Django's debugsqlshell command). It's very frustrating that something as basic as defining a function is totally broken. |
I think embed() is the wrong interface for those to be using. embed() is |
Not only ubuntu. debian wheezy also shows that. |
FYI, the Django ticket @liokm created above is https://code.djangoproject.com/ticket/18204, which now points to https://code.djangoproject.com/ticket/17078, which looks to have been fixed in trunk. It should land in 1.5. |
@bkvirendra that is fixed in django 1.6 |
But 1.6 is not even stable yet! |
Software are not always stable, and between releases there might still be bugs. But there is nothing that should be fixed in IPython. Even if we do something here, the fix would not be in IPython stable before it is released either... |
Problem still persists in ipython==4.2.0. Interestingly under windows it is no problem but together with ubuntu global variables are not recognized. Use case:
|
Problem remains. IPython 3.1.0, Debian Whezzy. |
If I recall correctly, this was fixed for a little while, and seems to be reintroduced (reproduced using IPython 5.1.0 on macOS here: https://git.io/vPDrJ). (edit: I may be mis-remembering the fix. It's possible I just threw everything I needed into a startup file instead of using embed) |
Just in case this helps anyone I've been using this piece of code as a workaround def fix_embed_globals(N=0):
# Get the parent frame
import inspect
frame_level0 = inspect.currentframe()
frame_cur = frame_level0
N = 2 # I may have this value off by one. I rewrote this function to use only standard calls
strict = False
for _ix in range(N + 1):
frame_next = frame_cur.f_back
if frame_next is None:
if strict:
raise AssertionError('Frame level %r is root' % _ix)
else:
break
frame_cur = frame_next
parent_frame = frame_cur
# Add locals to parent globals
parent_frame.f_locals.update(parent_frame.f_globals)
parent_frame.f_globals['_didfixipyglobals'] = True I just call that function whenever I get the name error. It puts all locals in your current frame into the global dictionary. Its hacky but it works in most circumstances. |
As noted at inducer/pudb#103, a temporary workaround while using an embedded shell is: |
Hmm, can the milestone of this bug be changed to one of the next versions? |
Done. |
will this ever get fixed? |
AFAIK, my comments from a few years ago stand:
|
I don't understand, and it makes me angry. |
sorry, I was frustrated that things which would work if you typed them into an empty .py file didn't work here but re-reading the history, it seems this is an issue with Django's embarrassingly I'm working a lot in obsolete version of Django (1.4) according to earlier comment, newer versions of Django shell use ipython differently and may not have the problem? eg https://code.djangoproject.com/ticket/17078 apologies for the misunderstanding |
Yup, I think it was fixed for Django 1.6. If you really can't upgrade, you might want to apply the fix manually. It looks like this was it: django/django@3570ff7 |
I found a workaround today that is posted at #10695. A simpler treatment for the case where the IPython is not embedded inside a function is shown at this thread. I am not an expert so please help to check the validity. |
Embedding IPython results in issues where import statements and other globals changes in the shell environment don't work as expected.[1] As an example, the following commands result in a NameError: In [1]: import time In [2]: def foo(): ...: print(time.time()) ...: In [3]: foo() --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-3-c19b6d9633cf> in <module> ----> 1 foo() <ipython-input-2-a038e7f91c93> in foo() 1 def foo(): ----> 2 print(time.time()) 3 NameError: name 'time' is not defined By switching to use "start_ipython()" instead, we get an environment more similar to what we get by running `ipython` directly. [1] ipython/ipython#62
Embedding IPython results in issues where import statements and other globals changes in the shell environment don't work as expected.[1] As an example, the following commands result in a NameError: In [1]: import time In [2]: def foo(): ...: print(time.time()) ...: In [3]: foo() --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-3-c19b6d9633cf> in <module> ----> 1 foo() <ipython-input-2-a038e7f91c93> in foo() 1 def foo(): ----> 2 print(time.time()) 3 NameError: name 'time' is not defined By switching to use "start_ipython()" instead, we get an environment more similar to what we get by running `ipython` directly. [1] ipython/ipython#62
Embedding IPython results in issues where import statements and other globals changes in the shell environment don't work as expected.[1] As an example, the following commands result in a NameError: In [1]: import time In [2]: def foo(): ...: print(time.time()) ...: In [3]: foo() --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-3-c19b6d9633cf> in <module> ----> 1 foo() <ipython-input-2-a038e7f91c93> in foo() 1 def foo(): ----> 2 print(time.time()) 3 NameError: name 'time' is not defined By switching to use "start_ipython()" instead, we get an environment more similar to what we get by running `ipython` directly. [1] ipython/ipython#62
Embedding IPython results in issues where import statements and other globals changes in the shell environment don't work as expected.[1] As an example, the following commands result in a NameError: In [1]: import time In [2]: def foo(): ...: print(time.time()) ...: In [3]: foo() --------------------------------------------------------------------------- NameError Traceback (most recent call last) <ipython-input-3-c19b6d9633cf> in <module> ----> 1 foo() <ipython-input-2-a038e7f91c93> in foo() 1 def foo(): ----> 2 print(time.time()) 3 NameError: name 'time' is not defined By switching to use "start_ipython()" instead, we get an environment more similar to what we get by running `ipython` directly. [1] ipython/ipython#62
Summary: This fixes issues like ipython/ipython#62. Reviewed By: zzl0 Differential Revision: D44759311 fbshipit-source-id: 5ce94040cef09ac67962c147e86711971c545f91
Original Launchpad bug 399627: https://bugs.launchpad.net/ipython/+bug/399627
Reported by: h-fangohr (Hans Fangohr).
The error can be reproduced as follows:
Within the just started ipython session, global variables are sometimes not visible. Two examples are:
Example 1:
Example 2:
There is no error if "b=1; (lambda :b)()" is put in a script, and this script is executed using ipython's "run" command.
There is no error if "b=1; (lambda :b)()" is executed interactively after having started ipython.
The only way the error seems comes up is if an embedded IPython shell is started from a Python prompt AND the commands are executed interactively at the prompt.
I find the same error when trying ipython-0.9.1 (with python2.4).
The bug was reported by Olivier Klein to the nmag team (http://nmag.soton.ac.uk).
The text was updated successfully, but these errors were encountered: