make_traceback can cause segfaults in certain environments

[ghost]

I've been running into a coredump that I can only replicate when answering RPC requests through a multi-threaded SWIG'd API. It's easy to duplicate the crash:

env = jinja2.Environment()
template = env.from_string("{{ pkg_dir")

The actual exception is (<class 'jinja2.exceptions.TemplateSyntaxError'>, TemplateSyntaxError("unexpected end of template, expected 'end of print statement'.",), <traceback object at 0x97d3504>), but instead, I get a segfault/coredump. The problem is that jinja2 is dynamically importing the debug.py module, which in turn dynamically imports ctypes. A dynamic import of ctypes can cause segfaults in many environments, particularly where threading, read-only filesystems, and SWIG is involved.

It would be nice if a parameter could be passed to Environment() that disabled the traceback annotation features. Traceback:

#0  0x080f4ea0 in vgetargs1 (args=0xb9b14ec, format=0x815e467 "s:zipimporter", p_va=0xf187305c, flags=0)    at ../../../Python/getargs.c:196#1  0x080f5ce4 in PyArg_ParseTuple (args=0xb9b14ec, format=0x815e467 "s:zipimporter")    at ../../../Python/getargs.c:85#2  0x08128888 in zipimporter_init (self=0xb9b150c, args=0xb9b14ec, kwds=0x0)    at ../../../Modules/zipimport.c:69#3  0x080ad60d in type_call (type=0x819e4a0, args=0xb9b14ec, kwds=0x0) at ../../../Objects/typeobject.c:742#4  0x0805e06f in PyObject_Call (func=0x819e4a0, arg=0x815e467, kw=<optimized out>)    at ../../../Objects/abstract.c:2492#5  0x0805f49e in PyObject_CallFunctionObjArgs (callable=0x819e4a0) at ../../../Objects/abstract.c:2723#6  0x080f6baa in get_path_importer (p=<optimized out>, path_hooks=<optimized out>,     path_importer_cache=<optimized out>) at ../../../Python/import.c:1140#7  find_module (fullname=<optimized out>, subname=<optimized out>, path=<optimized out>,     buf=<optimized out>, buflen=4097, p_fp=0xf18793e8, p_loader=0xf18793ec) at ../../../Python/import.c:1357#8  0x080fa3b2 in import_submodule (mod=0xb9aa08c, subname=<optimized out>, fullname=0xf187944f "ctypes.os")    at ../../../Python/import.c:2581#9  0x080fad87 in load_next (p_buflen=<optimized out>, buf=<optimized out>, p_name=<optimized out>,     altmod=<optimized out>, mod=<optimized out>) at ../../../Python/import.c:2411#10 import_module_level (level=<optimized out>, fromlist=<optimized out>, locals=<optimized out>,     globals=<optimized out>, name=<optimized out>) at ../../../Python/import.c:2133
#11 PyImport_ImportModuleLevel (name=0xf7520e94 "os", globals=0x12400bdc, locals=0x12400bdc, 
    fromlist=0x81862f8, level=-1) at ../../../Python/import.c:2184
#12 0x080da552 in builtin___import__ (self=0x0, args=0xb9a6874, kwds=0x0) at ../../../Python/bltinmodule.c:48
#13 0x0805cdc2 in PyObject_Call (func=0xf75102ac, arg=0xb9a6874, kw=0x0) at ../../../Objects/abstract.c:2492
#14 0x080dbb4c in PyEval_CallObjectWithKeywords (func=0xf75102ac, arg=0xb9a6874, kw=<optimized out>)
    at ../../../Python/ceval.c:3619
#15 0x080dce2d in PyEval_EvalFrameEx (f=0xb522fdc, throwflag=0) at ../../../Python/ceval.c:2159
#16 0x080e271b in PyEval_EvalCodeEx (co=0xb3edd58, globals=0x12400bdc, locals=0x12400bdc, args=0x0, 
    argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../../../Python/ceval.c:3000
#17 0x080e280d in PyEval_EvalCode (co=0xb3edd58, globals=0x12400bdc, locals=0x12400bdc)
    at ../../../Python/ceval.c:541
#18 0x080f7e5d in PyImport_ExecCodeModuleEx (name=0xf187d8e6 "ctypes", co=0xb3edd58, 
    pathname=0xf187a76b "/usr/grte/v2/piii-linux/lib/python2.6/ctypes/__init__.pyc")
    at ../../../Python/import.c:677
#19 0x080f875a in load_source_module (name=<optimized out>, 
    pathname=0xf187a76b "/usr/grte/v2/piii-linux/lib/python2.6/ctypes/__init__.pyc", fp=<optimized out>)
    at ../../../Python/import.c:1017
#20 0x080f9887 in load_package (name=0xf187d8e6 "ctypes", pathname=<optimized out>)
    at ../../../Python/import.c:1073
#21 0x080fa5f0 in load_module (loader=<optimized out>, type=<optimized out>, buf=<optimized out>, 
    fp=<optimized out>, name=<optimized out>) at ../../../Python/import.c:1835
#22 import_submodule (mod=0x81862f8, subname=<optimized out>, fullname=0xf187d8e6 "ctypes")
    at ../../../Python/import.c:2591
#23 0x080fb074 in load_next (p_buflen=<optimized out>, buf=<optimized out>, p_name=<optimized out>, 
    altmod=<optimized out>, mod=<optimized out>) at ../../../Python/import.c:2415
#24 import_module_level (level=<optimized out>, fromlist=<optimized out>, locals=<optimized out>, 
    lobals=<optimized out>, name=<optimized out>) at ../../../Python/import.c:2133#25 PyImport_ImportModuleLevel (name=0xb9b0734 "ctypes", globals=0x1002024c, locals=0x81862f8,     fromlist=0x81862f8, level=-1) at ../../../Python/import.c:2184#26 0x080da552 in builtin___import__ (self=0x0, args=0xb40dd24, kwds=0x0) at ../../../Python/bltinmodule.c:48#27 0x0805cdc2 in PyObject_Call (func=0xf75102ac, arg=0xb40dd24, kw=0x0) at ../../../Objects/abstract.c:2492#28 0x080dbb4c in PyEval_CallObjectWithKeywords (func=0xf75102ac, arg=0xb40dd24, kw=<optimized out>)    at ../../../Python/ceval.c:3619#29 0x080dce2d in PyEval_EvalFrameEx (f=0xb524894, throwflag=0) at ../../../Python/ceval.c:2159#30 0x080e271b in PyEval_EvalCodeEx (co=0x102bf188, globals=0x1002024c, locals=0x0, args=0xb524838,     argcount=0, kws=0xb524838, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../../../Python/ceval.c:3000#31 0x080e0cd2 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>,     pp_stack=<optimized out>, func=<optimized out>) at ../../../Python/ceval.c:3846#32 call_function (oparg=<optimized out>, pp_stack=<optimized out>) at ../../../Python/ceval.c:3771#33 PyEval_EvalFrameEx (f=0xb5246fc, throwflag=0) at ../../../Python/ceval.c:2412#34 0x080e271b in PyEval_EvalCodeEx (co=0x102bfb18, globals=0x1002024c, locals=0x1002024c, args=0x0,     argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at ../../../Python/ceval.c:3000#35 0x080e280d in PyEval_EvalCode (co=0x102bfb18, globals=0x1002024c, locals=0x1002024c)    at ../../../Python/ceval.c:541#36 0x080f7e5d in PyImport_ExecCodeModuleEx (name=0xf1880e8f "jinja2.debug", co=0x102bfb18, 
    pathname=0xf187fe27 "/g/cluster_workflow/g3/server/shell/shell_server.runfiles/G3/third_party/py/jinja2/debug.py")
    at ../../../Python/import.c:677
#37 0x080f875a in load_source_module (name=<optimized out>, 
    pathname=0xf187fe27 "/g/cluster_workflow/g3/server/shell/shell_server.runfiles/G3/third_party/py/jinja2/debug.py", 
    fp=<optimized out>) at ../../../Python/import.c:1017
#38 0x080fa669 in load_module (loader=<optimized out>, type=<optimized out>, buf=<optimized out>, 
    fp=<optimized out>, name=<optimized out>) at ../../../Python/import.c:1821
#39 import_submodule (mod=0xb67529c, subname=<optimized out>, fullname=0xf1880e8f "jinja2.debug")
    at ../../../Python/import.c:2591
#40 0x080fae60 in load_next (p_buflen=<optimized out>, buf=<optimized out>, p_name=<optimized out>, 
    altmod=<optimized out>, mod=<optimized out>) at ../../../Python/import.c:2411
#41 import_module_level (level=<optimized out>, fromlist=<optimized out>, locals=<optimized out>, 
    globals=<optimized out>, name=<optimized out>) at ../../../Python/import.c:2140
#42 PyImport_ImportModuleLevel (name=0xba7a8cc "jinja2.debug", globals=0xff59d74, locals=0x81862f8, 
    fromlist=0xb66130c, level=-1) at ../../../Python/import.c:2184
#43 0x080da552 in builtin___import__ (self=0x0, args=0xa14e02c, kwds=0x0) at ../../../Python/bltinmodule.c:48
#44 0x0805cdc2 in PyObject_Call (func=0xf75102ac, arg=0xa14e02c, kw=0x0) at ../../../Objects/abstract.c:2492
#45 0x080dbb4c in PyEval_CallObjectWithKeywords (func=0xf75102ac, arg=0xa14e02c, kw=<optimized out>)
    at ../../../Python/ceval.c:3619
#46 0x080dce2d in PyEval_EvalFrameEx (f=0x100d9604, throwflag=0) at ../../../Python/ceval.c:2159
#47 0x080e271b in PyEval_EvalCodeEx (co=0xba7c4e8, globals=0xff59d74, locals=0x0, args=0x100d39ac, 
    argcount=2, kws=0x100d39b4, kwcount=1, defs=0xb785c18, defcount=3, closure=0x0)
    at ../../../Python/ceval.c:3000
#48 0x080e0cd2 in fast_function (nk=<optimized out>, na=<optimized out>, n=<optimized out>, 
    pp_stack=<optimized out>, func=<optimized out>) at ../../../Python/ceval.c:3846

[mitsuhiko]

Traceback annotation can be disabled by overriding the handle_exception method:

class SimpleEnvironment(Environment):
    def handle_exception(self, exc_info=None, rendered=False, source_hint=None):
        if exc_info is None:
            exc_info = sys.exc_info()
        raise exc_info[0], exc_info[1], exc_info[2]

The real solution here however would be to force the import of jinja2.debug ahead of time I guess.