Passing "detailed_report" in GPTResearcher constructor (still) results in key error

[Handschrift]

I already opened an issue (see #431) but the issue got closed.

The mentioned problem still exists. So running the code:

report_type = "detailed_report" # using ReportType.DetailedReport.value does not work aswell
researcher = GPTResearcher(query, report_type, config_path="config.json")
await researcher.conduct_research()
report = await researcher.write_report()

does produce a KeyError because:

    report_type_mapping = {
        ReportType.ResearchReport.value: generate_report_prompt,
        ReportType.ResourceReport.value: generate_resource_report_prompt,
        ReportType.OutlineReport.value: generate_outline_report_prompt,
        ReportType.CustomReport.value: generate_custom_report_prompt,
        ReportType.SubtopicReport.value: generate_subtopic_report_prompt
    }

does not contain the "detailed_report" key.

This error still exists even if I use the latest version (the master branch, currently at commit 53f2da7) of the package.

[assafelovic]

@proy9714 can you take a look?

[assafelovic]

@Handschrift I understand where the misunderstanding is. GPT Researcher as a package does not yet support long detailed report, and not sure it will. The package is used for doing specific research reports on a topic. The way we're creating long detailed reports is by leveraging GPT Researcher package multiple times and on various formats. You can check out the implementation here: https://github.com/assafelovic/gpt-researcher/blob/master/backend/report_type/detailed_report/detailed_report.py

Due to this misunderstanding, I've refactored the codebase to make it more understandable and also added some better error logs in case this issue happens to others: #443

Thanks for raising this!
FYI @proy9714

[assafelovic]

I will leave this issue open for awhile in case others run into this issue

[NicoLiendro14]

I have the same error but using de UI. Using Detailed Report I get this:

🤔 Generating subtopics...
📋Subtopics: []
ERROR: Exception in ASGI application
....
File "G:\GPT SCRAPER\gpt-researcher\backend\report_type\detailed_report\detailed_report.py", line 61, in _get_all_subtopics
return subtopics.dict()["subtopics"]
~~~~~~~~~^^^^^^^^^^^^^
TypeError: list indices must be integers or slices, not str

So, subtopics is an empty list.

[assafelovic]

@proy9714 can you please help with this? Looks like either non determinstic issue with LLMs or some validation that needs to be added for better stability

[proy9714]

Hey @NicoLiendro14 can you provide me the details of the report you are generating so that I can try to emulate it and see where the error is! The report topic, source, the search provider and LLM you are using should be enough! Thanks!

[proy9714]

@assafelovic @Handschrift @NicoLiendro14

Found the problem : langchain-ai/langchain#21635

Anyone know of any fixes?

In the meantime I would highly advise using gpt-4-turbo....until the langchain library gets updated. The present version still uses pydantic v1 instead of v2.

[bat9527]

Has anyone ever run into this problem when using detailed reports？

Exception in parsing subtopics :  AzureOpenAIProvider.__init__() got an unexpected keyword argument 'model'
📋Subtopics: []
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi
    result = await self.app(self.scope, self.asgi_receive, self.asgi_send)  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/middleware/errors.py", line 151, in __call__
    await self.app(scope, receive, send)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/routing.py", line 373, in handle
    await self.app(scope, receive, send)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/routing.py", line 96, in app
    await wrap_app_handling_exceptions(app, session)(scope, receive, send)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/starlette/routing.py", line 94, in app
    await func(session)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/fastapi/routing.py", line 348, in app
    await dependant.call(**values)
  File "/Users/Andy/gitee/gpt-researcher/backend/server.py", line 53, in websocket_endpoint
    report = await manager.start_streaming(task, report_type, report_source, websocket)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/Andy/gitee/gpt-researcher/backend/websocket_manager.py", line 57, in start_streaming
    report = await run_agent(task, report_type, report_source, websocket)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/Andy/gitee/gpt-researcher/backend/websocket_manager.py", line 75, in run_agent
    report = await researcher.run()
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/Andy/gitee/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 36, in run
    subtopics = await self._get_all_subtopics()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/Andy/gitee/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 62, in _get_all_subtopics
    return subtopics.dict()["subtopics"]
           ^^^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'dict'
INFO:     connection closed