[BUG] improve error handling in bulk inserter to catch and log expected failures

[axiomofjoy]

when the unique constraint is violated in postgres, postgres correctly rejects the update on the span, but spits out an unhandled exception in the logs

ERROR [phoenix.db.bulk_inserter] Failed to insert evaluations
Traceback (most recent call last):
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 538, in _prepare_and_execute
    self._rows = await prepared_stmt.fetch(*parameters)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch
    data = await self.__bind_execute(args, 0, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 241, in __bind_execute
    data, status, _ = await self.__do_execute(
                      ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/asyncpg/prepared_stmt.py", line 230, in __do_execute
    return await executor(protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 207, in bind_execute
asyncpg.exceptions.UniqueViolationError: duplicate key value violates unique constraint "uq_span_annotations_span_rowid_name"
DETAIL:  Key (span_rowid, name)=(761, Q&A Correctness) already exists.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1971, in _exec_single_context
    self.dialect.do_execute(
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 919, in do_execute
    cursor.execute(statement, parameters)
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 572, in execute
    self._adapt_connection.await_(
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 131, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 550, in _prepare_and_execute
    self._handle_exception(error)
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 501, in _handle_exception
    self._adapt_connection._handle_exception(error)
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 789, in _handle_exception
    raise translated_error from error
sqlalchemy.dialects.postgresql.asyncpg.AsyncAdapt_asyncpg_dbapi.IntegrityError: <class 'asyncpg.exceptions.UniqueViolationError'>: duplicate key value violates unique constraint "uq_span_annotations_span_rowid_name"
DETAIL:  Key (span_rowid, name)=(761, Q&A Correctness) already exists.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/xandersong/phoenix/src/phoenix/db/bulk_inserter.py", line 185, in _insert_evaluations
    result = await insert_evaluation(session, evaluation)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/src/phoenix/db/insertion/evaluation.py", line 49, in insert_evaluation
    return await _insert_span_evaluation(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/src/phoenix/db/insertion/evaluation.py", line 114, in _insert_span_evaluation
    await session.execute(
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/ext/asyncio/session.py", line 461, in execute
    result = await greenlet_spawn(
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 201, in greenlet_spawn
    result = context.throw(*sys.exc_info())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2306, in execute
    return self._execute_internal(
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2191, in _execute_internal
    result: Result[Any] = compile_state_cls.orm_execute_statement(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/orm/bulk_persistence.py", line 1276, in orm_execute_statement
    result = conn.execute(
             ^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1422, in execute
    return meth(
           ^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 514, in _execute_on_connection
    return connection._execute_clauseelement(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1644, in _execute_clauseelement
    ret = self._execute_context(
          ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1850, in _execute_context
    return self._exec_single_context(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1990, in _exec_single_context
    self._handle_dbapi_exception(
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2357, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1971, in _exec_single_context
    self.dialect.do_execute(
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 919, in do_execute
    cursor.execute(statement, parameters)
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 572, in execute
    self._adapt_connection.await_(
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 131, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 196, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 550, in _prepare_and_execute
    self._handle_exception(error)
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 501, in _handle_exception
    self._adapt_connection._handle_exception(error)
  File "/Users/xandersong/phoenix/.venv/lib/python3.12/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 789, in _handle_exception
    raise translated_error from error
sqlalchemy.exc.IntegrityError: (sqlalchemy.dialects.postgresql.asyncpg.IntegrityError) <class 'asyncpg.exceptions.UniqueViolationError'>: duplicate key value violates unique constraint "uq_span_annotations_span_rowid_name"
DETAIL:  Key (span_rowid, name)=(761, Q&A Correctness) already exists.
[SQL: INSERT INTO span_annotations (span_rowid, name, label, score, explanation, metadata, annotator_kind) VALUES ($1::INTEGER, $2::VARCHAR, $3::VARCHAR, $4::FLOAT, $5::VARCHAR, $6::JSONB, $7::VARCHAR) RETURNING span_annotations.id]
[parameters: (761, 'Q&A Correctness', 'incorrect', 0.0, 'The question is asking about a hosting service that offers accelerated CDN delivery and tracking of usage data for running a website. The reference t ... (158 characters truncated) ...  delivery, or tracking of usage data for running a website. Therefore, the answer does not correctly answer the question based on the reference text.', '{}', 'LLM')]
(Background on this error at: https://sqlalche.me/e/20/gkpj)