Production related issue of order primary key

[Cinderella-Man]

Cascading effect on not storing trades that happened:

iex(1)> 03:13:11.739 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(17919) received an transaction of 73.50000000 for BUY order 306907004 @ 0.25591000
iex(1)> 03:13:12.004 module=gen_server [error] GenServer #PID<0.883.0> terminating
** (Ecto.ConstraintError) constraint error when attempting to insert struct:

    * trades_pkey (unique_constraint)

If you would like to stop this constraint violation from raising an
exception and instead add it as an error to your changeset, please
call `unique_constraint/3` on your changeset with the constraint
`:name` as an option.

The changeset has not defined any constraint.

    (ecto) lib/ecto/repo/schema.ex:687: anonymous fn/4 in Ecto.Repo.Schema.constraints_to_errors/3
    (elixir) lib/enum.ex:1327: Enum."-map/2-lists^map/1-0-"/2
    (ecto) lib/ecto/repo/schema.ex:672: Ecto.Repo.Schema.constraints_to_errors/3
    (ecto) lib/ecto/repo/schema.ex:274: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
    (hefty) lib/hefty/trades.ex:70: Hefty.Trades.create_trade/1
    (hefty) lib/hefty/algos/naive/trader.ex:269: anonymous fn/5 in Hefty.Algos.Naive.Trader.handle_info/2
    (ecto_sql) lib/ecto/adapters/sql.ex:874: anonymous fn/3 in Ecto.Adapters.SQL.checkout_or_transaction/4
    (db_connection) lib/db_connection.ex:1415: DBConnection.run_transaction/4
    (hefty) lib/hefty/algos/naive/trader.ex:268: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Broadcast{event: "trade_event", payload: %Hefty.Repo.Binance.TradeEvent{__meta__: #Ecto.Schema.Metadata<:loaded, "trade_events">, buyer_market_maker: true, buyer_order_id: 306907004
, event_time: 1574219590552, event_type: "trade", id: "aad67857-a5ee-4ccd-9832-307d26a4e3d5", inserted_at: ~N[2019-11-20 03:13:10], price: "0.25591000", quantity: "73.50000000", seller_order_id: 306919454, symbo
l: "XRPUSDT", trade_id: 40912131, trade_time: 1574219590491, updated_at: ~N[2019-11-20 03:13:10]}, topic: "stream-XRPUSDT"}
iex(1)> 03:13:12.005 application=hefty module=Hefty.Algos.Naive.Leader [info] Leader restarts process as it died
03:13:12.005 application=hefty module=Hefty.Algos.Naive.Leader [info] Trader found in the list of traders. Removing
03:13:12.006 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader on symbol XRPUSDT with budget of 30.0670539133375

As we can see it failed to write trade to a db and this was in the end completely ignored by the system, it started new trader and kept trading as money would never be spent..

Cased account to have insufficient founds...:

iex(1)> 09:10:42.935 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader(41042) on symbol XRPUSDT with budget of 30.0931284621375
iex(1)> 09:10:48.421 application=hefty module=Hefty.Algos.Naive.Trader [info] Trader(41042) - Placing BUY order for XRPUSDT @ 0.24914, quantity: 120.7
iex(1)> 09:10:48.969 module=gen_server [error] GenServer #PID<0.1420.0> terminating
** (MatchError) no match of right hand side value: {:error, %Binance.InsufficientBalanceError{reason: %{code: -2010, msg: "Account has insufficient balance for requested action."}}}
    (hefty) lib/hefty/algos/naive/trader.ex:643: Hefty.Algos.Naive.Trader.place_buy_order/2
    (hefty) lib/hefty/algos/naive/trader.ex:204: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3

This started a cycle of doom that resulted in OS' OOM crash erlang VM:

iex(1)> 09:11:15.033 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader on symbol  with budget of 
iex(1)> 09:11:15.158 module=gen_server [error] GenServer #PID<0.1439.0> terminating
** (FunctionClauseError) no function clause matching in Hefty.Algos.Naive.Trader.place_buy_order/2
    (hefty) lib/hefty/algos/naive/trader.ex:621: Hefty.Algos.Naive.Trader.place_buy_order("0.24928000", %Hefty.Algos.Naive.Trader.State{budget: nil, buy_down_interval: nil, buy_order: nil, id: nil, pair: nil, profit_interval: nil, rebuy_interval: nil, rebuy_notified: false, retarget_interval: nil, sell_order: nil, stop_loss_interval: nil, stop_loss_triggered: false, strategy: :blank, symbol: nil, trade: nil})
    (hefty) lib/hefty/algos/naive/trader.ex:204: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Last message: %Phoenix.Socket.Broadcast{event: "trade_event", payload: %Hefty.Repo.Binance.TradeEvent{__meta__: #Ecto.Schema.Metadata<:loaded, "trade_events">, buyer_market_maker: true, buyer_order_id: 307168352, event_time: 1574241074997, event_type: "trade", id: "c165553a-da52-4250-b2cc-8b8310b019ba", inserted_at: ~N[2019-11-20 09:11:15], price: "0.24928000", quantity: "1500.00000000", seller_order_id: 307168360, symbol: "XRPUSDT", trade_id: 40929113, trade_time: 1574241074991, updated_at: ~N[2019-11-20 09:11:15]}, topic: "stream-XRPUSDT"}
iex(1)> 09:11:15.159 application=hefty module=Hefty.Algos.Naive.Leader [info] Leader restarts process as it died
iex(1)> 09:11:15.159 application=hefty module=Hefty.Algos.Naive.Leader [info] Trader found in the list of traders. Removing
iex(1)> 09:11:15.159 application=hefty module=Hefty.Algos.Naive.Trader [info] Starting trader on symbol  with budget of 
iex(1)> 09:11:15.270 module=gen_server [error] GenServer #PID<0.1440.0> terminating
** (FunctionClauseError) no function clause matching in Hefty.Algos.Naive.Trader.place_buy_order/2
    (hefty) lib/hefty/algos/naive/trader.ex:621: Hefty.Algos.Naive.Trader.place_buy_order("0.24928000", %Hefty.Algos.Naive.Trader.State{budget: nil, buy_down_interval: nil, buy_order: nil, id: nil, pair: nil, profit_interval: nil, rebuy_interval: nil, rebuy_notified: false, retarget_interval: nil, sell_order: nil, stop_loss_interval: nil, stop_loss_triggered: false, strategy: :blank, symbol: nil, trade: nil})
    (hefty) lib/hefty/algos/naive/trader.ex:204: Hefty.Algos.Naive.Trader.handle_info/2
    (stdlib) gen_server.erl:637: :gen_server.try_dispatch/4
    (stdlib) gen_server.erl:711: :gen_server.handle_msg/6
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3