Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support Binance paper trading for sync sub-command #1605

Merged
merged 1 commit into from Mar 28, 2024
Merged

Conversation

lanphan
Copy link
Contributor

@lanphan lanphan commented Mar 26, 2024

  1. Enable Paper trading in sync command
  2. Refactor pkg/util
  3. take advantage of adshao/go-binance/v2 in order to set URL for paper trading
  • Sync for Binance paper trading works now
❯ ./bbgo sync
[0000]  INFO syncing symbols [ETHUSDT] from session binance
[0000]  INFO syncing binance ETHUSDT trades from 2024-01-01 00:00:00 +0000 UTC...
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.05 @ 3563.64 | AMOUNT 178.182 | FEE 0 ETH | OrderID 1923248 | TID 519244 | Mar 21 16:17:00.022
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.05 @ 3562.79 | AMOUNT 178.1395 | FEE 0 ETH | OrderID 1924216 | TID 519503 | Mar 21 16:21:30.874
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.029 @ 3535.23 | AMOUNT 102.52167 | FEE 0 ETH | OrderID 1928686 | TID 520851 | Mar 21 16:52:40.003
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.021 @ 3535.23 | AMOUNT 74.23983 | FEE 0 ETH | OrderID 1928686 | TID 520852 | Mar 21 16:52:40.029
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.05 @ 3547.33 | AMOUNT 177.3665 | FEE 0 ETH | OrderID 1972861 | TID 531552 | Mar 21 20:35:30.157
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.05 @ 3538.17 | AMOUNT 176.9085 | FEE 0 ETH | OrderID 1973010 | TID 531660 | Mar 21 20:37:00.011
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.05 @ 3547.24 | AMOUNT 177.362 | FEE 0 ETH | OrderID 1973997 | TID 531933 | Mar 21 20:42:23.886
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.0213 @ 3526.51 | AMOUNT 75.114663 | FEE 0 ETH | OrderID 1974883 | TID 532213 | Mar 21 20:48:17.063
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.0287 @ 3526.51 | AMOUNT 101.210837 | FEE 0 ETH | OrderID 1974883 | TID 532214 | Mar 21 20:48:17.084
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.05 @ 3520.83 | AMOUNT 176.0415 | FEE 0 ETH | OrderID 1975005 | TID 532360 | Mar 21 20:55:10.563
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.05 @ 3529.46 | AMOUNT 176.473 | FEE 0 ETH | OrderID 1975689 | TID 532725 | Mar 21 21:11:02.569
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.0288 @ 3493.65 | AMOUNT 100.61712 | FEE 0 ETH | OrderID 2131425 | TID 576386 | Mar 22 10:29:45.764
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.0112 @ 3493.65 | AMOUNT 39.12888 | FEE 0 ETH | OrderID 2131425 | TID 576387 | Mar 22 10:29:48.373
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.0224 @ 3335.88 | AMOUNT 74.723712 | FEE 0 ETH | OrderID 2419189 | TID 649622 | Mar 23 10:03:51.311
[0000]  INFO inserting types.Trade: TRADE binance ETHUSDT  BUY 0.0276 @ 3335.88 | AMOUNT 92.070288 | FEE 0 ETH | OrderID 2419189 | TID 649623 | Mar 23 10:03:51.311
[0000]  INFO syncing binance ETHUSDT orders from 2024-01-01 00:00:00 +0000 UTC...
[0000]  INFO querying closed orders ETHUSDT from 2024-01-01 00:00:00 +0000 UTC <=> 2024-03-26 10:36:41.613071 +0700 +07 m=+0.996091167 ... exchange=binance
[0001]  INFO querying closed orders ETHUSDT from 2024-03-23 10:10:05.357 +0700 +07 <=> 2024-03-26 10:36:41.613071 +0700 +07 m=+0.996091167 ... exchange=binance
[0001]  INFO inserting types.Order: ORDER binance | 1923248 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3563.64 | FILLED | Mar 21 09:16:36.244 -> Mar 21 09:17:00.022
[0001]  INFO inserting types.Order: ORDER binance | 1924216 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3562.79 | FILLED | Mar 21 09:21:05.469 -> Mar 21 09:21:30.874
[0001]  INFO inserting types.Order: ORDER binance | 1928686 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3535.23 | FILLED | Mar 21 09:42:05.688 -> Mar 21 09:52:40.029
[0001]  INFO inserting types.Order: ORDER binance | 1972861 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3547.33 | FILLED | Mar 21 13:35:05.456 -> Mar 21 13:35:30.157
[0001]  INFO inserting types.Order: ORDER binance | 1973010 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3538.17 | FILLED | Mar 21 13:36:05.465 -> Mar 21 13:37:00.011
[0001]  INFO inserting types.Order: ORDER binance | 1973997 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3547.24 | FILLED | Mar 21 13:42:05.606 -> Mar 21 13:42:23.886
[0001]  INFO inserting types.Order: ORDER binance | 1974883 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3526.51 | FILLED | Mar 21 13:48:05.577 -> Mar 21 13:48:17.084
[0001]  INFO inserting types.Order: ORDER binance | 1975005 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3520.83 | FILLED | Mar 21 13:50:05.492 -> Mar 21 13:55:10.563
[0001]  INFO inserting types.Order: ORDER binance | 1975689 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3529.46 | FILLED | Mar 21 14:09:05.397 -> Mar 21 14:11:02.569
[0001]  INFO inserting types.Order: ORDER binance | 2131425 | ETHUSDT | LIMIT BUY  | 0.04/0.04 @ 3493.65 | FILLED | Mar 22 03:29:00.278 -> Mar 22 03:29:48.373
[0001]  INFO inserting types.Order: ORDER binance | 2419189 | ETHUSDT | LIMIT BUY  | 0.05/0.05 @ 3335.88 | FILLED | Mar 23 03:03:30.974 -> Mar 23 03:03:51.311
[0001]  INFO syncing binance deposit records...
  • Run with Binance paper trading works with user data stream synced
❯ ./bbgo run --enable-webserver
[0000]  INFO otp key loaded: Q2PQI**********************JJW6S

To scan your OTP QR code, please run the following command:

	open otp.png

For telegram, send the auth command with the generated one-time password to the bbgo bot you created to enable the notification:

	/auth


For telegram, send the following command to the bbgo bot you created to enable the notification:

	/auth

And then enter your token

	itsme55667788!*2B

[0001]  INFO querying market info from binance... session=binance
[0001]  INFO querying account balances... session=binance
[0001]  INFO account binance balances:
BalanceMap[GTC: 208, LQTY: 269, KLAY: 1666, IMX: 141, QUICK: 5197, LRC: 1154, GHST: 410, ZEC: 14, PHA: 2501, ACA: 2209, PAXG: 1, OCEAN: 420, TIA: 28, MASK: 97, LUNA: 462, PYR: 52, MATIC: 392, CFX: 1495, ILV: 3, OG: 82, ZIL: 12286, HIGH: 208, DYM: 70, OM: 1557, BETA: 4326, SANTOS: 61, CRV: 576, SC: 18446, COS: 18446, FTM: 595, ORDI: 6, CTSI: 1223, GMX: 8, SYS: 1528, ALCX: 12, RPL: 13, ARPA: 5417, BTTC: 18446, NTRN: 324, BRL: 97, CYBER: 42, AR: 12, STRAX: 314, CELO: 341, ID: 359, HBAR: 3743, AUDIO: 1463, NEAR: 63, STEEM: 1391, CAKE: 103, LEVER: 18446, IOTX: 5826, PUNDIX: 716, CVC: 3265, WLD: 50, RONIN: 114, TRU: 5713, BIFI: 1, FIS: 694, WING: 45, MANA: 656, API3: 127, PIXEL: 602, AGLD: 271, ONG: 1070, WOO: 870, BNB: 1, ADA: 657, ROSE: 2893, BAT: 1329, PEPE: 18446, EDU: 441, PEOPLE: 10212, ARK: 402, CKB: 18446, MOVR: 18, CHESS: 1619, ENS: 19, MLN: 19, BADGER: 81, AERGO: 2622, RLC: 111, SUI: 309, WRX: 1382, CTXC: 902, MTL: 208, FLOW: 313, DASH: 11, ATOM: 36, FLUX: 403, PENDLE: 163, QI: 16127, IOST: 18446, SYN: 317, NFP: 540, HFT: 984, DEXE: 49, OMG: 374, ERN: 90, EOS: 419, TRB: 4, EUR: 456, DATA: 6035, APT: 37, CLV: 3830, RDNT: 1225, ARB: 231, KEY: 18446, DAI: 10000, BSW: 2854, UFT: 992, CREAM: 24, PHB: 164, BNT: 489, PSG: 93, SEI: 550, ZAR: 18466, OXT: 2836, VGX: 3222, USDC: 10000, XRP: 727, KSM: 8, DENT: 18446, KAVA: 451, OP: 110, HARD: 1819, REN: 4438, DOT: 45, JOE: 645, XAI: 349, GMT: 1347, BURGER: 573, DIA: 755, HOOK: 309, AEUR: 456, NMR: 11, RAD: 176, FDUSD: 10000, VIB: 4699, ASTR: 3098, MDX: 5688, WAVES: 112, GRT: 1121, FIRO: 213, MBOX: 871, NEO: 27, LOKA: 1156, JASMY: 18446, ARS: 18466, POLS: 464, IDRT: 18466, CHR: 1097, 1000SATS: 18446, TRY: 18466, BLUR: 711, ENJ: 766, FUN: 18446, ATM: 150, STRK: 202, ARKM: 165, AI: 251, PROM: 34, ORN: 208, KP3R: 5, NEXO: 331, MOB: 1223, GNO: 1, UTK: 3982, RARE: 2735, OAX: 2064, C98: 1124, CVX: 99, BLZ: 1311, FRONT: 479, VET: 10139, FXS: 50, WNXM: 6, DEGO: 139, ALGO: 1538, STMX: 18446, LTC: 5, FET: 181, STX: 164, OOKI: 18446, TFUEL: 5383, RON: 18466, UNI: 35, DCR: 15, LOOM: 3451, YGG: 509, RSR: 18446, IQ: 18446, RIF: 1784, AAVE: 3, SPELL: 18446, XEM: 8256, AMB: 18446, ETC: 13, AKRO: 18446, CITY: 123, NULS: 1227, XEC: 18446, SUSHI: 243, LUNC: 18446, SOL: 3, RAY: 383, BEL: 462, BCH: 1, NKN: 2762, DODO: 1599, FIDA: 945, ACM: 181, QTUM: 91, POND: 14183, WAXP: 5109, AXS: 39, BNX: 816, SSV: 9, WAN: 1624, ZEN: 29, VIDT: 10580, THETA: 149, TKO: 906, ONE: 13702, DOGE: 2940, DOCK: 11124, LINA: 18446, MANTA: 130, ALPINE: 187, ANKR: 8975, ONT: 1239, ICX: 1368, TUSD: 10000, CHZ: 3138, OGN: 1981, JTO: 182, WIF: 217, ACE: 40, IDEX: 5563, LIT: 278, VANRY: 1415, ATA: 2893, PERP: 287, WBTC: 1, METIS: 3, ETH: 1.54, BOND: 107, BIDR: 158, GLMR: 721, XNO: 279, XVS: 29, LSK: 235, CELR: 13157, VTHO: 18446, AVAX: 9, STORJ: 568, FIL: 46, IRIS: 11853, MAV: 717, VIC: 528, RUNE: 44, RVN: 15590, KNC: 517, TLM: 16029, LTO: 3643, USTC: 14321, JPY: 0, BICO: 720, COTI: 2288, FARM: 8, POLYX: 1780, LDO: 142, MKR: 1, ALPACA: 1962, ACH: 12838, REQ: 3280, HIFI: 383, DUSK: 1180, DF: 8123, SCRT: 627, TRX: 3798, TWT: 311, LINK: 24, DGB: 18446, PNT: 2278, EPX: 18446, SFP: 617, ALICE: 215, CTK: 503, PYTH: 561, DAR: 1753, BAND: 189, AEVO: 167, REEF: 18446, HIVE: 1085, PLN: 18466, USDP: 500, APE: 200, AUCTION: 18, HOT: 18446, OSMO: 306, KDA: 296, RNDR: 43, PROS: 871, DYDX: 125, GFT: 17574, SKL: 4407, FORTH: 85, BAR: 155, SAND: 660, ADX: 1913, VOXEL: 1335, GLM: 785, JST: 11575, SHIB: 18446, AGIX: 407, GAL: 107, FLOKI: 18446, WIN: 18446, MAGIC: 348, XVG: 18446, STG: 576, AST: 2844, POWR: 1048, SUN: 18446, IOTA: 1268, QKC: 18446, LPT: 24, BAKE: 1005, AVA: 506, BONK: 18446, XLM: 3315, EGLD: 6, SNT: 9023, SXP: 902, MINA: 330, GALA: 6794, QNT: 3, ARDR: 3695, LAZIO: 155, SNX: 100, DREP: 1171, BAL: 82, GNS: 82, VITE: 14068, CVP: 808, ZRX: 535, TROY: 18446, JUV: 167, ICP: 34, ASR: 155, SUPER: 362, XTZ: 330, ALPHA: 2552, FIO: 11137, COMP: 5, PDA: 2307, ALT: 856, PORTAL: 199, COMBO: 413, MDT: 4254, UAH: 18466, UNFI: 55, KMD: 1305, AMP: 18446, FTT: 205, BTC: 1, YFI: 1, ELF: 712, GAS: 62, FOR: 13725, BEAMX: 12475, UMA: 102, FLM: 3506, PIVX: 1063, REI: 7208, USDT: 8099.9, AXL: 229, T: 12639, INJ: 9, JUP: 577, MEME: 11123, PORTO: 159, 1INCH: 735, MBL: 18446, WBETH: 1, STPT: 5903, SLP: 18446] session=binance
[0001]  INFO syncing symbols [ETHUSDT] from session binance
[0001]  INFO syncing binance ETHUSDT trades from 2024-01-01 00:00:00 +0000 UTC...
[0001]  INFO syncing binance ETHUSDT orders from 2024-01-01 00:00:00 +0000 UTC...
[0001]  INFO querying closed orders ETHUSDT from 2024-03-23 03:03:30.974 +0000 UTC <=> 2024-03-26 10:50:40.881392 +0700 +07 m=+1.655627251 ... exchange=binance
[0001]  INFO querying closed orders ETHUSDT from 2024-03-23 10:10:05.357 +0700 +07 <=> 2024-03-26 10:50:40.881392 +0700 +07 m=+1.655627251 ... exchange=binance
[0002]  INFO syncing binance deposit records...
[0002]  INFO Deposit is not supported in paper trading
[0002]  INFO syncing binance withdraw records...
[0002]  INFO Withdraw is not supported in paper trading
[0002]  INFO syncing binance reward records...
[0002]  INFO Reward is not supported in paper trading
[0002]  INFO attaching strategy *bollmaker.Strategy on binance...
[0002]  INFO loading strategies states...
[0002]  INFO found symbol ETHUSDT based strategy from bollmaker.Strategy
[0002]  INFO querying kline ETHUSDT 5m {1000 <nil> 2024-03-26 10:50:39.344694 +0700 +07 m=+0.118902084} exchange=binance
[0002]  INFO querying kline ETHUSDT 1m {1000 <nil> 2024-03-26 10:50:39.344694 +0700 +07 m=+0.118902084} exchange=binance
[0003]  INFO querying kline ETHUSDT 1h {1000 <nil> 2024-03-26 10:50:39.344694 +0700 +07 m=+0.118902084} exchange=binance
[0003]  INFO ETHUSDT last price: 3641.4

@bbgokarma-bot
Copy link

Welcome back! @lanphan, This pull request may get 267 BBG.

Copy link

codecov bot commented Mar 26, 2024

Codecov Report

Attention: Patch coverage is 6.25000% with 15 lines in your changes are missing coverage. Please review.

Project coverage is 22.61%. Comparing base (f246077) to head (37a0ae5).
Report is 5 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1605      +/-   ##
==========================================
- Coverage   22.63%   22.61%   -0.02%     
==========================================
  Files         621      621              
  Lines       44903    44913      +10     
==========================================
- Hits        10164    10159       -5     
- Misses      34009    34022      +13     
- Partials      730      732       +2     
Files Coverage Δ
pkg/bbgo/environment.go 1.28% <ø> (+<0.01%) ⬆️
pkg/exchange/binance/exchange.go 5.77% <25.00%> (-0.26%) ⬇️
pkg/service/sync.go 0.00% <0.00%> (ø)

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f246077...37a0ae5. Read the comment docs.

@bbgokarma-bot
Copy link

Re-estimated karma: this pull request may get 283 BBG

const BinanceUSWebSocketURL = "wss://stream.binance.us:9443"
const WebSocketURL = "wss://stream.binance.com:9443"
const WebSocketTestURL = "wss://testnet.binance.vision"
const FutureTestBaseURL = "https://testnet.binancefuture.com"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are still required because we want to replace adshao/binance with the requestgen api

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I'll keep it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all constants containing test url are kept, done

if paperTrade() {
client.BaseURL = BinanceTestBaseURL
futuresClient.BaseURL = FutureTestBaseURL
}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is still required

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need

my new code is added before create new binance client as followings

        if util.IsPaperTrade() {
		binance.UseTestnet = true
	}
	var client = binance.NewClient(key, secret)

it'll help to set BaseURL of binance client to its test base url. I did run it and it works.

@@ -88,6 +89,10 @@ func (s *SyncService) SyncRewardHistory(ctx context.Context, exchange types.Exch
}

log.Infof("syncing %s reward records...", exchange.Name())
if util.IsPaperTrade() {
log.Info("Reward is not supported in paper trading")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

such check is done by checking the interface, you can check pkg/types/exchange.go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no check there
In fact, when I don't have that code, sync progress is corrupted with error below

[0001]  INFO syncing binance reward records...
Error: request failed with status code: 404, body: ""
[0002] FATAL cannot execute command error=request failed with status code: 404, body: ""

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inside the QueryRewardHistory method, return empty slice for paper trading? and actually you can turn off the reward history sync ;-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inside the QueryRewardHistory method --> there is no method with name QueryRewardHistory

and actually you can turn off the reward history sync --> you mean there is env to on/off this? ah, I found it, rewardHistory: true

At last, I think this code is ok at this time, just because it has the same pattern with deposit / withdrawn methods

@bbgokarma-bot
Copy link

Re-estimated karma: this pull request may get 364 BBG

@bbgokarma-bot
Copy link

Re-estimated karma: this pull request may get 363 BBG

@bbgokarma-bot
Copy link

Re-estimated karma: this pull request may get 378 BBG

@@ -304,7 +304,7 @@ func (s *Stream) getEndpointUrl(listenKey string) string {

if s.IsFutures {
url = FuturesWebSocketURL + "/ws"
} else if isBinanceUs() {
} else if util.IsBinanceUs() {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isBinanceUs should only be used inside the binance package, only common functions could be moved into the util package

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, moved back, done

@@ -109,6 +119,11 @@ func (s *SyncService) SyncDepositHistory(ctx context.Context, exchange types.Exc

func (s *SyncService) SyncWithdrawHistory(ctx context.Context, exchange types.Exchange, startTime time.Time) error {
log.Infof("syncing %s withdraw records...", exchange.Name())
if util.IsPaperTrade() {
log.Info("Withdraw is not supported in paper trading")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

log messages start with lower case :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@@ -97,6 +102,11 @@ func (s *SyncService) SyncRewardHistory(ctx context.Context, exchange types.Exch

func (s *SyncService) SyncDepositHistory(ctx context.Context, exchange types.Exchange, startTime time.Time) error {
log.Infof("syncing %s deposit records...", exchange.Name())
if util.IsPaperTrade() {
log.Info("Deposit is not supported in paper trading")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto, log messages start with lower case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@bbgokarma-bot
Copy link

Re-estimated karma: this pull request may get 443 BBG

Copy link
Owner

@c9s c9s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great! thanks!

@c9s c9s merged commit f300791 into c9s:main Mar 28, 2024
3 of 5 checks passed
@bbgokarma-bot
Copy link

Hi @lanphan,

Well done! 458 BBG has been sent to your polygon wallet. Please check the following tx:

https://polygonscan.com/tx/0x898a7a144065e12bacbf304b2e0f62707123e97a57a7a0230cc288c2f77c6ba0

Thank you for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants