[Feature] non-blocking mode for polling

[zhzLuke96]

When using the Telegraf library to implement a Telegram bot, I noticed that my bot would frequently hang and appear unresponsive, seemingly due to network issues. At first it was able to receive updates normally, but after I started handling chat_member update types, the bot became extremely slow and was basically non-functional.

I initially thought it was my network environment, but after debugging with packet capture I realized... it wasn't even sending getUpdates requests! I eventually tracked down the culprit to this code

here:

telegraf/src/core/network/polling.ts

Lines 77 to 89 in 087658a

	async loop(handleUpdate: (updates: tg.Update) => Promise<void>) {
	if (this.abortController.signal.aborted)
	throw new Error('Polling instances must not be reused!')
	try {
	for await (const updates of this)
	await Promise.all(updates.map(handleUpdate))
	} finally {
	debug('Long polling stopped')
	// prevent instance reuse
	this.stop()
	await this.syncUpdateOffset().catch(noop)
	}
	}

In this loop,each Array<Update> wait for the previous batch of updates to be processed before processing the next batch of updates
And my bot logic has a lot of waiting queues like this:

bot.on('text', async (ctx) => {
  const user_queue = get_queue(ctx);
  await user_quque.wait();
  // do something
});

which causes the entire handleUpdate calling to be blocked...

Clearly this severely limits the throughput of the bot. As a workaround I'm currently using a middleware to wrap all handleUpdate calls and catch errors separately, like this:

const quick_message: Middleware<Context> = async (ctx, next) => {
  next().catch((error) => {
  my_GlobalErrorCatcher.catch(error);
  });
};

So I'm thinking, should there be an option to switch between blocking-polling and non-blocking-polling?

like this

bot.launch({
  nonBlockingPolling: true,
});

I think it's worth bringing up to improve performance. and blocked-polling will cause the behavior of the bot to be inconsistent with the webhook mode

[shahradelahi]

Hi there! I understand what you're trying to achieve, and I have a suggestion that might be helpful. Instead of using the "blocking" approach, have you considered creating another async function inside the event callback? Here's an example of what I mean:

bot.on('text', (ctx) => {
  (async () => {
    // You can do something that takes a long time to process here
  })().catch(handleError);
});

This way, you can benefit from both blocking and non-blocking. Let me know if this helps!

[MKRhere]

Just remember you cannot access session from the async function because middleware handler will close before the unhandled function completes running.

[zhzLuke96]

Thank you suggestions. The solution proposed by @shahradelahi to create an async function inside the event callback is essentially another form of the quick_message middleware that I have implemented, as shown in the following code:

const quick_message: Middleware<Context> = async (ctx, next) => {
  next().catch((error) => {
    my_GlobalErrorCatcher.catch(error);
  });
};

// quick_message is top middleware
app.use(quick_message)
app.on('text', (ctx) => { ... })

While @shahradelahi's approach is indeed worth trying (and In fact, my current code uses quick_message to support parallel message processing in polling mode.), as @MKRhere pointed out, it may risk losing access to the session context as the middleware handler would close before the unhandled function completes.

In fact, my current middleware design is already coupled with the bot's message processing logic, as shown in the following code:

bot.on('text', async (ctx, next) => {
  const user_queue = get_queue(ctx);
  const status = await user_queue.wait();
  
  if (status.ok) {
    // Proceed to the next middleware
    return next();
  }
  
  // Not OK, alert the user
});

middleware includes complex filters, loggers, or checkers. This pattern works seamlessly when using the Webhook mode since the processing of each message is done in parallel.
However, when using the polling mode, a performance bottleneck arises. This is because the polling mode processes a batch of messages (potentially multiple messages from multiple different users from getUpdates) at once, while the webhook mode processes only one message at a time, making the webhook mode inherently parallel, whereas processing a getUpdates batch in the polling mode is not parallel.

so, Due issues mentioned above, I suggest refactoring the polling.ts file to support parallel message processing in the polling mode.

[mr-kenikh]

@zhzLuke96 hmmm, I use webhooks, but it also get stuck in the case of heavy middleware

const { Telegraf } = require('telegraf');
const { message } = require('telegraf/filters');

const wait = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const print = (...args) => console.log(new Date().toLocaleTimeString(), ...args);

const domain = 'example.com';
const port = 7701;
const token = '123456789:ABC...';

async function main() {
  const bot = new Telegraf(token);

  bot.use(async (ctx, next) => {
    print('Handling update from', ctx.from?.username, '...');
    await next();
    print('Update from', ctx.from?.username, 'handled');
  });

  bot.on(message('text'), async (ctx) => {
    await ctx.reply('Just wait a bit...');
    await wait(10_000);
    await ctx.reply('Done!');
  });


  await bot.launch({ webhook: { domain, port: port } });
  console.log('Webhook bot listening on port', port);
}

main();

I texted the bot from two accounts at the same time:

5:22:05 PM Handling update from alice ...
5:22:15 PM Update from alice handled
5:22:15 PM Handling update from bob ...
5:22:25 PM Update from bob handled

But expected something like this:

5:22:05 PM Handling update from alice ...
5:22:05 PM Handling update from bob ...
5:22:15 PM Update from alice handled
5:22:15 PM Update from bob handled