Possible deadlock in TParallelUnorderedTableWriter

[chegoryu]

Assume you added options.ThreadCount_ + 1 write tasks to writer.

options.ThreadCount_ tasks are running now and the last one is waiting in the ThreadPool_ queue, i.e. all TaskFutures_ are not ready and this part of code always ended up in else { break; }

ytsaurus/yt/cpp/mapreduce/library/parallel_io/parallel_writer-inl.h

Lines 204 to 214 in d2ed627

	while (TaskFutures_.size() > 0) {
	if (TaskFutures_.front().HasValue()) {
	TaskFutures_.pop_front();
	continue;
	} else if (TaskFutures_.front().HasException()) {
	Stopped_ = true;
	TaskFutures_.front().TryRethrow();
	} else {
	break;
	}
	}

Then you call Finish() and start waiting for the completion of the task here:

ytsaurus/yt/cpp/mapreduce/library/parallel_io/parallel_writer-inl.h

Line 171 in d2ed627

::NThreading::WaitAll(TaskFutures_).GetValueSync();

After that something went wrong, for example cluster downtime/network problems/etc.

Nothing will be returned to the WritersPool_ due to exception from task.Process:

ytsaurus/yt/cpp/mapreduce/library/parallel_io/parallel_writer-inl.h

Lines 231 to 235 in d2ed627

	task.Process(writer);
	// If exception in Process is thrown parallel writer is going to be aborted.
	// No need to Release writer in this case.
	WriterPool_.Release(std::move(writer));

And after first failed write task the last task (with options.ThreadCount_ + 1 number) will start waiting here:

ytsaurus/yt/cpp/mapreduce/library/parallel_io/parallel_writer-inl.h

Lines 251 to 253 in d2ed627

	while (WriterPool_.empty() && PoolSize_ >= MaxPoolSize_) {
	HasWriterCV_.Wait(Lock_);
	}

But no one will wake up this task here:

ytsaurus/yt/cpp/mapreduce/library/parallel_io/parallel_writer-inl.h

Line 273 in d2ed627

HasWriterCV_.Signal();

And also WriterPool_ will always be empty.