Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pool.terminate() Crashing Entire Application #442

Open
Camuvingian opened this issue Aug 11, 2022 · 0 comments
Open

pool.terminate() Crashing Entire Application #442

Camuvingian opened this issue Aug 11, 2022 · 0 comments

Comments

@Camuvingian
Copy link

I have the following code to spin up a workerpool.

const pool = Pool(
	() => spawn(new Worker('../workers/createCubeMapFaceWorker')),
	this.#cubeMapWorkerCount > Constants.FacePositions.length // Only makes sense to spin-up 6 - anymore and they are idle.
		? Constants.FacePositions.length
		: this.#cubeMapWorkerCount
);
logger?.info(`${this.#cubeMapWorkerCount} workers spun-up successfully`);

let workerId = 1;
const promises = Constants.FacePositions.map(async (face: 'nx' | 'ny' | 'nz' | 'px' | 'py' | 'pz') => {
	return pool.queue(async ({ createCubeMapFaceWorker }) => {
		return await createCubeMapFaceWorker(
			path,
			userDetails.organizationDetails,
			requestDto.tourId,
			requestDto.cameraId,
			requestDto.imageVersionName,
			requestDto.resolutions,
			requestDto.interpolation,
			requestDto.maxWidth,
			requestDto.sharpenLevel,
			requestDto.compression,
			face,
			this.#azureStorageOptions,
			workerId++
		);
	});
});

try {
	await Promise
		.all(promises)
		.then(async (results: CubeMapResult[]) => {

			let insertUpdateResult = await this.inserUpdateTourImageAudit(results.flatMap(r => r), requestDto);
			if (insertUpdateResult.succeeded == null || !insertUpdateResult.succeeded || insertUpdateResult.errors.length > 0) {

				responseDto.errors = insertUpdateResult.errors;
				responseDto.succeeded = false;

				logger?.warning(`Upload to toursImageAudit collection failed (tourId = ${requestDto.tourId})`);
			}

			this.checkAndConstructErrors(results, responseDto);

			if (responseDto.errors.length > 0) {

				responseDto.succeeded = false;
				httpStatusCode = HttpStatusCode.BadRequest;

				throw new Error('Cloud persistance and/or database-arbiter failure');
			}
			responseDto.succeeded = true;
		})
		.catch((exception) => {
			var exceptionDetails = Utils.getExceptionMessage(exception);
			let message = `${exceptionDetails.message}\n${exceptionDetails.stack}`;

			responseDto.errors.push(new RequestError(
				HttpStatusCode[HttpStatusCode.BadRequest],
				message
			));
			responseDto.succeeded = false;
			httpStatusCode = HttpStatusCode.BadRequest;

			logger?.error(message);
		});
	logger?.info(`Cubemap processing completed on ${this.#cubeMapWorkerCount} workers, terminating...`);

	await pool.completed(true);
	await pool.terminate();

	logger?.info(`${this.#cubeMapWorkerCount} workers completed successfully`);
}
finally { 
	const dir = getImageDir(requestDto.tourId, requestDto.cameraId);
	if (fs.existsSync(dir)) {
		logger?.info(`Attempting to remove temporary local storage...`);
		fs.rmSync(dir, {
			recursive: true, 
			force: true
		});
		logger?.info(`Temporary local storage directory \"${dir}\" removed successfully`);
	}
}

This is running great, but the entire program crashes on await pool.terminate();. This does not happen if I use await pool.completed(); but then the app just hangs.

I cannot seem to debug the cause of this and there is an idential issue here (#247). Any help here would be appreciated as the code works perfectly apart from this and I might be doing something silly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant