feat: move webhook handler to worker #8525
Conversation
WalkthroughWalkthroughThe recent update to the NocoDB project introduces significant enhancements and restructuring related to job handling and webhook processing. Key changes include the removal of Changes
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (15)
- packages/nocodb/src/app.module.ts (2 hunks)
- packages/nocodb/src/interface/Jobs.ts (1 hunks)
- packages/nocodb/src/modules/global/global.module.ts (4 hunks)
- packages/nocodb/src/modules/jobs/fallback/fallback-queue.service.ts (3 hunks)
- packages/nocodb/src/modules/jobs/jobs-service.interface.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs.controller.ts (3 hunks)
- packages/nocodb/src/modules/jobs/jobs.module.ts (2 hunks)
- packages/nocodb/src/modules/jobs/jobs/at-import/at-import.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/export-import/duplicate.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/health-check.processor.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/meta-sync/meta-sync.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/source-create/source-create.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/source-delete/source-delete.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/webhook-handler/webhook-handler.processor.ts (1 hunks)
- packages/nocodb/src/services/hook-handler.service.ts (4 hunks)
Files skipped from review due to trivial changes (3)
- packages/nocodb/src/interface/Jobs.ts
- packages/nocodb/src/modules/jobs/jobs-service.interface.ts
- packages/nocodb/src/modules/jobs/jobs/source-delete/source-delete.controller.ts
Additional comments not posted (27)
packages/nocodb/src/modules/jobs/jobs/health-check.processor.ts (3)
4-4: Ensure that theIJobsServiceinterface is correctly implemented in theJobsService.
11-11: Injection ofJobsServiceasIJobsServiceensures decoupling and flexibility. Good use of dependency injection.
16-16: Check if thejobsQueueproperty is properly exposed inIJobsServiceto support this usage.Verification successful
The
jobsQueueproperty is indeed defined in theIJobsServiceinterface, as shown in the output of the shell script. This confirms that the property is properly exposed to support its usage in thehealth-check.processor.tsfile.
jobsQueueis defined inpackages/nocodb/src/modules/jobs/jobs-service.interface.tsat line 5.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the exposure of `jobsQueue` in `IJobsService`. # Test: Search for the `jobsQueue` property in `IJobsService`. Expect: Property should be defined. ast-grep --lang typescript --pattern $'interface IJobsService { $$$ jobsQueue: $_; $$$ }'Length of output: 1080
packages/nocodb/src/modules/jobs/jobs/webhook-handler/webhook-handler.processor.ts (3)
1-6: Imports and class setup are correct. Ensure thatHookHandlerServiceis properly configured to be injectable.
12-13: UsingforwardRefis necessary to resolve the circular dependency. Good implementation.
16-29: Ensure that thehandleHooksmethod inHookHandlerServicecan handle the data structure provided injob.data.Verification successful
The
handleHooksmethod inHookHandlerServiceis designed to handle the data structure provided injob.data. It accepts an object with properties:hookName,prevData,newData,user,viewId,modelId, andtnPath, and processes them accordingly.
- The method retrieves the view and model based on
viewIdandmodelId.- It handles form view data submission if the
hookNamematches specific conditions.- It sends emails if configured and logs errors if any occur.
- It invokes webhooks based on the
hookNameand logs errors if any occur.This confirms that the
handleHooksmethod can handle the data structure provided injob.data.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify that `handleHooks` can handle the data structure. # Test: Search for the `handleHooks` method in `HookHandlerService`. Expect: Method should accept the data structure. ast-grep --lang typescript --pattern $'class HookHandlerService { $$$ handleHooks($_) { $$$ } $$$ }'Length of output: 115
Script:
#!/bin/bash # Locate the `handleHooks` method in the codebase and inspect its implementation. # Search for the `handleHooks` method definition in TypeScript files. rg 'handleHooks' --glob '*.ts' # Once located, extract the method implementation to verify if it handles the expected data structure. ast-grep --lang typescript --pattern 'function handleHooks($_) { $$$ }'Length of output: 1613
Script:
#!/bin/bash # Extract and inspect the `handleHooks` method implementation from `packages/nocodb/src/services/hook-handler.service.ts`. # Extract the `handleHooks` method implementation. ast-grep --lang typescript --pattern 'class HookHandlerService { $$$ handleHooks({ hookName, prevData, newData, user, viewId, modelId, tnPath }) { $$$ } $$$ }' packages/nocodb/src/services/hook-handler.service.tsLength of output: 214
Script:
#!/bin/bash # Locate the `handleHooks` method definition in `packages/nocodb/src/services/hook-handler.service.ts` and inspect its implementation. # Search for the `handleHooks` method definition. rg 'handleHooks' packages/nocodb/src/services/hook-handler.service.ts # Extract the method implementation to verify if it handles the expected data structure. ast-grep --lang typescript --pattern 'handleHooks({ hookName, prevData, newData, user, viewId, modelId, tnPath }) { $$$ }' packages/nocodb/src/services/hook-handler.service.tsLength of output: 832
Script:
#!/bin/bash # Locate the `handleHooks` method definition in `packages/nocodb/src/services/hook-handler.service.ts` and inspect its implementation. # Search for the `handleHooks` method definition. rg 'handleHooks' packages/nocodb/src/services/hook-handler.service.ts # Extract the method implementation to verify if it handles the expected data structure. sed -n '/handleHooks/,/}/p' packages/nocodb/src/services/hook-handler.service.tsLength of output: 755
Script:
#!/bin/bash # Fully extract the `handleHooks` method implementation from `packages/nocodb/src/services/hook-handler.service.ts`. # Extract the method implementation to verify if it handles the expected data structure. sed -n '/public async handleHooks/,/^}/p' packages/nocodb/src/services/hook-handler.service.tsLength of output: 4081
packages/nocodb/src/modules/jobs/jobs/source-create/source-create.controller.ts (2)
18-18: Correct import ofIJobsService. This aligns with the new architecture for job handling.
24-24: Proper use of dependency injection forIJobsService. This ensures that the controller is decoupled from the specific implementation of job services.packages/nocodb/src/modules/jobs/jobs/at-import/at-import.controller.ts (2)
16-16: Correct import ofIJobsService. Consistent with other job-related controllers.
22-22: Proper use of dependency injection forIJobsService. This ensures that the controller is decoupled from the specific implementation of job services.packages/nocodb/src/modules/global/global.module.ts (2)
15-17: Addition ofHookHandlerServiceandJobsModuleto the global module. This centralizes the hook handling and job management, aligning with the PR's objectives.
41-41: The configuration of imports, providers, and exports is correct. Ensure that all necessary services are properly initialized and available.Also applies to: 51-51, 60-60
Verification successful
The
globalModuleMetadatainglobal.module.tsincludes the necessary imports, providers, and exports. The configuration appears to be correct, ensuring that all necessary services are properly initialized and available.
- Imports:
EventEmitterModule,forwardRef(() => UsersModule),JobsModule- Providers:
InitMetaServiceProvider,AppHooksService,JwtStrategyProvider,GlobalGuard,SocketGateway,AppHooksListenerService,TelemetryService,HookHandlerService- Exports:
MetaService,JwtStrategyProvider,GlobalGuard,AppHooksService,AppHooksListenerService,TelemetryService,HookHandlerServiceScripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the initialization and availability of services in the global module. # Test: Check the module configuration for proper service initialization. Expect: All services should be initialized. rg --type typescript $'@Module({ $$$ imports: [$$$], providers: [$$$], exports: [$$$] })'Length of output: 428
Script:
#!/bin/bash # Description: Verify the initialization and availability of services in the global module. # Test: Check the module configuration for proper service initialization. Expect: All services should be initialized. rg --type typescript '@Module\(\{[^}]*imports: \[[^]]*\],[^}]*providers: \[[^]]*\],[^}]*exports: \[[^]]*\]\s*\}\)'Length of output: 151
Script:
#!/bin/bash # Description: Verify the initialization and availability of services in the global module. # Test: Check the module configuration for proper service initialization. Expect: All services should be initialized. rg --glob '*.ts' '@Module\(\{[^}]*imports: \[[^]]*\],[^}]*providers: \[[^]]*\],[^}]*exports: \[[^]]*\]\s*\}\)'Length of output: 112
Script:
#!/bin/bash # Description: Verify the initialization and availability of services in the global module. # Test: Check the module configuration for proper service initialization. Expect: All services should be initialized. rg --glob '*.ts' '@Module\(\{[^}]*imports: \[[^]]*\]'Length of output: 55
Script:
#!/bin/bash # Description: Verify the initialization and availability of services in the global module. # Test: Check the module configuration for proper service initialization. Expect: All services should be initialized. rg --glob '*.ts' '@Module\('Length of output: 596
Script:
#!/bin/bash # Description: Extract the content of the @Module decorator in global.module.ts to verify the configuration. # Extract the @Module decorator content from global.module.ts rg --glob 'packages/nocodb/src/modules/global/global.module.ts' -A 20 '@Module\('Length of output: 244
Script:
#!/bin/bash # Description: Locate the definition of globalModuleMetadata and verify its configuration. # Locate the definition of globalModuleMetadata rg --glob '*.ts' 'globalModuleMetadata'Length of output: 211
Script:
#!/bin/bash # Description: Extract the content of globalModuleMetadata to verify its configuration. # Extract the content of globalModuleMetadata from global.module.ts rg --glob 'packages/nocodb/src/modules/global/global.module.ts' -A 20 'export const globalModuleMetadata ='Length of output: 1708
packages/nocodb/src/modules/jobs/jobs/meta-sync/meta-sync.controller.ts (2)
16-16: Correct import ofIJobsService. This aligns with the new architecture for job handling.
22-22: Proper use of dependency injection forIJobsService. This ensures that the controller is decoupled from the specific implementation of job services.packages/nocodb/src/app.module.ts (1)
Line range hint
1-1: The configuration of the application module is correct. Ensure that all modules and services are properly initialized.Verification successful
The
ceModuleConfiginapp.module.tsincludes the necessary imports and configurations, indicating that all modules and services are properly initialized.
imports: IncludesGlobalModule,UsersModule,AuthModule,MetasModule,DatasModule,EventEmitterModule,JobsModule,NestJsEventEmitter,ConfigModule, and conditionallySentryModule.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the proper initialization of modules and services in the application module. # Test: Check the module configuration for proper service initialization. Expect: All services should be initialized. rg --type typescript $'@Module({ $$$ imports: [$$$], providers: [$$$], exports: [$$$] })'Length of output: 428
Script:
#!/bin/bash # Description: Verify the proper initialization of modules and services in the application module. # Test: Check the module configuration for proper service initialization. Expect: All services should be initialized. rg --type typescript '@Module\(\{[^}]*imports: \[[^]]*\],[^}]*providers: \[[^]]*\],[^}]*exports: \[[^]]*\]\}\)'Length of output: 148
Script:
#!/bin/bash # Description: Verify the proper initialization of modules and services in the application module. # Test: Check the module configuration for proper service initialization. Expect: All services should be initialized. rg --type ts '@Module\(\{[^}]*imports: \[[^]]*\],[^}]*providers: \[[^]]*\],[^}]*exports: \[[^]]*\]\}\)'Length of output: 105
Script:
#!/bin/bash # Description: Locate the @Module decorator and inspect the module configuration. # Test: Search for the @Module decorator in TypeScript files. rg --type ts '@Module\('Length of output: 592
Script:
#!/bin/bash # Description: Locate and inspect the definition of ceModuleConfig in the app.module.ts file. # Test: Search for the definition of ceModuleConfig in TypeScript files. rg --type ts 'ceModuleConfig'Length of output: 153
Script:
#!/bin/bash # Description: Extract and display the contents of ceModuleConfig in the app.module.ts file. # Test: Display the definition of ceModuleConfig. rg --type ts 'export const ceModuleConfig = {' -A 20 packages/nocodb/src/app.module.tsLength of output: 214
Script:
#!/bin/bash # Description: Extract and display the contents of ceModuleConfig in the app.module.ts file. # Test: Display the definition of ceModuleConfig. rg --type ts 'export const ceModuleConfig = \{' -A 20 packages/nocodb/src/app.module.tsLength of output: 665
packages/nocodb/src/modules/jobs/jobs.module.ts (2)
17-17: Addition ofWebhookHandlerProcessorto the jobs module. This supports the new functionality for handling webhooks.
81-81: Ensure thatWebhookHandlerProcessoris properly configured and initialized in the jobs module.Verification successful
The
WebhookHandlerProcessoris properly configured and initialized in theJobsModule. It is listed in theprovidersarray within theJobsModuleMetadataobject.
WebhookHandlerProcessoris included in theprovidersarray inpackages/nocodb/src/modules/jobs/jobs.module.ts.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the configuration and initialization of `WebhookHandlerProcessor` in the jobs module. # Test: Check the module configuration for proper processor initialization. Expect: Processor should be initialized. rg --type typescript $'@Module({ $$$ processors: [$$$], $$$ })'Length of output: 369
Script:
#!/bin/bash # Description: Verify the configuration and initialization of `WebhookHandlerProcessor` in the jobs module. # Test: Check the module configuration for proper processor initialization. Expect: Processor should be initialized. rg --type typescript '@Module\(\{[^}]*providers: \[[^]]*WebhookHandlerProcessor[^]]*\][^}]*\}\)' packages/nocodb/src/modules/jobs/jobs.module.tsLength of output: 181
Script:
#!/bin/bash # Description: Verify the configuration and initialization of `WebhookHandlerProcessor` in the jobs module. # Test: Check the module configuration for proper processor initialization. Expect: Processor should be initialized. rg '@Module\(\{[^}]*providers: \[[^]]*WebhookHandlerProcessor[^]]*\][^}]*\}\)' packages/nocodb/src/modules/jobs/jobs.module.tsLength of output: 128
Script:
#!/bin/bash # Description: Print the contents of the jobs.module.ts file to manually verify the presence of `WebhookHandlerProcessor` in the `providers` array. # Print the contents of the jobs.module.ts file cat packages/nocodb/src/modules/jobs/jobs.module.tsLength of output: 3553
packages/nocodb/src/modules/jobs/fallback/fallback-queue.service.ts (2)
35-35: The addition ofwebhookHandlerProcessorto the constructor enhances modularity by handling webhook-related jobs separately.
93-96: Integration ofHandleWebhookjob type ensures that webhook handling is managed as part of the job system, improving scalability and maintainability.packages/nocodb/src/services/hook-handler.service.ts (6)
1-1: Updated imports and dependencies align with the new responsibilities ofHookHandlerServiceto handle jobs, enhancing cohesion.Also applies to: 14-15
21-21: Initialization ofloggerand injection ofjobsServicein the constructor are crucial for logging and job management within the service.Also applies to: 26-26
118-122: Enhanced error logging inhandleHooksmethod improves debuggability and error traceability, which is essential for maintaining robustness in webhook handling.
147-173: ThetriggerHookmethod efficiently delegates webhook handling to the job system, encapsulating the logic in a maintainable way.
178-178: Binding thetriggerHookmethod to theHANDLE_WEBHOOKevent ensures that webhook triggers are processed asynchronously, enhancing system responsiveness.
139-143: Robust error logging for webhook invocation errors aids in quick identification and resolution of issues during webhook processing.packages/nocodb/src/modules/jobs/jobs/export-import/duplicate.controller.ts (1)
20-20: The integration ofIJobsServiceensures a consistent and type-safe approach to job management across different controllers, aligning with best practices.Also applies to: 26-26
packages/nocodb/src/modules/jobs/jobs.controller.ts (1)
24-24: The integration ofIJobsServiceinJobsControllerstandardizes job management and enhances modularity, which is crucial for maintaining clean architecture.Also applies to: 35-35
| @@ -6,6 +6,7 @@ import { AtImportProcessor } from '~/modules/jobs/jobs/at-import/at-import.proce | |||
| import { MetaSyncProcessor } from '~/modules/jobs/jobs/meta-sync/meta-sync.processor'; | |||
| import { SourceCreateProcessor } from '~/modules/jobs/jobs/source-create/source-create.processor'; | |||
| import { SourceDeleteProcessor } from '~/modules/jobs/jobs/source-delete/source-delete.processor'; | |||
| import { WebhookHandlerProcessor } from '~/modules/jobs/jobs/webhook-handler/webhook-handler.processor'; | |||
There was a problem hiding this comment.
Tip
Codebase Verification
Ensure that WebhookHandlerProcessor is correctly implemented and tested since it's now part of the job handling system.
The implementation of WebhookHandlerProcessor is present, but there are no tests found for it. Additionally, there are no indirect references to WebhookHandlerProcessor in the test files.
- Please add unit tests for
WebhookHandlerProcessorto ensure its functionality is properly verified.
Analysis chain
Ensure that WebhookHandlerProcessor is correctly implemented and tested since it's now part of the job handling system.
Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify the implementation and test coverage for `WebhookHandlerProcessor`.
# Test: Search for the implementation and tests of `WebhookHandlerProcessor`.
fd 'webhook-handler.processor.ts' --exec cat {}
fd 'webhook-handler.processor.spec.ts' --exec cat {}
Length of output: 950
Script:
#!/bin/bash
# Description: Verify if `WebhookHandlerProcessor` is tested indirectly in other test files.
# Search for references to `WebhookHandlerProcessor` in the test files.
rg 'WebhookHandlerProcessor' --type ts
Length of output: 848
|
Uffizzi Preview |
packages/nocodb/src/modules/jobs/jobs/webhook-handler/webhook-handler.processor.ts
Outdated
Show resolved
Hide resolved
Signed-off-by: mertmit <mertmit99@gmail.com>
Signed-off-by: mertmit <mertmit99@gmail.com>
There was a problem hiding this comment.
Actionable comments posted: 0
Review Details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (15)
- packages/nocodb/src/app.module.ts (2 hunks)
- packages/nocodb/src/interface/Jobs.ts (3 hunks)
- packages/nocodb/src/modules/global/global.module.ts (4 hunks)
- packages/nocodb/src/modules/jobs/fallback/fallback-queue.service.ts (3 hunks)
- packages/nocodb/src/modules/jobs/jobs-service.interface.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs.controller.ts (3 hunks)
- packages/nocodb/src/modules/jobs/jobs.module.ts (2 hunks)
- packages/nocodb/src/modules/jobs/jobs/at-import/at-import.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/export-import/duplicate.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/health-check.processor.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/meta-sync/meta-sync.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/source-create/source-create.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/source-delete/source-delete.controller.ts (1 hunks)
- packages/nocodb/src/modules/jobs/jobs/webhook-handler/webhook-handler.processor.ts (1 hunks)
- packages/nocodb/src/services/hook-handler.service.ts (3 hunks)
Files skipped from review as they are similar to previous changes (15)
- packages/nocodb/src/app.module.ts
- packages/nocodb/src/interface/Jobs.ts
- packages/nocodb/src/modules/global/global.module.ts
- packages/nocodb/src/modules/jobs/fallback/fallback-queue.service.ts
- packages/nocodb/src/modules/jobs/jobs-service.interface.ts
- packages/nocodb/src/modules/jobs/jobs.controller.ts
- packages/nocodb/src/modules/jobs/jobs.module.ts
- packages/nocodb/src/modules/jobs/jobs/at-import/at-import.controller.ts
- packages/nocodb/src/modules/jobs/jobs/export-import/duplicate.controller.ts
- packages/nocodb/src/modules/jobs/jobs/health-check.processor.ts
- packages/nocodb/src/modules/jobs/jobs/meta-sync/meta-sync.controller.ts
- packages/nocodb/src/modules/jobs/jobs/source-create/source-create.controller.ts
- packages/nocodb/src/modules/jobs/jobs/source-delete/source-delete.controller.ts
- packages/nocodb/src/modules/jobs/jobs/webhook-handler/webhook-handler.processor.ts
- packages/nocodb/src/services/hook-handler.service.ts
Change Summary
Change type