Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New dev commands and bypass unoconv node module at indexing #489

Merged
merged 3 commits into from May 15, 2024
Merged

New dev commands and bypass unoconv node module at indexing #489

merged 3 commits into from May 15, 2024

Conversation

ericlinagora
Copy link
Contributor

Do not use node module to call unoconv because of deprecation stderr warning triggering an error. Add 3 dev commands to the CLI:

  • debug: meant to be edited and not committed for little intermittent needs to code with platform running. Default implementation just prints all the data in a convenient way for diagnosis - and a very inconvenient way for large amounts of data. Including summaries of access rights, user-company topology, and comparison to search engine indexes.
  • labels: analyse status of i18n labels in the source code. Compares languages for identical or missing translations, for labels absent in the code and labels in the code absent in the translations (requires silver searcher), creates a report of all labels presenting a potential concern.
  • translate: opens a webservice defined in an environment variable for each translation language. eg. set the following in your .env file: TDCLI_TRANSLATOR_URL=https://translate.google.com/?sl=%{from:auto}&tl=%{to}&text=%{text}&op=translate and the command will use google translate with the provided first argument.

@github-actions GitHub Actions
Copy link

Coverage Report

  • Status: ✅ Passed
Click to view remaining coverage report
Status File Stmts Branch Funcs Lines
total 67.22 48.28 62.74 67.21
/usr/src/app/src/version.ts 100 100 100 100
/usr/src/app/src/core/config/index.ts 100 100 100 100
/usr/src/app/src/core/crypto/index.ts 95.65 100 100 95.45
/usr/src/app/src/core/crypto/legacy.ts 85.71 57.14 50 85.71
/usr/src/app/src/core/crypto/v1.ts 86.36 71.42 100 86.36
/usr/src/app/src/core/crypto/v2.ts 88 71.42 100 88
/usr/src/app/src/core/platform/platform.ts 80 100 50 80
/usr/src/app/src/core/platform/framework/configuration.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/event-bus.ts 93.75 83.33 100 93.75
/usr/src/app/src/core/platform/framework/execution-storage.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/factory.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/index.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/logger.ts 100 66.66 100 100
/usr/src/app/src/core/platform/framework/api/application-configuration.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/api/component.ts 92.59 81.81 100 92.3
/usr/src/app/src/core/platform/framework/api/constants.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/api/container.ts 70.37 50 66.66 76
/usr/src/app/src/core/platform/framework/api/crud-service.ts 96.29 83.33 90.9 95.55
/usr/src/app/src/core/platform/framework/api/index.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/api/service-options.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/api/service-state.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/api/service.ts 58.73 68.75 77.77 58.73
/usr/src/app/src/core/platform/framework/decorators/consumes.ts 100 0 100 100
/usr/src/app/src/core/platform/framework/decorators/index.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/decorators/prefix.ts 100 0 100 100
/usr/src/app/src/core/platform/framework/decorators/service-name.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/decorators/skip.ts 92.85 66.66 100 92.3
/usr/src/app/src/core/platform/framework/decorators/realtime/created.ts 100 80 100 100
/usr/src/app/src/core/platform/framework/decorators/realtime/deleted.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/decorators/realtime/index.ts 90 57.14 75 94.44
/usr/src/app/src/core/platform/framework/decorators/realtime/saved.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/decorators/realtime/updated.ts 100 100 100 100
/usr/src/app/src/core/platform/framework/utils/component-utils.ts 73.8 11.11 100 75.6
/usr/src/app/src/core/platform/framework/utils/loader.ts 76.19 60 85.71 73.68
/usr/src/app/src/core/platform/services/auth/index.ts 100 100 100 100
/usr/src/app/src/core/platform/services/auth/service.ts 85.71 50 66.66 85.71
/usr/src/app/src/core/platform/services/auth/web/index.ts 100 100 100 100
/usr/src/app/src/core/platform/services/auth/web/jwt.ts 100 88.88 100 100
/usr/src/app/src/core/platform/services/auth/web/routes.ts 62.5 100 33.33 71.42
/usr/src/app/src/core/platform/services/counter/index.ts 100 100 100 100
/usr/src/app/src/core/platform/services/counter/provider.ts 85.71 80 100 85.71
/usr/src/app/src/core/platform/services/cron/index.ts 34.78 0 28.57 34.78
/usr/src/app/src/core/platform/services/database/index.ts 74.07 0 75 72
/usr/src/app/src/core/platform/services/database/services/connector-factory.ts 66.66 25 100 66.66
/usr/src/app/src/core/platform/services/database/services/index.ts 77.77 50 60 77.77
/usr/src/app/src/core/platform/services/database/services/orm/manager.ts 84.05 70 76.92 87.87
/usr/src/app/src/core/platform/services/database/services/orm/utils.ts 84.61 73.91 90 86.48
/usr/src/app/src/core/platform/services/database/services/orm/connectors/abstract-connector.ts 66.66 100 33.33 50
/usr/src/app/src/core/platform/services/database/services/orm/connectors/cassandra/cassandra.ts 76.58 55.73 71.92 76.64
/usr/src/app/src/core/platform/services/database/services/orm/connectors/cassandra/pagination.ts 22.22 0 0 25
/usr/src/app/src/core/platform/services/database/services/orm/connectors/cassandra/query-builder.ts 92.3 73.07 84.61 93.47
/usr/src/app/src/core/platform/services/database/services/orm/connectors/cassandra/typeTransforms.ts 81.25 80.3 100 81.81
/usr/src/app/src/core/platform/services/database/services/orm/connectors/mongodb/mongodb.ts 7.69 0 0 8.08
/usr/src/app/src/core/platform/services/database/services/orm/connectors/mongodb/query-builder.ts 14.28 0 0 15.38
/usr/src/app/src/core/platform/services/database/services/orm/connectors/mongodb/typeTransforms.ts 12.28 0 0 9.25
/usr/src/app/src/core/platform/services/database/services/orm/connectors/postgres/postgres-data-transform.ts 72.58 77.55 100 72.88
/usr/src/app/src/core/platform/services/database/services/orm/connectors/postgres/postgres-query-builder.ts 95.87 80 100 95.4
/usr/src/app/src/core/platform/services/database/services/orm/connectors/postgres/postgres.ts 48.38 14.81 44.44 49.16
/usr/src/app/src/core/platform/services/database/services/orm/decorators/column.ts 100 100 100 100
/usr/src/app/src/core/platform/services/database/services/orm/decorators/entity.ts 100 50 100 100
/usr/src/app/src/core/platform/services/database/services/orm/decorators/index.ts 100 100 100 100
/usr/src/app/src/core/platform/services/database/services/orm/repository/manager.ts 84.61 100 100 84.61
/usr/src/app/src/core/platform/services/database/services/orm/repository/repository.ts 83.78 77.77 76.92 87.87
/usr/src/app/src/core/platform/services/email-pusher/index.ts 63.88 40.74 100 63.38
/usr/src/app/src/core/platform/services/message-queue/api.ts 81.25 57.14 77.77 82.6
/usr/src/app/src/core/platform/services/message-queue/bus.ts 83.33 100 66.66 83.33
/usr/src/app/src/core/platform/services/message-queue/factory.ts 83.33 50 100 83.33
/usr/src/app/src/core/platform/services/message-queue/index.ts 80.95 50 75 80
/usr/src/app/src/core/platform/services/message-queue/processor.ts 58.33 20 50 58.33
/usr/src/app/src/core/platform/services/message-queue/proxy.ts 52.63 23.8 40 53.57
/usr/src/app/src/core/platform/services/message-queue/amqp/client.ts 80.95 60 84.21 80.48
/usr/src/app/src/core/platform/services/message-queue/amqp/constants.ts 100 100 100 100
/usr/src/app/src/core/platform/services/message-queue/amqp/index.ts 73.07 100 55.55 73.07
/usr/src/app/src/core/platform/services/message-queue/amqp/manager.ts 76.74 33.33 66.66 76.74
/usr/src/app/src/core/platform/services/message-queue/amqp/pubsub.ts 84.21 47.82 83.33 84.21
/usr/src/app/src/core/platform/services/message-queue/amqp/pubsubclient.ts 54.83 38.88 46.15 58.62
/usr/src/app/src/core/platform/services/message-queue/local/index.ts 15.62 0 0 15.62
/usr/src/app/src/core/platform/services/push/index.ts 63.15 16.66 75 58.82
/usr/src/app/src/core/platform/services/push/connectors/fcm/service.ts 18.75 0 0 18.75
/usr/src/app/src/core/platform/services/realtime/bus.ts 100 100 100 100
/usr/src/app/src/core/platform/services/realtime/index.ts 80 100 50 78.57
/usr/src/app/src/core/platform/services/realtime/types.ts 94.44 66.66 75 94.11
/usr/src/app/src/core/platform/services/realtime/services/entity-manager.ts 81.81 0 66.66 81.81
/usr/src/app/src/core/platform/services/realtime/services/room-manager.ts 11.47 0 11.11 10.71
/usr/src/app/src/core/platform/services/search/api.ts 100 100 100 100
/usr/src/app/src/core/platform/services/search/index.ts 63.04 22.22 58.33 61.9
/usr/src/app/src/core/platform/services/search/repository.ts 88.88 33.33 100 88.46
/usr/src/app/src/core/platform/services/search/adapters/abstract.ts 54.54 100 28.57 54.54
/usr/src/app/src/core/platform/services/search/adapters/ascii-folder.ts 64.7 30 50 91.66
/usr/src/app/src/core/platform/services/search/adapters/utils.ts 76.92 0 77.77 77.27
/usr/src/app/src/core/platform/services/search/adapters/elasticsearch/elastic-open-search-adapter.ts 73.64 60.16 76.92 73.01
/usr/src/app/src/core/platform/services/search/adapters/elasticsearch/search.ts 90.16 81.6 100 90.16
/usr/src/app/src/core/platform/services/search/adapters/mongosearch/index.ts 10.41 0 0 10.41
/usr/src/app/src/core/platform/services/search/adapters/mongosearch/search.ts 11.11 0 0 11.42
/usr/src/app/src/core/platform/services/storage/index.ts 84.52 67.3 91.66 83.75
/usr/src/app/src/core/platform/services/storage/connectors/S3/s3-service.ts 70.17 58.82 83.33 71.69
/usr/src/app/src/core/platform/services/storage/connectors/local/service.ts 16.66 0 0 16.66
/usr/src/app/src/core/platform/services/tracker/index.ts 30.43 0 30 28.57
/usr/src/app/src/core/platform/services/tracker/adapters/segment.ts 11.11 0 0 11.53
/usr/src/app/src/core/platform/services/webserver/error.ts 100 100 100 100
/usr/src/app/src/core/platform/services/webserver/index.ts 81.94 9.09 81.25 83.09
/usr/src/app/src/core/platform/services/websocket/index.ts 93.75 100 75 92.85
/usr/src/app/src/core/platform/services/websocket/services/index.ts 36.11 50 41.66 36.11
/usr/src/app/src/services/global-resolver.ts 97.36 50 100 97.36
/usr/src/app/src/services/applications/index.ts 92.3 100 75 90.9
/usr/src/app/src/services/applications/realtime.ts 50 100 0 50
/usr/src/app/src/services/applications-api/index.ts 58.53 50 50 57.89
/usr/src/app/src/services/applications-api/web/index.ts 100 100 100 100
/usr/src/app/src/services/applications-api/web/routes.ts 72.22 0 25 76.47
/usr/src/app/src/services/applications-api/web/schemas.ts 100 100 100 100
/usr/src/app/src/services/applications-api/web/controllers/index.ts 15.68 0 0 15.68
/usr/src/app/src/services/applications/entities/application.ts 45.45 100 0 33.33
/usr/src/app/src/services/applications/services/applications.ts 50 0 25 60
/usr/src/app/src/services/applications/services/company-applications.ts 17.39 0 25 17.39
/usr/src/app/src/services/applications/services/hooks.ts 25.8 0 25 25.8
/usr/src/app/src/services/applications/web/index.ts 100 100 100 100
/usr/src/app/src/services/applications/web/routes.ts 100 100 100 100
/usr/src/app/src/services/applications/web/schemas.ts 100 100 100 100
/usr/src/app/src/services/applications/web/controllers/applications.ts 17.39 0 0 17.39
/usr/src/app/src/services/applications/web/controllers/company-applications.ts 27.27 0 0 27.27
/usr/src/app/src/services/console/client-factory.ts 37.5 0 0 37.5
/usr/src/app/src/services/console/index.ts 92.3 100 75 90.9
/usr/src/app/src/services/console/service.ts 88.23 100 50 88.23
/usr/src/app/src/services/console/clients/internal.ts 16.21 0 0 16.66
/usr/src/app/src/services/console/clients/remote-jwks-verifier.ts 10.81 0 0 10.81
/usr/src/app/src/services/console/clients/remote.ts 9.89 0 0 10
/usr/src/app/src/services/console/web/controller.ts 10.9 1.08 5.88 10.9
/usr/src/app/src/services/console/web/index.ts 100 100 100 100
/usr/src/app/src/services/console/web/routes.ts 86.66 100 33.33 86.66
/usr/src/app/src/services/console/web/schemas.ts 100 100 100 100
/usr/src/app/src/services/documents/const.ts 100 100 100 100
/usr/src/app/src/services/documents/index.ts 93.75 100 80 92.3
/usr/src/app/src/services/documents/types.ts 100 100 100 100
/usr/src/app/src/services/documents/utils.ts 84.09 75.4 90.9 83.51
/usr/src/app/src/services/documents/entities/drive-file.search.ts 100 83.33 100 100
/usr/src/app/src/services/documents/entities/drive-file.ts 100 100 100 100
/usr/src/app/src/services/documents/entities/drive-tdrive-tab.ts 68.75 100 0 100
/usr/src/app/src/services/documents/entities/file-version.ts 63.33 100 0 100
/usr/src/app/src/services/documents/entities/missed-drive-file.ts 58.82 100 0 100
/usr/src/app/src/services/documents/services/access-check.ts 72.27 61.88 75.86 71.77
/usr/src/app/src/services/documents/services/drive-file-dto-builder.ts 100 90.9 100 100
/usr/src/app/src/services/documents/services/index.ts 69.17 54.33 74.24 68.9
/usr/src/app/src/services/documents/services/engine/extract-keywords.ts 100 100 100 100
/usr/src/app/src/services/documents/services/engine/index.ts 96.55 100 100 96.42
/usr/src/app/src/services/documents/services/engine/save-keywords.ts 80.95 83.33 80 80.95
/usr/src/app/src/services/documents/web/index.ts 100 100 100 100
/usr/src/app/src/services/documents/web/routes.ts 100 100 100 100
/usr/src/app/src/services/documents/web/schemas.ts 100 100 100 100
/usr/src/app/src/services/documents/web/controllers/documents.ts 55.85 28.04 59.52 57.37
/usr/src/app/src/services/documents/web/controllers/index.ts 100 100 100 100
/usr/src/app/src/services/files/index.ts 92.3 100 75 90.9
/usr/src/app/src/services/files/entities/file.ts 86.36 100 50 100
/usr/src/app/src/services/files/services/index.ts 53.75 45.88 50 53.25
/usr/src/app/src/services/files/services/preview.ts 81.81 50 80 85
/usr/src/app/src/services/files/web/index.ts 100 100 100 100
/usr/src/app/src/services/files/web/routes.ts 90.47 100 33.33 89.47
/usr/src/app/src/services/files/web/controllers/files.ts 42.1 61.9 33.33 41.81
/usr/src/app/src/services/files/web/controllers/index.ts 100 100 100 100
/usr/src/app/src/services/general/index.ts 92.85 100 75 91.66
/usr/src/app/src/services/general/languages.ts 100 100 100 100
/usr/src/app/src/services/general/web/index.ts 100 100 100 100
/usr/src/app/src/services/general/web/routes.ts 58.82 0 14.28 58.82
/usr/src/app/src/services/i18n/i18n-service.ts 80 100 60 80
/usr/src/app/src/services/i18n/index.ts 100 100 100 100
/usr/src/app/src/services/previews/utils.ts 100 100 100 100
/usr/src/app/src/services/previews/services/files/engine/clear.ts 40 0 25 42.85
/usr/src/app/src/services/previews/services/files/engine/index.ts 100 100 100 100
/usr/src/app/src/services/previews/services/files/engine/service.ts 95.23 66.66 83.33 95.12
/usr/src/app/src/services/previews/services/files/processing/image.ts 86.36 83.33 100 86.36
/usr/src/app/src/services/previews/services/files/processing/pdf.ts 76.19 0 100 75
/usr/src/app/src/services/previews/services/files/processing/service.ts 92.85 100 100 92.85
/usr/src/app/src/services/previews/services/files/processing/video.ts 58.33 12.5 69.23 57.44
/usr/src/app/src/services/statistics/index.ts 83.33 100 66.66 83.33
/usr/src/app/src/services/statistics/types.ts 100 100 100 100
/usr/src/app/src/services/statistics/entities/statistics.ts 91.66 100 0 90
/usr/src/app/src/services/statistics/service/index.ts 47.05 44.44 50 47.05
/usr/src/app/src/services/tags/index.ts 93.33 100 80 91.66
/usr/src/app/src/services/tags/entities/index.ts 100 100 100 100
/usr/src/app/src/services/tags/entities/tags.ts 100 100 100 100
/usr/src/app/src/services/tags/services/tags.ts 91.66 100 100 91.66
/usr/src/app/src/services/tags/web/index.ts 100 100 100 100
/usr/src/app/src/services/tags/web/routes.ts 100 100 100 100
/usr/src/app/src/services/tags/web/controllers/index.ts 100 100 100 100
/usr/src/app/src/services/tags/web/controllers/tags.ts 85.29 83.33 100 85.29
/usr/src/app/src/services/user/index.ts 92.3 100 75 90.9
/usr/src/app/src/services/user/realtime.ts 71.42 100 42.85 71.42
/usr/src/app/src/services/user/utils.ts 100 62.68 100 100
/usr/src/app/src/services/user/entities/company.ts 100 100 100 100
/usr/src/app/src/services/user/entities/company_user.ts 100 100 100 100
/usr/src/app/src/services/user/entities/device.ts 100 100 100 100
/usr/src/app/src/services/user/entities/external_company.ts 100 100 100 100
/usr/src/app/src/services/user/entities/external_user.ts 100 100 100 100
/usr/src/app/src/services/user/entities/user.search.ts 100 88.88 100 100
/usr/src/app/src/services/user/entities/user.ts 88.37 40 75 87.8
/usr/src/app/src/services/user/services/companies.ts 50 27.38 41.66 51.85
/usr/src/app/src/services/user/services/external_links/index.ts 61.11 0 50 61.11
/usr/src/app/src/services/user/services/users/service.ts 51.87 42.62 53.65 54.1
/usr/src/app/src/services/user/web/controller.ts 78.78 65.78 80.76 77.65
/usr/src/app/src/services/user/web/index.ts 100 100 100 100
/usr/src/app/src/services/user/web/routes.ts 95.83 0 100 95.65
/usr/src/app/src/services/user/web/schemas.ts 100 100 100 100
/usr/src/app/src/services/user/web/types.ts 100 100 100 100
/usr/src/app/src/services/workspaces/index.ts 92.3 100 75 90.9
/usr/src/app/src/services/workspaces/realtime.ts 87.5 75 75 87.5
/usr/src/app/src/services/workspaces/entities/workspace.ts 100 100 100 100
/usr/src/app/src/services/workspaces/entities/workspace_counters.ts 100 100 100 100
/usr/src/app/src/services/workspaces/entities/workspace_invite_domain.ts 90.9 100 0 88.88
/usr/src/app/src/services/workspaces/entities/workspace_invite_tokens.ts 92.3 100 0 90.9
/usr/src/app/src/services/workspaces/entities/workspace_pending_users.ts 91.66 100 0 90
/usr/src/app/src/services/workspaces/entities/workspace_user.ts 100 66.66 100 100
/usr/src/app/src/services/workspaces/services/workspace.ts 54.89 37.79 62 55.02
/usr/src/app/src/services/workspaces/web/index.ts 100 100 100 100
/usr/src/app/src/services/workspaces/web/routes.ts 72.82 24.24 64.7 71.91
/usr/src/app/src/services/workspaces/web/schemas.ts 100 100 100 100
/usr/src/app/src/services/workspaces/web/controllers/workspace-invite-tokens.ts 8.33 0 0 8.47
/usr/src/app/src/services/workspaces/web/controllers/workspace-users.ts 55.47 44.64 62.85 57.14
/usr/src/app/src/services/workspaces/web/controllers/workspaces.ts 80.43 76.66 86.95 80.43
/usr/src/app/src/utils/coalesce.ts 25 0 0 25
/usr/src/app/src/utils/company.ts 82.6 75 50 82.6
/usr/src/app/src/utils/counters.ts 100 100 100 100
/usr/src/app/src/utils/exec.ts 69.76 0 50 71.05
/usr/src/app/src/utils/files.ts 88.67 100 80 89.47
/usr/src/app/src/utils/handleError.ts 33.33 0 0 33.33
/usr/src/app/src/utils/mime.ts 100 100 100 100
/usr/src/app/src/utils/password-encoder.ts 50 0 50 50
/usr/src/app/src/utils/pick.ts 100 100 100 100
/usr/src/app/src/utils/types.ts 100 100 100 100
/usr/src/app/src/utils/users.ts 85.71 73.33 100 88.46
/usr/src/app/src/utils/uuid-reducer.ts 40 0 0 33.33
/usr/src/app/src/utils/workspace.ts 33.33 14.28 25 33.33
/usr/src/app/test/e2e/utils.api.ts 93.75 100 83.33 93.33
/usr/src/app/test/e2e/utils.prepare.db.ts 78.49 76.92 65.21 79.77
/usr/src/app/test/e2e/common/user-api.ts 93.63 64.51 90.9 94.39
/usr/src/app/test/e2e/common/entities/mock_entities.ts 100 100 100 100
/usr/src/app/test/e2e/documents/utils.ts 100 100 100 100
/usr/src/app/test/e2e/setup/index.ts 88.88 50 100 88.88
/usr/src/app/test/unit/core/services/database/services/orm/connectors/postgres/utils.ts 79.16 100 37.5 100
Total 67.21%
Change from base: 0%
Covered Lines: 4907
Total Lines: 7300

@ericlinagora ericlinagora merged commit a8de167 into main May 15, 2024
7 checks passed
@ericlinagora ericlinagora deleted the bypass-unoconv-node-module-at-indexing branch May 15, 2024 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MontaGhanmy MontaGhanmy MontaGhanmy approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@ericlinagora @MontaGhanmy