Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Review package related files due to Azure module refactor #2689

Closed
2 tasks done
Tracked by #19576
nico-stefani opened this issue Dec 13, 2023 · 2 comments · Fixed by #2698
Closed
2 tasks done
Tracked by #19576

Review package related files due to Azure module refactor #2689

nico-stefani opened this issue Dec 13, 2023 · 2 comments · Fixed by #2698
Assignees
@nico-stefani
Labels
level/task Subtask issue type/enhancement Enhancement issue

Comments

@nico-stefani
Copy link
Member

nico-stefani commented Dec 13, 2023 •
edited

Wazuh version
4.9.0

Description

As part of wazuh/wazuh#19576, the Azure module is being refactored to get better organization. Some structural changes are being made to it, separating the module into different files to have a more maintainable and cohesive code.

Tasks

  • Review that the changes made to the AWS Module have their counterpart in the corresponding wazuh-packages files (e.g. solaris/solaris11/SPECS/template_agent.json)
  • Test the correct build of the packages that required a modification on their files.
@nico-stefani nico-stefani added level/task Subtask issue type/enhancement Enhancement issue labels Dec 13, 2023
@nico-stefani nico-stefani changed the title Review package related files due to AWS module refactor Review package related files due to Azure module refactor Dec 13, 2023
@nico-stefani nico-stefani mentioned this issue Dec 13, 2023
Merged
@Selutario Selutario mentioned this issue Dec 14, 2023
Closed
8 tasks
@nico-stefani nico-stefani self-assigned this Dec 14, 2023
@nico-stefani
Copy link
Member Author

I've been able to generate the package and install it correctly.

vagrant@solaris11:~$ sudo pkg install -g wazuh-agent_v4.9.0-19576.azure.refactor-sol11-i386.p5p wazuh-agent
           Packages to install:  1
            Services to change:  1
       Create boot environment: No
Create backup boot environment: No

DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                                1/1       127/127      6.0/6.0 27.6M/s

PHASE                                          ITEMS
Installing new actions                       185/185
Updating package state database                 Done
Updating package cache                           0/0
Updating image state                            Done
Creating fast lookup database                   Done
Reading search index                            Done
Updating search index                            1/1
Updating package cache                           2/2


vagrant@solaris11:~$ sudo tree -L 2 /var/ossec/wodles/azure
/var/ossec/wodles/azure
├── azure_services
│   ├── __init__.py
│   ├── analytics.py
│   ├── graph.py
│   └── storage.py
├── azure_utils.py
├── azure-logs
├── azure-logs.py
└── db
    ├── __init__.py
    ├── orm.py
    └── utils.py

2 directories, 10 files

Still, I need to do the proprer module testing.

@nico-stefani
Copy link
Member Author

nico-stefani commented Dec 18, 2023 •
edited

After doing a fix(wazuh/wazuh@30b21c1) in the agent installation, I was able to get the module up and running.

root@solaris11:/var/ossec# sudo tree -L 2 /var/ossec/wodles/azure
/var/ossec/wodles/azure
├── __pycache__
│   └── azure_utils.cpython-37.pyc
├── azure_services
│   ├── __init__.py
│   ├── __pycache__
│   ├── analytics.py
│   ├── graph.py
│   └── storage.py
├── azure_utils.py
├── azure-logs
├── azure.db
├── db
│   ├── __init__.py
│   ├── __pycache__
│   ├── orm.py
│   └── utils.py
├── graph.credentials
├── loganalytics.credentials
└── storage.credentials

5 directories, 14 files
root@solaris11:/var/ossec# wodles/azure/azure-logs --graph --graph_auth_path /var/ossec/wodles/azure/graph.credentials --graph_tenant_domain wazuh.onmicrosoft.com --graph_tag request_829166026 --graph_query 'auditLogs/directoryaudits' --graph_time_offset 30d --debug 2
2023/12/18 15:56:39 azure: INFO: Checking database integrity
2023/12/18 15:56:39 azure: INFO: Database integrity check finished
2023/12/18 15:56:39 azure: INFO: Azure Graph starting.
2023/12/18 15:56:39 azure: INFO: Graph: Getting authentication token.
2023/12/18 15:56:41 azure: INFO: Graph: Building the url.
2023/12/18 15:56:41 azure: INFO: Graph: The search starts for query: "auditLogs/directoryaudits" using activityDateTime+gt+2023-12-18T13:23:04.5421212Z
2023/12/18 15:56:41 azure: INFO: Graph: The URL is "https://graph.microsoft.com/v1.0/auditLogs/directoryaudits?&$filter=activityDateTime+gt+2023-12-18T13:23:04.5421212Z"
2023/12/18 15:56:41 azure: INFO: Graph: Pagination starts
2023/12/18 15:56:44 azure: DEBUG: Attempting to update a graph row object. MD5: "169e36eda7ce0ec7141250c5a2d03285", min_date: "2023-11-18T15:35:49.275815Z", max_date: "2023-12-18T15:49:31.868932Z"
2023/12/18 15:56:44 azure: INFO: Graph: Sending event by socket.
2023/12/18 15:56:44 azure: INFO: Graph: End


root@solaris11:/var/ossec# wodles/azure/azure-logs --log_analytics --la_auth_path /var/ossec/wodles/azure/loganalytics.credentials --la_tenant_domain wazuh.onmicrosoft.com --la_tag request_607367744 --la_query "AuditLogs" --workspace xxx --la_time_offset 60d --debug 2
2023/12/18 15:41:05 azure: INFO: Checking database integrity
2023/12/18 15:41:05 azure: INFO: Database integrity check finished
2023/12/18 15:41:05 azure: INFO: Azure Log Analytics starting.
2023/12/18 15:41:05 azure: INFO: Log Analytics: Getting authentication token.
2023/12/18 15:41:06 azure: INFO: Log Analytics: The search starts for query: "AuditLogs | order by TimeGenerated asc | where TimeGenerated >= datetime(2023-10-19T15:41:06.565943Z) "
2023/12/18 15:41:06 azure: INFO: Log Analytics: Sending a request to the Log Analytics API.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
...
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: INFO: Log Analytics: Sending event by socket.
2023/12/18 15:41:09 azure: DEBUG: Attempting to update a log_analytics row object. MD5: "63daf76c0c6951a249570d1a0c0a2201", min_date: "2023-10-19T15:40:16.009691Z", max_date: "2023-12-18T13:23:04.5421212Z"
2023/12/18 15:41:09 azure: INFO: Azure Log Analytics ending.


root@solaris11:/var/ossec#  wodles/azure/azure-logs --storage --storage_auth_path /var/ossec/wodles/azure/storage.credentials --container "frameworktestcontainer" --blobs "*" --storage_tag azure-activity --storage_time_offset 30d --debug 2
2023/12/18 15:43:05 azure: INFO: Checking database integrity
2023/12/18 15:43:05 azure: INFO: Database integrity check finished
2023/12/18 15:43:05 azure: INFO: Azure Storage starting.
2023/12/18 15:43:05 azure: INFO: Storage: Authenticating.
2023/12/18 15:43:05 azure: DEBUG: String_to_sign=GET
2023/12/18 15:43:09 azure: INFO: Storage: Authenticated.
2023/12/18 15:43:09 azure: INFO: 82e049b81fa6eb88ebf85f1677785f2b was not found in the database for storage. Adding it.
2023/12/18 15:43:09 azure: DEBUG: Attempting to insert row object into storage with md5="82e049b81fa6eb88ebf85f1677785f2b", min_date="2023-11-18T15:43:09.541235Z", max_date="2023-11-18T15:43:09.541235Z"
2023/12/18 15:43:09 azure: INFO: Storage: Getting blobs.
2023/12/18 15:43:09 azure: DEBUG: String_to_sign=GET
2023/12/18 15:43:09 azure: INFO: Client-Request-ID=25046914-9dbc-11ee-a7e0-610520c0a70f Outgoing request: Method=GET, Path=/frameworktestcontainer, Query={'restype': 'container', 'comp': 'list', 'prefix': None, 'delimiter': None, 'marker': None, 'maxresults': None, 'include': None, 'timeout': None}, Headers={'x-ms-version': '2019-02-02', 'User-Agent': 'Azure-Storage/2.1.0-2.1.0 (Python CPython 3.7.10; SunOS 5.11)', 'x-ms-client-request-id': '25046914-9dbc-11ee-a7e0-610520c0a70f', 'x-ms-date': 'Mon, 18 Dec 2023 15:43:09 GMT', 'Authorization': 'REDACTED'}.
2023/12/18 15:43:10 azure: INFO: Client-Request-ID=25046914-9dbc-11ee-a7e0-610520c0a70f Receiving Response: Server-Timestamp=Mon, 18 Dec 2023 15:43:08 GMT, Server-Request-ID=8c2ee194-801e-0086-47c8-31644e000000, HTTP Status Code=200, Message=OK, Headers={'transfer-encoding': 'chunked', 'content-type': 'application/xml', 'server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0', 'x-ms-request-id': '8c2ee194-801e-0086-47c8-31644e000000', 'x-ms-client-request-id': '25046914-9dbc-11ee-a7e0-610520c0a70f', 'x-ms-version': '2019-02-02', 'date': 'Mon, 18 Dec 2023 15:43:08 GMT'}.
2023/12/18 15:43:10 azure: INFO: Storage: The search starts from the date: 2023-11-18 15:43:09.552026+00:00 for blobs in container: "frameworktestcontainer" and prefix: "/"
2023/12/18 15:43:10 azure: INFO: Storage: End

@nico-stefani nico-stefani mentioned this issue Dec 18, 2023
Merged
30 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nico-stefani nico-stefani

Labels
level/task Subtask issue type/enhancement Enhancement issue
Projects
Release 4.9.0
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add new Azure module files wazuh/wazuh-packages
2 participants
@nico-stefani @Selutario