[COST-4912] Calculate node based unattributed storage (Azure)

[myersCody]

Jira Ticket

COST-3912

Description

This change will calculate the node based unattributed storage for Azure.

Testing

1. Checkout Branch
2. Create the test customer
3. Update the unleash flag:

def is_feature_unattributed_storage_enabled(account):
    """Should unattributed storage feature be enabled."""
    unleash_flag = "cost-management.backend.unattributed_storage"
    account = convert_account(account)
    context = {"schema": account}
    # TODO: Turn on fallback when QE is ready to start testing
    # UNLEASH_CLIENT.is_enabled(unleash_flag, context, fallback_development_true)
    # return UNLEASH_CLIENT.is_enabled(unleash_flag, context)
    return True

4. make create-test-customer
5. make load-test-customer-data test_source=Azure
6. Ensure that the Storage unattributed namespace was populated:

select sum(infrastructure_raw_cost), resource_id from reporting_ocpusagelineitem_daily_summary where namespace='Storage unattributed' group by resource_id;

        sum        |               resource_id
-------------------+-----------------------------------------
 4.236326764000000 | pv-123-claimless
 2.337303951000000 | azure-cloud-prefix-pvc-partial-matching
 1.554439789000000 | disk-id-1234567

The claimless will be be 100% unattributed cost; however, we can check our other resource to see that the cost was split:

select namespace, sum(infrastructure_raw_cost) from reporting_ocpusagelineitem_daily_summary where resource_id='azure-cloud-prefix-pvc-partial-matching' group by namespace;

      namespace       |        sum
----------------------+-------------------
 news-site            | 0.001427496000000
 Storage unattributed | 2.337303951000000

7. Undo the changes to the unleash flag:

git checkout -- koku/masu/processor/__init__.py

This will set it back to being False.
8. Resummarize the Azure source
9. Recheck the database to see if unattributed storage was added:

select count(*) from reporting_ocpusagelineitem_daily_summary where namespace='Storage unattributed';

 count
-------
    0
(1 row)

Release Notes

• proposed release note

* [COST-3912](https://issues.redhat.com/browse/COST-3912) Fix some things

[myersCody]

/retest

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.1%. Comparing base (e2f0934) to head (2764919).
Report is 1 commits behind head on main.

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #5066   +/-   ##
=====================================
  Coverage   94.1%   94.1%           
=====================================
  Files        375     375           
  Lines      31194   31202    +8     
  Branches    3727    3728    +1     
=====================================
+ Hits       29360   29372   +12     
+ Misses      1168    1166    -2     
+ Partials     666     664    -2     

[myersCody]

Successful Azure Smokes: https://ci.ext.devshift.net/job/koku-pipeline-pr-check-main/2468/

[pedrolp85]

checking out to this branch:

((koku) ) [plopezpe@fedora koku]$ git status
En la rama cost-4912-unattributed-storage
Tu rama está actualizada con 'origin/cost-4912-unattributed-storage'.

and enabling unnatributed cost in unleash.
using nise current QE version at IQE

(.iqe_venv) [plopezpe@fedora iqe-cost-management-plugin]$ pip freeze | grep nise
koku-nise==4.4.16

run the IQE command:

ENV_FOR_DYNACONF=local iqe tests plugin cost_management --trace -k "test_api_ocp_on_azure_source_raw_calc" -raA --pdb

this will set a breakpoint at the beggining of the tests, when the sources finish their setup

double check storage.unnatributed is ON

http://localhost:8000/api/cost-management/v1/reports/openshift/infrastructures/azure/costs/?group_by[project]=*

thes test is gonna fail, because we did not add to the reports the volume handles, and some storage cost will move to unnatributed

the weird thing is that the API calls:

v1/reports/openshift/infrastructures/azure/storage/?start_date=2024-05-01&end_date=2024-06-07

returns empty values

"date": "2024-05-02",
"values": []

I am not aware at any changes at this API are expected

setting the flag to off makes raw calc pass, and the API returns not empty values