Generate and add equipment availability data

[sakshimohan]

Fixes #1305

This PR generates availability data on equipment using HHFA 2018-19 and maps it to TLO model equipment codes.

[sakshimohan]

Hi @tbhallett. I'll add the equipment availability resource file to this PR. There are a couple of steps of cleaning the data left. It should be ready by Friday. But the format will be as follows:

<style> </style>

Equipment_code	Facility_Level	District	Proportion of facilities which reported equipment as available	Proportion of facilities which reported equipment as available & functional	Proportion of facilities which reported equipment as available, functional, and calibrated/prepared
1	1a	Lilongwe	0.9	0.85	0.8
1	1b	Lilongwe	0.8	0.8	0.8
2	1a	Blantyre	0.9	0.9	0.8
2	1b	Blantyre	0.7	0.6	0.5
...	...	...	...	...	...

[tbhallett]

Hi @tbhallett. I'll add the equipment availability resource file to this PR. There are a couple of steps of cleaning the data left. It should be ready by Friday. But the format will be as follows:

<style> </style>

Equipment_code Facility_Level District Proportion of facilities which reported equipment as available Proportion of facilities which reported equipment as available & functional Proportion of facilities which reported equipment as available, functional, and calibrated/prepared
1 1a Lilongwe 0.9 0.85 0.8
1 1b Lilongwe 0.8 0.8 0.8
2 1a Blantyre 0.9 0.9 0.8
2 1b Blantyre 0.7 0.6 0.5
... ... ... ... ... ...

Hi @sakshimohan and @nchagoma503 -- this is going to be brilliant. Thank you so much.

Just one comment.... for the column District, it would be really handy if this could match as well as possible with the District names defined in resources/demography/ResourceFile_PopulationSize_2018Census.csv. These are:

{'Chitipa', 'Karonga', 'Nkhata Bay', 'Rumphi', 'Mzimba', 'Likoma',
'Mzuzu City', 'Kasungu', 'Nkhotakota', 'Ntchisi', 'Dowa', 'Salima',
'Lilongwe', 'Mchinji', 'Dedza', 'Ntcheu', 'Lilongwe City',
'Mangochi', 'Machinga', 'Zomba', 'Chiradzulu', 'Blantyre',
'Mwanza', 'Thyolo', 'Mulanje', 'Phalombe', 'Chikwawa', 'Nsanje',
'Balaka', 'Neno', 'Zomba City', 'Blantyre City'}

[tbhallett]

Also, please could I suggest that the destination for the ResourceFiles that will be created to be resources/healthsystem/infrastructure_and_equipment

[sakshimohan]

Hi @tbhallett, @nchagoma503. This PR is now ready for review. The resourcefile has two columns -

1. 'available' - this represents whether equipment was recorded as available in HHFA
2. 'functional' - this represents whether equipment was recorded as functional in HHFA
   Data is available for 62 unique Item_code (s)/equipment.

There are some rows with missing data. For example, data on several items of equipment was not recorded at level 0. I've left these as such. I'm not sure whether we can assume that missing data implies missing. For instance, 126 (Biochemistry analyser) is recorded as available in some district which for others there is no data. I'm not sure how to deal with this.

Here is a summary of the data. The trend is as we would expect.

1. Overall summary ('available', 'functional')

<style> </style>

Facility_level	Average of 'available'	Average of 'functional'
0	19.4%	16.4%
1a	58.2%	52.9%
1b	81.7%	78.1%
2	88.5%	85.2%
3	96.2%	94.8%
4	30.6%	27.7%
Grand Total	69.9%	65.9%

2. Summary of data by district and facility level ('available')
   

[tbhallett]

Thanks so much for this @sakshimohan

I've just done a quick change to update (ever so slightly) the name of the resulting file.

I've tried running the script, equipment_availability_estimation.py but get an error on line 99: TypeError: '>=' not supported between instances of 'float' and 'str'.

I think this comes from the aggregation using max on the groupby object. It could be the columns that are strings need to be coerced into floats.

Does it run cleanly from top-to-bottom on your machine?

[sakshimohan]

Hi @tbhallett. Thanks for looking at this. The script did run cleanly for me but you're right that there were two string value columns (calibrated and prepared) which were not really passing through the aggregate max command. I have removed these from the script altogether because there is very limited data on these for these to be of value (I was already not extracting them into the resource file). Hopefully the script should run on your system as well now?

I should note that I'm still using python 3.10 not 3.11. Will update this soon!

[tbhallett]

Hi @tbhallett. Thanks for looking at this. The script did run cleanly for me but you're right that there were two string value columns (calibrated and prepared) which were not really passing through the aggregate max command. I have removed these from the script altogether because there is very limited data on these for these to be of value (I was already not extracting them into the resource file). Hopefully the script should run on your system as well now?

I should note that I'm still using python 3.10 not 3.11. Will update this soon!

Thanks @sakshimohan. Now runs perfectly on my machine too.

[tbhallett]

Thanks again for this @sakshimohan.

I've added some stuff onto the end of your script that

• formats exported dataset into what is required for reading into the Equipment class
• does the extrapolation to all item_codes and facilities (Rather than having this hidden in the Equipment class).

I'll add some comments onto the edits about both these points.

[tbhallett]

Remove checks on handling of item_codes/facility_id that are not in the availability data (we now insist that they all are).

[tbhallett]

recreated file from update script

[sakshimohan]

@tbhallett This should be ready to merge now.

[tbhallett]

I think this average should be computed from across facility levels 0-3. (Otherwise it gets dragged down by the zeros at level 5).

I'll make that last change

[sakshimohan]

That's a good point. Thanks, Tim. Just as an FYI, there isn't any data being imputed by this line of code as the series is complete at this stage. But I have deleted the imputation in case any new source of data requires this step.