Interpreting "reliability"

[0xCoto]

I've successfully managed to install itmlogic and all of its dependencies, but I'm having trouble interpreting the example plots. I'm not sure if the terminology is introduced in the original work of the Irregular Terrain Model, but what exactly does "Reliability" mean? Is it possible to get an idea of what this term implies without looking through/studying the code of p2p.py?

As for the "Confidence", I assume this just gives an estimate for the 'probability' of the upper propagation loss limit.

Thanks in advance, this looks like an interesting package (more questions to come for sure)! 🙂

---

(I'm also getting these warnings, but I assume they're not important)

/Users/User/Library/Python/3.9/lib/python/site-packages/rasterstats/io.py:302: UserWarning: Setting nodata to -999; specify nodata explicitly
  warnings.warn("Setting nodata to -999; specify nodata explicitly")

/Users/User/Downloads/itmlogic/vis/vis.py:33: FutureWarning: In a future version of pandas all arguments of DataFrame.sort_index will be keyword-only
  data = data.set_index(['distance_km','confidence_level_%']).unstack().swaplevel(0,1,1).sort_index(1)
/Users/User/Downloads/itmlogic/vis/vis.py:59: FutureWarning: In a future version of pandas all arguments of DataFrame.sort_index will be keyword-only
  data = data.set_index(

[godelayheehoo]

Reliability and confidence have slightly different meanings based on context, it is explained in the original paper but there's a brief discussion under /references/itm_man.txt.

The rasterstats warning might be important to pay attention to. Your elevation data might use some value to say "there is no elevation data here". If it's, for instance, -9999 instead of -999 (which I've ran across), then rasterstats will assume that those are "real" heights unless you set the nodata value appropriately (or, perhaps easier if you're just using the package, modify your map appropriately).

I'm not involved with this though, just a fan, so please take what I've said with a grain of salt!

[tomalrussell]

Just to caveat - it's a while since I've looked at this (I would also check in with @edwardoughton and point you to the original ITM references).

I read confidence as statistical confidence, reliability as link reliability - so from the output you can say that attenuation loss will be at most some value, within some confidence interval, for some value of reliability % of the time.

In Hufford's description of the algorithm, in itm_alg.pdf the output is described as a 2-dimensional cumulative distribution to account for variability over time and situations (3-dimensional in the case of area prediction, which has multiple locations to consider as well):

One can now say "With probability (or confidence) qS the attenuation will not exceed A (qT ; qS) for at least qT of the time."

Confidence and reliability parameters are set in the p2p script as qc and qr if you want to sample differently - https://github.com/edwardoughton/itmlogic/blob/master/scripts/p2p.py#L75-L80

---

Thanks for flagging the other warnings - the rasterio one depends on your data, as @godelayheehoo describes. The pandas one we'll need to fix for compatibility with future pandas.

[0xCoto]

Thank you @godelayheehoo and @tomalrussell for your input! The warning is a bit confusing to be honest. It wouldn't be completely unexpected if I had entered in the elevation data or some coordinates myself, but I'm simply running the scripts/p2p.py file as suggested in the README. Is this warning normal, or is the preset data in the example script somewhat of a buggy example?

---

On another note (let me know if I should make another issue for this), I couldn't find which elevation dataset is actually being used. E.g. ASTER, EU-DEM, SRTM? [Edit: after a bit of digging, I found an ASTER dataset in the /data directory]

Can you select which elevation dataset to use (other than ASTER) for a propagation loss computation based on the location of your communication system (in order to obtain the highest resolution/accuracy)? If not, is it possible to input a simple array of points representing the elevation profile between a transmitter and a receiver, and have itmlogic compute the propagation loss along that path?

---

One more question (for now.. 😄 ):
The paper states:

Two modes of prediction are available: “area prediction” and “point-to-point”. Area prediction mode uses a terrain irregularity parameter based on the inter-decile range of terrain elevations (the range after removing the top 10% and bottom 10% of elevations). Point-to-point mode uses a sample of up to 600 points from the terrain profile of the straight line between transmitter and receiver (Hufford, 1995).

Does this imply that point-to-point provides higher-accuracy link loss computations and area prediction for a giving out a rough estimate for the propagation loss as a function of distance? If I run both scripts for the same points (P1, P2), will area prediction give the same propagation loss at Distance=P2 as the point-to-point script, or will it be less accurate?

[edwardoughton]

@0xCoto On the last question. P2P vs area prediction mode really represent different antenna types and deployment scenarios. P2P would be a directed set of (microwave) antennas connecting 2 towers, probably providing 'backhaul' connectivity. Whereas area prediction mode is where you have a traditional cell area (e.g. a hexagon with a 3-sectored site at the centroid) and users (often mobile ones) spread out across a wide area trying to access the site radio resources. Thus, area prediction mode is much more like your traditional 2G, 3G, 4G or 5G cell areas, with the aim of trying to predict the capacity each user can access.

[0xCoto]

I'm looking under if __name__ == '__main__': in scripts/p2p.py, and some things are unclear.

#Define distance between terminals in km (from Longley Rice docs)
main_user_defined_parameters['d'] = 77.8

Why do we need to define the (line-of-sight) distance between the transmitter and receiver? Doesn't the code account for this based on the coordinates of the transmitter and receiver dictionaries?

On that note, @godelayheehoo:

The rasterstats warning might be important to pay attention to. Your elevation data might use some value to say "there is no elevation data here". If it's, for instance, -9999 instead of -999 (which I've ran across), then rasterstats will assume that those are "real" heights unless you set the nodata value appropriately (or, perhaps easier if you're just using the package, modify your map appropriately).

This is what I find in the sample code of p2p.py:

'coordinates': (-0.07491679518573545, 51.42413477117786)

Looks like this matches the coordinates of the Crystal Palace station found here. I don't know if the sample code is incorrect, e.g. the location of latitude and longitude should be flipped or something, but this is definitely not an ocean for me to be getting -999. Any idea what the issue is?

---

Secondly, by surface_profile_m, I understand that the package expects me to provide the (line-of-sight) elevation profile between the transmitter and receiver. This is fine, but what does the package do with the coordinates of the stations then?

[tomalrussell]

Why do we need to define the (line-of-sight) distance between the transmitter and receiver? Doesn't the code account for this based on the coordinates of the transmitter and receiver dictionaries?

We don't need to - we could use the coordinates and the terrain profile values measured from the elevation raster. The script as written hard-codes the distance and terrain profile from the original Longley-Rice docs. Check out lines 410-415 for how to calculate distance and profile from the data.

this is definitely not an ocean for me to be getting -999. Any idea what the issue is?

The warning is not saying that you are reading a data value of -999 - it is saying that this raster file does not have a special "NODATA" value set in its metadata, and that the code will assume that if it does happen to read any cell with the value -999, then it will interpret that to mean "no data here" or null. You can verify this by inspecting the tif file in QGIS or using gdalinfo.

Secondly, by surface_profile_m, I understand that the package expects me to provide the (line-of-sight) elevation profile between the transmitter and receiver. This is fine, but what does the package do with the coordinates of the stations then?

Nothing further than what's described above - the coordinates are not passed in to the core model function itmlogic_p2p. The p2p script demonstrates writing the data to shapefiles for further processing or visualisation.

[tomalrussell]

Take a look at demo#L415-450 to see how you could compare each way of using the coordinates and terrain profile.

[ab3ap]

At the risk of commenting on a possibly dead thread (new to me!), the 1982 "A Guide to the Use of the ITS Irregular Terrain Model in the Area Prediction Mode" at NTIA/ITS's web site has helpful information starting in Section 6.3 'Reliability and Confidence.' I've also plotted confidences for a given reliability and reliabilities for a given confidence at https://udel.edu/~mm/itm/