Convention General_1.0 Missing

[Firionus]

The General_1.0 convention is missing in this repo. I think this is due to all General conventions being excluded here: https://github.com/pyfar/sofar/blob/040e729274d5af5a60558cf6e7b5abd913eaa053/sofar/update_conventions.py#L99-L101.

SOFAtoolbox 2.1.5 (with SOFA 2.2 support) writes and reads those files without a problem.

What is the reason to exclude all General conventions from the update process?

[f-brinkmann]

If I remember correctly I had two reasons. First, I think it is not well defined. For example the dimension e is defined by EmitterPosition (see https://www.sofaconventions.org/mediawiki/index.php/General). This is fine at first, however, if you change the data type to FIR-E the dimension e would be defined by Data.IR (see https://www.sofaconventions.org/mediawiki/index.php/GeneralFIR-E). Second there are General conventions for each data type, so I saw no use in having a not well defined convention, when those could be used instead.

If you want to read the data from that convention, you can probably do so with read_sofa_as_netcdf.

In what way are you using the General convention? Could that also be done with another conention, e.g., GenerelTF, or am I missing something and should think about a way to include General?

[Firionus]

My use case was that I wanted to save a RIR with xyz Direction of Arrival (DOA) data for each sample, coming out of the SDM algorithm. I guess GeneralFIR or GeneralFIR-E would work for that instead. However, I wanted to give myself a heads-up when I later look at those files that I'm using a non-standard "data type". I don't plan to publish such files at the moment.

But after looking at the standard I'd agree the General convention is useless. Chapter 4.6 says "Data (...) shall be stored as variables with associated attributes following one of the data types defined in Annex C" and there is a General-convention for every data type in Annex C, so it's not clear why General is needed at all.
I guess it could be used as an "escape hatch" out of validation whenever you need to come up with your own data type but still want a "SOFA-ish" file.
But then again, the csv file from https://sofaconventions.org/conventions/ is really quite restrictive for that use case. Why would Data.IR be mandatory, when I want to use my own data type? Judging only from the standard, it looks like that csv file was created quite arbitrarily.

I'd suggest 2 things:

1. putting a note in the sofar documentation that the "General" convention is not supported, that the specific variants like "GeneralFIR" should be used instead when writing files and such files can be read with read_sofa_as_netcdf
2. printing a short notice when detecting "General" in code. It could look like this:

>>> sofar.Sofa("General")
ValueError: Convention 'General' not supported, use a specific variant like 'GeneralFIR-E'. See `sofar.list_conventions()` for available conventions.

>>> sofar.read_sofa("File_with_General_convention.sofa")
ValueError: Convention 'General' not supported. Try `sofar.read_sofa_as_netcdf`.

I can put together a PR when I find the time.

[f-brinkmann]

I like both suggestions and would welcome the PR. An other convention that is not supported by sofar is GeneralString, which SOFAtoolbox only lists for testing purposes. What do you think of taking care of that as well? I could provide fake SOFA files of those conventions for testing.