Better explain out of memory error in message queue

[Faless]

Also effectively saves one unnecessary call when everything is fine.

Please expand the file the same condition (buffer_end + room_needed) >= buffer_size was in fact checked at the beginning of each indented block. Instead of failing there, another FAIL_COND_V check was performed outside of the loop.

Please bear with me and double check, I don't want to break the core but saving one if on every call can be a performance improvement 💃

Fixes #8483 (not really a buffer overflow, more like an out of memory error).