Partial stream parsing and O(n) memory requirement

[mrnuke]

I am trying to use jsmn in a constrained environment:

• I can look at 8KiB of a JSON stream at any given time.
• JSON object or array can be longer than the buffer

In this situations, I would need to fit the tokens in a constant memory -- O(1). It is my understanding that the "tokens" array passed to jsmn_parse() should not be cleared in-between calls. This would make it an O(n) memory requirement.

In my situation, the O(n) "tokens" array doesn't work. I found that if I clear the tokens on every iteration via jsmn_init(), I am able to work around this limitation.

The issue appears when the parsing ends midway within an array or object, for example, inside "array_1". A subsequent operation will see the closing bracket ']', assume the JSON is corrupt, and stop parsing. We don't reach "array_2"

	"title" : "Example JSON file with long arrays",
	"array_1" : [
		{...},
		...
	]
	"array_2" : [
		{...},
		...
	]

I propose that the parser context keep track of the recursion depth. Them, if the parser finds a stray ']' or '}', it could distinguish between a JSON error recurse down.