function to emit metrics

[technillogue]

we need the model code to emit accurate input token counts using the correct prompt formatting and tokenizer for the model in question. after some internal discussion from cog import emit_metric seems similar to the best approach for this. presently these metrics are ignored for non-official models and shouldn't really matter anywhere else, so it shouldn't be a concern with billing. this particular approach is janky but should unblock us for now.

usage:

     async def predict(self, ...) -> ...:
        ...
        formatted_prompt = prompt_template.format(prompt=prompt, system_prompt=system_prompt)
        input_token_count = len(self.tokenizer(formatted_prompt )["input_ids"])
        cog.emit_metric(name="input_token_count", value=input_token_count)
        ...

[technillogue]

TODO new doc file

[zeke]

I can't speak to the technical implementation of this, but I think it needs docs.

I would suggest creating a docs/metrics.md file to document why these exists, and the API for sending metrics. Maybe also some recommendations like "Use snake case for metric names" or "Put an x_ prefix on your custom metrics names" or similar.

[zeke]

What does "When running outside of Cog" mean? Should this be "outside of Replicate"?

[technillogue]

fixed; we'll also have another chance to review this when merging into main