We’re migrating to Gemini API for some products from OpenAI for our company, but a lot of them utilize the “logit_bias” parameter to basically ban words, or just to force the model to choose between a few certain dictionary of words (classification tasks). We need it in Gemini too.
You could start by open-sourcing the tokenizer for gemini-2.0-flash and up (like OpenAI’s), maybe providing a page similar to https://platform.openai.com/tokenizer which’d allow developers to see the IDs they need to put into the logit_bias map.
This is an important feature for use cases that require strict generation of certain logits, say a Yes/No only generation, or categorization (things that the model just can’t provide any other answers for) and logit_bias helps make the generation much much more reliable for these purposes, besides of course word-banning & biasing (e.g. favorite/disliked dictionary the agent might use) or for writing purposes, making it not use certain words cause they’re cliched and so on.
Thank you, Gemini/Vertex AI team.