Skip to content

Giskard Research

Discover new findings, research papers, and expert insights driving progress in AI safety, red teaming, and robust LLM evaluation. This space connects scientists and practitioners passionate about shaping secure, trustworthy artificial intelligence systems

OpenAI won't let you “escape” freely in JSON mode

Accented characters like é may be escaped in JSON as \u00e9. We found that OpenAI's and Azure OpenAI's endpoints can't emit these correctly in JSON mode: after the prefix \u00, the decoder allows only control-character completions (\u0000–\u001f). The output stays valid JSON but holds the wrong bytes — typically a NUL plus literal e9. Once parsed, those control bytes could break some systems and lead to unexpected errors. This is not a JSON limitation but an undocumented endpoint constraint. We describe the failure mode, reproduce it experimentally, and offer practical mitigations.

StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

We introduce StereoTales, a multilingual framework for discovering harmful stereotypes in open-ended LLM generation. By prompting 23 frontier models to write 650,000+ stories across 10 languages and statistically analyzing the demographic associations they produce, we surface over 1,500 significant stereotypes — many shared universally across all models. A human study with 247 participants provides a harmfulness classification of the associations. It shows that all LLMs generate harmful associations, and reveals systematic blind spots in how LLMs themselves judge harm. The harmful stereotypes are usually language-specific, or shared through cultural regions. Harmful associations adapt culturally to the prompt language and amplify bias against locally salient protected groups.

RealHarm: Real-world failure cases of language models applications

Large language model deployments in consumer-facing applications raise significant concerns about potential harms and risks. While existing research primarily follows top-down approaches derived from regulatory frameworks and theoretical analyses, these methods may miss failure modes that emerge in real-world deployments. In this work, we introduce RealHarm, a dataset of problematic interactions with AI agents built from a systematic review of publicly reported incidents. Analyzing harms, causes, and hazards specifically from the deployer's perspective, we find that reputational damage constitutes the predominant organizational harm, while misinformation emerges as the most common hazard category. Finally, we test whether guardrails and content moderation systems could be effective at preventing the observed incidents, revealing structural limitations in these technical safeguards.