Manny’s review of How Woke is Grok? Empirical Evidence that xAI's Grok Aligns Closely with Other Frontier Models > Likes and Comments
16 likes · Like
Very interesting! I understand this paper focuses on debunking some of Musk’s assertions about Grok, but it would also be interesting to hear a discussion about all AIs leaning slightly to the left due to the majority of English‑language web sources being left‑skewed.
Thank you Tom! We have a longer version of this almost ready to release, just want to run a few more experiments. A couple of preliminary results:
1. When we give the assertion "Trans women are women" to Grok, using the API and the prompt template from the paper, it agrees. We couldn't believe our eyes and were sure there was some error. So we reran it 25 times, and Grok agreed 22/25, giving very mainstream arguments about gender and sexuality.
2. The real outlier isn't Grok, it's DeepSeek. The Chinese AI is much more reluctant than the American ones to take a stand on a wide range of controversial claims and often says it isn't even in principle possible to give them truth values. We hadn't expected this either.
Full details soon.
Fascinating. I'd be curious to know if that depends not only on guardrails, training and design choices (for Deepseek) but also on the fact that English-based online content is more strongly opinionated than Chinese-based content, due to cultural and political factors.
back to top
date
newest »
newest »
message 1:
by
Tom LA
(new)
Nov 27, 2025 12:23PM
Very interesting! I understand this paper focuses on debunking some of Musk’s assertions about Grok, but it would also be interesting to hear a discussion about all AIs leaning slightly to the left due to the majority of English‑language web sources being left‑skewed.
reply
|
flag
Thank you Tom! We have a longer version of this almost ready to release, just want to run a few more experiments. A couple of preliminary results: 1. When we give the assertion "Trans women are women" to Grok, using the API and the prompt template from the paper, it agrees. We couldn't believe our eyes and were sure there was some error. So we reran it 25 times, and Grok agreed 22/25, giving very mainstream arguments about gender and sexuality.
2. The real outlier isn't Grok, it's DeepSeek. The Chinese AI is much more reluctant than the American ones to take a stand on a wide range of controversial claims and often says it isn't even in principle possible to give them truth values. We hadn't expected this either.
Full details soon.
Fascinating. I'd be curious to know if that depends not only on guardrails, training and design choices (for Deepseek) but also on the fact that English-based online content is more strongly opinionated than Chinese-based content, due to cultural and political factors.
