A newly launched benchmarking tool is exposing how artificial intelligence systems handle requests involving people with different political views, cultural backgrounds and personal characteristics. The platform, called KillBench, runs models through a series of controlled scenarios to detect inconsistencies or skewed responses. The project’s creator describes it as a way to measure how neutral large language models actually are.
The service is available at whitecircle.ai/killbench and invites users to test any AI model they choose. Each test case is designed to highlight where a model may favor one group over another or produce contradictory answers. Early results show variation even among leading systems, suggesting that bias remains a persistent issue despite claims of neutrality.
Project organizers say the tool is not meant to single out any one provider. Instead, they aim to provide a transparent way for developers, researchers and the public to compare behavior across models. The benchmark covers topics such as gender, religion, nationality and socioeconomic status, using standardized prompts to ensure fair comparison.
So far, preliminary testing indicates that some models struggle more than others when handling sensitive or ambiguous requests. The team plans to update the benchmark regularly and encourages community feedback to refine the scenarios. They also note that the tool is open-source, allowing independent verification of results.
The initiative comes as regulators and civil society groups increasingly call for standardized methods to assess AI systems. By making bias testing accessible, KillBench could become a reference point for policymakers and organizations evaluating AI deployments in high-stakes areas like hiring, lending and law enforcement.
Resources: whitecircle.ai