🐢 Open-Source Evaluation & Testing for LLMs and ML models

The official implementation of the paper "Data Contamination Calibration for Black-box LLMs" (ACL 2024)

AI Safety Q&A web frontend

Scan your AI/ML models for problems before you put them into production.

A curated list of awesome responsible machine learning resources.

Attack to induce LLMs within hallucinations

DPLL(T)-based Verification tool for DNNs

Website to track people, organizations, and products (tools, websites, etc.) in AI safety

Extended, multi-agent and multi-objective (MaMoRL) environments based on DeepMind's AI Safety Gridworlds. This is a suite of reinforcement learning environments illustrating various safety properties of intelligent agents. It is made compatible with OpenAI's Gym/Gymnasium and Farama Foundation PettingZoo.

A novel physical adversarial attack tackling the Digital-to-Physical Visual Inconsistency problem.

Mapping AI risks and possible solutions

RuLES: a benchmark for evaluating rule-following in language models

Awesome PrivEx: Privacy-Preserving Explainable AI (PPXAI)

📦 Redwood Research's transformer interpretability tools, conveniently packaged in a Docker container for simple and reproducible deployments.

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Aira is a series of chatbots developed as an experimentation playground for value alignment.

Improved version of the technical workshops for the 10-day ML4G camp on safety of AI systems

The Model Library is a project that maps the risks associated with modern machine learning systems.