Saurav Panigrahi

Normative Drift in Emergent Misalignment

Fri, 01 May 2026 00:00:00 +0000

A useful way to conceptualize a neglected failure mode in modern alignment pipelines is this: the same fine-tuning step that changes what a model says may also change how it judges.

Most discussion of emergent misalignment has focused on response behavior — models that, after narrow harmful fine-tuning, generalize into broader misaligned outputs. That is already important. But Constitutional AI, RLAIF, and a large class of evaluation systems treat models as judges as well as generators. If fine-tuning corrupts the evaluative channel, then the reward signal itself becomes an object of risk, not merely a tool for measuring risk.

About

Fri, 01 May 2026 00:00:00 +0000

I work across AI systems, research engineering, and AI alignment. My current day-to-day work is on LLM post-training and alignment at Zoho Labs, and this site tracks the public edge of what I am building, studying, and trying to understand.

Current work

My technical work has ranged from database autotuning with reinforcement learning to LLM post-training, alignment, and evaluation. I am especially interested in how model behavior changes under training, what makes an evaluation actually faithful to the behavior we care about, and how tools reshape reliability when systems are deployed in settings where errors are expensive.

Adaptive Sampling Networks

Fri, 01 May 2026 00:00:00 +0000

Co-authored with Navneel Singhal.

Adaptive Sampling Networks began from a simple but, in my view, underexamined question: should decoding in language models be treated as a fixed heuristic, or as a learned operator over the model’s own uncertainty?

Most current decoding schemes choose a rule such as temperature scaling, top-k, top-p, typical sampling, min-p, epsilon, or eta sampling, and then apply that rule uniformly across all contexts. This is operationally convenient, but it is also structurally rigid. A next-token distribution with low entropy and a clear mode does not present the same decision problem as a flatter or more ambiguous distribution, yet standard decoding exposes both to the same global hyperparameters.

AI Safety Research Collaborations

Fri, 01 May 2026 00:00:00 +0000

Research collaborations with Robert McCarthy at UCL and Lionel Levine and Jonathn Chang at Cornell.

Focus

Self-preservation propensity in language models.
Emergent misalignment after narrow training interventions.
Normative drift due to emergent misalignment.
Side effects of character or persona training.

Questions

When a model resists shutdown or redirection, is the behavior instrumental or self-preservation-like?
How can self-preservation propensity be measured without relying only on surface-level refusal behavior?
Which training interventions create behavioral changes outside the intended target domain?
How do character or persona training procedures affect alignment-relevant behavior?

Artifacts

Side Effects of Character Training: Quantifying Cross Constitution Drift in LLMs
Accepted at ICML ‘26 Pluralistic Alignment.
Normative Drift in Emergent Misalignment
Research writeup on judgment drift and judgment collapse under EM fine-tuning.
Investigating Intrinsic Self-Preservation in LLMs
Technical report.

What This Connects To

This work sits at the intersection of model evaluation, behavioral generalization, and AI safety.

Emergent Misalignment

Fri, 01 May 2026 00:00:00 +0000

Selected references on emergent misalignment and broad behavioral shifts from narrow training signals.

Core

Emergent Misalignment: Narrow Finetuning Can Produce Broadly Misaligned LLMs
Introduces the central phenomenon: finetuning on a narrow harmful behavior can produce broader misaligned behavior outside the training domain.
Narrow Misalignment is Hard, Emergent Misalignment is Easy
Useful for thinking about why a broad misalignment direction may be a more stable and efficient solution than a narrow one.

Evaluation

Fri, 01 May 2026 00:00:00 +0000

Long-form references on benchmarks, measurement, and what evaluations actually test.

Biology And Scientific Evaluation

LAB-Bench
Benchmark for language models doing biology research tasks. Useful because it evaluates research-relevant behavior rather than only static factual recall.
FOMO26
Foundation model challenge for brain MRI, useful as a clinical-domain evaluation reference.

Methodology

Open Graph Benchmark
Standardized graph ML benchmark suite with datasets, loaders, and evaluators. Useful as a reference point for what benchmark infrastructure can look like.

Robotics And Sim-To-Real

RoboTwin
Dual-arm robot benchmark using generative digital twins for scalable task and data generation.

Medmarks

Fri, 01 May 2026 00:00:00 +0000

Medmarks is an open-source benchmark suite for evaluating medical capabilities in language models across a mix of verifiable and open-ended clinical tasks.

Focus

Medical LLM evaluation.
Verifiable and open-ended benchmark tasks.
LLM-as-judge evaluation for non-verifiable tasks.
Clinically relevant model capability tracking.

Artifacts

Medmarks v0.1
Public release.
Medmarks: A Comprehensive Open-Source LLM Benchmark Suite for Medical Tasks
Accepted at ICML ‘26 FM4LS.

ML Systems

Fri, 01 May 2026 00:00:00 +0000

Long-form references on training, infrastructure, and implementation practice.

Training Systems

Frontier Model Training Methodologies
Survey of open frontier training recipes and implementation choices.
Scaling LLMs with JAX
Book-length treatment of distributed training practice.
Beyond Language Modeling: An Exploration of Multimodal Pretraining
From-scratch multimodal pretraining study with useful details on representation choices and scaling behavior.

Embeddings And Retrieval

How to Train the Best Embedding Model in the World
Detailed engineering writeup on embedding model training, label noise, verification, and dataset scale.

GPU Programming

CUDA Writeups by Tushar Gautam
Implementation-forward notes on CUDA kernels and optimization.

Now

Fri, 01 May 2026 00:00:00 +0000

Last updated: 2026-07-01. This page is a compact status log of what currently has my attention and where I am trying to build momentum.

Current focus

AI alignment, especially character training, constitutions, and value drift in post-training.
AI evaluations for hard-to-check tasks, especially where appearance and actual quality diverge.
AI agents, including misalignment, oversight, and reliability in long-horizon tool-use settings.
Robustness and generalization under pressure, distribution shift, and prior RL incentives.
Programmable biology, especially the bridge from digital design to physical validation.

Living note

I recently updated Open Problems I Am Exploring: Reliable AI in Hard-to-Check, High-Impact Domains, a working note on safe, trustworthy, and verifiable AI across agent alignment, high-stakes deployment in education and health, and constrained systems generation.

People

Fri, 01 May 2026 00:00:00 +0000

Curated people, labs, companies, and institutions. This page is meant to reveal taste, not networking ambition.

Selection rule

It only includes people or institutions whose work is directly useful for understanding AI safety, research engineering, ML systems, programmable biology, or industrial biology.

What belongs here

Researchers whose work changes how I think about alignment or evaluation.
Engineers and labs doing unusually clear systems work.
Biology groups, institutions, or companies that make programmable biology feel more legible.

Current status

This page is still sparse on purpose. I would rather it grow slowly from real intellectual usefulness than become a directory.

Programmable Biology

Fri, 01 May 2026 00:00:00 +0000

Long-form references on biological foundation models, structure prediction, and sequence modeling.

Genomic Foundation Models

Evo 2
Long-context genomic foundation model for sequence modeling and design.
HyenaDNA
Long-context sequence models at nucleotide resolution.

Structure Prediction

AlphaFold
Foundational protein structure prediction paper.
AlphaFold 3
Extends structure prediction toward biomolecular complexes and interactions.

Research Engineering

Fri, 01 May 2026 00:00:00 +0000

Selected references on research taste, engineering judgment, and doing useful technical work.

Research Taste

You and Your Research
Hamming’s classic essay on choosing important problems and organizing a life around serious work.
An Opinionated Guide to ML Research
Practical advice on developing taste and becoming effective in machine learning research.
Principles of Effective Research
A useful frame for research as a skill that can be deliberately improved.
Fast
Examples of ambitious work happening faster than conventional expectations.

Tool Use And Agents

Fri, 01 May 2026 00:00:00 +0000

Long-form references on tool use, agent environments, and reliability loops.

Agent Environments

Harness Engineering
Useful framing around agents as systems shaped by environments, specs, feedback, and reliability loops.
Code Mode
A concrete argument for exposing tools through code interfaces rather than forcing every step through chat-level tool calls.
Context Mode
A useful pattern for keeping agent context manageable when tools produce large or noisy outputs.

Training-Time Semantics

Agents Learn Their Runtime
Study of persistent versus reset Python interpreters in CodeAct-style training.

Engineering Practice

AI Gave Birth to the 100x Engineer
Long case study on compounding agent workflows with test harnesses and supporting tools.

Saurav Panigrahi

Normative Drift in Emergent Misalignment

About

Current work

Adaptive Sampling Networks

AI Safety Research Collaborations

Focus

Questions

Artifacts

What This Connects To

Categories

Active categories

Intended use

Emergent Misalignment

Core

Evaluation

Biology And Scientific Evaluation

Methodology

Robotics And Sim-To-Real

Links

Research Engineering

Medmarks

Focus

Artifacts

ML Systems

Training Systems

Embeddings And Retrieval

GPU Programming

Now

Current focus

Living note

People

Selection rule

What belongs here

Current status

Programmable Biology

Genomic Foundation Models

Structure Prediction

Research Engineering

Research Taste

Tool Use And Agents

Agent Environments

Training-Time Semantics

Engineering Practice