manuscript

COMAL: A Convergent Meta Algorithm for Aligning LLMs with General Preferences