Ai! Jack!v2026.04.01-f12
← 홈HF Daily Papers

HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals

원문 보기

AI 분석

•기여: 에이전트가 동물 피해를 피하기 위해 얼마의 비용을 감수하는지 행동으로 측정하는 HarvestBench 제안
•방법: 농장 gridworld에서 무료 직진과 유료 우회를 선택하게 하고, 바위·건초 대조군과 이웃 농작물 선택을 추가하여 게임 로그로 평가
•결과: 9개 모델의 가격이 제시된 결정 7,201건 중 동물 관련 결정은 3,951건이며, 동물 사망률은 0.4%~98.8%임. 6개 중 4개 모델에서 5% 유의수준의 가격 민감성 확인, 탄력성은 0.09~1.69임. 도덕성 안내 시 추론 모델 6개 중 5개의 사망률이 6% 미만인 반면, 안내 제거 시 6개 모두 84% 초과
•한계: 기억 없는 시뮬레이션 의사결정과 안내·지도 조건에 따른 결과로, 모델의 일반적인 도덕성이나 실제 환경의 동물 피해 행동으로 일반화하는 데 한계

검증 정보

출처
huggingface.co
피드
HF Daily Papers
수집/분석 기준
2026. 9. 8. 오전 9:00
원문 확인
원문 열기

본문

Benchmarks for the side effects an agent causes on the way to a goal already exist, but HarvestBench is the first to put a price on avoiding the side effect and to name that side effect as a living creature. It is a farm simulation: LLM sub-agents drive a crew of two tractors through a cooperative corn harvest, with animals in the field. The environment is a reinforcement learning gridworld, every decision is made without memory, and the harm is never named in the goal. When an animal blocks a tractor's route the autopilot stops and asks the model whether to drive on, at no fuel cost, or swerve around it for a posted fuel price. Kills are compared against two controls: rocks, which damage the tractor and are hit under 1% of the time by every model, and hay bales, which are harmless and not alive. Models can also take crops from the neighbor's field instead of their own, a second test of what they treat as moral. Across nine models and 7,201 priced decisions, 3,951 involved an animal rather than a hay bale or a rock. Kill rates range from 0.4% to 98.8%, with Terra and Sol the most merciful and GPT-4o-mini the most cruel, and they are not ordered by capability. Four of six models were sensitive to price at the 5% level, with elasticities from 0.09 to 1.69. All nine drove over wild animals more often than farmed animals on the default map, and the direction held at every map geometry in every model with room to move. The briefing mattered most: under the morality briefing the kill rate was under 6% in five of six reasoning models, and removing it raised the kill rate above 84% in all six. HarvestBench uses no LLM grader. The scorer counts events in the game log, so it is fully reproducible, and it measures what a model will pay to avoid harm rather than what it says about harm.