Project ยท In progress

WhatIf: A Simulated Twin of German Vote Intention

A simulation of German national vote intention that answers with outcome distributions under scenarios and is checked against what actually happened.

View repository

Overview

WhatIf simulates one measurable outcome, the weekly national poll average in Germany, under different scenarios. It does not give a single prediction. It gives a distribution of outcomes and scores it against what happened afterwards and against simple baselines. A main research question is whether language-model agents look better in backtests only because the model already knows what happened (hindsight leak).

This project is still in progress. The data pipeline, baselines, backtests and synthetic population are built; the voter and party agents are not built yet.

What I built so far

  • Collected published national polls and averaged them per week.
  • Added two baselines: a naive "same as last week" forecast and a damped-trend statistical model.
  • Built backtests that start at past dates and score outcome distributions with CRPS.
  • Loaded official federal election results and constituency statistics for 2017, 2021 and 2025.
  • Built a synthetic voter population matched to official statistics per constituency.
  • Stored each downloaded dataset with a timestamp and metadata so runs can be reproduced.
  • Wrote tests for each part.

Planned next

  • Voters whose party preference shifts with events, the economy and other voters.
  • Party agents driven by language models, compared with rule-based stand-ins.
  • A hindsight leak experiment with masked and unmasked prompts on two models with different training cutoffs.
  • Scenarios, rolling backtests with plots, and forward forecasts.

Tools

  • Python
  • NumPy
  • pytest
  • Agent-based simulation
  • Probabilistic forecasting
View repository