{"id":23,"date":"2024-11-07T19:40:09","date_gmt":"2024-11-07T19:40:09","guid":{"rendered":"https:\/\/shaheenahmedc.com\/?page_id=23"},"modified":"2026-04-29T14:50:12","modified_gmt":"2026-04-29T14:50:12","slug":"high-dimensional-bayesian-optimisation-on-expensive-simulators","status":"publish","type":"page","link":"https:\/\/shaheenahmedc.com\/?page_id=23","title":{"rendered":"High-dimensional Bayesian Optimisation on Expensive Simulators"},"content":{"rendered":"\n

MSc Thesis<\/h2>\n\n\n\n

If you build a computationally expensive simulation (of entire economies, climate systems, brains, disease spread, galaxies, text), and it has a number of hyperparameters which you need to tune (interest rates, number of synaptic connections per neuron, vaccination rates, the Hubble constant \\(H_0\\), number of attention heads), how do you do this? <\/p>\n\n\n\n

The volume of your parameter space will grow exponentially with every hyperparameter. Say you want to test at least 10 values for each hyperparameter, across 10 parameters. If you want to explore every combination, that’s immediately \\( 10^{10} \\) (10 billion)!<\/p>\n\n\n\n

On top of this, if one run of your simulation takes 100 seconds (which seems modest), the total runtime will be \\( 10^{10} * 100 \\) seconds, or, in other words, 31,710 years! <\/p>\n\n\n\n

Of course, we might be able to distribute these runs across compute nodes, having access to \\( 10^6 \\) nodes would reduce the runtime to around 12 days. But, this would be, in most situations, prohibitively expensive. <\/p>\n\n\n\n

Can we be smarter about how we explore this parameter space? Do we need all \\( 10^{10} \\) samples, or can we get an acceptable loss value with fewer? <\/p>\n\n\n\n

This is the game Bayesian Optimisation plays, and this is what I chose to write my MSc thesis on, in the context of expensive, high-dimensional macro-economic simulation models. In particular, a flavour of simulations known as agent-based modelling (ABM), which is typically described in this agent-environment lingo, but I like to just think of it as little computer people\/animals\/bugs running around a computer world, bumping into things and each other, buying stuff, eating stuff, running away from and towards each other (in sometimes beautiful fashion<\/a>). <\/p>\n\n\n\n

I was particularly interested in how we could use Bayesian Optimisation to efficiently calibrate expensive macro-economic ABMs, which often had dozens of hyperparameters. <\/p>\n\n\n\n

Within the thesis, I:
<\/p>\n\n\n\n