Part 8/14:
Proxy Model: Uses GP to build a surrogate that predicts outcomes across the search space. The covariance matrix size, which grows with the number of evaluated points, impacts computational feasibility.
Acquisition Function: Guides the search process by balancing exploration (sampling uncertain areas) and exploitation (sampling promising areas). Expected Improvement (EI) is a common choice, quantifying the expected gain over the current best observation.
Sequential Sampling: The optimizer iteratively evaluates the true function at the most promising points suggested by the acquisition function, updating the surrogate model after each evaluation.
This process continues until convergence criteria are met or computational budgets are exhausted.