Fine-tuning and fusion
Towards accurate protein–peptide binding-affinity prediction. My B.Sc. Physics thesis at the Chair of Theoretical Biophysics, Technical University of Munich, supervised by Prof. Dr. Martin Zacharias and advised by M.Sc. Niklas Halbwedl.
Three questions
Question 1
Does Boltz-2’s affinity head transfer to peptides?
No. It reaches r = 0.30 at best, and that signal largely tracks molecular weight.
Question 2
Does fine-tuning help, and do four architectures learn the same thing?
It helps every model, lifting three of them to about r = 0.44. Yet they learn different signals and correlate only weakly with each other.
Question 3
Does combining them beat any single model?
Yes. A de-biased mean of all four reaches r = 0.573, with a mean absolute error of 1.46 kcal/mol.
The problem
Cyclic peptides are large enough to engage protein surfaces that small molecules cannot reach, yet can stay cell-permeable. That makes them attractive drugs, and hard to model.
Boltz-2 predicts the structure of biomolecular complexes and, through a separate head, their binding affinity. That affinity head was trained only on protein–small-molecule data, so peptides sit outside its training distribution. It also accepts the binder only as a SMILES string, a text notation for small molecules.
Structure first
Encoding the peptide as SMILES hurt structure prediction badly. The median centroid distance to the crystal pose was 3.56 Å, against 0.74 Å when the peptide was encoded as a chain of residues. To keep the better structure and still use the SMILES-only affinity head, I built Boltzina, a fork of Boltz-2 that skips the structure module and injects externally computed coordinates. It runs 7.3 to 11.8 times faster.
Then affinity
Untuned, the affinity head reached r ≈ 0.30 at best, well below the r ≈ 0.62 Boltz-2 reports for small-molecule ligands, and the signal mostly tracked molecular weight.
I fine-tuned the head under both encodings. On SMILES input, test-set r rose from 0.257 to 0.438. On residue input it rose from 0.059 to 0.431. Alongside, I fine-tuned a graph neural network (GEMS, r = 0.431) and a protein language model (r = 0.340) on the same receptor-disjoint split of about 3,000 complexes.
The four models correlate only weakly with each other, even the two that share an architecture. Averaging their de-biased predictions gave r = 0.573, with a mean absolute error of 1.46 kcal/mol, better than any single model and better than fusing the untuned versions (r = 0.435). The gain holds after controlling for peptide size and interface residue count.
Fine-tuning and fusion each help on their own, and more so together.
Combining several independently fine-tuned models looks like a cheaper alternative to designing and training one large, purpose-built model from scratch.
One fused score
0.573
The four fine-tuned models, averaged, beat every one of them alone. The test set is receptor-disjoint at 90 % sequence identity and spans far more receptor families than the training data, so the number measures generalization rather than recall.
Fine-tuning and combining existing models could be a cheaper route than training one large purpose-built model from scratch.