EMBO Workshop “Computational structural biology”

Value of the Information & Strength of the Argument

The talk provides substantial value by conducting a systematic evaluation of contemporary ML docking and affinity prediction methods. The argumentation is strong: PoseBusters checks are intentionally lenient (permitting energies 100 times the average and severe van der Waals clashes), yet many deep learning methods still fail them. She illustrates this with concrete examples and quantitative results across diverse methods and datasets. Her analysis of binding affinity prediction is particularly compelling: simple baselines that ignore interactions achieve correlations near 0.76 on CASF 2016, outperforming more sophisticated methods. This demonstrates that current models often rely on data biases rather than learning physics. The reasoning is logical, supported by data, and presented with clarity, making a convincing case for the need for rigorous benchmarking and cautious interpretation of ML results in structural biology.

Scientific Rigor, Source Quality, Title Accuracy

The scientific rigor is high: Deane references established databases (PDBbind, CASF 2016) and prior publications, and she discusses the limitations of her own benchmark. She also acknowledges that their approach is not perfect and encourages community use of their tools. The title ‘EMBO Workshop Computational structural biology’ is the workshop’s general title, and the talk’s content directly addresses that theme, focusing on computational methods for understanding protein-ligand interactions. The presentation is well-organized and based on published research, though the talk format inevitably compresses some technical details. No external links were provided in the video description, but the speaker cites specific tools and methods (e.g., DiffDock, Uni-Mol, PoseBusters) that are identifiable in the literature.

The title is the workshop name, which is broad; the talk focuses on evaluating ML for docking and affinity prediction, fitting the theme.

8/10

Talk by a prominent researcher presenting rigorous benchmarking and critical analysis of ML methods in structural biology. Methods are well documented, but as a talk it simplifies some details.

Markers derived by PSI from the transcript: the creator did not define chapters.

• 01:00Introduction and context: rising deep learning docking papers, and the initial excitement.
• 05:00Martin's finding: many deep learning docking outputs are physically impossible; PoseBusters introduced.
• 10:00Description of the PoseBusters benchmark creation and its checks.
• 15:00Results: deep learning methods perform well on ASTEX but fail on PoseBusters; classical methods are more robust.
• 20:00Analysis of performance vs. sequence identity to training data; deep learning methods fail on novel targets.
• 25:00Discussion of simple baselines for binding affinity prediction and their surprising effectiveness.
• 30:00Demonstration that CASF 2016 is highly biased, and methods are not outperforming bias baselines.
• 35:00Conclusion: ML methods often learn biases; the need for better benchmarks and baseline models.

• PoseBusters — Tool and benchmark developed by Deane's group to validate molecular poses.
• ASTEX Diverse Set — Standard test set for docking, but heavily overlapping with training data of ML methods.
• PDBbind — Database used for training and testing binding affinity models.
• CASF 2016 — Benchmark for scoring functions; shown to be biased with high sequence identity to training data.
• DiffDock — Deep learning-based docking method evaluated in the talk.
• Uni-Mol — Deep learning model for molecular interaction prediction, also evaluated.
• GNINA and Vina — Classical docking tools used as baselines.
• RDKit — Used for computing 2D descriptors for ligand bias baseline.