PPI benchmark evaluation

Data integrity

24 rows removed during cleaning (0.2%)

24 duplicate pairs (0 reversed), 0 self-pairs and 0 pairs with conflicting labels were removed; 11,164 canonical pairs remain.

For model building: Reversed duplicates (A–B and B–A) placed on both sides of a random split leak labels directly. Train on the canonical pair list, not the raw file.

Degree bias in supplied negatives

Endpoint degree SMD 1.16 between positives and supplied negatives

Positive pairs join proteins with mean positive-network degree 6.72, supplied negatives 3.34. Degree-matched N2 brings the SMD to 0.01.

For model building: A model can tell the classes apart by how well-studied the proteins are, without learning anything about interaction. Compare against degree-matched negatives before trusting a score.

Constructed negatives are putative (unobserved) pairs, not experimentally established non-interactions.

Degree-only shortcut

Degree alone reaches ROC-AUC 0.962 on the reference labels

A logistic model on four endpoint-degree summaries, with no sequence information, scores 0.962 on N0 and 0.502 on degree-matched N2 negatives.

For model building: Most of the benchmark's label separation is reproducible from network degree. Headline scores on these labels mix interaction signal with annotation density.

Degree is computed from the same incomplete positive graph that supplies the labels; it is a retrospective audit feature, not an input available for a protein with no known interactions.

Length-only shortcut

Sequence length alone: ROC-AUC 0.540

Endpoint sequence-length summaries used as the only features.

For model building: Sequence length does not separate the labels.

Performance on unseen proteins

ROC-AUC falls by 0.109 from seen (R1) to unseen proteins (R3)

AAC+CTD MLP with the training set held fixed: R0 0.701, R1 0.697, R3 0.587.

For model building: Random pair splits overstate performance on new proteins. Report the protein-disjoint R3 split if the model is meant for proteins without known interactions.

R3 test sets are small; see the uncertainty chapter.

Signal after degree matching

Close to chance (0.517) with degree-matched negatives on unseen proteins

Best representation AAC+CTD, MLP trained and tested with N2 negatives at R3: ROC-AUC 0.517 (N0: 0.587). Component bootstrap 95% CI 0.400–0.960; above chance under both corrected tests: no.

For model building: With degree shortcuts removed and proteins unseen, there is little sequence signal for a model to learn from this dataset as constructed.

A non-significant test does not establish equivalence to chance; few connected components limit what can be concluded.

Independent test units

R3 test graphs contain 5–19 connected components

Test pairs that share proteins are not independent. Uncertainty is computed by resampling whole connected components of the test-pair graph.

For model building: Confidence intervals are wide and tests have little power; differences between models of a few hundredths are not interpretable.

A non-significant test does not establish equivalence to chance; few connected components limit what can be concluded.