Chapter four
The frontier
A afirmação
The two cells we do not win are Biological Process for the proteins we know little or nothing about: the biological-process wall. This chapter says exactly where it is, and each step is one measurement. The information is not missing: 97 of every 100 process terms we fail to predict already exist in the vocabulary the method can see before the evaluation window opens, and 95 of every 100 once each term is weighted by how much it actually tells you, which is how the score counts them. What we retrieve is not the limit either: hand our own candidate list to a perfect ranker and it is worth 0.752, and up to 0.776 if that ranker also keeps the candidates whose ancestors are right. We deliver 0.213 of it, so we extract 27 percent of what our own shortlist already allows. That is why adding candidates does not pay, and a co-occurrence expansion that lifts recall by half moves almost nothing. It is not where we cut the list either: handing each protein its true number of terms, an oracle no method could have, buys +0.036. Almost all of the rest is ordering, and no clue we carry separates the right candidates from the rest. Nor is it for want of turning the knobs: a different training objective, one model per cell, rank features, class weights, a pruned pool, every variation scores below the recipe we already run. The wall is a fact about our ranking, not about biology, and the signal that would cross it is one we have not found.
Abra o marcador para ver o comprovativo e a operação que regenera o número.
A evidência
| f_micro_w | ||
|---|---|---|
| What we deliver today | 0.213 | the deployed recipe over the pool |
| Perfect ranking of the SAME pool | 0.752 | at precision 1.000; the best ordering of it reaches 0.776. We capture 27 percent: the loss is ranking |
| Perfect per-protein term count | +0.036 | an oracle no method could have, measured against the per-cell arm at 0.2017 |
| Best technique variation we found | none | every knob we turned made it worse |
These are lab evaluations on a freshly retrained booster rather than the sealed f_micro_w board, and they track it closely: the deployed recipe measures 0.213 here against the board's 0.218 on this cell. Every row shares one ground truth and one harness, which is what makes the comparison between the rows the result.
As nossas ressalvas
Declaramo-las antes que alguém as declare por nós.
- The oracle ceiling is what a perfect ranking of our current shortlist would score. It is a bound on this pool, not a score any method could reach, and two thirds of the true process terms are still outside the pool.
- A recall number does not tell you what binds a pipeline; the ceiling of the pool does, and it costs one evaluation with the labels used as the score. That is how the low recall of our shortlist was shown not to be the wall.
- One variation deserves its own line: the binary objective carries a better AUC than the recipe we run (0.823 against 0.790) and a worse score on the metric that decides. AUC ranks these recipes in the opposite order to the benchmark. We do not triage ranking levers by AUC.
- A text-aligned representation (ProtST) lifts process for novel proteins by about 0.062 in an aspect-aware lab evaluation, and remains the most promising outside signal. It is reported here as a direction, not a headline, and the lift is specific to ProtST: a second text model, ProTrek, did not reproduce it.
- Structure recovers the molecular-function cells and leaves the process cells where they were, which is consistent with structure being conserved for function and not for process.