How the MIC prediction model was built, tested, and improved — every dataset, feature set, and experiment that got us from a first physicochemical baseline to the model running behind this site.
Every model in this project is trained on antimicrobial peptide activity measurements pulled from DBAASP, the Database of Antimicrobial Activity and Structure of Peptides.
Sequences are matched to their target organism’s taxonomy
(Phylum → Class → Order → Family → Genus) and
Gram status, then de-duplicated by taking the median log-MIC per
sequence–target pair. The regression target is
log10(MIC), so a validation MAE of 0.39
corresponds to an average error of about
10^0.39 ≈ 2.4× in raw MIC — the scale
used throughout this page.
Each stage of the project added a new family of features on top of the last, rather than replacing it. The final production model combines all of them into a single 5,502-dimensional input vector.
Started as 28 global descriptors from Biopython (amino-acid
composition, molecular weight, pI, aromaticity, GRAVY,
secondary-structure fractions). Later expanded via modlAMP and
custom code to 492 columns: amino-acid fractions, terminal and
windowed composition, hydrophobic-moment descriptors, and
reduced-alphabet 2–3-mer motif counts (pos / neg / hyd
/ aro / pol / gly / pro).
One-hot encoded organism context — Phylum, Class, Order, Family, Genus, and Gram status — resolved from NCBI taxonomy lookups against each experiment’s target species. Unknown taxonomy gets its own explicit category rather than being dropped.
Hand-designed ratios such as charge-per-length, hydrophobic-to-charge balance, Boman index per length, and moment balance — compact signals that helped the smaller baseline models before a protein language model was in the picture.
Sequences are passed once through facebook/esm2_t12_35M_UR50D,
a pretrained protein language model, with no fine-tuning.
Per-residue outputs are mean-pooled into a single 480-dim vector
and cached to disk so the transformer never has to run twice on
the same sequence. A PCA-compressed variant reduces this block
further for a cheaper, near-equivalent alternative.
Ten tracked stages, from an empty repository to the model currently deployed on this site. Metrics are validation-split MAE and R² on log10(MIC), single-seed unless noted.
Data loaders, a sequence cleaner, and the first encoder
interfaces (physicochemical, one-hot, Word2Vec, PLM
placeholder) behind a shared BaseModel contract.
No trained model yet — this stage built the scaffolding
every later experiment reused.
The first working regressor: 28 physicochemical descriptors, no organism context at all.
Adding organism taxonomy and Gram status was the single biggest free win in the whole project — a large jump in validation R² before touching the model architecture at all.
Refactored training and evaluation into one shared MIC runner plus a named model registry. Unglamorous, but this is what made the next dozen experiments cheap to run instead of copy-pasted scripts.
Expanded the descriptor set to 492 features (motif k-mers, windowed composition, autocorrelation) and trained XGBoost on top. It did not beat the much simpler Stage 3 baseline — an early lesson that more features and a stronger model aren’t automatically better on a noisy MIC target.
Introduced facebook/esm2_t12_35M_UR50D as a frozen
encoder and ran five parallel experiments to see how it
behaved alone, regularized, and across model families.
Two findings survived this stage: an MLP squeezes more out of dense ESM2 embeddings than a tree model does, and frozen ESM2 alone still doesn’t beat hand-crafted physicochemical descriptors.
Feeding physicochemical descriptors, frozen ESM2 embeddings, and taxonomy/Gram context into one regularized MLP produced the best validation result of the entire project — the PLM adds information on top of hand-crafted features, not instead of them.
Compressed the ESM2 block with PCA to cut dimensionality while keeping validation performance almost unchanged — a near-tie with Stage 7 at a fraction of the embedding size. This is the model this website calls to serve predictions.
The most recent addition to the model registry: an XGBoost variant trained with a pseudo-Huber loss, aimed at making the tree-based branch more robust to MIC outliers. Results aren’t finalized yet.
A new architecture rather than a bigger MLP: a peptide
tower (frozen esm2_t33_650M_UR50D, 14×
larger than the ESM2 model used earlier, followed by a
learnable Transformer encoder and attention pooling) combined
with an organism tower (an FT-Transformer over
species identity and numeric organism features). Trained on
~106k aggregated pairs, split by k-mer Jaccard clustering to
keep near-duplicate sequences out of the test set. For the
first time, the model outputs a calibrated 90% confidence
interval alongside each point prediction, via one of two
interchangeable heads.
Evaluated on a stricter, leakage-safe split than the random validation split used by earlier stages, so these numbers aren’t directly comparable to the MAE/R² chart below — see the dedicated section further down the page for the full breakdown.
Validation-split metrics for all nine tracked variants, in the order they were built. Shorter MAE bars and longer R² bars are better.
Physicochemical descriptors outperformed frozen ESM2 embeddings whenever each feature family was tested alone. Pretrained protein language models did not make domain expertise obsolete on this dataset.
Combining frozen ESM2 embeddings with physicochemical descriptors and taxonomy context beat either feature family alone — the best model in the project is the combined one, not the purely neural one.
Every time regularization was pushed harder to shrink the train–validation gap, validation error got worse, not better — a repeated underfitting trap rather than successful generalization.
The non-PCA sibling (mlp_mic_physchem_esm2_context_regularized)
scored marginally better on validation (MAE 0.387 vs. 0.391), but the
PCA-compressed embedding keeps the served feature vector smaller for
a near-identical result — the tradeoff chosen for this live demo.
The newest addition to the registry swaps the single MLP for a two-tower deep learning architecture and, for the first time, reports a calibrated 90% confidence interval alongside every point prediction — not just a single MIC estimate.
Uppercased sequences pass through a frozen
esm2_t33_650M_UR50D protein language model
(14× larger than the 35M-parameter model used earlier in
the project) to get per-residue embeddings, then through a
learnable Transformer encoder with attention pooling.
A Feature Tokenizer + Transformer jointly encodes species identity and numeric organism-level features, replacing the earlier one-hot taxonomy block with a learned representation.
Trained on ~106k aggregated (peptide, organism) pairs across 15 named species plus an “other” category, with replicate measurements combined by geometric mean in log-space. The train/test split uses k-mer Jaccard similarity clustering to keep near-duplicate sequences out of the test set — a stricter, leakage-safe split than the random validation split used earlier in the project, so its numbers aren’t directly comparable to the MAE/R² chart above.
| Metric (held-out test) | Gaussian head | Quantile head |
|---|---|---|
| RMSE (log₁₀ MIC) | 0.753 | 0.744 |
| MAE | 0.576 | 0.555 |
| Pearson r | 0.455 | 0.493 |
| Spearman ρ | 0.449 | 0.478 |
| Within 2-fold | 34.3% | 37.8% |
| Within 4-fold | 61.8% | 65.1% |
| PICP (nominal 90%) | 82.8% | 75.9% |
| MPIW (interval width) | 1.923 | 1.564 |
The quantile head beats the Gaussian head on every point-prediction metric and produces narrower 90% intervals (1.56 vs. 1.92 log₁₀ units), at the cost of under-covering the nominal 90% target (75.9% vs. 82.8%). Both heads support post-hoc split-conformal calibration to correct this gap without retraining.
Gram-negative species (P. aeruginosa, A. baumannii, K. pneumoniae) get the best Spearman correlation (0.47–0.54), while the fungal C. albicans is the weakest (0.24–0.28) — the same pattern seen throughout the project.
D-amino acid peptides are predicted better than standard L-peptides (Spearman 0.55–0.57 vs. 0.46–0.49), likely because they cluster in a distinct MIC range; modified/X peptides remain the hardest group (Spearman 0.31–0.38).