Personal project
Attention to Chemistry
A transformer built from raw PyTorch tensors to generate molecules, with tools to inspect what happens inside.
PythonPyTorchRDKit
James Liu
I’m an engineering student at the University of Waterloo.
I’m interested in machine learning, building software, and understanding how things work. I study Management Engineering with an option in AI, graduating in 2028.
Projects
Personal project / Machine learning
Attention to Chemistry
Building a transformer from scratch, then testing what it learned.
I built the model from raw PyTorch tensors, without
torch.nn. Attention, masking, normalization,
training, and activation caching are all accessible in the
implementation.
I then trained it as a target-conditioned molecular generator on 27,236 ChEMBL ligands across ten protein targets. The model has about 821K parameters: small enough to inspect and intervene on directly.
Attention vs. intervention
Looking at attention and testing a model’s behavior can tell different stories. In the ring-closure analysis, the head with the most attention was nearly dispensable. Removing another head, with much less attention, caused a much larger drop in accuracy.
That made interventions more useful than an attention map alone for figuring out what the model actually relied on.
Experience
Full résumé ↗ (PDF, opens in a new tab)2026National Research Council CanadaMachine Learning Research Assistant
May–Aug 2026 · Mississauga, ON
At Clean Energy Innovation, I worked on battery-leaching models. I combined kinetic-model priors with Gaussian-process residual correction to learn from 15 initial experiments. Held-out R² improved from 0.78 to 0.96.
I also developed a constrained Bayesian optimization pipeline that reduced the projected number of additional experiments by 83%. This was part of that placement; the manuscript is in preparation.
Manuscript details ↗2025Loblaw Companies LimitedData Science Intern
Sep–Dec 2025 · Toronto, ON
I built an XGBoost model to forecast backhaul volume across more than 200 transport lanes at 3% weighted absolute percentage error. I also worked with Oracle and Teradata data to understand demand spikes and improve SKU-location forecasts.
2025University College LondonUndergraduate Research Assistant
Apr–Sep 2025 · London, UK
I worked on noise- and constraint-aware Bayesian optimization for formulation development, comparing seven methods across 4,000 simulated campaigns. The work contributed to a peer-reviewed paper in the Journal of Controlled Release.
View publication ↗2025Atlantic Industries LimitedProcess and Data Engineering Intern
Jan–Apr 2025 · Waterloo, ON
I used Statistical Process Control to help reduce production scrap by 20%, and built a Power BI and Python dashboard that automated KPI tracking and reduced manual data entry by 90%.
Publications
Published as Hongdian LiuFast-tracking complex formulation development with multi-objective Bayesian optimisation (opens in a new tab)
Hongyu Liu, Antonia Gucic, Hongdian Liu, Michael T. Cook, David Shorthouse
Physics-informed Bayesian optimization for battery-leaching process development
Liu, H.†; Jaberi, A.†; et al. · †Equal contribution