Research Papers in Nov 2023: Tackling Hallucinations, Boosting Reasoning Abilities, and New Insights into the Transformer Architecture
FIXEDD This month, I want to focus on three papers that address three distinct problem categories of Large Language…
FIXEDD
Ten Noteworthy AI Research Papers of 2023
This year has felt distinctly different. I’ve been working in, on, and with machine learning and AI for…
Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs
This article will teach you about self-attention mechanisms used in transformer architectures and large language models (LLMs) such…
Research Papers in Jan 2024: Model Merging, Mixtures of Experts, and Towards Smaller LLMs
2023 was the year when the potential and complexity of Large Language Models (LLMs) were growing rapidly. Looking…
Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch
Low-rank adaptation (LoRA) is a machine learning technique that modifies a pretrained model (for example, an LLM or…
Research Papers in February 2024: A LoRA Successor, Small Finetuned LLMs Vs Generalist LLMs, and Transparent LLM Research
Once again, this has been an exciting month in AI research. This month, I’m covering two new openly…
Tips for LLM Pretraining and Evaluating Reward Models
It’s another month in AI research, and it’s hard to pick favorites. Besides new research, there have also…
Thomas Flannery: delivering digital for McLaren Construction
Improving digital construction was the brief McLaren Construction gave Thomas Flannery when he joined in late 2022, and…
Learning from Crossrail’s information management challenge
Nearly a month ago, BIMplus highlighted how the Crossrail project team learned the hard way the trouble associated…