FIXEDD 
Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs
This article will teach you about self-attention mechanisms used in transformer architectures and large language models (LLMs) such…
Research Papers in Jan 2024: Model Merging, Mixtures of Experts, and Towards Smaller LLMs
2023 was the year when the potential and complexity of Large Language Models (LLMs) were growing rapidly. Looking…
Improving LoRA: Implementing Weight-Decomposed Low-Rank Adaptation (DoRA) from Scratch
Low-rank adaptation (LoRA) is a machine learning technique that modifies a pretrained model (for example, an LLM or…
Research Papers in February 2024: A LoRA Successor, Small Finetuned LLMs Vs Generalist LLMs, and Transparent LLM Research
Once again, this has been an exciting month in AI research. This month, I’m covering two new openly…
Tips for LLM Pretraining and Evaluating Reward Models
It’s another month in AI research, and it’s hard to pick favorites. Besides new research, there have also…
Learning from Crossrail’s information management challenge
Nearly a month ago, BIMplus highlighted how the Crossrail project team learned the hard way the trouble associated…
Thomas Flannery: delivering digital for McLaren Construction
Improving digital construction was the brief McLaren Construction gave Thomas Flannery when he joined in late 2022, and…
University of Liverpool digital twin helps decarbonise HVAC
The University of Liverpool has made a five-figure operational cost saving thanks to using a digital twin as…
Galliford Try’s Barry Kingscote crowned CIOB Construction Manager of the Year
Galliford Try project manager Barry Kingscote has scooped the prestigious CIOB Construction Manager of the Year at the…