Non-LLM neural networks & ML foundations for advanced learners
106 resources · page 3 of 4
DenseNet
by Gao Huang et al. (Cornell University)
arXiv paper (2016) "Densely Connected Convolutional Networks", introducing DenseNet, where each layer receives feature maps from all earlier layers.
DETR
by Nicolas Carion et al. (Meta)
arXiv HTML version of the DETR paper (2020) on end-to-end object detection with transformers.
DINOv2: Learning Robust Visual Features without Supervision
by Maxime Oquab et al. (Meta AI)
arXiv HTML version of the DINOv2 paper (2023) on self-supervised visual features.
Deep Learning Recommendation Model for Personalization and Recommendation Systems (DLRM)
arXiv PDF of Meta's DLRM paper (2019), a deep learning recommendation model for personalisation.
DQN (Atari)
by Volodymyr Mnih et al. (Google DeepMind)
arXiv paper (2013) "Playing Atari with Deep Reinforcement Learning", introducing DQN, deep Q-learning from pixels on Atari games.
DreamerV3
by Danijar Hafner et al. (Google DeepMind)
arXiv paper (2023) introducing DreamerV3, a world-model RL agent that masters diverse domains with fixed settings.
EfficientNet
by Mingxing Tan and Quoc V. Le (Google Brain)
ICML 2019 paper (PDF) EfficientNet: scaling convolutional networks by depth, width and resolution.
ESPnet
by ESPnet
ESPnet documentation: end-to-end speech processing toolkit with recipes for recognition, synthesis and translation.
Faster R-CNN
by Shaoqing Ren et al. (Microsoft Research)
NeurIPS paper (PDF, 2015) Faster R-CNN: real-time object detection with region proposal networks.
GAT
by Petar Veličković et al. (University of Cambridge)
arXiv paper (2017) "Graph Attention Networks", introducing GAT, which weights neighbour nodes with masked self-attention.
GCN
by Thomas N. Kipf and Max Welling (University of Amsterdam)
arXiv paper (2016) "Semi-Supervised Classification with Graph Convolutional Networks", introducing GCN layers for node classification.
GraphSAGE
by William L. Hamilton et al. (Stanford University)
arXiv paper (2017) "Inductive Representation Learning on Large Graphs", introducing GraphSAGE for inductive node embeddings.
GRU
by Kyunghyun Cho et al. (Université de Montréal)
arXiv paper (2014) introducing the RNN encoder–decoder with gated recurrent units (GRU) for machine translation.
GRU empirical evaluation
by Junyoung Chung et al. (Université de Montréal)
arXiv paper (2014) empirically comparing gated recurrent units (GRU) and LSTM on sequence modelling.
IMPALA
by Lasse Espeholt et al. (Google)
arXiv paper (2018) introducing IMPALA, a scalable distributed deep RL architecture.
Inception / GoogLeNet
by Christian Szegedy et al. (Google)
arXiv PDF of the GoogLeNet paper (2014) introducing the Inception architecture.
Jumanji
Jumanji documentation: InstaDeep's suite of JAX reinforcement-learning environments for combinatorial and decision problems.
Mask R-CNN
by Kaiming He et al. (Facebook AI Research)
arXiv paper (2017) "Mask R-CNN", introducing Mask R-CNN for instance segmentation.
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
by Andrew G. Howard et al. (Google)
arXiv PDF of the MobileNets paper (2017) on efficient convolutional networks for mobile vision.
Moirai
by Gerald Woo et al. (Salesforce)
arXiv PDF of the Moirai paper (2024), Salesforce's universal time-series forecasting transformer.
Montreal Forced Aligner
by Montreal Forced Aligner project (McGill University)
Montreal Forced Aligner documentation: tool for aligning speech audio with transcripts and training pronunciation models.
MuZero
by Julian Schrittwieser et al. (Google DeepMind)
arXiv paper (2019) introducing MuZero, planning with a learned model to master Atari, Go, chess and shogi.
Neural Collaborative Filtering
by Xiangnan He et al. (National University of Singapore)
arXiv PDF of the Neural Collaborative Filtering paper (2017) for neural recommender systems.
OpenAI Five (Dota 2)
by OpenAI
arXiv paper (2019) "Dota 2 with Large Scale Deep Reinforcement Learning", describing OpenAI Five's large-scale reinforcement learning for Dota 2.
PPO
by John Schulman et al. (OpenAI)
arXiv paper (2017) "Proximal Policy Optimization Algorithms", introducing PPO, a policy gradient method with a clipped surrogate objective.
Relational inductive biases, deep learning, and graph networks
by Peter W. Battaglia et al. (DeepMind)
arXiv paper (2018) "Relational inductive biases, deep learning, and graph networks", defining the graph network block that unifies many GNN variants.
ResNet
by Kaiming He et al. (Microsoft)
arXiv paper (2015) "Deep Residual Learning for Image Recognition", introducing ResNet residual networks.
RL4CO
RL4CO documentation: reinforcement-learning benchmark and library for combinatorial optimisation problems.
RWKV
by Bo Peng et al. (University of British Columbia)
arXiv paper (2023) introducing RWKV, a recurrent architecture trained in parallel like a transformer.
Efficiently Modeling Long Sequences with Structured State Spaces (S4)
by Albert Gu et al. (Stanford University)
arXiv paper (2021) "Efficiently Modeling Long Sequences with Structured State Spaces", introducing S4, structured state space models for long sequences.
SAC
by Tuomas Haarnoja et al. (UC Berkeley)
arXiv paper (2018) introducing Soft Actor-Critic (SAC), off-policy maximum-entropy deep reinforcement learning.
SAM
by Alexander Kirillov et al. (Meta AI)
arXiv paper (2023) introducing Segment Anything (SAM), a promptable image segmentation model and dataset.