Non-LLM neural networks & ML foundations for expert learners
100 resources · page 3 of 4
DINOv2: Learning Robust Visual Features without Supervision
by Maxime Oquab et al. (Meta AI)
arXiv HTML version of the DINOv2 paper (2023) on self-supervised visual features.
Deep Learning Recommendation Model for Personalization and Recommendation Systems (DLRM)
arXiv PDF of Meta's DLRM paper (2019), a deep learning recommendation model for personalisation.
DQN (Atari)
by Volodymyr Mnih et al. (Google DeepMind)
arXiv paper (2013) "Playing Atari with Deep Reinforcement Learning", introducing DQN, deep Q-learning from pixels on Atari games.
DreamerV3
by Danijar Hafner et al. (Google DeepMind)
arXiv paper (2023) introducing DreamerV3, a world-model RL agent that masters diverse domains with fixed settings.
EfficientNet
by Mingxing Tan and Quoc V. Le (Google Brain)
ICML 2019 paper (PDF) EfficientNet: scaling convolutional networks by depth, width and resolution.
Faster R-CNN
by Shaoqing Ren et al. (Microsoft Research)
NeurIPS paper (PDF, 2015) Faster R-CNN: real-time object detection with region proposal networks.
GAT
by Petar Veličković et al. (University of Cambridge)
arXiv paper (2017) "Graph Attention Networks", introducing GAT, which weights neighbour nodes with masked self-attention.
GCN
by Thomas N. Kipf and Max Welling (University of Amsterdam)
arXiv paper (2016) "Semi-Supervised Classification with Graph Convolutional Networks", introducing GCN layers for node classification.
GraphSAGE
by William L. Hamilton et al. (Stanford University)
arXiv paper (2017) "Inductive Representation Learning on Large Graphs", introducing GraphSAGE for inductive node embeddings.
GRU
by Kyunghyun Cho et al. (Université de Montréal)
arXiv paper (2014) introducing the RNN encoder–decoder with gated recurrent units (GRU) for machine translation.
GRU empirical evaluation
by Junyoung Chung et al. (Université de Montréal)
arXiv paper (2014) empirically comparing gated recurrent units (GRU) and LSTM on sequence modelling.
IMPALA
by Lasse Espeholt et al. (Google)
arXiv paper (2018) introducing IMPALA, a scalable distributed deep RL architecture.
Inception / GoogLeNet
by Christian Szegedy et al. (Google)
arXiv PDF of the GoogLeNet paper (2014) introducing the Inception architecture.
Mask R-CNN
by Kaiming He et al. (Facebook AI Research)
arXiv paper (2017) "Mask R-CNN", introducing Mask R-CNN for instance segmentation.
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
by Andrew G. Howard et al. (Google)
arXiv PDF of the MobileNets paper (2017) on efficient convolutional networks for mobile vision.
Moirai
by Gerald Woo et al. (Salesforce)
arXiv PDF of the Moirai paper (2024), Salesforce's universal time-series forecasting transformer.
MuZero
by Julian Schrittwieser et al. (Google DeepMind)
arXiv paper (2019) introducing MuZero, planning with a learned model to master Atari, Go, chess and shogi.
Neural Collaborative Filtering
by Xiangnan He et al. (National University of Singapore)
arXiv PDF of the Neural Collaborative Filtering paper (2017) for neural recommender systems.
OpenAI Five (Dota 2)
by OpenAI
arXiv paper (2019) "Dota 2 with Large Scale Deep Reinforcement Learning", describing OpenAI Five's large-scale reinforcement learning for Dota 2.
PPO
by John Schulman et al. (OpenAI)
arXiv paper (2017) "Proximal Policy Optimization Algorithms", introducing PPO, a policy gradient method with a clipped surrogate objective.
Relational inductive biases, deep learning, and graph networks
by Peter W. Battaglia et al. (DeepMind)
arXiv paper (2018) "Relational inductive biases, deep learning, and graph networks", defining the graph network block that unifies many GNN variants.
ResNet
by Kaiming He et al. (Microsoft)
arXiv paper (2015) "Deep Residual Learning for Image Recognition", introducing ResNet residual networks.
RWKV
by Bo Peng et al. (University of British Columbia)
arXiv paper (2023) introducing RWKV, a recurrent architecture trained in parallel like a transformer.
Efficiently Modeling Long Sequences with Structured State Spaces (S4)
by Albert Gu et al. (Stanford University)
arXiv paper (2021) "Efficiently Modeling Long Sequences with Structured State Spaces", introducing S4, structured state space models for long sequences.
SAC
by Tuomas Haarnoja et al. (UC Berkeley)
arXiv paper (2018) introducing Soft Actor-Critic (SAC), off-policy maximum-entropy deep reinforcement learning.
SAM
by Alexander Kirillov et al. (Meta AI)
arXiv paper (2023) introducing Segment Anything (SAM), a promptable image segmentation model and dataset.
A decoder-only foundation model for time-series forecasting (TimesFM)
by Abhimanyu Das et al. (Google)
arXiv paper (2023) introducing TimesFM, a decoder-only foundation model for time-series forecasting.
U-Net
by Olaf Ronneberger et al. (University of Freiburg)
arXiv paper (2015) "U-Net: Convolutional Networks for Biomedical Image Segmentation", introducing the encoder-decoder architecture with skip connections.
VGG
by Karen Simonyan and Andrew Zisserman (University of Oxford)
arXiv PDF of the VGG paper (2014) on very deep convolutional networks for large-scale image recognition.
ViT
by Alexey Dosovitskiy et al. (Google)
arXiv paper (2020) introducing the Vision Transformer (ViT) for image recognition.
WaveNet
by Aaron van den Oord et al. (DeepMind)
arXiv paper (2016) "WaveNet: A Generative Model for Raw Audio", introducing WaveNet, an autoregressive generative model of raw audio.
Wide & Deep
by Heng-Tze Cheng et al. (Google)
arXiv paper (2016) "Wide & Deep Learning for Recommender Systems", introducing Google's Wide & Deep recommender model.