Portrait

Postdoctoral Research Associate
Department of Experimental Psychology, University of Oxford

Supervisors: Christopher Summerfield and Tim Behrens

Education:
PhD in Electrical Engineering, Stanford University
Thesis: Deep linear neural networks: A theory of learning in the brain and mind
Advisers: Jay McClelland (primary), Andrew Ng, Christoph Schreiner, and Surya Ganguli

BSE in Electrical Engineering, Princeton University (summa cum laude)

Curriculum Vitae: pdf

Research

The theory of deep learning and its applications to phenomena in neuroscience and psychology.

Publications

Saxe, A.M., McClelland, J.L., Ganguli, S. (2018). A mathematical theory of semantic development in deep neural networks. arXiv.
pdf | arxiv

Bansal, Y., Advani, M., Cox, D. D., & Saxe, A. M. (2018). Minnorm training: an algorithm for training over-parameterized deep neural networks. arXiv.
pdf | arxiv

Masís, J., Saxe, A. M., & Cox, D. D. (2018). Rats optimize reward rate & learning speed in a 2-AFC task. Poster at COSYNE. Denver.
pdf

Saxe*, A. M., & Advani*, M. (2018). A theory of memory replay and generalization performance in neural networks. Poster at COSYNE. Denver. *Equal contributions.
pdf

Zhang, Y., Saxe, A. M., Advani, M. S., & Lee, A. A. (2018). Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning. Molecular Physics, 1–10.
pdf | arxiv

Earle, A. C., Saxe, A. M., & Rosman, B. (2018). Hierarchical Subtask Discovery with Non-Negative Matrix Factorization. In Y. Bengio & Y. LeCun (Eds.), the International Conference on Learning Representations. Vancouver, Canada.
pdf | workshop version

Saxe, A. M., Bansal, Y., Dapello, J., Advani, M., Kolchinsky, A., Tracey, B. D., & Cox, D. D. (2018). On the Information Bottleneck Theory of Deep Learning. In Y. Bengio & Y. LeCun (Eds.), the International Conference on Learning Representations. Vancouver, Canada.
pdf | code

Nye, M., & Saxe, A. (2018). Are Efficient Deep Representations Learnable? In Y. Bengio & Y. LeCun (Eds.), Workshop Track at the International Conference on Learning Representations. Vancouver, Canada.
pdf

Advani*, M., & Saxe*, A.M. (2017). High-dimensional dynamics of generalization error in neural networks. arXiv. *Equal contributions.
pdf | arxiv

Musslick, S., Saxe, A.M., Ozcimder, K., Dey, B., Henselman, G., & Cohen, J.D. (2017, July). Multitasking Capability Versus Learning Efficiency in Neural Network Architectures. In M. Knauff, M. Paulen, N. Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 39th annual meeting of the Cognitive Science Society. (pp. 829-834). Austin, TX: Cognitive Science Society.
pdf

Saxe, A.M., Earle, A.C., & Rosman, B. (2017). Hierarchy Through Composition with Multitask LMDPs. In ICML 2017.
pdf | supplementary material

Tsai*, C.Y., Saxe*, A., & Cox, D. (2016) Tensor Switching Networks. In NIPS 2016. *Equal contributions.
pdf | supplementary material

McClelland, J.L., Sadeghi, Z. & Saxe, A.M. (2016). A Critique of Pure Hierarchy: Uncovering Cross-Cutting Structure in a Natural Dataset. Neurocomputational Models of Cognitive Development and Processing (pp 51-68). World Scientific.
pdf

Saxe, A.M. (2016, August). Inferring actions, intentions, and causal relations in a neural network. Poster at CogSci 2016, Quebec City, Canada.
pdf

Saxe, A.M., & Norman, K. (2016, February). Optimal storage capacity associative memories exhibit retrieval-induced forgetting. Poster at COSYNE 2016. Salt Lake City.
pdf

Baldassano*, C., & Saxe*, A.M. (2016, February). A theory of learning dynamics in perceptual decision-making. Poster at COSYNE 2016. Salt Lake City. *Equal contributions.
pdf

Saxe, A.M. (2015, March). A deep learning theory of perceptual learning dynamics. Poster at COSYNE 2015. Salt Lake City.
pdf

Lee, R., & Saxe, A.M. (2015, March). The effect of pooling in a deep learning model of perceptual learning. Poster at COSYNE 2015. Salt Lake City.
pdf

Goodfellow, I.J., Vinyals, O., & Saxe, A.M. (2015). Qualitatively characterizing neural network optimization problems. In Y. Bengio & Y. LeCun (Eds.), International Conference on Learning Representations. San Diego, CA.
pdf | arxiv

Saxe, A.M. (2014, July) Multitask model-free reinforcement learning. Poster at CogSci 2014, Quebec City, Canada.
pdf | code upon request

Lee, R., Saxe, A., & McClelland, J.L. (2014, July). Modeling perceptual learning with deep networks. Poster at CogSci 2014, Quebec City, Canada.
pdf

Saxe, A.M., McClelland, J.L., & Ganguli, S. (2014). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. In Y. Bengio & Y. LeCun (Eds.), International Conference on Learning Representations. Banff, Canada.
pdf | arxiv

Saxe, A.M., McClelland, J.L., and Ganguli, S. (2013) Dynamics of learning in deep linear neural networks. In NIPS Workshop on Deep Learning 2013.
pdf | supplementary material

Saxe, A.M., McClelland, J.L., and Ganguli, S. (2013) Learning hierarchical category structure in deep networks. In M. Knauff, M. Paulen, N. Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 35th annual meeting of the Cognitive Science Society. (pp. 1271-1276). Austin, TX: Cognitive Science Society.
pdf

Saxe, A.M., McClelland, J.L., and Ganguli, S. (2013) A Mathematical Theory of Semantic Development. Poster at COSYNE 2013, Salt Lake City.
pdf

Saxe, A., Bhand, M., Mudur, R., Suresh, B., & Ng, A. (2011) Unsupervised learning models of primary cortical receptive fields and receptive field plasticity. In NIPS 2011.
pdf | supplementary material | data upon request

Saxe, A., Koh, P.W., Chen, Z., Bhand, M., Suresh, B., & Ng, A. (2011). On random weights and unsupervised feature learning. In ICML 2011.
pdf | code

Saxe, A., Bhand, M., Mudur, R., Suresh, B., & Ng, A. (2011, February). Modeling cortical representational plasticity with unsupervised feature learning. Poster at COSYNE 2011, Salt Lake City.
pdf

Saxe, A., Koh, P.W., Chen, Z., Bhand, M., Suresh, B., & Ng, A. (2010). On random weights and unsupervised feature learning. In NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning.
pdf | supplementary material | code

Balci, F., Simen, P., Niyogi, R., Saxe, A., Hughes, J.A., Holmes, P., Cohen, J.D. (2010). Acquisition of decision making criteria: reward rate ultimately beats accuracy. Attention, Perception, & Psychophysics, 1–18.
pdf

Goodfellow, I. J., Le, Q. V., Saxe, A. M., Lee, H., & Ng, A.Y. (2009). Measuring invariances in deep networks. In NIPS 2009.
pdf

Baldassano, C.A., Franken, G.H., Mayer, J.R., Saxe, A.M., & Yu, D.D. (2009). Kratos: Princeton University’s entry in the 2008 Intelligent Ground Vehicle Competition. Proceedings of SPIE.
pdf

Atreya, A.R., Cattle, B.C., Collins, B.M., Essenburg, B., Franken, G.H., Saxe, A.M., et al. (2006). Prospect Eleven: Princeton University’s entry in the 2005 DARPA Grand Challenge. Journal of Field Robotics, 23(9), 745-753.
pdf

Teaching

Lecture slides on backpropagation

Software

Object recognition with features from random weight TCNNs

Matlab maximally informative dimension solver