Ms. Alicia Tsai

Ph.D. Candidate

University of California, Berkeley

Deep neural networks (DNNs) have been extremely successful across many domains. However, despite their prevalence, there are major gaps in our understanding of the fundamental behavior of deep neural net models. First, there is a challenging set of problems involving robustness, uncertainty and adversarial perturbations in DNNs. Second, mission critical systems that employ DNNs often lack verifiability guarantees as a result of their complex and black-box nature. Specifically, the non-convexity of the decision regions complicates the analysis of fundamental questions such as stability and reachability in dynamical systems.

My research focuses on developing a unified framework of deep implicit models that encompasses DNNs as special cases to address the challenges noted above as well as others in interpretability and robustness. By allowing implicitly defined hidden variables, we simplify the representation and notation of complex DNNs and enable a host of techniques, ranging from linear algebra and Perron-Frobenius theory to robust optimization. This will not only further basic scientific knowledge in machine learning and optimization; it will also put us in a better position to make progress on understanding and interpreting complex deep models, making it robust and more trustworthy. As mentioned earlier, implicit models are in some way equivalent to the “state-space” representations that have been introduced in control theory in the 1970s, bringing to bear linear algebra techniques on problems involving rational functions. However, these connections remain unexplored in the context of neural networks.