Training an Autonomous Pentester with Deep RL
Presented at Strange Loop 2021
Abstract: Deep reinforcement learning has proven useful in training agents that learn useful tasks through trial and error. Can we use these techniques in the infosec space to create an autonomous pentesting agent? Previously successful agents have been built mostly in the context of games like Go or DOTA that can be sped up to make the techniques practical with the massive training data size requirements that come with deep RL, and can be naturally broken down into state and action spaces. Penetration testing does not have an obvious discrete state or action space and resetting an environment built out of virtual machines for every training episode would be too slow to be practical.
To solve these problems, we use the popular Metasploit penetration testing framework to break out a space of possible actions and state. Then, we simulate vulnerable networks using partially observed Markov decision processes to allow the agent to rapidly acquire training data. Finally, we remove the agent from the simulation in order to test that the behaviors learned in simulation can be used to pilot Metasploit to compromise a real-life vulnerable host.
Client Side Deep Learning Optimization with PyTorch
Presented at Strange Loop 2021
Abstract: Deep learning has the capacity to take in rich, high dimensional data and produce insights that can create totally new mobile experiences for developers. However, the constraints of network availability and latency limit what kinds of work can be done in the mobile application space and vastly increase the cost to developers. We have recently developed a customer facing mobile application that leverages real-time computer vision models and will share our experiences of moving multiple deep learning models from the server onto the client. In this presentation, we dive into technical solutions for porting custom architectures for various vision tasks and how to serialize them from Python to binary assets, while avoiding common issues such as unsupported hardware instructions. We also discuss the theory and practice of quantizing models, model fusion, and storing tensors in last memory format for optimization. We conclude by demonstrating how to benchmark the performance of client-side models for various devices and operating systems.