Broadly speaking, we work at the intersection of computational neuroscience and machine learning, aka AI4(Neuro)Science. Ultimately, we are interested in reverse engineering the algorithms of the brain, in order to figure out how the brain works and to build better artificial intelligence systems.
Check out group's website for more information, and see our open source code below!
- DeepLabCut: for animal pose estimation
- DLC2action: for action segmentation
Computer Vision and Behavioral Analysis:
- HOISDF: Constraining 3D Hand-Object Pose Estimation with Global Signed Distance Fields: Code for Haozhe Qi, Chen Zhao, Mathieu Salzmann, & Alexander Mathis. CVPR (in press) 2024
- WildCLIP: Scene and animal attribute retrieval from camera trap data with domain-adapted vision-language models: Code for Gabeff, Russwurm, Tuia & Mathis BioRxiv 2023. International Journal of Computer Vision 2024
- Bottom-up conditioned top-down pose estimation (BUCTD): Code for Zhou*, Stoffl*, Mathis and Mathis ICCV 2023. State of the art code for performing 2D pose estimation in crowded scenes.
- End-to-end trainable multi-instance pose estimation with transformers: Code for POET model, Stoffl, Vidal & Mathis arxiv 2021
- AcinoSet: A 3D Pose Estimation Dataset and Baseline Models for Cheetahs in the Wild, Joska et al. ICRA 2021
- Primer on Motion Capture, Mathis et al. Neuron 2020
Reinforcement learning (mostly for motor skills):
- Acquiring musculoskeletal skills with curriculum-based reinforcement learning: Code for Chiappa, et al. BioRxiv 2024
- Winning code for the object manipulation track of the MyoChallenge at NeurIPS 2023, article forthcoming
- Latent Exploration for Reinforcement Learning: Code for Chiappa, et al. NeurIPS 2023
- DMAP: a Distributed Morphological Attention Policy for Learning to Locomote with a Changing Body: Code for Chiappa, Marin-Vargas & Mathis NeurIPS 2022
- Winning code for the Baoding ball MyoChallenge at NeurIPS 2022, joint work with Pouget Lab (University of Geneva). Caggiano et al. Proceedings of Machine Learning Research 2022
AI4Science including modeling sensorimotor control:
- Task-driven-proprioception: Code for modeling the proprioceptive system of primates. Marin Vargas* & Bisi* et al. Cell 2024
- ODEformer: symbolic regression of dynamical systems with transformers: Code from d'Ascoli*, Becker*, Schwaller, Mathis & Kilbertus ICLR 2024 (spotlight). Cool code to infer symbolic formulas from data
- DeepDraw: Code for modeling proprioception with task-driven modeling, Sanbrink*, Mamidanna* et al. eLife 2023
🌈 Please reach out, if you want to work with us! We love collaborative, open-source science.
We often collaborate with the group of Mackenzie Mathis, and also recommend checking out their GitHub repository!
