Dieses Bild zeigt Jenny Schmalfuss

Jenny Schmalfuss

Frau M. Sc.

Wiss. Mitarbeiterin
Institut für Visualisierung und Interaktive Systeme (VIS)
Abteilung Computer Vision

Kontakt

Universitätsstraße 38
70569 Stuttgart
Deutschland
Raum: 1.465

Fachgebiet

My research interests are at the intersection of computer vision and machine learning. Here, I research new ways to quantify and improve the robustness of motion estimation and specifically optical flow methods under adversarial attacks and distribution shifts. My goal is to enable robust motion estimation by understanding what influences the robustness of current methods.

No defense is better than a bad one against adversarial patches...
...because most defenses lower both quality and robustness if the attacks considers the defense. Our paper illuminates the empty promises of current patch-defenses for optical flow methods, and will appear at WACV 2024!

Weather perturbs motion estimation. But can we turn it into an adversarial attack?
Test your optical flow methods under Distracting Downpour, which was accepted to ICCV 2023! Also, my recent presentation of the project at ICVSS 2023 won a Poster Award and our precursor work on Attacking motion estimation with adversarial snow won a Best Paper Award at the  ECCV AROW Workshop 2022.

Are all optical flow networks equally vulnerable to adversarial attacks?
Check out our paper on evaluating the robustness of optical flow methods, which was an Oral at ECCV 2022!

Short Bio

I am a doctoral researcher at the Computer Vision Department of the University of Stuttgart, and a scholar of the International Max Planck Research School for Intelligent Systems (IMPRS-IS). I hold a MSc and BSc degree in Simulation Technology from the University of Stuttgart, where I focused on numerical mathematics, computer vision and machine learning. During my studies, I worked as research intern at the National University of Singapore and the University of Houston. 

Find me on google scholar or research gate.

News: I will be visiting the Data and Web Science Group at University of Mannheim for a talk on March 05, 2024.

Talks: I gave a talk at the TrustML Young Scientist Seminars about "Challenges in Adversarial Attacks for Motion Estimation". You can find the video on YouTube.

  1. 2024

    1. Scheurer, E., Schmalfuss, J., Lis, A., & Bruhn, A. (2024). Detection Defenses: An Empty Promise against Adversarial Patch Attacks on Optical Flow. In Proc. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE/CVF. https://arxiv.org/abs/2310.17403
  2. 2023

    1. Mehl, L., Jahedi, A., Schmalfuss, J., & Bruhn, A. (2023, Januar). M-FUSE: Multi-frame Fusion for Scene Flow Estimation. Proc. Winter Conference on Applications of Computer Vision (WACV). https://doi.org/10.48550/arXiv.2207.05704
    2. Schmalfuss, J., Scheurer, E., Zhao, H., Karantzas, N., Bruhn, A., & Labate, D. (2023). Blind image inpainting with sparse directional filter dictionaries for lightweight CNNs. Journal of Mathematical Imaging and Vision (JMIV), 65, 323–339. https://doi.org/10.1007/s10851-022-01119-6
    3. Schmalfuss, J., Mehl, L., & Bruhn, A. (2023, Oktober). Distracting Downpour: Adversarial Weather Attacks for Motion Estimation. Proc. International Conference on Computer Vision (ICCV). https://arxiv.org/abs/2305.06716
    4. Mehl, L., Schmalfuss, J., Jahedi, A., Nalivayko, Y., & Bruhn, A. (2023). Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://spring-benchmark.org/
  3. 2022

    1. Schmalfuss, J., Mehl, L., & Bruhn, A. (2022). Attacking Motion Estimation with Adversarial Snow. Proc. ECCV Workshop on Adversarial Robustness in the Real World (AROW). https://doi.org/10.48550/arXiv.2210.11242
    2. Schmalfuss, J., Scholze, P., & Bruhn, A. (2022). A perturbation-constrained adversarial attack for evaluating the robustness of optical flow. In S. Avidan, G. Brostow, M. Cissé, G. M. Farinella, & T. Hassner (Hrsg.), Proc. European Conference on Computer Vision (ECCV) (Bd. 13682, S. 183--200). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-20047-2_11
  4. 2021

    1. Schmalfuss, J., Riethmüller, C., Altenbernd, M., Weishaupt, K., & Göddeke, D. (2021). Partitioned coupling vs. monolithic block-preconditioning approaches for solving Stokes-Darcy systems. Proc. International Conference on Computational Methods for Coupled Problems in Science and Engineering (COUPLED PROBLEMS). https://doi.org/10.23967/coupled.2021.043
  5. 2016

    1. Wanigasekara, N., Schmalfuss, J., Carlson, D., & Rosenblum, D. S. (2016). A bandit approach for intelligent IoT service composition across heterogeneous smart spaces. Proc. International Conference on the Internet of Things, 121–129. https://doi.org/10.1145/2991561.2991562

Winter 2023/2024:

  • Computer Vision and Intelligent Systems - Colloquium (Organization)

Summer 2023:

  • Recent Advances in Computer Vision - Seminar (Supervision)
  • Computer Vision and Intelligent Systems - Colloquium (Organization)

Winter 2022/2023:

  • Computer Vision and Intelligent Systems - Colloquium (Organization)

Summer 2022:

  • Recent Advances in Computer Vision - Seminar (Supervision)
  • Computer Vision and Intelligent Systems - Colloquium (Organization)

Winter 2021/2022:

  • Computer Vision - Tutorial (Coordinator)
  • Computer Vision and Intelligent Systems - Colloquium (Organization)

Summer 2021:

  • Imaging Science - Tutorial (Coordinator)
  • Recent Advances in Computer Vision - Seminar (Supervision)
  • Computer Vision and Intelligent Systems - Colloquium (Organization)

Winter 2020/2021:

  • Computer Vision - Tutorial (Coordinator)
  • Bildverarbeitung und Computer Vision - Seminar (Supervision)
  • Computer Vision and Intelligent Systems - Colloquium (Organization)

Summer 2020:

  • Imaging Science - Tutorial (Coordinator)

Before Summer Term 2020:

  • Imaging Science - Tutorial (Tutor)
  • Computer Vision - Tutorial (Tutor)

"Preconditioning Techniques for Coupled Stokes Darcy Systems" (Master's thesis, 2019/2020)

"Learning to Transform: Neural Networks with Sparse Directional Filters for Blind Inpainting" (research project in cooperation with the University of Houston, 2018/2019)

"Detection of  Alternative Splicing in Coral Reef Fish Exposed to Climate Change" (research project with the King Abdullah University of Science and Technology, 2017)

"Simulation and Adaptation of Contextual Bandit Algorithms for IoT Service Discovery" (Bachelor's thesis in cooperation with the National University of Singapore, 2016)

"Differentiable illumination for naturalistic particle attacks on motion estimation". Bachelor’s Thesis. University of Stuttgart, 2023.

"A feature-level analysis of the adversarial robustness of RAFT". Work in progress. Master’s Thesis. University of Stuttgart, 2023.

"Assessing the robustness of optical flow algorithms: A comparative analysis of adversarial attacks and training". Research Project. University of Stuttgart, 2023.

"Attacks on defended optical flow networks for action recognition". Research Project. University of Stuttgart, 2023.

"A Global Adversarial Attack for Scene Flow". Master’s Thesis. University of Stuttgart, 2023.

"Filter dictionaries for optical flow prediction with RAFT". Master’s Thesis. University of Stuttgart, 2023.

"Sparse filters for optical flow robustness". Research Project. University of Stuttgart, 2022.

"An optimization approach for attacking the Horn and Schunck model". Bachelor’s Thesis. University of Stuttgart, 2022.

"Differentiating the Variational Horn and Schunck Method with Implicit Functions". Bachelor’s Thesis. University of Stuttgart, 2022.

"Atacking a defended optical flow network". Master’s Thesis. University of Stuttgart, 2022.

"A framework for attacking optical flow networks". Research Project. University of Stuttgart, 2022.

"Switching sparsity: investigating a training time adaptive layer". Research Project. University of Stuttgart, 2021.

"Stability of neural network architectures for optical flow estimation under adversarial attacks". Awarded Best SimTech Bachelor's thesis 2021. Bachelor’s Thesis. University of Stuttgart, 2021.

"Investigating the influence of learning rates on the learning speed of neural networks". Bachelor’s Thesis. University of Stuttgart, 2021.

"Analysing the quality of interframe interpolation with optical flow". Bachelor’s Thesis. University of Stuttgart, 2021.

"Improved RAFT architectures for optical flow estimation". Master’s Thesis. University of Stuttgart, 2021.

Internship: I will join NVIDIAs Autonomous Vechicle Perception Research Group in Santa Clara, CA, USA, for an internship in 2024.

News: I will be visiting the Data and Web Science Group at University of Mannheim for a talk on March 05, 2024.

News: I will be at WACV'24 from January 04-08 at Waikoloa Beach, Hawaii.

No defense is better than a bad one against adversarial patches...
...because most defenses lower both quality and robustness if the attacks considers the defense. Our paper illuminates the empty promises of current patch-defenses for optical flow methods, and will appear at WACV 2024!

News: I will be at ICCV'23 from October 02-06 in Paris.

Weather perturbs optical flow. But can we turn it into an adversarial attack?
Test your optical flow methods under Distracting Downpour, which was accepted to ICCV 2023! Also, my recent presentation of the project at ICVSS 2023 won a poster award! More info here.

News: I will be at ICVSS'23 from July 09-15 in Sicily.

Meet Spring: The new benchmark for scene flow, optical flow and stereo!
Test our dataset at spring-benchmark.org, and read the preprint for our CVPR 2023 paper.

TrustML YSS Talk: I gave a talk at the TrustML Young Scientist Seminars about "Challenges in Adversarial Attacks for Motion Estimation". You can find the video on YouTube.

News: I will be visiting the Institute of Computer Graphics and Vision at TU Graz for a talk from December 19-20, 2022.

News: I will be visiting the Robust Machine Learning Group at MPI-IS Tübingen for a talk on November 17, 2022.

Snowy conditions can break your optical flow method.
Our work on Attacking motion estimation with adversarial snow won a Best Paper Award at the  ECCV AROW Workshop 2022! More info here.

News: I will be at ECCV'22 from October 23-27 in Tel Aviv.

Are all optical flow networks equally vulnerable to adversarial attacks?
Check out our paper on evaluating the robustness of optical flow methods, which was an Oral at ECCV 2022!

Zum Seitenanfang