Position within the page tree

Institute for Visualization and Interactive Systems
Team
Lukas Mehl

Lukas Mehl

M. Sc.

Doctoral Researcher
Institute for Visualization and Interactive Systems (VIS)
Research Group Computer Vision

Contact

+49 711 685 88325
Email

Universitätsstraße 38
70569 Stuttgart
Germany
Room: 1.465

Publications

2023
1. Mehl, L., Jahedi, A., Schmalfuss, J., & Bruhn, A. (2023, January). M-FUSE: Multi-frame Fusion for Scene Flow Estimation. Proc. Winter Conference on Applications of Computer Vision (WACV). https://doi.org/10.48550/arXiv.2207.05704
  - BibTeX
  - Link
  BibTeX
  @inproceedings{Mehl2023_MFUSE, abstract = {Recently, neural network for scene flow estimation show impressive results on automotive data such as the KITTI benchmark. However, despite of using sophisticated rigidity assumptions and parametrizations, such networks are typically limited to only two frame pairs which does not allow them to exploit temporal information. In our paper we address this shortcoming by proposing a novel multi-frame approach that considers an additional preceding stereo pair. To this end, we proceed in two steps: Firstly, building upon the recent RAFT-3D approach, we develop an improved two-frame baseline by incorporating an advanced stereo method. Secondly, and even more importantly, exploiting the specific modeling concepts of RAFT-3D, we propose a U-Net architecture that performs a fusion of forward and backward flow estimates and hence allows to integrate temporal information on demand. Experiments on the KITTI benchmark do not only show that the advantages of the improved baseline and the temporal fusion approach complement each other, they also demonstrate that the computed scene flow is highly accurate. More precisely, our approach ranks second overall and first for the even more challenging foreground objects, in total outperforming the original RAFT-3D method by more than 16%. Code is available at https://github.com/cv-stuttgart/M-FUSE.}, author = {Mehl, Lukas and Jahedi, Azin and Schmalfuss, Jenny and Bruhn, Andr{\'e}s}, booktitle = {Proc. Winter Conference on Applications of Computer Vision (WACV)}, doi = {10.48550/arXiv.2207.05704}, month = {01}, preprinturl = {https://doi.org/10.48550/arXiv.2207.05704}, title = {M-FUSE: Multi-frame Fusion for Scene Flow Estimation}, url = {https://doi.org/10.48550/arXiv.2207.05704}, year = 2023 }
  Link
  https://doi.org/10.48550/arXiv.2207.05704
2. Mehl, L., Schmalfuss, J., Jahedi, A., Nalivayko, Y., & Bruhn, A. (2023). Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4981–4991. https://openaccess.thecvf.com/content/CVPR2023/html/Mehl_Spring_A_High-Resolution_High-Detail_Dataset_and_Benchmark_for_Scene_Flow_CVPR_2023_paper.html
  - BibTeX
  - Link
  BibTeX
  @inproceedings{Mehl_2023_CVPR, abstract = {While recent methods for motion and stereo estimation recover an unprecedented amount of details, such highly detailed structures are neither adequately reflected in the data of existing benchmarks nor their evaluation methodology. Hence, we introduce Spring -- a large, high-resolution, high-detail, computer-generated benchmark for scene flow, optical flow, and stereo. Based on rendered scenes from the open-source Blender movie "Spring", it provides photo-realistic HD datasets with state-of-the-art visual effects and ground truth training data. Furthermore, we provide a website to upload, analyze and compare results. Using a novel evaluation methodology based on a super-resolved UHD ground truth, our Spring benchmark can assess the quality of fine structures and provides further detailed performance statistics on different image regions. Regarding the number of ground truth frames, Spring is 60x larger than the only scene flow benchmark, KITTI 2015, and 15x larger than the well-established MPI Sintel optical flow benchmark. Initial results for recent methods on our benchmark show that estimating fine details is indeed challenging, as their accuracy leaves significant room for improvement. The Spring benchmark and the corresponding datasets are available at http://spring-benchmark.org.}, author = {Mehl, Lukas and Schmalfuss, Jenny and Jahedi, Azin and Nalivayko, Yaroslava and Bruhn, Andr\'es}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {06}, pages = {4981-4991}, title = {Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo}, url = {https://openaccess.thecvf.com/content/CVPR2023/html/Mehl_Spring_A_High-Resolution_High-Detail_Dataset_and_Benchmark_for_Scene_Flow_CVPR_2023_paper.html}, year = 2023 }
  Link
  https://openaccess.thecvf.com/content/CVPR2023/html/Mehl_Spring_A_High-Resolution_High-Detail_Dataset_and_Benchmark_for_Scene_Flow_CVPR_2023_paper.html
3. Schmalfuss, J., Mehl, L., & Bruhn, A. (2023). Distracting Downpour: Adversarial Weather Attacks for Motion Estimation. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 10106–10116. https://openaccess.thecvf.com/content/ICCV2023/html/Schmalfuss_Distracting_Downpour_Adversarial_Weather_Attacks_for_Motion_Estimation_ICCV_2023_paper.html
  - BibTeX
  - Link
  BibTeX
  @inproceedings{Schmalfuss_2023_ICCV, abstract = {Current adversarial attacks on motion estimation, or optical flow, optimize small per-pixel perturbations, which are unlikely to appear in the real world. In contrast, adverse weather conditions constitute a much more realistic threat scenario. Hence, in this work, we present a novel attack on motion estimation that exploits adversarially optimized particles to mimic weather effects like snowflakes, rain streaks or fog clouds. At the core of our attack framework is a differentiable particle rendering system that integrates particles (i) consistently over multiple time steps (ii) into the 3D space (iii) with a photo-realistic appearance. Through optimization, we obtain adversarial weather that significantly impacts the motion estimation. Surprisingly, methods that previously showed good robustness towards small per-pixel perturbations are particularly vulnerable to adversarial weather. At the same time, augmenting the training with non-optimized weather increases a method's robustness towards weather effects and improves generalizability at almost no additional cost. Our code is available at https://github.com/cv-stuttgart/DistractingDownpour.}, author = {Schmalfuss, Jenny and Mehl, Lukas and Bruhn, Andr\'es}, booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, month = {10}, pages = {10106-10116}, title = {Distracting Downpour: Adversarial Weather Attacks for Motion Estimation}, url = {https://openaccess.thecvf.com/content/ICCV2023/html/Schmalfuss_Distracting_Downpour_Adversarial_Weather_Attacks_for_Motion_Estimation_ICCV_2023_paper.html}, year = 2023 }
  Link
  https://openaccess.thecvf.com/content/ICCV2023/html/Schmalfuss_Distracting_Downpour_Adversarial_Weather_Attacks_for_Motion_Estimation_ICCV_2023_paper.html
4. Jahedi, A., Luz, M., Rivinius, M., Mehl, L., & Bruhn, A. (2023). MS-RAFT+: High Resolution Multi-Scale RAFT. International Journal of Computer Vision, 1573–1405. https://doi.org/10.1007/s11263-023-01930-7
  - BibTeX
  - Link
  BibTeX
  @article{jahedi2023msraft, abstract = {Hierarchical concepts have proven useful in many classical and learning-based optical flow methods regarding both accuracy and robustness. In this paper we show that such concepts are still useful in the context of recent neural networks that follow RAFT’s paradigm refraining from hierarchical strategies by relying on recurrent updates based on a single-scale all-pairs transform. To this end, we introduce MS-RAFT+: a novel recurrent multi-scale architecture based on RAFT that unifies several successful hierarchical concepts. It employs a coarse-to-fine estimation to enable the use of finer resolutions by useful initializations from coarser scales. Moreover, it relies on RAFT’s correlation pyramid that allows to consider non-local cost information during the matching process. Furthermore, it makes use of advanced multi-scale features that incorporate high-level information from coarser scales. And finally, our method is trained subject to a sample-wise robust multi-scale multi-iteration loss that closely supervises each iteration on each scale, while allowing to discard particularly difficult samples. In combination with an appropriate mixed-dataset training strategy, our method performs favorably. It not only yields highly accurate results on the four major benchmarks (KITTI 2015, MPI Sintel, Middlebury and VIPER), it also allows to achieve these results with a single model and a single parameter setting. Our trained model and code are available at https://github.com/cv-stuttgart/MS_RAFT_plus.}, affiliation = {Jahedi, A; Luz, M (Corresponding Author), Univ Stuttgart, Inst Visualizat & Interact Syst, Stuttgart, Germany. Luz, M (Corresponding Author), Univ Freiburg, Robot Learning Lab, Freiburg, Germany. Jahedi, Azin; Luz, Maximilian; Mehl, Lukas; Bruhn, Andres, Univ Stuttgart, Inst Visualizat & Interact Syst, Stuttgart, Germany. Luz, Maximilian, Univ Freiburg, Robot Learning Lab, Freiburg, Germany. Rivinius, Marc, Univ Stuttgart, Inst Informat Secur, Stuttgart, Germany.}, author = {Jahedi, Azin and Luz, Maximilian and Rivinius, Marc and Mehl, Lukas and Bruhn, Andrés}, doi = {10.1007/s11263-023-01930-7}, issn = {{0920-5691} and {1573-1405}}, journal = {International Journal of Computer Vision}, pages = {1573-1405}, publisher = {Springer}, pubstate = {prepublished}, research-areas = {Computer Science}, title = {MS-RAFT+: High Resolution Multi-Scale RAFT}, unique-id = {WOS:001126025000002}, url = {https://doi.org/10.1007/s11263-023-01930-7}, year = 2023 }
  Link
  https://doi.org/10.1007/s11263-023-01930-7
2022
1. Schmalfuss, J., Mehl, L., & Bruhn, A. (2022). Attacking Motion Estimation with Adversarial Snow. Proc. ECCV Workshop on Adversarial Robustness in the Real World (AROW). https://doi.org/10.48550/arXiv.2210.11242
  - BibTeX
  - Link
  BibTeX
  @inproceedings{schmalfuss2022attacking, abstract = {Current adversarial attacks for motion estimation (optical flow) optimize small per-pixel perturbations, which are unlikely to appear in the real world. In contrast, we exploit a real-world weather phenomenon for a novel attack with adversarially optimized snow. At the core of our attack is a differentiable renderer that consistently integrates photorealistic snowflakes with realistic motion into the 3D scene. Through optimization we obtain adversarial snow that significantly impacts the optical flow while being indistinguishable from ordinary snow. Surprisingly, the impact of our novel attack is largest on methods that previously showed a high robustness to small L_p perturbations. }, author = {Schmalfuss, Jenny and Mehl, Lukas and Bruhn, Andrés}, booktitle = {Proc. ECCV Workshop on Adversarial Robustness in the Real World (AROW)}, doi = {https://doi.org/10.48550/arXiv.2210.11242}, preprinturl = {https://arxiv.org/abs/2210.11242}, title = {Attacking Motion Estimation with Adversarial Snow}, url = {https://arxiv.org/abs/2210.11242}, year = 2022 }
  Link
  https://arxiv.org/abs/2210.11242
2. Jahedi, A., Mehl, L., Rivinius, M., & Bruhn, A. (2022). Multi-Scale RAFT: combining hierarchical concepts for learning-based optical flow estimation. IEEE International Conference on Image Processing (ICIP). https://doi.org/10.48550/arXiv.2207.12163
  - BibTeX
  - Link
  BibTeX
  @inproceedings{Jahedi2022_ICIP, abstract = {Many classical and learning-based optical flow methods rely on hierarchical concepts to improve both accuracy and robustness. However, one of the currently most successful approaches -- RAFT -- hardly exploits such concepts. In this work, we show that multi-scale ideas are still valuable. More precisely, using RAFT as a baseline, we propose a novel multi-scale neural network that combines several hierarchical concepts within a single estimation framework. These concepts include (i) a partially shared coarse-to-fine architecture, (ii) multi-scale features, (iii) a hierarchical cost volume and (iv) a multi-scale multi-iteration loss. Experiments on MPI Sintel and KITTI clearly demonstrate the benefits of our approach. They show not only substantial improvements compared to RAFT, but also state-of-the-art results -- in particular in non-occluded regions.}, author = {Jahedi, Azin and Mehl, Lukas and Rivinius, Marc and Bruhn, Andrés}, booktitle = {IEEE International Conference on Image Processing (ICIP)}, doi = {10.48550/arXiv.2207.12163}, title = {Multi-Scale {RAFT}: combining hierarchical concepts for learning-based optical flow estimation}, url = {/brokenurl# https://doi.org/10.48550/arXiv.2207.12163}, year = 2022 }
  Link
  /brokenurl# https://doi.org/10.48550/arXiv.2207.12163
2021
1. Mehl, L., Beschle, C., Barth, A., & Bruhn, A. (2021). An Anisotropic Selection Scheme for Variational Optical Flow Methods with Order-Adaptive Regularisation. Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision (SSVM), 140--152. https://doi.org/10.1007/978-3-030-75549-2_12
  - BibTeX
  - Link
  BibTeX
  @inproceedings{10.1007/978-3-030-75549-2_12, abstract = {Approaches based on order-adaptive regularisation belong to the most accurate variational methods for computing the optical flow. By locally deciding between first- and second-order regularisation, they are applicable to scenes with both fronto-parallel and ego-motion. So far, however, existing order-adaptive methods have a decisive drawback. While the involved first- and second-order smoothness terms already make use of anisotropic concepts, the underlying selection process itself is still isotropic in that sense that it locally chooses the same regularisation order for all directions. In our paper, we address this shortcoming. We propose a generalised order-adaptive approach that allows to select the local regularisation order for each direction individually. To this end, we split the order-adaptive regularisation across and along the locally dominant direction and perform an energy competition for each direction separately. This in turn offers another advantage. Since the parameters can be chosen differently for both directions, the approach allows for a better adaption to the underlying scene. Experiments for MPI Sintel and KITTI 2015 demonstrate the usefulness of our approach. They not only show improvements compared to an isotropic selection scheme. They also make explicit that our approach is able to improve the results from state-of-the-art learning-based approaches, if applied as a final refinement step – thereby achieving top results in both benchmarks.}, author = {Mehl, Lukas and Beschle, Cedric and Barth, Andrea and Bruhn, Andrés}, booktitle = {Proceedings of the International Conference on Scale Space and Variational Methods in Computer Vision (SSVM)}, doi = {10.1007/978-3-030-75549-2_12}, pages = {140--152}, publisher = {Springer}, title = {An Anisotropic Selection Scheme for Variational Optical Flow Methods with Order-Adaptive Regularisation}, url = {https://link.springer.com/chapter/10.1007%2F978-3-030-75549-2_12}, year = 2021 }
  Link
  https://link.springer.com/chapter/10.1007%2F978-3-030-75549-2_12

Teaching

Summer term 2022

Tutorial Correspondence Problems in Computer Vision (Coordination)
Seminar Recent Advances in Computer Vision (Supervision)

Winter term 2021/2022

Lecture Computer Vision (Coordination)

Summer term 2021

Tutorial Correspondence Problems in Computer Vision (Tutor)

Winter term 2020/2021

Tutorial Correspondence Problems in Computer Vision (Tutor)
Seminar Bildverarbeitung und Computer Vision (Supervision)

Summer term 2020

Tutorial Correspondence Problems in Computer Vision (Tutor)
Seminar Recent Advances in Computer Vision (Supervision)

Supervised Theses and Projects

Multi-Frame Approaches for Learning Optical Flow Predictions: Master's Thesis, 2020. (co-supervision)
Zwischenbildinterpolation mit optischem Fluss: Bachelor's Thesis, 2020. (co-supervision)
Occlusion-Aware Variational Optical Flow Refinement: Master's Thesis, 2021.
Motion-compensated Frame Interpolation using Multiple Frames: Bachelor's Thesis, 2021.
Diffusion-Based Refinement of Optical Flow: Bachelor's Thesis, 2021.
Pre-training ProFlow: Research Project, 2022.
Variationelle Verfeinerung zur Schätzung von Szenenfluss: Master's Thesis, 2022.
A Global Adversarial Attack on Scene Flow: Master's Thesis, 2023. (co-supervision)

Team page

Lukas Mehl

Contact

Publications

2023

BibTeX

Link

BibTeX

Link

BibTeX

Link

BibTeX

Link

2022

BibTeX

Link

BibTeX

Link

2021

BibTeX

Link

Teaching

Supervised Theses and Projects

Audience

Formalities

Services

Organization

Lukas Mehl

Contact

Publications

2023

BibTeX

Link

BibTeX

Link

BibTeX

Link

BibTeX

Link

2022

BibTeX

Link

BibTeX

Link

2021

BibTeX

Link

Teaching

Supervised Theses and Projects

Here you can reach us

Audience

Formalities

Services

Organization