I am interested in solving real world problems using computer vision and machine learning. At Adobe, I work on research and tech transfer projects related to deep learning, image processing and intelligent systems. Before joining Adobe, I studied computer vision with Prof. Stan Sclaroff at Boston University and got my PhD in 2016. I recevied my M.S. and B.S. degree in Math at Tsinghua University in 2008 and 2011 respectively.
[CV] [Google Scholar] [LinkedIn] [My Old Website]
Contact: jianmzha AT adobe.com
News
- [09/22/2023] Three papers accepted at ICCV 2023.
- [03/10/2023] Seven papers accepted at CVPR 2023. Three of them are selected as highlights.
- [07/06/2022] Three papers accepted at ECCV 2022.
- [03/08/2022] Three papers accepted at CVPR 2022. One of them is selected for oral presentation.
- [10/25/2021] I will serve as an AC for CVPR 2022.
- [10/01/2021] One paper accepted at ICCV 2021.
- [03/03/2021] Four papers accepted at CVPR 2021. Two of them are selected for oral presentation.
- [10/20/2020] Sky replacement is released in PS [teaser].
- [07/02/2020] Five papers accepted at ECCV 2020.
- [03/04/2020] Three papers accepted at CVPR 2020.
- [07/22/2019] Three papers accepted at ICCV 2019.
- [04/04/2019] Our code for single image shallow DoF is released [code]
Professional Service
- Area Chair: CVPR 2022, WACV 2023
Selected Publications
PhD Thesis
- Visual Saliency Computation for Image Analysis. Boston University, 2016. [PDF]
Books
- Jianming Zhang, Filip Malmberg, Stan Sclaroff. “Visual Saliency: From Pixel-Level to Object-Level Analysis”. Springer, 2019. [Pub]
Journals & Conferences
Image Editing
- Yiqun Mei, He Zhang, Xuaner Zhang, Jianming Zhang, Zhixin Shu, Yilin Wang, Zijun Wei, Shi Yan, HyunJoon Jung, Vishal M Patel. “LightPainter: Interactive Portrait Relighting with Freehand Scribble”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
- Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, Daniel Aliaga. “ObjectStitch: Object Compositing With Diffusion Model”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
- Yu Zeng, Zhe Lin, Jianming Zhang, Qing Liu, John Collomosse, Jason Kuen, Vishal M Patel. “Scenecomposer: Any-level semantic image synthesis”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. (spotlight)
- Yichen Sheng, Jianming Zhang, Julien Philip, Yannick Hold-Geoffroy, Xin Sun, He Zhang, Lu Ling, Bedrich Benes. “PixHt-Lab: Pixel Height Based Light Effect Generation for Image Compositing”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. (spotlight)
- Yichen Sheng, Yifan Liu, Jianming Zhang, Wei Yin, A Cengiz Oztireli, He Zhang, Zhe Lin, Eli Shechtman, Bedrich Benes. “Controllable Shadow Generation Using Pixel Height Maps”. In Proc. European Conference on Computer Vision (ECCV), 2022.
- Juewen Peng, Jianming Zhang, Xianrui Luo, Hao Lu, Ke Xian, Zhiguo Cao. “MPIB: An MPI-Based Bokeh Rendering Framework for Realistic Partial Occlusion Effects”. In Proc. European Conference on Computer Vision (ECCV), 2022.
- Juewen Peng, Zhiguo Cao, Xianrui Luo, Hao Lu, Ke Xian, Jianming Zhang. “BokehMe: When Neural Rendering Meets Classical Rendering”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. (oral)
- Yichen Sheng, Jianming Zhang, Bedrich Benes. “SSN: Soft Shadow Network for Image Compositing”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (oral)
- Yifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, Zhangyang Wang. “SSH: A Self-Supervised Framework for Image Harmonization”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2021.
- Xihui Liu, Zhe Lin, Jianming Zhang, Handong Zhao, Quan Tran, Xiaogang Wang, and Hongsheng Li. “Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions”. In Proc. European Conference on Computer Vision (ECCV), 2020.
- Zijun Wei, Jianming Zhang, Min Hoai, Xiaohui Shen, Zhe Lin, Radomir Mech and Dimitris Samaras. “Good View Hunting: Learning Photo Composition from Dense View Pairs”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. [PDF] [Project Page]
- Lijun Wang, Xiaohui Shen, Jianming Zhang, Oliver Wang, Zhe Lin, Chih-Yao Hsieh, Sarah Kong, Huchuan Lu. “DeepLens: Shallow Depth Of Field From A Single Image”. In Proc. Siggraph Asia, 2018. [PDF] [Project Page] [code]
- Wei-Chih Hung, Jianming Zhang, Xiaohui Shen, Zhe Lin, Joon-Young Lee,Ming-Hsuan Yang. “Learning to Blend Photos”. In Proc. European Conference on Computer Vision (ECCV), 2018. [PDF] [Project Page]
Structure from A Single Image
- Linyi Jin, Jianming Zhang, Yannick Hold-Geoffroy, Oliver Wang, Kevin Blackburn-Matzen, Matthew Sticha, David F Fouhey. “Perspective Fields for Single Image Camera Calibration”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023. (spotlight)
- Yiran Wang, Min Shi, Jiaqi Li, Zihao Huang, Zhiguo Cao, Jianming Zhang, Ke Xian, Guosheng Lin. “Neural Video Depth Stabilizer”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2023.
- Dominique Piché-Meunier, Yannick Hold-Geoffroy, Jianming Zhang, Jean-François Lalonde. “Lens Parameter Estimation for Realistic Depth of Field Modeling”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2023.
- Byeong-Uk Lee, Jianming Zhang, Yannick Hold-Geoffroy, In So Kweon. “Single View Scene Scale Estimation Using Scale Field”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
- Soo Ye Kim, Jianming Zhang, Simon Niklaus, Yifei Fan, Simon Chen, Zhe Lin, Munchurl Kim. “Layered Depth Refinement with Mask Guidance”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
- Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, Chunhua Shen. “Learning to Recover 3D Scene Shape from a Single Image”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (oral and Best Paper Candidate)
- Lijun Wang, Jianming Zhang, Yifan Wang, Huchuan Lu, Xiang Ruan. “CLIFFNet for Monocular Depth Estimation with Hierarchical Embedding Loss”. In Proc. European Conference on Computer Vision (ECCV), 2020.
- Ke Xian, Jianming Zhang, Oliver Wang, Long Mai, Zhe Lin, Zhiguo Cao. “Structure-Guided Ranking Loss for Single Image Depth Prediction”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [Project Page]
- Lijun Wang, Jianming Zhang, Oliver Wang, Zhe Lin, Huchuan Lu. “SDC-Depth: Semantic Divide-and-Conquer Network for Monocular Depth Estimation”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [PDF]
Segmentation and Masking
- Qihang Yu, Jianming Zhang, He Zhang, Yilin Wang, Zhe Lin, Ning Xu, Yutong Bai, Alan Yuille. “Mask Guided Matting via Progressive Refinement Network”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
- Lu Zhang, Jianming Zhang, Zhe Lin, Radomir Mech, Huchuan Lu, You He. “Unsupervised Video Object Segmentation with Joint Hotspot Tracking”. In Proc. European Conference on Computer Vision (ECCV), 2020.
- Lu Zhang, Zhe Lin, Jianming Zhang, Huchuan Lu, You He. “Fast Video Object Segmentation via Dynamic Targeting Network”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2019.
- Lu Zhang, Jianming Zhang, Zhe Lin, Huchuan Lu, You He. “CapSal: Leveraging Captioning to Boost Semantics for Salient Object Detection”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. [PDF] [Project Page]
- Yi Zeng, Pingping Zhang, Jianming Zhang, Zhe Lin, Huchuan Lu. “Towards High-Resolution Salient Object Detection”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2019. [arXiv]
Earlier PHD Work
- Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Xiaohui Shen, Jonathan Brandt, Stan Sclaroff. “Top-down Neural Attention by Excitation Backprop”. International Journal of Computer Vision (IJCV), 2018. [Pub] [Project Page]
- Jianming Zhang and Stan Sclaroff. “Exploiting Surroundedness for Saliency Detection: A Boolean Map Approach”. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016. [PDF] [Supp] [Project Page]
- Jianming Zhang, Shugao Ma, Mehrnoosh Sameki, Stan Sclaroff, Margrit Betke, Zhe Lin, Xiaohui Shen, Brian Price and Radomír Měch. “Salient Object Subitizing”. International Journal of Computer Vision (IJCV), 2017. [Pub] [Project Page]
- Jianming Zhang, Stan Sclaroff, Zhe Lin, Xiaohui Shen, Brian Price and Radomír Měch. “Unconstrained Salient Object Detection via Proposal Subset Optimization”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. (spotlight) [PDF] [Supp] [Project Page]
- Jianming Zhang, Stan Sclaroff, Zhe Lin, Xiaohui Shen, Brian Price and Radomír Měch. “Minimum Barrier Salient Object Detection at 80 FPS”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2015. (oral) [PDF] [Project Page]
- Jianming Zhang, Shugao Ma and Stan Sclaroff. “MEEM: Robust Tracking via Multiple Experts using Entropy Minimization”. In Proc. European Conference on Computer Vision (ECCV), 2014. [PDF] [Supp] [Project Page]
Patents
100+ pending or issued patents:
- Large-scale image tagging using image-to-topic embedding (US 10216766B2)
- Semantic Class Localization in Images (US 9846840)
- Generating Image Features Based on Robust Feature-Learning (US 9830526)
- Accurate Tag Relevance Prediction for Image Search (Pending, International)
- Image Zooming (US 9805445)
- Image Cropping Suggestion Using Multiple Saliency Maps (US 9626584)
- Saliency Map Computation (US 9454712)
Honors and Awards
- Best Paper Honorable Mention, CHI 2019
- Technical Excellence Award, Adobe 2019
- Outstanding Reviewer, CVPR 2017, CVPR 2021
Tech Transfer Highlights
- Adobe Firefly Image 2. I was one of the main contributors in developing the new text2image foundation model.
- Depth Blur in Photoshop 2022 Neural Filter and Lens Blur in Lightroom 2023. These two features are based on our multi-year research effort in single image depth estimation.
- Sky Replacement in Photoshop 2021 [Youtube]. Later shipped to Lightroom. Starting as a research project years ago, we spent a lot of effort to make this technology achieve the quality level that would satifiy the picky professionals. I developed the backbone masking and compositing technologies and worked closely with the PS team to polish this feature. The final form of this feature is simple, flexible and powerful. It just shines! See what professionals say [rephotojournal.com] [PHLEARN] [PiXimperfect].
- Object Selection Tool in Photoshop 2020 [Youtube]. Later shipped to Lightrrom. After nearly three years’ development, engineering and testing, we shipped this new interative selection tool. It makes the selection so intuitive and easy. See reviews from the profesionals [PiXimperfect] [PTC] [YES].
- Conten Aware Fit in InDesign CC 2019 [Youtube]. The key technology behind this is an automatic image cropping algorithm we started to develop years ago. Since then, the algorithm went through several iterations and I was the main developer for the latest version.
- Select Subject in Photoshop CC 2018 [Youtube]. Deep learning based salient object detection with high boundary accuracy. This is one of the first few Sensei powered features in Adobe flagship products. I designed and developed the deep learning algorithm behind this feature. Media coverage: [PetaPixel], [TechCrunch], [Engaget].
Former Interns
- Kaihong Wang, Boston University, 2023
- Jiaxin Xie, Hong Kong University of Science and Technology, 2023
- Jaskirat Singh, Australian National University, 2023
- Bowen Zhang, University of Adelaide, 2022
- ByeongUk Lee, KAIST, 2022
- Yunze Man, UIUC, 2022
- Soo Ye Kim, Korea Advanced Institute of Science and Technology, 2021
- Yifan Liu, Adelaide University, 2021
- Wei Yin, Adelaide University, 2020
- Qihang Yu, John Hopkins University, 2020
- Lu Zhang, Dalian University of Technology, 2019.
- Yichen Sheng, Purdue University, 2019, 2021
- Ke Xian, Huazhong University of Science and Technology, 2019
- He Zhang, Rutgers University, 2018
- Haichao Yu, UIUC, 2018
- Kang Hao, Purdue University, 2018
- Lijun Wang, Dalian University of Technology, 2017.
- Jianan Li, Beijing Institute of Technology, 2017.
- Zhe Wang, UC Irvine, 2017.
- Rameswar Panda, UC Riverside, 2017.
- Wei-Chih (Wayne) Hung, UC Merced, 2017.
- Zijun Wei, Stony Brook University, 2017, 2018.