I am interested in solving real world problems using computer vision and machine learning. At Adobe, I work on research and tech transfer projects related to deep learning, image processing and intelligent systems. Before joining Adobe, I studied computer vision with Prof. Stan Sclaroff at Boston University and got my PhD in 2016.
[CV] [Google Scholar] [LinkedIn] [My Old Website]
Contact: jianmzha AT adobe.com
News
- [03/03/2021] Four papers accepted at CVPR 2021. Two of them are selected for oral presentation.
- [10/20/2020] Sky replacement is released in PS [teaser].
- [07/02/2020] Five papers accepted at ECCV 2020.
- [03/04/2020] Three papers accepted at CVPR 2020.
- [07/22/2019] Three papers accepted at ICCV 2019.
- [04/04/2019] Our code for single image shallow DoF is released [code]
- [03/15/2019] Two papers accepted at CVPR 2019.
- [03/15/2019] Our paper SmartEye received Best Paper Honorable Mention at CHI.
- [02/06/2019] Received Adobe’s Tech Excellence Award with three amazing colleagues.
- [01/01/2019] One paper on Layout GAN gets accepted at ICLR.
- [01/01/2019] One paper on Mobile Composition Guidance gets accepted at CHI.
- [10/11/2018] The Content Aware Fit and Content Aware Crop feature are shipped in InDesign CC and Illustrator CC (Demo). I developed the key auto cropping technology behind these features.
- [09/17/2018] Our paper on pointer networks for sequence analysis accepted at NeurIPS.
Selected Publications
PhD Thesis
- Visual Saliency Computation for Image Analysis. Boston University, 2016. [PDF]
Books
- Jianming Zhang, Filip Malmberg, Stan Sclaroff. “Visual Saliency: From Pixel-Level to Object-Level Analysis”. Springer, 2019. [Pub]
Journals
- Andrea Zunino, Sarah Adel Bargal, Pietro Morerio, Jianming Zhang, Stan Sclaroff, Vittorio Murino. “Excitation Dropout: Encouraging Plasticity in Deep Neural Networks”. International Journal of Computer Vision (IJCV), 2021. [Pub]
- Zijun Wei, Boyu Wang, Minh Hoai, Jianming Zhang, Xiaohui Shen, Zhe Lin, Radomir Mech, Dimitris Samaras. “Sequence-to-Segments Networks for Detecting Segments in Videos”. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019. [Pub]
- Xin Ye, Zhe Lin, Joon-Young Lee, Jianming Zhang, Shibin Zheng, Yezhou Yang. “GAPLE: Generalizable Approaching Policy LEarning for Robotic Object Searching in Indoor Environment”. IEEE Robotics and Automation Letters (RAL), 2019. [arXiv]
- Hao Kang, Haoxiang Li, Jianming Zhang, Xin Lu, Bedrich Benes. “FlyCam: Multi-touch Gesture Controlled Drone Gimbal Photography”. IEEE Robotics and Automation Letters, 2018. (selected to be presented at IROS 2018, Spain) [Pub]
- Yuqiu Kong, Jianming Zhang, Huchuan Lu, Xiuping Liu, “Exemplar-aided Salient Object Detection via Joint Latent Space Embedding”. IEEE Transactions on Image Processing, 2018. [Pub]
- Danna Gurari, Kun He, Bo Xiong, Jianming Zhang, Mehrnoosh Sameki, Suyog Dutt Jain, Stan Sclaroff, Margrit Betke, Kristen Grauman. “Predicting Foreground Object Ambiguity and Efficiently Crowdsourcing the Segmentation (s)”. International Journal of Computer Vision (IJCV), 2018. [Pub] [arXiv]
- Jianming Zhang, Sarah Adel Bargal, Zhe Lin, Xiaohui Shen, Jonathan Brandt, Stan Sclaroff. “Top-down Neural Attention by Excitation Backprop”. International Journal of Computer Vision (IJCV), 2018. [Pub] [Project Page]
- Jianming Zhang, Shugao Ma, Mehrnoosh Sameki, Stan Sclaroff, Margrit Betke, Zhe Lin, Xiaohui Shen, Brian Price and Radomír Měch. “Salient Object Subitizing”. International Journal of Computer Vision (IJCV), 2017. [Pub] [Project Page]
- Shugao Ma , Jianming Zhang, Leonid Sigal, Nazli Ikizler-Cinbis and Stan Sclaroff. “Space-Time Tree Ensemble for Action Recognition”. International Journal of Computer Vision (IJCV), 2017. [Pub]
- Shugao Ma, Sarah Adel Bargal, Jianming Zhang, Leonid Sigal, Stan Sclaroff. “Do Less and Achieve More: Training CNNs for Action Recognition Utilizing Action Images from the Web”. Pattern Recognition, 2017. [arXiv] [Project Page]
- Jianming Zhang and Stan Sclaroff. “Exploiting Surroundedness for Saliency Detection: A Boolean Map Approach”. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2016. [PDF] [Supp] [Project Page]
Conferences
- Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, Chunhua Shen. “Learning to Recover 3D Scene Shape from a Single Image”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (oral)
- Yichen Sheng, Jianming Zhang, Bedrich Benes. “SSN: Soft Shadow Network for Image Compositing”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. (oral)
- Qihang Yu, Jianming Zhang, He Zhang, Yilin Wang, Zhe Lin, Ning Xu, Yutong Bai, Alan Yuille. “Mask Guided Matting via Progressive Refinement Network”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
- Xin Yuan, Zhe Lin, Jason Kuen, Jianming Zhang, Yilin Wang, Michael Maire, Ajinkya Kale, Baldo Faieta. “Multimodal Contrastive Training for Visual Representation Learning”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
- Lu Zhang, Jianming Zhang, Zhe Lin, Radomir Mech, Huchuan Lu, You He. “Unsupervised Video Object Segmentation with Joint Hotspot Tracking”. In Proc. European Conference on Computer Vision (ECCV), 2020.
- Lijun Wang, Jianming Zhang, Yifan Wang, Huchuan Lu, Xiang Ruan. “CLIFFNet for Monocular Depth Estimation with Hierarchical Embedding Loss”. In Proc. European Conference on Computer Vision (ECCV), 2020.
- Yu Zeng, Zhe Lin, Jimei Yang, Jianming Zhang, Eli Shechtman, and Huchuan Lu. “High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling”. In Proc. European Conference on Computer Vision (ECCV), 2020.
- Shikun Liu, Zhe Lin, Yilin Wang, Jianming Zhang, Federico Perazzi, and Edward Johns. “Shape Adaptor: A Learnable Resizing Module”. In Proc. European Conference on Computer Vision (ECCV), 2020.
- Xihui Liu, Zhe Lin, Jianming Zhang, Handong Zhao, Quan Tran, Xiaogang Wang, and Hongsheng Li. “Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions”. In Proc. European Conference on Computer Vision (ECCV), 2020.
- Jane L. E, Ohad Fried, Jingwan Lu, Jianming Zhang, Radomír Mech, Jose Echevarria, Pat Hanrahan, James A. Landay. “Adaptive Photographic Composition Guidance”. In ACM CHI Conference on Human Factors in Computing Systems Proceedings (CHI), 2020. [Project Page]
- Ke Xian, Jianming Zhang, Oliver Wang, Long Mai, Zhe Lin, Zhiguo Cao. “Structure-Guided Ranking Loss for Single Image Depth Prediction”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [Project Page]
- Lijun Wang, Jianming Zhang, Oliver Wang, Zhe Lin, Huchuan Lu. “SDC-Depth: Semantic Divide-and-Conquer Network for Monocular Depth Estimation”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [PDF]
- Zijun Wei, Jianming Zhang, Zhe Lin, Joon-Young Lee, Niranjan Balasubramanian, Minh Hoai Nguyen, Dimitris Samaras. “Learning Visual Emotion Representations from Web Data”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [Project Page]
- Lu Zhang, Zhe Lin, Jianming Zhang, Huchuan Lu, You He. “Fast Video Object Segmentation via Dynamic Targeting Network”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2019.
- Yi Zeng, Pingping Zhang, Jianming Zhang, Zhe Lin, Huchuan Lu. “Towards High-Resolution Salient Object Detection”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2019. [arXiv]
- Jason Kuen, Federico Perazzi, Zhe Lin, Jianming Zhang, Yap-Peng Tan. “Scaling Object Detection by Transferring Classification Weights”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2019. [arXiv]
- Lu Zhang, Jianming Zhang, Zhe Lin, Huchuan Lu, You He. “CapSal: Leveraging Captioning to Boost Semantics for Salient Object Detection”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. [PDF] [Project Page]
- Siyuan Qiao, Zhe Lin, Jianming Zhang, and Alan Yuille. “Neural Rejuvenation: Improving Deep Network Training by Enhancing Computational Resource Utilization”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. (oral) [PDF][Project Page]
- Shuai Ma, Zijun Wei, Feng Tian, Xiangmin Fan, Jianming Zhang, Xiaohui Shen, Zhe Lin, Jin Huang, Radomír Měch, Dimitris Samaras, Hongan Wang. “SmartEye: Assisted Instant Photo Taking via the Integration of User Preferences into the View Proposal Network”. In ACM CHI Conference on Human Factors in Computing Systems Proceedings (CHI), 2019. (Best Paper Honorable Mention) [PDF]
- Jianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang and Tingfa Xu. “LayoutGAN: Generating Graphic Layouts with Wireframe Discriminator”. In Proc. International Conference on Learning Representations (ICLR), 2019. [PDF]
- Zijun Wei, Boyu Wang, Minh Hoai, Jianming Zhang, Xiaohui Shen, Zhe Lin, Radomir Mech and Dimitris Samaras. “Sequence-to-Segment Networks for Segment Detection”. In Proc. Advances in Neural Information Processing Systems (NeurIPS), 2018. [PDF]
- Lijun Wang, Xiaohui Shen, Jianming Zhang, Oliver Wang, Zhe Lin, Chih-Yao Hsieh, Sarah Kong, Huchuan Lu. “DeepLens: Shallow Depth Of Field From A Single Image”. In Proc. Siggraph Asia, 2018. [PDF] [Project Page] [code]
- Wei-Chih Hung, Jianming Zhang, Xiaohui Shen, Zhe Lin, Joon-Young Lee,Ming-Hsuan Yang. “Learning to Blend Photos”. In Proc. European Conference on Computer Vision (ECCV), 2018. [PDF] [Project Page]
- Rameswar Panda, Jianming Zhang, Haoxiang Li, Joon-Young Lee, Xin Lu, Amit Roy-Chowdhury, “Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias”. In Proc. European Conference on Computer Vision (ECCV), 2018. [PDF] [Project Page]
- Yufei Wang, Zhe Lin, Xiaohui Shen, Scott Cohen, Jianming Zhang. “ConceptMask: Large-Scale Segmentation from Semantic Concepts”. In Proc. European Conference on Computer Vision (ECCV), 2018.
- Zijun Wei, Jianming Zhang, Min Hoai, Xiaohui Shen, Zhe Lin, Radomir Mech and Dimitris Samaras. “Good View Hunting: Learning Photo Composition from Dense View Pairs”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. [PDF] [Project Page]
- Sarah Adel Bargal*, Andrea Zunino*, Donghyun Kim, Jianming Zhang, Vittorio Murino, Stan Sclaroff. “Excitation Backprop for RNNs.” In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018. [PDF] [Code]
- Filip Malmberg, Robin Strand, Jianming Zhang, Stan Sclaroff. “The Boolean Map Distance: Theory and Efficient Computation”. In Proc. International Conference on Discrete Geometry for Computer Imagery (DGCI), 2017. [Pub]
- Vasili Ramanishka, Abir Das, Jianming Zhang, Kate Saenko. “Top-down Visual Saliency Guided by Captions”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. [PDF] [Project Page]
- Jianming Zhang, Zhe Lin, Jonathan Brandt, Xiaohui Shen, Stan Sclaroff. “Top-down Neural Attention by Excitation Backprop”. In Proc. European Conference on Computer Vision (ECCV), 2016. (oral) [PDF] [Supp] [Project Page]
- Jianming Zhang, Stan Sclaroff, Zhe Lin, Xiaohui Shen, Brian Price and Radomír Měch. “Unconstrained Salient Object Detection via Proposal Subset Optimization”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. (spotlight) [PDF] [Supp] [Project Page]
- Jianming Zhang, Stan Sclaroff, Zhe Lin, Xiaohui Shen, Brian Price and Radomír Měch. “Minimum Barrier Salient Object Detection at 80 FPS”. In Proc. IEEE International Conference on Computer Vision (ICCV), 2015. (oral) [PDF] [Project Page]
- Jianming Zhang, Shugao Ma, Mehrnoosh Sameki, Stan Sclaroff, Margrit Betke, Zhe Lin, Xiaohui Shen, Brian Price and Radomír Měch. “Salient Object Subitizing”. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. [PDF] [Supp] [Project Page]
- Jianming Zhang, Shugao Ma and Stan Sclaroff. “MEEM: Robust Tracking via Multiple Experts using Entropy Minimization”. In Proc. European Conference on Computer Vision (ECCV), 2014. [PDF] [Supp] [Project Page]
- Jianming Zhang and Stan Sclaroff. “Saliency Detection: A Boolean Map Approach”. In Proc. IEEE Internetional Conference on Computer Vision (ICCV), 2013. [PDF] [Project Page]
- Shugao Ma, Jianming Zhang, Nazli Ikizler-Cinbis and Stan Sclaroff. “Action Recognition and Localization by Hierarchical Space-Time Segments”. In Proc. IEEE Internetional Conference on Computer Vision (ICCV), 2013. [PDF] [Project Page]
- Jianming Zhang, Liliana Lo Presti and Stan Sclaroff, “Online Multi-Person Tracking by Tracker Hierarchy”. In Proc. IEEE Conference on Advanced Video and Signal Based Surveillance (AVSS), 2012. [PDF] [Project Page]
Patents
50+ pending or issued patents:
- Large-scale image tagging using image-to-topic embedding (US 10216766B2)
- Semantic Class Localization in Images (US 9846840)
- Generating Image Features Based on Robust Feature-Learning (US 9830526)
- Accurate Tag Relevance Prediction for Image Search (Pending, International)
- Image Zooming (US 9805445)
- Image Cropping Suggestion Using Multiple Saliency Maps (US 9626584)
- Saliency Map Computation (US 9454712)
Honors and Awards
- Best Paper Honorable Mention, CHI 2019
- Technical Excellence Award, Adobe 2019
- Outstanding Reviewer, CVPR 2017
Tech Transfer Highlights
- Sky Replacement in Photoshop 2021 [Youtube]. Starting as a research project years ago, we spent a lot of effort to make this technology achieve the quality level that would satifiy the picky professionals. I developed the backbone masking and compositing technologies and worked closely with the PS team to polish this feature. The final form of this feature is simple, flexible and powerful. It just shines! See what professionals say [rephotojournal.com] [PHLEARN] [PiXimperfect].
- Object Selection Tool in Photoshop 2020 [Youtube]. After nearly three years’ development, engineering and testing, we shipped this new interative selection tool. It makes the selection so intuitive and easy. See reviews from the profesionals [PiXimperfect] [PTC] [YES].
- Conten Aware Fit in InDesign CC 2019 [Youtube]. The key technology behind this is an automatic image cropping algorithm we started to develop years ago. Since then, the algorithm went through several iterations and I was the main developer for the latest version.
- Select Subject in Photoshop CC 2018 [Youtube]. Deep learning based salient object detection with high boundary accuracy. This is one of the first few Sensei powered features in Adobe flagship products. I designed and developed the deep learning algorithm behind this feature. Media coverage: [PetaPixel], [TechCrunch], [Engaget].
Former Interns
- Wei Yin, Adelaide University, 2020
- Qihang Yu, John Hopkins University, 2020
- Lu Zhang, Dalian University of Technology, 2019.
- Yichen Sheng, Purdue University, 2019
- Ke Xian, Huazhong University of Science and Technology, 2019
- He Zhang, Rutgers University, 2018
- Haichao Yu, UIUC, 2018
- Kang Hao, Purdue University, 2018
- Lijun Wang, Dalian University of Technology, 2017.
- Jianan Li, Beijing Institute of Technology, 2017.
- Zhe Wang, UC Irvine, 2017.
- Rameswar Panda, UC Riverside, 2017.
- Wei-Chih (Wayne) Hung, UC Merced, 2017.
- Zijun Wei, Stony Brook University, 2017, 2018.