TextSLAM: Visual SLAM with Semantic Planar Text Features

Authors:   Boying Li   Danping Zou   Yuan Huang   Xinghan Niu   Ling Pei   Wenxian Yu

CODE   DATASET   EVALUATION

Related Papers:

  • TextSLAM: Visual SLAM with Planar Text Features
    •   Boying Li, Danping Zou, Daniele Sartori, Ling Pei, Wenxian Yu
      IEEE International Conference on Robotics and Automation (ICRA). 2020

  • TextSLAM: Visual SLAM with Semantic Planar Text Features
    •   Boying Li, Danping Zou, Yuan Huang, Xinghan Niu, Ling Pei, Wenxian Yu
      IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). 2023

    Motivation

     TextSLAM is a novel visual Simultaneous Localization and Mapping system (SLAM) tightly coupled with semantic text objects.

      Humans can read texts and navigate complex environments using scene texts, such as road markings and room names. Why not robots?

      TextSLAM explores scene texts as the basic feature both geometrically and semantically. It achieves superior performance even under challenging environments, such as image blurring, large viewpoint changes, and significant illumination variations (day and night).

    Overview of TextSLAM

    Overview of TextSLAM dataset

    Abstract

       We propose a novel visual SLAM method that integrates text objects tightly by treating them as semantic features via fully exploring their geometric and semantic prior. The text object is modeled as a texture-rich planar patch whose semantic meaning is extracted and updated on the fly for better data association. With the full exploration of locally planar characteristics and semantic meaning of text objects, the SLAM system becomes more accurate and robust even under challenging conditions such as image blurring, large viewpoint changes, and significant illumination variations (day and night). We tested our method in various scenes with the ground truth data. The results show that integrating texture features leads to a more superior SLAM system that can match images across day and night. The reconstructed semantic 3D text map could be useful for navigation and scene understanding in robotic and mixed reality applications.

    Results

    BibTeX

    @article{li2023textslam,
    title={TextSLAM: Visual SLAM with Semantic Planar Text Features},
    author={Li, Boying and Zou, Danping and Huang, Yuan and Niu, Xinghan and Pei, Ling and Yu, Wenxian},
    booktitle={arXiv preprint},
    year={2023} }

    @inproceedings{li2020textslam,
    title={TextSLAM: Visual SLAM with Planar Text Features},
    author={Li, Boying and Zou, Danping and Sartori, Daniele and Pei, Ling and Yu, Wenxian},
    booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
    year={2020} }