This study proposed an improved method for synthesizing 1-hydroxyoctane-1,1-bisphosphonic acid (HOBA), and ilmenorutile (one of the important representative niobium-bearing minerals in Bayan Obo) was selected as the research object. Compared with octyl hydroxamic acid (OHA), the flotation mechanism of the synthetic ilmenorutile and HOBA is systematically investigated through contact angle, Zeta potential, FTIR, SEM-EDS, and XPS methods. The results show that under acidic conditions, HOBA can be chelated with ions on the surface of ilmenorutile and stably adsorbed on the surface of ilmenorutile through chemical adsorption, which could improve the floatability of ilmenorutile. Its selectivity for ilmenorutile was significantly better than that of OHA, well explaining the superior flotation results of HOBA over OHA for the actual niobium-bearing minerals in Bayan Obo. Solution chemistry calculation provided more information for the interaction of HOBA and ions on the surface of ilmenorutile in the flotation process, and the reasons for the inhibition effect of Ce3+ and the synergetic enhancement effect of Pb2+ in the interaction between HOBA and ilmenorutile were further discussed. This study indicates that HOBA can serve as an effective collector for ilmenorutile and other niobium-bearing minerals in Bayan Obo under appropriate pH range and ion conditions, and the optimization of its synthesis route provide the possibility for industrial application.