Accurate and robust short-term bus travel prediction facilitates operating the bus fleet to provide comfortable and flexible bus services. The built environment, including land use, buildings, and public facilities, has an important influence on bus travel demand prediction. However, previous studies regarded the built environment as a static feature thus even ignored its influence on bus travel in deep learning framework. To fill this gap, we propose a graph deep learning-based approach coupling with spatiotemporal influence of built environment (GDLBE) to enhance short-term bus travel demand prediction. A time-dependent geographically weighted regression method is used to resolve the dynamic influence of the built environment on bus travel demand at different times of the day. A graph deep learning module is used to capture the comprehensive spatial and temporal dependency behind massive bus travel demand. The short-term bus travel demand is predicted by fusing the dynamic built environment influences and spatiotemporal dependency. An experiment in Shenzhen is conducted to evaluate the performance of the proposed approach. Baseline methods are compared, and the results demonstrate that the proposed approach outperforms the baselines. These results will help bus fleet dispatch for smart transportation.