Millimeter-wave (mmWave) frequencies offer the multi-gigabit throughput required by next-generation UAV networks, but their directional propagation makes link quality sensitive to the material composition of nearby obstacles. Geometric planners treat all obstacles as binary blockers, while signal-strength planners chase high-SNR zones without distinguishing stable line-of-sight from fragile multipath. Both fail in urban environments where concrete, metal, glass, and vegetation coexist. We propose a control framework that fuses mmWave radar sensing with visual semantic classification to construct an effective capacity metric that decouples raw signal strength from material-dependent link reliability. A two-layer planner generates collision-free paths that avoid both hard blockers and unreliable multipath zones, then deforms the trajectory via gradient optimization to maximize reliable throughput. Hardware characterization with a 60 GHz sensor confirms that radar returns alone cannot distinguish spectrally distinct materials. Monte Carlo simulations show that the proposed controller improves effective throughput by 65% over signal-strength baselines while maintaining 0.96 link reliability, reaching 97.3% of an oracle with perfect global channel knowledge.