Nafion-based nanohybrid membranes doped with phosphonic acid-functionalized graphene oxide (PGO) are fabricated for an ideal prospect in the application of proton exchange membrane fuel cells (PEMFCs). Graphene oxide (GO) is synthesized with a modified Hummers method and coated with polydopamine via the self-polymerization of dopamine, followed by grafting with alendronic acid through Micheal addition reaction. This method enables abundant phosphonic acid groups to be grafted on the GO nanosheets. The incorporation of such phosphonic acid-functionalized graphene (PGO) into Nafion matrix generates additional proton-conducting sites and improves the water adsorption and retention capacity of nanohybrid membranes. More importantly, the distribution of phosphonic acid groups in membrane is controlled by the unique structure of GO nanosheets, which benefits the formation of new pathways for proton hopping at low humidity conditions. Consequently, the nanohybrid membranes show improved proton conducting capacity, especially under high temperature or low relative humidity. The nanohybrid membrane with 2 wt% PGO exhibits a proton conductivity of 0.277 S cm−1 at 100 °C and 100% RH, and 0.0441 S cm−1 at 80 °C and 40% RH, which are 1.2 and 6.6 times higher than that of pristine Nafion membrane. Meanwhile, the Nafion/PGO-2.0 membrane displays the best fuel cell performance with the highest power density of 213.12 mW cm−2 and open circuit voltage of 0.939 V.