© 2021 The AuthorsThis paper provides a new technique for techno-economic analysis of an off-grid hybrid renewable energy system (HRES). In this study, a photovoltaic (PV) system has been utilized as a primary mover of the HRES which uses a Proton Exchange Membrane Fuel Cells (PEMFC) system as a backup system. Also, H2 storage tank and Electrolyzer (EL) are utilized for supplying the PEMFC. The system has been designed to provide an optimum size selection for the HRES components with considering a suitable total Net Present Cost and loss of power supply probability (LPSP). To get the best results, a new improved metaheuristic, called Fractional-Order Neural Network Algorithm (FONNA) has been utilized for the optimization. The designed system was then applied to a rural building in Yuli County, China. To analyze the system performance, a sensitivity analysis based on the cost variation of the PV, FC, H2 storage tanks and EL is assessed. Simulations show that by using the suggested FONNA, 2.49% LPSP and 5.49% PEE, that will be achieved by selecting 45 ELs, 20 FCs, 25 PVs, and 35 H2 storage tanks. Final results indicate that the suggested approach provides an efficient HRES for use in the studied location.