Fiscal rules are thought to help maintain the sustainability of public finances and underpin fiscal consolidation. This paper estimates a model which regresses the primary balance of local governments in Japan on debt, to the power of debt, the level of temporary government spending, a business cycle indicator and a set of fiscal rules enacted in 2007. Local governments include all prefecture governments over the period from FY1985 to FY2011. Overall, results are consistent with the Bohn (1998) debt sustainability definition. The primary balance initially falls at low levels of debt/GDP. As debt increases beyond certain levels, however, the primary balance rises significantly with increasing debt. The improvement of the primary balance was achieved by expenditure cuts and local tax revenue increases, while intergovernmental transfer played no decisive role. The debt service ratio serves to improve the primary balance of local governments, while net revenue and the current account balance do not play a disciplining role. The introduction of the fiscal rule reduced the explanatory power of fiscal variables.