Policy Claims
With Weak Evidence:
A Critique of the
Reason Foundation Study of
in the
Victoria
Basolo and Nico Calavita
EXECUTIVE SUMMARY
The Reason Foundation study of
• The research design lacks a comparison between cities with and without IH and the authors do not provide long-term data by year. It is therefore impossible to ascertain whether the decline in housing production in localities with IH is: 1) due to IH per se or, 2) part of an overall downward trend in housing production in the Bay Area or, 3) due to the time of enactment of IH in relation to the economic cycle. For example, fifteen of the IH programs in the Bay Area were passed between 1989 and 1992, just before or at the very beginning of the economic recession of the early and mid-1990s.
• Shortcomings in data collection plague the analysis throughout the report and case counts are presented in confusing ways. For instance, the authors have incomplete information on 17 out of 50 cities they list and use the average of the remaining 33 cities to "fill" in the incomplete information.
• The authors use largely anecdotal data to substantiate some of their claims, (e.g., the comments of a builder on page 28), and do not provide scientific evidence on which to base conclusions. More rigorous, systematic qualitative and/or survey research is needed before generalizing about the perspectives of developers.
• In calculating the cost of IH, the report utilizes homeownership figures only, when a large percentage of units provided with IH are less costly rental units. The report's authors also insist that developers do not pay any of the costs of IH, but only landowners - who sell to developers at a reduced price to reflect the cost of IH - and the buyers and renters of IH. However, there is general agreement among observers and researchers of IH that developers are likely to share some of the cost.
• In calculating the additional cost of IH to the developer, the authors assume that the developer is mandated in all cases to build the IH units, that the units will cost the same as the market units to build, that there are no incentives and subsidies available to the developer, and that developers are not capable of being innovative in finding solutions that reduce their costs. These assumptions do not reflect reality.
• The authors view IH abstractly and rigidly, and not as the outcome of political and economic processes that attempt to balance the community's interests with those of the developers. IH has evolved as a multifaceted approach aimed in most cases at reducing political opposition by neutralizing or reducing the additional costs that developers incur in providing affordable housing. Any analysis of the costs associated with IH needs to take into account the evolving nature of a system marked by flexibility, adaptability and change.
• Similarly, the authors present an elementary discussion of supply and demand relying on a static model and ignoring that housing is a special case of a good that can be viewed as having many quality sub-markets and a finite supply of available land for development.
We agree with the authors that empirical research is needed
to evaluate IH as a policy mechanism to produce affordable housing. The need
for future research was emphasized in the recent report,
BACKGROUND
In April 2004, the Reason Foundation (RF) -- a libertarian think tank -- published a study titled, “Housing Supply and Affordability: Do Affordable Housing Mandates Work?” This study, authored by Professors Benjamin Powell and Edward Stringham, claims to investigate empirically the “effects of inclusionary zoning and examine whether it is an effective public policy response to high housing prices” (Executive Summary, 1st page). The RF report provides a preliminary and substantially flawed analysis of the effects of inclusionary housing (IH) in the San Francisco Bay (SFB) area. The purposes of this critique paper are: 1) to identify the major weaknesses of the research; 2) to examine the reasonableness of policy claims based on the research; and 3) to recommend a more rigorous and policy-relevant route for future research on IH.
WEAKNESSES OF THE RESEARCH
The research in the RF report has many weaknesses that severely diminish its power to provide useful policy recommendations. While this critique paper does not address all the weaknesses of the research, it discusses several major shortcomings that render the RF report wholly insufficient for the purposes of policy development and/or evaluation. This critique is organized around several problem areas including the scope and assumptions of the research, the research design, the quality of the data analysis, and misleading assertions.
Scope
The research is narrowly defined and lacks depth and balance in its framing and analytic interpretation. The authors comment, “Without knowing the economic and real-world consequences of inclusionary zoning, policymakers have difficulty assessing the merits or faults of inclusionary zoning” (p. 2). The implication is that they will provide the information to adequately assess IH policy. They do not. The research falls considerably short of a full analysis of the “real world consequences” of IH and, as a result, has very limited utility for policymakers desiring to assess the merits or faults of the policy. Specifically, the authors never acknowledge, or even ponder, any of the benefits of IH, and instead, seem to dismiss IH’s direct value: the provision of affordable housing.
The analytic framework presented in Part 2 of the RF report consists of an elementary discussion of supply and demand. The authors describe a static model and ignore that housing is a special commodity. That is, markets are dynamic and housing can be viewed as having many quality sub-markets and a finite supply of available land for development .[1] These factors, and the highly regulated and differentiated development contexts at the local level, make it difficult for developers to enter or exit particular housing markets, one of the assumptions of the RF study. The works of economists, Rothenberg et al. (1991) and Galster and Rothenberg (1991), are good examples of a more complex analysis of housing markets along these lines.[2]
A number of questionable assumptions occur throughout the report. We provide several examples of these assumptions and briefly discuss their shortcomings.
1. Every IH housing unit would have been a for-sale, market-rate home
In calculating the costs associated with IH, the authors assume that every unit created through an IH program would have been a for-sale, market-rate unit. IH, however, is applied to both rental and for-sale projects. Even in the case of for-sale projects, not all cities require that the affordable units be for sale. Similarly, granny-flats can be used to meet single-family requirements. The cost associated with making a for-sale unit affordable to low-income households that is the same as market-rate units is much higher than rental, granny flats or other types of units. Calculating the cost of IH on the basis of for-sale units only makes the authors' claim that "In half the Bay Area jurisdictions the cost associated with selling each inclusionary unit exceeds $346,000. In one fourth of the jurisdictions the cost is greater than $500,000 per unit, and the cost of inclusionary zoning in the average jurisdiction is $45 million, bringing the total cost for all inclusionary units in Bay Artea to date to $2.2 billion," highly bombastic but not based on the actual functioning of IH.
This assumption is particularly troublesome in the context of Part 5 of the report where the authors calculate the fiscal costs of IH. Specifically, it results in compounding any error in the assumption. For example, overestimating the number of counterfactual market units (the one IH unit to one single-family, market-rate unit assumption) overestimates the tax and other fiscal consequences of IH.
2.
Costs associated with below-market units
In calculating the additional cost of IH, the authors assume that the developer is mandated in all cases to build the IH units, that the units will cost the same as the market units to build, that there are no incentives and subsidies available to the developer and that developers are not capable of becoming innovative in finding solutions that reduce their costs. These assumptions do not reflect reality.
• In lieu fees. According to the report
• Unit modifications. IH units can be smaller, with fewer bathrooms, and witha reduction in the quality of finishes. Also, the design of the subdivision can be changed (for example, the width of streets can be reduced) to reduce the cost of development.
• Incentives and subsidies. As the report,
• Innovative approaches. Innovative developers have learned to work with non-profit housing developers and take advantage of subsidy programs, such as state and federal tax credits, which allow them to provide affordable units at a much reduced cost.
3. Developers will not build housing in
communities with IH policy
The authors imply that IH chases off housing developers. However, their report also shows that housing units have been built in communities with IH. Clearly, some developers have produced under IH and, as discussed above, have effectively competed in IH environments through innovative physical design (unit modifications) and cooperation (public, nonprofit, and private sectors).
In short, the authors view IH abstractly and rigidly, and not as the outcome of political and economic processes that attempt to balance the community's interest with the developers'. IH has evolved as a multifaceted approach aimed in most cases at reducing political opposition by neutralizing or reducing the additional costs that developers incur in providing affordable housing. Any analysis of the costs associated with IH needs to take into account the evolving nature of a system marked by flexibility, adaptability and change.
As discussed above, the authors present a constrained analysis with little complexity or breadth. However, sometimes, for reasons of feasibility, researchers analyze only one aspect of a policy. Even in these cases, the research should possess two attributes. First, the report of the research should include a strong statement of what the research does and does not include. Second, the research should be designed and analyzed to provide the most valid results and the conclusions should follow from those results. In the RF report, the design and analysis have significant flaws, and therefore the results call into question the policy recommendations. These issues are discussed in the following sections.
Research Design
A major flaw in the research design is the authors’ decision
to focus only on cities that adopted IH.
The need for a comparative approach was emphasized in the recent report,
The authors also do not provide long-term data by each year of their study period. It is therefore impossible to ascertain whether the decline in housing production in localities with IH is: 1) due to IH per se; 2) part of an overall downward trend in housing production in the Bay Area (see further discussion of this point in the Appendix); or 3) due to the time of enactment of IH in relation to the economic cycle. This last point needs some elaboration. Quite often the enactment of IH programs are a response to the political pressures that build up from the steep increases in housing costs, usually associated with periods of economic expansion. Not surprisingly, it takes time for the momentum for IH to build up and for the IH program to be developed and approved, with most programs passed at the apex or the beginning of the waning of the economic boom. For example, fifteen of the IH programs in the Bay Area were passed between 1989 and 1992, just before or at the very beginning of the economic recession of the early and mid-1990s. Housing production fell sharply during those times, a decline that the report attributes to IH.
The proposed design is quasi-experimental and should involve regression analysis with control variables. In other words, the authors do not control for many potential threats to the validity of their results (e.g., the adoption of growth controls, available buildable land in the city, etc.).
Data and
Analysis
The report contains some useful descriptive information
gleaned from secondary sources. The
National Association of Builders’ index (Part 1 of report) is well-circulated
and underscores the problem of housing affordability in
The missing data for number of units produced per year in the 17 communities with incomplete information are “filled” with the average of the remaining 33 cities (result shown in Figure 2, p. 6). Given that the overall number of cases (cities) is relatively small, the assumption that the average for the other cities can substitute for the actual value of the missing cities is troublesome. However, the authors use this technique without even a rudimentary analysis to determine if the missing cases are systematically different from the cases with full data (e.g., Are population growth rates significantly different statistically? Did the number of building permits differ in a statistically significant way? Did the cost/value of housing differ? What is the amount of land available for residential development in the two sets of cities? Are there differences in regulations controlling growth between these cities? etc.). For this type of analysis, the authors could have used available data from Census 2000 and performed difference of means tests as an elementary and quick check of their assumption.
Shortcomings in data collection plague the analyses throughout the report and case counts are presented in confusing ways. Recall that Table 2-3 (pp. 4-5) lists 50 cities of which 17 have incomplete information. At times the authors use all 50 cases in their analyses using the missing values. On page 20, they write about “the 45 cases in the sample” (looking at permits issued one year before and after the adoption of IH) and then on page 21, they are analyzing “33 cities” (looking at permits for longer before and after periods). On page 21, a reader may think they are analyzing the 33 cities with complete information, but they are not, according to footnote #32: “Our data is [sic] from 1970 to 2002, so we have data for seven years prior and seven years following for 33 cities that created their ordinances between 1977 and 1995.” The 33 cities referred to in this latter analysis exclude six cities with complete information in Table 2-3 (pp. 4-5) and include six cities with missing data (filled with the mean of cities with information). The changing sample sizes and composition cast major doubts on their conclusions.
The shortcomings in analysis are compounded by the data problems, but, even with better data, the analyses fall short of producing valid results for policymakers. For example, the authors use building permit information to “show” that IH results in a reduced supply of housing overall.[3] Their analysis considers the time period before and after the IH policy went into effect for the “the 45 cities in the sample” that had IH policies. Their Figure 13 displays results using data for the average number of permits one year before and after the policy adoption and the authors comment that they found similar results in three-, five-, and seven-year averages, but of course, the sample size has 26% fewer cases in these longer periods and therefore, is not comparable to the one -ear averages.
The
authors assert with great confidence that, as a result of IH, 2,982 fewer units
were produced the year after the adoption of IH for 45 cities. Besides the data problems outlined above, the
one-year decline would be especially puzzling if, indeed, it was associated
with IH. Builders complain that
processing plans takes an inordinate amount of time, certainly more than one
year. In fact, it is likely that developers would push through projects before
the enactment of IH so as not to be subjected to its requirement, with final
approval probably happening the year after the enactment of IH. The point is
that projects approved one year after the enactment of IH would be in the
pipeline at the time of the passage of IH or before. It is extremely unlikely that the reduction
in one year is due to the passage of IH requirements.
The authors present mostly quantitative analyses, but occasionally include qualitative data (from interviews) to bolster their claims. But these data are anecdotal (e.g., the comments of a builder on page 28), and do not provide scientific evidence on which to base conclusions. Clearly, more rigorous, systematic qualitative and/or survey research is needed before drawing conclusions about the perspectives of developers.
Misleading Assertions
The report contains a number of assertions that are not supported by evidence and could be misleading to many readers. We discuss three examples below.
1. Builders will avoid bearing the cost of IH
The authors maintain: "All theory and evidence suggest that the cost of inclusionary zoning will not be borne by builders, but by new homebuyers and landowners," but cite only one reference (Burchell and Galley, 2003). Actually, economists have pointed out that the costs are born, in various degrees depending on the circumstances, by homebuyers and renters, the developer, and landowners. If the demand for housing is elastic (i.e., sensitive in price), developers cannot pass down the cost increases to homebuyers or renters, and so have to reduce their profits. Furthermore, many observers have pointed out that in the long run it is probable that the landowner will bear most of the costs of IH (Mallach 1984, Hagman 1982, Ellickson 1985). A recent report critical of IH states that with IH, "The value of existing properties as potential sites for new housing development will be reduced” (Kosmont Partners 2004, p. 4). These arguments, however, have remained largely theoretical. For now, it would be safe to concur with Watkins (1999) who surmises that the cost of development fees - and IH is a development fee - "will always be split between all players in the development process" (Nelson and Moody 2003, p. 6). The costs of IH, then, are not automatically passed down to homebuyers and renters only.
The argument that builders will not pay is based on the assumption that "builders can simply move their construction to more profitable locations." As mentioned earlier, however, housing markets are segmented and are becoming more difficult to enter due to a decreasing supply of land and more complex local regulatory environments.
2. The findings are
generalizable.
The authors state that the cities used in their research “provide a good sample to tell us how inclusionary zoning is probably working nationwide” (p. 2). Unfortunately, because of the narrow scope and assumptions, the flawed research design, the data limitations, and the weaknesses of the analyses in the report, the analysis of the Bay Area cities cannot tell us how IH is working in the Bay Area, let alone nationwide.
2. IH "cannot meet the area's affordable
housing needs."
The authors point out that IH produces few units in general,
especially compared to the need for affordable housing. That is true. It is
misleading, however, to belittle one type of affordable housing program for not
meeting the entire need. IH was never
intended to be the only solution to the affordable housing crisis in
CONCLUSION
The RF report presents empirical results concerning the consequences of IH in the San Francisco Bay Area. The authors of the Reason Foundation report apparently intended to provide policymakers with information to evaluate the impacts of IH on communities. Our purpose in writing this critique was to assess the quality of the research and note its major weaknesses, consider the reasonableness of claims based on the research results, and recommend a more fruitful path for empirical work on IH policies. Based on our assessment, the research has many shortcomings and is unsuitable as evidence for the authors' claim that IH leads to a decline in the overall production of housing and is a costly policy for communities. We have made a number of suggestions for better empirical research on IH, including a more sophisticated research design, improved data collection (quantitative and qualitative), and more rigorous analyses.
REFERENCES
Burchell, R and C.
Galley. 2003. "Inclusionary Zoning: Pros and Cons," in The California Inclusionary Housing
Reader. Sacramento
California Northern California . 2003.
Inclusionary Housing in
California
Galster, G. and J. Rothenberg. 1991.Filtering in Urban Housing. Journal of Planning Education and Research 11:37-50.
Hagman, D. 1982. Taking Care of
One's Own Through
Kosmont Partners. 2004. Review
of the David Rosen & Associates Report on
Mallach,, A. 1984.
Nelson, A. and M. Moody. 2003. "Paying for Prosperity:
Impact Fees and Job Growth." Cities
and Suburbs Report.
Rothenberg, J., Galster, G.C., and J.R. Pitkin. 1991. The Maze of Urban
Housing Markets: Theory, Evidence, and Policy.
Ellickson, R. 1985.
Watkins, W.A. 1999. "Impact of Land Development Charges." Land Economics 75(3): 415-424.
APPENDIX
Figure 1. Research Designs
(a) Reason Foundation Research Design:
Time
No random
Assignment Oit1 Xi Oit2
Where: Oit1 = the number of residential permits issued in time 1 (t2 is time 2) for
city i (cities with IH policy)
Xi = the adoption year of IH policy in city i
(b) Proposed design:
Time
No random Oit1 Xi Oit2
Assignment
Ojt1 Ojt2
Where: Oit1 = the same as above
Ojt1 = the number of residential permits issued in time 1 (t2 is time 2) for
city j (cities without an IH policy) i.e., a comparison group
Xi = the same as above
Residential
Building
The graph in Figure 2 shows the numbers of permits over the
years in a hypothetical city. While
permits vary from year to year, an overall downward trend is present, even
before IH policy was adopted by the city.
Therefore, it is no surprise that these data would show that permits
went down, on average, after IH went into effect (using the averages for 1-,
3-, 5-, and 7-year before and after periods).
This example is only one city and the authors use the average of a
number of cities (45 apparently in one analysis, and 33 in the others). If all the cities or the majority of the
cities had a downward trend in permits over the entire study period, then the
difference between the average permits before or after the IH policy was
adopted would be even more dramatic than the single case – but the difference,
averaged in this manner, would be very misleading. The report does not provide long-term data by
year to the reader, and therefore, we cannot know if the difference in permits
found by the authors is an effect of IH, the reflection of a long-term trend[4], or, as
stated in the body of this critique, the date of approval of IH in relation to
the economic cycle.
Figure 2. Hypothetical Permit Data for 1975-1995
(using 1986 as the year IH was
adopteda)
a The data in this graph are hypothetical and are intended
only to illustrate a problem
with the analyses in the RF report.