This is the second piece in Hot Rocks: Commercializing Next-Generation Geothermal Energy, a joint series by Employ America and the Institute for Progress, examining the potential to commercialize next-generation geothermal energy, the lessons we might learn from the shale revolution, and the federal policy changes needed to make it happen. The introduction to the series is available here, and follow-up pieces will be published over the coming weeks.
Introduction
In the last piece in our geothermal series, Brian Potter explored the accumulated technological advances in drilling and fracking that led to the shale revolution – advances that are now key to next generation geothermal energy.
These advances weren’t “Eureka!” innovations. Instead, they were incremental, the result of considerable time and investment. Private-sector investment dramatically accelerated commercialization of shale drilling. Moreover, that investment led to a massive production and productivity boom that we’re still experiencing. The game-changing fracking techniques developed in the Barnett were the result of Mitchell spending “17 years and over $250 million dollars drilling wells… gathering data and experimenting with different fracking methods, before they developed a method that worked and began to turn a profit.” The profit and productivity gains came well after the substantial investment, but they wouldn’t have occurred without it.
As we look to dramatically expand and commercialize new energy technologies like geothermal, it’s worth exploring what drove a long period of sustained investment in the energy sector’s recent past. Many factors contributed to the investment boom, but specific policy decisions facilitated investment, accelerating the shale revolution. The same policy tools that enabled the shale revolution can supercharge new clean energy technologies like next-generation geothermal.
The case for policy to support more investment
Investment supports technological advancement
Why should policymakers even care about investment in the first place? Beyond the traditional debates about the role of government in markets and the merits of public versus private investment, there are more academic sources for skepticism. For example, investments in sectors like crude oil extraction (and mining more generally) can be subject to diminishing returns if viewed in isolation.
“Crowding in” more investment helps industries learn from how investment is deployed. New learnings and technical knowledge can emerge; novel production techniques and more production become feasible. This class of mechanisms includes “learning-by-doing,” knowledge spillovers, and other forms of technological diffusion. All rely on an economic actor taking the first step of irreversibly sinking their liquidity for the sake of creating a less liquid real fixed asset.
Disrupting any industry can require considerable capital investment. Energy and mining are particularly capital-intensive sectors. Relative to the operating labor expense involved, the requisite capital expenditure just for physical structures and equipment are substantial. While capital intensity is relatively high for utilities and many segments of manufacturing, and higher still for mining sectors writ-large, it is especially high for crude oil produced through fracking techniques. Maintaining existing levels of production requires persistent investment. By contrast, conventional oil extraction techniques are better understood, are less vulnerable to cyclical market dynamics, and require less frequent investment to maintain the same level of output over time. In critical contexts like developing new clean energy sources, investment is an essential ingredient both to production and learning.
Active policy can crowd-in more investment
Even if we presume investment to be worthwhile for the sake of advancing technological outcomes, it may not be immediately obvious why policy should play an active role. The conventional argument centers around the positive externalities of investment and technological advancement. The more that society can learn about how to produce energy abundantly and affordably, the higher the standard of living can be for everyone. However, in the absence of active policy, the private producers of these benefits might not be sufficiently incentivized to deliver optimal social value.
A related but less stated reason for active policy is the necessity to help firms overcome high and sticky investment hurdle rates. The hurdle rate is a firm’s minimum acceptable rate of return for a project. Decarbonization will require many individual firms making conscious decisions to invest despite high and sticky hurdle rates.
In theory, calculating the hurdle rate for a given project is relatively straightforward. But in practice, perceived risks tend to push the hurdle rate higher than what theory would suggest (See Note 1).
Even as financial markets might reflect a lower general cost of capital, managers of firms are often reluctant to respond with proportionally higher levels of investment, even though more projects should be worth pursuing as a result. Hurdle rates in the mining sector are especially high relative to other sectors.
A tractable explanation for high and sticky hurdle rates is that deeper uncertainties paralyze risk-taking. These types of uncertainties are difficult to anticipate ex-ante with a specific probability distribution, and typically involve dire worst-case scenarios. Nascent technology always comes with its own set of issues. Litigation risks can also loom large. And where costs and final products are highly variable and vulnerable to big cyclical swings, all kinds of possibilities can emerge that are difficult to fully plan for. The energy sector is filled with all of these kinds of problems.
For private firms with finite balance sheets, these uncertainties can be life or death issues. Using policy effectively to address these uncertainties — or at least cushion the potential blow — helps ensure that requisite investment is crowded in, and that technological advancement is more likely to materialize.
Specific policies enabled investment
Going back to the 1980s, a specific set of policies facilitated investment in shale oil and gas production. The scale of investment was staggering: per a 2020 Deloitte report, in just the two shale boom and bust cycles (between 2010 and 2020), the industry had negative $300B in free cash flow and impaired $450B of capital. This was funded in large part by banks and private equity, with reports indicating a handful of major investment banks provided over half of the financing for shale firms. And this doesn’t even account for the considerable private investment in the 1980s and 1990, when technological risk was particularly high.
While this activity may look like the result of purely private decision-making, private capital was in fact “unlocked” by proactive policy choices. Over the decades that companies experimented and iterated to build the shale revolution, various federal institutions proactively made it easier to overcome high hurdle rates. Those institutions conducted research and development, but they also provided funding for private companies to undertake those activities when risk was high, commercial incentives that supported investment after technological breakthroughs, and accommodative macroeconomic policy that facilitated the capital investment necessary to achieve widespread commercialization.
These policy choices came from different federal entities with different missions (sometimes, wholly unrelated to energy policy). But they worked together over decades, at times in concert and at times in succession, to help companies overcome the hurdle rate for risky investments in shale production.
Flexible financing incentivized private participation
Direct federal research and development funding, especially by the Eastern Gas Shales Project (EGSP), led to several innovations and industry “firsts” that pioneers like George Mitchell and Harold Hamm built on. As the Breakthrough Institute wrote in 2012, “In the 1980s Mitchell relied on DOE mapping techniques and research to understand the complex geology of tight shale formations.” Furthermore, “technologies like diamond-studded drill bits and microseismic imaging were developed by federal agencies for non-shale applications, demonstrating the clear and present value of publicly-funded basic research.”
Beyond its direct role in innovation, the federal government facilitated private investment so that companies could experiment and perfect techniques. At the early stage, when technological risk is highest, flexible financing arrangements are particularly useful to incentivize private participation (as another example, consider the importance of Other Transaction Authority in the development of SpaceX’s Falcon 9 rocket).
The Eastern Gas Shales project that began in 1976 included funding agreements that offloaded the risk of investment for producers. Mitchell Energy participated in cost-sharing agreements with DOE to drill and core several wells in the Marcellus Shale. Bespoke participation agreements with individual companies are not typical for government funding projects, given concerns about corruption and “picking winners.” But the contracting flexibility to enter into cost-sharing agreements was crucial for the early uptake of private-public participation agreements. As described in the report of one cost-sharing agreement with Mitchell, an explicit objective of the program was to “stimulate interest among commercial gas suppliers in the concept of producing large quantities of gas from low-yield shallow Devonian Shale wells.” And DOE was successful: Mitchell built 35 shale gas wells with government funding.
Supply- and demand-side support made costly production economical
Once development became viable, the federal government offered strong economic incentives for scale that supported production and spurred a profitable private market. Two tax subsidies significantly accelerated investment in the shale sector: the Unconventional Gas Credit and the Intangible Drilling Deduction.
The unconventional gas credit, a production tax credit for gas produced from shale (and other unconventional sources), was created in 1980, and offered an economic incentive for riskier, high-cost shale drilling. The production credit was passed through the Crude Oil Windfall Profit Tax Act of 1980, and applied to gas produced from unconventional gas wells that were drilled from 1980-1992. It remained available to natural gas extracted from those wells through 2002. The credit applied to shale, tar sands, gas produced from coal seams, tight sands, geopressurized brine, Devonian shales, synthetic fuels from coal, and steam from agricultural products. In a report on the legislation, the Senate Finance Committee wrote:
“[T]hese alternative energy sources involved new technologies, and some subsidy is needed to encourage these industries to develop to the stage where they can be competitive with conventional fuels. The information gained from the initial efforts at producing these energy sources will be of benefit to the entire economy.”
The committee was correct to include the production tax credit. The credit drove the shale industry forward and underwrote shale investment. The tax credit more than doubled the production of unconventional gas in a two-decade period, spurred innovation in drilling technologies, and savedproducers about $10 billion. As Jason Burwen and Jane Flegal wrote in 2013:
“The… creation of production-based credits opened up a new domain of financing available for gas well operations.
Since many small operators did not have substantial enough liabilities to take advantage of tax credits, they effectively “sold” their credits to larger firms in tax equity financing transactions. While the level of tax equity investment was modest, nonetheless Section 29 credits generated more investor interest and leveraged more private dollars in unconventional gas than existed previously. Perhaps more importantly, the credit stimulated industry to drill more wells and collect more data, contributing to applied knowledge of well operators. This learning-by-doing drove incremental improvements in technology, finding rates, and well productivities, thereby keeping unconventional gas resources economic even following the expiration of the Section 29 tax credits in 1992.”
Also importantly, the tax credit was designed to respond to price volatility in the commodities sector:
“The formula accounted for inflation and contained a factor that would gradually phase out the effect of the tax credit when the price of oil was high, reflecting the consideration that the credits were to take effect when oil prices were so low as to limit the competitiveness of unconventional fuels.”
By essentially establishing a price floor for riskier investment, the credit helped firms overcome the high hurdle rate associated with unconventional gas drilling.
Another important tax subsidy was the intangible drilling costs deduction (“IDC”) which has incentivized riskier plays since its inception in 1918through the IRS rulemaking process. The IDC allows producers to deduct the charges for non-salvageable expenses (like wages, fuel and repairs) that are necessary to prepare and drill wells. Perhaps the most important feature of the IDC deduction is that expenses are deductible in the year in which they are incurred, rather than being amortized over a longer period. This frees up cash flow (smaller firms can deduct 100% of their IDCs immediately, particularly useful as they are often the producers responsible for additional barrels at the margin). As the Stockholm Environment Institute wrote in a recent report, “this immediate or accelerated deduction (rather than over the life of a well) again allows firms to lower their taxable income, freeing up cash flow and increasing profitability.”
While the IDC deduction is available to all drillers, it is disproportionately beneficial for less conventional forms of drilling. As Wood Mackenzie noted in a 2013 report, while IDCs typically accounted for about 70% of a conventional well’s costs, that number was at that time, 80% for offshore drilling, and 85% for shale drilling (and can be reasonably assumed to be even higher before the production efficiencies of the 2000s). In short, the tax subsidies in place made it easier to overcome the considerable hurdle rate for high-cost production.
In addition to subsidies on production, there was demand-side policy in place as well. In 1954, the Phillips decision dramatically expanded the system of price controls (administered by the Federal Power Commission) for natural gas sold interstate. Over time, these controls limited the supply that producers were willing to bring online. In the wake of the 1970s energy crises, Congress passed the Natural Gas Policy Act in 1978, pulling back the price controls and instituting higher price ceilings intended to incentivize producers to produce more natural gas.
Most important for shale development was a carveout in Section 107 of the Act, which gave FERC (created by the Act to replace the FPC and regulate a unified intrastate and interstate gas market) the authority to essentially exempt certain high-cost production (including Devonian aged shale) from the federal price regulations, if “such special price is necessary to provide reasonable incentives for the production of such high-cost natural gas.” These price incentives created a “huge advantage” for the risky, high-cost production of shale companies. Mitchell Energy was also incentivized to keep investing because of a higher-than-market price contract with Natural Gas Pipelines of America (NGPA).
The deregulation of prices for high-cost production that produced a demand-side incentive, coupled with the tax subsidies that helped subsidize production, worked in conjunction to boost shale investment. Dan Steward, a former Mitchell Energy executive, summarized the dynamic to the Breakthrough Institute in 2011:
“The tax credits helped, as did the different pricing scenarios for newly discovered gas. We had a gas contract with a natural gas pipeline that gave us a higher price. We had a basket of prices and gasses and with the different categories we could keep our gas price. So you could say that those pricing scenarios, and the tight gas tax credit, created the possibility for shale gas.”
A limited categorical exclusion sped up oil and gas exploration
From the 1970s through the 2000s, fiscal policy incentivized riskier, high-cost production like shale. But from 2005 onwards, another important policy driver was regulatory reform, which reduced the costs of exploration for oil and gas, and helped push the US to becoming the world’s number one producer.
In 2005, a bipartisan majority passed the Energy Policy Act, which aimed to boost production and build US energy security. Among many provisions, the Act categorically excluded certain limited-footprint oil and gas exploration (including some well drilling) from the more rigorous, lengthy review (herein referred to as the “390 CE”) required under the National Environmental Policy Act (NEPA).
Categorical exclusions generally take considerably less time to permit than the environmental assessments or environmental impact statements required under NEPA. The National Renewable Energy Laboratory found that the average exploration EA takes 337 days, compared to only 88 days for the average categorical exclusion.
The 390 CE had an impact immediately following its passage. From fiscal years 2006 through 2008, the 390 CE accounted for over a quarter of all wells approved by the Bureau of Land Management (BLM). A 2011 GAO report surveying both BLM officials and industry participants found that processing times for Applications for Permits to Drill (APDs) under the 390 CE decreased dramatically:
“The vast majority of BLM officials we spoke with told us that using section 390 categorical exclusions expedited the application review and approval process… Industry officials with whom we spoke also agreed that BLM’s use of section 390 categorical exclusions had generally decreased APD processing times.”
By reducing the regulatory costs of exploration and lighter-touch drilling, the 390 CE certainly played a role in unlocking oil production, but it’s not clear that it dramatically shaped the shale revolution.
For one, onshore oil and gas production from federal lands is typically a small percentage of all US production (See Note 2). Many shale producers were able to avoid permitting on federal lands entirely. After the innovations of horizontal drilling and fracking in the Barnett by Mitchell and Devon Energy, producers began to explore other shale resources, including in the Marcellus and Fayetteville Shales. While the 390 CE may have been used by some shale producers, a 2009 GAO analysis indicates that the BLM field offices relying most on the 390 CE were not ones that would have approved many of the shale plays between 2006 and 2008.
However, absent the considerable subsidies aforementioned, it’s likely that shale exploration would have been considerably less worthwhile for producers to undertake.
In our current context, the importance of regulatory reform for geothermal energy is considerably more pronounced. Shallow geothermal resources are concentrated on federal lands in the American West:
This means the permitting process at BLM will have far more effect on the nature of geothermal energy deployment. Furthermore, in a higher-interest rate environment, delays are even more costly. The increased cost of delays could hinder production activities of all kinds.
Interest rates shaped the investment boom
While fracking technology continued to mature through the 2000s, financial factors led investment to truly explode from 2009 to 2012. The most obvious and straightforward financial factor — the price of oil — was clearly helpful over this period. Oil prices were persistently above $100 a barrel, thanks to tight OPEC policy and relatively strong growth in emerging market economies. Another financial factor goes less frequently stated: the low level of interest rates and financing costs.
The Federal Reserve had chosen to keep interest rates low throughout this period. Due to the anemic employment and output recovery following the Global Financial Crisis and the Great Recession, the Federal Reserve engaged in both conventional and unconventional efforts to support broader risk-taking across financial markets. While the direct effects of lower interest rates are well-understood, the Fed’s policies also helped to lower financing costs through channels that directly reduced risk premiums, thereby making it much more affordable for firms to scale up debt issuance. Plausible macroeconomic models even suggest that the Fed’s policies in this period were disinflationary, as the effects on crude oil supply growth may have outstripped their effects on demand.
The net effect of high prices and low interest rates was a historic boom in fixed investment for crude oil production. The Baker Hughes crude oil rig count octupled, from 179 in June 2009 to 1,432 in October 2012 (See Note 3). For historical perspective, crude oil rig counts in the United States had generally been in decline over the previous decades, from about 600 in November 1987 to just over 100 in September 2002. As a share of total US fixed investment expenditures devoted to nonresidential structures, structures for crude oil and natural gas extraction purposes doubled.
This kind of growth and scale was primarily funded through debt, a fact exposed through waves of bankruptcies that followed multiple crashes in the price of oil. At a time when industry has the capacity to grow quickly, debt capital can be especially attractive, and from 2009 to 2012, benchmark interest rates were generally low and corporate credit risk premiums in the energy sector were generally falling.
Abundant and affordable debt capital in the shale boom was not the product of a conscious policy choice. But the serendipitous occurrence likely sped up the timeline for onshore US shale production to reach maturity. And now, although investment since 2016 has generally been in decline, it has not come at the expense of production or productivity. And as we argue throughout this piece, the growth of investment over the 2009-2012 period helped to unlock productivity gains in the present and future. Having reached critical mass to allow for experiential learnings to take place, industry is proving to be better positioned to translate those learnings into both greater financial efficiency and greater resource efficiency (See Note 4).
Should the serendipity of low financing costs not be available to other leading-edge industries primed for growth, active fiscal policy efforts may be worth considering to accelerate investment and technological advancement.
Investment supported an ecosystem of entrepreneurs and suppliers, not just additional wells
The considerable debt (and equity) that shale producers issued during the growth phase is staggering. Of course, much of these proceeds were used to invest in additional wells, particularly between 2008 and 2015. In 2008, there were fewer than 500 oil wells in operation in the United States, and by 2015, that number had increased to more than 1,500.
This capital investment also funded a growing ecosystem of parts suppliers and machinery manufacturers specifically designed to improve the output of shale wells as the industry matured. As one industry source wrote in 2014:
“As the US unconventional market continues to grow, with operators ramping up activity in newly discovered plays, service companies have followed suit with ongoing research and development efforts. In 2013, FTSI opened a corporate technology center in Houston to focus on developing more effective fracture designs, new fluids, proppant transport systems and customized solutions for specific reservoir requirements in the US unconventional market. ‘We now understand that a lot of hydrocarbons are being left behind, so we’re developing tools and methodologies to improve recovery,’ Mr Holms, Technical Advisor for the company, said.”
This buildout gave entrepreneurs new opportunities to adapt to a wider range of terrains. Advances in horizontal and multi-stage drilling were commercialized not only because they were successful technologies, but also because the scale of investment and spending built a corresponding supplier ecosystem for the necessary tools.
Without sustained demand for new tools, suppliers do not produce them, and the material basis for the technological improvement is lost. Firms need some way to pay these suppliers, who play a crucial role in driving down the cost of new techniques by continually refining actual production and manufacturing processes. If revenue isn’t there yet, that means capital, in the form of debt provided by funders who are less worried about immediate profitability than capturing long-term efficiency gains.
The productivity boom that followed the investment boom
While production and productivity dramatically increased after fracking became more widespread and commercialized, (the first-year output of the average fracked well tripled between 2007 and 2016), the greatest productivity gains came after the largest increases in investment. In the period between 2014 and 2018, production increased even as rig counts (a good proxy for investment) and employment in the sector went down, per the Kansas City Fed:
More recently, the same researchers, Jason Brown and David Rodziewicz, found the same conclusion up to 2022:
“Typically, increases in production are thought to come from a higher number of active drilling rigs. However, for much of the past decade, the number of rigs (orange line) has remained below its 2000 level. Instead, the increase in production (green line) has been driven largely by improved drilling productivity (blue line). The number of barrels of oil produced per foot of drilling has more than doubled since 2014.”
During the period that producers iterated in new geographies, increased investment did not necessarily correlate with productivity. In the chart above, production increased in the periods from 2004 to 2008, and 2010 to 2014, but it was the result of higher rates of investment and employment growth.
By contrast, in the production boom from 2014 to 2018, production increased despite rig counts and employment levels that were well below previous peaks. The paper attributed a number of factors, but key among them was the technology enhancements that were perfected over the previous period:
“While this recent surge in production per rig and per worker has been driven in part by a focus on more core areas of plays in recent years, which naturally are more productive, there have been a number of key drilling and technology enhancements. These include longer horizontal laterals, multiwell pad drilling, walking rig systems and increased proppant concentrations in hydraulic fracturing. Firms also are making greater use of data analytics to increase drilling accuracy and create process efficiencies.”
Put somewhat crudely, repetitive deployment of the same capital goods can reveal more opportunities for continuous improvement. New wells are often drilled with a related workforce and network of suppliers, with operations improved by the lessons of past difficulties.
But this is still not the end of the story. For new industries, profitability and major efficiency gains come after the technological gains are secured, not before. Despite all this doom and gloom, fracking may now be set up to turn a profit thanks to a growing international market for natural gas — which can be cheaply produced through fracking.
In fact, this connection between spending on capital investment, debt, and innovation is so strong in fracking that the development of the industry in North Dakota has explicitly been used as a case study for the importance of access to cheap capital for innovation.
Conclusion
A common adage is “success has a thousand fathers, but failure is an orphan.” From the few wildcat producers who, often amidst scorn and ridicule, continued to invest in challenging technologies, to the scientists and engineers designing cutting-edge new devices, thousands of decisions over decades helped realize the transformation that was the shale revolution. However, at every turn, key policy choices facilitated and accelerated shale development, culminating in a production and productivity boom we are still experiencing today.
Those policy choices spanned a wide range of interventions: direct investments in research and development, cost-sharing agreements and other incentives for private participation and investment, targeted production tax subsidies, demand incentives, regulatory reform that made exploration easier, and an accommodative monetary environment that made it much easier for firms to scale up debt issuance. All played an important role in making the US the world’s number one producer for oil and gas.
Now, as we turn to dramatically scaling new sources of clean firm energy, the lessons of the shale revolution are critical to policy design. Novel and capital-intensive energy sources, like next-generation geothermal energy, will have high hurdle rates for investment to overcome. With a markedly different macroeconomic environment, decision makers will need to design policy in a manner that allows technologies like next-generation geothermal production to achieve the same trajectory as the shale revolution.
Notes
Note 1: Traditional corporate finance theory suggests that hurdle rates should reflect the firm’s weighted average cost of capital and a risk premium that compensates for the systematic risk of the given investment project. In practice, hurdle rates are much higher than conventional theory predicts, and appear to reflect compensation for the perceived idiosyncratic risks of a given project.
Note 2: In the period between 2008 and 2017, federal onshore oil production as a percentage of total US production declined from a high near 10%. The same is true for federal onshore natural gas production, which peaked at around 15%. This decline has continued through 2021.
Note 3: Oil rigs are structures used to extract petroleum and develop productive wells.
Note 4: As has become more apparent since 2020, the industry has had to shift away from balance sheet structures built for growth and towards ones that support profitability. In simple terms, that means less reliance on debt to fuel growth and more reliance on equity-financing through retained earnings.
