Hot Rocks, Part I: The Technological Innovations that Produced the Shale Revolution

A Joint Series from Employ America and Institute for Progress

This piece is part of Hot Rocks: Commercializing Next-Generation Geothermal Energy, a joint series by Employ America and the Institute for Progress, examining the potential to commercialize next-generation geothermal energy, the lessons we might learn from the shale revolution, and the federal policy changes needed to make it happen. The introduction to the series is available here, and links for all available pieces are here.

In the process of moving to clean, firm sources of energy, one promising technology is geothermal. Historically, geothermal power could only be produced in places where water naturally moves through heated rock near the earth’s surface, like hot springs. This meant that geothermal power plants could only be built in a small number of places, and as of 2022, only about 0.4% of the US’s electricity came from geothermal power. 

But heat from the earth is available anywhere, if you can drill deep enough and find a way to extract it. Many of the geothermal energy technologies being developed today could be built almost anywhere. The amount of heat energy in the earth’s crust is so enormously vast (41x more than that of all known petroleum and nuclear fuel reserves) that large-scale construction of geothermal plants could provide an abundant source of zero-carbon energy, without the intermittency problems of solar and wind.

New approaches make extensive use of drilling technology originally developed for the oil and gas industry. As easily available sources of petroleum and natural gas have been exhausted, the industry looked to develop drilling methods to economically tap less accessible deposits.

Hydraulic fracturing, or fracking, was developed in the 1990s to extract natural gas from previously inaccessible shale deposits (some sources, particularly older industry sources, use the spelling “fraccing” or “fracing"). Thanks to fracking, shale gas went from 2% of US natural gas production in 1998 to nearly 80% of American natural gas production by 2022 (for 1998 values, see this USGS sheet and Golden and Wiseman (2015)). Today, next-generation geothermal companies are using advanced drilling and fracking technologies from the oil and gas industry. Perhaps ironically, technology developed for the oil and gas industry could be crucial in our effort to transition to zero-carbon sources of energy.

Birth of Shale Gas Fracking

Until the 1990s, most natural gas in the US was extracted by drilling a well into a natural gas reservoir, a large crack or space between layers of rock where natural gas had accumulated (gas was also often found in oil deposits, which is known as “associated conventional gas”). In this type of natural gas deposit (known as “conventional gas”), the gas will readily flow into the well.

But natural gas can also be found within the pores of some rock formations, such as shale layers. These so-called “unconventional” gas deposits (also known as “continuous” or “tight” gas), had long been known about, (Shale gas had been tapped as early as the 1820s), but the rock’s low permeability limited gas extraction. This changed in the 1990s, when Mitchell Energy developed novel fracking techniques to extract large amounts of gas from the Barnett Shale in Texas. By injecting high-pressure fluid into its gas wells, the rock below the surface would fracture, increasing its permeability and thus the flow of natural gas. These cracks would be kept open by infusing the fracking liquid with “proppants,” small sand-like particles that were carried into the cracks and prevented them from closing.

Fracking itself was not a new technology in the 90s. Its use in oil and gas dates back to 1947 and the Stanolind Oil and Gas Company. At the time, Stanolind injected acid into its wells to widen the pores in the rock and increase well production (a technique first developed in 1895, and still used today). A Stanolind engineer, Floyd Farris, noticed that the higher the pressure the acid was injected at, the more the well produced. Farris speculated that the high-pressure injections might be fracturing the rock, and that it might be possible to increase well production simply by injecting a fluid under high pressure. Stanolind’s first attempt at hydraulic fracturing was on a natural gas well in Kansas, and used a fluid composed of gasoline and napalm mixed with sand. Over the next two years, the company fracked 23 wells, increasing gas production in 11 of them. When Haliburton licensed Stanolind’s fracking technique in 1949, it found that fracking increased well production by 75% on average, and went on to frack thousands of its wells. By 1955, over 100,000 oil and gas wells around the world had been fracked, and fracking had increased the supply of oil in the US “far beyond anything anticipated."

Fracking technology advanced over the next decades. Early fracking used comparatively small amounts of fluid and proppant, on the order of 400 pounds of sand and 750 gallons of fluid per well, but by the 1970s so-called Massive Hydraulic Fracturing was injecting on the order of 500,000 gallons of fluid and 1 million pounds of proppant into natural gas wells. Fracking fluids evolved, from gasoline and napalm, to water, to cross-linked guar gels. Different kinds of proppants, including “plastic pellets, steel shot, Indian glass beads, aluminum pellets, high-strength glass beads, rounded nut shells, resin-coated sands, sintered bauxite, and fused zirconium” were all experimented with.

By the 1990s, fracking was a mature technology, used by Mitchell Energy for many years (in fact, it had set a record for the largest frack ever on a tight gas well back in 1978). Mitchell had drilled wells into the Barnett Shale since 1981, after an employee published a paper suggesting that the Barnett might contain large amounts of oil and gas. By 1995, Mitchell had drilled more than 250 wells in the Barnett, experimented with different fracking fluids and chemicals, and had performed massive fracks to stimulate well production with up to 3 million pounds of proppant and 1 million gallons of fluid. But production rates on Barnett wells remained unimpressive.

Mitchell’s breakthrough came in 1997. At the time, Mitchell, like most of the industry, was using viscous cross-linked gels. After one gel frack, Mitchell engineer Nick Steinsberger noticed that the gel had not properly set, making the fluid similar in consistency to water. But even without the aid of gel, the well was producing a surprising amount of gas.

Water had previously been used as a fracking fluid in the 50s and 60s, but had mostly been replaced by gels designed to be thick enough to carry proppants deep into fractured rock. Later, chatting with a friend at an industry event, Steinsberger learned that Union Pacific Resources used a fracking mix in sandstone that was mostly just water and sand, with a few added chemicals as friction reducers. These were known as “slick-water fracks."

At the time, water was considered an inappropriate fluid for fracking in shales. It was believed that the shales would absorb the water and swell, reducing rather than increasing permeability. Despite this, Steinsberger began to wonder if water might work as well as gel as a fracking fluid in shale. Because it used fewer chemicals and proppants, a slick-water frack would be much cheaper than a gel frack.

After learning more about Union Pacific’s methods, Steinsberger began to test slick-water fracks on wells in the Barnett Shale in early 1997. The fracks appeared to work roughly as well as gel fracks, but didn’t result in particularly impressive levels of gas production. But Steinsberger tweaked his methods, adding more horsepower to the pumps and adjusting when the sand was added. Well performance improved. The gas production rates from slick-water fracked wells proved much steadier, declining less over time than gel-fracked wells. By 1998, the company had switched to Steinsberger’s methods for all of its Barnett wells, reducing the costs of fracking the wells by 50-70% and increasing gas production.

Mitchell’s Barnett fracking received a further boost in 1999 when Kent Bowker, a Mitchell geologist who had studied the Barnett while working at Chevron, showed that the Barnett held up to 3x as much gas as previously thought. The Barnett consisted of two layers, the Lower and Upper Barnett. Until 1999, Mitchell’s wells had been drilled into the Lower Barnett, but it began to drill wells into the Upper Barnett as well, often “refracking” already-drilled wells to tap the Upper layer. As Mitchell more aggressively drilled wells, gas production in the Barnett greatly increased. Between 1993 and 2002, the year Mitchell was sold to Devon Energy, gas production in the Barnett Shale increased by more than 20x, mostly due to Mitchell’s wells.

Horizontal Drilling

The next major evolution in fracking technology combined hydraulic fracturing with horizontal drilling. Many oil and gas deposits are in layers of rock that are only a few tens of feet thick, but that extend horizontally for miles. By drilling a horizontal well into these deposits, the well intersects a much larger portion of the reservoir, increasing how much oil or gas it produces.

Drilling non-vertical wells dates back to the 1930s, when unscrupulous well operators in the US would drill angled wells to steal oil from their neighbors. But most early directional drilling was done in the Soviet Union. The Soviets heavily developed turbodrilling technology, which used a small motor at the end of the drill that was driven by the flow of drilling mud, rather than rotating the entire drilling assembly (called the drillstring) from a motor on the surface. The turbo drill made it much easier to steer the drillstring and drill horizontally, and over the next several decades continued improvements (such as the invention of the positive displacement motor in the 1950s) greatly advanced the art of horizontal drilling. In 1968, the Soviets set a record by drilling a 2,500 meter-deep well with a horizontal portion of more than 600 meters. 

Most of these efforts at horizontal drilling weren’t particularly successful. From aboveground, it was difficult to know where exactly the drill was, making it hard to successfully intersect a thin layer of petroleum-bearing rock. It was also difficult to precisely control the movement of the drill. But this began to change in the 1970s, when drill sensors that could provide information while the drill was operating, known as Measurement While Drilling, began to appear. Downhole motors got better, as did the bendable portion of the drill housing (known as the bent sub), which made it possible to steer the drill more precisely. By the end of the 1980s, horizontal drilling had become “economically viable,” with costs only slightly exceeding or on-par with vertical drilling.

Mitchell Energy had made several instructive attempts to drill horizontal wells in the Barnett, but production hadn’t been impressive. This changed after its acquisition by Devon Energy. Devon’s first five horizontal wells drilled in 2002 “outperformed anything previously seen in the Barnett,” with production rates over 3x Mitchell’s average rate. Horizontal wells both exposed more reservoir area to the well and often allowed much larger frack treatments than vertical wells. Devon’s success with horizontal drilling was in part due to its extensive use of 3D seismic data, a technology which first began to appear in the 1970s, and which enabled Devon to precisely locate the layers to be drilled into. Devon also made use of microseismic fracture mapping — using sensors to record minute seismic events when the rock fractures to map fracture extents and better plan gas field development.

Beyond the Barnett

From the Barnett, fracking moved on to other large American shale gas resources. In 2004, operators began to drill extensively in the Fayetteville Shale in Arkansas after data suggested it had similar geological properties as the Barnett. That same year the Marcellus Shale in the Northeast began to be extensively drilled.

Drilling other shale resources (known as “plays”) used the same combination of slick-water hydraulic fracturing and horizontal drilling that had been so successful in the Barnett. But different shales had different geological conditions, requiring adjustments to the basic fracking “recipe." The Marcellus Shale required significant trial and error with different fluids, proppants, pumping rates, and pressures. Mitchell’s methods eventually managed to extract oil from shale deposits as well as gas, but this also took significant trial and error. Oil molecules are much larger than natural gas molecules, and move much less readily through impermeable shales, requiring modified fracking methods. 

As Barnett proved fracking could work in shale, the dynamics of incremental improvement in industry kicked in, leading to transformative advances over the following years. When oil drillers used horizontal drilling and slick-water fracking on the Bakken Shale in North Dakota, they found that extracting oil required fracking the well in stages — isolating one portion of the well, fracking it, and repeating on another portion of the well. Similar multistage fracking on vertical wells proved successful when extracting oil from Permian Basin shales in 2005. Multistage fracking would also be successfully used in shale gas wells. 

As region-specific frack methods were being developed, drilling technology continued to make the process more efficient. Polycrystalline Diamond Cutter (PDC) drill bits, made from synthetic diamond, began to replace the previous roller-cone bits, going from 15% of oil and gas drilling footage in 1995 to 50% in 2005 (Mau 2015 p199). With PDC bits, wells could be drilled much faster. Pad drilling, where multiple horizontal wells are drilled from a single pad, avoiding the time consuming setup and teardown process for the drilling rig, also began to appear, and by 2014, 70% of horizontal wells used pad drilling. To further reduce setup and teardown times, walking drilling rigs were developed, allowing a rig to be moved to another well site without disassembling it. Pumps and downhole motors improved, and new drilling automation and control systems reduced wear on the bits, extending lifespan and reducing maintenance downtime.

The scale of fracking operations also continued to increase, using more larger pumps to pump more fluid and proppant into wells. Horizontal wells got longer. Whereas Devon’s initial wells extended horizontally for around 2000 feet (Steward 2007), today the horizontal portions of wells routinely exceed 15,000 feet and might have dozens of separate fracking stages.

The amount of gas a well can produce and the speed wells are drilled have both steadily increased. Between 2007 and 2023, gas productivity per drilling rig (the amount of gas produced per day by all the wells a drilling rig drilled in a given month) increased by nearly 100x in the Marcellus Shale region. Automation, scale, and smaller technological advances turned the exciting frontier of shale fracking to a mature, dominant industry.

Improvement in other shale fields has been substantial, although not as meteoric: Drilling rig production increased 5x in Eagle Ford, roughly 30x in the Bakken, and roughly 12x in Haynesville. 

From oil and gas to geothermal

Today, this oil and gas drilling technology is being used to produce geothermal energy.

As we’ve mentioned, though heat from the earth is available anywhere, not everywhere has water flowing naturally flowing through fractured rock that can be tapped to extract this heat. But using hydraulic fracturing, a network of interconnected fractures can be manually created in hot rocks deep below the surface. By pumping water into these fractures (a technique similar to another oil and gas technology, waterflooding), heat may be extracted.

Using fracking to create artificial fracture networks is the goal of so-called Enhanced Geothermal Systems (EGS). EGS technology dates back to the Los Alamos Hot Dry Rock Program of the 1970s, but has been given a boost from the technology used to frack shale reservoirs, such as using slick-water, multi-stage fracks in long horizontally-drilled wells. EGS has also benefited from monitoring technology such as microseismic fracture mapping to determine the size and extent of the fracture network. Fervo Energy tested a successful EGS in Nevada that created a fracture network between two horizontal wells with multi-stage fracks, achieving higher fluid flow rates (a proxy for system performance) than any prior EGS project. Fervo specifically credits shale gas drilling technology in making its EGS technology possible:

"Leveraging technology innovations from the unconventional oil and gas industry provides a pathway to unlocking new geologic resources and improving project economics in a way that could enable geothermal developers to mimic the rapid scale-up observed in shale development over the past two decades."

Oil and gas drilling technology is also being used to develop other types of geothermal energy systems. Eavor, for instance, is using directional drilling combined with drilling multiple laterals from a single vertical well to create closed-loop geothermal systems. Eavor’s system also seems to be based on another oil and gas technology, called Steam Assisted Gravity Drainage (SAGD). With SAGD, two horizontal wells are drilled next to each other, and steam is pumped into the upper one. This heats up the surrounding oil, causing it to drain into the lower well, making it possible to extract very viscous oil from reservoirs. With closed-loop geothermal, heat is extracted by pumping water through a series of drilled holes, rather than via fractures in the rock. While the amount of drilling such a system requires has historically made it uneconomical, advances in drilling productivity enabled by things such as PDC bits could potentially change this calculus.

What lessons can we learn from the shale revolution?

First, it’s difficult to predict the trajectory of technology, and what some capability will ultimately be used for. Over and over again, we see technology spillovers, where technology developed in one industry for some particular purpose ends up spreading to other industries, often to solve very different problems.

Oil and gas drilling technology repurposed to create geothermal power systems is of course one example, but there are many others. Steadily advancing microchip technology made it possible to generate the downhole telemetry that made horizontal drilling feasible, and greatly lowered the cost of processing large 3D seismic datasets. PDC drill bits, which greatly increased oil and gas well drilling productivity, were (ironically) originally developed by the DOE for geothermal well drilling. 3D seismic mapping, which proved critical to Devon Energy’s horizontal drilling efforts, was based on technology originally developed to find Soviet submarines. Early demand for directional drilling in the US came from offshore oil and gas platforms, as well as the Atomic Energy Commission, which needed to drill directional wells to gather samples from underground nuclear tests.

Fracking also shows the importance of serendipity, and of a keen eye for unusual patterns in scientific and technological discovery. Fracking was originally developed when Floyd Farris happened to notice the relationship between acid injection pressure and subsequent well performance. Mitchell’s breakthrough came when a fracking gel failed to set properly, and Nick Steinsberger began investigating the well’s subsequent performance. Such serendipitous developments are common, from the creation of titanium dental implants to the invention of fast-drying automotive paint.

Mitchell also benefited from government oil and gas funding efforts such as the Eastern Gas Shales Program, begun in 1976 by the DOE’s predecessor ERDA, which was part of a larger government effort to develop alternative sources of natural gas following the 1970s energy crisis. Mitchell’s record-breaking frack in 1978 was funded by the Gas Shales program, and Mitchell’s first 35 shale gas wells were drilled with government funds. Mitchell (along with other gas shale producers) also extensively studied the maps and other data produced by the gas shales program when trying to “crack” the Barnett. Government support was also critical for the development of other oil and gas technology, such as 3D seismic mapping, microseismic fracture mapping, and PDC drill bits.

Nevertheless, technology development remains risky and uncertain. There were many times, according to Mitchell geologist Dan Steward, that development efforts in the Barnett were “on the verge of failing."

This underscores one final lesson: technology development and commercialization often requires an extremely long time, decades or more. Mitchell Energy spent 17 years and over $250 million dollars drilling wells into the Barnett Shale, gathering data and experimenting with different fracking methods, before it developed a method that worked and began to turn a profit. It took years of learning and experimentation before Mitchell cracked the code, and similar learning and experimentation would be required when adapting Mitchell’s methods to other shale gas resources. Mitchell’s success in turn was built upon decades of previous advances in fracking technology, 3D seismic measurement, downhole monitoring technology, improved drill bits, and so on.

First 12 months of gas production in Barnett Shale wells, by date of well drilling, via Steward 2007

As oil and gas drilling technology gets repurposed and further developed as a geothermal energy technology, it will be useful to keep these lessons in mind.