Transfer learning from the oil industry

In the US, the petroleum industry was born in 1859, when Edwin Drake first drilled for oil in Titusville, Pennsylvania. A lot of technology for the oil industry was borrowed from coal refining, drilling for salt, and storage, handling, and transportation of water, some contemporary industries of the time.

John D. Rockefeller got into the oil business in 1863 at the age of 24. He incorporated Standard Oil in 1870, which at its peak in 1899 was refining 90% of America's oil!

To understand the landscape, let's look at the three areas into which the oil industry is divided depending on the physical location of crude oil in the value chain:

1. Upstream: Searching for oil wells and drilling for oil

2. Midstream: Transportation and storage infrastructure

3. Downstream: Making different types of products like paints, chemicals, gasoline..

In the early days, a lot of people were focused on drilling for oil (upstream). However, Rockefeller had a key insight - the real value was not in drilling for MORE OIL, it was in REFINING OIL to make it suitable for different downstream tasks - the points of value creation (Think about cotton gin in the first industrial revolution, Bessemer converter in the iron and steel industry).

Standard Oil created a much better refining process, followed by a transportation infrastructure. This led to other producers relying on Standard Oil's end-to-end infrastructure of gathering, storing, refining, transportation, and wholesale marketing (upstream + midstream + downstream).

The subsequent story of Standard Oil is shown well here.

Generally, when a large industry consolidates, a few major players creating value in complementary parts of the industry remain. Lets take a historic look at the strategies of the oil majors. Each of them historically focused on specific parts of the value chain.

1. Exxon - Refining and marketing, NOT exploring - downstream

2. Chevron - Upstream, refining, and marketing - all stages

3. Mobil - Marketer - downstream

4. Texaco - sell crude, license I/P from Standard Oil - upstream

5. Gulf - exploring and producing - upstream

6. Shell - upstream (geologists), refining and marketing - all stages

In today's terms, Rockefeller created a platform for oil services, by leveraging upon a key insight which was overlooked by his contemporaries. REFINING is a deep innovation that can help build a better platform business than COLLECTION.

This is a deep insight from the oil industry. Now, lets do transfer learning.

I'll consider the industry vertical which involves making physical machines intelligent - drones, UxVs, high capex industrial machines like jet engines, wind turbines, satellite data analysis, CNC machines, remote surveillance, etc. We're collecting TBs of sensor data deployed on these assets to ensure their timely upkeep and operation.

Today, data collection + visualization and AI analytics are analogous to oil collection and refining respectively. Better analytics (NOT more data collection) is what solves specific downstream tasks like anomaly detection, classification, regression, thresholding, etc. which create monetary value for real-world AI applications.

This type of visual thinking can help see parallels between seemingly disjoint ideas. Maybe we need to spend more time on building 10x better refining processes. Some other architectural ideas like the multi-headed self attention of the transformer neural network! And transfer learning always helps!