|
Blog
|
|
Oil crash only a foretaste of what awaits energy industry
17-Mar-2020
The major certainty of the year is that the oil and gas industry is facing global challenges. The major uncertainty of the year is which industry incumbents will fail to innovate and which will invest strategically to revolutionize their efficiency.
Traditionally, oil downturns are weathered by cutting costs across the board and waiting for good times to come back. The best word for such an approach this time around is suicide. The good old days are not coming back, and those organizations that fail to invest in sustainable operational efficiencies will die. This is made abundantly clear given the issues outlined in such articles as this March 15th piece from the Financial Times. Survivors in the energy industry are now and will indefinitely be those that organize their biggest assets (their unstructured data) so they can leverage them to enable lean data-driven processes and intelligent decision-making.
We are proud of our clients, from the oil majors to the non-profit professional societies, for doubling down on using our platform to organize their unstructured data. Ask us how we do it.
Follow us at www.delfinsia.com
Read reference material, here
|
|
|
|
Putting data at the Center of the Oil & Gas Industry
10-Mar-2020
Open Subsurface Data Universe, now wrapped up in the Open Group, is an initiative started by Shell engineers to overcome the industry’s conventional tendency not to share any data and instead to enable expedited sharing when sharing does make sense. The concept is that there exist some categories of data that, when shared by all, benefit all. There is an emphasis on subsurface data, but the group recognizes a need for general enablement of sharing through the creation of repositories and of shared standards for storage and access. Along these lines, the group touts the benefits of relying on organizations external to the firm in helping to realize the full potential of digital transformation.
At the end of April 2019 in Amsterdam, Johan Krebers, GM Digital Emerging Technologies/VP IT Innovation at Shell, gave a talk on “Putting Data at the Center of the Oil and Gas Industry”. In the talk, Johan declared that the major problem holding the industry back is a data access problem. Too many resources are devoted to trying to find data, and the ability of practitioners to add value in the firm is reduced because of the narrow scope of their data access. The result is that the industry is operating at far less than full capacity. Dramatic boosts in productivity, cost saving, and safety performance are immediately realizable when this problem is overcome. Krebers emphasized five recurring issues that lead to these two major problems.
- Data currently needs to be formatted in a particular way to be acceptable as input by a given software tool. As a result, people are having to spend a lot of time converting between formats, leading to human error.
- Data is currently stored in silos within the firm. These silos prevent any aside from the creators of the data from accessing the data without considerable hassle. In addition, the silos tend to be recreated by each generation within teams, so decisions are not informed by historical data buried in other silos.
- There is a distinct lack of metadata stored with files in folders. This makes it difficult for practitioners to know in which workflows a file could be useful or what kind of value exists within the file without opening and exploring it exhaustively.
- Energy firms are dealing with no or limited search capability. As a result, finding files relevant to a search query and the particular needed data from within relevant files is an arduous manual process.
- Even the structured data within the firm is set up and stored in ways that are not suitable for machine learning or other data driven applications to capture their potential value.
The solution, as Krebers sees it, is to put data at the center, both within the company and across the industry. He believes that a related cultural shift is inevitable. The emerging dominant culture in the industry is exemplified by individuals who rely on full data sets to make transparent decisions and who take the broader needs of the whole company into consideration as they create and share their data. Driving cultural change is always difficult, even with market demand on the side of the shift. There are, however, four specific actions Krebers and the Open Subsurface Data Universe community recommend that firms across the industry take.
- Separate data from the applications that it is created from or traditionally used with. The goal is to have data formatted in such a way that any application can potentially access it.
- All data, whether structured or unstructured, needs to go into a single data platform accessible across the company and usable by powerful data science techniques.
- Develop support methods for meta and master data such that files are assigned multiple tags each designating them relevant to a certain team or workflow.
- Develop search functions that enable real-time data access so that practitioners can make fully informed decisions without delays. Krebers believes this is increasingly important.
The realization of each of these four strategies relies on a common tactic: the conversion of unstructured data into structured data. Considering Kreber’s first point above, one type of “unstructured data” is actually structured within a file according to an existing protocol or template, but that protocol or template is simply the wrong one for the intended use case. The output structure from one application, in the most common example, is different from the required input structure for the application at hand. If an era is coming in which all applications for oil and gas practitioners share common data input and output formatting, that era is not yet here. In the meantime, systematic machine conversion between formatting is essential lest impractical amounts of employee time get wrapped up accomplishing diminishing returns in rote manual data transfer efforts. Some of this automation can be accomplished in-house, but there is also a place for external service providers to offer file format transition solutions to expedite existing workflows and to make better use of data currently untouched by existing workflows. This holds true whether the applications in question are incumbent software tools or data science tools that are new to the industry, and it holds true whether the data in question is being passed form one workflow to the next or is accessed from a central repository by tools designed to handle big data.
Another type of unstructured data is buried in textual files and needs to be converted into tabulated data in order to be useful to practitioners attempting to crunch numbers. These unstructured files are often historical, produced at a time when the typed word was the best and most useful way to ensure utility in the future (they may contain tables, but not easily reconciled with the master tables baked into contemporary workflows). This format has become obsolete when it can be replaced by automated tabulated reporting lending itself much more readily to today’s data analytics initiatives. For example, if a person used to walk around a facility and check meters manually and then write up a report, likely this practice has been replaced by automatic tabulated data population by remote sensors, and yet the old files still exist. In these cases, the buried data in the historical textual files is still useful, but it needs to be scanned and converted into tabulated data in large batches impractical for manual labor. There are other instances where unstructured textual data arguably can never be converted into today’s structured data without loss of value. These include the recorded observations of individuals in complex and dynamic environments irreducible to options on a dropdown menu. These also include the distillations of wisdom from industry veterans that deal with grey area or the space between cells in the spreadsheet. Decision trees dictating best practices would be too enormous and dynamic to construct manually, and so the typed word prevails in communication of observations and best practices. In these instances, however, the unstructured data scattered across these valuable files still needs to be considered by the applications dealing with structured data. A system for extraction needs to be established and maintained.
Another way to derive structure from unstructured data is to tag the file containing the unstructured data with a handle that is recognizable by applications and humans alike and that affiliates that file with a workflow, a location, a team, a time period, or another designation of value to the firm. The proliferation of data files is accelerating rapidly across all industries, but the collection of files in oil and gas was large to begin with and was siloed. Vast amounts of structured and unstructured files are stored in file folders that are themselves unstructured. Given its lack of accessibility, all of this data is, practically speaking, unstructured. The task, therefore, of manually sorting through all data silos for various files and tagging each based on their affiliation with any of multiple designations is madness. Firms are faced with the option of either starting over or of automating the tagging process of historical records. Starting over would involve creating a new and smart system allowing tags on all new files going forward while either ignoring all the accumulated files from the past or else committing to decades of rote manual processing rife with human error. Automating the tagging would allow the new system to be informed by lessons from historical data before it is fully solidified. Tagging is useful not only for files but also for specific passages of text within files. For example, tags are often used in the categorization of events in the field so as to allow for optimal resource allocation and predictive maintenance.
Finally, workers already tend to have real-time access to their own structured data through dashboards connected to sensors. If search functionality is to create additional value to this, it will be through including in search results the relevant unstructured data across all the types mentioned above. The intents of questions entered into search bars can be broken out using natural language processing, but if the only answers that come back are complete files with tags matching key words or are specific passages containing key words, the application will fall short of functional utility. This is because there are too many recurring key words across many nuanced contexts within the industry for the list of all of the instances matching any given key word from a search to be relevant, let alone helpful. In addition, if key words or intents are matched with whole files, the user still then needs to search within each of the unstructured files for the buried intelligence they seek. For enterprise search to work effectively for firms in a complex, technical, and dynamic industry like oil and gas, the search function needs to match the intents of the questions with the intents of the passages throughout all relevant unstructured textual files.
Overall, the creation of a data-centric work culture in firms across the industry seems inevitable, and the efforts internally to drive the necessary changes in order to realize this culture are complex and will take considerable time and effort. One recurring need that sits at the beginning of all identified categories of effort towards this goal is for the conversion of unstructured to structured data, whether by reformatting, tagging, or finding. Organizations interested in realizing the benefits of having data at the center of their firm and their industry will need to define numerous specific work scopes for the conversion of unstructured to structured data, and then they will need to execute those work scopes either making use of current internal resources or leveraging external knowhow. Given that a large part of this conversion is a batch process rather than an ongoing need, and given the industry’s lack of experience with unstructured data management, the devotion of major resources to building internal capability would seem ill advised.
|
|
|
|
Engineering Software Market Size will reach US$ 46 Billion by 2024
3-Mar-2020
Last year, Market Watch reported that "The global engineering software program market is anticipated to witness sizeable boom opportunities within the oil & gas enterprise," with the full engineering software market expected to exceed $46B by 2024. In the course of the last year, the move toward adoption of software has grown significantly in the energy industry even as the industry has continued to consolidate. Perhaps less foreseen overall has been the valiant attempts made by some industry incumbents to develop their own internal fit-for-purpose software solutions, potentially limiting growth toward this market projection depending on how it is measured.
The competition rages among new data analytics firms, the software majors, the management consulting firms made over as software solution providers, and internal oil and gas teams for a share of this growing market. Yet, few seem to want to touch the dirty unstructured data, historical or contemporary. But to get viable data-centric workflows set up and utilized to fully extract value, someone needs to find, tag, organize, and convert this unstructured data into useable formats. We're proud of our ability to fill this growing need!
Read reference material, here
|
|
|
|
View From the Top: A Leading Energy Economist’s Perspective on Digital Energy
03-Feb-2020
The oil and gas industry is saturated with conferences, publications, and sales pitches pertaining to digital strategies for efficiency and performance. Industry incumbent participation in the hype reflects something of a mix between FOMO (fear of missing out), FOAB (fear of appearing behind), and fatigue (“When can I leave this session and go back to work?”). Will digital transformation continue to drive business impact beyond the duration of the hype? Is it important enough to receive devoted resources today? To answer these questions, I thought to seek a perspective positioned a few steps back from the operations business and focused on the greater forces shaping the digital transformation trend to begin with. For such a perspective, I turned to Ken Medlock, (https://www.bakerinstitute.org/experts/kenneth-b-medlock-iii/) the James A. Baker III and Susan G. Baker Fellow in Energy and Resource Economics at the Baker Institute for Public Policy and the senior director of the Center for Energy Studies at Rice University.
I met with Dr. Medlock toward the end of 2019, grateful for his having given me an hour of his time. Medlock is the kind of person whom the heads of countries solicit as an economic consultant. He is a highly decorated economist and thought leader, and his making time for me and this article was very generous. The rest of this article is a transcript of our conversation, starting with me laying down the context for the meeting.
AW: The oil and gas industry is undergoing an unprecedented collection of widely discussed simultaneous changes: A generation of experts is in the process of retiring; a new and much younger generation is stepping in; onshore development has grown enormously; domestic prices have dropped and would seem poised to remain low; operational efficiency has become a primary goal; digital transformation has become a hot topic toward that end; collaborative data optimization efforts between industry players are under way; and third-party venders are lining up for a piece of the action. The scope of discussions on these topics within industry incumbents is deep and focused, but these are tied to larger global economic trends and are more thoroughly understood when also viewed from high-level economic perspectives. Along those lines, Ken, what is happening right now in energy at a macroeconomic scale?
KM: The global energy economy is very dynamic, and many of the changes currently are regional. Talking at too high a level runs the risk of whitewashing the important regional dynamics. Let’s start with energy-producing regions in the US. They have become increasingly outward facing, meaning export orientation has become more the norm. If you go back to the middle of the last decade, nobody in the US was talking about exporting. Now, it can sometimes seem to be all anyone is talking about. That said, the domestic market remains massive. The US consumes enormous amounts of energy across the power generation, industrial, residential, commercial, and transportation sectors. The opportunities to deliver to domestic consumers still matter.
That said, in terms of growth opportunity, aside from natural gas, it is really a discussion about the developing world, with Asia in the limelight. When we look at demand collectively in OECD [Organization for Economic Cooperation and Development], that demand is flat. The interesting dynamics there are about how certain fuels are outcompeting other fuels. So, some energy sources are increasing but at the expense of others. All the growth in global energy demand is occurring outside of that umbrella of countries. That is very important because you are talking about growth at a scale that is unprecedented for the next 20 to 30 years. A lot of energy companies are getting their heads around that right now. It’s unprecedented because there are 1.3 billion people in the developed world, the OECD, and 7.7 billion in total. So, we have propensity for demand growth from roughly 6.4 billion people today, plus another 2 billion between now and 2050—all outside the OECD—that is unlike anything we have ever seen. Just to be blunt and make it simple, this means that everything is needed.
A lot of energy companies are trying to find their way in that new reality and figure out how will they fit in. Does a decline in market share mean a decline in market? Generally, no. For example, if I own 50% of a pie that doubles in size and my market presence needs to be of the same scale, what share do I need? 25%. This concept is something that folks in the energy space are still getting their heads around. Many companies and their investors are worried about declining market shares, and some of that worry is misplaced. In the policy arena, this misplaced worry also exists. A lot of that is engendered by the fact that we all view the world from where we sit. Here in the developed world, demand is not really growing, so there is an intense competition for market share. Gas is knocking coal off the stack. Renewables are capturing various margins of growth in electrification. Electrification continues to increase, which is natural in the course of development. How that all ties together is something that everyone grapples with on their own scale and within their own part of the energy economy.
AW: How do you reconcile the growth in demand you mention internationally with the touted longevity of domestic break-even pricing that I keep hearing about?
KM: Particularly in the US, onshore upstream development has been a challenging situation because of the pricing that you mention. Unconventional resource development is still a new frontier. A lot of people believe that shale is what it is and the processes for developing it have reached an asymptote. Those people say that development is going to cost what it’s going to cost, and so the price needs to be above a certain threshold for it to make sense. That is just not true. People had those same thoughts in 2008 and in 2014. Here we are in 2019, and, despite excellent reductions in development costs, many still cling to the same false assumption that no more innovation can be achieved. What’s remarkable about that is that the price environment needed to make everything work has been different in each of those cases. The space is still evolving.
Another consideration is that the thin margin environment is not bad for all players. It serves to increase the comparative advantages that larger companies enjoy. The firms that can bring scale efficiencies to the field benefit as their smaller competitors suffer. They can acquire contiguous acreage and connect up wellsites, allowing for management of processes like water distribution, crew allocation, how rigs are rotating, and in-field produced-fluid treatment. This lowers their costs. The pricing environment, being where it is today domestically, rewards scale efficiency. That means these companies expand their presence, driving production increases that keep prices low. In a Darwinian sense, we are in a heard-thinning time.
The smaller competitors need to lean into their comparative advantage, which is their relative agility, and adopt new technologies quickly to create more efficiency and hence more margin. Some of them are doing that, and those will be the ones that survive. When margins are thin, companies do not necessarily just bunker down and lay people off; they actually try all kinds of new things. There is evidence that innovation accelerates when margins are thin, because intense competition serves a higher potential reward from any successful innovation. Coasting at high margins is what discourages innovation. If you have $100+ oil, you may be just interested in drilling the next well and not really be thinking about efficiency in operation and capital the way you do when margins are thin.
AW: What’s keeping domestic prices low if exporting is such a good idea right now?
KM: In the short term, you bring enough supply on that demand cannot absorb it all and the price has to drop to encourage demand to be higher. At the end of the day, the market has to balance, and price is the signal around which they balance. There are a lot of concerns recently about global economic health and trade wars. That puts a negative bearing on price, because there doesn’t appear to be a consensus that the economy will continue to grow sustainably. When companies are looking at a greenfield investment opportunity, like the full set of value chain investments aimed at exporting their products to developing countries in Asia, they do not just look at the next 5 years; they look at the next 30 to 40 years. If you have a big enough balance sheet, you can absorb short-term challenges and will continue to push forward with a view to the long-term future.
Upstream onshore is the major casualty of the weak price environment, because it is a lot about short-cycle development. The capital costs are spread out over many smaller leases and individual wells, so it is easier to scale back in reaction to short-term political and pricing signals. This leads to a slowdown in drilling and a decline in rig count, which means a decline in production, which we saw in a major way just a few years ago. If people are not sensing that growth is going to be robust in the next 9 to 24 months, they slow down a little bit. When you get into bigger infrastructure projects like offshore rigs or facilities designed to trade hydrocarbon products internationally, those are not impacted in a similar way when it comes to near-term concerns about the economy. Overall, long-term global demand is there and it is not going anywhere, so projects with long lead times and operating lives tend to be focused differently.
AW: How are new popular perceptions of trade wars and environmental regulations adjusting oil company political affiliations?
KM: Energy and other capital-intensive industries just want clarity on regulation. They tend to have much more long-lived assets, and they seek confidence that they can generate a predictable return over the course of a few decades from those assets. If the regulatory environment shifts dramatically, the forecasts they counted on are disrupted, and they are less likely to invest in the next project until they sense stability again. Any connection to conservative or liberal politics are more an indication of how things are moving recently. If a US president announced that there is going to be a carbon price of X and that it is going to be legislatively determined and hence written into law, there would be some hand-wringing, particularly by firms that have already sunk investments in assets that will do poorly under the new law. But, even a drastic regulatory shift could help overall if it firmly and clearly establishes a new norm under which all firms can adjust and operate. The polarization of politics injects more uncertainty into what legislation is likely to be passed and what regulations are likely to last. Political uncertainty like we are seeing today is generally bad for all capital-intensive industries, including energy.
AW: Is the current importance of efficiency the cause for all the consolidation in the energy industry currently, and what can companies without large-scale operations do to become more efficient?
KM: It has a lot to do with it, certainly. We are in a thin-margin environment in onshore upstream. That favors efficiency. If you are a firm that can’t bring scale efficiency to the table, you’ll have to look for other efficiency or else get out of the industry. Some of the small independents and mid-caps are looking at divestiture, and others are throwing all their weight into new efficiency technologies. The big IOCs [international oil companies] constantly look for ways to improve operational efficiency regardless of the pricing environment, but they are the slow-moving behemoths in the energy space. They are not generally the entrepreneurs but, instead, tend to move in once a concept is proven and often find ways to improve upon it or better integrate it. In that sense, when an oil major starts using a technology, it sends a signal to the industry that the technology is proven and valuable. In some cases, it is innovation by acquisition, but you have to understand that the innovation space is diverse. It’s not just about trying new physical technologies in drilling or completions; it’s also about how you approach your operations. As you see something develop in technology generally that could have a place within your organization to create operational efficiencies, you can move projects from low to reasonable margins and you can even transform negative returns into positive returns. This part of the oil major’s approach to innovation is particularly encouraged by the current pricing environment and is also typically where you see the majors looking at making acquisitions.
AW: Are there cultural barriers to achieving the focus on operational efficiency?
KM: That is a firm-by-firm type of question. The oil and gas industry actually has a pretty broad range of culture from company to company. While I can make generalizations by firm size as I did answering your last question, historically there have been firms that lag the operational efficiency frontier, and there are firms that are pushing the cutting edge through internal research, partnerships, and contracting. Overall, though, I think it is fair to say that the US oil and gas industry is currently looking for ways to improve efficiency across the board.
AW: What are some ways you’ve seen that firms are trying to boost efficiency?
KM: A lot of what firms are doing to try and boost efficiency involves data. There is a tremendous amount of data gathered about how crews are moving, how water is transported, how waste management is occurring, and many other of these sorts of things. Bringing this all together and making sense of it is a classic big-data problem. There are new and innovative ways to analyze it and streamline it that do not necessarily come from within the oil industry. The oil companies are being pushed into digitalization, artificial intelligence, and data science. This is a big category with a lot of low-hanging fruit. There is also a tendency toward remote operation and toward robotization, which creates more data from increased communication between humans and from more sensor output. This feeds into the data science area, and they are capable of driving big efficiencies. If someone in Houston can manage a well in West Texas without driving out there, and they can operate it using remote control and automation, that is already cutting costs. Add on to that how they can make faster and better-informed decisions from their data, and you can begin to see how efficiencies can really increase.
Something else to recognize is that benefits from all these new technologies do not only fall in the realm of efficiency. Companies can discover entirely new opportunities using data science or through robotization. We hear about companies using cutting-edge techniques to review their knowledge and data and discovering new ways to stage their wells or a previously overlooked pay zone. Technology can also help them to dodge important pitfalls that they might not have otherwise. Efficiency may be a primary driver, but oil companies are deriving other categories of value from embracing new technologies.
AW: How practical is it for oil companies to try to tackle software development?
KM: Wow. Now that is an interesting question. There is an element of that occurring with internal tools that are relatively simple to develop and closely tied to the physical expertise of the industry. The oil and gas majors are very technically savvy and are doing some Space Age stuff in the field and offshore. That said, there are many different disciplines in engineering, and it should come as no surprise that software is not the oil industry’s comparative advantage. We are starting to see companies in the industry explore more third-party contracts and partnerships. They are look at how to work with Google and other large software firms, but they are also looking at how to work with smaller and industry-focused solution providers. In software, there is a place for all three: internal development, outsourcing to major software companies, and outsourcing to smaller industry-focused players. They make these decisions depending on where they see the most direct path toward achieving operational efficiencies. Despite fantastic engineering capabilities internally, the fact is that they still often get more value more quickly and cheaply from working with companies that have a specific focus and proven track record rather than reinventing the wheel internally. I often get some looks of surprise from my students when they begin to realize that the oil industry is one of the world’s top high-tech consumers.
AW: Where do you recommend leaders in oil and gas turn for information on how to evaluate these decisions?
KW: The statement “paralysis by analysis” is commonly applied to this situation in the energy industry. There is a widely noted risk right now that leaders can wind up doing nothing because they are overly focused on learning what they should do. In reality, the first thing that leaders in the energy industry need to do is to recognize and accept that the world is changing. From there, it becomes a matter of considering how various changes in the global energy-economy affect your firm. Then, while heavily utilizing their internal expertise, they can lean on consultancies and other institutions for aid in that domain. This is largely what they are doing already. If there are ways of better utilizing internal knowledge to make decisions more accurately and more efficiently, that should be done, and it generally is. From that perspective, investing resources into finding the best way to capitalize on internal knowledge and creating an environment where its dissemination is encouraged can be a valuable exercise.
|
Kenneth B. Medlock III is the James A. Baker III and Susan G. Baker Fellow in Energy and Resource Economics at the Baker Institute and the senior director of the Center for Energy Studies at Rice University. He is also the director of the Masters of Energy Economics program, holds adjunct professor appointments in the Department of Economics and the Department of Civil and Environmental Engineering, and is the chair of the faculty advisory board at the Energy and Environment Initiative at Rice University. Medlock holds a PhD degree in economics from Rice University.
|
Read reference material, here
|
|
|
|
How artificial intelligence is already impacting today’s jobs
28-Jan-2020
More thoughts for the AI build-or-buy conundrum:
LinkedIn's Q3 2018 research suggests that the industries embracing AI are experiencing faster changes and more innovation: "When we zoomed in from the industry level to the skill level, we found that many of the changes were due to a rise in (1) data and programming skills that are complementary to AI, (2) skills to use products or services that are powered by data, such as search engine optimization for marketers, and (3) interpersonal skills."
Embracing AI means attracting to your firm people with AI-related skills, but it also means empowering your firm with AI to attract people ready to utilize AI-empowered workplaces. Both groups are highly sought after for the positive impact they have on their firm, but oil and gas is already great at attracting people with degrees like petroleum engineering and geoscience who have the skills to use products and services powered by data. This does not mean these folks need to become competitive data scientists; it just means that oil companies need to make sure their data is clean, accessible, and fed to AI tools these practitioners can use.
Read reference material, here
|
|
|
|
AAPG Interview on Alec's Unstructured Data Course
16-Dec-2019
Barry Friedman, a writer for AAPG Explorer in Tulsa, Oklahoma interviewed me and then wrote a story on my "Intro to Unstructured Data and Machine Learning in Oil and Gas" course offered through AAPG. We're publishing the interview in this blog.
1. Talk some about unstructured and structured data
Structured data is data with a home. A cell in a spreadsheet is a home for the information in that cell. It's easy to reference it using labels and tags like the row and column header. It's easy to compare to other data across a row or down a column. Structured data is usually numerical, and often it's created by machines (sensors, lab equipment, computer calculations, etc). Unstructured data, by contrast, is the data that people and traditional software have a hard time finding and using. It's the vast majority of all data: documents, manuals, presentations, reports, etc. Think of this sentence as an example. It's buried away in a paragraph of other text, so finding it and using it is difficult. Unstructured data in the industry tends to be textual and tends to be created by humans, not machines. Let's say that somewhere down the road someone wants to see an example of unstructured data, perhaps having forgotten what it is. How would they find the above example sentence buried in this paragraph, in this article, in this publication? Typically, the incumbent solution is key-word search. Most enterprise search falls into this category: a search tool lets users type in queries about what they're looking for and then comes back with documents that contain the words used in the query. That doesn't work very well in the enterprise, because there are a lot of redundant terms (TOC, permeability, fracture, etc). Ask a question like, "Where is the most mature source rock in the Delaware Basin?", and the tool will give you back tons of documents containing the words "Delaware Basin" or "source rock". Those tools do not understand the context or intent of your question. This limitation has existed in software since text was first digital, and it's the reason why companies spend millions on data organization and tabulation projects. It's also the reason why the energy industry reportedly spends at much as 80% of their time looking for their own data despite having enterprise search tools. However, new methods are now allowing software to access buried and scattered unstructured data, unlocking enormous potential for organizations to boost produce, cut costs, and improve safety.
2. About the workshop: How did it come about? What made you realize that there was a need?
Typically, I try to use design thinking to figure out what people are interested in for workshops, but this one was just as much the product of my own interest. I really like operational efficiency, and I'm passionate about eliminating rote work because I am allergic to boredom. All the rote work, boredom, and operational inefficiency that comes from traditional methods for handling unstructured data have me very caught up in the topic. I would still be offering this workshop even if it wasn't in demand and even if my company wasn't affiliated. As it is, however, this topic does address a strong need across the industry. I moved around a lot when I worked for an oil major. I was in research and development for chemicals, then external consulting, then software product management, and then reservoir engineering. I've moved from downstream to upstream, and I've also sat in a few different types of engineering teams. I saw the same ignorance around unstructured data and encountered the same challenges everywhere I went. After grad school, I started doing digital transformation consulting for multinationals. A big part of it had to do with unstructured data management best practices. This workshop was first an offering to clients to get everyone on the same page about the nature of the problems so that solutions could be discussed. I've since tailored it to the oil and gas industry. Thanks to AAPG for letting me try it out in San Antonio a few months ago, I've seen that it's well received and have since begun to offer it elsewhere for the industry. Given the overlap between machine learning and some of the solution options for handling unstructured data, I've begun to incorporate a quick intro to the concept of machine learning in the class, too. It serves as a kind of demystification segment so everyone can be on the same page when they're discussing the concept of machine learning. That segment is very popular, so I've increased the focus on it in the course.
3. Five-hour workshop and you’re covering seven distinct segments. What is the realistic expectation of what skills people will leave with at 6 pm on the 22nd?
The workshop is about raising awareness of how different types of data create different types of bottlenecks and challenges within organizations. So far, attendees have told me that they've exited with a strong intuition on structured and unstructured data and the tradeoffs in various methods of addressing the challenges presented by each. I've tried in the description to make it really clear that this workshop will not teach the technical parts, but even so, sometimes people attend hoping to learn how to build effective natural language processing solutions to navigate unstructured data right then and there. It took us the better part of four years to build a tool that deftly handles unstructured data for the energy industry, and my own personal technical contributions to that solution were pretty small, so I can't claim to be able to offer a single short class that gets people up to speed on building effective software. To do this stuff at a competitive level requires devoted effort from an all-star team in a very agile environment plugged into the rapidly evolving techniques in the burgeoning world of data science. Even incumbent "big tech" firms are struggling to build these things internally. My goal with this course is to get practitioners in the energy industry up to speed on the issues and tradeoffs. Everyone is eager to create value, and their decisions about how they go about getting their work done need to be informed by the basics of contemporary data management.
4. The topic, I gather, is more relevant today than ever. Is the industry making progress on this?
I'm pretty frustrated by the progress the industry is making. It seems to me that many decision makers are still hoping to wait out the storm, as if the good old days of easy money will come back. The reality is that sophistication in data-driven decision making is a sustainable competitive advantage. Operational efficiency is software driven, and remaining ignorant about software is not a viable option. Just as frustrating to see is oil companies throwing all their weight being the first potential solution they see without proper technical or business vetting. Knee-jerk data science hires, corporate venture capital investments, and partnerships can lead to problematic inertia preventing adoption of practical solutions that did not emerge in the vetting process. For example, one leading oil company invested significantly in an AI vendor that had not yet built a solution. They did not bother to explore whether that vendor startup had competitors. When another vendor comes along with a finished solution, the oil company now turns them away, because supporting another vendor hurts the equity they placed in the first vendor. Or, another example, an oil major takes a quick look at an AI vendor and decides to build the solution internally with a "how hard could this be?" mentality. A couple years go by and they realize they cannot do it, so they circle back to a vendor which could have helped them from the start. The fact that the oil major led the vendor on, offering projects and pay, and then tried to take some IP from the vendor is also not very encouraging. Part of what I'm out to accomplish in these courses and all my publications and talks is raising awareness about how to frame data problems and vet potential solutions quickly and effectively. My biggest goal to this end is to promote open communication of options and capabilities. No more hiding behind buzzwords for vendors, and no more knee-jerk or impractically-motivated decisions for operators. I want the industry to organize hackathons so we can compare our tech openly to our competitors for everyone to objective determine the differences. That's not a radical idea in the software world; why is that so hard in this industry?
5. With that in mind, this section in the course description struck me: "The drastic inefficiencies created by mismanagement of unstructured data will be given context by the growth in private equity backing, the lower hydrocarbon price environment, the proliferation of data sources, and the aftermath of the big crew change.” What is the scope of the inefficiencies due to the lack of knowledge?
Well, the number oil operators are throwing around is 80%. 80% of time is spent looking for data already contained in the organization. From my own experience, the number is more like 50-60%, but the point is that it is higher than it could be. This prevents oil and gas engineers with software talent from wanting to join the industry. This prevents wall street investors from believing in the future of companies in this industry. This prevents current employees from feeling that their talent, skills, and ideas are fully utilized on the job. Private equity is famous for driving efficiency, so they are focusing in on this point, demanding that their portfolio companies clean up their data practices and use their data more efficiently. When oil companies have lower margins, they have less profit and hence less free cash flow. This can drive down their stock prices, creating a push for more operational efficiency. If they operate more efficiently, their costs are lower, so they free up cash flow even when hydrocarbon pricing is down. Proliferation of remote sensors is creating more structured data, causing oil and gas practitioners to have to create more unstructured data in the form of summaries and reports on that structured data. I read that unstructured data is 90% of the total data and growing. The big crew change saw a bunch of people retire, leaving a huge expertise gap between them and the people who had to fill their shoes. If I have less expertise but need to make decisions with the same level of competency, I spend a lot more time researching precedents within the company. Looking through mountains of unstructured data is rote, time consuming, and hit-and-miss. As a millennial and digital native, I don't consider this kind of work to be a necessary part of life; instead, I loath it and expect better data management and better tools.
One challenge that our data management solution faces is that while ROI is obvious, it's not necessarily quantifiable up front. There is a lot of bottom-up demand, but leaders without experience focusing on data management lack the know-how to evaluate. Say their workers have instant access to all of their data rather than extremely slow access to just part of their data -- exactly how much better will be the quality of their work? Better enough to justify the cost of a software product, but still, how much better, exactly? The answer to that depends on the use case. For example, if an oil operator has two weeks to place a bid on a lease and needs to know whether and how much to bid, and they only have time at their current data analysis rate to explore 30% of their related data, and the last time they were in this situation they overbid by $1MM, there's a pretty compelling argument to use a tool to help them navigate their unstructured documents at the speed of thought. They could evaluate all of their data at a fraction of the total time required and ensure that their bid reflects the value of the lease. And yet, decision makers can still cling to the question, "Exactly how much value will your tool yield?" as an excuse to hold off on evaluating implementation of something if they have decided that they do not want to understand. To answer a question like this in a sales room, I can make educated guesses from strong experience, but I'm not going to pretend to know their use case well enough to tell them exactly what will happen. The writing is on the wall, however, and the companies making their data accessible to their people will drive better data-informed decisions and be more successful generally. Those using our tool have been happy about their ROI.
6. I am curious about the course not requiring the use of any computer. Explain.
I took some software classes and did a project when I was at Stanford with a genius instructor named Marty Stepp. He taught difficult concepts like recursion and the perfect marriage algorithm without the use of computers. I learned those computer science concepts better than my student peers in other programs. I'm no Marty Stepp, but I'm certainly inspired. I also took a bunch of classes at Stanford's d.school and focused that on educating corporate clients in workshops tied to consulting gigs after grad school, and I learned that engaging and interactive learning sticks best. Since, my goal for this course is to inform people who are not interested in building software themselves how to make smart decisions about it, I need to make this course attractive to those folks. If you want to learn machine learning, I recommend Andrew Ng's Coursera courses for the theory and the fast.ai stuff for the implementation. Also, follow the Github and Medium blog posts of the people presenting at prestigious data science and software conferences. If you want to learn about how to make smart strategic decisions in data management in oil and gas, I hope you consider my courses.
Read reference material, here
|
|
|
|
The Three D's of Data Science Management
2-Dec-2019
Juan Contreras is a data science manager at Uber. He led digital analytics for a political party, and then he built a team at a fintech company before joining Uber as a data scientist. Once he was promoted to a data science manager, Juan began spending time reflecting on the variety of demands the job entailed. He noted the frequency and importance of the demands, grouping them into three categories. He believes data science managers can hone their skills in each of the three categories to continuously improve their effectiveness on the job. According to Juan, the role of a data science manager should entail giving the data science team strategic direction and tactical advice, finding and sharing relevant contextual information with the data science team, and working to “unblock” the data science team when they are at risk of being unhappy, ineffective, and inefficient. Juan notes that this third part involves a balance between taking care of the team and pushing their limits to continue their own development. Juan organizes these responsibilities and their corresponding capacities into three categories: the “3 D’s of Data Science Leadership”.
The first D is for Diplomat. What it takes to become an excellent leader in this area is a relentless focus on getting the “lay of the land”. The diplomat can speak the language of cross-functional partners, understanding where they are coming from and what their needs are. This can help position the data science team as an ally with other teams in the organization. The diplomat constantly considers adjustments to the strategic roadmap of the team to align with the needs of other teams and the greater organization. The diplomat also speaks the language of the data science team, communicating with ease their aspirations, strengths, and challenges when conversing with other teams. The diplomat data science manager has the special obligation to get everyone in the organization excited about data science. This involves connecting data science to the goals of other teams as well as the goals of the entire organization. Juan shared an example of diplomat skills in practice: Leadership needs data analysis but hires data scientists instead. They then grow frustrated at the slow process in the resulting analysis. Meanwhile, the data scientists do what they are trained to do, focusing on creating and tuning models at the expense of using those models to analyze the data. The data scientists grow frustrated at the pushes to apply undeveloped models and organize results. It is the job of the data science manager to notice the differences in expectations between the two groups and to help each to understand the perspective of the other. The new awareness earned by both is fed into the strategic plan, and resources are then allocated to resolve the issue.
The second D is for Diagnostician. What it takes to be an excellent leader in this area is to discover prospective opportunities for data science to shine and then, just as importantly, to filter out the opportunities that are a poor fit for data science. As Juan puts it, “data science is a good way to solve some problems but a bad way to solve most problems”. The data science manager is constantly asking whether any of the organizations’ objectives can be furthered by data science. While it is tempting to get creative imagining possibilities to help another team using data science, the diagnostician thinks critically about how the data science team would practically devote resources to lend assistance to the other team as well as what is stopping the other team from deriving the value or resolving the issue themselves. Juan gave the example of a certain collection of data deemed relevant to the decision-making process but that was being ignored in the process because it was too noisy and too large for manual analysis. The data science manager discovered this opportunity and then designed and coordinated the creation of a data product to meet this unstated decision-making need. At the same time, the data science manager found that a certain category of this data was sufficiently organizable without the need of the data science team. The data product from the data science team, therefore, did not handle this data, and another team’s resources were used to handle that data sufficiently.
The third D is for Developer. What it takes to be an excellent leader in this area is to stay sharp on the advantages and disadvantages of the latest theories and methods in data science. As Juan puts it, “Data science is really hard, so you could probably do better.” The developer reads, thinks, talks, and writes often about data science and about data science leadership. The data scientist manager as developer is also aware and makes the team of data scientists aware that they know more about data science than does the manager. Getting involved does not mean getting in the way. Juan’s example is that, when a deep learning model was decided as the best approach to solve a particular problem, the data science manager did not personally develop the model but was the primary contributor of the factors that needed to be considered. The tradeoffs and objectives in approach and product that the data science manager as a developer contributed to the team were functions of the work of the data science manager as diplomat and as diagnostician. Many data science managers earn the position through being an excellent data scientist, so it can be hard to let go of the implementation details and stay out of the way of the data science team. One way of expediting this challenge is to focus on channeling personal developer experience to helping the organization avoid the common mistake of underestimating what it will take to get data analysis to be valuable and timely.
|
|
| |
|
|