The State of Gen AI insiders Report FINAL 0724 - Flipbook - Page 14
THE
EMERGENCE
OF A
“GENERATIVE
AI TAX”
Generative AI applications are built upon
Simply put, software companies using LLMs
inferences, which are multiple calculations to
will definitely see the ongoing effect of the
determine a reasoned conclusion. An example
Generative AI tax on project hurdle rates and
of an inference would be when a user prompts
company P&Ls. While not as immediately
ChatGPT to “write a poem about a cat.” Given
noticeable to date, this tax will actually
that this is an open-ended task, ChatGPT could
increase significantly. There are a number of
approach this in multiple ways - from writing a
strategies to keep this generative AI tax in
silly, lighthearted poem about a cat playing with
check.
a ball of yarn to something more serious and
introspective.
The cost of inferences needs to be scaled up in a
way that will not accrue the same $4-5 million
dollar per day cost as ChatGPT. All businesses
are going to face this ‘generative AI tax’
associated with building models that are in
production and serving up inference. LLMs
processing costs across vast data sets, housed in
both defined warehouses and data lakes, are
One of the major
inflection points was when
everyone could see that
there was a potential impact
for their day-to-day job,
or to their occupation.
not cheap. Even though the cost is declining,
Florian Douetteau
there will be larger inference precision
CEO & Co-founder
Dataiku
requirements that will continue to drive the
expense curve upward overall.
THE STATE OF GENERATIVE AI INSIDERS REPORT
14
