Reasons and Effects of Rapidly Growing AI Spending in Business

Introduction

Businesses today are rapidly adopting artificial intelligence (AI) technology to solve more complex and resource-intensive problems. This is leading to a significant increase in AI spending. According to data from Big Technology's AI Summit, while individual token costs are decreasing, the volume of tokens consumed for each task is increasing exponentially. This trend shows that efficiency gains are lagging behind the demand for complex agent-based workflows.

Problem Analysis: Key Reasons for the Increase in AI Spending

1. The Rise of Complex Workflows

Businesses have started to use AI not only for simple data analysis but also in multi-step and highly dependent workflows. For example:

• Automatic response systems and problem solving bots in customer service
• Forecasting models and risk analysis in supply chain management
• Candidate evaluation and recruitment processes in human resources

Such workflows cause the consumption of a large number of tokens at each step. For example, the number of tokens required to resolve a customer request can be 10-100 times more than a simple query.

2. Increasing Trend in Token Consumption

AI models generally work through text-based inputs (prompts) and outputs (tokens). The number of tokens represents the amount of data the model processes. Current trends are:

• Decline in Token Costs: AI model providers are reducing costs per token due to competition. For example, the token cost of the gpt-4 model has decreased by up to 80% over the years.
• Explosion in Token Volume: In contrast, the number of tokens consumed for each task is increasing exponentially. For example, as the steps involved in a workflow increase, token consumption grows exponentially rather than linearly.

3. The Rise of Agent-Based AI

Agent-based AI (agentic AI) are systems that can make decisions on their own and automate complex tasks. These systems manage workflows that often consist of multiple steps and interact with the AI ​​model at each step. Examples:

• Order processing process on an e-commerce site
• Automatic evaluation of a loan application at a bank
• Automatic updating of patient records in a hospital

In such systems, the number of tokens consumed at each step is much higher than in a simple query. For example, the number of tokens required to process an order can be up to 10,000.

Solution Steps: Controlling AI Spending

1. Optimizing Token Consumption

One of the most effective ways to reduce AI spending is to optimize token consumption. The following steps can be followed for this:

# An example of a good prompt (for example, to summarize a text)
prompt = """
Convert the following text into a 5-sentence summary:

[Text goes here]
"""

# Example of Python code for model selection (using OpenAI API)
import openai

# Choose a model that consumes less tokens
response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello, how can I help you?"}]
)
print(response)

# Using OpenAI API response to track token consumption
response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Hello"}]
)

# Print the number of tokens to the screen
print(f"Total number of tokens: {response['usage']['total_tokens']}")

# Python example for caching (e.g. using Redis)
import redis

# Create Redis connection
r = redis.Redis(host='localhost', port=6379, db=0)

# Cache the query
query = "Hello, how can I help you?"
response = r.get(query)

if response is None:
    # If the query is not in the cache, send it to the AI model
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": query}]
    )
    # Cache the response
    r.set(query, response.choices[0].message.content)

print(response.choices[0].message.content)

# AWS EC2 instance selection (for example, to host the AI model)
# Avoid using an unnecessarily powerful server
instance_type = "t3.medium" # Medium CPU and memory

# A good prompt example (for order processing)
prompt = """
Verify and confirm the following order information:

Customer Name: [Name]
Product: [Product Name]
Quantity: [Quantity]
Total Price: [Price]

Type 'Yes' to confirm.
"""