The world’s abuzz with the power of AI. ChatGPT has really captured the public’s imagination – with good reason. But yesterday Reuters ran a story in which OpenAI’s CEO Sam Altman said (quoted from Twitter) the computing costs of operating ChatGPT were “eye watering”; a couple or more cents per conversation.
In the same article, Alphabet’s Chairman John Hennessy said that having an exchange with AI probably costs 10 times more than a standard keyword search.
I wanted to know how the costs for AI racked up, so I asked the “person” who have the answers, ChatGPT. Here’s out conversation.
What are the costs of AI exchanges?
How much does it cost you to process a text query like this?
As an AI language model, I don’t have a direct cost associated with processing text queries like this. The cost is primarily determined by the hardware and infrastructure used to run my algorithms and process the requests.
My training and ongoing maintenance require significant resources, including large amounts of computing power and data storage. However, once I’m up and running, the cost to process individual text queries like this one is typically negligible in terms of both time and resources.
In general, the processing time for a text query will depend on the complexity of the task and the length of the input text. Some tasks, such as language translation or summarization, may require more computational resources and take longer to complete. However, for simple tasks like answering a straightforward question like this one, the processing time is typically very fast.
What are the hardware and infrastructure costs?
The hardware and infrastructure costs for training and maintaining large language models like me can be significant. Here are a few factors that contribute to the cost:
- Computing power: Training and running large language models requires a lot of computing power, which is typically provided by high-performance CPUs and GPUs. These can be expensive to purchase and maintain, particularly for the most powerful models.
- Data storage: The data used to train language models can be massive, and storing it can be expensive. Additionally, as the model is trained, the intermediate checkpoints need to be saved, which adds to the storage cost.
- Networking: As data is being processed and moved around, high-speed networking infrastructure is required to ensure that the model’s data is processed quickly and efficiently.
- Staffing: Running and maintaining large language models often requires a team of researchers, developers, and engineers with specialized knowledge and expertise, which can add to the overall cost.
Overall, the cost of hardware and infrastructure varies depending on the size and complexity of the language model and the amount of data used to train it. However, it is safe to say that these costs can run into the millions of dollars for the largest and most powerful models.
How current is the data?
As training the AI takes so much resource and OpenAI is a start-up, I wondered how up to date ChatGPT’s data set is. I’ve experienced quite a few exchanges where the results are URLs and they return a 404. The data felt old. Sam Altman is clear about the data’s accuracy, he’s even tweeted about it:
“ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness. It’s a mistake to be relying on it for anything important right now. it’s a preview of progress; we have lots of work to do on robustness and truthfulness.”
I wanted to know how aware of its own limitations ChatGPT is. So I asked.
What is the most recent date on the data set you were trained on?
As an AI language model, my training data consists of a diverse set of text documents and web pages that have been collected from various sources over time. The exact date range of the data used to train me may vary depending on the specific version of the model in use. However, my training data was last updated in September 2021, which means my knowledge cutoff date is also September 2021. Any events, developments, or information that have occurred after that date may not be reflected in my responses or knowledge.
How worried are you that people rely on the results of exchanges with you, but your data was last updated in September 2021?
As an AI language model, my responses are based on the information and patterns I learned from my training data, which includes a vast amount of text data from various sources. While my training data was last updated in September 2021, I still have access to a vast amount of information and knowledge that can be useful to users.
However, it’s essential to note that the world is constantly changing, and new information becomes available every day. As a language model, I cannot guarantee that my responses are always the most up-to-date or accurate, particularly for information that has changed significantly since my training data was last updated. It’s always a good idea to verify any information I provide with additional sources or consult a human expert to ensure that you have the most accurate and up-to-date information.
Overall, while I strive to provide accurate and helpful responses to users, I recommend using my responses as a starting point for further research and inquiry, particularly for time-sensitive or critical information.
Where does the majority of your training data come from?
The majority of my training data comes from a diverse set of text sources that have been collected from various websites, documents, and other text sources on the internet. The specific sources and distribution of the data vary depending on the version of the model being used, but typically, my training data includes a large corpus of general text from a variety of sources, such as books, web pages, news articles, and scientific papers.
My training data is also pre-processed and filtered to ensure its quality and relevance. The data is often cleaned and standardized to remove noise and ensure consistency, and it is also frequently filtered to remove any sensitive or inappropriate content.
It’s important to note that while my training data is designed to be as diverse and representative as possible, there may still be some biases or gaps in my knowledge. As an AI language model, I strive to be as helpful and informative as possible, but it’s always a good idea to verify any information I provide with additional sources or consult a human expert to ensure that you have the most accurate and up-to-date information.
It’s early days, but watch this space!
My conclusion? It’s early days for this technology. There’s clearly an end-user appetite for it, But given the costs the pioneering developers are incurring in development and operations – up to ten times that of ordinary search – the results need to be at least 10x better. Right now, I don’t think they are. But, we all know that digital moves at the speed of light, so watch this space!