DeepSeek's $124/Hour Question

A week on, DeepSeek is still in the news, because it is truly different from everything we’ve seen so far.

Open source and deployment: For starters, DeepSeek is truly open source, unlike Google or Meta’s models, which have usage restrictions and corresponding licensing models. This means that the way DeepSeek is deployed will likely differ from most LLMs' API-based models.

Cloud might be expensive: We now know that GCP, AWS and Azure aren’t charging customers to use DeepSeek like they would the other models (based on output, i.e., text generated). So, while deploying DeepSeek, you only pay for the compute you use.

This can be cheaper. However, it might be more expensive to rent cloud servers and run DeepSeek on them yourself. In the Applied AI newsletter, Jon Victor writes that “AWS, for example, charges as much as $124 per hour to rent a server that’s tailored to running AI. Running such a server continuously would cost around $90,000 per month.”

“In comparison, using the highest-quality version of Meta’s Llama 3.1 model through AWS costs $3 per 1 million tokens, a much lower commitment for companies that aren’t using AI as much or only do so sporadically.”

Time to try on-prem: No, you read that right. For compute-heavy workloads, especially for unpredictable usage patterns, on-prem implementations can be secure and cheaper.

If you’d like to see more about how this works in reality, hit me up. We’ll show you.

Unlike last week, this week had some news other than DeepSeek.

Eric Schmidt wants us all to be cautious about AI

Speaking to NPR, Schmidt pointed out that the proliferation of AI means it’s available to those who are “really really evil,” making them more dangerous than we understand. He warns us of the implications that he argues we haven’t even begun to imagine.

What struck me most was when he said, “Companies are doing what companies do. They're trying to maximize their revenue. What's missing is a social consensus of what's right and what's wrong."

[You can listen to the entire interview on NPR]

Google adjusts ethical guidelines

Speaking of right and wrong, Alphabet, Google’s parent company, overhauled its ethical guidelines to not prohibit AI for developing weapons or surveillance tools. In fact, it is now encouraging companies, governments and individuals to come together to create AI that “protects people, promotes global growth and supports national security.”

In a way, Google places you solely responsible for how you’re using AI. More LLM and cloud providers are likely to follow suit. What that means is that your AI governance mechanisms are more important than ever before!

Is the chatbot model good for all AI use cases?

So far, all generative AI has been in the form of chatbots, right? You ask an interface a question, and it responds. Product designer Daniel De Laney argues that this model is a bad UI pattern for coding tasks.

“Current AI tools pretend writing software is like having a conversation. It’s not. It’s like writing laws,” he argues.

What do you think? Is the model of questions and answers serving your users well? I’d love to hear from you.

Best,
Anshuman Pandey

P.S. DeepSeek R1 is now available on Tune Chat