😺 AI is expensive

PLUS: Micro-expensive

Welcome, humans.

ChatGPT-4 has become noticeably less helpful for us over the past six months. The honeymoon phase in the Spring was exhilarating, but it feels like that fiery spark has gradually faded. Anyone have advice for rekindling that passion? More date nights? Perhaps a bouquet of flowers? SOS!

Here’s what you need to know about AI today:

Powering AI demands an extraordinary amount of energy.
The financials for Anthropic? High costs, low margins.
Today’s AI is not economical enough to automate most jobs.
Google's Chrome is getting smarter with new AI integrations.

AI is costing a fortune to run…

AI is super freaking costly to use, and it’s causing big problems.

Why? Well, every time you ask ChatGPT something, it needs to perform new calculations behind the scenes, which requires insanely expensive GPUs, servers, and infrastructure.

“Using [ChatGPT] to summarize an email is like getting a Lamborghini to deliver a pizza.”

WSJ

We're talking a Lamborghini Huracán Super Trofeo Omologato level of extravagance.

Sam Altman once said that training GPT-4 cost them at least $100M and that future AI will require “an energy breakthrough”.
Processing a single ChatGPT question could cost OpenAI 10x more than a standard Google Search.
“The amount of electricity required to power the world’s data centers could jump by 50% by 2027, thanks to AI alone.”

And even though we’re still in an AI wonderland where it rains spaghetti and $100M Series As, costs are starting to weigh heavy on Big Tech and their customers.

Take Anthropic, for example. Despite billions in funding, their server expenses have slashed their gross profits well below typical software company standards.

Yikes.

Oh, and BTW, when AI builders face higher costs, so do we, the end-users. In fact, a recent MIT study found that computer vision AI is still far too pricey to replace human workers in 99.6% of jobs.

But don't get too comfy—automation is still creeping up (keep reading).

FROM OUR PARTNERS

Mixtral on OctoAI: 12x lower costs, 30% faster speeds vs GPT-4-Turbo

Some of the savviest builders in AI are ditching OpenAI for OctoML, and here’s why:

Building products on top of OpenAI can be expensive AF, and if you’re not throwing big bucks at them, latency and scale become problems.

Enter OctoML—the fastest, most affordable option to run Mixtral 8x7B (an open-source model that outperforms GPT-3.5).

Customers switching from GPT-4 Turbo see comparable quality, 12x savings, and 30% lower latency.

Try OctoAI Text Generation and snag $10 in free credits when you sign up today.

Costs will come down, and so will prices.

Yes, there are significant uncertainties facing many businesses we talk to about implementing AI, with cost being a major hurdle.

(quick pro tip for decision-makers: start with OpenAI for rapid prototyping, then consider hosting your models once you've got the hang of it).

As AI matures, though, we’ll see costs come down (wink wink Moore’s Law).

Per The Information, here's a glimpse at some of the groundbreaking developments already underway:

Older AI chips, like Nvidia’s A100, are seeing price reductions.
Newer chips, such as the H100s, are accelerating model training.
Cost-effective training techniques like quantization and Mixture of Experts are gaining traction.

Just yesterday, reports surfaced that Microsoft is forming a new team to develop cheaper AI models, just three months after it announced an effort to create cost-effective AI chips.

Around the Horn.

Google is upgrading Chrome and Ad campaigns with new generative AI features.
Here’s how AI improved teaching and learning in Harvard’s CS50 class.
Here’s a collection of FREE machine learning YouTube courses.
Three more Google DeepMind engineers left to launch their own AI startup.

Wednesday Wirings.

Blueshift, an AI tool that automates marketing campaigns, raised $40M.
Vicarius, a cybersecurity startup building AI-powered vulnerability detection tools, raised $30M.
Qdrant, an open-source vector database, raised $28M.
Recraft, an AI graphic design tool, raised $11M.
Artisse, an AI photo creation app, raised $6.7M.

We curated the top 27 tools you need for work in our “Top Tools For Business”!