The pace of AI certainly isn’t slowing down, but
where will it be by the end of the year?
Well we’ve put together nine trends that we expect to merge throughout the year.
Some of them are broad and high level, some are a bit more technical, so let’s get into them.
Oh, and if you’ve stumbled across this article in 2025, let us know how we did.
Trend number one
This is the year of the reality check. It is the year of more realistic expectations. When generative AI first hit mass awareness, it
was met with breathless news coverage, everyone was messing around with chat GBT, Dali and the like, and now the dust has settled, we’re starting to develop a more refined understanding of what AI powered solutions can do. Now Many generative AI tools are now being implemented as integrated elements, rather than standalone chatbots and the like. They enhance and compliment existing tools
rather than revolutionize or replace them. So think co-pilot features in Microsoft Office or generative fill in Adobe Photoshop.
And embedding AI into everyday workflows like these
helps us to better understand what generative AI can and cannot do in its current form. And one area generative AI is really extending its capabilities. Trend Two That is in multi-modal AI.
Now, AI multi-modal models can take multiple layers of data as input,
and we already have interdisciplinary models today, like OpenAI’s GPT-4V and Google Gemini,
that can move freely between natural language processing and computer vision tasks.
so users can for example like ask about an image and then receive a natural language answer
or they could ask out loud for instructions to let’s say repair something and receive
visual aids alongside step-by-step text instructions. New models are also bringing
video into the fold and where this really gets interesting is in how multimodal AI
allows for models to process more diverse data inputs and that expands the information
available for training and inference, for example, by ingesting data captured by video cameras
for holistic learning. So there’s lots more to come this year.
Now, trend three, that relates to smaller models.
Now, massive models, they jumpstarted the generative AI age, but they’re not without drawbacks.
According to one estimate from the University of Washington, training a single GPT-3 size model
requires the yearly electricity consumption of over a thousand households. And you might be thinking sure that’s training we
know that’s expensive but what about inference? Well a standard day of chat
GPT queries rivals the daily energy consumption of something like 33,000
households. Smaller models meanwhile are far less resource intensive. Much of
the ongoing innovation in LLMs has focused on yielding greater output from
fewer parameters. Now GPT-4, that is rumored to have around 1.76 trillion parameters, but
many open source models have seen success with model sizes in the 3 to 17 billion parameter
range, so billions instead of trillions. Now in December last year, Mistral released
Mixtral. That is a mixture of experts, or an MOE model, integrating eight neural
networks, each with 7 billion parameters.
And Mistral claims that Mixtral not only outperforms
the 70 billion parameter variant of Lama 2
on most benchmarks at six times faster influence speeds,
no less, but that it even matches or outperforms
OpenAI’s far larger GPT 3.5 on most standard benchmarks.
Smaller parameter models can be run at a lower cost
and run locally on many devices like personal laptops,
which conversely brings us to trend number four which is GPU and cloud costs.
The trend towards smaller models is pin driven as much by necessity as it is by entrepreneurial vigor.
The larger the model the higher the requirement on GPUs for training and inference.
Relatively few AI adopters maintain their own infrastructure,
so that puts upward pressure on cloud costs as providers update and optimize
their own infrastructure to meet gen.ai demand. All while everybody is scrambling to obtain the necessary GPUs to power the infrastructure. If only these models were a bit more optimized, they’d need less compute. Haha, yes that is trend
Number five, that is model optimization. Now this past year, we’ve already seen adoption
of techniques for training, tweaking, and fine tuning pre-trained models like quantization.
You know how you can reduce the file size of an audio file or a video file
just by lowering its bit rate? Well, quantization lowers the precision used to represent model data points.
For example, from 16 bit floating point to 8-bit integer to reduce memory usage and speed up inference.
Also, rather than directing directly fine-tuning billions of model parameters,
something called LoRa, or low-rank adaptation, entails freezing pre-trained model weights
and injecting trainable layers in each transformer block. And LoRa reduces the number of parameters
that need to be updated, which in turn dramatically speeds up fine-tuning
and reduces the memory needed to store model updates. So expect to see more model optimization techniques emerge this year.
Okay, let’s knock out a few more. Trend six the next one is all about custom local models.
Open source models afford the opportunity to develop powerful custom AI models.
That means trained on an organization’s proprietary data and fine-tuned for their specific needs.
Keeping AI training and inference local avoids the risk of proprietary data or
sensitive personal information being used to train closed source models or
otherwise pass through to the hands of third parties. And then using things
like RAG or Retrieval Augmented Generation to access relevant
information, rather than storing all of that information directly within the LLM itself, that helps to reduce model size.
Trend number seven, that is virtual agents. Now that goes beyond the straightforward
customer experience chat bot, because virtual agents relate to task automation,
where agents will get stuff done for you. They’ll make reservations, or they’ll complete checklist tasks,
or they’ll connect to other services. So lots more to come there.
Trend number eight, that is all about regulation. Now in December of last year,
the European Union reached provisional agreement on the Artificial Intelligence Act.
Also the role of copyrighted material in the training of AI models used for content generation
remains a hotly contested issue, so expect much more to come in the area of regulation.
And finally we’re at trend number nine which is the continuance of something
called shadow AI. What’s that? Well it’s the unofficial personal use of AI in the
workplace by employees. It’s about using gen AI without going through IT for
approval or oversight. Now in one study from Ernest and Young, 90% of
said they used AI at work. But without corporate AI policies in place, and importantly, policies that are observed,
this can lead to issues regarding security, privacy, compliance, that sort of thing.
So for example, an employee might unknowingly
seed trade secrets to a public-facing AI model that continually trains the model on user input.
Or they might use copyright-protected material to train a proprietary model,
and then that could expose the company to legal action. The dangers of generative AI rise kind of almost
in a linear line with its capabilities and that line’s going up.
With great power comes great responsibility. So there you have it,
nine important AI trends for this year.
But why nine?
Don’t these things almost always come in tens?
Well yes, yes they do and that’s your job.
