Managing technology teams should be straightforward: Set clear expectations, provide the right tools and environment, and measure your results. 

Dwayne Forde believes more founders should manage AI the same way.

“At times it feels like there’s no clear feedback or a sense of progress when working with this evolving new technology.”

As the co-founder and CTO of Mantle, a next-generation equity management platform built for modern founders, operators, and law firms, Forde talks constantly with company leaders about the potential applications of AI.

He believes many tech leaders are still struggling to effectively manage the technology’s use by their own internal teams. And so he decided to help.

Forde recently released a series of blog posts on the Mantle site designed to offer step-by-step training for tech teams using Large Learning Models.

Part one of his instructional series, “Working with AI: Understanding the Art of the Prompt,” explains the purpose and calibration of LLM settings, and is designed to help startup teams produce better results through the use of AI.

“I’ve worked with LLMs for years, now reliable patterns are starting to emerge but they are still a black box,” said Forde. “At times it feels like there’s no clear feedback or a sense of progress when working with this evolving new technology.”

An engineer and trusted industry figure with more than 20 years of experience with companies including VMware and Xtreme Labs, Forde has been interacting with LLMs for years. He hopes his insights can serve as a handy reference point for others.

“I’ve captured my current experience building production systems with the assistance of LLMs and distilled it for the community at large,” he said. “We really value the time of our employees and I know other founders do too, so how can we use AI to accelerate cycles without a reduction in uncertainty & quality?”

Extra help, right now

AI is supposed to be a time saver. 

But Forde says many technology companies have still not fully embraced its utility in moving projects forward.

When faced with new work, startups still tend to expand or reassign their engineering teams. But Forde thinks more companies should explore using AI to reduce the task’s timeline or the number of people assigned to the work.

“We need to start thinking: ‘How can we use AI to augment these tasks?” he said.

Forde believes many companies are deterred by the idea that they will have to train an LLM to produce the work they want, a process that can seem long and overwhelming.

According to a study by PWC, only one in 10 Canadians are using AI daily or weekly, and 50 percent have never used it at all. Almost half of employers (48 percent) believe their employees are not at all prepared to use AI or GenAI.

But Forde’s instructional guide to LLMs is designed to demystify the tool and provide a simpler approach that can start introducing results today.

“What we’re doing is taming LLMs,” he said. “This is about educating folks on the best ways to prompt/interact so they can get the desired results.”

Set yourself up for success

Most people start to engage with an LLM by asking it a question or giving it a prompt.

Forde’s blog encourages technology teams to take a step back.

Each LLM offered through an API comes with settings, he points out, which can be adjusted to establish the right parameters for a particular response.

“The ability to effectively harness the power of LLMs hinges on understanding the delicate interplay between prompts, settings, and the underlying mechanisms that govern them,” Forde noted.

To break it down even further, Forde’s blog post explains that the settings of an LLM determine how creative the model is going to be. If you want the model to write a poem, for example, the settings should be calibrated for maximum experimentation. If you want a list or more detailed output, the settings should be calibrated for accuracy.

Forde’s posts offer a detailed, step-by-step guide for teams to understand and experiment with LLM settings to establish the desired balance of style, consistency and accuracy before asking them to produce any work.

He presents readers with different scenarios, from asking an LLM to write a poem to asking for the title of an article.

“For writing a poem, I want an LLM to be very creative,” Forde explains. “If we are thinking about a task like asking for the title of an article, there are often exact outcomes that you are aiming to generate.”

Tame don’t train

The settings that guide an LLM’s response are called Temperature, Top P, and Top K, all of which come calibrated to default settings. Turning these dials can change the outcomes that are produced, just like turning the dials on a stereo used to produce different frequencies.

In this case, the settings affect the randomness or specificity of an LLM’s responses. Depending on your task, Forde recommends tailoring settings to be concise and deterministic or exploratory and creative.

“The Temperature setting determines how much randomness is in the considered tokens,” said Forde. “Top P is the setting that determines how many token/word choices the LLM considers, and Top K is how many token/word options the LLM looks at for each choice.”

Once users understand how settings and prompts affect each other, Forde said it is much easier to get the most out of LLM interactions.

A temperature of 0 can produce more literal interpretations, while a temperature of 1 can produce more verbose and natural language responses, and a temperature of 2 can elicit longer summaries where the messaging can start going off track.

In other words, higher temperatures are good for creative tasks like asking AI to write a haiku, while lower temperatures are beneficial for succinct tasks, like asking for the title of an article.

To get a glimpse of how this works in practice, Forde’s blog demonstrates what happens when he asks an LLM to prepare a list of the top 10 programming languages.

The results shift every time he adjusts the temperature settings. 

“It will be up to you to decide what settings will work for you for your task and the LLM you’re using,” said Forde.

By understanding how LLM settings work, Forde believes technology teams can more effectively unlock their power and get the most out of the tools.

“LLMs and the layers of technologies surrounding them are getting more powerful by the day. Everyone should be thinking about how to incorporate them into their workflows.”

For more detailed information about how to understand LLM prompts, read Forde’s blog post here.

All images provided by Mantle.

The post How to tame a large language model first appeared on BetaKit.