Start by defining the objective for each agent or prompt. Stick to one cognitive process type per agent, such as: conceptualizing a landing page, selecting components, or generating content for specific sections.
Having clear boundaries maintains focus and clarity in your LLM interactions, aligning with the Engineering Techniques apex of the LLM Triangle Principle.
“Each step in our flow is a standalone process that must occur to achieve our task.”
For example, avoid combining different cognitive processes in the same prompt, which might yield suboptimal results. Instead, break these into separate, focused agents:
def generate_landing_page_concept(input_data: LandingPageInput) -> LandingPageConcept:
"""
Generate a landing page concept based on the input data.
This function focuses on the creative process of conceptualizing the landing page.
"""
passdef select_landing_page_components(concept: LandingPageConcept) -> List[LandingPageComponent]:
"""
Select appropriate components for the landing page based on the concept.
This function is responsible only for choosing components,
not for generating their content or layout.
"""
pass
def generate_component_content(component: LandingPageComponent, concept: LandingPageConcept) -> ComponentContent:
"""
Generate content for a specific landing page component.
This function focuses on creating appropriate content based on the component type and overall concept.
"""
pass
By defining clear boundaries for each agent, we can ensure that each step in our workflow is tailored to a specific mental task. This will improve the quality of outputs and make it easier to debug and refine.
Define clear input and output structures to reflect the objectives and create explicit data models. This practice touches on the LLM Triangle Principles‘ Engineering Techniques and Contextual Data apexes.
class LandingPageInput(BaseModel):
brand: str
product_desc: str
campaign_desc: str
cta_message: str
target_audience: str
unique_selling_points: List[str]class LandingPageConcept(BaseModel):
campaign_desc_reflection: str
campaign_motivation: str
campaign_narrative: str
campaign_title_types: List[str]
campaign_title: str
tone_and_style: List[str]
These Pydantic models define the structure of our input and output data and define clear boundaries and expectations for the agent.
Place validations to ensure the quality and moderation of the LLM outputs. Pydantic is excellent for implementing these guardrails, and we can utilize its native features for that.
class LandingPageConcept(BaseModel):
campaign_narrative: str = Field(..., min_length=50) # native validations
tone_and_style: List[str] = Field(..., min_items=2) # native validations# ...rest of the fields... #
@field_validator("campaign_narrative")
@classmethod
def validate_campaign_narrative(cls, v):
"""Validate the campaign narrative against the content policy, using another AI model."""
response = client.moderations.create(input=v)
if response.results[0].flagged:
raise ValueError("The provided text violates the content policy.")
return v
In this example, ensuring the quality of our application by defining two types of validators:
- Using Pydanitc’s
Fieldto define simple validations, such as a minimum of 2 tone/style attributes, or a minimum of 50 characters in the narrative - Using a custom
field_validatorthat ensures the generated narrative is complying with our content moderation policy (using AI)
Structure your LLM workflow to mimic human cognitive processes by breaking down complex tasks into smaller steps that follow a logical sequence. To do that, follow the SOP (Standard Operating Procedure) guiding principle of the LLM Triangle Principles.
“Without an SOP, even the most powerful LLM will fail to deliver consistently high-quality results.”
4.1 Capture hidden implicit cognition jumps
In our example, we expect the model to return LandingPageConcept as a result. By asking the model to output certain fields, we guide the LLM similar to how a human marketer or designer might approach creating a landing page concept.
class LandingPageConcept(BaseModel):
campaign_desc_reflection: str # Encourages analysis of the campaign description
campaign_motivation: str # Prompts thinking about the 'why' behind the campaign
campaign_narrative: str # Guides creation of a cohesive story for the landing page
campaign_title_types: List[str]# Promotes brainstorming different title approaches
campaign_title: str # The final decision on the title
tone_and_style: List[str] # Defines the overall feel of the landing page
The LandingPageConcept structure encourages the LLM to follow a human-like reasoning process, mirroring the subtle mental leaps (implicit cognition “jumps”) that an expert would make instinctively, just as we modeled in our SOP.
4.2 Breaking complex processes into multiple steps/agents
For complex tasks, break the process down into various steps, each handled by a separate LLM call or “agent”:
async def generate_landing_page(input_data: LandingPageInput) -> LandingPageOutput:
# Step 1: Conceptualize the campaign
concept = await generate_concept(input_data)# Step 2: Select appropriate components
selected_components = await select_components(concept)
# Step 3: Generate content for each selected component
component_contents = {
component: await generate_component_content(input_data, concept, component)
for component in selected_components
}
# Step 4: Compose the final HTML
html = await compose_html(concept, component_contents)
return LandingPageOutput(concept, selected_components, component_contents, html)
This multi-agent approach aligns with how humans tackle complex problems — by breaking them into smaller parts.
YAML is a popular human-friendly data serialization format. It’s designed to be easily readable by humans while still being easy for machines to parse — which makes it classic for LLM usage.
I found YAML is particularly effective for LLM interactions and yields much better results across different models. It focuses the token processing on valuable content rather than syntax.
YAML is also much more portable across different LLM providers and allows you to maintain a structured output format.
async def generate_component_content(input_data: LandingPageInput, concept: LandingPageConcept,component: LandingPageComponent) -> ComponentContent:
few_shots = {
LandingPageComponent.HERO: {
"input": LandingPageInput(
brand="Mustacher",
product_desc="Luxurious mustache cream for grooming and styling",
# ... rest of the input data ...
),
"concept": LandingPageConcept(
campaign_title="Celebrate Dad's Dash of Distinction",
tone_and_style=["Warm", "Slightly humorous", "Nostalgic"]
# ... rest of the concept ...
),
"output": ComponentContent(
motivation="The hero section captures attention and communicates the core value proposition.",
content={
"headline": "Honor Dad's Distinction",
"subheadline": "The Art of Mustache Care",
"cta_button": "Shop Now"
}
)
},
# Add more component examples as needed
}sys = "Craft landing page component content. Respond in YAML with motivation and content structure as shown."
messages = [{"role": "system", "content": sys}]
messages.extend([
message for example in few_shots.values() for message in [
{"role": "user", "content": to_yaml({"input": example["input"], "concept": example["concept"], "component": component.value})},
{"role": "assistant", "content": to_yaml(example["output"])}
]
])
messages.append({"role": "user", "content": to_yaml({"input": input_data, "concept": concept, "component": component.value})})
response = await client.chat.completions.create(model="gpt-4o", messages=messages)
raw_content = yaml.safe_load(sanitize_code_block(response.choices[0].message.content))
return ComponentContent(**raw_content)
Notice how we’re using few-shot examples to “show, don’t tell” the expected YAML format. This approach is more effective than explicit instructions in prompt for the output structure.
Carefully consider how to model and present data to the LLM. This tip is central to the Contextual Data apex of the LLM Triangle Principles.
“Even the most powerful model requires relevant and well-structured contextual data to shine.”
Don’t throw away all the data you have on the model. Instead, inform the model with the pieces of information that are relevant to the objective you defined.
async def select_components(concept: LandingPageConcept) -> List[LandingPageComponent]:
sys_template = jinja_env.from_string("""
Your task is to select the most appropriate components for a landing page based on the provided concept.
Choose from the following components:
{% for component in components %}
- {{ component.value }}
{% endfor %}
You MUST respond ONLY in a valid YAML list of selected components.
""")sys = sys_template.render(components=LandingPageComponent)
prompt = jinja_env.from_string("""
Campaign title: "{{ concept.campaign_title }}"
Campaign narrative: "{{ concept.campaign_narrative }}"
Tone and style attributes: {{ concept.tone_and_style | join(', ') }}
""")
messages = [{"role": "system", "content": sys}] + few_shots + [
{"role": "user", "content": prompt.render(concept=concept)}]
response = await client.chat.completions.create(model="gpt-4", messages=messages)
selected_components = yaml.safe_load(response.choices[0].message.content)
return [LandingPageComponent(component) for component in selected_components]
In this example, we’re using Jinja templates to dynamically compose our prompts. This creates focused and relevant contexts for each LLM interaction elegantly.
“Data fuels the engine of LLM-native applications. A strategic design of contextual data unlocks their true potential.”
Few-shot learning is a must-have technique in prompt engineering. Providing the LLM with relevant examples significantly improves its understanding of the task.
Notice that in both approaches we discuss below, we reuse our Pydantic models for the few-shots — this trick ensures consistency between the examples and our actual task! Unfortunately, I learned it the hard way.
6.1.1 Examples Few-Shot Learning
Take a look at the few_shots dictionary in section 5. In this approach:
Examples are added to the messages list as separate user and assistant messages, followed by the actual user input.
messages.extend([
message for example in few_shots for message in [
{"role": "user", "content": to_yaml(example["input"])},
{"role": "assistant", "content": to_yaml(example["output"])}
]
])
# then we can add the user prompt
messages.append({"role": "user", "content": to_yaml(input_data)})
By placing the examples as messages, we align with the training methodology of instruction models. It allows the model to see multiple “example interactions” before processing the user input — helping it understand the expected input-output pattern.
As your application grows, you can add more few-shots to cover more use-cases. For even more advanced applications, consider implementing dynamic few-shot selection, where the most relevant examples are chosen based on the current input.
6.1.2 Task-Specific Few-Shot Learning
This method uses examples directly related to the current task within the prompt itself. For instance, this prompt template is used for generating additional unique selling points:
Generate {{ num_points }} more unique selling points for our {{ brand }} {{ product_desc }}, following this style:
{% for point in existing_points %}
- {{ point }}
{% endfor %}
This provides targeted guidance for specific content generation tasks by including the examples directly in the prompt rather than as separate messages.
While fancy prompt engineering techniques like “Tree of Thoughts” or “Graph of Thoughts” are intriguing, especially for research, I found them quite impractical and often overkill for production. For real applications, focus on designing a proper LLM architecture(aka workflow engineering).
This extends to the use of agents in your LLM applications. It’s crucial to understand the distinction between standard agents and autonomous agents:
Agents: “Take me from A → B by doing XYZ.”
Autonomous Agents:“Take me from A → B by doing something, I don’t care how.”
While autonomous agents offer flexibility and quicker development, they can also introduce unpredictability and debugging challenges. Use autonomous agents carefully — only when the benefits clearly outweigh the potential loss of control and increased complexity.
Continuous experimentation is vital to improving your LLM-native applications. Don’t be intimidated by the idea of experiments — they can be as small as tweaking a prompt. As outlined in “Building LLM Apps: A Clear Step-by-Step Guide,” it’s crucial to establish a baseline and track improvements against it.
Like everything else in “AI,” LLM-native apps require a research and experimentation mindset.
Another great trick is to try your prompts on a weaker model than the one you aim to use in production(such as open-source 8B models) — an “okay” performing prompt on a smaller model will perform much better on a larger model.
