Axis of optimization

1. Context optimization, what the model needs to know.
2. LLM optimization, how the model needs to act.

Prompt engineering

• Start with
  • Write clear instructions. The more specific you can be, the better it can perform.
  • Split complex tasks into simpler subtasks.
  • Give GPTs time to “think”.
  • Test changes systematically.
• Extend to
  • Provide reference text.
  • Use external tools.
• Good for
  • Testing and learning early.
  • When paired with evaluation, it provides a baseline and sets up further optimization.
• Not good for
  • Introducing new information.
  • Reliably replicating a complex style or method, i.e. learning a new programming language.
  • Minimizing token usage.

• After prompt generation, think about the gaps.
  • If there is a short-term memory gap where your model needs specific information, then go with RAG.
    • Finding the right content.
  • If there is a long-term memory gap where you want your model to use a specific structure and format, then go with fine-tuning.
    • Presenting the content in the right way.

Retrieval Augmented Generation

• RAG generally starts with a knowledge base, which could just be a set of documents.