Here’s the revised list including the Proactive Chain-of-Thought Prompting (ProCoT) from recent research:

1. Zero-Shot Prompting:

• Description: The model is given a task without any prior examples.
• Goal: To provide a concise explanation or answer based on general knowledge.
• Example 1: « Describe the process of photosynthesis. »
• Example 2: « Explain the significance of the Treaty of Versailles. »

1. Few-Shot Prompting:

• Description: The model is provided with a few examples to better understand the task.
• Goal: To generate responses that follow the pattern shown in the examples.
• Example 1: « Text: Neil Armstrong landed on the moon in 1969. Event: Moon Landing, 1969. Text: The first iPhone was released in 2007. Event: »
• Example 2: « Problem: If I have 5 apples and give away 2, how many do I have left? Solution: 3 apples. Problem: If a train travels 100 miles in 1 hour, how far will it travel in 3 hours? Solution: »

1. Chain of Thought (CoT) Prompting:

• Description: Guides the model to decompose the problem into intermediate steps before providing the final answer.
• Goal: To facilitate complex problem-solving by breaking down the process.
• Example 1: « To solve the math problem ‘8 divided by 2(2+2)’, let’s think step by step. »
• Example 2: « To determine the capital of France, let’s consider the major cities in France and identify the one that is the political and cultural center. »

1. Proactive Chain-of-Thought Prompting (ProCoT):

• Description: Involves planning and taking initiative towards a conversational goal, enhancing proactivity in dialogue systems.
• Goal: To develop a proactive and strategic response to a situation or problem.
• Example 1: « A customer complains about a late delivery. Let’s plan the steps to address this issue. »
• Example 2: « To decide on a marketing strategy for a new product, let’s first analyze the target market and then determine the most effective approach. »

1. Contrastive Chain of Thoughts:

• Description: Uses contrasting explanations (correct and incorrect) to enhance understanding.
• Goal: To clarify a concept by differentiating correct information from misconceptions.
• Example 1: « Correct: Plants release oxygen during photosynthesis. Incorrect: Plants consume oxygen during photosynthesis. Now, explain photosynthesis. »
• Example 2: « Correct: The Earth orbits the Sun. Incorrect: The Sun orbits the Earth. Describe the solar system’s structure. »

1. Self-Reflection Prompting:

• Description: Adds a verification layer to the generated response to detect errors or inconsistencies.
• Goal: To ensure accuracy and completeness in the response.
• Example 1: « After summarizing the article, review the summary for accuracy and completeness. »
• Example 2: « Translate this text and then evaluate the translation for any possible errors or improvements. »

1. Decomposed Prompting:

• Description: Breaks down a complex prompt into sub-prompts, each addressing a part of the overall task.
• Goal: To tackle complex, multi-faceted tasks by addressing each aspect separately.
• Example 1: « Break down the process of cellular respiration into its major stages and then explain each stage. »
• Example 2: « Divide the history of the Roman Empire into key periods and summarize the main events of each period. »

1. Self Consistency Prompting:

• Description: Generates multiple responses to the same prompt and selects the most consistent or frequent answer.
• Goal: To achieve a more reliable and consistent answer by considering multiple possibilities.
• Example 1: « Generate multiple explanations for why the sky is blue, and then identify the most accurate explanation. »
• Example 2: « Provide several predictions for the future of renewable energy, and then choose the most likely scenario. »

1. System 2 Attention Prompting:

• Description: Focuses on extracting relevant information from a text while ignoring biases and irrelevant details.
• Goal: To respond to queries based solely on pertinent and factual information.
• Example 1: « Extract factual information from this political speech, then analyze the policy proposals based on these facts. »
• Example 2: « Identify key scientific facts in this article about global warming, then summarize the implications of these facts. »

1. Simulation Theory of Mind Prompting:
   • Description: Involves taking the perspective of a specific persona or knowledge base to answer questions.
   • – Goal: To provide insights or answers from a unique or specific perspective.
   • – Example 1: « Imagine you’re an AI from the future. What information do you have about advanced technology? Based on this, predict future tech trends. »
   • – Example 2: « Assume you are a historian from the 1800s. What is your understanding of industrialization? Explain its impact based on that perspective. »
   • These methods showcase the versatility of LLMs in handling various tasks and queries, each technique offering unique advantages for specific types of problems.

Merging all the prompting methods into a single approach is complex but conceivable, especially for multifaceted or highly nuanced tasks. The goal would be to leverage the strengths of each method for different aspects of the task. Here are 10 samples illustrating how multiple methods can be integrated:

1. Integrated Sample 1:

• Prompt: « Imagine you’re an AI analyzing climate data. First, identify key patterns in this climate report (System 2 Attention). Then, using a Chain of Thought approach, evaluate how these patterns indicate global warming trends. Finally, compare these findings with historical climate data (Contrastive Chain of Thoughts) and provide a summarized prognosis (Decomposed Prompting). »

1. Integrated Sample 2:

• Prompt: « As a historian from the 1800s (Simulation Theory of Mind), review this modern article on the Industrial Revolution. Extract factual data (System 2 Attention), then critically analyze the differences in perspectives (Contrastive Chain of Thoughts). Conclude with a few-shot prompted summary comparing past and present views. »

1. Integrated Sample 3:

• Prompt: « To solve this complex math problem, let’s break it down (CoT). Consider alternative methods to solve each step (Self Consistency). Reflect on each solution’s validity (Self-Reflection). Finally, provide a concise explanation of the solution process (Decomposed Prompting). »

1. Integrated Sample 4:

• Prompt: « First, read these contrasting opinions on renewable energy (Contrastive Chain of Thoughts). Analyze their factual accuracy (System 2 Attention) and reasoning (CoT). Then, synthesize a proactive plan to increase renewable energy adoption, considering economic and environmental factors (ProCoT). »

1. Integrated Sample 5:

• Prompt: « Imagine you’re a 22nd-century environmentalist (Simulation Theory of Mind). Review these historical documents on deforestation (System 2 Attention), identify key changes over time (Contrastive Chain of Thoughts), and predict future trends (Decomposed Prompting). Summarize your findings with potential solutions (Few-Shot Prompting). »

1. Integrated Sample 6:

• Prompt: « As a medical AI, analyze these patient reports (System 2 Attention), identify symptoms (CoT), and diagnose (Decomposed Prompting). Compare your diagnosis with similar historical cases (Contrastive Chain of Thoughts) and suggest a treatment plan (ProCoT). »

1. Integrated Sample 7:

• Prompt: « Read this debate on AI ethics (System 2 Attention). Identify the main arguments (CoT), compare them with established ethical standards (Contrastive Chain of Thoughts), and propose a balanced ethical guideline for AI development (ProCoT). »

1. Integrated Sample 8:

• Prompt: « Review this economic report as a 20th-century economist (Simulation Theory of Mind), extract key economic indicators (System 2 Attention), compare with current economic data (Contrastive Chain of Thoughts), and predict future economic trends (Decomposed Prompting). »

1. Integrated Sample 9:

• Prompt: « Analyze this new technology from a future perspective (Simulation Theory of Mind). Break down its potential impacts (CoT), compare with past technological advancements (Contrastive Chain of Thoughts), and propose future applications (ProCoT). »

1. Integrated Sample 10:
   • Prompt: « Examine this legal case file (System 2 Attention). Identify key legal precedents (CoT), compare with similar cases (Contrastive Chain of Thoughts), and predict the outcome (Self Consistency). Finally, draft a judgment summary (Decomposed Prompting). »

In each of these samples, multiple prompting methods are combined to tackle different parts of the task, creating a comprehensive approach that leverages the strengths of each technique. This integrated approach can be particularly useful for complex tasks requiring nuanced analysis, synthesis of different viewpoints, and strategic planning.

• Introduces NoGAN, a faster alternative to traditional tabular data synthetization.
• Runs 1000x quicker than GAN, delivering superior results with a new, sophisticated evaluation metric.
• A significant cost reducer, minimizing cloud/GPU time and training time.
• Replaces manual fine-tuning parameters with auto-tuning.
• Now available as open-source software.
• Real-life case studies: synthetization in <5 seconds (compared to 10 minutes with GAN).
• Produces higher quality results, confirmed via cross-validation.
• Fast implementation enables automatic, efficient hyperparameter fine-tuning.
• Future improvements discussed: speed enhancement, data faithfulness, auto-tuning, Gaussian NoGAN, and broader applications.

https://docs.google.com/presentation/d/1kDlAhS8yh_-Yu19ICxFk0Hxfq3ZXc4iy/mobilepresent?slide=id.p1

Nixtla’s TimeGPT is a generative pre-trained forecasting model for time series data. TimeGPT can produce accurate forecasts for new time series without training, using only historical values as inputs. TimeGPT can be used across a plethora of tasks including demand forecasting, anomaly detection, financial forecasting, and more.

The TimeGPT model “reads” time series data much like the way humans read a sentence – from left to right. It looks at windows of past data, which we can think of as “tokens”, and predicts what comes next. This prediction is based on patterns the model identifies in past data and extrapolates into the future.

The API provides an interface to TimeGPT, allowing users to leverage its forecasting capabilities to predict future events. TimeGPT can also be used for other time series-related tasks, such as what-if scenarios, anomaly detection, and more.

https://nixtla.github.io/nixtla/docs/getting-started/getting_started_short.html

https://youtube.com/watch?v=POBcYr0sbcg&si=VD5odGEisjt2fzUi

DSPy is the framework for solving advanced tasks with language models (LMs) and retrieval models (RMs). DSPy unifies techniques for prompting and fine-tuning LMs — and approaches for reasoning, self-improvement, and augmentation with retrieval and tools. All of these are expressed through modules that compose and learn.

To make this possible:

• DSPy provides composable and declarative modules for instructing LMs in a familiar Pythonic syntax. It upgrades « prompting techniques » like chain-of-thought and self-reflection from hand-adapted string manipulation tricks into truly modular generalized operations that learn to adapt to your task.
• DSPy introduces an automatic compiler that teaches LMs how to conduct the declarative steps in your program. Specifically, the DSPy compiler will internally trace your program and then craft high-quality prompts for large LMs (or train automatic finetunes for small LMs) to teach them the steps of your task.

The DSPy compiler bootstraps prompts and finetunes from minimal data without needing manual labels for the intermediate steps in your program. Instead of brittle « prompt engineering » with hacky string manipulation, you can explore a systematic space of modular and trainable pieces.

For complex tasks, DSPy can routinely teach powerful models like GPT-3.5 and local models like T5-base or Llama2-13b to be much more reliable at tasks. DSPy will compile the same programinto different few-shot prompts and/or finetunes for each LM.

If you want to see DSPy in action, open our intro tutorial notebook.

Graph RAG is an Knowledge-enabled RAG approach to retrieve information from Knowledge Graph on given task. Typically, this is to build context based on entities’ SubGraph related to the task.

GraphStore backed RAG vs VectorStore RAG

As we compared how Graph RAG helps in some use cases in this tutorial, it’s shown Knowledge Graph as the unique format of information could mitigate several issues caused by the nature of the “split and embedding” RAG approach.

Graph RAG

https://www.philschmid.de/gptq-llama

• Hugging Face Optimum team collaborated with AutoGPTQ library for a simple API to apply GPTQ quantization on language models.
• GPTQ quantization allows open LLMs to 8, 4, 3, or 2 bits, enabling them to run on smaller hardware with minimal performance loss.
• The blog covers:

1. Setting up the development environment.
2. Preparing the quantization dataset.
3. Loading and quantizing the model.
4. Testing performance and inference speed.
5. Bonus: Running inference with text generation.

• GPTQ’s purpose is explained before diving into the tutorial.

• Large Language Models (LLMs) and prompt-based heuristics are being used for off-the-shelf solutions to various NLP problems.
• LLM-based few-shot methods have shown promise but lag in Named Entity Recognition (NER) compared to other methods.
• « PromptNER » is introduced as a new algorithm for few-shot and cross-domain NER.
• PromptNER needs entity definitions and few-shot examples for a new NER task.
• PromptNER uses LLM to generate potential entities and explanations for their compatibility with entity type definitions.
• PromptNER achieves state-of-the-art performance in few-shot NER on ConLL, GENIA, and FewNERD datasets.
• It also outperforms previous methods in Cross Domain NER, setting new records on 3 out of 5 CrossNER domains with an average F1 gain of 3%.

https://arxiv.org/pdf/2305.15444.pdf

https://github.com/promptslab/Promptify

https://cameronrwolfe.substack.com/p/the-history-of-open-source-llms-better

• Value of Open-source LLM Research: Aims to democratize influential technology; despite initial struggles and criticism, open-source LLMs gained popularity and significance.
• Early Challenges: Initial open-source LLMs performed poorly and faced criticism, posing difficulties for advancement.
• Transformative Research Line: Focuses on enhancing open-source LLMs, leading to high-performing pre-trained models accessible to all.
• Significance of High-Performing Models: Creation of powerful, cost-effective pre-trained LLMs revolutionized research accessibility.
• Series Overview: Part two of a three-part series on open-source LLM history. The first part explored initial open-source LLM attempts.
• Study Focus: This overview delves into the most popular open-source base models, emphasizing pre-trained models not yet fine-tuned or aligned.
• Future Exploration: Subsequent installment will discuss fine-tuning and alignment of models for diverse practical applications.

https://cameronrwolfe.substack.com/p/practical-prompt-engineering-part

• Prompt engineering: An empirical science focused on optimizing LLM (Large Language Model) performance through various prompting strategies.
• Aims to understand prompting mechanics and employs techniques to enhance LLM capabilities.
• Zero/few-shot learning: A fundamental technique where LLMs perform tasks with minimal or no training examples, showcasing their remarkable adaptability.
• Instruction prompting: Another vital technique involving explicit instructions in prompts to guide LLM behavior.
• Overview intends to impart practical insights and strategies for effective prompt engineering and LLM utilization.
• Provides actionable tricks and takeaways for prompt engineers and LLM practitioners to enhance their effectiveness.

https://cameronrwolfe.substack.com/p/the-history-of-open-source-llms-early

• Language modeling research traces back to models like GPT, GPT-2, and pre-transformer methods such as ULMFit.
• GPT-3’s proposal marked the initial rise in popularity by showcasing impressive few-shot learning through self-supervised pre-training and in-context learning.
• The recognition of GPT-3 led to the creation of various large language models (LLMs), including InstructGPT and ChatGPT, sparking widespread interest in generative AI.
• Early LLMs often remained closed source, limiting researchers’ understanding and improvement of their workings.
• Open-source variants of popular language models began to emerge gradually, although they initially lagged behind proprietary models in performance.
• These early open-source models laid the groundwork for increased transparency in LLM research and inspired the development of more potent subsequent models like Falcon and LLaMA-21.
• The overview is part of a three-part series that delves into the history of open-source language models, exploring their beginnings, recent developments, and the application of imitation and alignment techniques to enhance their performance.