context regression tests to ensure that changes to supplied context do not silently alter system behaviour.

Rationale

Research has found that the performance of large language models (LLMs) is highly sensitive to choices such as the ordering of examples, the quality of demonstration labels, and even small variations in phrasing. In some cases, reordering examples in a prompt produced accuracy shifts of more than 40 percent.

In-context learning

A model's ability

1.Previous
3.Next