Orca, a new language model in the field of generative AI, has emerged as a strong competitor to ChatGPT. With around 13 billion parameters, Orca has surpassed its competitors in various evaluations, including the Big Bench Hard (BBH) challenge and esteemed academic exams such as SAT, LSAT, GRE, and GMAT.
According to the presented paper, Orca has outperformed a wide range of foundational models including OpenAI ChatGPT in the Vicuna evaluation set. Orca’s performance in the BBH benchmark is remarkable as it has outperformed instruction-tuned models by a margin of over 113%. Similarly, in the AGIEval benchmark, Orca has showcased its superiority by delivering a 42% improvement over the previous standard.
What sets Orca apart is its ability to learn progressively from elaborate explanation traces, including step-by-step thought processes derived from the advanced GPT-4 model. This extraordinary approach to learning has enabled Orca to acquire previously untapped skills, enhancing its capabilities.
To evaluate Orca’s proficiency in various areas such as writing, comprehension, analytical skills, mathematical prowess, logical reasoning, and safety considerations, a team of researchers carried out an extensive array of tests.
Orca is equipped to handle zero-shot tasks across multiple benchmarks, and its performance in these tasks reflects its high level of proficiency. Its impressive skill set and exceptional performance make Orca a formidable competitor in the field of AI language models.
