An excessive amount of duplication
Some degree of competitors and parallel improvement is wholesome for innovation, however the present scenario seems more and more wasteful. A number of organizations are constructing comparable capabilities, with every contributing an enormous carbon footprint. This redundancy turns into notably questionable when many fashions carry out equally on normal benchmarks and real-world duties.
The variations in capabilities between LLMs are sometimes refined; most excel at comparable duties equivalent to language technology, summarization, and coding. Though some fashions, like GPT-4 or Claude, might barely outperform others in benchmarks, the hole is usually incremental somewhat than revolutionary.
Most LLMs are skilled on overlapping information units, together with publicly out there web content material (Wikipedia, Widespread Crawl, books, boards, information, and so forth.). This shared basis results in similarities in information and capabilities as fashions take up the identical factual information, linguistic patterns, and biases. Variations come up from fine-tuning proprietary information units or slight architectural changes, however the core common information stays extremely redundant throughout fashions.
