Valued at $1B, Kai-Fu Lee’s LLM startup unveils open supply mannequin

Kai-Fu Lee, the pc scientist identified within the West for his bestseller AI Superpowers and in China for his bets on synthetic intelligence unicorns, has a brand new enterprise — and an amazing ambition.

In late March, Lee launched an organization known as 01.AI with the imaginative and prescient to develop a homegrown massive language mannequin for the Chinese language market. The enterprise places him in competitors with different distinguished Chinese language tech leaders, together with Sogou’s founder Wang Xiaochuan, who’ve been swiftly gathering expertise and enterprise capital to ascertain China’s equivalents of OpenAI.

“I feel necessity is the mom of innovation, and there’s clearly an enormous necessity in China,” Lee informed Information World in an interview, explaining the motive behind beginning 01.AI. “In contrast to the remainder of the world, China doesn’t have entry to OpenAI and Google as a result of these two firms didn’t make their merchandise out there in China, so I feel many doing LLM are attempting to do their half in creating an answer for a market that actually wants this.”

01.AI’s progress is a becoming reflection of the fast growth within the generative AI discipline. Seven months after its founding, the startup has launched its first mannequin, the open-source Yi-34B. The choice to introduce an open LLM as its debut product is a method to “give again” to society, stated Lee. For individuals who have felt LLaMA is a “godsend” to them, “we’ve supplied a compelling different,” he added.

As of writing, Yi-34B, which is a bilingual (English and Chinese language) base mannequin educated with 34 billion parameters and considerably smaller than different open fashions like Falcon-180B and Meta LlaMa2-70B, got here in first amongst pre-trained LLM fashions, according to a ranking by Hugging Face.

“We nonetheless consider that bigger fashions, when educated nicely, on a considerable amount of high-quality knowledge, will at all times outperform considerably smaller fashions of comparable high quality and comparable expertise, so I feel [Yi-34B] outperforming a lot bigger fashions is one thing that we don’t normally see,” stated Lee. “We really feel fairly assured as we launched fashions which are 100 billion to 400 billion over the subsequent coming 12 months, 12 months and a half, these fashions shall be dramatically higher than at the moment’s mannequin that we introduced.”

The startup’s skill to start mannequin coaching shortly is little doubt an end result of its clean fundraising, which is crucial to securing top-tier expertise and AI processors. Whereas declining to reveal how a lot 01.AI has raised, Lee stated it’s valued at $1 billion after receiving financing from Sinovation Ventures, Alibaba Cloud and different undisclosed traders.

01.AI has already grown to greater than 100 staff, over half of whom are LLM specialists from main multinational and Chinese language tech companies. Its vice chairman of expertise, as an example, is an early member of Google’s Bard, and its chief architect was a founding member of TensorFlow and labored alongside famend researchers like Jeff Dean and Samy Bengio at Google Mind. The important thing figures behind Yi-34B are Wenhao Huang, a Microsoft Analysis Asia veteran, and Ethan Dai, who held senior AI positions at Huawei and Alibaba.

Having backed over ten unicorns and venture-built seven firms via Sinovation Ventures, Lee is probably probably the most well-connected traders and entrepreneurs in China.

“It’s been, you understand, over 25 years for the reason that founding of Microsoft Analysis Asia, and every thing I’ve executed has been about getting tremendous nice expertise,” stated Lee, who launched Microsoft Analysis Asia, the U.S. big’s largest analysis middle overseas, earlier than heading Google China. Through the years, Microsoft Analysis Asia has earned the repute because the “West Level” for nurturing China’s AI entrepreneurs.

“Now, in fact, you need to pay individuals pretty, and it is advisable to be aggressive in pay, however I actually suppose that it’s additionally about individuals believing they will make a distinction and believing the corporate can succeed,” Lee added.

It’s no secret that constructing LLMs is a pricey enterprise. To maintain its cash-intensive operations, 01.AI has plans for monetization proper from the beginning. Whereas the corporate will proceed to open supply a few of its fashions, its goal is to construct a state-of-the-art proprietary mannequin that serves as a basis for a various vary of business merchandise.

“We can’t open supply every thing,” stated Lee. “We have been fairly cognizant of the truth that these massive language fashions require a variety of compute, and due to this fact, are very costly. Once we increase some huge cash, most of will probably be spent on the GPU. Provided that, we would have liked to first purchase as a lot GPU as we might, which we did.”

Like different LLM gamers in China, 01.AI has proactively stockpiled GPUs in anticipation of U.S. sanctions; it borrowed cash to purchase processors even earlier than it landed funding. Over the previous 12 months, the Biden administration has heightened restrictions on China’s entry to high-end AI chips, prompting Chinese firms to pay inflated prices for chips. The foresight was rewarded — 01.AI now has a provide that may suffice for at the very least the subsequent 12-18 months.

Apart from inflicting complications for Chinese language companies, U.S. sanctions have been a catalyst for innovation by encouraging them to optimize the usage of computing energy. “With a really high-quality infrastructure workforce, for each 1000 GPUs, we would have the ability to squeeze 2000 GPUs workload out of them,” stated Lee.

01.AI’s path to monetization hinges largely on its skill to search out product-market match for its costly AI fashions. Whereas top-notch LLM scientists are scarce, there’s no scarcity of product expertise in China.

“China’s not forward of the U.S. in LLM, however there’s little doubt China can construct higher functions than American builders largely due to the exceptional cellular web ecosystem that was constructed over the past 12 years or so,” argued Lee.

Whereas the founder gave no particulars on the providers within the pipeline, he hinted that the corporate is experimenting with ideas within the productiveness and social instructions, and he’d be “disenchanted” if 01.AI didn’t launch an app inside this calendar 12 months.

The startup’s final purpose, in accordance with Lee, is to develop into an ecosystem the place outdoors builders can construct functions simply. “The responsibility is not only to push out good analysis fashions, however much more importantly to make utility growth straightforward in order that there could be compelling functions,” he stated. “On the finish of the day. It’s an ecosystem play.” Time will inform if Lee’s AI endeavor will repay.