Huazhe Wang

The high-end graphics cards from Nvidia, such as the H100, have catalyzed the emergence of dedicated Rent-A-GPU farms, expanding beyond traditional hyperscale providers like AWS, GCP, and Azure. Initially, supply chain limitations coupled with high demand enabled these specialized farms to purchase these cards—each costing nearly $40,000—in bulk and offer them for rent. By late 2023, rates were around $8 per hour under commitments ranging from one to three years. Today, the landscape has shifted significantly. Some of the largest Rent-A-GPU providers have adapted to changing market dynamics by offering GPU spot markets with commitments as short as 48 hours, with rates as low as $1.8 per hour or even less. This suggests that the supply chain for high-end GPUs has finally caught up with demand, potentially heralding a substantial decrease in costs not only in the rental market but also in capital expenditures for owning these GPUs. For the first time, there is downward pressure on the prices of high-end GPUs. This development is particularly promising for real-world applications of Generation AI. The significant reduction in inference costs is crucial for the viability and value creation of GenAI within enterprises. It represents the first of many indicators that we are approaching the rapid realization of game-changing use cases, transforming how businesses leverage advanced AI capabilities. At KamiwazaAI we believe the start of an 5th Industrial Revolution Enterprise #5IR is around #1trillionInferences a day or in the range of ~$3b dollars a day utilizing chat GPT4 at .003 per Inference (100 tokens). A 1000x decrease in the cost of Inference is only 18 months away or so and its time NOW to begin implementing the technology to keep up!

21

5 Comments

Most major tech companies are at Computex in Taiwan this week, and we're seeing a flood of announcements. The industry is buzzing following Jensen Huang's announcement of the next-generation Rubin GPU platform and associated Vera Arm CPUs. These will replace Grace, Hopper, and Grace Hopper as NVIDIA's scalable AI platform in 2026 and beyond. It's excellent to see NVIDIA honor another woman in science with their platform names, in this case American astronomer Vera Rubin who passed away in 2016. #Rundown #Computex #AI

1

Back in May, I was invited by the local chapter of IEEE SSCS in Santa Clara to give a talk on how to use LLM and Gen AI for high-speed design. The video has been uploaded to YouTube. It is a good summary of how we are using LLM and AI for SI. https://lnkd.in/eUy2HDhB A few months later, Eric Schmidt gave a controversial talk on where he thought AI would progress in the next few years in a Stanford computer science class. He was a little too frank on telling students to steal TikTok IPs and why he thought Google would fail. https://lnkd.in/epMKkmvv Both talks seem to completely overlap on how to create domain knowledge dominance using advanced LLM. He gave examples on taking over TikTok and in chemical engineering, I was focusing on high-speed SI designs. But it can be applied to any other hardware design disciplines.  What we are pursuing, and he seems to suggest the same approach : 1) Use LLM with deep context window to scan through large volume of knowledge (text, video or sound), in my case for SI/PI we will be scanning large volumes of design specs for design domain knowledge 2) Use LLM agents to execute task based on the ingested knowledge (context to action), in my case, we will be initiating simulations based on design spec range and look for optimal solutions 3) Repeat the above with multiple chain of thoughts and store the observation to draw a conclusion, in my case, after scanning through the design specs and obtain optimal spacing for performance, generate routing and constraint rules 4) We also add an optimization step which is necessary for engineering  5) Improve the LLM agent performance using an adversarial agent to critique the LLM agent's performance 6) We also add using fine-tuning and transfer learning to future proof the domain specific models. I apologize both are long talks ~1 hour.  I hope this will be a good starting point to discuss the use of LLM and Gen AI for SI/PI. Feedback and comments are most welcome.

13

2 Comments

Are LLMs capable of Artificial General Intelligence (AGI)? I came across this article in the NY Times: https://lnkd.in/geXXbi8B (for non-subscribers here's the paper it talks about https://lnkd.in/gP-m7sxA) The upshot is that researchers are finding strong evidence that language is a very distinct part of the human brain whose purpose is to communicate and that it is not intrinsically involved in reasoning. This gets at a very fundamental question about the use of LLMs as the basis for Artificial General Intelligence. Can a model based on language be used as the basis for AGI? Time will tell I'm sure, but when I've asked this question to researchers I know, they generally say yes. However, I'm not sure what the basis is for this belief. CAVEAT: I'm not in the AI field, nor have I delved deeply into the scholarly articles. I have a fairly rudimentary understanding of LLMs, but would be happy to learn from those who know more. One of the fundamental articles shaping my personal views on LLM based AI is Ted Chiang's excellent piece "ChatGPT is a blurry JPEG of the web" (https://lnkd.in/gmkRBe6A). For me, it captured the idea that LLMs are just summarizing what we know. To be sure, they are capable of really cool things, but are they really doing any thinking? More important are they even capable of thinking? Again, time will tell, but I'm increasingly skeptical. The kicker for me is that they have no common sense. One of the reasons we find LLMs so amusing is they crazy ways they go off the rails (aka hallucinations). I think that these are largely because there's no common sense 'boundary'. When a human's thoughts start to get carried away, we have ways of reigning them in by asking ourselves "is this actually a reasonable thought?" Indeed, it can be quite alarming to encounter people who communicate about ideas that we find to be outside the bounds of common reason. It feels like this 'framework of rationality' is very important but I don't think it's something you can just use as guard rails. It seems like you need some kind of other AI to be able to generate that framework. What do you think? Are LLMs capable of AGI? Or are they like our own language processing center, just a small piece of a much larger 'intelligence framework'? [Edited to use AGI instead of GenAI so as not to be confused with Generative AI] [Edit: Thanks to Tareq Alkhatib here is a direct link to the paper in question: https://lnkd.in/g9b7Q66c]

41

21 Comments

According to Eric Schmidt, in the next year or so, the union of the large context window, agents, and text-to-action, when delivered at scale, is going to have an impact on the world that no one understands yet. Listen to the fascinating discussion in the "The Age of AI, Eric Schmidt" youtube video - https://lnkd.in/dFTKgBVY

52

1 Comment