Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Edgeai Engr245 2021 Lessons Learned

•Download as PPTX, PDF•
•
Stanford University
Stanford University

The document discusses EdgeAI, a startup developing an AI chip with a custom machine learning accelerator and new embedded memory technologies targeting low-power, high-performance edge applications. In early stages, EdgeAI aimed to enable AI vision on battery-powered cameras but faced challenges competing with GPUs. It later found product-market fit enabling solar-powered security cameras by developing a chip that performs inference with no idle power consumption. EdgeAI will validate this approach with pilots and plans to fabricate a second silicon chip and raise seed funding to develop end-to-end prototypes.

1 of 19
EdgeAI The future of edge computing Mentor Mentor Serena DAYAL 95 total interviews Day One: Offering a custom silicon chip with new embedded memories and a custom Machine Learning accelerator targeting low-power, high-throughput, and low-latency applications. Today: Enable AI vision applications on next generation battery powered surveillance cameras. Massimo GIORDANO Kartik PRABHU Jo ZHU Weier WAN Alon DROR Picker PhD, EE Hacker PhD, EE Hustler MBA, M.Ed Designer PhD, EE Hustler MBA Dave NEWMAN
EDGE AI - THE FIRST ALL-IN-ONE AI CHIP ● In the lab, we: ○ 2 years of research and development ○ Collaboration with a foundry that offers the new memory ○ Built the first all-in-one AI chip prototype Our already fabricated AI chip Possible use cases We envisioned ● Objective: Enabling new AI applications at the edge, with a faster and lower power DNN accelerator enabled by new memory technology (Resistive RAM). Unclear purpose statement ?What does it mean? Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
WEEK 1 - EdgeAI was: ● A silicon AI accelerator with new memories for low power and high performance inference on edge devices Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 EdgeAI Security Cameras
WEEK 1 - EdgeAI was: ● A silicon AI accelerator with new memories for low power and high performance inference on edge devices Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 EdgeAI Security Cameras No need for AI accelerator (Not compute intensive) Wrong power target (0.1 mW instead of 100mW)
WEEK 1 - EdgeAI was: ● A silicon AI accelerator with new memories for low power and high performance inference on edge devices Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 EdgeAI Security Cameras Computation power much smaller than propulsion power
WEEK 1 - EdgeAI was: ● A silicon AI accelerator with new memories for low power and high performance inference on edge devices Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 EdgeAI Security Cameras High-end products can afford expensive hardware (GPUs) Needs high compute power (4k images at 60fps) - hard to compete with NVIDIA GPUs Battery life is not critical
By WEEK 2 we got confused Other embedded non-volatile memories offer similar performance Competitors already use eFLASH and MRAM eFLASH won’t scale, but TODAY has the same performance of RRAM. MRAM offers the same benefits of RRAM We FORGOT why RRAM was great... Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
“Lost in Space”: How we reached market fit! ● Radiation resilient & affordable alternative to FPGAs for AI applications in space almost Huge interest from NASA engineers: Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 Market: ● Big satellites (NASA & ESA) ● Nano satellites “A huge opportunity! We can help you make this happen. Let’s write a grant together”
“Lost in Space”: How we reached market fit! almost Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 ● Big satellites (NASA & ESA) ○ RRAM is resistant to radiation, but is still not space graded ● Nano satellites ○ Small-market, 1000 satellite/year less than 10k whole mission “I just launched into space an Intel’s Myriad vision processor” - AI expert, ESA Commercial products already sent in space Users Customers
Focusing on the value proposition AHA moment: “Who cares about RRAM?! What matter is what we do with it!” Heidi Roizen Threshold Ventures Focus on the value proposition! What is unique to RRAM, that is otherwise NOT possible? Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
Customers Need AI! “Something that consumes 10mWh/day would be amazing” - Ring “Currently I need a GPU card for every camera and most of the time nothing is happening” “AI would solve a lot of problems, but it consumes too much power” Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
Searching for Optimal Product/Market Fit ● Low-power solutions can not run large complex DNNs ● High-performance compute consumes a lot of power Low-power… but can’t do much! Prius High performance… but high power! vs. Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
Aha! RRAM Enables No Idle Power. Search for these customers! ● Great specs for AI ● No power to retain memory ● Fast wake-up Edge AI High-performance when needed 10ms 1 second Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 It goes FAST! But no power when stopped.
We Find Product/Market Fit! ● Edge AI supports 50MB DNN models, capable of waking-up and performing 1 inference in just 1 mJ Solar panel provides only 5 mW*h/day We enable AI vision on solar-powered security camera Not possible today! Higher security, Fewer false alarms No need to change batteries Happier customers! Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
MVP: smart security cameras w/o changing batteries We are really efficient in doing this 1 frame / second No person Person 10 MB model 1mJ 30 frame / second higher resolution 0.3% Face recognition 40 MB model 5 mJ Owner Unknown Less Efficient but infrequent 10% Unsafe 1000 MB model 200 mJ safe Above memory available on-chip Limited power budget Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
Aha! Software is Key! Use Existing AI IP ● Custom ML accelerator needs complex software stack (hard to accomplish) ● Instead, use an existing AI IP that offers the full software stack (Expedera) “We’ve spent a lot of time writing software for our current chip, and can’t afford to redo everything for a new chip” - Embedded systems startup Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
Quick wake-up powerful AI acceleration that consume no power when idle IP providers Only chips: OEM TSMC already collaborating IC Design Tools Software Run infrequent complex AI tasks preserving battery life. Engineering, Development Customer Customization Work with customers to enable their applications Supply Chain IP protection Provide custom features based on customer need Physical: System Integrator Radiation resistant, space-graded AI ASIC accelerators NASA, ESA Sell custom-designed silicon chips Development: MPW runs, EDA tools, IP licenses, Packagings, mask costs Develop custom silicon Contracts for AI expert consulting for co-designing specific applications Develop custom DNN accelerators AI IP with full software support Enable new AI applications on their device Product Managers Offering new AI features EE/CS Engineers Development and design Smart AI battery powered Cameras Lower-power compute Ease of integration and use
What’s Next: ● More product validation: ○ Further potential customers interviews ● Run pilots: ○ With various security cameras and doorbells companies ● Fabricate 2nd silicon chip: ○ Scheduled for October 2021 ● Raise first seed investment in Q3-Q4 ‘21 ○ Development of end-to-end prototype Contact us: Massimo Giordano - mgiordan@stanford.edu Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1

More Related Content

Edgeai Engr245 2021 Lessons Learned

  • 1. EdgeAI The future of edge computing Mentor Mentor Serena DAYAL 95 total interviews Day One: Offering a custom silicon chip with new embedded memories and a custom Machine Learning accelerator targeting low-power, high-throughput, and low-latency applications. Today: Enable AI vision applications on next generation battery powered surveillance cameras. Massimo GIORDANO Kartik PRABHU Jo ZHU Weier WAN Alon DROR Picker PhD, EE Hacker PhD, EE Hustler MBA, M.Ed Designer PhD, EE Hustler MBA Dave NEWMAN
  • 2. EDGE AI - THE FIRST ALL-IN-ONE AI CHIP ● In the lab, we: ○ 2 years of research and development ○ Collaboration with a foundry that offers the new memory ○ Built the first all-in-one AI chip prototype Our already fabricated AI chip Possible use cases We envisioned ● Objective: Enabling new AI applications at the edge, with a faster and lower power DNN accelerator enabled by new memory technology (Resistive RAM). Unclear purpose statement ?What does it mean? Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 3. WEEK 1 - EdgeAI was: ● A silicon AI accelerator with new memories for low power and high performance inference on edge devices Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 EdgeAI Security Cameras
  • 4. WEEK 1 - EdgeAI was: ● A silicon AI accelerator with new memories for low power and high performance inference on edge devices Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 EdgeAI Security Cameras No need for AI accelerator (Not compute intensive) Wrong power target (0.1 mW instead of 100mW)
  • 5. WEEK 1 - EdgeAI was: ● A silicon AI accelerator with new memories for low power and high performance inference on edge devices Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 EdgeAI Security Cameras Computation power much smaller than propulsion power
  • 6. WEEK 1 - EdgeAI was: ● A silicon AI accelerator with new memories for low power and high performance inference on edge devices Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 EdgeAI Security Cameras High-end products can afford expensive hardware (GPUs) Needs high compute power (4k images at 60fps) - hard to compete with NVIDIA GPUs Battery life is not critical
  • 7. By WEEK 2 we got confused Other embedded non-volatile memories offer similar performance Competitors already use eFLASH and MRAM eFLASH won’t scale, but TODAY has the same performance of RRAM. MRAM offers the same benefits of RRAM We FORGOT why RRAM was great... Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 8. Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 9. “Lost in Space”: How we reached market fit! ● Radiation resilient & affordable alternative to FPGAs for AI applications in space almost Huge interest from NASA engineers: Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 Market: ● Big satellites (NASA & ESA) ● Nano satellites “A huge opportunity! We can help you make this happen. Let’s write a grant together”
  • 10. “Lost in Space”: How we reached market fit! almost Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 ● Big satellites (NASA & ESA) ○ RRAM is resistant to radiation, but is still not space graded ● Nano satellites ○ Small-market, 1000 satellite/year less than 10k whole mission “I just launched into space an Intel’s Myriad vision processor” - AI expert, ESA Commercial products already sent in space Users Customers
  • 11. Focusing on the value proposition AHA moment: “Who cares about RRAM?! What matter is what we do with it!” Heidi Roizen Threshold Ventures Focus on the value proposition! What is unique to RRAM, that is otherwise NOT possible? Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 12. Customers Need AI! “Something that consumes 10mWh/day would be amazing” - Ring “Currently I need a GPU card for every camera and most of the time nothing is happening” “AI would solve a lot of problems, but it consumes too much power” Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 13. Searching for Optimal Product/Market Fit ● Low-power solutions can not run large complex DNNs ● High-performance compute consumes a lot of power Low-power… but can’t do much! Prius High performance… but high power! vs. Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 14. Aha! RRAM Enables No Idle Power. Search for these customers! ● Great specs for AI ● No power to retain memory ● Fast wake-up Edge AI High-performance when needed 10ms 1 second Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1 It goes FAST! But no power when stopped.
  • 15. We Find Product/Market Fit! ● Edge AI supports 50MB DNN models, capable of waking-up and performing 1 inference in just 1 mJ Solar panel provides only 5 mW*h/day We enable AI vision on solar-powered security camera Not possible today! Higher security, Fewer false alarms No need to change batteries Happier customers! Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 16. MVP: smart security cameras w/o changing batteries We are really efficient in doing this 1 frame / second No person Person 10 MB model 1mJ 30 frame / second higher resolution 0.3% Face recognition 40 MB model 5 mJ Owner Unknown Less Efficient but infrequent 10% Unsafe 1000 MB model 200 mJ safe Above memory available on-chip Limited power budget Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 17. Aha! Software is Key! Use Existing AI IP ● Custom ML accelerator needs complex software stack (hard to accomplish) ● Instead, use an existing AI IP that offers the full software stack (Expedera) “We’ve spent a lot of time writing software for our current chip, and can’t afford to redo everything for a new chip” - Embedded systems startup Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1
  • 18. Quick wake-up powerful AI acceleration that consume no power when idle IP providers Only chips: OEM TSMC already collaborating IC Design Tools Software Run infrequent complex AI tasks preserving battery life. Engineering, Development Customer Customization Work with customers to enable their applications Supply Chain IP protection Provide custom features based on customer need Physical: System Integrator Radiation resistant, space-graded AI ASIC accelerators NASA, ESA Sell custom-designed silicon chips Development: MPW runs, EDA tools, IP licenses, Packagings, mask costs Develop custom silicon Contracts for AI expert consulting for co-designing specific applications Develop custom DNN accelerators AI IP with full software support Enable new AI applications on their device Product Managers Offering new AI features EE/CS Engineers Development and design Smart AI battery powered Cameras Lower-power compute Ease of integration and use
  • 19. What’s Next: ● More product validation: ○ Further potential customers interviews ● Run pilots: ○ With various security cameras and doorbells companies ● Fabricate 2nd silicon chip: ○ Scheduled for October 2021 ● Raise first seed investment in Q3-Q4 ‘21 ○ Development of end-to-end prototype Contact us: Massimo Giordano - mgiordan@stanford.edu Week 0 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 The Future Week 1