Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

TII_Thierry_LESTABLE_WCNC_2022_v10_Short.pdf

•
•
Thierry Lestable
Thierry Lestable

industry overview of technology (A.I, C-V2X, 5G/6G) and application (DMS/ADAS, A.V, A.R, ...) trends being deployed or coming in near future, to adress the challenge of the Decade for Road Safety: cutting deaths and injuries by 50% by 2030. Taking this opportunity to identify some 5G/B5G/6G promising features to support the vision.

1 of 59
tii.ae A.I & Telecom for Road Safety: Unlocking the Potential of Sustainable Smart Cities (SSC) Thierry LESTABLE, Ph.D Executive Director Digital Science Research Center (DSRC) Acting Executive Director, Artificial Intelligence Cross-Center Unit Source: Nvidia WCNC 2022, Austin, US
TII – Technology Innovation Institute Global Status Report on Road Safety - 2018 VRU: Vulnerable Road Users 50 Millions injured
TII – Technology Innovation Institute A.I for Road Safety → 2030 « Accelerating the United Nations Sustainable Development Goals (SDG)» Road traffic deaths and injuries to be reduced by at least 50 % from 2021 to 2030 6G to address UN’s SDG ➔ Contribute to the Quality & opportunities of human Life
TII – Technology Innovation Institute 4 Alignment of Planets 6G Timeline
TII – Technology Innovation Institute 5 6G KPIs - Purpose Built System 6G KPIs Let’s handle Mobility as Keystone of Sustainable Cognitive City
tii.ae WCNC 2022, Austin, US ADAS, DMS, OMS Safer Vehicles & Road Users ADAS: Advanced Driver Assistance System, DMS: Driver Monitoring System, OMS: Occupant Monitoring System
TII – Technology Innovation Institute A.C.E.S : 4 Mega-Trends Taking over the Automotive Industry
TII – Technology Innovation Institute Software Defined Vehicles – OEM Market trends Big Tech’s Trojan’s Horse $4.5 Billion (ADAS lead) Brand new SW Business Unit (2021) How to change the Mindset?
TII – Technology Innovation Institute Automotive Grade HW platforms: Centralized & Powerful SA8155AP SA8195AP STB « on-wheels » iMX9 GPU/NPU IVI (In-Vehicle Infotainment)
TII – Technology Innovation Institute 10 Beyond Moore’s Law thanks to Specialized Processing (GPU, NPU) NPU and GPU are ‘Must have’ – no way back 1000 x By 2025 Moore’s Law is stalling. Introduction of new architecture with GPU, then NPU enables to keep impressive computing power growth rate. Moore’s Law stalled.
TII – Technology Innovation Institute 11 Native A.I + Open Source Everything Delivering RISC-V Purpose built Architecture ARM’s Next Gen architecture & Mobility For sake of efficiency (Processing power, Energy consumption, Cost) main Vertical Markets would require a specialized ‘Killer Reference Design’ best suited to address their own strategic Key Use cases. Those architectures will emerge as optimized (a.k.a purpose built) combination of CPU/GPU/NPU. RISC-V reduces investment barrier for silicon, therefore a new ‘generation’/Ecosystem will emerge, Designing & delivering such ‘Killer’ architecture for prioritized verticals on the mid term, whilst disruption can then be achieved on long term via Neuromorphic chipsets. Native A.I :: Embedded in all system components ➔ obtain & evaluate massive amount of real-time information. / Federated learning
TII – Technology Innovation Institute 12 Neuromorphic ‘Brain-like’ architectures – 6G companions?
TII – Technology Innovation Institute Autonomous Vehicle (AV): The Ultimate Sensor Fusion Source: Waymo « Space Odissey »/ « Moonshot » with some Direct Short Terms & Affordable Enhancements for our Daily Life!
TII – Technology Innovation Institute LiDAR + Video Camera Fusion + A.I 80% of all car accidents involve Distraction (rationale for SAFE act in the USA). 25% of fatal accidents involve distraction
TII – Technology Innovation Institute Two (2) Distinctive Paths towards Autonomous Vehicle Source: Yole 2019 Automotive camera module industry has reached $3Billion in 2018. Sales are expected to reach $5.7B in revenue by 2024. The average number of « Cameras per end- product » will exceed 3 in 2024, for Automotive Market & Mobile phones. Last mile delivery Robot Taxi Shuttle
TII – Technology Innovation Institute Virtuous Circle of Connected Fleets Crashes -91% Cost reduction Police fines -50% Fines reduction Insurance -20% Insurance premium Energy -10% Consumption AfterMarket (A/M) example for Telematics
TII – Technology Innovation Institute A.I based Vision: from Innovation Labs to R&D Implementation From Simulation To implementation Mobile phone Snapdragon Benchmarking with competition Driver Monitoring (Drowsiness, Distraction…) Mobile platform Snapdragon based Passive ADAS In-depth understanding of Key technologies is critical: Credibility for an efficient Design (Choice of HW & SW platforms + integration).
TII – Technology Innovation Institute Paving the way to Digital Twin Digital Twin: stepping stone to Metaverse Vulnerable Road Users (VRU)
TII – Technology Innovation Institute DMS/ADAS Stakeholders : ‘Tip of the Iceberg’ May 2021 Acquisition of AFFECTIVA: $73 Million Feb. 2021 Sale completed: €100 Million July 28th 2021 IPO Tel Aviv Stock Exchange (TASE) DMS/OMS ADAS
TII – Technology Innovation Institute Complete In-car SDK sensing solutions Driver Sense (DMS) Cabin Sense (OMS) - Drowsiness - Distraction - Smoking - Phone - Drinking - Seat belt - Demographics - Driver intention - Profile Cabin Personalization / recognition of driver - Occupancy - Personalization - Demographics - Posture - Objects OMS: Occupant Monitoring System DMS: Driver Monitoring System
TII – Technology Innovation Institute Driver Attention Alert (DAA) :: building blocks Drowsiness Distraction Green point = Target to be followed by Driver == ‘Cat Laser Chaser’ ! Being able to track accurately the eye gaze can: (1) improve efficiency for detection of lack of attention, (2) gather insights about driver behaviour in context, (3) trigger new services
TII – Technology Innovation Institute Mobile Phone: Texting/Calling & Drinking Mobile Phone: Texting, Calling Drinking The main target cluster : young adults, since road crashes are #1 cause of death for this category, and it’s proven that distraction due to Mobile phone usage amongst them is dramatic.
TII – Technology Innovation Institute DMS/OMS :: Seat Belt detection OMS: Occupant Monitoring System DMS: Driver Monitoring System
TII – Technology Innovation Institute Child presence Detection :: using OMS building blocks Child Seat Detection Child Detection (via Age estimation) OMS: Occupant Monitoring System
TII – Technology Innovation Institute On the road to Identifying Impaired Drivers…. Health Monitoring: Heart Rate (H.R) & Breath Rate (B.R) This might require additional sensors to complement the camera.
TII – Technology Innovation Institute DMS for Face Detection Liveness (Anti-spoofing) Mainly used for Commercial Transport of hazardous Goods == brings somehow security of usage, and thus safety. E.g. only qualified/experienced & rested Drivers can be authorized to drive vehicles. Then this could be used also for Car/Environment personalized configuration (Comfort, Multimedia). DMS: Driver Monitoring System
TII – Technology Innovation Institute Vision based ADAS Road Safety Improvement: • FCW (Forward Collision Warning) • LDW (Lane Departure Warning) • PCW (Pedestrian Collision Warning ) • HMW (Headway Monitoring Warning) • FVSA (Front Vehicle Start Alarm) PCW FCW LDW In-cabin integration & HMI is critical. Avoid Distraction & False Alarms.
TII – Technology Innovation Institute Vulnerable Road Users (VRU): 54% of Road Deaths ! It’s time to focus (more) on Human! Pedestrian Cyclist Pedestrian Source: courtesy of Humanising Autonomy
TII – Technology Innovation Institute VRU* :: Prediction of pedestrian Crossing intention Alphapose YOLOv3 DeepSORT High interest to assess feasibility & TRL (Technology Readiness Level) for further integration into real world product == Differentiating Factor w.r.t Vulnerable Road Users (VRU) SUP’COM Students crossing the street, again & again… *VRU : Vulnerable Road User
TII – Technology Innovation Institute VRU :: Monocular camera based (not LiDar!) Source: courtesy of Humanising Autonomy Paving the Way to Autonomous Vehicles ROI Distance estimate
TII – Technology Innovation Institute Traffic Sign Recognition (TSR) with A/M Products (overview) For an After Market (A/M) Product, within the scope of TSR, mostly 2 types are of real interest: 1) STOP/YIELD & NO ENTRY: to generate ALARM 2) SPEED LIMITS: Intelligent Speed Assistance (ISA) TSR: Traffic Sign Recognition
TII – Technology Innovation Institute Augmented Reality (AR) AR brought to Head-Up Displays Source: courtesy of Wayray - navigation + car status information - surrounding environment, and conditions, - LBS (Location Based services), - C-V2X could warn of potentially ragged edges, approaching accidents can be identified, - road signs and route trackers to be magnified. Road Safety improvement, since the Driver doesn’t need to get his eyes off the road… However UI still require some good design to avoid Driver distraction by feeding too many information at a time…
TII – Technology Innovation Institute Video-Telematics cloud platform: DMS & ADAS Cloud Telematics platform integrating Video (DMS & ADAS) is available. Besides Telematics (position, acceleration, speed, mileage, diagnostic, …), short videos triggered by warnings and alarms from DMS/ADAS are stored in the platform. Key for Driver Coaching, insurance claims... + ’Pay as you Drive’ Models Field Qualification are still necessary
tii.ae WCNC 2022, Austin, US Regulation trends overview Towards Road Safety Improvement
TII – Technology Innovation Institute EU, USA Driver Drowsiness & ISA Driver (advanced) Distraction 2023 2025 Sudden Sickness GSR v2 Driver under influence* (impaired) Child presence Detection face recognition, Genre classification, occupant counting, object detection & classification, Driver Gaze Zone tracking (for 3D control), Health Monitoring, Body tracking 2026 Market demands for Next Gen Digital Cockpit (2026) USA (2021): bipartisan $1.2 Trillion Infrastructure Bill GSR: Global Safety Regulation, NCAP: New Car Assessment Programme ISA: Intelligent Speed Assistance features Minimum set Extended set Lack of Ambition, still very slow adoption: Will OTA change/accelerate this? Once True Software Defined Vehicle reached
TII – Technology Innovation Institute UK regulations : Direct Vision Standard (DVS) & FORS Camera sensors & Radar means more opportunity for A.I based warning system to support the Driver to take care of its surroundings, especially with VRUs. DVS: Direct Vision Standard FORS: Fleet Operator Recognition Scheme Safety Permit for Heavy Good Vehicle (HGV) to access Greater London
tii.ae WCNC 2022, Austin, US Smart/Cognitive City Road Infrastructure & Traffic Management
TII – Technology Innovation Institute The Top 10 Smart City Use Cases 2020 survey “ 55% of the world’s population lives in urban areas and this percentage is expected to rise to 68% by 2050. Additionally, freight volumes are expected to grow by 40% during this time period” traffic congestion cost U.S. commuters $305 billion in 2017 due to wasted fuel, lost time and the increased cost of transporting goods through congested areas Source: INRIX
TII – Technology Innovation Institute Smart Traffic Analysis Stereo-vision OTUS3D
TII – Technology Innovation Institute Urban Traffic Management :: A.I + 5G & C-V2X Cross-Working Group Work Item VRU: 2020 A.I combined with 5G & C-V2X is paving the way to disrupting road traffic Management services, saving and changing lives of citizens. Heterogeneous Interconnected devices, with Data privacy concerns make it a perfect playground for Federated Learning & A.I on Edge !
TII – Technology Innovation Institute C-V2X services (very brief) Overview: Keystone of nextGen Mobility Emergency First Responder priority Smart Traffic Light Blind curve: Hazard warning
TII – Technology Innovation Institute Road Side Unit (RSU) – C-V2X + LiDAR + Video & A.I Road Side Units (RSU) deployment jointly with LiDAR & Cameras to capture activities & intention of Vehicles + Pedestrian to improve safety, then prioritized emergency vehicles, and finally smooth/efficient traffic. Smart City :: Security & Safety Edge Computing (MEC) will also play a Key role within the IT infrastructure, to secure Low Latency (RTT), gathering/fusion of sensors data & processing power. MEC: Multi-Access Edge Computing
TII – Technology Innovation Institute Traffic Management :: Classifying & counting – Flow control Fixed Camera Flying Camera onboard Drone 6G Non- Terrestrial Networks (NTN) Reconfigurable Intelligent Surface (RIS)
TII – Technology Innovation Institute Autonomous Vehicle (A.V) Teleoperation Dispatch Center A “Must have” for Liability purpose? … Still in the infancy. Quite some room for improvements in terms of User/Technician Immersive experience! Needs interaction mechanisms to maintain perceptual illusions. Possible thanks to high resolution imaging and sensing, and 5G/6G wireless networks (URLLC) URLLC Multisensory Telepresence & Immersion URLLC: Ultra-Reliable Low Latency Communication
TII – Technology Innovation Institute Traffic Management :: Carpooling & Tunnel Watch Carpooling :: High Occupancy Vehicle (HOV) - Lateral & frontal detection for occupants counting - Vehicle class recognition - License plate matching - GDPR compliant (Privacy: blurring of personnal data) Tunnel Watch :: Automatic Incident Detection (AID) - High Accuracy of Incident Detection - Time saving for first responders - vehicle categories & counting per Lane, for real-time traffic informations - ➔ Safety improved within such complex environment Video Semantic Segmentation
TII – Technology Innovation Institute Prioritizing Emergency Vehicles flow :: A.I+C-V2X+(NG)eCall A.I Jointly with C-V2X and (NG)e-Call system, will save lives, by minimizing time for intervention (first responders), whilst also increasing global security of cities.
TII – Technology Innovation Institute Urban investments in road safety :: How to Prioritize? NYC – TOTAL Collisions NYC – VRU (Cyclists) Collisions Acquisition of relevant and quality Data from Road Traffic Crashes (RTC) must be a Keystone for Sustainable Urban Planning, and still unmet challenge, in spite of some progress thanks to open Data policies in some Major cities worldwide. Machine Learning + Geolocation applied to Social Media Crowdsourced Crash Report (City of Nairoby) Milusheva S. et al.
TII – Technology Innovation Institute Improving Road Safety via Infrastructure Hazards detection (e.g. Pothole) Sensor Fusion : Video + Accelerometer (IMU) + GNSS Potholes are amongst the primary Road Infrastructure Hazards jeopardizing safety of drivers (especially Motorbikes, Cycles). Thanks to latest Leading Edge technologies such as Computer Vision, fusioned with Inertial sensors, it is now possible to detect and monitor such hazards, and set-up efficient repair campaigns via predictive Maintenance.
TII – Technology Innovation Institute Potholes Detection & Mapping: Smart City Cloud Platform Extension Prioritization & Optimization of Urban investment / Road Maintenance, whilst cross-referencing with Road Traffic Crashes (RTC) Risk Management.
TII – Technology Innovation Institute Motorway Maintenance: Cost efficiency “The costs of road maintenance in the Netherlands add up to 1.6 Billion euros on an annual basis” A.I Driving footage “a team of specialized technicians to go frame by frame through the footage in order to recognize anomalies. These defects are then manually categorized before they are reported for maintenance” The past: Costly & Lengthy Now Semantic segmentation Cost efficient Maintenance
tii.ae Micro-Mobility Smarter & Safer e-Scooters, e-Bikes WCNC 2022, Austin, US
TII – Technology Innovation Institute Micro-Mobility is Key to Sustainable Smart Cities Faster growth than ride-hailing ! Rebound after Covid-19 crisis. Reduce Carbon Gaz emission
TII – Technology Innovation Institute A.I Taking Fleet e-Scooter safety ( and Business) to the next level (1/2) • Lane recognition – Is the scooter riding on the Road, the Sidewalk or the Bike lane? • Pedestrian detection • Helmet detection • Camera assisted parking (using QR code recognition in designated spaces that don’t have any GPS signal – e.g. underground car parks or ‘urban canyons’) • Smart City Insights (Surface conditions, obstructions, near misses etc) Source: Luna & Voi Sidewalk detection/Protection Parking assistance Main challenge for business growth: Sidewalk Protection
TII – Technology Innovation Institute A.I Taking Fleet e-Scooter safety ( and Business) to the next level (2/2)
TII – Technology Innovation Institute What about Consumer e-Scooters? Model XI Crash Assist A.I ADAS Camera sensor, making difference between: stop-light, stop-sign, person, car, or inanimate oject ➔ Audible & visual warnings!!.
TII – Technology Innovation Institute Qualcomm & Spoke: Bringing C-V2X to e-Bikes
TII – Technology Innovation Institute Boréal Bikes - Holoscene X (Developer Edition)
TII – Technology Innovation Institute Intelligent Helmets : A.I + A.R A.R: Augmented Reality Far beyond « Standard Helmets », however fueling Innovation towards more Safety. Potential Concerns with Distraction….likely to require quite some Feedback Experience from the Field / Real World…before any commercial launch.
tii.ae Let’s make a Difference for a Better World !! WCNC 2022, Austin, US

More Related Content

TII_Thierry_LESTABLE_WCNC_2022_v10_Short.pdf

  • 1. tii.ae A.I & Telecom for Road Safety: Unlocking the Potential of Sustainable Smart Cities (SSC) Thierry LESTABLE, Ph.D Executive Director Digital Science Research Center (DSRC) Acting Executive Director, Artificial Intelligence Cross-Center Unit Source: Nvidia WCNC 2022, Austin, US
  • 2. TII – Technology Innovation Institute Global Status Report on Road Safety - 2018 VRU: Vulnerable Road Users 50 Millions injured
  • 3. TII – Technology Innovation Institute A.I for Road Safety → 2030 « Accelerating the United Nations Sustainable Development Goals (SDG)» Road traffic deaths and injuries to be reduced by at least 50 % from 2021 to 2030 6G to address UN’s SDG ➔ Contribute to the Quality & opportunities of human Life
  • 4. TII – Technology Innovation Institute 4 Alignment of Planets 6G Timeline
  • 5. TII – Technology Innovation Institute 5 6G KPIs - Purpose Built System 6G KPIs Let’s handle Mobility as Keystone of Sustainable Cognitive City
  • 6. tii.ae WCNC 2022, Austin, US ADAS, DMS, OMS Safer Vehicles & Road Users ADAS: Advanced Driver Assistance System, DMS: Driver Monitoring System, OMS: Occupant Monitoring System
  • 7. TII – Technology Innovation Institute A.C.E.S : 4 Mega-Trends Taking over the Automotive Industry
  • 8. TII – Technology Innovation Institute Software Defined Vehicles – OEM Market trends Big Tech’s Trojan’s Horse $4.5 Billion (ADAS lead) Brand new SW Business Unit (2021) How to change the Mindset?
  • 9. TII – Technology Innovation Institute Automotive Grade HW platforms: Centralized & Powerful SA8155AP SA8195AP STB « on-wheels » iMX9 GPU/NPU IVI (In-Vehicle Infotainment)
  • 10. TII – Technology Innovation Institute 10 Beyond Moore’s Law thanks to Specialized Processing (GPU, NPU) NPU and GPU are ‘Must have’ – no way back 1000 x By 2025 Moore’s Law is stalling. Introduction of new architecture with GPU, then NPU enables to keep impressive computing power growth rate. Moore’s Law stalled.
  • 11. TII – Technology Innovation Institute 11 Native A.I + Open Source Everything Delivering RISC-V Purpose built Architecture ARM’s Next Gen architecture & Mobility For sake of efficiency (Processing power, Energy consumption, Cost) main Vertical Markets would require a specialized ‘Killer Reference Design’ best suited to address their own strategic Key Use cases. Those architectures will emerge as optimized (a.k.a purpose built) combination of CPU/GPU/NPU. RISC-V reduces investment barrier for silicon, therefore a new ‘generation’/Ecosystem will emerge, Designing & delivering such ‘Killer’ architecture for prioritized verticals on the mid term, whilst disruption can then be achieved on long term via Neuromorphic chipsets. Native A.I :: Embedded in all system components ➔ obtain & evaluate massive amount of real-time information. / Federated learning
  • 12. TII – Technology Innovation Institute 12 Neuromorphic ‘Brain-like’ architectures – 6G companions?
  • 13. TII – Technology Innovation Institute Autonomous Vehicle (AV): The Ultimate Sensor Fusion Source: Waymo « Space Odissey »/ « Moonshot » with some Direct Short Terms & Affordable Enhancements for our Daily Life!
  • 14. TII – Technology Innovation Institute LiDAR + Video Camera Fusion + A.I 80% of all car accidents involve Distraction (rationale for SAFE act in the USA). 25% of fatal accidents involve distraction
  • 15. TII – Technology Innovation Institute Two (2) Distinctive Paths towards Autonomous Vehicle Source: Yole 2019 Automotive camera module industry has reached $3Billion in 2018. Sales are expected to reach $5.7B in revenue by 2024. The average number of « Cameras per end- product » will exceed 3 in 2024, for Automotive Market & Mobile phones. Last mile delivery Robot Taxi Shuttle
  • 16. TII – Technology Innovation Institute Virtuous Circle of Connected Fleets Crashes -91% Cost reduction Police fines -50% Fines reduction Insurance -20% Insurance premium Energy -10% Consumption AfterMarket (A/M) example for Telematics
  • 17. TII – Technology Innovation Institute A.I based Vision: from Innovation Labs to R&D Implementation From Simulation To implementation Mobile phone Snapdragon Benchmarking with competition Driver Monitoring (Drowsiness, Distraction…) Mobile platform Snapdragon based Passive ADAS In-depth understanding of Key technologies is critical: Credibility for an efficient Design (Choice of HW & SW platforms + integration).
  • 18. TII – Technology Innovation Institute Paving the way to Digital Twin Digital Twin: stepping stone to Metaverse Vulnerable Road Users (VRU)
  • 19. TII – Technology Innovation Institute DMS/ADAS Stakeholders : ‘Tip of the Iceberg’ May 2021 Acquisition of AFFECTIVA: $73 Million Feb. 2021 Sale completed: €100 Million July 28th 2021 IPO Tel Aviv Stock Exchange (TASE) DMS/OMS ADAS
  • 20. TII – Technology Innovation Institute Complete In-car SDK sensing solutions Driver Sense (DMS) Cabin Sense (OMS) - Drowsiness - Distraction - Smoking - Phone - Drinking - Seat belt - Demographics - Driver intention - Profile Cabin Personalization / recognition of driver - Occupancy - Personalization - Demographics - Posture - Objects OMS: Occupant Monitoring System DMS: Driver Monitoring System
  • 21. TII – Technology Innovation Institute Driver Attention Alert (DAA) :: building blocks Drowsiness Distraction Green point = Target to be followed by Driver == ‘Cat Laser Chaser’ ! Being able to track accurately the eye gaze can: (1) improve efficiency for detection of lack of attention, (2) gather insights about driver behaviour in context, (3) trigger new services
  • 22. TII – Technology Innovation Institute Mobile Phone: Texting/Calling & Drinking Mobile Phone: Texting, Calling Drinking The main target cluster : young adults, since road crashes are #1 cause of death for this category, and it’s proven that distraction due to Mobile phone usage amongst them is dramatic.
  • 23. TII – Technology Innovation Institute DMS/OMS :: Seat Belt detection OMS: Occupant Monitoring System DMS: Driver Monitoring System
  • 24. TII – Technology Innovation Institute Child presence Detection :: using OMS building blocks Child Seat Detection Child Detection (via Age estimation) OMS: Occupant Monitoring System
  • 25. TII – Technology Innovation Institute On the road to Identifying Impaired Drivers…. Health Monitoring: Heart Rate (H.R) & Breath Rate (B.R) This might require additional sensors to complement the camera.
  • 26. TII – Technology Innovation Institute DMS for Face Detection Liveness (Anti-spoofing) Mainly used for Commercial Transport of hazardous Goods == brings somehow security of usage, and thus safety. E.g. only qualified/experienced & rested Drivers can be authorized to drive vehicles. Then this could be used also for Car/Environment personalized configuration (Comfort, Multimedia). DMS: Driver Monitoring System
  • 27. TII – Technology Innovation Institute Vision based ADAS Road Safety Improvement: • FCW (Forward Collision Warning) • LDW (Lane Departure Warning) • PCW (Pedestrian Collision Warning ) • HMW (Headway Monitoring Warning) • FVSA (Front Vehicle Start Alarm) PCW FCW LDW In-cabin integration & HMI is critical. Avoid Distraction & False Alarms.
  • 28. TII – Technology Innovation Institute Vulnerable Road Users (VRU): 54% of Road Deaths ! It’s time to focus (more) on Human! Pedestrian Cyclist Pedestrian Source: courtesy of Humanising Autonomy
  • 29. TII – Technology Innovation Institute VRU* :: Prediction of pedestrian Crossing intention Alphapose YOLOv3 DeepSORT High interest to assess feasibility & TRL (Technology Readiness Level) for further integration into real world product == Differentiating Factor w.r.t Vulnerable Road Users (VRU) SUP’COM Students crossing the street, again & again… *VRU : Vulnerable Road User
  • 30. TII – Technology Innovation Institute VRU :: Monocular camera based (not LiDar!) Source: courtesy of Humanising Autonomy Paving the Way to Autonomous Vehicles ROI Distance estimate
  • 31. TII – Technology Innovation Institute Traffic Sign Recognition (TSR) with A/M Products (overview) For an After Market (A/M) Product, within the scope of TSR, mostly 2 types are of real interest: 1) STOP/YIELD & NO ENTRY: to generate ALARM 2) SPEED LIMITS: Intelligent Speed Assistance (ISA) TSR: Traffic Sign Recognition
  • 32. TII – Technology Innovation Institute Augmented Reality (AR) AR brought to Head-Up Displays Source: courtesy of Wayray - navigation + car status information - surrounding environment, and conditions, - LBS (Location Based services), - C-V2X could warn of potentially ragged edges, approaching accidents can be identified, - road signs and route trackers to be magnified. Road Safety improvement, since the Driver doesn’t need to get his eyes off the road… However UI still require some good design to avoid Driver distraction by feeding too many information at a time…
  • 33. TII – Technology Innovation Institute Video-Telematics cloud platform: DMS & ADAS Cloud Telematics platform integrating Video (DMS & ADAS) is available. Besides Telematics (position, acceleration, speed, mileage, diagnostic, …), short videos triggered by warnings and alarms from DMS/ADAS are stored in the platform. Key for Driver Coaching, insurance claims... + ’Pay as you Drive’ Models Field Qualification are still necessary
  • 34. tii.ae WCNC 2022, Austin, US Regulation trends overview Towards Road Safety Improvement
  • 35. TII – Technology Innovation Institute EU, USA Driver Drowsiness & ISA Driver (advanced) Distraction 2023 2025 Sudden Sickness GSR v2 Driver under influence* (impaired) Child presence Detection face recognition, Genre classification, occupant counting, object detection & classification, Driver Gaze Zone tracking (for 3D control), Health Monitoring, Body tracking 2026 Market demands for Next Gen Digital Cockpit (2026) USA (2021): bipartisan $1.2 Trillion Infrastructure Bill GSR: Global Safety Regulation, NCAP: New Car Assessment Programme ISA: Intelligent Speed Assistance features Minimum set Extended set Lack of Ambition, still very slow adoption: Will OTA change/accelerate this? Once True Software Defined Vehicle reached
  • 36. TII – Technology Innovation Institute UK regulations : Direct Vision Standard (DVS) & FORS Camera sensors & Radar means more opportunity for A.I based warning system to support the Driver to take care of its surroundings, especially with VRUs. DVS: Direct Vision Standard FORS: Fleet Operator Recognition Scheme Safety Permit for Heavy Good Vehicle (HGV) to access Greater London
  • 37. tii.ae WCNC 2022, Austin, US Smart/Cognitive City Road Infrastructure & Traffic Management
  • 38. TII – Technology Innovation Institute The Top 10 Smart City Use Cases 2020 survey “ 55% of the world’s population lives in urban areas and this percentage is expected to rise to 68% by 2050. Additionally, freight volumes are expected to grow by 40% during this time period” traffic congestion cost U.S. commuters $305 billion in 2017 due to wasted fuel, lost time and the increased cost of transporting goods through congested areas Source: INRIX
  • 39. TII – Technology Innovation Institute Smart Traffic Analysis Stereo-vision OTUS3D
  • 40. TII – Technology Innovation Institute Urban Traffic Management :: A.I + 5G & C-V2X Cross-Working Group Work Item VRU: 2020 A.I combined with 5G & C-V2X is paving the way to disrupting road traffic Management services, saving and changing lives of citizens. Heterogeneous Interconnected devices, with Data privacy concerns make it a perfect playground for Federated Learning & A.I on Edge !
  • 41. TII – Technology Innovation Institute C-V2X services (very brief) Overview: Keystone of nextGen Mobility Emergency First Responder priority Smart Traffic Light Blind curve: Hazard warning
  • 42. TII – Technology Innovation Institute Road Side Unit (RSU) – C-V2X + LiDAR + Video & A.I Road Side Units (RSU) deployment jointly with LiDAR & Cameras to capture activities & intention of Vehicles + Pedestrian to improve safety, then prioritized emergency vehicles, and finally smooth/efficient traffic. Smart City :: Security & Safety Edge Computing (MEC) will also play a Key role within the IT infrastructure, to secure Low Latency (RTT), gathering/fusion of sensors data & processing power. MEC: Multi-Access Edge Computing
  • 43. TII – Technology Innovation Institute Traffic Management :: Classifying & counting – Flow control Fixed Camera Flying Camera onboard Drone 6G Non- Terrestrial Networks (NTN) Reconfigurable Intelligent Surface (RIS)
  • 44. TII – Technology Innovation Institute Autonomous Vehicle (A.V) Teleoperation Dispatch Center A “Must have” for Liability purpose? … Still in the infancy. Quite some room for improvements in terms of User/Technician Immersive experience! Needs interaction mechanisms to maintain perceptual illusions. Possible thanks to high resolution imaging and sensing, and 5G/6G wireless networks (URLLC) URLLC Multisensory Telepresence & Immersion URLLC: Ultra-Reliable Low Latency Communication
  • 45. TII – Technology Innovation Institute Traffic Management :: Carpooling & Tunnel Watch Carpooling :: High Occupancy Vehicle (HOV) - Lateral & frontal detection for occupants counting - Vehicle class recognition - License plate matching - GDPR compliant (Privacy: blurring of personnal data) Tunnel Watch :: Automatic Incident Detection (AID) - High Accuracy of Incident Detection - Time saving for first responders - vehicle categories & counting per Lane, for real-time traffic informations - ➔ Safety improved within such complex environment Video Semantic Segmentation
  • 46. TII – Technology Innovation Institute Prioritizing Emergency Vehicles flow :: A.I+C-V2X+(NG)eCall A.I Jointly with C-V2X and (NG)e-Call system, will save lives, by minimizing time for intervention (first responders), whilst also increasing global security of cities.
  • 47. TII – Technology Innovation Institute Urban investments in road safety :: How to Prioritize? NYC – TOTAL Collisions NYC – VRU (Cyclists) Collisions Acquisition of relevant and quality Data from Road Traffic Crashes (RTC) must be a Keystone for Sustainable Urban Planning, and still unmet challenge, in spite of some progress thanks to open Data policies in some Major cities worldwide. Machine Learning + Geolocation applied to Social Media Crowdsourced Crash Report (City of Nairoby) Milusheva S. et al.
  • 48. TII – Technology Innovation Institute Improving Road Safety via Infrastructure Hazards detection (e.g. Pothole) Sensor Fusion : Video + Accelerometer (IMU) + GNSS Potholes are amongst the primary Road Infrastructure Hazards jeopardizing safety of drivers (especially Motorbikes, Cycles). Thanks to latest Leading Edge technologies such as Computer Vision, fusioned with Inertial sensors, it is now possible to detect and monitor such hazards, and set-up efficient repair campaigns via predictive Maintenance.
  • 49. TII – Technology Innovation Institute Potholes Detection & Mapping: Smart City Cloud Platform Extension Prioritization & Optimization of Urban investment / Road Maintenance, whilst cross-referencing with Road Traffic Crashes (RTC) Risk Management.
  • 50. TII – Technology Innovation Institute Motorway Maintenance: Cost efficiency “The costs of road maintenance in the Netherlands add up to 1.6 Billion euros on an annual basis” A.I Driving footage “a team of specialized technicians to go frame by frame through the footage in order to recognize anomalies. These defects are then manually categorized before they are reported for maintenance” The past: Costly & Lengthy Now Semantic segmentation Cost efficient Maintenance
  • 51. tii.ae Micro-Mobility Smarter & Safer e-Scooters, e-Bikes WCNC 2022, Austin, US
  • 52. TII – Technology Innovation Institute Micro-Mobility is Key to Sustainable Smart Cities Faster growth than ride-hailing ! Rebound after Covid-19 crisis. Reduce Carbon Gaz emission
  • 53. TII – Technology Innovation Institute A.I Taking Fleet e-Scooter safety ( and Business) to the next level (1/2) • Lane recognition – Is the scooter riding on the Road, the Sidewalk or the Bike lane? • Pedestrian detection • Helmet detection • Camera assisted parking (using QR code recognition in designated spaces that don’t have any GPS signal – e.g. underground car parks or ‘urban canyons’) • Smart City Insights (Surface conditions, obstructions, near misses etc) Source: Luna & Voi Sidewalk detection/Protection Parking assistance Main challenge for business growth: Sidewalk Protection
  • 54. TII – Technology Innovation Institute A.I Taking Fleet e-Scooter safety ( and Business) to the next level (2/2)
  • 55. TII – Technology Innovation Institute What about Consumer e-Scooters? Model XI Crash Assist A.I ADAS Camera sensor, making difference between: stop-light, stop-sign, person, car, or inanimate oject ➔ Audible & visual warnings!!.
  • 56. TII – Technology Innovation Institute Qualcomm & Spoke: Bringing C-V2X to e-Bikes
  • 57. TII – Technology Innovation Institute Boréal Bikes - Holoscene X (Developer Edition)
  • 58. TII – Technology Innovation Institute Intelligent Helmets : A.I + A.R A.R: Augmented Reality Far beyond « Standard Helmets », however fueling Innovation towards more Safety. Potential Concerns with Distraction….likely to require quite some Feedback Experience from the Field / Real World…before any commercial launch.
  • 59. tii.ae Let’s make a Difference for a Better World !! WCNC 2022, Austin, US