Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Alexandr Golyak, Nikolay Chertkov "Automotive Testing vs Test Automatio"

•
•

The document discusses automotive testing versus test automation. It provides an agenda for a meeting covering topics like career paths in automotive testing, test theory, and test environments. The speakers will share specifics and challenges of automotive testing and validation, and discuss test automation frameworks and examples. They aim to answer questions about working in automotive testing, how to enter the field, its future, and automotive testing in Ukraine.

1 of 39
Download to read offline

1

Automotive Testing vs Test Automation LoGeek Night Ukraine September 2020 Speakers: Alexandr Golyak & Nikolay Chertkov

2

2www.luxoft.com Agenda LoGeek Night Ukraine September 2020 1. Career Path 3. Test Environment 1. What is being automated? 2. Test Automation Frameworks 3. Test Automation Solutions examples Automotive Testing Test Automation 2. A bit of Test Theory Aim: to share Automotive Test & Validation Specifics & Challenges, answering on the following questions: • What is Automotive Testing in general? • Is it interesting to work there? • How to get there? • What’s the future of this area? • What about Automotive Testing in Ukraine? Aim: to share more technical information regarding specifics of Test Automation in Automotive, answering the following questions: • What is TAF in Automotive? • Which TA coverage could be achieved? • Does the “silver bullet” exist?

3

3www.luxoft.com About speaker LoGeek Night Ukraine September 2020  Solution Architect leading the Business Office activities for Luxoft’s Automotive System Test & Validation practice.  15+ years experience in IT industry 12 of them in Automotive area providing complex business and technical solutions with a major emphasis on Test & Validation.  Specializing in pre-sales, solution architecture, project and program management and enjoy leading a successful and productive team in a fast-paced and challenging environment.  Passionate about driving the Automotive LOB and expanding customer footprint by using proven accumulated expertise.  Prior Luxoft worked in Government Enterprise (Court Information Centre), publishing house, pharma company. https://www.linkedin.com/in/oleksandrgoliak/

4

4www.luxoft.com DXC History Created by the merger of CSC and the Enterprise Services business of Hewlett Packard Enterprise, DXC Technology boasts a long and proud history of innovation, service and value LoGeek Night Ukraine September 2020 Roy Nutt and Fletcher Jones pooled $100 to form CSC, providing computer manufacturers with complex programs Electronic Data Systems was founded in Dallas, Texas, by Ross Perot Hewlett Packard acquired Electronic Data Systems in 2008, and soon created HP Enterprise Services Hewlett Packard Company split into HP Inc. and Hewlett Packard Enterprise, and the Enterprise Services business became part of HPE DXC Technology was founded as the result of the spin-off of Hewlett Packard Enterprise's Enterprise Service segmentand its merger with Computer Sciences Corporation DXC Technology is world’s leading IT services company, with around $20 billion in annual revenues and nearly 6,000 customers in more than 70 countries! 1959 1962 20172020 2008 2015

5

5www.luxoft.com Luxoft Global Presence DXC Global Centers Luxoft & DXC at a Glance Global Centers & Top Offerings $900+M Revenue FY2019 (USD) 245+ Active Clients 13,000+ Employees Worldwide 8 Industries 14 years Automotive Experience 40+ Active Clients 3,050+ Employees Worldwide 36 Delivery Offices Automotive IndustryGlobal Leader in Innovative Tech Consulting, IT Services & SW Development $20+B Revenue FY2019 (USD) 137,000+ Employees Worldwide 6,000+ Active Clients 70+ countries 9 Streamlined Offering Families

6

6www.luxoft.com Major Automotive System Test & Validation Locations in Luxoft LoGeek Night Ukraine September 2020 Dearborn, MI, USA Bucharest, Romania Odesa, Ukraine Kyiv, Ukraine Ukraine: Kyiv & Odesa Test & Validation of IVI system & HMI SW, IOP Testing Romania: Bucharest Test & Validation of IVI systems, IOP Testing Vietnam: Ho Chi Minh City Test & Validation USA: Dearborn Onsite presence, Virtual Testing Platform Solution Mexico: Guadalajara Test & Validation Guadalajara, Mexico Ho Chi Minh City, Vietnam Germany: Böblingen, Berlin & Lindau Test & Validation of ADAS systems, UI testing Böblingen, Berlin & Lindau, Germany

7

7www.luxoft.com 1. Career Path in Automotive Testing LoGeek Night Ukraine September 2020

8

8www.luxoft.com ISTQB Automotive Software Tester Certification LoGeek Night Ukraine September 2020 https://www.istqb.org/certification-path-root/foundation-level-automotive-software-tester.html https://astqb.org/benefits/road-map/ ISTQB® CTFL ISTQB Foundation Level software testing ISTQB® CTFL-AuT ISTQB Foundation Level Automotive Software Tester If you work in the automotive industry, it comes with its own set of testing challenges, making Automotive Testing Certification important. English / German Ukrainian / Russian • Expleo Technology Germany GmbH, Köln, Germany (GTB) • imbus AG Möhrendorf, Germany (GTB) • ALTEN Italia S.p.A., Milan, Italy, (ITA-STQB) Not Available as for Sep 2020 ISTQB® Accredited Training Providers by language: https://www.udemy.com/course/istqb-automotive-software-tester-practice-tests/ ISTQB® CTFL-AuT Exam Providers (English): www.ita-stqb.org, atsqa.org, brightest.org/en/, www.cert-it.com, www.certinstitute.com, www.gasq.org, hstqb.org, www.isqi.org

9

9www.luxoft.com SHADOW Team approach example as starting point of Automotive Tester Career LoGeek Night Ukraine September 2020 Candidate • Usually has 0 experience in Automotive Testing • Joins to the project for “Learning by Doing” as intern • Project Specific Training Program is applied Project • Delegates boring but important tasks (often routine) • Has strategic reserve in case of attrition • Seniors can train their soft & mentorship skills Customer • Usually pays nothing for shadow team but aware • Gets extra mile & attrition insurance • Provides access to systems & tools

10

10www.luxoft.com Own Story: from Test Engineer to PgM & SA Career path in Automotive Testing (example) LoGeek Night Ukraine September 2020 2006 – Started IT career as Junior Engineer in non-Automotive area 2007 – graduated from the University 2008 – Regular Test Engineer 2009 – Senior Test Engineer 2010 – Test Team Lead 2015 – Project Manager 2017 – Program Manager 2019 – Acting Head of Automotive Testing CoC 2020 – Head of Business Office System Test & Validation Enablers: • Technical Background • No fear of wires • English skills • Learning by Doing • Continuous Education & Self Development • Soft skills • Luck & adventurism • Good vestibular apparatus Another Career Path examples: • From Intern to Test Manager • From Test Engineer to Program Manager • From Test Engineer to Head of Technology Office

11

11www.luxoft.com 2. A bit of Automotive Testing Theory LoGeek Night Ukraine September 2020

12

12www.luxoft.com Automotive Testing Challenges Current Automotive Test & Validation business challenges:  Increasingly intelligent and interconnected products  Increasing product testing complexity  Increasing digitization & proliferation  Hardware-Software separation  Shorter product life cycles  Stricter regulatory norms  Increasing demand of engineering capacity Automotive testers are to be able to support Automotive manufacturers to translate these challenges into strategic business opportunities The Automotive Industry in the state of Digital Transformation. Automotive manufacturers are being challenged by fast-changing customer needs, advancing digital technologies, comprehensive connectivity, and stricter environmental regulations. Testing is a critical enabler for the digital transformation journey LoGeek Night Ukraine September 2020

13

13www.luxoft.com Autonomous Shared Electric The Future of Automotive Industry (as well as Testing and Validation) is C.A.S.E. LoGeek Night Ukraine September 2020 Connected • V2V, V2X, 5G • Cybersecurity • SOTA & Telematics • Advanced Diagnostics • Emergency Services • Intelligent Traffic • SAE Levels of driving automation (1-5) • Steer by Wire, Brakes by Wire • Radars and LiDARs • CV, AR, AI, ML • Advanced Virtualization like Digital Twins • Data Ingesting, Processing, Analyzing & Annotation • Virtual Test Drive • Personalized Smart Mobility • Personal vehicle sharing (PVS) • P2P Carsharing • Ridesharing • Cloud Based Infrastructure • App. Security • Mobility as a Service * Shifting priorities in the age of COVID-19 • Electric Vehicles Powertrain technologies: Hybrids, PHEV, BEV, FCEV(H2), EREV • Battery Management System • EV + Charging Stations IOP & Communication • EV applications • Combined Charging System (CCS)

14

14www.luxoft.com SAE Levels of driving automation LoGeek Night Ukraine September 2020 Level Definitions Driving mode Example ~ Year 0 Zero Automation Everything on Gauges or system issues warnings 1885 1 Driver Assistance Everything on ACC, LKA, PA 2000 2 Partial Automation Feet Off full control of the vehicle: accelerating, braking, and steering. 2013 3 Conditional Automation Hands Off Traffic Jam Chauffeur or ALKS 2022 4 High Automation Eyes Off Supported only in limited specific areas or under special circumstances 2026 5 Full Automation Mind Off No human intervention is required at all on all roads all over the world, all year around, in all weather conditions 2032 Ways of Testing: • physically, in a closed environment • on public roads, where permitted • in a virtual environment, i.e. using computer simulations Active AD players: Google (Waymo), Baidu (Apollo), BMW, Nissan, Ford, General Motors (Cruise), Delphi Automotive Systems, Tesla, Mercedes-Benz, Bosch, Amazon (Zoox), Nuro, Pony.ai, Aurora, Apple, Audi, Uber, Lyft, Yandex, Volvo, Robocar, AutoX, TuSimple and others. DXC Robotic Drive: accelerates end-to-end engineering and testing in AD development by enabling the autonomous driving platform, toolkit, and expertise.

15

15www.luxoft.com Standards for the testing of E/E systems in Automotive LoGeek Night Ukraine September 2020 Which Standards influence the work of Test Professional in Automotive? • 4 Automotive Safety Integrity Levels: ASIL A, ASIL B, ASIL C, ASIL D • Relevant volumes (parts) for the tester: Volume 4 - 6 ASPICE • Test Specific Processes: SWE.4; SWE.5; SWE.6; SYS.4; SYS.5 • Assessment levels and capability indicators: e.g. 0 - 3 ISO 26262 FuSa AUTOSAR • Objectives of AUTOSAR, Classic & Adaptive Platforms • Three-layer architecture: BSW, RTE, SW-C ISO IEEE 29119 • Series of 5 international standards for software testing • Can be used within any software development lifecycle

16

16www.luxoft.com Test Levels ASPICE based & comparison LoGeek Night Ukraine September 2020 Test levels are not completely consistent ASPICE 3.0 ISTQB CTFL ISO 26262 No equivalent Acceptance Test Safety validation (4-9) System Qualification Test (SYS.5) System of Systems Test 1 Item integration and test (4-8) 2 System Integration Test (SYS.4) System Integration Test Verification of the Software-safety requirements (6-11) Software integration and test (6-10) Software Qualification Test (SWE.6) System Test Software Integration Test (SWE.5) Component Integration Test Software Unit Verification (SWE.4) Component Test Software-Unit-Test (6- 9) 1. The testing of several heterogenic distributed systems 2. Item integration and test includes three phases: the integration and the test of hardware and software of an element, the integration and the test of all elements belonging to the item, and the integration and the test of the item in connection with other items in the vehicle

17

17www.luxoft.com 3. Specifics of the Test Environment in Automotive LoGeek Night Ukraine September 2020

18

18www.luxoft.com Test Environment in general Open-Loop-System  Outputs has no relation to the inputs  Reactive behavior  E.g. traffic light, washing machine, TV remote Closed-Loop-System Also known as in-the-Loop  Outputs collected and forwarded to the input  Complex functions could be tested  E.g. controllers, ABS, air conditioner, refrigerator LoGeek Night Ukraine September 2020 Start Testing as early as possible Use different environment on different development phases Often realistic environment is needed Motivation for a test environment: To be able to perform his activities tester needs a test environment in which the missing parts are simulated. 1. The test environment is used to stimulate the input interfaces of the device under test and monitor its outputs through the output interfaces. 2. Afterwards, the behavior at the output interfaces is analyzed. 3. In a successful test, the observed behavior corresponds to the expected output. According to ISO/IEC/IEEE 29119, a test environment consists of the following parts: • Hardware of the test environment (computer, electronic control unit (ECU), test bench, development kit, …) • Software of the test environment (operating system, simulation software, environment models, …) • Facilities of communication (access to networks, data loggers, …) • Tools (oscilloscope, measuring tools) • Laboratory (protection from electromagnetic radiation and noise) Testing in XiL test environments  Model in the Loop (MiL)  Software in the Loop (SiL)  Process in the Loop (PiL)  Hardware in the Loop (HiL)  Vehicle in the Loop (ViL)

19

19www.luxoft.com Test Lab & Test Farm equipment LoGeek Night Ukraine September 2020 19  Test Lab workplace is designed for technology domain specifics  Test Farm could consist hundreds of Test Benches equipped for Manual & Automated Testing  Possibility to run tests 24/7 including overnight and over weekend  Could be accessible remotely by using an intranet timeslot booking tool  Vehicle breadboard is also being used by Test Engineers in some cases Source: https://www.mathworks.com/ Source: https://www.ni.com/

20

20www.luxoft.com Test Garage & Test Fleet Management LoGeek Night Ukraine September 2020 20 Test Drives Performed in different countries & regions, e.g. (New Zealand, India, Saudi Arabia, Oman, South Africa, Mexico, United States, Brazil, Spain, Portugal, Ireland, Turkey, Greece, Romania, China) Two types of test drives are being performed:  Test drives on-demand – where there is a clear requirement to drive to a specific location or perform a specific test during the test drive.  Recurrent test drives – which happen during the working hours, in the evening or during the week-end. The main purpose is to drive as many kilometers as possible, using the system as an end-user would. Secure Test Fleet Management  Closed TISAX compliant perimeter in Garage  The engineers have to pass a defensive driving training and first-aid training which provided by local companies agreed with the customer.  “How to garage” training provided internally for new employees  When not in use, the experimental HW was covered by a textile cover.  After test drives, the cars had to be parked in closed garages. Test car setup activities  Removal of old equipment & installation of new equipment (e.g. Head unit, Touch Pad, Touch Display, Multimedia Boxes, USB Hub, GNSS antenna, Audio Amplifier)  Test cars hardware and mechanical modifications according to testing process, installation of data loggers & sensors.  Daily maintenance tasks for electronic equipment

21

21www.luxoft.com Vehicle Prototyping & Measurement Systems LoGeek Night Ukraine September 2020 21 Wiring LiDAR Radar Camera DGPS Power Management HMI Modular Measurement Systems & Dataloggers System conception  Clarify requirements  Definition of system and KPIs  Conception of whole system System development & setup  Setup in laboratory  Check & Validation  Adaptation of system to prototype vehicle System integration  Integration of various sensors and components  Adaptation of HMI  Space saving wiring in car Modular Measurement Solutions

22

22www.luxoft.com Scenario Based Testing Approach LoGeek Night Ukraine September 2020 Testing Approach example for the most complex projects  Hundreds of End User Behavior Scenarios created and executed for complex projects.  Each week certain number of scenarios is re-planned taking into consideration certain test focus.  Each scenario consist of ~ 10 – 15 basic Domain Test Cases created by Domain test specialist.  Scenarios Executed in Laboratory or Test Car (using pre-defined routes). Different End User Profiles:  Housewife  Taxi Driver  Car rental  Manager  Salesman  Senior  Youngster Main benefits:  increase cross domain coverage  Stress the system  Check the most common scenarios  have always a quick overview of the stability of SW Difficulties:  using re-mastered or non-prototype car  all components (ECU) needed to be updated (time-consuming)  Scenarios to be executed by very experienced testers having deep knowledge of all domains. Driver Co- Driver Tester Rear Left Tester Rear Right Tester Data / Trace logger

23

23www.luxoft.com Summary and Conclusions Automotive Testing is very cool place to work for both manual & automated testers What cool things automotive testers can do?  Drive prototype camouflaged cars not even known by Paparazzi yet  Execute tests in Virtual Environment  Deal with the top megatrends transforming businesses  Set up and operate Test Farms in car production line  Drive Special Test Drive Roads e.g. in Shanghai and California  Operate Climate Chamber and Sun Simulator  Burn and brick Head Units LoGeek Night Ukraine September 2020 Pros: • Exceptional talent pool • Strong technical background • Mindset of the people • Nearshore to OEMs and Tier1s Cons: • Driving prototype cars issues • HW import complexity & price • Roads condition & safety Location Specific: Ukraine Although COVID-19 Crisis, Ukraine locations (Kyiv & Odessa) are growing in Automotive Line of Business. What other related benefits?  To obtain knowledge in technology domain  To develop test tools and Smart Test Bench Solutions  To develop & Test Automation Frameworks and Virtual Testing Platforms  To collect and annotate data  To learn German, Chinese, etc.  Games Development & Testing  Consulting & Advisory

24

24www.luxoft.com Part II: Test Automation in Automotive LoGeek Night Ukraine September 2020

25

25www.luxoft.com About speaker LoGeek Night Ukraine September 2020 Part II: Test Automation in Automotive  Lead Expert driving Technology Office activities for Luxoft's System Test & Validation practice.  Having more than 10 years of knowledge and experience in Automotive Test Automation.  Responsible for providing and building the relevant expertise and capabilities to support both customer requests as well as ongoing activities within Automotive LOB across all practices.  Focusing on establishing best practices across the entire line of business as well as providing his technical expertise for the team when they engage in pre-sales support  Prior to Luxoft developed Test Automation solutions for Gyroscope Manufacturer  Joined Luxoft as Test Engineer www.linkedin.com/in/nikolay-chertkov-25522585

26

26www.luxoft.com Part II. Test Automation in Automotive LoGeek Night Ukraine September 2020 What is being automated? Test Automation Frameworks Automation solutions examples

27

27www.luxoft.com Automotive vs Automation LoGeek Night Ukraine September 2020 Employer: now show me what you can!​ Me:​

28

28www.luxoft.com Automation vs Manual LoGeek Night Ukraine September 2020

29

29www.luxoft.com Automated vs Manual LoGeek Night Ukraine September 2020 ASPICE levels and Test Automation coverage SYS.5ENG.10 System Qualification Test Software Integration TestENG.7 SWE.6ENG.8 Software Qualification Test SWE.5 ENG.9 SYS.4 System Integration Test ENG.6 SWE.4 Software Unit Verification Amount of tests ASPICE group Manual Automated

30

30www.luxoft.com TA example: domain Diagnostic LoGeek Night Ukraine September 2020 Computer Diagnostic

31

31www.luxoft.com Example: domain Diagnostic LoGeek Night Ukraine September 2020 Purpose • Self-diagnosis and error reporting • Additional/hidden system info • Change of ECU mode • Sensor calibration • Feature activation/deactivation TA • Potential up to 100% • Protocol tests • Functional tests • Fault injection tests • Endurance tests Source: dSpace World Conference 2019

32

32www.luxoft.com In-car Infotainment trends LoGeek Night Ukraine September 2020 • Autonomous driving • Functional safety • Cyber security • ADAS • Connected mobility Source: Joyofapple.com Source: Uslugio.com

33

33www.luxoft.com In-car Infotainment trends (McKinsey, 2019) LoGeek Night Ukraine September 2020 “In-car infotainment SW complexity has grown 4 times from 2010 till 2020. Meanwhile SW- development productivity increased just x1.5” Dr. Ondrej Burkacky, McKinsey & Company, Inc dSpace World Conference 2019

34

34www.luxoft.com Test Automation. Cruise ship analogy LoGeek Night Ukraine September 2020 Source: vk.com

35

35www.luxoft.com Test Environment. VTBxx LoGeek Night Ukraine September 2020 A picture where you see a biggest office mess ever with a lot of wires, car head-units, dozen displays around, aluminum test rack and a chair in the middle. You realize it’s not a warehouse but a workplace and start laughing… and I start crying Source: forums.elderscrollsonline.com

36

36www.luxoft.com Test Automation Framework LoGeek Night Ukraine September 2020  Customer- and project-specific  Reusable design  Automation vs Manual balance  Independent instruments  Common Interface (API)  Scalable approach  Complex scenarios  Continuous testing  "Shift-left" enabler Unique Uniting Level Up Script Editor Diagnostic Tool CAN Gen Tool Some/IP Tool D-Bus Comm Logs & Reports Tool Tracing Tools (DLT, IOC, CAN, PCAP) Peripherals Tools (Frame Grabber, Climate Chamber, USB Matrix, Relay Box, ...) TAF Hub

37

37www.luxoft.com Smart Test Bench Solution LoGeek Night Ukraine September 2020 Smart Test Bench Micro HIL Bench Head Unit ModemGateway Virtual Test Platform SD CARD ETH Audio IN Audio OUT LVDS IN HDMI USB UI Line Out / SPKR MIC Display  Display virtualization  Remote Connectivity  Automation Framework

38

38www.luxoft.com Smart Test Bench Farm Solution LoGeek Night Ukraine September 2020 Vehicle Fleet Smart Bench Farm Region 1 Smart Bench Farm Region N Internal Network Internal Network Internal Network Scheduler Internet System Quality early feedback • Vast test coverage • Remote access • Running tests 24/7 • Region free

39

Thank You!

More Related Content

Alexandr Golyak, Nikolay Chertkov "Automotive Testing vs Test Automatio"

  • 1. Automotive Testing vs Test Automation LoGeek Night Ukraine September 2020 Speakers: Alexandr Golyak & Nikolay Chertkov
  • 2. 2www.luxoft.com Agenda LoGeek Night Ukraine September 2020 1. Career Path 3. Test Environment 1. What is being automated? 2. Test Automation Frameworks 3. Test Automation Solutions examples Automotive Testing Test Automation 2. A bit of Test Theory Aim: to share Automotive Test & Validation Specifics & Challenges, answering on the following questions: • What is Automotive Testing in general? • Is it interesting to work there? • How to get there? • What’s the future of this area? • What about Automotive Testing in Ukraine? Aim: to share more technical information regarding specifics of Test Automation in Automotive, answering the following questions: • What is TAF in Automotive? • Which TA coverage could be achieved? • Does the “silver bullet” exist?
  • 3. 3www.luxoft.com About speaker LoGeek Night Ukraine September 2020  Solution Architect leading the Business Office activities for Luxoft’s Automotive System Test & Validation practice.  15+ years experience in IT industry 12 of them in Automotive area providing complex business and technical solutions with a major emphasis on Test & Validation.  Specializing in pre-sales, solution architecture, project and program management and enjoy leading a successful and productive team in a fast-paced and challenging environment.  Passionate about driving the Automotive LOB and expanding customer footprint by using proven accumulated expertise.  Prior Luxoft worked in Government Enterprise (Court Information Centre), publishing house, pharma company. https://www.linkedin.com/in/oleksandrgoliak/
  • 4. 4www.luxoft.com DXC History Created by the merger of CSC and the Enterprise Services business of Hewlett Packard Enterprise, DXC Technology boasts a long and proud history of innovation, service and value LoGeek Night Ukraine September 2020 Roy Nutt and Fletcher Jones pooled $100 to form CSC, providing computer manufacturers with complex programs Electronic Data Systems was founded in Dallas, Texas, by Ross Perot Hewlett Packard acquired Electronic Data Systems in 2008, and soon created HP Enterprise Services Hewlett Packard Company split into HP Inc. and Hewlett Packard Enterprise, and the Enterprise Services business became part of HPE DXC Technology was founded as the result of the spin-off of Hewlett Packard Enterprise's Enterprise Service segmentand its merger with Computer Sciences Corporation DXC Technology is world’s leading IT services company, with around $20 billion in annual revenues and nearly 6,000 customers in more than 70 countries! 1959 1962 20172020 2008 2015
  • 5. 5www.luxoft.com Luxoft Global Presence DXC Global Centers Luxoft & DXC at a Glance Global Centers & Top Offerings $900+M Revenue FY2019 (USD) 245+ Active Clients 13,000+ Employees Worldwide 8 Industries 14 years Automotive Experience 40+ Active Clients 3,050+ Employees Worldwide 36 Delivery Offices Automotive IndustryGlobal Leader in Innovative Tech Consulting, IT Services & SW Development $20+B Revenue FY2019 (USD) 137,000+ Employees Worldwide 6,000+ Active Clients 70+ countries 9 Streamlined Offering Families
  • 6. 6www.luxoft.com Major Automotive System Test & Validation Locations in Luxoft LoGeek Night Ukraine September 2020 Dearborn, MI, USA Bucharest, Romania Odesa, Ukraine Kyiv, Ukraine Ukraine: Kyiv & Odesa Test & Validation of IVI system & HMI SW, IOP Testing Romania: Bucharest Test & Validation of IVI systems, IOP Testing Vietnam: Ho Chi Minh City Test & Validation USA: Dearborn Onsite presence, Virtual Testing Platform Solution Mexico: Guadalajara Test & Validation Guadalajara, Mexico Ho Chi Minh City, Vietnam Germany: Böblingen, Berlin & Lindau Test & Validation of ADAS systems, UI testing Böblingen, Berlin & Lindau, Germany
  • 7. 7www.luxoft.com 1. Career Path in Automotive Testing LoGeek Night Ukraine September 2020
  • 8. 8www.luxoft.com ISTQB Automotive Software Tester Certification LoGeek Night Ukraine September 2020 https://www.istqb.org/certification-path-root/foundation-level-automotive-software-tester.html https://astqb.org/benefits/road-map/ ISTQB® CTFL ISTQB Foundation Level software testing ISTQB® CTFL-AuT ISTQB Foundation Level Automotive Software Tester If you work in the automotive industry, it comes with its own set of testing challenges, making Automotive Testing Certification important. English / German Ukrainian / Russian • Expleo Technology Germany GmbH, Köln, Germany (GTB) • imbus AG Möhrendorf, Germany (GTB) • ALTEN Italia S.p.A., Milan, Italy, (ITA-STQB) Not Available as for Sep 2020 ISTQB® Accredited Training Providers by language: https://www.udemy.com/course/istqb-automotive-software-tester-practice-tests/ ISTQB® CTFL-AuT Exam Providers (English): www.ita-stqb.org, atsqa.org, brightest.org/en/, www.cert-it.com, www.certinstitute.com, www.gasq.org, hstqb.org, www.isqi.org
  • 9. 9www.luxoft.com SHADOW Team approach example as starting point of Automotive Tester Career LoGeek Night Ukraine September 2020 Candidate • Usually has 0 experience in Automotive Testing • Joins to the project for “Learning by Doing” as intern • Project Specific Training Program is applied Project • Delegates boring but important tasks (often routine) • Has strategic reserve in case of attrition • Seniors can train their soft & mentorship skills Customer • Usually pays nothing for shadow team but aware • Gets extra mile & attrition insurance • Provides access to systems & tools
  • 10. 10www.luxoft.com Own Story: from Test Engineer to PgM & SA Career path in Automotive Testing (example) LoGeek Night Ukraine September 2020 2006 – Started IT career as Junior Engineer in non-Automotive area 2007 – graduated from the University 2008 – Regular Test Engineer 2009 – Senior Test Engineer 2010 – Test Team Lead 2015 – Project Manager 2017 – Program Manager 2019 – Acting Head of Automotive Testing CoC 2020 – Head of Business Office System Test & Validation Enablers: • Technical Background • No fear of wires • English skills • Learning by Doing • Continuous Education & Self Development • Soft skills • Luck & adventurism • Good vestibular apparatus Another Career Path examples: • From Intern to Test Manager • From Test Engineer to Program Manager • From Test Engineer to Head of Technology Office
  • 11. 11www.luxoft.com 2. A bit of Automotive Testing Theory LoGeek Night Ukraine September 2020
  • 12. 12www.luxoft.com Automotive Testing Challenges Current Automotive Test & Validation business challenges:  Increasingly intelligent and interconnected products  Increasing product testing complexity  Increasing digitization & proliferation  Hardware-Software separation  Shorter product life cycles  Stricter regulatory norms  Increasing demand of engineering capacity Automotive testers are to be able to support Automotive manufacturers to translate these challenges into strategic business opportunities The Automotive Industry in the state of Digital Transformation. Automotive manufacturers are being challenged by fast-changing customer needs, advancing digital technologies, comprehensive connectivity, and stricter environmental regulations. Testing is a critical enabler for the digital transformation journey LoGeek Night Ukraine September 2020
  • 13. 13www.luxoft.com Autonomous Shared Electric The Future of Automotive Industry (as well as Testing and Validation) is C.A.S.E. LoGeek Night Ukraine September 2020 Connected • V2V, V2X, 5G • Cybersecurity • SOTA & Telematics • Advanced Diagnostics • Emergency Services • Intelligent Traffic • SAE Levels of driving automation (1-5) • Steer by Wire, Brakes by Wire • Radars and LiDARs • CV, AR, AI, ML • Advanced Virtualization like Digital Twins • Data Ingesting, Processing, Analyzing & Annotation • Virtual Test Drive • Personalized Smart Mobility • Personal vehicle sharing (PVS) • P2P Carsharing • Ridesharing • Cloud Based Infrastructure • App. Security • Mobility as a Service * Shifting priorities in the age of COVID-19 • Electric Vehicles Powertrain technologies: Hybrids, PHEV, BEV, FCEV(H2), EREV • Battery Management System • EV + Charging Stations IOP & Communication • EV applications • Combined Charging System (CCS)
  • 14. 14www.luxoft.com SAE Levels of driving automation LoGeek Night Ukraine September 2020 Level Definitions Driving mode Example ~ Year 0 Zero Automation Everything on Gauges or system issues warnings 1885 1 Driver Assistance Everything on ACC, LKA, PA 2000 2 Partial Automation Feet Off full control of the vehicle: accelerating, braking, and steering. 2013 3 Conditional Automation Hands Off Traffic Jam Chauffeur or ALKS 2022 4 High Automation Eyes Off Supported only in limited specific areas or under special circumstances 2026 5 Full Automation Mind Off No human intervention is required at all on all roads all over the world, all year around, in all weather conditions 2032 Ways of Testing: • physically, in a closed environment • on public roads, where permitted • in a virtual environment, i.e. using computer simulations Active AD players: Google (Waymo), Baidu (Apollo), BMW, Nissan, Ford, General Motors (Cruise), Delphi Automotive Systems, Tesla, Mercedes-Benz, Bosch, Amazon (Zoox), Nuro, Pony.ai, Aurora, Apple, Audi, Uber, Lyft, Yandex, Volvo, Robocar, AutoX, TuSimple and others. DXC Robotic Drive: accelerates end-to-end engineering and testing in AD development by enabling the autonomous driving platform, toolkit, and expertise.
  • 15. 15www.luxoft.com Standards for the testing of E/E systems in Automotive LoGeek Night Ukraine September 2020 Which Standards influence the work of Test Professional in Automotive? • 4 Automotive Safety Integrity Levels: ASIL A, ASIL B, ASIL C, ASIL D • Relevant volumes (parts) for the tester: Volume 4 - 6 ASPICE • Test Specific Processes: SWE.4; SWE.5; SWE.6; SYS.4; SYS.5 • Assessment levels and capability indicators: e.g. 0 - 3 ISO 26262 FuSa AUTOSAR • Objectives of AUTOSAR, Classic & Adaptive Platforms • Three-layer architecture: BSW, RTE, SW-C ISO IEEE 29119 • Series of 5 international standards for software testing • Can be used within any software development lifecycle
  • 16. 16www.luxoft.com Test Levels ASPICE based & comparison LoGeek Night Ukraine September 2020 Test levels are not completely consistent ASPICE 3.0 ISTQB CTFL ISO 26262 No equivalent Acceptance Test Safety validation (4-9) System Qualification Test (SYS.5) System of Systems Test 1 Item integration and test (4-8) 2 System Integration Test (SYS.4) System Integration Test Verification of the Software-safety requirements (6-11) Software integration and test (6-10) Software Qualification Test (SWE.6) System Test Software Integration Test (SWE.5) Component Integration Test Software Unit Verification (SWE.4) Component Test Software-Unit-Test (6- 9) 1. The testing of several heterogenic distributed systems 2. Item integration and test includes three phases: the integration and the test of hardware and software of an element, the integration and the test of all elements belonging to the item, and the integration and the test of the item in connection with other items in the vehicle
  • 17. 17www.luxoft.com 3. Specifics of the Test Environment in Automotive LoGeek Night Ukraine September 2020
  • 18. 18www.luxoft.com Test Environment in general Open-Loop-System  Outputs has no relation to the inputs  Reactive behavior  E.g. traffic light, washing machine, TV remote Closed-Loop-System Also known as in-the-Loop  Outputs collected and forwarded to the input  Complex functions could be tested  E.g. controllers, ABS, air conditioner, refrigerator LoGeek Night Ukraine September 2020 Start Testing as early as possible Use different environment on different development phases Often realistic environment is needed Motivation for a test environment: To be able to perform his activities tester needs a test environment in which the missing parts are simulated. 1. The test environment is used to stimulate the input interfaces of the device under test and monitor its outputs through the output interfaces. 2. Afterwards, the behavior at the output interfaces is analyzed. 3. In a successful test, the observed behavior corresponds to the expected output. According to ISO/IEC/IEEE 29119, a test environment consists of the following parts: • Hardware of the test environment (computer, electronic control unit (ECU), test bench, development kit, …) • Software of the test environment (operating system, simulation software, environment models, …) • Facilities of communication (access to networks, data loggers, …) • Tools (oscilloscope, measuring tools) • Laboratory (protection from electromagnetic radiation and noise) Testing in XiL test environments  Model in the Loop (MiL)  Software in the Loop (SiL)  Process in the Loop (PiL)  Hardware in the Loop (HiL)  Vehicle in the Loop (ViL)
  • 19. 19www.luxoft.com Test Lab & Test Farm equipment LoGeek Night Ukraine September 2020 19  Test Lab workplace is designed for technology domain specifics  Test Farm could consist hundreds of Test Benches equipped for Manual & Automated Testing  Possibility to run tests 24/7 including overnight and over weekend  Could be accessible remotely by using an intranet timeslot booking tool  Vehicle breadboard is also being used by Test Engineers in some cases Source: https://www.mathworks.com/ Source: https://www.ni.com/
  • 20. 20www.luxoft.com Test Garage & Test Fleet Management LoGeek Night Ukraine September 2020 20 Test Drives Performed in different countries & regions, e.g. (New Zealand, India, Saudi Arabia, Oman, South Africa, Mexico, United States, Brazil, Spain, Portugal, Ireland, Turkey, Greece, Romania, China) Two types of test drives are being performed:  Test drives on-demand – where there is a clear requirement to drive to a specific location or perform a specific test during the test drive.  Recurrent test drives – which happen during the working hours, in the evening or during the week-end. The main purpose is to drive as many kilometers as possible, using the system as an end-user would. Secure Test Fleet Management  Closed TISAX compliant perimeter in Garage  The engineers have to pass a defensive driving training and first-aid training which provided by local companies agreed with the customer.  “How to garage” training provided internally for new employees  When not in use, the experimental HW was covered by a textile cover.  After test drives, the cars had to be parked in closed garages. Test car setup activities  Removal of old equipment & installation of new equipment (e.g. Head unit, Touch Pad, Touch Display, Multimedia Boxes, USB Hub, GNSS antenna, Audio Amplifier)  Test cars hardware and mechanical modifications according to testing process, installation of data loggers & sensors.  Daily maintenance tasks for electronic equipment
  • 21. 21www.luxoft.com Vehicle Prototyping & Measurement Systems LoGeek Night Ukraine September 2020 21 Wiring LiDAR Radar Camera DGPS Power Management HMI Modular Measurement Systems & Dataloggers System conception  Clarify requirements  Definition of system and KPIs  Conception of whole system System development & setup  Setup in laboratory  Check & Validation  Adaptation of system to prototype vehicle System integration  Integration of various sensors and components  Adaptation of HMI  Space saving wiring in car Modular Measurement Solutions
  • 22. 22www.luxoft.com Scenario Based Testing Approach LoGeek Night Ukraine September 2020 Testing Approach example for the most complex projects  Hundreds of End User Behavior Scenarios created and executed for complex projects.  Each week certain number of scenarios is re-planned taking into consideration certain test focus.  Each scenario consist of ~ 10 – 15 basic Domain Test Cases created by Domain test specialist.  Scenarios Executed in Laboratory or Test Car (using pre-defined routes). Different End User Profiles:  Housewife  Taxi Driver  Car rental  Manager  Salesman  Senior  Youngster Main benefits:  increase cross domain coverage  Stress the system  Check the most common scenarios  have always a quick overview of the stability of SW Difficulties:  using re-mastered or non-prototype car  all components (ECU) needed to be updated (time-consuming)  Scenarios to be executed by very experienced testers having deep knowledge of all domains. Driver Co- Driver Tester Rear Left Tester Rear Right Tester Data / Trace logger
  • 23. 23www.luxoft.com Summary and Conclusions Automotive Testing is very cool place to work for both manual & automated testers What cool things automotive testers can do?  Drive prototype camouflaged cars not even known by Paparazzi yet  Execute tests in Virtual Environment  Deal with the top megatrends transforming businesses  Set up and operate Test Farms in car production line  Drive Special Test Drive Roads e.g. in Shanghai and California  Operate Climate Chamber and Sun Simulator  Burn and brick Head Units LoGeek Night Ukraine September 2020 Pros: • Exceptional talent pool • Strong technical background • Mindset of the people • Nearshore to OEMs and Tier1s Cons: • Driving prototype cars issues • HW import complexity & price • Roads condition & safety Location Specific: Ukraine Although COVID-19 Crisis, Ukraine locations (Kyiv & Odessa) are growing in Automotive Line of Business. What other related benefits?  To obtain knowledge in technology domain  To develop test tools and Smart Test Bench Solutions  To develop & Test Automation Frameworks and Virtual Testing Platforms  To collect and annotate data  To learn German, Chinese, etc.  Games Development & Testing  Consulting & Advisory
  • 24. 24www.luxoft.com Part II: Test Automation in Automotive LoGeek Night Ukraine September 2020
  • 25. 25www.luxoft.com About speaker LoGeek Night Ukraine September 2020 Part II: Test Automation in Automotive  Lead Expert driving Technology Office activities for Luxoft's System Test & Validation practice.  Having more than 10 years of knowledge and experience in Automotive Test Automation.  Responsible for providing and building the relevant expertise and capabilities to support both customer requests as well as ongoing activities within Automotive LOB across all practices.  Focusing on establishing best practices across the entire line of business as well as providing his technical expertise for the team when they engage in pre-sales support  Prior to Luxoft developed Test Automation solutions for Gyroscope Manufacturer  Joined Luxoft as Test Engineer www.linkedin.com/in/nikolay-chertkov-25522585
  • 26. 26www.luxoft.com Part II. Test Automation in Automotive LoGeek Night Ukraine September 2020 What is being automated? Test Automation Frameworks Automation solutions examples
  • 27. 27www.luxoft.com Automotive vs Automation LoGeek Night Ukraine September 2020 Employer: now show me what you can!​ Me:​
  • 28. 28www.luxoft.com Automation vs Manual LoGeek Night Ukraine September 2020
  • 29. 29www.luxoft.com Automated vs Manual LoGeek Night Ukraine September 2020 ASPICE levels and Test Automation coverage SYS.5ENG.10 System Qualification Test Software Integration TestENG.7 SWE.6ENG.8 Software Qualification Test SWE.5 ENG.9 SYS.4 System Integration Test ENG.6 SWE.4 Software Unit Verification Amount of tests ASPICE group Manual Automated
  • 30. 30www.luxoft.com TA example: domain Diagnostic LoGeek Night Ukraine September 2020 Computer Diagnostic
  • 31. 31www.luxoft.com Example: domain Diagnostic LoGeek Night Ukraine September 2020 Purpose • Self-diagnosis and error reporting • Additional/hidden system info • Change of ECU mode • Sensor calibration • Feature activation/deactivation TA • Potential up to 100% • Protocol tests • Functional tests • Fault injection tests • Endurance tests Source: dSpace World Conference 2019
  • 32. 32www.luxoft.com In-car Infotainment trends LoGeek Night Ukraine September 2020 • Autonomous driving • Functional safety • Cyber security • ADAS • Connected mobility Source: Joyofapple.com Source: Uslugio.com
  • 33. 33www.luxoft.com In-car Infotainment trends (McKinsey, 2019) LoGeek Night Ukraine September 2020 “In-car infotainment SW complexity has grown 4 times from 2010 till 2020. Meanwhile SW- development productivity increased just x1.5” Dr. Ondrej Burkacky, McKinsey & Company, Inc dSpace World Conference 2019
  • 34. 34www.luxoft.com Test Automation. Cruise ship analogy LoGeek Night Ukraine September 2020 Source: vk.com
  • 35. 35www.luxoft.com Test Environment. VTBxx LoGeek Night Ukraine September 2020 A picture where you see a biggest office mess ever with a lot of wires, car head-units, dozen displays around, aluminum test rack and a chair in the middle. You realize it’s not a warehouse but a workplace and start laughing… and I start crying Source: forums.elderscrollsonline.com
  • 36. 36www.luxoft.com Test Automation Framework LoGeek Night Ukraine September 2020  Customer- and project-specific  Reusable design  Automation vs Manual balance  Independent instruments  Common Interface (API)  Scalable approach  Complex scenarios  Continuous testing  "Shift-left" enabler Unique Uniting Level Up Script Editor Diagnostic Tool CAN Gen Tool Some/IP Tool D-Bus Comm Logs & Reports Tool Tracing Tools (DLT, IOC, CAN, PCAP) Peripherals Tools (Frame Grabber, Climate Chamber, USB Matrix, Relay Box, ...) TAF Hub
  • 37. 37www.luxoft.com Smart Test Bench Solution LoGeek Night Ukraine September 2020 Smart Test Bench Micro HIL Bench Head Unit ModemGateway Virtual Test Platform SD CARD ETH Audio IN Audio OUT LVDS IN HDMI USB UI Line Out / SPKR MIC Display  Display virtualization  Remote Connectivity  Automation Framework
  • 38. 38www.luxoft.com Smart Test Bench Farm Solution LoGeek Night Ukraine September 2020 Vehicle Fleet Smart Bench Farm Region 1 Smart Bench Farm Region N Internal Network Internal Network Internal Network Scheduler Internet System Quality early feedback • Vast test coverage • Remote access • Running tests 24/7 • Region free