Highlighted notes of article while studying Concurrent Data Structures, CSE:
Real-world Concurrency
Bryan Cantrill and Jeff Bonwick,
Sun Microsystems
ACM Queue, September 2008 https://doi.org/10.1145/1454456.1454462
BRYAN CANTRILL is a Distinguished Engineer at Sun Microsystems, where he has worked on concurrent systems since coming to Sun to work with Jeff Bonwick on Solaris performance in 1996. Along with colleagues Mike Shapiro and Adam Leventhal, Cantrill developed DTrace, a facility for dynamic instrumentation of production systems that was directly inspired by his frustration in understanding the behavior of concurrent systems.
JEFF BONWICK is a Fellow at Sun Microsystems, where he has worked on concurrent systems since 1990. He is best known for inventing and leading the development of Sun’s ZFS (Zettabyte File System), but prior to this he was known for having written (or rather, rewritten) many of the most parallel subsystems in the Solaris kernel, including the syn- chronization primitives, the kernel memory allocator, and the thread-blocking mechanism.
https://dl.acm.org/doi/10.1145/1454456.1454462
This document contains interview questions for a Linux administrator role. It includes questions about shell scripting, system administration tasks, networking, and more. Some example questions are how to take input in a shell script, write a script to convert file path slashes, and explain the differences between UDP and TCP. The document provides technical questions to assess a candidate's Linux knowledge and experience.
The document provides an overview of a course on software engineering. It discusses key concepts like structured programming, object-oriented programming, design principles of abstraction and modularity. It also covers programming in languages like C and Matlab. The goals of the course are to understand basic program design techniques, produce well-structured programs, and have a basic understanding of object-oriented design.
The document describes a course on software engineering that covers basic design principles and techniques like structured programming, object-oriented programming, and data structures. It aims to give students an understanding of how to produce well-structured, maintainable code. Examples will be shown in MATLAB and C/C++. Key concepts covered include abstraction, modularity, procedural and object-oriented programming, functions, classes, and arrays.
This was a talk, largely on Kamaelia & its original context given at a Free Streaming Workshop in Florence, Italy in Summer 2004. Many of the core
concepts still hold valid in Kamaelia today
The document introduces Parallel Pixie Dust (PPD), a cross-platform thread library that aims to guarantee deadlock-free and race-condition free schedules that are optimal. It discusses the need for multiple threads due to factors like the memory wall. Current threading models are problematic because testing and debugging threaded code is difficult. PPD uses futures and thread pools to simulate data flow and generate tree-like thread schedules. It provides parallel versions of functions and thread-safe containers to enable multi-threaded standard library algorithms. The goal is to make writing correct multi-threaded programs easier.
This document discusses clustering PHP applications. It begins by defining computer clustering as linking computers to work together closely and act as a single computer. The document then discusses why clustering is used, including to service more users faster and increase reliability. It outlines objectives like linear capacity and cost increases with clustering as well as exponential reliability gains. The rest of the document summarizes various techniques for clustering PHP applications, such as load balancing, database scaling, replicated storage, backups, caching, sessions, staging strategies, and debugging clustered applications.
This document discusses the major tenets of the Linux philosophy. It explains that everything is treated as a file, there are small single-purpose programs, and programs can be chained together to perform complex tasks. Configuration data is stored in text files for easy editing and portability. Programs act as filters that take input, modify it, and output the results. The philosophy values portability over efficiency and uses software leverage through small reusable programs.
The document discusses the pros and cons of using open source software in projects. It notes that open source code comes with various licenses like GPL, BSD, and MIT that differ in how they can be used. While open source lacks perceived accountability, it actually has governance structures. The document aims to provide context on open source licensing and use within the scope of incorporating external code into a project. It addresses common concerns about open source like licensing complexity but clarifies the intent is to protect author rights, not hinder code use.
The first compilers were written by hand-translating programs into machine code opcodes. The earliest programs were written directly in binary opcodes by looking up each instruction and its corresponding opcode. Later, the first assemblers were created by hand to automate this process. These early assemblers could then be used to assemble more advanced assemblers and compilers. This process of using each new tool to build the next was called bootstrapping. The first high-level programming language was FORTRAN, created in 1954, which had an optimizing compiler that produced very efficient code and helped establish modern programming practices.
The document provides an overview of Linux interview essentials related to operating system concepts, system calls, inter-process communication, and threads. It discusses topics such as the role and components of an operating system, multi-tasking and scheduling policies, differences between function calls and system calls, static and dynamic linking, common code and stack errors, memory leaks, kernel modes, monolithic and micro kernels, interrupts, exceptions, system calls implementation in Linux, and synchronous vs asynchronous communication methods.
The document discusses parallel computing over the past 25 years and challenges for using multicore chips in the next decade. It aims to provide context to scale applications effectively to 32-1024 cores. Key challenges include expressing inherent application parallelism while enabling efficient mapping to hardware through programming models and runtime systems. Future work includes developing methods to restore lost parallelism information and tradeoffs between programming effort, generality and performance.
This document contains the questions and answers from a computer architecture and organization exam. It includes questions about the differences between computer architecture and organization, instruction formats, bus definitions, cache memory advantages, and virtual memory. The responses provide detailed explanations of concepts like locality of reference, thrashing, address mapping, cache hits and misses, and hierarchical memory systems. Justification is given for using a hierarchical approach to improve performance across different memory types. The differences between paging and segmentation in virtual memory are also distinguished.
The document describes an instructional operating system called Nachos that was developed for teaching undergraduate operating systems courses. Some key points:
- Nachos simulates a RISC CPU and devices to allow students to modify an operating system kernel without needing physical hardware. It runs as a UNIX process.
- The assignments guide students through implementing core OS functionality like threads, file systems, multiprogramming, and networking. This allows students to build their understanding of modern operating system concepts through hands-on projects.
- Nachos emphasizes simplicity over realism so students can understand the entire system. The authors found simplicity was important for students to grasp concurrency and other challenging OS concepts.
The document describes an instructional operating system called Nachos that was developed for teaching undergraduate operating systems concepts. Some key points:
- Nachos simulates a RISC CPU and devices to allow students to modify an operating system kernel as a project. It runs as a UNIX process for simplicity.
- There are 5 main assignments that cover thread management, file systems, multiprogramming, virtual memory, and networking. Each builds on concepts from previous assignments.
- The goal was to create a simple yet realistic system that illustrates modern OS techniques and allows quantitative evaluation of design tradeoffs through benchmarks.
Highlighted notes of article while studying Concurrent Data Structures, CSE:
Software and the Concurrency Revolution
Herb Sutter
Software Architect, Microsoft
Software Development Consultant, www.gotw.ca/training
Herb Sutter is a prominent C++ expert. He is also a book author and was a columnist for Dr. Dobb's Journal. He joined Microsoft in 2002 as a platform evangelist for Visual C++ .NET, rising to lead software architect for C++/CLI.
MYIR Product Brochure - A Global Provider of Embedded SOMs & Solutions
This brochure gives introduction of MYIR Electronics company and MYIR's products and services.
MYIR Electronics Limited (MYIR for short), established in 2011, is a global provider of embedded System-On-Modules (SOMs) and
comprehensive solutions based on various architectures such as ARM, FPGA, RISC-V, and AI. We cater to customers' needs for large-scale production, offering customized design, industry-specific application solutions, and one-stop OEM services.
MYIR, recognized as a national high-tech enterprise, is also listed among the "Specialized
and Special new" Enterprises in Shenzhen, China. Our core belief is that "Our success stems from our customers' success" and embraces the philosophy
of "Make Your Idea Real, then My Idea Realizing!"
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
This document provides an introduction and overview of operating systems. It discusses what an operating system is, how operating systems have evolved over time, and why operating systems are important to study. Specifically, it describes how operating systems started as simple programs to run on expensive hardware and evolved to support time-sharing, multiprocessing, networking and more as hardware became less expensive and more powerful. The key roles of an operating system are to provide a standard library of resources and coordinate access to resources like the CPU and memory.
Threads allow concurrency within a process by separating the execution state (program counter, registers, stack) from the process (address space, resources). This allows a single process to have multiple execution streams, sharing data through the shared address space. Key benefits are improved responsiveness, easy resource sharing between threads, and ability to utilize multiple cores. Threads can be implemented in the user space or kernel space.
Threads allow concurrency within a process by separating the execution state (program counter, registers, stack) from the process. This allows a process to have multiple execution streams (threads) that can run independently while sharing the process's resources like memory. Key benefits of threads include improved responsiveness, easy resource sharing between threads, and better utilization of multiprocessor systems.
Highlighted notes of article while studying Concurrent Data Structures, CSE:
Real-world Concurrency
Bryan Cantrill and Jeff Bonwick,
Sun Microsystems
ACM Queue, September 2008 https://doi.org/10.1145/1454456.1454462
BRYAN CANTRILL is a Distinguished Engineer at Sun Microsystems, where he has worked on concurrent systems since coming to Sun to work with Jeff Bonwick on Solaris performance in 1996. Along with colleagues Mike Shapiro and Adam Leventhal, Cantrill developed DTrace, a facility for dynamic instrumentation of production systems that was directly inspired by his frustration in understanding the behavior of concurrent systems.
JEFF BONWICK is a Fellow at Sun Microsystems, where he has worked on concurrent systems since 1990. He is best known for inventing and leading the development of Sun’s ZFS (Zettabyte File System), but prior to this he was known for having written (or rather, rewritten) many of the most parallel subsystems in the Solaris kernel, including the syn- chronization primitives, the kernel memory allocator, and the thread-blocking mechanism.
https://dl.acm.org/doi/10.1145/1454456.1454462
This document contains interview questions for a Linux administrator role. It includes questions about shell scripting, system administration tasks, networking, and more. Some example questions are how to take input in a shell script, write a script to convert file path slashes, and explain the differences between UDP and TCP. The document provides technical questions to assess a candidate's Linux knowledge and experience.
The document provides an overview of a course on software engineering. It discusses key concepts like structured programming, object-oriented programming, design principles of abstraction and modularity. It also covers programming in languages like C and Matlab. The goals of the course are to understand basic program design techniques, produce well-structured programs, and have a basic understanding of object-oriented design.
The document describes a course on software engineering that covers basic design principles and techniques like structured programming, object-oriented programming, and data structures. It aims to give students an understanding of how to produce well-structured, maintainable code. Examples will be shown in MATLAB and C/C++. Key concepts covered include abstraction, modularity, procedural and object-oriented programming, functions, classes, and arrays.
This was a talk, largely on Kamaelia & its original context given at a Free Streaming Workshop in Florence, Italy in Summer 2004. Many of the core
concepts still hold valid in Kamaelia today
The document introduces Parallel Pixie Dust (PPD), a cross-platform thread library that aims to guarantee deadlock-free and race-condition free schedules that are optimal. It discusses the need for multiple threads due to factors like the memory wall. Current threading models are problematic because testing and debugging threaded code is difficult. PPD uses futures and thread pools to simulate data flow and generate tree-like thread schedules. It provides parallel versions of functions and thread-safe containers to enable multi-threaded standard library algorithms. The goal is to make writing correct multi-threaded programs easier.
This document discusses clustering PHP applications. It begins by defining computer clustering as linking computers to work together closely and act as a single computer. The document then discusses why clustering is used, including to service more users faster and increase reliability. It outlines objectives like linear capacity and cost increases with clustering as well as exponential reliability gains. The rest of the document summarizes various techniques for clustering PHP applications, such as load balancing, database scaling, replicated storage, backups, caching, sessions, staging strategies, and debugging clustered applications.
This document discusses the major tenets of the Linux philosophy. It explains that everything is treated as a file, there are small single-purpose programs, and programs can be chained together to perform complex tasks. Configuration data is stored in text files for easy editing and portability. Programs act as filters that take input, modify it, and output the results. The philosophy values portability over efficiency and uses software leverage through small reusable programs.
The document discusses the pros and cons of using open source software in projects. It notes that open source code comes with various licenses like GPL, BSD, and MIT that differ in how they can be used. While open source lacks perceived accountability, it actually has governance structures. The document aims to provide context on open source licensing and use within the scope of incorporating external code into a project. It addresses common concerns about open source like licensing complexity but clarifies the intent is to protect author rights, not hinder code use.
The first compilers were written by hand-translating programs into machine code opcodes. The earliest programs were written directly in binary opcodes by looking up each instruction and its corresponding opcode. Later, the first assemblers were created by hand to automate this process. These early assemblers could then be used to assemble more advanced assemblers and compilers. This process of using each new tool to build the next was called bootstrapping. The first high-level programming language was FORTRAN, created in 1954, which had an optimizing compiler that produced very efficient code and helped establish modern programming practices.
The document provides an overview of Linux interview essentials related to operating system concepts, system calls, inter-process communication, and threads. It discusses topics such as the role and components of an operating system, multi-tasking and scheduling policies, differences between function calls and system calls, static and dynamic linking, common code and stack errors, memory leaks, kernel modes, monolithic and micro kernels, interrupts, exceptions, system calls implementation in Linux, and synchronous vs asynchronous communication methods.
The document discusses parallel computing over the past 25 years and challenges for using multicore chips in the next decade. It aims to provide context to scale applications effectively to 32-1024 cores. Key challenges include expressing inherent application parallelism while enabling efficient mapping to hardware through programming models and runtime systems. Future work includes developing methods to restore lost parallelism information and tradeoffs between programming effort, generality and performance.
This document contains the questions and answers from a computer architecture and organization exam. It includes questions about the differences between computer architecture and organization, instruction formats, bus definitions, cache memory advantages, and virtual memory. The responses provide detailed explanations of concepts like locality of reference, thrashing, address mapping, cache hits and misses, and hierarchical memory systems. Justification is given for using a hierarchical approach to improve performance across different memory types. The differences between paging and segmentation in virtual memory are also distinguished.
The document describes an instructional operating system called Nachos that was developed for teaching undergraduate operating systems courses. Some key points:
- Nachos simulates a RISC CPU and devices to allow students to modify an operating system kernel without needing physical hardware. It runs as a UNIX process.
- The assignments guide students through implementing core OS functionality like threads, file systems, multiprogramming, and networking. This allows students to build their understanding of modern operating system concepts through hands-on projects.
- Nachos emphasizes simplicity over realism so students can understand the entire system. The authors found simplicity was important for students to grasp concurrency and other challenging OS concepts.
The document describes an instructional operating system called Nachos that was developed for teaching undergraduate operating systems concepts. Some key points:
- Nachos simulates a RISC CPU and devices to allow students to modify an operating system kernel as a project. It runs as a UNIX process for simplicity.
- There are 5 main assignments that cover thread management, file systems, multiprogramming, virtual memory, and networking. Each builds on concepts from previous assignments.
- The goal was to create a simple yet realistic system that illustrates modern OS techniques and allows quantitative evaluation of design tradeoffs through benchmarks.
Software and the Concurrency Revolution : NotesSubhajit Sahu
Highlighted notes of article while studying Concurrent Data Structures, CSE:
Software and the Concurrency Revolution
Herb Sutter
Software Architect, Microsoft
Software Development Consultant, www.gotw.ca/training
Herb Sutter is a prominent C++ expert. He is also a book author and was a columnist for Dr. Dobb's Journal. He joined Microsoft in 2002 as a platform evangelist for Visual C++ .NET, rising to lead software architect for C++/CLI.
Similar to "Hints" talk at Walchand College Sangli, March 2017 (20)
MYIR Product Brochure - A Global Provider of Embedded SOMs & SolutionsLinda Zhang
This brochure gives introduction of MYIR Electronics company and MYIR's products and services.
MYIR Electronics Limited (MYIR for short), established in 2011, is a global provider of embedded System-On-Modules (SOMs) and
comprehensive solutions based on various architectures such as ARM, FPGA, RISC-V, and AI. We cater to customers' needs for large-scale production, offering customized design, industry-specific application solutions, and one-stop OEM services.
MYIR, recognized as a national high-tech enterprise, is also listed among the "Specialized
and Special new" Enterprises in Shenzhen, China. Our core belief is that "Our success stems from our customers' success" and embraces the philosophy
of "Make Your Idea Real, then My Idea Realizing!"
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
How Social Media Hackers Help You to See Your Wife's Message.pdfHackersList
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
Quality Patents: Patents That Stand the Test of TimeAurora Consulting
Is your patent a vanity piece of paper for your office wall? Or is it a reliable, defendable, assertable, property right? The difference is often quality.
Is your patent simply a transactional cost and a large pile of legal bills for your startup? Or is it a leverageable asset worthy of attracting precious investment dollars, worth its cost in multiples of valuation? The difference is often quality.
Is your patent application only good enough to get through the examination process? Or has it been crafted to stand the tests of time and varied audiences if you later need to assert that document against an infringer, find yourself litigating with it in an Article 3 Court at the hands of a judge and jury, God forbid, end up having to defend its validity at the PTAB, or even needing to use it to block pirated imports at the International Trade Commission? The difference is often quality.
Quality will be our focus for a good chunk of the remainder of this season. What goes into a quality patent, and where possible, how do you get it without breaking the bank?
** Episode Overview **
In this first episode of our quality series, Kristen Hansen and the panel discuss:
⦿ What do we mean when we say patent quality?
⦿ Why is patent quality important?
⦿ How to balance quality and budget
⦿ The importance of searching, continuations, and draftsperson domain expertise
⦿ Very practical tips, tricks, examples, and Kristen’s Musts for drafting quality applications
https://www.aurorapatents.com/patently-strategic-podcast.html
INDIAN AIR FORCE FIGHTER PLANES LIST.pdfjackson110191
These fighter aircraft have uses outside of traditional combat situations. They are essential in defending India's territorial integrity, averting dangers, and delivering aid to those in need during natural calamities. Additionally, the IAF improves its interoperability and fortifies international military alliances by working together and conducting joint exercises with other air forces.
GDG Cloud Southlake #34: Neatsun Ziv: Automating AppsecJames Anderson
The lecture titled "Automating AppSec" delves into the critical challenges associated with manual application security (AppSec) processes and outlines strategic approaches for incorporating automation to enhance efficiency, accuracy, and scalability. The lecture is structured to highlight the inherent difficulties in traditional AppSec practices, emphasizing the labor-intensive triage of issues, the complexity of identifying responsible owners for security flaws, and the challenges of implementing security checks within CI/CD pipelines. Furthermore, it provides actionable insights on automating these processes to not only mitigate these pains but also to enable a more proactive and scalable security posture within development cycles.
The Pains of Manual AppSec:
This section will explore the time-consuming and error-prone nature of manually triaging security issues, including the difficulty of prioritizing vulnerabilities based on their actual risk to the organization. It will also discuss the challenges in determining ownership for remediation tasks, a process often complicated by cross-functional teams and microservices architectures. Additionally, the inefficiencies of manual checks within CI/CD gates will be examined, highlighting how they can delay deployments and introduce security risks.
Automating CI/CD Gates:
Here, the focus shifts to the automation of security within the CI/CD pipelines. The lecture will cover methods to seamlessly integrate security tools that automatically scan for vulnerabilities as part of the build process, thereby ensuring that security is a core component of the development lifecycle. Strategies for configuring automated gates that can block or flag builds based on the severity of detected issues will be discussed, ensuring that only secure code progresses through the pipeline.
Triaging Issues with Automation:
This segment addresses how automation can be leveraged to intelligently triage and prioritize security issues. It will cover technologies and methodologies for automatically assessing the context and potential impact of vulnerabilities, facilitating quicker and more accurate decision-making. The use of automated alerting and reporting mechanisms to ensure the right stakeholders are informed in a timely manner will also be discussed.
Identifying Ownership Automatically:
Automating the process of identifying who owns the responsibility for fixing specific security issues is critical for efficient remediation. This part of the lecture will explore tools and practices for mapping vulnerabilities to code owners, leveraging version control and project management tools.
Three Tips to Scale the Shift Left Program:
Finally, the lecture will offer three practical tips for organizations looking to scale their Shift Left security programs. These will include recommendations on fostering a security culture within development teams, employing DevSecOps principles to integrate security throughout the development
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.
Performance Budgets for the Real World by Tammy EvertsScyllaDB
Performance budgets have been around for more than ten years. Over those years, we’ve learned a lot about what works, what doesn’t, and what we need to improve. In this session, Tammy revisits old assumptions about performance budgets and offers some new best practices. Topics include:
• Understanding performance budgets vs. performance goals
• Aligning budgets with user experience
• Pros and cons of Core Web Vitals
• How to stay on top of your budgets to fight regressions
Details of description part II: Describing images in practice - Tech Forum 2024BookNet Canada
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and transcript: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/07/intels-approach-to-operationalizing-ai-in-the-manufacturing-sector-a-presentation-from-intel/
Tara Thimmanaik, AI Systems and Solutions Architect at Intel, presents the “Intel’s Approach to Operationalizing AI in the Manufacturing Sector,” tutorial at the May 2024 Embedded Vision Summit.
AI at the edge is powering a revolution in industrial IoT, from real-time processing and analytics that drive greater efficiency and learning to predictive maintenance. Intel is focused on developing tools and assets to help domain experts operationalize AI-based solutions in their fields of expertise.
In this talk, Thimmanaik explains how Intel’s software platforms simplify labor-intensive data upload, labeling, training, model optimization and retraining tasks. She shows how domain experts can quickly build vision models for a wide range of processes—detecting defective parts on a production line, reducing downtime on the factory floor, automating inventory management and other digitization and automation projects. And she introduces Intel-provided edge computing assets that empower faster localized insights and decisions, improving labor productivity through easy-to-use AI tools that democratize AI.
AC Atlassian Coimbatore Session Slides( 22/06/2024)apoorva2579
This is the combined Sessions of ACE Atlassian Coimbatore event happened on 22nd June 2024
The session order is as follows:
1.AI and future of help desk by Rajesh Shanmugam
2. Harnessing the power of GenAI for your business by Siddharth
3. Fallacies of GenAI by Raju Kandaswamy
4. Systems Research
Designing large and/or complex computing systems
computers (from PCs to supercomputers)
networks
printers
smartphones, tablets…
Includes the software:
operating systems
programming languages
networking stacks
file systems
distributed systems
etc.
5. Systems Research…
How it’s different from say, algorithm design:
external interface (requirement) is less precise, more complex, subject
to change…
more internal structure, thus more internal interfaces
module-level design choices have wider implications
measure of success is unclear
No such thing as a ‘best’ design
more important to avoid making terrible choices
Usually, the physical system doesn’t yet exist!
simultaneous development of the hardware and software
how do you design and test the software?
how do you design and test the hardware?
7. A 1-page Biography
Born in 1943, in Washington, DC
Education:
B.A. in Physics at Harvard, 1964
Ph.D. in EECS at Univ. of California at Berkeley, 1967, aged 23!
Work:
Xerox PARC, 1971-1984
DEC SRC, 1984-1995
Microsoft Research, 1995-
Systems:
Alto (workstation)
Bravo (WYSIWYG editor)
Cal (timesharing system)
Dover (laser printer)
Awards:
1992 ACM Turing Award
ACM Software Systems Award
IEEE Computer Pioneer Award
IEEE von Neumann Award
3 papers in ACM SIGOPS Hall of Fame!
8. The Xerox Alto
The first personal computer, built in 1973 at
PARC
First computer to use the desktop metaphor
First computer with a mouse-driven GUI
Conceived in a 1972 memo written by
Lampson
Design led by Charles Thacker (2009
Turing)
Lampson contributed to the design
Wrote the OS!
Lampson et al. won the 1984 ACM
Software System Award and the 2004 NAE
Draper prize for the Alto
Later work: led the design of Dorado and
9. The Bravo editor
First WYSIWYG editor
Shipped with the Xerox
Alto
Developed by Lampson,
Simonyi et al. in 1974
10. The Xerox 9700 and Dover Laser
Printers
First laser printer, at PARC, 1969, led by Gary
Starkweather
Lampson co-designed the electronics and
software
prototype for the Xerox 9700
Dover Laser Printer
later version, in 1976
much cheaper
Lampson designed the electronics
11. SDS 940
First general-purpose time-sharing system,
with char-by-char interaction
Project Genie at Berkeley
Lampson joined as graduate
student
wrote parts of the OS
created several programming
languages!
Cal: interactive language for
numerical computation
QSPL: system programming
12. Other Research
Two-phase commit protocol for distributed
transactions (with Sturgis)
Cal time-sharing system for CDC 6400 (at Berkeley)
pioneered shadow pages, redo logs
capability-based system: not a good basis for long-term
security
Programming Languages:
Mesa and SPL for systems programming
Mesa’s process mechanism modern thread systems
Cedar: combining the virtues of Mesa and Lisp
Euclid: first language designed to enable program
verification
Modula 2+: extends Wirth’s Modula-2
13. But wait, there’s more!
Security
Access Matrix model, unifying capabilities and ACLs
Information Flow Control
Theory of Principals “speaking for” other principals
Microsoft Palladium (TCB)
Scrubbing disk storage
How economic factors (not technology) inhibit security
Networking
co-inventor on Xerox patent for Ethernet!
switched LAN at DEC SRC
Formal specification and proof
TCP connection establishment (at-most-once messaging)
Leslie Lamport’s Paxos protocol
16. The Hints paper...
Presented at the 9th SOSP,
appeared in SIGOPS OSR October
1983, reprinted in IEEE Software Jan
1984
Summarizes the learnings from years of
systems research and implementation
Still influential after all these years... e.g.:
Werner Vogels, CTO, Amazon
Butler's paper shows a great mix of
fundamentals and best practices from the
early days of large scale system design.
Almost all of his advice has withstood the
test of time and as such they are even
more important now than in 1983
17. How it’s organized
“Hints”:
not laws
not foolproof recipes
not consistent
not always appropriate
in short, no guarantees
Each hint is:
summarized by a slogan
illustrated with examples from systems work
preceded by an appropriate quotation from Hamlet!
Hints for functionality (does it work?), speed (is it fast enough?) and fault
tolerance (does it keep working?)
Pre-req: Notion of an interface that separates an implementation of some
abstraction from the clients who use the abstraction
“Defining interfaces is the most important part of system design”
18. Do one thing at a time, and do it
well
KISS
Don’t generalize, generalizations
are generally wrong!
Interface should capture the
minimum essentials of an
abstraction
mustn’t promise more features
than the implementer knows
how to deliver (without
penalizing other clients)
Service should have a fairly
predictable cost
But, get it right!
Perfection is reached not when
there is no longer anything to
add, but when there is no longer
anything to take away!
A. Saint-Exupery
We are faced with an
insurmountable opportunity
W. Kelley
Everything should be made as
simple as possible, but no
simpler
A. Einstein
Hints for Functionality
19. Hints for Functionality...
Make it fast, rather than general or powerful
Analogous to the RISC principle...
Better to have fast, basic operations than slow, powerful
ones
Clients who don’t need the power shouldn’t have to pay
more for the basic functionality
As long as it’s fast, a client can program the additional
functionality it needs; another client can program some
other functionality that it needs!
Don’t hide power
If a low-level abstraction allows something to be done
quickly, higher levels should not bury this power inside
something more general
abstractions should conceal only undesirable properties!
20. Hints for Functionality...
Use procedure arguments to provide flexibility in an
interface
restricted or encoded as necessary
e.g. an interface for finding all elements of a set that satisfy
a property
let clients pass in a filter procedure vs. a special language of
patterns
Leave it to the client
keep the interface simple, flexible and high-performance
caveat: as long as it is cheap to pass control back and
forth
e.g. success of monitors for synchronization, because
locking and signalling do very little
clients take care of process scheduling, buffer allocation,
resource accounting, etc. or use other libraries for that
21. Hints for Functionality...
Keep basic interfaces stable
embodies assumptions shared by many parts of a
system
if type-checking is available, may be possible to check
for number and types of arguments
but still requires programmers to rework the integration
may not detect semantic changes in interfaces
Keep a place to stand...
... if you do change interfaces
e.g. a compatibility package, implements an old
interface on top of the new one
e.g. OS simulators
dev cost may be less vs. cost of fixing all client software
performance hit, but often acceptable
22. Hints for Functionality...
Plan to throw one away...
... you will, anyhow!
Keep secrets...
assumptions about an implementation that clients aren’t allowed
to make
tension vs. the desire not to hide power
tension vs. the need for performance
Divide and conquer
reduce a problem to several easier ones
bite off as much as will fit, leaving the rest for the next iteration
e.g. Alto’s file system defragmentation and its use of memory
Handle normal and worst cases separately
the normal case must be fast, the worst case must make
progress
e.g. caches and hints help the normal case
e.g. Garbage Collector in Cedar doesn’t count refs in local
frames; those are scanned completely during GC
23. Hints for Speed
Split resources in a fixed way, rather than sharing them
faster to allocate, faster to access, more predictable
e.g. using register banks vs. memory
cost of extra resources is usually low
multiplexing overheads may be larger than the fragmentation waste
Dynamic translation...
... from a convenient representation to one that can be quickly
interpreted
e.g. Smalltalk compiler generates bytecodes, implementation translates
(and caches) a single procedure’s bytecodes to machine language when
invoked
Cache answers to expensive computations
store the triple [f, x, f(x)] in an associative store
basic example: processor cache or virtual memory systems
[Fetch, address, contents of address]
need to ensure cache invalidation/update for non-functional f() such as
Fetch()
24. Hints for Speed...
Use hints
like a cache entry (saved result of some computation), but:
not necessarily reached by associative lookup
may be wrong!
if using a hint for unrecoverable actions, need to check if
it’s wrong
e.g. store-and-forward routing tables in Arpanet
based on each node’s opinion about its links to neighbours
periodically broadcast (can be lost, or delivered out of order)
When in doubt, use brute force
hardware is cheap (even in 1983!)
brute force allows for cheaper, faster implementation
e.g. Ken Thompson’s chess-playing computer uses brute-
force vs. sophisticated chess strategies
25. Hints for Speed...
Compute in the background where possible
especially in interactive or real-time systems
e.g., Cedar garbage collector, e-mail delivery, etc.
Use batch processing if possible
doing things incrementally usually costs more
disks and tapes work better when accessed
sequentially
e.g. banking systems use online data, but discard
it after nightly batch
26. Hints for Speed...
Safety first
in allocating resources, avoid disaster vs. being
optimal
“a general-purpose system cannot optimize the use of
resources”
again, “hardware is cheap, and getting cheaper”
cleverness works only if you have very well-known
loads
“The nicest thing about the Alto is that it doesn’t run
faster at night” – Morris
no fancy processor scheduling, fixed share of cycles to
each job
27. Hints for Fault-Tolerance
End-to-end
Saltzer et al. classic paper, 1981 ICDCS
error recovery at app level is necessary for a reliable system
any other error detection or recovery is strictly for performance
e.g. file transfer across a network from A to B
reading from B’s disk and validating checksum against A’s disk is necessary
checking the transfer from A’s disk to A’s memory, or from A to B over the
network, is not sufficient
not necessary either, but can help performance (retransfer only corrupted
parts)
Log updates to record the truth about an object’s state
current state of object is treated like a hint
log entry must include update procedure (functional) and its arguments
sequence of log entries can be re-executed if necessary to re-create the
true state
Make actions atomic or restartable
atomic: failure during the action has no effect
log entries need to be restartable / idempotent
can be partially executed any number of times before a complete execution
28. In Conclusion...
Most humbly do I take my leave, my lord.
“Such a collection of good advice and
anecdotes is rather tiresome to read; perhaps
it is best taken in small doses at bedtime”!
“I can only plead that I have ignored most of
these rules at least once, and nearly always
regretted it”
29. For more information…
http://amturing.acm.org/award_winners/lampso
n_1142421.cfm - Profile of Lampson on ACM
site
http://amturing.acm.org/bib/lampson_1142421.
cfm - citations for the 3 SIGOPS HoF papers
http://research.microsoft.com/en-
us/um/people/blampson/default.htm -
Lampson’s homepage at MSR
Editor's Notes
e.g. Alto OS has simple file system interface (read/write n bytes to file), and paging system that stores each virtual page in a dedicated disk page. Pilot system allows virtual pages to be mapped to file pages, subsuming file IO into virtual memory. Much more complex and slower implementation.
“Make it fast”: RISC processors vs. others like VAX with more general/powerful instruction sets.
“Don’t hide power”: Alto disk can transfer a full cylinder at disk speed. File system is aware of this, and exploits it for performance.
“Procedure args”: e.g. sort routine that has to compare elements – numeric? ASCII? Lexicographic?. Pass a procedure rather than encoding the possibilities.
“Keep secrets”: e.g. if a set is known to be sorted, a test for membership can take log n time, rather than n.
“Divide and conquer”: Alto Scavenger rebuilds file system index and directory structure. Builds a data structure in memory, mapping disk pages to file pages. If disk is fragmented, the struct will not fit in memory. So, discard info for half the files and continue with the other half. If necessary, repeat.
“Normal and worst cases”: Bravo editor piece table. Doc stored as array of pieces (strings). Editing results in adding pieces. Occasional defrag.
“Cache answers”: e.g., f = sort, x = a set of numbers. Can cache f(x), the sorted array of numbers in x.
Cache size can be adaptive, to ensure a good hit rate.