Using Parallel Computing Platform - NHDNUG

Editor's Notes

1. Let’s use this slide for an “Architectural Perspective” of UMS.<CLICK>S1 and S2 are the first threads created within a UMS solution. These are “Scheduler Threads” or “Primary Threads”. These threads represent “core” or physical CPU’s from a Scheduler perspective. These are normal threads to begin with, but you would typically first establish processor affinity using the new CreateRemoteThreadEx API and the use a new API, EnterUmsSchedulingMode, to specify that the new thread is a Scheduler thread.You pass in a callback, i.e. UMSSchedulerProc, function pointer to begin executing instructions on the Scheduler thread.A UMS worker thread is created by calling CreateRemoteThreadEx with the PROC_THREAD_ATTRIBUTE_UMS_THREAD attribute and specifying a UMS thread context and a completion list. The OS places these threads into the Completion List and your Scheduler logic takes over typically placing the new threads onto the Scheduler’s Ready List.<CLICK>The first thing that a Scheduler should do is move it’s associated Worker threads onto the Scheduler’s Ready List. Then, it can began executing your customer scheduler logic.<CLICK>Each of the Scheduler threads should then pop a Worker thread off of the Ready List and run it on the associated Core. When this occurs, the Scheduler thread context is essentially lost forever… the Worker thread now owns the core and is executing. The Scheduler thread will not regain the core until a processor Yield event occurs.<CLICK>The first thing that could happen is that this thread could yield. Yield is again a Scheduler callback mechanism and perhaps the single most important function of UMS. It’s within the Yield that you will implement your own synchronization primitives and scheduling logic.Ideally, the yielding thread provides some contextual information to the scheduler (maybe it wants to wait on some specific application domain event to occur). Your Scheduler would look at this Yield request and associated context and make a scheduling decision.<CLICK>Maybe the Scheduler places the Worker thread within a Wait list for that specific event or event type.Now your Scheduler has to decide what to run next. <CLICK>Maybe the next Worker thread from the Ready List, for instance… and we’re back running again. Note, that no kernel scheduling context switch was necessary. Maybe that wait event handling took 200 cycles in user-mode. It may have cost 10 times that with a kernel context switch.<CLICK>Let’s now assume that this worker performs a system call… At this point, we switch the worker thread to it’s kernel-mode context and the thread continues to run within the kernel. If it does not block (in other words, if it doesn’t use one of the kernel synchronization primitives, then it just continues to run. If the thread never blocks in the kernel, then it just returns to user-mode and continues to run and do work.<CLICK>Let’s assume that the thread does block. Maybe a page fault occurred, for instance. Now our Scheduler thread regains control of the processor via a callback from the kernel. Now, the kernel is telling your Scheduler that a worker thread is blocked and the reason for that block. This is the point where we integrate kernel synchronization with user synchronization. But now, you get to decide what to run next.<CLICK>The Scheduler looks at the state of it’s affairs and perhaps decides to run the next Worker thread from the Ready List, for example.Let’s assume that later in time Worker 3 unblocks. <CLICK>The kernel will now place this unblocked Worker thread into the UMS Completion List.<CLICK>At the next Yield event, for instance, we get another Scheduler decision opportunity. Maybe this Yield contains information that affects the state of our Wait list.<CLICK>The first thing that the Scheduler should do, however, is manage the Completion List and move any unblocked threads to the Ready List.<CLICK>Next, our Scheduler must make a priority decision. Maybe our Waiting thread gets to run again and our Yielding thread gets placed upon the Ready List.And we’re done…
2. UMS is an enabler for:Finer-grained parallelismMore deterministic behaviorBetter cache localityUMS allows your Scheduler to boost performance in certain situations:Apps that have a lot of blocking, for example
3. Think Tasks not Threads.Threads represent execution flow, not workHard-coded; significant system overheadMinimal intrinsic parallel constructsQueueUserWorkItem() is handy for fire-and-forgetBut what about…WaitingCancelingContinuingComposingExceptionsDataflowIntegrationDebugging
4. NOW, LET’S FIRST CONSIDER THE TOOLS ARCHICTECTURE FROM A .NET DEVELOPER’S PERSPECTIVE.LET’S START WITH THE .NET Runtime AND THE .NET Parallel Extensions library. In a moment, we’ll look at how a developer uses the extensions within their application. The .NET PARALLEL EXTENSIONS provide the benefits of concurrent task scheduling without YOU having to build a custom scheduler that is appropriately reentrant, thread-safe, and non-blocking.<CLICK>The Parallel Extensions library contains a Task Scheduler and a Resource Manager component that integrates with the underlying .NET Runtime. The Resource Manager manages access to system resources like the collection of available CPU’s. <CLICK>The Scheduler leverages only thread pools for task scheduling. <CLICK>The Parallel Extensions also supports multiple Programming Models. <CLICK>The Task Parallel Library (TPL) is an easy and convenient way to express fine-grain parallelism within your applications. The TPL provides patterns for Task Execution, Synchronization, and Data Sharing.<CLICK>The PLINQ (or Parallel LINQ) enables parallel query execution not only on SQL Data but also on XML or Collections Data.<CLICK>The Parallel Extensions also includes Data Structures that are “scheduler aware” enabling you to optimally specify task scheduling requests and custom scheduler policies.<CLICK>Again, Visual Studio 2010 includes new tools for parallel application development and testing. These include:<CLICK>A new parallel debugger. And…<CLICK>A new parallel application profiler.Let’s take a brief look at a simple .NET parallel application along with the Visual Studio 2010 Debugger and Parallel Performance Analyzer.Pure .NET librariesFeature areasTask Parallel LibraryParallel LINQSynchronization primitives and thread-safe data structuresEnhanced ThreadPool