1. OpenCL Components:
   • C Host API: basically saying what devices do I want to use, what do I want them to do, what functions I want to call, where should memory be
     • What you call from the host
     • Directs devices
   • OpenCL C
     • Used to program the device
     • Based on C99
     • Many built-in functions
   • Models
     • Memory, execution, etc

如上图，the host is going to call the host API to manage devices on the right. Devices are programmed using OpenCL C. Underneath all of these are models. These models are here to guide everything.

2. OpenCL Model:
   • Device Model: what devices look like inside
     • Inside the Device:
       
       The device is broken down into further pieces, each of the small rectangle is a compute unit (CU), in this picture we have 15 compute units, and we have 8 processing elements per CU
     • Inside the Compute Unit:
       
       PE stands for processing element. Let’s take apart one of these blocks of the PE and private memory and see what that’s about.
     • Inside the Processing Element (PE):
       Think of the PE as a very simple processor. In particular, all instructions are executed on the processing element (means that everything that you are going to do in terms of actually making devices do work, the PE is going to be responsible for all of that.
   • Execution Model: How work gets done on devices
     • Kernel Functions:
       • OpenCL executes kernel functions on the device. The kernel functions are just ordinary functions with a special signature
       • Kernel calls have two parts:
         • Ordinary function argument list
         • External execution parameters that control parallelism
           
     • Role of the host in kernel execution
       • Coordinates execution (the host tells the device to call this function, but does not participate itself)
       • Provides arguments to the kernel ( the host tell the device what to do and to provide it arguments)
       • Provides execution parameters to launch the kernel
     • NDRange: execution strategy
       • The same kernel function will be invoked many times
         • The argument list is identical for all invocations
       • Basically we call the same function over and over
         • How many times we do this is dictated by the execution parameters.
       • Host sets extra execution parameters prior to launch
     • NDRange: Identifying the call
       • How do kernel functions know what to work on?
         • The argument list is identical
       • Insight: execution paramenters provide an index space
         • each function invocation can access its index
       • The index space is n-dimensional
     • NDRange: Some Definitions
       • Work-item: invocation of the kernel for a particular index
       • Global ID: globally unique id for a work-item (from index space)
       • Global Work Size: the number of work-items (per dimension)
       • Work Dimension: dimension of the index space
       • Work-groups: Partition the global work into smaller pieces. Work-groups execute on compute units, work-items (inside a work-group) mapped to CU PEs. All work-items in a work-group share local memory
         • work-group size has a physical meaning: it is device specific
           • Maximum work-group size is a device characteristic: you can query a device to determine this value
           • Maximum work-group size is an integer: Handle n-dimensional work-groups in a special way
           • How to determine the best work-group size: this is too advanced for now
         • work-items can find out: their work-group id, size of work-groups, global id, global work size
       • The work-item perspective: each work-item has its own private memory, all of the work-items within the work-group or compute unit are able to share the local memory. Every work-item on the device can access the constant memory and the global memory
         
     • 一些Kernel Call Points
       • Host will provide execution dimensions to the device, this creates an index space
       • Parameters can be values or global memory objects
       • Global memory is persistent between calls. But constant、local、private memory is just scratch space, they are going to be reset per kernel call
       • OpenCL implementation has considerable flexibility:
         • How to map work-items to PEs?
         • How to schedule work?
   • Memory Model: How devices and host see data
     • Global Memory: where you load data and run functions
       • Shared with all processing elements
       • Host can access this memory too
         • Memory map
         • Copy data to/from global memory
       • This is OpenCL persistent storage (the memory remains across subsequent executions)
         • Other memory regions are scratch space
     • Constant Memory:
       • Shared with all processing elements
       • Read-only memory
       • Very different way to share data with all device PEs
       • Not persistent (will change over time)
     • Local Memory:
       
       • Shared with all PEs in a CU
       • Very efficient way to share data with all CU PEs
       • Cannot be accessed by other compute units
       • Not persistent
     • Private Memory:
       • Accessible by a single processing element (PE)
       • No other PE can access this memory
       • Not persistent
   • Host API: How the host control the devices

     • Platform

       • A platform is an implementation of OpenCL
       • Platforms are like drivers for particular devices: platforms expose devices to you
       • Example: A system with two GPUs (AMD+nVIDIA) and a Xeon Phi (Intel)
         • A platform from AMD for one GPU and the CPU
         • A platform from Intel for the Xeon Phi
         • A platform from nVIDIA for the other GPU
       • Use the platform to discover devices available to you

     • Context: when you write an OpenCL program, creating a context is the first thing you do. What you’re going to do is: discover the platform -> get a context -> start locating memory -> start controlling devices

       • You create a context for a particular platform (you cannot have multiple platforms in a context)
       • A context is a container:
         • Contains devices
         • Contains memory
       • Most operations are related to a context (Implicitly or explicitly)

     • Program:

       • Programs are just collections of kernels (you extract kernels from your program to call them)
       • OpenCL applications have to load kernels
         • Compile OpenCL C source code
         • Load binary representation
       • Programs are device specific

     • Asynchronous Device Calls:
       The host manages devices asynchronously. You can have multiple devices attached to your host (for example you may have a Xeon Phi, an AMD GPU, an Nvidia GPU and you can use a CPU as another device). Now you want to manage all of these devices asynchronously for best performance. OpenCL has an asynchronous interface to do this.

       • Asynchronous Device Management
         • Host issues commands to device
         • Commands tell the device to do something
         • Device take commands and do as they say
         • Host waits for commands to complete: this means the device has completed that action
         • Commands can be dependent on other commands
         • OpenCL commands are issued by clEnqueue* calls:
           • A cl_event object returned by clEnqueue* calls is used for dependencies
       • Command overview:
         
         • clEnqueueFoo enqueue the command “Foo” to run on a particular device
         • e1 is a handle to represent this command
         • {deps}: this is a set of previously issued commands that have to be finished before. Commands take a list of dependencies
       • An example:
         
         I have 2 commands I’m going to run called Foo and 1 command called bar. e1 and e2 have no dependencies because their dependent set is empty. But the command bar cannot be completed until these two previous calls to Foo have been finished. In real life, Foo might be doing memory copies and bar might be a kernel
       • Where do commands go? Now we have talked about enqueueing thins but haven’t really saied where they go or what they do
         • OpenCL has command-queues
         • A command-queue is attached to a single device
         • You can create as many command-queues as you want
         • clEnqueue* commands have a command-queue parameter
           

           1

           
           

           2

     • Host API Summary:

       • Host API controls the device (Devices can’t do anything themselves)
       • Asynchronous execution model
         • Important for speed
         • A bit different from traditional asynchronous APIs (because of the command queue system and everything else)
3. Mapping NDRange to Devices
   • Remember the PE runs instructions
     • So work-items should run on PEs
   • Assign multiple work-items to each PE
     • Need to handle the case that global work size > number PEs
   • Partition the global work into smaller pieces (work-groups)
     • Work-groups execute on compute units. All work-items in the work-group share local memory and mapped to CU PEs.
4. Conceptual Work-Group Launching
   
5. Geometric Visualization:
   • 1D:
     
   • 2D:
     
   • 3D