About Me
I'm Laurence Gonsalves, and this is my blog. I'm an independent software developer, a maker, and also a dad.
Rasterization
Rasterization is the process of determining the set of pixels that will be drawn to. This is accomplished by generating a “maskâ€, which is a alpha-channel image. Opaque (0xFF) pixels on this mask indicate areas we want to draw to at “full strengthâ€, transparent (0x00) areas are areas we don't want to draw to at all, and partially transparent areas will be drawn to at “partial strengthâ€. This is explained more at the end of the final phase. (When visualizing this process I find that it helps to think of opaque as white and transparent as black.)
Rasterization has two completely different behaviors depending on whether a Rasterizer has been set on the Paint.
If no Rasterizer has been set then the default rasterization process is used:
Path is scan-converted based on parameters from the Paint (eg: the style property) and the Path (eg: the fillType property) to produce an initial mask.
Pixels “inside†the Path will become opaque, those “ outside†will be left transparent, and those on the boundary may become partially transparent (for anti-aliasing). The mask will end up containing an opaque silhouette of the object.
The Path object's fillType determines the rule used to determine which pixels are inside versus outside. See Wikipedia's article on the non-zero winding rule for a good explanation if these different rules.
MaskFilter set, then the initial mask is transformed by the MaskFilter. The MaskFilter is essentially a function that takes a mask (an ALPHA_8 Bitmap) as input and returns a mask as output. For example, a BlurMaskFilter will blur the mask image.
If no MaskFilter is set then the initial mask is passed on to the next phase unmodified. That is, the default MaskFilter is the identity function.
If a Rasterizer is set on the Paint then, instead of the above two steps, the Rasterizer creates the mask from the Path. The MaskFilter is not invoked after the Rasterizer. (This seems like a bug, but I've verified this behavior experimentally. Romain Guy agreed that this is probably a bug.)
The only Rasterizer implementation in Android is LayerRasterizer. LayerRasterizer makes it possible to create multiple “layersâ€, each with its own Paint and offset (translation). This means that when n LayerRasterizer layers are present there are n + 1 Paint objects in use: the “top-level†Paint (passed to the draw* method) and an additional n Paint objects, one for each Layer.
LayerRasterizer takes the Path and for each layer runs the Path through the pipeline of that layer's Paint starting at the PathEffects step and rendering to the mask. This has some interesting consequences:
Each layer can have its own PathEffect. These are applied to the Path that was generated by the top-level PathEffect (if one was set). So if the PathEffect of the top-level's Paint is set to a CornerPathEffect and a layer's PathEffect set to DashPathEffect that layer will render a dashed shape with rounded corners.
Each layer can have its own Rasterizer. Recursive rasterization is recursive.
Each layer can have its own MaskFilter. This MaskFilter applies to a separate mask in the sub-pipeline. Remember, the entire pipeline is being run again. For example, if there are two layers and one has a BlurMaskFilter the output of the other layer will not be blurred regardless of the order of the layers.
The destination Bitmap of this sub-pipeline is an alpha bitmap, so only the alpha-channel component of the Shading and Transfer phases have any relevance.
Also note that LayerRasterizer does not make use of the MaskFilter in the top-level Paint. Since the top-level MaskFilter is not invoked after invoking the Rasterizer, there is no point in setting a MaskFilter on a Paint if the Rasterizer has been set to a LayerRasterizer. (Perhaps other Rasterizer implementations could make use of the top-level MaskFilter, but LayerRasterizer is the only implementation included with Android.)
Shading
Shading is the process of determining the “source colors†for each pixel. A color (can) consist of alpha, red, green, and blue components (ARGB for short) each of which ranges from 0 to 1. (In reality these are typically represented as bytes from 0x00 to 0xFF.)
At a high level, the output of the Shader can be thought of as a virtual image containing the source colors: the “source†image. The actual implementation doesn't use a Bitmap, but rather uses a function that maps from (x,y) to an ARGB color (the “source colorâ€) for the given pixel, and this function is only called for coordinates where the corresponding pixal may be altered by the source color. This is really just an optimization, however.
Like the previous phases, Shading also has two sub-phases:
An initial “source†image is generated by the Shader. If no Shader has been set it's as if a Shader that produced a single solid color (the Paint's Color) was used.
The Shader does not get the mask, the Path, or the destination image as inputs.
If a ColorFilter has been set then the colors in the source color image are transformed by this ColorFilter.
The only input to the ColorFilter during the pipeline are ARGB colors. The ColorFilter does not get the mask, the Path, the destination image, or the coordinates of the pixel whose color it is transforming, as inputs.
Transfer
Transfer is the process of actually transferring color to the destination Bitmap. The transfer phase has the following inputs:
The mask generated by Rasterization.
The “source color†for each pixel as determined by Shading.
The destination bitmap, which tells us the “destination color†for each pixel.
The transfer mode (XferMode).
Once again, there are two sub-phases:
XferMode. This function takes the source color and destination color and returns the color for the intermediate image's pixel at (x,y).
Note that the mask is not used in this sub-phase. In particular, the source-alpha comes from the Shader, and the destination alpha comes from the destination image.
If an XferMode hasn't been set on the Paint then the behavior is as though it was set to PorterDuffXferMode(SRC_OVER).
The second sub-phase takes the intermediate image, the destination image, and the mask as inputs and modifies the destination image. It does not use the XferMode.
The intermediate image is blended with the destination image through the mask. Blending means that each pixel in the destination image will become a weighted average (or equivalently, linear interpolation) of that pixel's original color and the corresponding pixel in the intermediate image. The opacity of the corresponding mask pixel is the weight of the intermediate color, and its transparency is the weight of the original destination color.
In other words, a pixel that is transparent (0x00) in the mask will be left unaltered in the destination, a pixel that is opaque (0xFF) in the mask will completely overwritten by the corresponding pixel in the intermediate image, and pixels that are partially transparent will result in a destination pixel color that is proportionately between its original color and the color of the corresponding intermediate image pixel.
This is the final phase. The pipeline is now complete.
More on Porter Duff Transfer Modes
The most commonly used transfer modes are instances of PorterDuffXferMode. The behavior of a PorterDuffXferMode is determined by its PorterDuff.Mode. The documentation for each PorterDuff.Mode (except OVERLAY) shows the function that is applied to the source and destination colors to obtain the intermediate color. For example, SRC_OVER is documented as:
[Sa + (1 - Sa)*Da, Rc = Sc + (1 - Sa)*Dc]
This means:
Ra = Sa + (1 - Sa) * Da Rr = Sr + (1 - Sa) * Dr Rg = Sg + (1 - Sa) * Dg Rb = Sb + (1 - Sa) * Db
Where Rx, Sx, and Dx are the intermediate (result), source and destination values of the x color component.
Some additional notes on the PorterDuff.Mode documentation:
The documentation uses “Sc†and “Dc†rather than describing each red, green, and blue component separately. This is because Porter Duff transfer modes always treat the non-alpha channels the same way and each of these channels is unaffected by all other channels except for alpha.
SRC_OVER and DST_OVER are the only two modes that have the left hand side of this equation, “Rcâ€, in their documentation. I'm guessing this inconsistency is a copy-and-paste error.
The alpha channel is always unaffected by non-alpha channels. That is, Ra is always a function of only Sa and Da.
The documentation for ADD refers to a “Saturate†function. This is just clamping to the range [0,1]. (I don't know why they use such an odd name for clamping, especially “saturation†usually refers to an entirely unrelated concept when talking about colors.)
The definition of many of these modes, including OVERLAY, can be found in the SVG Compositing Specification. The Skia code actually links to (an older version of) this document. It has some good diagrams, too.
References
android.graphics documentation. This answer to “Android Edit Bitmap Channels†on Stack Overflow. Seeing this answer motivated me to learn more about how the pipeline actually works. The Android codebase. Since the documentation was so sparse and there didn't seem to be much information I looked to the source. My initial look stopped short when I realized everything was just a wrapper around “native†code. Skia documentation, particularly SkPaint. Skia is the vast bulk of “native†(C++) code involved. “Stack Overflow: How do the pieces of Android's (2D) Canvas drawing pipeline fit together?â€, a question I asked on Stack Overflow. One member of the Android team actually responded, but didn't really provide the details I was looking for. The Skia codebase. The code for SkCanvas::drawPath is a good place to start. SVG Compositing Specification: W3C Working Draft 30 April 2009. This document is mentioned in the Skia code. SVG Compositing Specification: W3C Working Draft 15 March 2011. This document supercedes the one mentioned in the Skia code. I believe the relevant bits still apply, but there's more detailed explanation and some good diagrams.