Best Practice VDP DeveloperEdition
Best Practice VDP DeveloperEdition
Best Practice VDP DeveloperEdition
i
Merge equivalent spot colors 32
Avoid overprinting spot colors 33
Optimizing VDP layouts 35
Place graphics in consistent locations whenever possible 35
Avoid interleaving static and variable elements on a page 37
Minimize object overlaps 38
Nest ‘forms’ and images appropriately 40
Don’t mix variable and static data in form XObjects 40
Don’t draw the same graphic multiple times 41
Don’t add unnecessary white fills 41
Security marking 42
Managing jobs with many unique designs 42
Use standards when they’re relevant 43
Instant improvement 44
Relevant standards 45
PDF/X (ISO 15930) 45
PDF/X conformance levels 47
PDF/VT (ISO 16612-2 and 16612-3) 48
PDF/VT conformance levels 48
Key advantages of PDF/VT 50
PDF 2.0 (ISO 32000-2) 50
Print product metadata for PDF (ISO 21812-1 )
[11]
53
PDF PROCESSING STEPS (ISO 19593-1[10]) 55
Bibliography 57
ii
Introduction
The use of variable data printing is expanding across multiple sectors of
the print industry, from its roots in transactional, through commercial,
wide format, labels, packaging and into industrial print. It’s one of the
key reasons for adoption of digital printing and can help the print service
provider, converter or manufacturer to increase their profitability.
But that profitability often requires that the digital press is kept running
at full engine speed, and the way in which a variable data PDF file has
been designed and constructed can have a significant influence on
whether that’s possible or not. The same visual appearance can be
achieved with various different ways of constructing the file. Some of
those ways will be much more efficiently processed in the Digital Front
End (DFE) for the press than others.
This guide, and its companion for designers (Best Practice in creating
PDF files for variable data printing - Designer Edition), are therefore
intended to help you, as a designer, composition tool operator or
software vendor, to avoid creating inefficient PDF files.
Martin Bailey
Distinguished Technologist, Global Graphics Software
This document was developed under the auspices of the PDF
Association’s Print Product Metadata (PPM) Liaison Working Group (LWG)
and the PDF/VT Technical Working Group (TWG). Questions or
suggestions for improvement are welcome at info@pdfa.org.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 1
Why should I read and act on this guide?
If you’re a software developer working on composition or performance
enhancing tools for variable data print workflows you’ll probably already
know that the efficiency of the PDF files that your code produces is a key
part of its value to your customers. This guide will give you a lot of the
information you need to ensure that those PDF files can be processed
(rendered and printed) as quickly as possible without compromising
their visual appearance.
If you work in print production it will help you understand what might be
happening when PDF files process slowly, and to talk to the composition
tool and or press vendors about possible options for improvement.
But if you’re a designer, on the sharp end of creating PDF files for print,
you will probably want to read the separate application note that’s
specifically written for you to get the overview of how these issues relate
to design software first. Of course, you’re always welcome to come back
here and dig deeper afterwards.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 2
What is variable data printing?
For the purposes of this guide, “variable data printing” is any job where
all the pages are related (in other words you’re not simply printing a
huge number of separate jobs), but where many of the pages are similar
but nonetheless unique.
That uniqueness may be in only a small way – maybe only a single
graphic or line of text that’s different to every other instance of the same
underlying page design – while the majority of the artwork and content
is the same. Or the uniqueness may be very significant, for example
when printing photo books, or designs where the graphics are all
variations from a small number of ‘seed’ designs.
The variation may be for process control; shipping and returns; security;
customer- or campaign-driven or for any other reason. Most of this guide
(and all of the recommendations in chapter 12) are applicable for any of
those.
This guide will talk about ‘pages’, even though many print sectors don’t
print ‘pages’ of output, for example in labels, packaging, industrial print
etc. If you’re not printing pages, please simply mentally translate to
‘label’ or ‘carton’ or whatever else is appropriate for your use case.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 3
Why does optimization of VDP jobs matter?
Let’s assume that you’re printing, for example, 50 copies of a series of
booklets, or of an imposed form of labels. In this case the DFE (Digital
Front End) on your digital press only needs to RIP (Raster Image
Processor) each PDF page once and then print the same raster 50 times.
Let’s assume that you’re printing static jobs on a press that can produce
100 pages per minute (or the equivalent area for labels etc.). If all your
jobs are 50 copies long, you therefore need to RIP jobs at only two pages
per minute (ppm): 100ppm/50 copies. Once a job is fully RIPped and the
copies are running on the press you have plenty of time to get the next
job prepared before the current one clears the press.
VDP jobs place additional demands on the processing power available
in a DFE because most pages are different to every other page and must
therefore each be RIPped separately. If you’re printing at 100 pages per
minute the DFE must RIP at 100 pages per minute; fifty times faster than
needed for processing fifty copies of a static job.
Each minor inefficiency in a VDP job will often only add between a
few milliseconds and a second or two to the processing of each page,
but those times are multiplied by the number of pages in the job. An
individual delay of half a second on every page of a 10,000-page job adds
up to around an hour and a half for the whole job. For a really big job of a
million pages it only takes an extra tenth of a second per page to add 24
hours to the total processing time.
If you’re printing at 120ppm the DFE must process each page in an
average of half a second or less to keep up with the press. The fastest
continuous feed inkjet presses at the time of writing are capable of
printing an area equivalent to over 13,000 pages per minute, which
means each page must be processed in just over 4ms. It doesn’t take
much of a slow-down to start impacting throughput.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 4
High level view of VDP optimizations: RIP once, use
many times
A very short run of a non-variable job on a digital press tends to mean
that you’re producing at least a few dozen copies. In other words, each
page is processed once in the DFE (color managed, RIPped, halftone
screened etc.), and then sent multiple times to the press. The DFE
doesn’t need to process pages at the same speed that the press engine
can print them.
But if you’re printing a variable data job it’s likely that many pages will be
unique; most pages will be at least slightly different to every other page.
Obviously, this is not a universal rule; if you’re printing invoices, for
example, it’s common for the back of every sheet to be the same as the
back of every other … but even in that case there may be an invoice
number or date added onto the back of the sheet.
The DFEs for many digital production presses include optimizations
designed specifically to reduce the processing requirements for VDP
jobs.
When a VDP piece is designed, a variety of assets of various forms are
collected together. Some assets are intended to be used multiple times,
while others are associated with a single recipient or personalization.
Both single- and multiple-use assets may include images, graphics (for
example barcodes, maps, logos etc.) and, of course, text.
All of the assets are placed and positioned according to a set of rules.
Those rules might be as simple as a mail merge in Microsoft Word,
where placeholders are included in a template for the document, and
then replaced with text from a separate data file. In more sophisticated
environments additional information from a database about each
recipient is used to select from the assets available.
A classic direct marketing example is a mailer sent out to people who
have previously bought a particular make of car, inviting them to come
in and view this year’s model. Each piece might include a photograph of
a car of the same class as the one they purchased, and perhaps even in
PDF Association - Best Practice: PDF files for VDP - Developer Edition 5
the same color. Thus, if they had bought a sedan, they’d see an image of
this year’s sedan; if they bought a sports car, they’d see a sports car.
In addition, there might be a map to the dealer that they bought
from last time, the name and contact details for an appropriate sales
representative, and so on.
All of the assets required to reproduce the pages are included in the
PDF file sent for printing. The PDF can be viewed in any PDF reader and
would display as a series of fully laid out pages. It could be processed
through a DFE in that way as well … but often not at high enough speed
to keep the press running at full engine speed.
A DFE can optimize how it handles VDP jobs by processing any graphical
elements that are reused on many pages just once and then reusing
the already processed versions of the graphical elements. Processing
here typically means applying all color management and rendering to a
raster format at the same resolution, and using the same colorants, as
the digital press. The process of combining reused components with the
remaining page content can be done in software, on a GPU, or on the
driver board for the digital press itself.
In a well-designed and implemented system the overhead for splitting
the job apart and then re-compositing the rendered components again is
a lot less than the time saved by not having to process each asset in the
job hundreds, thousands or even millions of times.
Various products and technologies approach this optimization process
in different ways. Some, for example, identify reused elements by relying
on the encoding of elements inside the PDF file while others also review
all graphics and identify sequences that are repeated regardless of how
they are structured into objects within the file.
PDF provides a method to reuse graphical assets using the concept of
XObjects. Each XObject can hold a collection of graphics such as images,
text and vector graphics in a way that means they can be reused in the
same job without having to include multiple copies of the data in the file.
An XObject can also refer to other XObjects making it possible to reuse
complex combinations of content.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 6
A composition engine can use XObjects in the generated PDF file to help
identify reused elements and to reduce overall file size. Note that the
amount of reuse encoded by composition engines varies greatly, from
none at all, or just images, all the way up to maximum possible reuse.
This usually happens without being controlled or even noticed by the
“user of the composition engine” - for example the designer. However,
from a print output point of view it is an important feature that has a
significant effect on how smoothly a PDF can be processed.
Some variable data PDF consumers (such as DFEs) not only identify
reused elements but also try to determine which collections of elements
are used together on multiple pages with consistent positioning relative
to each other. This further minimizes the number of components
required to construct every final page in the job. Achieving this
coalescence automatically, flexibly and intelligently can have a huge
impact on the overall throughput of the DFE and is a key distinguishing
factor between composition engines, RIPs and DFEs from different
vendors.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 7
Who can affect the processing speed of the
VDP PDF file?
The speed at which a variable data PDF file can be processed in a
printing workflow depends on many people, including the originator of
the design concept, the designer, the composition tool operator and the
composition tool software developers.
Variable data PDF files are typically created with composition tools, and
those tools vary significantly in what assets or graphical components
(created by a designer) each uses to do so.
Some work with basic graphical elements such as a line or block of text;
an image or a value to be represented in a barcode.
At the other end of the scale, some tools expect to be given large pieces
of the final design of the print already built, perhaps as a PDF file. Simple
graphics are then often laid over the top. In the extreme this can be
compared to the historical method of pre-printing shells, perhaps on an
offset or flexo press, and then using a digital press to overprint simple
variable data on top.
Obviously, there is a complete spectrum between these two end points.
Many individual tools span a range of that spectrum by giving control to
decide how much of the final design will be provided as a single pre-
created PDF asset, and how much will be added from simple graphics
following the placement rules appropriate for this project.
In general, tools that combine basic individual graphical elements tend
to generate simpler variable data PDF files, and often don’t have the
capability to use live PDF transparency. The responsibility for making
efficient files at this end of the scale is almost all with the operator of the
composition tool, and with the software developers. In this context the
most obvious opportunities to make inefficient files tend to be around
issues such as image resolution.
When large parts of the design are supplied as a pre-created PDF asset
it’s possible to achieve all of the normal rich graphics that PDF supports.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 8
That PDF asset is normally created by a designer based on a concept
provided by the originator of the design, often representatives of a brand
owner. If the designer has followed the best practice guide intended
specifically for designers, then that PDF file will probably not cause any
problems further downstream. If they have not, the PDF may be
inefficiently built, which could slow down processing in the print
workflow.
Composition tools don’t usually do much, if any, analysis and pre-
processing of the internals of pre-created PDF assets before inserting
them into the variable data PDF that they are generating. They don’t
need to because the PDF standard is written in such a way that one
PDF file can be read and placed into a larger PDF file that happens to
also include other graphics fairly simply in a clean, self-contained way,
without any confusion around which copy of a font or a color space or
whatever should be used for which graphic.
This normally means that the only people who have any real impact
on the efficiency of the final variable data PDF file are (a) those who
create the design of that asset in the first place, and (b), those who
subsequently export it to create the PDF asset. If multiple pre-created
PDF assets are supplied, and then combined differently for each instance
of the variable data, then any inefficiencies in each of them can have a
significant impact on DFE throughput.
But a composition tool vendor may choose to enhance their product by
adding the ability to analyze and optimize PDF assets. As an example,
the composition tool could identify images with a depth of 8 or 16
bits and automatically down-sample them if they are at too high a
resolution.
It would, of course, then be the responsibility of the composition tool
operator to ensure that the correct options are selected to generate
optimized PDF files for each project, perhaps by setting a target image
resolution.
If that pre-created PDF asset is used as a static background across every
PDF page of the output it won’t usually make much difference to how
PDF Association - Best Practice: PDF files for VDP - Developer Edition 9
long the job takes to process in a DFE, at least if that DFE has reasonable
variable data support. That’s because it will usually only be processed
once and then treated as a backdrop for processing the graphics placed
on top.
The composition tool operator has a role to play here, too, in ensuring
that the assets are ordered in the file for maximum efficiency. As an
example, variable data should always be placed on top of assets
representing reused graphics.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 10
Making efficient PDF files
In every print workflow one rule overrides virtually everything else: the
printed result must be what the person signing the check wanted and
expected. This guide is not intended to restrict the ability of marketing
departments and graphic designers to achieve the desired visual
appearance of printed work. It provides guidance on easing the path to
the most efficient production of that design … whatever that desired
result might be.
This section sets out guidelines for avoiding tripping up the print
production workflow with your PDF files for VDP. At the highest level
almost all of them boil down to a very simple maxim:
PDF Association - Best Practice: PDF files for VDP - Developer Edition 11
some subset of the recipients, usually based on some metadata about
the recipient, fall somewhere in between.
If you only have the time to focus on parts of your workflow you should
concentrate on the graphics that are individual to each recipient.
Optimizing images
In a DFE, as a general rule, images tend to take longer to process than
vector graphics and text. A photographic image will often use quite a
large number of different colors, each of which must be appropriately
color managed. In addition, there is simply more data involved which
must often be copied between memory locations, and the difference
between the effective resolution of the source image and the resolution
of the output device must be resolved.
These operations only take a few milliseconds individually but multiplied
over all the images in a job they can add up to a significant total.
At the same time images are commonly reused within a VDP job; they
may form part of a static page background, or a small number of images
may be selected from, each being used for a proportion of the recipients.
The ability to process each of a relatively small number of images only
once, and then reuse the result many times can significantly increase the
throughput of the DFE.
Note that many of these recommendations around image handling
can also make a PDF file more appropriate for multi-channel delivery,
for example by the web, email or to a mobile device because they will
reduce the file size and allow a more resource-constrained viewer to
display them correctly.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 13
photographic images, as long as they have not had non-photographic
content, such as text, merged into them, and as long as the down-
sampled result is not lossily compressed. Resolution optimization should
also only be done if the resolution change is reasonably large; reducing
the resolution only a small amount can impact image quality.
The resolution of both photographic and synthetic images should not
normally be increased if the original is too low a resolution. In such cases
a tool might, for instance, trigger a suggestion to check that the correct
image asset has been provided instead.
FIG 1: Effective resolution changes if the same image data is scaled to different sizes. If the
image is cropped the calculation must be made from the uncropped dimensions.
Reprinted with permission from Global Graphics Software.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 14
Discard cropped pixels from images
If an image is heavily cropped the portions outside the cropped area
should be completely discarded rather than simply hidden using a
clipping path, as shown in FIG 2. Even though the clipped-out pixels
won’t typically be color managed or otherwise processed for print
purposes by the DFE, they will typically still need to be read from the
PDF file and decompressed in order to find the pixels that are actually
required.
Cropping images can sometimes be efficiently combined with a
resolution reduction step.
FIG 2: Removing image data that will be cropped on the page instead of just hiding it.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 16
Again, this recommendation is most relevant for photographic images.
There are cases where pre-rendered images of assets such as text,
especially saved at one bit per pixel, can benefit from image
interpolation, most notably when the effective image resolution is higher
than the printing resolution.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 17
If multiple copies of the same image are embedded in the PDF that will
evidently bloat the file size. Less obviously it will reduce the efficiency of
the VDP optimizations in some DFEs because the images may be seen as
different and therefore each copy may be processed separately,
increasing the work required unnecessarily and slowing the job down.
Whenever possible only one copy of each image should be embedded.
If the same source image is used at multiple different sizes on the pages,
those may either use the same embedded copy or a separate copy at a
suitable resolution may be used for each final size.
Another inefficient practice to avoid is embedding images as inline
images instead of image XObjects, because inline images cannot be
reused and must be included in the PDF file every time they’re used.
Optimizing transparency
The very rich and flexible support for live transparency in PDF is an
incredibly useful aspect of the format and is one of the reasons for
preferring PDF for production print.
On the other hand, compositing transparent regions in a PDF file is much
more processor intensive than handling opaque areas of a page.
As an example, consider a PDF page containing two overlapping RGB
images, both tagged with an ICC profile for ECI RGB.
When outputting to a digital press printing in CMYK with no live
transparency involved, the color of each pixel in each image must be
transformed into tone values for CMYK, usually using ICC profiles. In
most DFEs the results of the calculation for each set of RGB values from
the image will be cached and reused when another pixel using exactly
the same RGB values is processed.
There’s a reasonable amount of calculation involved, but nothing too
heavyweight.
Now consider the same example, but where the two images are within a
“transparency group” in the PDF file. In most cases that group will have a
color space associated with it called the “blending color space”, and in
PDF Association - Best Practice: PDF files for VDP - Developer Edition 19
most cases that blending space will be sRGB, if only because that’s the
default in many design applications.
In addition, a “blend mode” will be set. The blend modes allowed in PDF
include commonly used modes such as ‘Normal’, ‘Overlay’ and ‘Multiply’
and more specialized ones such as ‘Soft Light’ and ‘Saturation’.
The colors of each pixel are transformed from the source RGB (ECI RGB)
to the blend color space (sRGB). Once in the blend space the two images
are composited together according to the blending mode.
It’s unlikely that the pixels of the two images are exactly aligned, so
this composition means that the number of apparent pixels in the area
where they overlap will increase.
And finally the resulting colors in sRGB must be transformed to the
output CMYK of the press.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 21
Avoid invisible transparency effects
The most common usage for live transparency in PDF is for drop
shadows, but even that use should be avoided if it doesn’t result in
an effect that’s visible on the final printed piece. For example, do not
include drop shadows on images that are printed on a black background,
as shown in FIG 5, unless the shadow will also fall on another element
where it will be visible, such as another image on the page.
FIG 5: If an image with a drop shadow is placed over a black background the drop shadow is not
usually visible.
Clearly there are exceptions to this where the drop shadow would still be
visible on a print, even if it isn’t on a computer monitor, such as where
the drop shadow paints in a rich black (for example black plus 40% cyan)
and the background is printed with only black ink.
In the same way, if all you’re doing is adding drop shadows to text or
images that fall entirely on a white background, you don’t need to use
transparency at all; a simple shading pattern will do everything that you
need. Of course, if any of the graphics with drop shadows overlap each
other you will need to use transparency, so that the shadows fall across
the elements behind correctly.
If assets are being created in off-the-shelf design tools and then
integrated with variable elements in a composition tool it may be
difficult to avoid the use of transparency in drop shadows, because
many design tools offer a simple switch to add a drop shadow, which
includes turning on the transparency. On the other hand, if everything is
created and laid out within the composition tool it should be very
achievable.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 22
Use overprinting instead of transparency for black text and rules
Printers using offset lithography, flexography and other conventional
print technologies have used a little trick to avoid registration errors
between small black text and fine rules running over other graphics on
a page for many years: they set the black elements to overprint. This
means that the text and rules don’t knock out of the other graphics,
which means that you’ll never see any white outlines as a result
of misregistration. More recently, in some cases people have used
transparency instead, using Overlay or Darken blend modes.
The potential for objectionable artifacts when using either approach is
vanishingly small. The only visible effect likely is that the black won’t be
pure, but may have varying amounts of cyan, magenta and yellow added
by graphics behind it. If these techniques are used only for small black
text and rules it’s hard to see that variation at all, even with a lens.
Where overprinting and transparency do differ, however, is in the speed
at which the DFE can process them. A simple black overprint will often
be significantly faster, especially if the background behind the black
elements is complex or includes high-resolution images.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 23
Some applications use a soft mask to clip an image only because a hard
mask at the same resolution as the main image would result in visible
stepping around the edge. A vector clipping path will yield a smoother
edge than most hard masks and would be a suitable alternative to a soft
mask in many cases.
When a special effect frame is added to an image it is usually printed on
top of the image. It is far more efficient to reveal the real image through
the frame using one of the following techniques than to add a soft mask
to a frame supplied as an image:
Draw the frame using vector objects (far easier for some visual effects
than for others). In this case nothing extra is required to reveal the image
through the center of the frame.
Apply a clipping path to the frame object.
Use a masked image (with a Mask entry) rather than an image with a
SMask entry
PDF Association - Best Practice: PDF files for VDP - Developer Edition 25
copy of the image, rather than just clipped out. Care must be taken
to ensure that the two images are exactly aligned in that case. Even
though this technique increases the amount of image processing
required, it can increase overall performance because image
processing is much faster than transparency compositing.
FIG 8: Choosing the blend color space carefully can greatly reduce color transformations
required.
Reprinted with permission from Global Graphics Software.
Switching the blending color space, especially between RGB and CMYK
spaces, will often change the final printed color. If you’re going to change
the blend color space from something like sRGB to the output CMYK for
maximum DFE performance, you need to finalize that decision early in
your design process and validate the resulting output. If you need to stay
PDF Association - Best Practice: PDF files for VDP - Developer Edition 27
with blending in RGB you should ensure that the blend color space
matches the source color space of as many of your images as possible (or
vice versa).
Occasionally, transparency group operations may be chained together
if a group is defined within another group, although that is relatively
rare. There can be good reasons for using this kind of construct in a
non-variable job for commercial print, publication or newsprint work.
One common example is when multiple PDF/X files that use different
ICC profiles in their output intents are placed or imposed together;
this might arise if you’re placing display ads, for instance. If that kind
of situation occurs in a VDP print job, however, you would be advised
to review the creation workflow and unify your asset design process
further upstream to ensure consistent and predictable output. In general
nesting transparency groups should be avoided for VDP.
Use a constant opacity rather than a soft mask with constant values
There are two ways of specifying how transparent a graphic should
be within a PDF file: you can set a constant opacity value for fills and
strokes (using the CA/ ca keys), or you can attach a soft mask (SMask
in the PDF, or within a JPEG2000 image). Soft masks can be very useful
if the transparency should vary across the graphic, for example for
softening the edges of an image. But they are also used quite a lot
where the transparency is uniform across the whole graphic. The most
inefficient examples add a soft mask where all of the values are either 1.0
(indicating that the element is fully opaque) or 0.0 (indicating that the
element is fully transparent and should not be visible at all).
If the element should be fully opaque the best way to represent that is to
omit the SMask entry completely.
If the element should be fully transparent (not visible) then don’t include
it in the PDF file at all!
If the element should have a constant transparency that is neither fully
opaque, nor fully transparent, just use the CA or ca keys to set that value
and omit the SMask key.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 28
Optimizing vector graphics
Vector graphics are relatively quick to process compared to images,
which is why this section is so short.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 29
the image resolution to avoid preflight warnings by using multiple pixels
per module will tend to result in slower processing, but not normally
enough to worry about too much.
Composition vendors can assist with barcode quality (readability) if
they’re using vector graphics by turning on automatic stroke adjustment
for bar codes (using the SA graphics state parameter in the PDF) to
minimize issues if the scaling is not absolutely correct.
Optimizing colors
It’s common practice to use spot colors when defining a color that will
be used many times in the same design, especially when those are brand
colors and will be used in multiple jobs. But using too many, or using
them in sub-optimal ways, can impact the processing speed on the DFE.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 33
FIG 10: Example of flexible packaging using a white ink under the colors.
And you’d usually do the same for a varnish; place it on top of all other
graphics in the PDF and set it to overprint.
But if you’re using spot colors for the color, most of the time you’ve
chosen that color because that’s what you want to see; you don’t want it
mixing with other colors. And if that’s the case, make sure that you don’t
set it to overprint. On a digital press spot colors will very often be
emulated using CMYK inks (or CMYKOG etc.) and managing overprinted
spot colors so that they will emulate not just the color of the solid spot,
but also the color resulting from overprinting with graphical elements
underneath them, takes more processing, and therefore more time, not
to mention being very specific to the press and inks that are being
emulated
PDF Association - Best Practice: PDF files for VDP - Developer Edition 34
Optimizing VDP layouts
As mentioned above (see “High level view of VDP optimizations: RIP
once, use many times”) the ability to coalesce multiple graphics together
to reduce the number of components that need to be re-composed
together to form a final page can have a very significant impact on the
throughput of the DFE.
The coalescing process typically requires that multiple graphics must all
appear on a significant number of pages together, and with exactly the
same positions relative to each other in order to be grouped together
into a single component.
Some RIPs have the capability to adjust the drawing order of the assets
and other graphics placed on the page, that is the order in which they
are to be placed, with some behind or in front of others. Being able to
re-order graphics allows them to be coalesced into groups even if they
are not adjacent to each other in the drawing order. Of course, those
solutions place great importance on avoiding any changes to the visual
appearance of the printed page as a result.
Most of the recommendations in this section are aimed at maximizing
the efficiency of the coalescing process so that fewer components are
required to construct every final page.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 35
FIG 11: These four pages illustrate different instances of a document where all of the blue ‘text’
and the blue ‘image’ (marked ‘AAA’) are intended to be used together, while the red ‘text’ and the
green ‘image’ are unique to that instance.
On the second page, the green image is in a different location to those on pages 1, 3, and 4; but
that will only matter if it uses transparency (or spot colors set to overprint that will be emulated
using the process inks) as it’s unique data and won’t be shared between pages.
The blue image on page 3 is also in a different position to the blue images on the other pages.
Here it does matter because it prevents the DFE from treating the combination of the blue text
and blue image as being used together in exactly the same way on every page. The incorrect
positioning may mean that the blue image is processed twice: once for pages 1, 2, and 4 (in
combination with the blue text), and again for page 3 on its own.
Reprinted with permission from Global Graphics Software.
If a group of graphics are used together many times, then it’s even
more important to ensure that their positions relative to each other
are consistent than it is for a single graphic. Varying their relationship
will mean that the DFE’s ability to optimize the job by coalescing them
together as a group will be reduced.
The same comments go for placing all copies of a graphic or group of
graphics at the same size and rotation.
If you have a good reason to move things around on the page then go ahead,
but finding that the throughput of the DFE is reduced because you accidentally
didn’t place them in exactly the same position would be frustrating!
Unfortunately this does mean exactly the same; a fraction of a point
difference will often be enough to subvert any optimizations; after all, at
1200dpi each point is nearly 17 device pixels!
In the same way, some composition engines offer the capability to ‘flex’
layouts, to move some assets in response to differing sizes of something
like a text block because some recipients have longer names or
addresses, or the length of a list of items varies.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 36
Again, if that produces the exact visual result that you’re looking for, go
ahead and use the option. But if flexing the layout doesn’t provide a
benefit for you in the design or readability, turn it off and allow the job to
process a bit faster at the print stage.
Consistent placement of graphics also extends to consistently placing
clipping paths around repeats of the same set of graphics. If you’re
imposing something like postcards or labels, make sure that you use the
same clip for all of them, even if that means the clip on some may extend
off the edge of the media.
FIG 12: The gray area in this diagram represents the substrate in a narrow-web label press, with
four lanes of labels imposed on it. Each label station in the PDF file had a clipping path around it,
shown in the diagram as a dashed red box. If the clipping path for each station is exactly the same
with respect to the graphical content, then the job can probably be processed faster in the DFE,
even if that means that the clipping paths extend outside of the PDF page. The same would apply
for imposed postcards etc.
Reprinted with permission from Global Graphics Software.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 37
The coalescing process in the DFE (see “High level view of VDP
optimizations: RIP once, use many times”) will typically work best if it
can merge all of the assets and other graphics for the ‘background’ into
one or a small number of components to be re-composited later.
It may collect sets of semi-variable assets and elements together as well,
if they are used together in a consistent way. To take the example given
in “High level view of VDP optimizations: RIP once, use many times”, of
a map to the recipient’s nearest store, it may be that that map is always
used with a logo and a text address for that specific store, and with a
sales representative’s image and telephone number.
It’s common to see PDF files where the assets and graphics on a
single page are drawn onto the page in a fairly arbitrary order, so that
‘background’ graphics are actually drawn quite late, after many of the
variable and semi-variable graphics This often makes no difference to the
visual appearance as long as the graphics drawn later don’t fall on top of
those drawn earlier. But it does mean that the coalescing step must work
harder and its rules, designed to ensure that the visual appearance is not
compromised, mean that it may not be able to collect graphics into as
small a number of large components, typically reducing throughput.
If you can design your assets and layouts in such a way that static
background elements are drawn first, followed by semi-variable
graphics, and then those specific to the current recipient, the coalescing
stage can often perform better. At a slightly more detailed level, it’s often
worth trying to make sure that an image and the key line for that image
are next to each other in the drawing order.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 38
The same goes for imposing multiple instances of something together on
a larger PDF page, whether that’s direct mail, postcards, labels, or
anything else. It’s very common to construct the PDF by creating a single
instance of static, semi-static and variable graphics, and then to impose
or step and repeat that across the available area (often with each
instance encoded as a form XObject).
If the background of one instance overlaps with the (identical)
background of another instance, then the DFE may decide that it cannot
safely treat the two as two copies of the same graphic.
That’s especially likely if they use any live PDF transparency, because
the overlap needs to be processed as an overlap to generate the correct
visual appearance of that transparency. And if they can’t be treated as
separate graphics, then the DFE must process much more data because
it must render both of them … or tens, or hundreds of them, however
many there are on that PDF page. Some DFEs can be configured to
ignore small overlaps in such cases, but in general it’s better to ensure
that the multiple instances don’t overlap in the imposition.
FIG 13: Overlapping label designs; the yellow lines indicate the extent of each copy.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 39
Nest ‘forms’ and images appropriately
While some DFEs coalesce graphics automatically (see “High level view
of VDP optimizations: RIP once, use many times”, others require that
the coalescing is guided entirely by how assets and other graphics have
been written into form and image XObjects in the PDF file. If you’re
using one of these DFEs the throughput can be significantly increased
if you use some care in creating your own compound assets before
placing them in the composition tool. Unfortunately, this can cross
the lines of responsibility between graphic designers and composition
tool operators, and can make late changes to the page layout, or
customization for markets using multiple output sizes (for example both
US Letter and A4 pages) more difficult.
In the same way, a composition vendor can optimize throughput in some
cases by replicating the hierarchy of single-use and reused graphics in a
hierarchy of form XObjects.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 40
FIG 14: Tree of objects in a typical variable data job where multiple instances of postcards, labels
etc. are imposed together on each PDF page.
Reprinted with permission from Global Graphics Software.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 41
each one takes a tiny fraction of a second to process, every little bit
helps, so don’t add those rectangles if you have the ability to avoid doing
so. This obviously doesn’t apply to adding background elements in white
ink, for example for printing on transparent, metallic or off-white
substrates. If you need the white there, add it!
Security marking
Some software adds security marks as an overlay of a pattern of very
fine dots over the top of the other graphics. If that’s done using live
PDF transparency then it may completely disable all variable data
optimizations in the DFE, slowing everything down significantly. On
the other hand, if it’s simply done with an overprinted black it’ll have
virtually no impact on the performance of the DFE.
Software that adds security by amending raster objects or elements
(for example using steganography) rather than adding an overlay has
no detectable impact on processing speed for the final variable data
PDF file, unless it means that the PDF ends up containing many more
different images than it would have otherwise.
Very complex guilloches used for security and drawn as vector strokes
can also slow processing, especially if each instance is unique.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 43
You may also find the “PDF/X-plus” specifications from the Ghent PDF
workgroup (gwg.org) useful in avoiding problems with unprintable
graphics in your files.
There are some notes on several helpful standards set out in “Relevant
standards”.
Instant improvement
At the end of 2020 a new PDF/X-6 standard[7], based on PDF 2.0, and a
new PDF/VT-3 standard[9], based on PDF/X-6, were published. In parallel
with this work the first of a series of “dated revisions” to the PDF 2.0
standard was also published[2] (think of it as a 2nd edition with some
minor corrections to the text and a few new small features).
These are intended, and expected, to become adopted as the
mainstream formats for preparing files for production printing. But it
may take a few years to achieve that, so some care should be taken to
ensure that you‘re not sending files that use a later standard than the
printing company or converter can handle.
There will, of course, be more work done on the PDF family of standards
in the future, so whenever you’re reading and acting on this document
you should always refer to the latest ISO standards, including any errata,
clarifications and corrections in order to make your workflows as robust
as possible. You may find the notes at https://pdf-issues.pdfa.org/
useful.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 44
Relevant standards
“Use standards when they’re relevant” recommends using standards
such as PDF/X and PDF/VT to support good practice in creating PDF files
for variable data printing. This section provides some more information
on relevant standards.
The PDF/VT standards are built on PDF/X which, in turn, are built on PDF.
The Processing Steps and Print Product Metadata standards can be used
in conjunction with several PDF/VT, PDF/X and PDF standards.
In developing the PDF/X and PDF/VT standards care has also been taken
to align them with PDF/A, (used for archiving and business exchange)
and/or PDF/UA (used for accessibility and repurposing), to ensure that a
single file can be created that is compliant with all of them at the same
time.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 45
The PDF/X standards mandate that certain items are included in a file
when it’s created to ensure best practice is followed. The various PDF/X
standards all require that:
1. All fonts required in the document are embedded, avoiding
problems with missing fonts or the use of a different version of a
font by the same name at the print service provider.
2. The colors used for all objects must be defined sufficiently
completely that they can be reproduced consistently and
accurately.
3. The color reproduction of the output device for which the job was
designed is specified, allowing accurate and consistent proofing
and emulation of the job on other devices, and preflight to identify
upstream mistakes quickly and easily.
The real value of PDF/X standards, however, isn’t obvious from a simple
list of technical prohibitions, requirements and recommendations. It
boils down to two things:
1. Application vendors can add an option to export to a specific PDF/X
conformance level, and users can then create files that are highly
likely to print well on a press run by some other company, without
knowing exactly what that conformance level requires. Even
technically savvy users can benefit because that same option will
reinforce self-discipline in following best practice.
2. A print service provider, converter, or other company printing for
other people can say very simply “give me a PDF/X-4 file” and know
that they will almost certainly be able to process what they get
sent. The designer, upstream prepress operator or composition
tool operator can translate that request into action because of
point a) above.
A PDF/X file also fully conforms to the PDF specification, so if you’re
creating files you can make them as PDF/X in order to gain the benefits of
automatic checking of things like font embedding, even if you’ve just
been asked for a PDF file.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 46
PDF/X conformance levels
The PDF specification itself has evolved over the last couple of decades,
as has the demand for richer graphics in printed work, and for formats to
support increased automation in the print workflow. As a result there are
now a number of PDF/X ‘conformance levels’ to choose from:
PDF/X-1a[3] – first published in 2001, and then re-issued in 2003, PDF/X-1a
is now regarded as a very conservative format. It’s based on PDF-1.4
and all color data must be expressed in CMYK or spots; no device
independent color spaces (ICC), RGB or Lab data are allowed. It also
prohibits live PDF transparency and doesn’t allow constructs added in
later PDF specifications, such as PDF layers (optional content).
PDF/X-2 – didn’t see significant adoption and is therefore best avoided.
PDF/X-3[4] – first published in 2002, and then re-issued in 2003, PDF/X-3
is a slightly more open standard than PDF/X-1a It’s still based on PDF-1.4
and prohibits transparency and PDF layers, but it does allow device
independent color spaces.
PDF/X-4[5] – first published in 2008, and then re-issued in 2010, PDF/X-4
is based on PDF-1 6 and allows device independent color spaces like
PDF/X-3. It also allows live PDF transparency and PDF layers to be used,
with some appropriate limitations. At the time of writing PDF/X-4 is the
most obvious format to use for delivering the majority of PDF files for
production printing.
PDF/X-4p[5] and PDF/X-5[6] – are specialist variants designed for specific
workflows where the receiver of the file has access to additional data
that must also be available to process the files correctly. These standards
should only be used if the company who will be receiving the PDF files
asks for them.
PDF/X-6[7] – is a new PDF/X standard published in November 2020 and
built on PDF 2 0 (“PDF 2.0 (ISO 32000-2)”). It takes advantage of new
functionality in PDF 2.0 such as page-level output intents, and also adds
value in multi-channel delivery because of the extended accessibility
PDF Association - Best Practice: PDF files for VDP - Developer Edition 47
support in PDF 2.0. Confirm with your print provider if they have
upgraded to their RIPs and DFEs to support PDF 2.0.
PDF/X-6p and PDF/X-6n – like PDF/X-4p and PDF/X-5, these are specialist
variants designed for specific workflows and should only be used if the
company who will be receiving the PDF files asks for them.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 48
standard that we would recommend for current workflows unless all
parties in that workflow agree to use one of the others.
PDF/VT-2[8] – designed to support a ‘chunking’ workflow to allow
something almost indistinguishable from streaming, that is where the
first pages of the job are being printed before the last ones have been
created by the composition engine. It does this by providing a method
whereby large assets such as images that are used multiple times
(for example for many recipients each) can be saved into a single PDF
file, known as a target file. A series of ‘chunks’, each defining a range
of pages to be printed and saved as a PDF/VT-2 file, is then produced.
Each PDF/VT-2 file includes references to the assets in the target file(s),
which means that those large assets don’t need to be repeated in every
PDF/VT-2 file. PDF/VT-2 is not widely implemented or used.
PDF/VT-2s – is a variant of PDF/VT-2 where both the target files containing
reused assets and the PDF/VT-2 files themselves are wrapped into a
single MIME stream. The intention is to simplify delivery of a stream for
printing where there isn’t a shared file system accessible to both the
submission tool and the DFE. PDF/VT-2s is even less widely implemented
than PDF/VT-2 and should be avoided.
PDF/VT-3[9] – Was published in late 2020 and is based on PDF/X-6, which,
in turn, is based on PDF 2.0. Amongst other things this allows better
color management of jobs that are printed on multiple different media,
and allows unification of PDF generation for multi-channel delivery
with excellent accessibility capabilities; see “PDF 2.0 (ISO 32000-2)”.
Just like PDF/X-6, PDF/ VT-3 is expected to become the most commonly
used PDF/VT standard over time, replacing PDF/VT-1. But check with
your print service provider or converter that they’re ready to accept it
already because not all press DFEs will necessarily have been upgraded
to support PDF 2.0 yet.
As with PDF/X, the real value of PDF/VT is more in simplifying
communication of requirements and best practice than in defining
anything significantly different from what can be achieved in baseline
PDF. In a sense it relieves the graphic designer and composition tool
operator of the need to consider some of these constraints when they
PDF Association - Best Practice: PDF files for VDP - Developer Edition 49
make a file; just select “PDF/VT” in the menu when generating the file for
print and it will be done for you.
The PDF/VT standard concentrates on providing support for predictable
and repeatable output and for automation; it does not focus on how the
desired elements should be written into that file in order to maximize
the efficiency of processing.
Accordingly, using PDF/VT is a very good way to improve the document
delivery workflow in many ways and is definitely recommended, but it’s
not the whole story. There are many things that users can do to optimize
processing and avoid last-minute problems in PDF/VT jobs. These are
the subject of this guide, and most are equally applicable to both PDF/VT
and ‘baseline’ PDF.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 51
PDF 2.0 has adopted and extended the original PDF/X output intent, and
it’s now possible to include an output intent on every page. With PDF 2.0,
one output intent can be created for each substrate, and can be
referenced from each page so that the right color management is
triggered.
PDF 2.0 has also adopted the DPart structure used for print metadata from
the first PDF/VT standard (see “PDF/VT (ISO 16612-2 and 16612-3)” and
“Print product metadata for PDF (ISO 21812-1[11])”), and a considerable
amount of work has been done on areas outside of production printing
workflows, such as better support for accessibility tools such as screen
readers. If you’re doing any form of multi- or omni- channel publishing you
may find that additional accessibility support very useful, for example to
achieve Section 508 compliance, and the equivalents in other countries.
At the end of 2020 a new “Dated Revision” of the ISO PDF 2.0 standard
was published[2]. The primary aim of this new publication was to clarify
additional areas and to resolve problems that had been encountered
in real-world usage, not just of the 2017 PDF 2.0 standard, but also of
previous PDF specifications. Further updates are anticipated over the
next few years, each one clarifying more issues where different vendors
have read the current text in different ways.
You must use the latest dated revision rather than the 2017 edition when
implementing both PDF/X-6 and PDF/VT-3.
PDF 2.0 adds value to many workflows, including those for production
printing, but it does also bring a small amount of risk. If a file has used
some of the new features in PDF 2.0 those will usually be silently ignored
by an older reader. PDF was designed to be very flexible, and to allow
custom and proprietary data to be embedded virtually anywhere in the
file structure. It does that by saying that a reader should simply ignore
anything it doesn’t recognize. To a PDF 1.7 reader, most new PDF 2.0
features are just objects that it won’t recognize and should therefore
ignore. The most common exception to that rule is around security; if the
PDF file uses the new AES-256 security introduced in PDF 2.0 then an
older reader will probably be unable to read that file at all.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 52
As with any new standard it’s important to plan adoption across your
workflows. It’s better to start at the end, with products that need to read
files, and only to start using newer functionality in upstream products
such as composition tools once the readers are ready to accept it.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 54
FIG 16: Hierarchy of pages in a personalized saddle-stitched booklet.
The print product metadata is consumed by the print workflow, which may mean inside the DFE
for a digital press, or may be slightly further upstream, to enable a job to be split over multiple
presses, and for managing off-line finishing as well.
When it is consumed, the print product metadata may be simply presented to an operator so
that they are aware of how the designer/purchaser expected the job to be printed. In a more
sophisticated workflow, the metadata may be merged with a pre-existing job ticket, either fully
automatically, or by offering the operator additional information in support of their configuration
of the system.
Reprinted with permission from Global Graphics Software.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 55
That inspection and configuration takes time and creates an opportunity
for making mistakes. Even proprietary metadata in PDF files often only
identifies spot colors as “technical”, without providing sufficient detail
for automatic processing.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 56
Bibliography
[1] ISO 32000-1:2008 - Document management — Portable document
format – Part 1: PDF 1.7
Available from https://www.pdfa.org/resource/iso-32000-pdf/
[2] ISO 32000-2:2020 - Document management — Portable document
format – Part 2: PDF 2.0
Available from https://www.pdfa.org/resource/iso-32000-pdf/
[3] ISO 15930-4:2003 - Graphic technology — Prepress digital data
exchange using PDF — Part 4: Complete exchange of CMYK and spot
color printing data using PDF 1.4 (PDF/X-1a)
Available from https://www.iso.org/standard/39938.html
[4] ISO 15930-6:2003 - Graphic technology — Prepress digital data
exchange using PDF — Part 6: Complete exchange of printing data
suitable for color-managed workflows using PDF 1.4 (PDF/X-3)
Available from https://www.iso.org/standard/39940.html
[5] ISO 15930-7:2010 - Graphic technology — Prepress digital data
exchange using PDF — Part 7: Complete exchange of printing data
(PDF/X-4) and partial exchange of printing data with external profile
reference (PDF/X-4p) using PDF 1.6
Available from https://www.iso.org/standard/55843.html
[6] ISO 15930-8:2010 - Graphic technology — Prepress digital data
exchange using PDF — Part 8: Partial exchange of printing data using PDF
1.6 (PDF/X-5)
Available from https://www.iso.org/standard/55844.html
[7] ISO 15930-9:2020 - Graphic technology — Prepress digital data
exchange using PDF — Part 9: Complete exchange of printing data
(PDF/X-6) and partial exchange of printing data with external profile
reference (PDF/X-6p and PDF/X-6n) using PDF 2.0
Available from https://www.iso.org/standard/77103.html
[8] ISO 16612-2:2010 - Graphic technology — Variable data exchange
— Part 2: Using PDF/X-4 and PDF/X-5 (PDF/VT-1 and PDF/VT-2)
Available from https://www.iso.org/standard/46428.html
PDF Association - Best Practice: PDF files for VDP - Developer Edition 57
[9] ISO 16612-3:2020 - Graphic technology — Variable data exchange
— Part 3: Using PDF/X-6 (PDF/VT-3)
Available from https://www.iso.org/standard/75218.html
[10] ISO 19593-1:2018 - Graphic technology — Use of PDF to associate
processing steps and content data — Part 1: Processing steps for
packaging and labels
Available from https://www.iso.org/standard/65428.html
[11] ISO 21812-1:2019 - Graphic technology — Print product metadata
for PDF files — Part 1: Architecture and core requirements for metadata
Available from https://www.iso.org/standard/74407.html
ISO standards can also be purchased through your national standards
body.
PDF Association - Best Practice: PDF files for VDP - Developer Edition 58