2.1. DOZOR

One of the core features of the protocol described here is the ability to automatically recognize and rank the series of single diffraction patterns collected during the low-dose mesh scan of the sample holder. This is carried out using the program DOZOR. As the algorithm used will be illustrated in more detail elsewhere, it will be only briefly described here.

In a first step, DOZOR determines the distribution of background intensity on a diffraction image as a function of the diffraction vector length h. This is accomplished by the iterative summation of pixel intensities and the sequential rejection of outliers. After azimuthal averaging this produces the one-dimensional background function . This function should be smooth: any sharp peaks are an indication of ice rings or salt diffraction, and such areas are not used in further calculations.

In the case of diffraction from a crystal of a biological macromolecule, the function

where N(h) is the number of detector pixels and I_i,j is the intensity in any pixel which belongs to the resolution shell, h), will give the estimate of the mean intensity of Bragg spots as a function of resolution and will represent the well known Wilson plot, which for any protein crystal can be modelled using , the unique pattern of average squared structure-factor magnitudes (Bourenkov & Popov, 2006). DOZOR approximates the experimental data by applying an isotropic Debye–Waller factor to the standard protein Wilson plot model,

The quality of the resulting fit is evaluated via the correlation coefficient between the left and right parts of (2), CC_powder. The program also identifies individual Bragg spots and makes a few simple geometrical checks which additionally validate the presence of diffraction from macromolecular crystals and allow the rejection of ice or salt contamination. Finally, a score of diffraction strength is estimated as the total averaged diffraction intensity multiplied by CC_powder, where V(h) is the reciprocal volume of the resolution shell,

In the case where DOZOR cannot find any Bragg spots, the score is determined as zero.

3.1. Bacteriorhodopsin

Crystals of bacteriorhodopsin (BR) were prepared as described previously (Gordeliy et al., 2003). In this study, two batches of bacteriorhodopsin crystals were used: BR1 (Fig. 2a), with dimensions of ∼20 × 20 × 5 µm, and BR2 (Fig. 3a), with dimensions of ∼5 × 5 × 2 µm. Diffraction data (Table 1) were collected on ESRF beamline ID29 (de Sanctis et al., 2012) using a PILATUS3 6M pixel detector (Dectris, Baden, Switzerland).

Figure 2 Multi-crystal data collection and structure solution from larger crystals of bacteriorhodopsin. (a) Crystals of bacteriorhodopsin obtained from crystallization in lipidic mesophase (Borshchevskiy et al., 2011); the average crystal size is ∼20 × 20 × 5 µm. (b) Heat map after initial mesh scan of the sample holder. The colours from dark red to yellow represent the intensity of the detected diffraction signal at the respective position; the white crosses mark the positions that have been used for collection of partial data sets. In all heat plots shown the x axis represents the grid points along the horizontal translation of the sample holder and the y axis the vertical grid points. For both, the unit is the beam size. (c) Dendrogram based on HCA of CCI(i, j) values produced by XSCALE. The blue rectangle shows the partial data sets merged to produce the final data set. (d) Wilson plot derived from the final data set using BEST (Bourenkov & Popov, 2006). (e) Detail of the final 2mFobs − DFcalc, αcalc electron-density map (contoured at 1.5 × r.m.s.) obtained, with the refined structure shown in ball-and-stick representation. (f) OMIT difference density (mFobs − DFcalc, αcalc) map at the end of the refinement procedure (contoured at 2.5 × r.m.s.) for a retinal molecule (ball-and-stick representation).

Figure 3 Multi-crystal data collection and structure solution from microcrystals of bacteriorhodopsin. (a) Microcrystals of bacteriorhodopsin obtained from crystallization in lipidic mesophase; average crystal size ∼5 × 5 × 2 µm. (b) Heat map after initial mesh scan of the sample holder. (c) Dendrogram based on HCA of CCI(i, j) values produced by XSCALE. (d) Wilson plot from the final data set derived using BEST. (e) Detail of the final 2mFobs − DFcalc, αcalc electron-density map (contoured at 1.5 × r.m.s.), with the refined structure shown in ball-and-stick representation. (f) OMIT difference density (mFobs − DFcalc, αcalc) map at the end of the refinement procedure (contoured at 2.0 × r.m.s.) for a retinal molecule (ball-and-stick representation).

For BR1 the initial mesh scan was carried out using a Gaussian X-ray beam of 20 µm in diameter with a flux of 3 × 10¹¹ photons s⁻¹. The resulting heat map (Fig. 2b) revealed ten well diffracting positions from which partial data sets were collected. All partial data sets could be automatically processed and, after HCA (Fig. 2c), nine were chosen for scaling and merging to produce a final data set to d_min = 2.3 Å (Table 1; Wilson plot shown in Fig. 2d).

For BR2, the initial mesh scan (X-ray beam of 10 µm in diameter with a flux of 1.5 × 10¹¹ photons s⁻¹) produced a heat map (Fig. 3b) showing 59 diffracting positions in the sample holder from which partial data sets were collected. 38 partial data sets could be automatically processed and, after HCA (Fig. 3c), ten were merged to produce a final data set to d_min = 2.6 Å; Table 1; Wilson plot shown in Fig. 3d).

For both BR1 (twinning fraction 0.06) and BR2 (twinning fraction 0.39) structure solution was carried out by molecular replacement using MOLREP (Vagin & Teplyakov, 2010) with PDB entry 3ns0 (Borshchevskiy et al., 2011) stripped of water molecules and ligands as a search model. Structure refinement (Table 2) was carried out using the twinning refinement option in REFMAC5 (Murshudov et al., 2011) interspersed with rounds of manual rebuilding in Coot (Emsley et al., 2010). In both crystal structures assignment of the retinal cofactor was possible from the interpretation of both electron-density and difference density maps and is well defined both in the final 2mF_obs − DF_calc electron density and in OMIT difference density maps (Figs. 2e, 2f, 3e and 3f).

3.2. Thaumatin

Thaumatin (Sigma–Aldrich catalogue No. T7638) was dissolved in double-distilled water to a concentration of 20 mg ml⁻¹. Crystals of approximate dimensions 40 × 40 × 60 µm were obtained in 2 µl (1:1 ratio) hanging drops using 0.1 M HEPES pH 7.5, 0.7 M potassium/sodium tartrate, 20% glycerol as a reservoir. Crystals were mounted as described in §2 without further cryoprotection. Data were collected on ESRF beamline ID29. The initial mesh scan was performed with an X-ray beam of 10 µm in diameter with a flux of 8.7 × 10¹¹ photons s⁻¹. From the resulting heat map (Fig. 4a), 100 well diffracting points were chosen for the collection of partial data sets, of which 78 could be automatically integrated. After HCA (Fig. 4b) 74 were merged to produce a final data set to d_min = 1.2 Å (Table 1; Wilson plot shown in Fig. 4c).

Figure 4 Multi-crystal data collection and structure solution from crystals of thaumatin. (a) Heat map after initial mesh scan of the sample holder. (b) Dendrogram based on HCA of CCI(i, j) values produced by XSCALE. (c) Wilson plot from the final data set derived using BEST. (d) A ribbon diagram of the refined crystal structure of thaumatin produced (tartrate molecule in stick representation). (e) Detail of the final 2mFobs − DFcalc, αcalc electron-density map (contoured at 1.5 × r.m.s.), with the refined structure shown in ball-and-stick representation. (f) Difference density (mFobs − DFcalc, αcalc) for a tartrate molecule after structure refinement (OMIT map). The difference density is shown at a contour level of 3 × r.m.s.

Structure solution was carried out by molecular replacement using MOLREP with PDB entry 4axu (Cipriani et al., 2012) stripped of water molecules and ligands as a search model. Structure refinement (Table 2, Fig. 4d), during which analysis of difference electron-density maps clearly allowed the assignment of tartrate (one molecule; Figs. 4e and 4f) and glycerol (one molecule) moieties bound to the protein, was carried out in REFMAC5 alternated with manual rebuilding in Coot.

3.3. Monoclinic lysozyme

Lysozyme (Roche Applied Science, catalogue No. 10837059001) was dissolved in double-distilled water to a concentration of 40 mg ml⁻¹. `Flowers' of monoclinic (space group P2₁) lysozyme crystals (Fig. 5a), with each petal ∼80 µm in the largest dimension, were then obtained from 2 µl (1:1 ratio) hanging drops using 0.6 M NaNO₃ as the precipitant/reservoir. Prior to mounting, 1 µm 75% glycerol was added to the crystallization drop for cryoprotection. Diffraction data were collected on ESRF beamline ID23-1 (Nurizzo et al., 2006) using an X-ray beam of 10 µm in diameter with a flux of 3.5 × 10¹⁰ photons s⁻¹. The initial mesh scan produced a heat map (Fig. 5b) which was used as the basis for the collection of 54 partial data sets, of which 40 could be automatically processed. After HCA (Fig. 5c) 21 partial data sets were merged to produce a final data set to d_min = 1.6 Å (Table 1; Wilson plot shown in Fig. 5d). Structure solution and refinement (Table 2, Fig. 5d) were then carried out as described above for thaumatin (using PDB entry 4axt stripped of water molecules and ligands as the search model for molecular replacement; Cipriani et al., 2012), during which analysis of electron-density and difference electron density maps allowed the assignment of a nitrate (NO₃⁻) ion bound to one of the lysozyme molecules in the asymmetric unit (Fig. 5f).

Figure 5 Multi-crystal data collection and structure solution from crystals of monoclinic lysozyme. (a) The `flowers' of monoclinic lysozyme crystals produced by the crystallization procedure. (b) The heat map after an initial mesh scan of the sample used in the workflow described here. (c) Dendrogram based on HCA of CCI(i, j) values produced by XSCALE. (d) Wilson plot from the final data set derived using BEST. (e) Detail of the 2mFobs − DFcalc, αcalc electron-density map at the end of the refinement procedure (contoured at 1 × r.m.s; amino-acid residues shown in ball-and-stick representation). (f) Difference density (mFobs − DFcalc, αcalc) for a nitrate molecule at the end of the structure-refinement procedure (OMIT map). The difference density is shown at a contour level of 3 × r.m.s. (g) Plots showing comparisons of the completeness (top panel) and quality of data sets obtained following either the HCA-directed merging of data sets (21 data sets merged, blue) or the `blind' merging of 39 of the 40 data sets collected. (h) Difference density (mFobs − DFcalc, αcalc) for a nitrate molecule at the end of the structure-refinement procedure based on the data set obtained by merging 39 of the 40 data sets collected. The difference density is shown at a contour level of 2.5 × r.m.s.

3.4. Thermolysin

Bacillus thermoproteolyticus thermolysin (Sigma–Aldrich catalogue No. T0331) was dissolved to 100 mg ml⁻¹ in 45% DMSO, 0.05 M MES pH 6.0. The reservoir contained 35% saturated ammonium sulfate, whereas the drops were composed of the protein solution and a solution consisting of 0.05 M MES pH 6.0, 1 M NaCl, 45% DMSO in a 1:1 ratio. Rod-shaped crystals of between 40 × 40 × 150 and 40 × 40 × 300 µm in size were quick-soaked in 6 M trimethylamine N-oxide (TMAO; Mueller-Dieckmann et al., 2011) for cryoprotection before mounting on a sample support (Fig. 6). Diffraction data were collected using an X-ray beam of 10 µm in diameter with a flux of 4.0 × 10¹⁰ photons s⁻¹ at the peak of the Zn K absorption edge (λ = 1.256 Å) on beamline ID23-1 of the ESRF. The initial mesh scan produced a heat map (Fig. 6a) which was used as a basis for the collection of 96 partial data sets, 77 of which were automatically processed and 49 were manually merged after HCA analysis to produce a final data set to d_min = 1.37 Å (Table 1, Figs. 6b and 6c). Structure solution (Fig. 6d) was carried out using the SAD method (Dauter et al., 2002) using the SHELXC/D/E pipeline (Sheldrick, 2008) as implemented in HKL2MAP (Pape & Schneider, 2004), with the initial de novo-obtained model of the crystal structure refined (Table 2, Fig. 6e) using iterative rounds of REFMAC5 and manual rebuilding in Coot.

Figure 6 Multi-crystal data collection and SAD structure solution from crystals of thermolysin. (a) The sample holder mounted on ID23-1 immediately before launching the MeshAndCollect workflow. (b) The heat map after an initial mesh scan of the sample clearly shows the size and disposition of the crystals contained on the sample holder. (c) Dendrogram based on HCA of CCI(i, j) values produced by XSCALE. (d) A plot of CCall versus CCweak from SHELXD/HKL2MAP for trial substructures, clearly indicating successful substructure solution. (e) Detail of the final 2mFobs − DFcalc, αcalc electron-density map at the end of the refinement procedure (contoured at 1.5 × r.m.s; amino-acid residues in ball-and-stick representation). (f) Detail showing both anomalous difference map (ΔFano, αcalc + 90°) peaks (purple chicken wire) around the catalytic Zn2+ ion (grey sphere) and three Ca2+ ions (yellow spheres) and OMIT difference density (mFobs − DFcalc, αcalc, green chicken wire) in the region of a Val-Lys dipeptide found bound in the active site. The OMIT difference density is contoured at 3 × r.m.s. and the anomalous difference density at 4.5 × r.m.s.

Our experiments with crystals of thermolysin reveal other features of the developed pipeline. In particular, when, as was the case here, the sample holder contains a series of crystals much larger than the X-ray beam (Fig. 6a) multi-crystal/multi-position data collection is also automated. Indeed, for crystals that are larger than the X-ray beam the rapid online analysis and ranking of diffraction characteristics using DOZOR (§2.1) provides diffraction cartographs (Bowler et al., 2010) of the crystals contained on the sample mount (Fig. 6b). The workflow thus ensures that partial data sets are collected from only well diffracting areas of any given crystal.

3.5. MAEL domain of Bombyx mori Maelstrom

Diffraction data from crystals of the selenomethionyl derivate of the MAEL domain of B. mori Maelstrom (for crystallization conditions, see Chen et al., 2015) were collected using an X-ray beam of 10 µm in diameter with a flux of ∼9.5 × 10¹⁰ photons s⁻¹ at the peak of the Se K absorption edge (λ = 0.979 Å) on beamline ID23-1 at the ESRF. Crystals of this system (20–50 µm in the largest dimension) diffract rather poorly; therefore, in order to increase the data multiplicity to allow a more accurate determination of anomalous differences, six different sample holders were used in this experiment. The initial mesh scans produced heat maps (Fig. 7a) used to direct the collection of 137 partial data sets, 122 of which could be automatically processed and 45 of which were merged to produce a final data set to d_min = 3.46 Å after HCA (Table 1, Figs. 7b and 7c). Structure solution (Figs. 7d and 7e) was carried out using the SAD technique as implemented in the CRANK2 pipeline (Skubák & Pannu, 2013).

Figure 7 Multi-crystal data collection and SAD structure solution of Maelstrom. (a) Heat maps from initial mesh scans of the six sample holders analysed. (b) Dendrogram based on HCA of CCI(i, j) values produced by XSCALE. (c) Wilson plot from the final data set derived using BEST. (d) A plot of CCall versus CCweak from SHELXD/HKL2MAP for trial substructures, clearly indicating successful substructure solution. (e) Representative part of the 2mFobs − DFcalc, αcalc electron-density map after initial model building and refinement, with two α-helices shown in ribbon representation.

4.1. General comments

The method that we describe here, while similar to the multi-crystal data-collection methods for samples mounted in micro-meshes described previously (Soares et al., 2014), presents fundamental differences. Notably, a very low X-ray dose pre-screening of a sample mount is used to both identify the positions of crystals contained on the sample mount and to rank the diffraction characteristics of the crystals in order to create a priority for the subsequent automatic collection of partial data sets, and a HCA protocol is used to choose which partial data sets to merge to produce the best final data set. Moreover, when the sample holder contains a series of crystals much larger than the X-ray beam the method also automates the type of multi-crystal/multi-position data collection (Riekel et al., 2005) that has become essential in the structural study of G protein-coupled receptors (GPCRs; Rasmussen et al., 2011; Hollenstein et al., 2013; Lebon et al., 2011). Furthermore, for crystals larger than the X-ray beam the rapid online analysis and ranking of diffraction characteristics using DOZOR (§2.1) also provides diffraction cartographs (Bowler et al., 2010) of the crystals contained on the sample mount, ensuring that partial data sets are only collected from well diffracting areas of any given crystal.

To demonstrate the general applicability of the workflow described here, we have applied it to various systems and scenarios in which many crystals of the same type are mounted on the same cryocooled sample holder. In all of the cases presented our workflow has yielded data sets that are fit for purpose (Table 1, §4.2). As might be expected (Fry et al., 1996), the protocol described here is particularly amenable to systems (i.e. thaumatin, bacteriorhodopsin, Maelstrom) that crystallize in high-symmetry space groups. However, our experiments using monoclinic crystals of lysozyme show that the method can also be applied to low-symmetry systems. Furthermore, as the monoclinic form of lysozyme crystallized as clumps of intergrown crystals (Fig. 5a), the success of this latter experiment demonstrates that the protocol developed also automates the collection of diffraction data using mini-focus or micro-focus X-ray beams under conditions where mounting single crystals of a particular sample may prove to be difficult or impossible.

It is worth noting that the completeness of the data set obtained for monoclinic lysozyme following the HCA-directed merging of the partial data set collected is rather incomplete (21 of 40 automatically processed partial data sets merged, 85% completeness; Table 1). However, this is not the result of a combination of low-symmetry crystals lying in preferred orientations in the sample holder. Indeed, merging 39 of the 40 automatically processed partial data sets greatly improves the completeness (Fig. 5g). However, the quality of the resulting data set is seriously degraded compared with that obtained by merging only partial data sets in the main HCA cluster (Fig. 5g). Moreover, in contrast to what is observed following HCA-directed merging, the resulting difference electron density does not allow the proper identification of nitrate ions bound to the protein (Figs. 5f and 5h). It is thus clear that HCA is an indispensable tool for the proper merging of partial data sets. Nevertheless, that the merged data set for monoclinic lysozyme obtained following HCA is somewhat incomplete suggests, for some low-symmetry systems at least, that data collection from samples in two loops with different orientations in the X-ray beam may be required to ensure a fully complete, high-quality data set.

The examples that we present include the compilation of complete diffraction data from partial data sets collected from a series of microcrystals (∼5 µm in the largest dimension), contained on the same sample holder, of a membrane protein (bacteriorhodopsin) grown in lipidic mesophase. Such mesophases are very important media for the growth of membrane-protein crystals (Gordeliy et al., 2003), but are often opaque in nature, particularly when cooled. It can thus be challenging to identify, mount and centre in the X-ray beam small crystals produced in such media. That the workflow described here uses diffraction-based methods to identify the positions of crystals in a sample holder is clearly a major advantage in such cases as it obviates such problems, particularly when entire crystallization drops are harvested, by automating the collection of partial data sets from multiple crystals.

4.2. Structure solution and refinement

4.3. Perspectives

We have developed an automatic procedure to locate, rank the diffraction characteristics of and collect partial data sets from large numbers of crystals contained on the same sample holder. Subsequent HCA of the partial data sets collected then allows the choice of which partial data sets to merge to produce a final data set for downstream structure solution and refinement. Compared with previously presented SSX protocols (Gati et al., 2014; Nogly et al., 2015; Stellato et al., 2014), MeshAndCollect has several advantages, notably that small but contiguous data sets can, if desired, be collected from all crystals contained on the sample holder. Crystal wastage is thus not an issue, data reduction from raw diffraction images to structure factors and standard deviations is comparably straightforward and the quality of the final data set is improved. Moreover, the experiments described in §3 clearly demonstrate the capability of DOZOR to detect diffraction signal in low-dose two-dimensional mesh scans even for the smallest crystals (BR2; §3.1) studied in this work, which had an average volume of ∼50 µm³.

When starting this work, we presumed that cryocooled crystals contained on the same loop would be relatively isomorphous as all crystals are from the same crystallization drop and subject to similar handling during mounting and cryocooling (Giordano et al., 2012). The dendrograms shown in Figs. 2, 3, 4, 5 and 6 suggest that this is the case, although in several of our examples many of the partial data sets collected are not used to construct the final result. Most of the above histograms contain one main cluster with high mutual correlation coefficients and a continuum of data sets with decreasingly low correlation to the main cluster. Such a pattern is indicative of strongly varying data quality between partial data sets rather than crystal non-isomorphism and suggests that some partial data sets were collected from positions with overlapping crystal lattices or other issues such as crystal damage. Clearly, the evaluation of initial two-dimensional mesh scans with DOZOR did not filter such positions out. Furthermore, with only a 10° rotation range measured at each position it is difficult to detect such problematic data sets on the basis of their internal processing statistics, and HCA is required to filter out the worst partial data sets. In the case of Maelstrom, where partial data sets were measured from crystals on several different sample mounts, the dendrogram (Fig. 7) shows well populated clusters above a cutoff of dist(i, j) = 0.15 and a continuum of poorly correlated data sets below this cutoff. This suggests that both non-isomorphism and variation in data quality between partial data sets is present. However, as can be seen, both poor-quality and non-isomorphous partial data sets are successfully filtered by the HCA procedure.

Despite the success of the experiments described above, the procedure developed will eventually be improved in many areas. Here, all samples were mounted and cryocooled manually, and it may be that better results can be achieved by taking advantage of robotic crystal-handling methods both for the removal of mother liquor from the crystallization drop and the mounting and cryocooling of crystals in a suitable sample holder (Cipriani et al., 2012). Moreover, for the different experiments described here the total absorbed doses per crystal (Table 1; calculated post-experiment using RADDOSE; Paithankar & Garman, 2010) are rather low compared with the Henderson/Garman limits (Henderson, 1990; Owen et al., 2006) generally used in diffraction data collection from cryocooled single crystals of macromolecules. In future versions of the pipeline presented here, following the low-dose two-dimensional mesh scan the optimum total exposure time per crystal (partial data set) will be calculated before the data-collection step using the EDNA characterization software (Bourenkov & Popov, 2010; Incardona et al., 2009), the result being better quality and/or higher resolution data collected per crystal. For crystals that are highly radiation-sensitive one might even imagine the use of a `Burn Strategy' workflow (Leal et al., 2011) to provide a precise estimation of the maximum allowable total absorbed dose per crystal.

As the EDNA procedure implies the indexing of diffraction patterns (Incardona et al., 2009), comparison, for crystals larger than the X-ray beam, of orientation matrices will allow either the pre-clustering of partial data sets collected from different points on the same crystal or the measurement of crystal size and alignment in the sample holder. In the latter case this information could be used to automatically guide helical data collections (Flot et al., 2010; de Sanctis et al., 2012) that, provided that diffraction is homogenous, may allow the collection of complete data sets from each of the different crystals contained in the sample holder. For crystals of a similar or smaller size than the X-ray beam prior knowledge of the crystal orientation in the X-ray beam will allow a broader range of experiments than is currently the case. In particular, the order of the collection of partial data sets could be constructed to ensure the compilation of a complete data set when only a few crystals are available or to ensure the collection of as highly redundant data as possible. Finally, for sample mounts containing many small robust, well diffracting crystals one can also imagine a modification to the pipeline in which complete diffraction data sets for structure solution and subsequent refinement are collected from all crystals contained in the sample holder. Separating such data sets into different clusters would result in ensembles of crystal structures for each target.

Once data collection and processing have been completed, a final improvement to the pipeline is in the choice of partial data sets to merge to produce a final data set. This choice clearly depends on the aim of the experiment in hand (i.e. structure solution by molecular replacement, de novo structure solution using SAD etc.), and in principle is best made using HCA based on CC_I(i, j) (§2; Giordano et al., 2012). However, for partial data sets from low-symmetry crystals the number of common unique reflections for each pair of data sets may be low, thus leading to artefacts, and a better approach may be to combine HCA with the type of `scale-and-merge' algorithms currently implemented in the PHENIX package (Adams et al., 2010; https://www.phenix-online.org/version_docs/dev-1977/reference/scale_and_merge.html ) or recently described for other SSX protocols (Huang et al., 2015).