STRUCTURAL DYNAMICS 2, 041714 (2015) Data collection strategies for time-resolved X-ray free-electron laser diffraction, and 2-color methods Chufeng Li, Kevin Schmidt, and John C. Spencea) Department of Physics, Arizona State University, Tempe, Arizona 85287, USA (Received 18 February 2015; accepted 29 May 2015; published online 12 June 2015) We compare three schemes for time-resolved X-ray diffraction from protein nanocrystals using an X-ray free-electron laser. We find expressions for the errors in structure factor measurement using the Monte Carlo pump-probe method of data analysis with a liquid jet, the fixed sample pump-probe (goniometer) method (both diffract-and-destroy, and below the safe damage dose), and a proposed two-color method. Here, an optical pump pulse arrives between X-ray pulses of slightly different energies which hit the same nanocrystal, using a weak first X-ray pulse which does not damage the sample. (Radiation damage is outrun in the other cases.) This two-color method, in which separated Bragg spots are impressed on the same detector readout, eliminates stochastic fluctuations in crystal size, shape, and orientation and is found to require two orders of magnitude fewer diffraction patterns than the currently used Monte Carlo liquid jet method, for 1% accuracy. Expressions are given for errors in structure factor measurement for the four approaches, and detailed simulations provided for cathepsin B and IC3 crystals. While the error is independent of the number of shots for the dose-limited goniometer method, it falls off inversely as the square root of the number of shots for the two-color and Monte Carlo methods, with a much smaller pre-factor for the C 2015 two-color mode, when the first shot is below the damage threshold. V Author(s). All article content, except where otherwise noted, is licensed under a Creative Commons Attribution 3.0 Unported License. [http://dx.doi.org/10.1063/1.4922433] I. INTRODUCTION In recent experiments aimed at the measurement of structure-factors using a free-electron laser (XFEL),1 protein nanocrystals are sprayed in single-file across a pulsed hard-X-ray beam, using a technique known as serial femtosecond X-ray crystallography (SFX). The crystals, often of submicron dimensions, vary in size, are randomly oriented, and are destroyed by the beam after providing a high-resolution diffraction pattern. In addition, the intensity of the X-ray beam may vary from shot to shot by up to 15%, and the time-structure of the femtosecond pulses used also varies from shot to shot. Diffraction patterns are read out at perhaps 120 Hz, so that large amounts of data are collected. Nevertheless, using improved data analysis methods, the number of diffraction patterns needed to determine a structure at better than 0.2 nm resolution has recently been reduced to less than 6000.24 The extraction of structure factors then requires an integration across the angular width of the Bragg reflections from these many “stills,” snapshots, or partial reflections, in each of which the Ewald sphere cuts through a small slice of the intensity distribution around each Bragg condition. For the smallest nanocrystals, containing perhaps just a few dozen unit cells, since the XFEL is spatially coherent, and assuming that the beam is wider than the crystal, the Bragg spots are broadened by “shape transform” functions;1,11 for larger crystals, mosaicity may be present.2 The case of a coherent beam smaller than the crystal (or smaller than one mosaic block, or unit cell) is discussed elsewhere.18 a) Author to whom correspondence should be addressed. Electronic mail: spence@asu.edu. 2329-7778/2015/2(4)/041714/19 2, 041714-1 C Author(s) 2015 V 041714-2 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) Where a wide beam illuminates a mosaic crystal, slightly tilted blocks of crystal monochromate different component wavelengths of the beam, scattering them into slightly different directions around the Bragg spots, across which an integration is required. The divergence of the incident beam and the energy-spread in the beam must also be considered, since these contribute to the "thickness" of the Ewald sphere. These considerations lead to the well-established multiplescattering theory of primary and secondary extinction in mosaic crystals,19 which assumes incoherent multiple scattering between blocks but coherent multiple scattering within each block. We do not consider that here, since a modern XFEL coherent beam diameter of 200 nm is comparable with a typical mosaic block size, and the mosaic block model may not apply to the layer structures such a membrane proteins.22 In all cases, the precise deviation of the diffraction conditions from the ideal Bragg condition is needed for each spot in every shot in order to estimate the degree of partiality for each reflection. So far, it has not been possible to measure this quantity directly; however, several groups have recently used optimization methods to estimate partiality.2,15,16,24 Building on earlier synchrotron work,21 pump-probe SFX experiments3,4,20 have also been undertaken, aimed at imaging time-resolved changes (TR-SFX) in three-dimensional protein charge-density maps due to optical illumination, such as that which occurs in photosynthesis. In a typical experiment, alternate nanocrystals in a liquid jet stream might be illuminated optically (causing a change in structure factors) and the difference in the measured intensities between illuminated (bright) and un-illuminated (dark) angle-integrated Bragg reflections is used, after phasing, to provide a real-space density map showing the change in molecular structure due to illumination. The differences are taken between a very large number of bright and dark nanocrystals of different sizes (leading to large scale-factor differences covering orders of magnitude). In this paper, we obtain expressions for the number of patterns needed to reduce the errors in structure factor measurement to below that needed to observe optical pumping effects, using three different methods, which we compare. In order to merge data (by adding together Bragg partial reflections with the same Miller indices from nanocrystals of different sizes), subject to these many sources of stochastic variation, it was suggested that the only reasonable method is a Monte Carlo (MC) type of angular integration across the Bragg reflections, in which the angular coordinate then consists of a random sample of abscissa (crystal orientation) values. This integration will then average over all stochastic fluctuations, such as shot-to-shot beam intensity variation and differences in crystal size. The contributions of these fluctuations to the final structure factor measurement might then be expected to add in quadrature, giving a signal-to-noise ratio (SNR) which improves as the square root of the number of diffraction patterns, and this behavior has been confirmed experimentally.5 Thus, a hundred times more data are needed to add one significant figure to the results. Improvements on this behavior require experimental characterization of the sources of error and their distributions and more accurate specification of experimental parameters, such as the assignment of a scattering vector to each pixel on the area detector and deviation from the exact Bragg condition. Model-based data analysis methods using the EMC (Expectation maximization and compression) algorithm also show great promise for the smallest crystals.6 More recently, we have been involved with experiments in which data are collected from larger crystals in a fixed orientation mounted on a goniometer, with provision to scan the sample to a new position laterally. For pump-probe experiments, the incident X-ray intensity can be adjusted for either destructive readout (in which case, the sample must be translated after each shot has drilled a hole in the sample) or defocussed to a level below the damage threshold, giving a poorer statistics.7,8 In principle, this method allows measurements at equally spaced increments across the rocking curve, with a known abscissa error; however, the total dose for all exposures must fall below the Henderson safe dose.28 Finally, new modes of XFEL operation have been demonstrated, dubbed “split and delay,” in which the coherent X-ray beam is split into two beams of slightly different wavelengths, with the femtosecond pulse in one beam delayed relative to the other.9,10,25 Several methods are possible including a “slotted foil,” the use of mirrors (for softer X-rays with high efficiency) and Bragg crystal splitters (harder X-rays with lower efficiency). The two beams can be focused 041714-3 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) onto the same sample, arriving at slightly different times and beam energies, or from slightly different directions at the same energy. Delays are currently in the range of 100 fs but could, in principle, be extended to the range more useful for biology or organic chemistry (with very long path lengths), in which case a pump laser could be inserted between the two pulses of a pair, and both diffraction patterns then impressed on the same detector readout. For larger nanocrystals, the sharp partial Bragg spots at the two slightly different beam energies will then be displaced on the detector, and the intensity differences are merged to provide a difference density map after phasing. By obtaining pairs of diffraction patterns from the same nanocrystal (before and after optical illumination), errors due to both size and orientation variation are eliminated; however, the first pulse must clearly not destroy the sample, resulting in poorer SNR relative to the diffract-and-destroy mode. Among the methods developed at LCLS for split-anddelay research, different limitations apply. Use of mirrors limits the X-ray energy to below 2 keV and a short time delay, thus cannot provide high-resolution reflections needed for biological imaging. Bragg crystals used as beam splitters result in excessively collimated and monochromatic pulses giving low efficiency in structure factor measurement. Similarly, a two-color scheme based on use of two sets of undulators generates two X-ray pulses at slightly different energies (2% difference) and separated in time with an adjustable delay up to 40 fs, potentially extendable to up to 200 fs.9 This two-color approach, which is also applicable to hard-X-rays with time delay from a few femtoseconds up to 200 fs, is most suitable for the study of the earliest stages of conformational change and bond formation in biochemistry. We therefore focus our analysis and discussion on these two-color approaches in Secs. II B and III B. In this paper, we compare the accuracy of structure factor measurement for each of these modes for pump-probe time-resolved diffraction experiments, in which the error should be less than the changes in structure factor due to pump illumination. Since many poorly characterized experimental factors influence such a complex comparison (such as crystal quality, jet hit rate, sample concentration, and fixed-sample scan time), we make here a simplified comparison which focuses on establishing signal-to-noise ratio as a function of number of shots for each method, with other factors equal. Some of the many additional experimental considerations might include the following. For irreversible processes, the pump laser must be directed to a new area (or crystal) for each shot. Since Laue diffraction is not possible using an XFEL, many shots (both bright and dark) are needed in the vicinity of every Bragg condition to perform the required angular integration over these partial reflections. With many pixels within the angular profile of the Bragg reflection, the intensity of these partial reflections is proportional to the square of the number of electrons in the illuminated region of the sample, while the angleintegrated intensity is proportional to the number of electrons or molecules. A doubling of beam size on a large crystal by defocus (with constant number of photons per shot) leaves the intensity of Bragg beams unchanged (in the absence of damage). The ideal maximum of diffraction information is obtained with the largest possible ideally imperfect crystal fully illuminated at a level below the Henderson safe dose. (This maximizes the number of undamaged molecules contributing to the diffraction pattern.) The use of diffract-and-destroy methods allows a dose of up to 100 times this safe dose without damage, in principle, providing much more intense high angle scattering and so better resolution, with data obtained from submicron regions of crystals, in some cases thereby reducing the contribution from defects. The use of femtosecond pulses allows us to outrun radiation damage effects at all beam intensity levels (including low intensity), while subsequent vaporization of the crystal at high intensities prevents the collection of pumped data from the same crystal. The theory of diffraction from protein nanocrystals is given elsewhere;5 the theory of diffraction from larger mosaic crystals is given in textbooks.19 II. XFEL-BASED APPROACHES FOR STRUCTURE FACTOR MEASUREMENT A. Monte-Carlo approach The MC approach5 merges all diffraction data from many crystal sizes and sample orientations and performs a simple average over microcrystal size, shot-to-shot beam intensity, and the 041714-4 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) partiality of the reflections. For different patterns, the Ewald sphere intersects the Bragg orders in reciprocal space at different points on the intensity distribution. The intensity of the reflection is thus dependent on the crystal size and orientation, assuming a parallelepiped crystal as11 I ð iÞ       ð iÞ ð iÞ ðiÞ ðiÞ ðiÞ ðiÞ 2 2 2 sin N W sin N W sin N W 3 3 1 1 2 2 ðÞ       DX; ¼ I0i jFðDkÞj2 re2 Pðk0 Þ ðiÞ ð iÞ ðiÞ 2 2 2 sin W1 sin W2 sin W3 (1) where the sin2 terms are known as "shape transforms" and the N(i)’s represent the number of unit cells in a given dimension (hence crystal size), and ðiÞ W1 ¼ pDk  aðiÞ ; ðiÞ ðiÞ W2 ¼ pDk  bðiÞ ; W3 ¼ pDk  cðiÞ : (2) The superscript index “(i)” indicates the “i”th shot event. a(i), b(i), and c(i) are the lattice vectors of the nano-crystal in the frame fixed to the laboratory at the “i”th shot. The extracted structure factor is estimated from the average intensity of the Bragg beam over all shots with index(i)       ðiÞ ðiÞ + ðiÞ ðiÞ ð iÞ ð iÞ 2 2 2 sin N W sin N W sin N W 3 3 1 1 2 2 ðÞ       hIðiÞ i ¼ I0i jFðDkÞj2 re2 Pðk0 ÞDX; ð iÞ ðiÞ ðiÞ 2 2 2 sin W1 sin W2 sin W3     * 2  ð iÞ ð iÞ  ðiÞ ðiÞ + ðiÞ ðiÞ 2 2 sin N W sin N W sin N W 3 3 1 1 2 2 ðÞ       hI0i ijFðDkÞj2 re2 Pðk0 ÞDX; ðiÞ ð iÞ ðiÞ 2 2 2 sin W3 sin W1 sin W2 * 2  ðiÞ ðiÞ +* 2  ðiÞ ðiÞ +* 2  ðiÞ ðiÞ + sin N1 W1 sin N2 W2 sin N3 W3 ðÞ        re2 Pðk0 ÞDX: (3) hI0i ijFðDkÞj2 ðiÞ ðiÞ ð iÞ 2 2 2 sin W1 sin W2 sin W3 * ðiÞ If we use CðhklÞ to denote the combined effect of crystal size, orientation, and other constants, then Eq. (3) can be written in the following form: ðiÞ ðiÞ IðiÞ ¼ I0 jFðDkÞj2  CðhklÞ ; 0 ðÞ CðihklÞ      1 ðiÞ ðiÞ ðiÞ ðiÞ ðiÞ ðiÞ 2 2 2 sin N W sin N W sin N W 3 3 C 2 1 1 2 2 B       Are Pðk0 ÞDX; ¼@ ðiÞ ð iÞ ðiÞ 2 2 2 sin W3 sin W1 sin W2 (4) ðiÞ ðiÞ hIðiÞ i ¼ hI0 ijFðDkÞj2 hCðhklÞ i: (5) (6) The structure factors can then be estimated from the average Bragg beam intensity using the following relation: 2 jFexp ðhklÞ j ¼ hIðiÞ i ðÞ ðÞ hI0i ihCðihklÞ i ; (7) ðiÞ where hCðhklÞ i includes the average shape transform, which can be modeled, based on experimental parameters. As shown elsewhere,23 this average shape transform is a smooth curve, rather than the sinc-function profile of a single cubic nano-crystal. B. Two-color approach for pump-probe experiments The two-color approach offers the possibility of eliminating the randomness of several stochastic variables, as shown below. The first of a pair of pulses hits a nano-crystal and, after a 041714-5 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) set time delay, the second hits the same crystal in an identical orientation, since the rotational diffusion time of micron-sized microcrystals in the buffer solution of a liquid jet is much larger than the delay. Between these two pulses, the crystal may be pumped optically; however, the first X-ray pulse must not cause damage, and if it excites the crystal, sufficient time must be allowed for the excitation to decay before optical pumping. Both patterns are recorded by the detector within the same read-out event. Since the two patterns are from two pulses with slightly different wavelengths, they can be separated in data analysis, if the crystals are large enough to minimize an overlap of the diffraction spots at the two wavelengths. This method is not restricted to the use of a liquid or viscous jet, and our analysis can equally be applied to microcrystals mounted on a scanned fixed-target arrangement. Since the two diffraction patterns are from almost the same scattering geometry, the intensities may be expressed as ðiÞ ðiÞ ðiÞ ðiÞ ðiÞ ðiÞ I1;ðhklÞ ¼ I01  jF1;ðhklÞ j2  C1;ðhklÞ ; I2;ðhklÞ ¼ I02  jF2;ðhklÞ j2  C2;ðhklÞ ; ðiÞ ðiÞ C1;ðhklÞ  C2;ðhklÞ ; (8) (9) (10) where the indices “1” and “2” indicate the first and the second of the paired pulses, or ground state and excited state. As can be seen from Eqs. (8)–(10), the beauty of the two-color approach is that we can now divide out the common orientation factor to obtain the change in structure factor amplitude ðÞ RðihklÞ      D jFðhklÞ j jF2;ðhklÞ j  jF1;ðhklÞ j ¼ ¼ ;   jF1;ðhklÞ j jF1;ðhklÞ j vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u ðiÞ u uI ðÞ ujF2;ðhklÞ j2 u 2;ðhklÞ I i t  1 ¼ t ðiÞ  01  1; ¼ 2 ðÞ jF1;ðhklÞ j I1;ðhklÞ I02i vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u ðiÞ uI qffiffiffiffiffiffiffiffiffiffi u 2;ðhklÞ ð Þ ðÞ ¼ t ðiÞ  k12i  1 ¼ AðihklÞ  1; I1;ðhklÞ ðÞ k12i ðÞ ðÞ  I01i ðÞ I02i (11) ; ðÞ AðihklÞ  I2;i ðhklÞ ðÞ I1;i ðhklÞ ðÞ  k12i ; (12) ðiÞ where k12 denotes the ratio of the first pulse intensity to the second for the (i)th shot. The ratio of the change in Bragg beam intensity is independent of crystal size and orientation. It is equal to the ratio of the change in the squared structure factor magnitudes. Experimentally, this means that each frame from paired pulses which contains two slightly displaced diffraction patterns gives exactly the same ratio of the change in the Bragg beam intensity. The randomness in crystal size and orientation are therefore eliminated, suggesting that this two-color approach might be superior to a Monte-Carlo approach in a liquid jet, where bright and dark differences are taken from crystals of different sizes. However, the weak signal from the first pulse (needed to avoid damaging the sample) degrades SNR. C. Large crystal fixed on a goniometer For fixed-sample experiments, the sample orientation can be controlled using a goniometer to allow a slow scan across reflections from a large single crystal at controlled increments for both bright and dark conditions. The total dose deposited in the sample must be lower the Henderson safe dose to obtain damage-free data. If the diffract-and-destroy mode is used 041714-6 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) (drilling holes with the beam in a large crystal), the many orientations and bright and dark conditions must all be obtained from different regions of the same crystal, separated by several microns to allow for the range of damage and strain caused by hole-drilling. This approach has the advantage of allowing a much higher dose13 (with resulting stronger high-angle scattering) and the absence of radiation damage on the Bragg data. A third possibility uses microcrystals trapped, perhaps by filtration, on the sites of a calibrated lattice in random orientations. Then, under diffract-and-destroy conditions, bright and dark data are collected from different microcrystals, and the methodology is similar to the GDVN (Gas Dynamic Virtual Nozzle) liquid jet, but with a hit rate approaching 100% and possibly slower readout, depending on scan speed. If a goniometer and large crystal are used (either above or below the damage threshold), the extracted structure factor from a series of exposures around Bragg conditions is jFes j2 ¼ Ns X ðIi  DWÞ; i¼1 ¼ DW Ns X Ii ; i¼1 (13) Ns X Ii ¼ WT i¼1 Ns ; where WT is the effective angular width of the abscissa of Bragg reflection that is scanned across and DW is the sampling increment of the scanning process. Ii is the measured intensity of the ith sampled point and Ns is the number of sampling points across the reflection. III. ERROR METRICS In order to determine if the two-color (or split-and-delay) approach is more accurate than the Monte- Carlo method, the errors in structure factor extraction are estimated below for both approaches. In addition, we determine approximately the number of patterns needed to achieve a given accuracy in structure factor, and whether it is feasible for both approaches, with a 15% beam intensity fluctuation, to identify a 1% change in structure factors. A. Monte-Carlo approach The extracted structure factor converges to its true value by Monte-Carlo integration over 5 crystal size, orientation, and beam intensity p fluctuation. This convergence has a diminishing ffiffiffiffi efficiency described by error reduction as 1= N , which makes Monte-Carlo approach wasteful of protein sample and beam resources. For the study of radiation damage dynamics or subpico-second time-resolved imaging, the change in structure factor is very small and likely to be less than 10% at best, and 1% in some cases. To recognize this small change from random errors, a huge number of patterns may be needed; nevertheless, near-atomic resolution “movies” of the photo-detection cycle in photo-sensitive bacterial yellow protein have recently produced by this approach.20 In the following, we estimate this number based on error analysis. The error in structure factor from each shot can be derived from Eq. (7) based on error propagation as11 vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u !2 !2 0  ðiÞ 12 u ð iÞ ð Þ ð Þ i i u ð Þ ð Þ rMC ðjFjÞ 1 t r I rC @r I0 A ; ¼ þ þ ðÞ hjFji 2 hIðiÞ i hCðiÞ i hI0i i (14) where “r” denotes the error (or standard deviation) in each random variable and “hi” represents the average value. After merging N patterns by Monte-Carlo integration pffiffiffiffiover crystal size and orientation, the error in the structure factor is reduced by a factor of 1= N 041714-7 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u !2 !2 0  ðiÞ 12 u ðNÞ ð Þ ð Þ i i u r I0 ð Þ ð Þ rMC ðjFjÞ 1 rI rC ¼ pffiffiffiffi t þ þ @ ðiÞ A : ð Þ ð Þ i i hjFji hI i hC i 2 N hI0 i (15) If we now neglect the error in intensity detection due to shot noise for a relatively strong Bragg beam, then the first term in the parenthesis in Eq. (15) vanishes. Also, for the purpose of approximation, a Monte-Carlo simulation has been conducted to obtain the approximate percentage error (ratio of standard deviation to mean) in the C(i) factor, which represents the effect of crystal shape and orientation. For crystals of Trypanosoma brucei cysteine protease cathepsin B (TbCatB) used recently,12 the value of the relative error in C(i) was found to be 5.7 for microcrystals of 0.9  0.9  11 lm average size and 10% deviation, with Gaussian distribution (see Appendix A). The shot-to-shot beam intensity fluctuation is 15%, so that the percentage error in a structure factor extracted using the Monte-Carlo approach is N ðjFjÞ rMC 1 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 2:85 ¼ pffiffiffiffi 5:72 þ 15%2 þ 0%2 ¼ pffiffiffiffi : hjFji 2 N N ð Þ (16) Therefore, for a 1% error tolerance in structure factor magnitude jFj, up to 8.12  104 patterns with the Bragg order (hkl) sampled are needed to achieve this accuracy. From the above analysis, the dominant error contribution comes from the random variation in crystal size, shape, and orientation represented by the first term under the root sign in Eq. (16). The contribution from the shot-to-shot intensity fluctuation represented by the second term could be reduced or even eliminated by measuring the intensity of the incident beam for each shot; however, this required the assumption that the beam hits the center of the crystal, not the side, and these "impact parameters" also affect scaling. Although this effect is relatively small compared to the first term, it does make the Monte-Carlo integration converge faster, and the extracted structure factors achieve a higher accuracy. B. Two-color approaches for TR-SFX The two-color approach determines changes in structure factors from two diffraction patterns that are recorded by pulse pairs from the same crystal in the same orientation. Therefore, ðiÞ for each shot (i), RðhklÞ is independent of the crystal size, shape, and orientation ðÞ RðihklÞ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u ðiÞ intensity uhI u 2;ðhklÞ ij ðÞ ¼ t ðiÞ k i  1 ¼ RðhklÞ ; intensity 12 hI1;ðhklÞ ij (17) where R(hkl) denotes the true value of the change in magnitude of the structure factor (hkl), and k12 is given by Eq. (12). ðiÞ 0ðiÞ We estimate RðhklÞ with RðhklÞ by replacing k12 and the recorded intensities with their expectation values, giving 0ðiÞ RðhklÞ vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u ðiÞ ð iÞ uI qffiffiffiffiffiffiffiffiffiffi u 1;ðhklÞ  hk12 i ðÞ ¼t  1 ¼ AðihklÞ  1: ð iÞ I1;ðhklÞ (18) Thus, for each shot (i), the error in the estimate of the change in magnitude of structure magni0ðiÞ tude RðhklÞ is 041714-8 Li, Schmidt, and Spence  0ðiÞ r RðhklÞ Struct. Dyn. 2, 041714 (2015)  ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffi v AðhklÞ i u 1 u 1  t ðiÞ þ ðiÞ þ a2 ; ¼ 2 I1;ðhklÞ I2;ðhklÞ h rðk12 Þ : hk12 i a (19) 0ðiÞ The percent error in RðhklÞ for one shot is inversely related to the intensity of the Bragg beam and directly related to the percentage error in k12, which is denoted by a. Thus, brighter Bragg beams give smaller errors and weaker ones give larger errors. Even for a particular Bragg order and constant incidence fluence, different shots correspond to different points on the rocking curve and thus give different Bragg beam intensities. Therefore, to reduce the error in determination of the percent change in structure factor magnitude, we make use of data from all shots by assigning a weighting function which weighs brighter reflections more than weaker ones [Eq. (20)]. Alternatively, we may simply sum up the intensities from all shots for the same Bragg reflection (hkl) and take the ratio of the sums [Eq. (21)]. This is actually a self-weighted average with the weighting function being the intensity itself. These two methods can be shown ðiÞ to be equivalent, with a proper choice of the weighting function Wop as shown in Eq. (24) 1: ðNÞ AðhklÞ ¼ N X ðiÞ 0ðNÞ W ðiÞ AðhklÞ ; RðhklÞ ¼ qffiffiffiffiffiffiffiffiffiffi ðNÞ AðhklÞ  1; (20) i¼1 N X ð Þ N Aðhkl Þ ¼ 2: i¼1 N X ðÞ I2;i ðhklÞ  k12  1; ð Þ N Rð0 hkl Þ ¼ qffiffiffiffiffiffiffiffiffiffi ðNÞ Aðhkl Þ  1; (21) ðÞ I1;i ðhklÞ i¼1 ðÞ ðiÞ Wop ¼ I1;i ðhklÞ N P i¼1 : (22) ðÞ I1;i ðhklÞ 0ðiÞ It is shown that RðhklÞ is indeed a valid estimate of RðhklÞ , the true value of relative change 0ðiÞ in structure factor magnitude jFðhklÞ j, and shows that the average value of RðhklÞ approaches the true value RðhklÞ if the number of shots N is sufficiently large (Appendix B). According to the theory of error analysis,11 the errors in measured variables propagate into 0 R according to pffiffiffi rðR0 Þ ¼ r A ; vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u 2 N  u X ð iÞ u I u 2 1 pffiffiffiffiffiffiffi u 1 1 i¼1 2 hAi  u þ þ a  ¼ !2 ; uX N N X N 2 X u ðiÞ ðiÞ ð Þ t I2 I1 I2i i¼1 ¼ i¼1 1 pffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi hAi  T1 þ T2 þ T3: 2 i¼1 (23) We now discuss the error contributions from the terms T1, T2, and T3 above. In Eq. (23), T1, T2, and T3 can be approximately evaluated directly from experimental data, for an given ðiÞ value of the number of shots N. This requires simulations using a full data set of reflections I1 ðiÞ and I2 . For a small value of N, this is necessary and can be readily undertaken. However, in case of a large value of N, it is impractical and unnecessary since the sampling can cover the 041714-9 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) whole intensity distribution of Bragg reflections ergodically, with much less fluctuation than for small values of N. We therefore estimate the error in R0 using the expectation value of the ðiÞ ðiÞ refection intensity I1 and I2 over the entire intensity distribution 1 pffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi hAi  T1 þ T2 þ T3; 2 sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi   1 pffiffiffiffiffiffiffi 1 1 1 hAi  N!1 þ ðiÞ intensity þ 1 þ b2 a2  pffiffiffiffi ; ð Þ intensity i 2 N hI2 ijshots hI1 ijshots rðR0 Þ ¼ ðiÞ (24) (25) ðiÞ and hI2 ijintensity are the expectation values over the distribution of Bragg where hI1 ijintensity shots shots reflection intensities from the first and second pulses, respectively. b is the relative standard ðiÞ deviation in I2 over the rocking curve, and a denotes the relative error in k12 . The ratio of the intensities of the two pulses k12 varies from shot to shot, and this fluctuation is characterized by a and determined by the stability of the emittance spoiler as well as the photon generating process (SASE (self-amplified spontaneous emission) or self-seeded).9 Using the two-color approach, this may depend on the stability of the seeding process. As shown by Eq. (25), with a sufficiently large number of diffraction patterns, the error in R0 depends on the Bragg beam intensity via T1 and T2, the accuracy of the incident flux ratio a, and the statistics of nano-crystal size, shape, and orientation distribution b (which may be evaluated by Monte-Carlo simulation) via T3. Among the three contributing terms, T1 and T2 are dependent on experimental conditions, such as the flux of the two pulses and their ratio, while T3 is determined by the photon generation stability, and the nano-crystal samples. Contributions from these terms are determined by the parameters of the sample and the experimental settings, such as the statistics of nano-crystal size, shape, orientation, X-ray flux, the relative intensity of the paired pulses, and the stability of the LCLS system. Without involving specific instrumental specifications and parameters, we can discuss below two different regimes of experiments: a relatively high flux of both of the paired pulses with unstable beam intensity ratio (e.g., two-color) and low flux for the first pulse, with perfect beam intensity control (as expected from a beam-splitting device). In the case of high X-ray flux and unstable beam intensity, we expect small Poisson noise due to counting at the detector, but a large error in control of the relative intensity of the two pulses. Then, in Eq. (25), T3 would dominate over the negligible terms T1 and T2. Assuming the same value of b ¼ 5.7 as in the Monte-Carlo approach, the error in R0 is pffiffiffiffiffiffiffi 1 rðR0 Þ  2:89 hAi  a  pffiffiffiffi ; N (26) which indicates that the error in the determination of the relative change of structure factor magnitudes is proportional to the relative error in the intensity ratio of the two paired pulses and hence depends on the stability of the emittance spoiler and the photon generation process. This error decreases as the square root of the number of patterns pffiffiffiffiffiffiffi recorded, which is similar to the Monte-Carlo approach [Eq. (15)] but with a prefactor hAi  a. Comparing split-and-delay and Monte-Carlo approaches, we can easily establish a criterion for superiority of the former over the latter a pffiffiffiffiffiffiffi hAi < 1: (27) For 20% change in structure factor magnitude as an example, the critical value of a is 0.83. In other words, any two-color system with an error of less than 83% in intensity ratio makes the two-color approach preferable. In the case of a weak first pulse (which does not destroy the sample) but with perfect beam intensity control, the Poisson noise T1 and T2 become the dominant error contribution rather than the negligible relative intensity fluctuation T3. Then, the error in R0 is 041714-10 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u 1 p ffiffiffiffiffiffiffi 1 1 u hAi  u N rðR0 Þ  þ N ; uX ð Þ X 2 ðiÞ t Ii I1 2 0 i¼1 i¼1 sffiffiffiffiffiffi1 pffiffiffi p ffiffiffiffiffiffiffi 1A 1 2 pffiffiffiffi : hAi  @ 2 hIi N (28) Thus, the error is now independent of the specific statistics of the nano-crystal samples and is only determined by the summed reflection intensities from all patterns. Additionally, a smaller error is expected for a brighter Bragg reflection than a weaker one. For TbCatB crystals of 0.9  0.9  11 lm average size, assuming structure factors F  104, an X-ray beam with photon energy of 9.4 keV, and beam diameter of 4 lm, at the Henderson "safe-dose" limit28 of 1 MGy at room temperature (allowing study of dynamics), the average number of photons of a reflection in a pattern is estimated to be 77. Hence, the error in R0 is 1 rðR0 Þ  0:087 pffiffiffiffi : N (29) For two-color experiments, the intensity or energy of each pulse can be measured by using a inline spectrometer.26 In this case, the uncertainty in k12, denoted by a, becomes dependent on the accuracy of the intensity measurements. The error in R0 is then equal to that given by Eq. (28). C. Fixed-sample experiments with goniometer With sample fixed to a holder and a goniometer, the crystal orientation can be controlled accurately to facilitate scans across the rocking curve. In contrast to the stills obtained from different crystals in random orientations, this scan process may generate a sampling over the angular profile of the Bragg reflections with equally spaced increments and the relative error due to Poisson noise in intensity measurement is vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi u 2  rR ðjFes jÞ 1 u rðDWÞ  1 ¼ t 1 þ b2 Ns þ ; hjFes ji 2 WT hIijshots intensity Ns ¼ 1 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi E1 þ E2; 2 (30) where W is the angular variable, as the abscissa of the rocking curve. WT is the total angular width of the rocking curve and DW is the sampling increment. Ns is the number of sampling points and hIijshots intensity denotes the mean intensity of each sample point averaged over both Poisson noise and the entire rocking curve. b is the relative standard deviation in measured intensity over the rocking curve, consistent with the previous discussion of the Monte-Carlo and two-color approaches. Beside the error contributions from goniometer control and intensity measurement, another contribution comes from the systematic error resulting from integration by quadrature. For one-point quadrature, the error is proportional to square of the sampling increment DW and the first derivative of the curve f 0 13     1 rS jFes j2 ¼ O f 0  DW2 / 2 : Ns (31) In the destructive-readout mode, where the X-ray beam must be translated to a fresh point on the sample sufficiently far away from the hole drilled by the previous shot to avoid damage, fixed-sample experiments sampling rocking curves with even increments and maximum beam ffi as indicated by Eq. (30). (We assume intensity would give a random error which goes as p1ffiffiffi Ns perfect goniometer control.) In this regard, fixed-sample experiments and M.C. experiments are 041714-11 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) essentially equivalent from the point of view of error reduction and data efficiency. However, the prefactor in the M.C. approach is much larger than in the fixed-sample approach since the former uses a random sampling, whereas crystal orientation and sampling are totally controllable using a goniometer. For CXI beam line at LCLS, with a typical pulse energy of 2 mJ, the estimated average photon counts for the same condition are approximately 100 times than that of the non-destructive mode resulting in the prefactor of 0.0057. We must also note that, however, the number of shots we can take on a single large crystal Ns is limited by the crystal size as well as the safe distance between shots to avoid radiation damage caused by previous shots. Therefore, an upper limit might exist for the accuracy in structure factor measurement using this diffract-and-destroy mode in fixed sample experiments. If the beam intensity is adjusted below the Henderson safe dose threshold so that the sample is not destroyed, the error from a fixed sample is then rR;min ðjFes jÞ 1 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ¼ E1 þ E2; hjFes ji 2 sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 pffiffiffiffiffiffi 1 1  E2 ¼ ; 2 2 hIijshots intensity Ns sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 1 1 ; / jFj 2 DH qLA (32) where DH is the Henderson safe dose, q is the mass density, L is the attenuation length of the sample, jFj is the magnitude of the structure factor, and A is the effective beam area. Equation (32) indicates that the random Poisson error in detector counts is independent of the number of sampling points on the rocking curve and is only dependent on the sample and X-ray beam parameters. This is reasonable, since the total photon signal is limited by the Henderson safe dose no matter how many sampling points are used in a scan. Therefore, combining systematic and random errors for consideration, an optimal value of Ns exists for minimal error [Eqs. (31) and (32)]. To determine the experimental design, detailed simulations need to be carried out to estimate the errors in the different approaches for specific samples. For TbCatB crystals12 of 0.9  0.9  11 lm average size, assuming a structure factor F  104, a photon energy of 9.4 keV, and a beam diameter of 4 lm, the Henderson dose limit of 1 MGy at room temperature, we show the number of patterns needed to achieve 1% accuracy in structure factor measurement for Monte-Carlo, two-color, and goniometer-based XFEL experiments in Fig. 1. pffiffiffiffiffi The error follows the inverse square root rule (1= Ns ) in the Monte-Carlo, two-color, or splitand-delay approaches. However, the error falls more rapidly with number of diffraction patterns for the two-color or split-and-delay method than for the Monte-Carlo approach. To identify a 1% change in a structure factor in pump-probe experiments, less than 100 pairs of patterns with the corresponding Bragg order indexed are needed for the two-color or split-and-delay approach, whereas 80 000 patterns are required in the conventional Monte-Carlo approach. This improvement in error reduction and data efficiency is a direct result of the elimination of the stochastic factors, such as random orientation and varying size and shape of the crystals. Two-color or split-and-delay experiments have the advantage of sensitivity to change in structure factors over the other approaches, rather than any superior accuracy of direct structurefactor measurement. At the safe dose which minimizes damage, fixed-sample experiments give an error independent of sampling procedure, but limited by the X-ray dose the sample can tolerate. Complete data sets must be obtained to solve the time-resolved structure. The number of patterns required for this purpose is definitely much more than the number of patterns needed to achieve a certain accuracy in a single structure factor since we need sufficient patterns that cover the whole reciprocal space to produce the electron density maps. We assume that the number of patterns needed to form a complete data in two-color approach is about the same as that needed in liquid jet sample delivery, based on the Monte-Carlo approach, since the 041714-12 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) FIG. 1. Relative error in structure factor magnitude measured in Monte-Carlo (MC, middle curve), split-and-delay or twocolor (SD, 2C, lower curve), and non-destructive mode of goniometer-based fixed sample (FS, upper curve) approaches for TR-SFX. To identify 1% change in structure factor in pump-probe experiments, less than 100 pairs of patterns are needed in two-color or split-and-delay mode, compared to approximately 80 000 patterns required in the Monte-Carlo approach. The non-destructive mode of goniometer-based fixed sample approach gives an error limited by the X-ray dose, but independent of sampling. The diffract-and-destroy mode, using fixed samples, yields an error with a prefactor of 0.57%, but the number of patterns collected from one single crystal is limited by the crystal size and the distance between consecutive shots in order to avoid radiation damage. Diffraction from micro-crystals trapped on a calibrated lattice follows essentially the same error reduction behavior as the Monte-Carlo approach using the liquid jet delivery system. statistics of the crystal orientation distribution is the same for both methods. Also, in case that the crystal is much larger than the typical beam size of 4 lm, we can expand the beam to match the size of the crystal by defocussing, to maximize the total signal hence to reduce the error in structure factor measurement. An increase in the beam size by the factor of 25 (100 lm) reduces the error in fixed sample experiments to approximately 1%, which is comparable to the other approaches. But certainly, larger crystals not only favor the fixed sample experiments but are also preferred in all modes, since they yield stronger diffraction signals and so higher resolution data unless this is limited by crystal quality. IV. DISCUSSION AND CONCLUSION The Monte-Carlo approach has been widely adopted for (SFX in recent years. Using tens of thousands of patterns, merged partial intensities converge accurately to yield the structure factors, allowing structures to be solved at better than 0.2 nm resolution which might not otherwise have been solved due to small crystal size or radiation sensitivity.24 The low data efficiency mainly results from uncontrollable stochastic variables contributing to the error in structure factors. These contributions add in quadrature, and the large intensity variation of the same Bragg reflection on different shots (covering several orders of magnitude) due to partiality (i.e., different deviations from the exact Bragg condition) dominates the error in the MonteCarlo approach. To improve on the traditional Monte-Carlo integration and merging procedure, new methods of treating the partial intensities, intensity integration, scaling, and post-refinement have been proposed and studied. By modeling the angular profile of the Bragg spots from mosaic crystals,2 an integration mask can be customized for each reflection. Using a geometrical model for partiality, the diffraction conditions for each pattern can be refined to estimate the partiality, so that full reflection intensities can be predicted, and this refinement procedure repeated 041714-13 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) iteratively to obtain the best estimate.14 Common reflections on different shots also assist scaling, using post-refinement and the Ewald offset correction, which assumes a Gaussian rocking curve for a sufficiently large crystal.15 Our two-color method complements these algorithmic approaches for improved accuracy, going beyond the Monte-Carlo method, for time-resolved diffraction. The two-color approach eliminates variations in crystal size, shape, and orientation which dominate the Monte-Carlo approach, by probing the same crystal twice in the same orientation with two pulses of different energies, separated in time. The ratio of partial intensities of Bragg spots with identical Miller indices from two pulses is recorded for each pattern and then summed with a weighting to obtain the percentage change in structure factor. The accuracy in structure factor change is determined by the total signal summed over all patterns. Therefore, in spite of the low dose limit for the first pulse (which must not destroy the crystal), the accuracy improves with the number of patterns collected. For the TbCatB crystals used recently,13 at the Henderson safe dose limit of 1 MGy at room temperature, less than 100 patterns are needed to achieve 1% accuracy, compared to 80 000 patterns for the Monte-Carlo approach (Fig. 1). From the point of view of error reduction and data efficiency, the two-color approach appears to be a better choice for pump-probe time-resolved experiments, provided that a sufficiently long delay between X-ray pulses can be obtained for the process of interest. A difference Fourier charge-density map is normally applied to study structural changes. The difference map shows changes in the electron density much more sensitively than a normal Fourier Map.27 With unknown phases, the peak height in a map is half that with phase information17 if the conditions DjFj/jFj  1 and r(DjFj)/DjFj  1 are satisfied. For most pump-probe experiments, these conditions are satisfied, making the difference Fourier map applicable to two-color data. However, our two-color approach and error analysis are based on several essential assumptions which must be considered here. First, the time interval between the two pulses must be much shorter than the rotational diffusion time of the crystal in solution (typically milliseconds for a 1–lm crystallite in buffer) so that it can be treated as stationary. Second, the difference in wavelength between the two pulses must be sufficient to separate the two diffraction patterns in the same readout, but not too large so that the corresponding Ewald spheres are far from each other intersecting different Bragg reflections. Third, the crystal size must neither be too small such that the broad shape transform will not allow us to separate the two patterns nor too big to invalidate our shape transform analysis. (Our error analysis assumes that the two patterns are taken from almost the same point on the rocking curve.) To investigate these assumptions for future two-color experiments, diffraction patterns from I3C (“magic triangle”)26 micron-sized inorganic crystals were simulated for X-ray pulses at energies of 6.6 keV and 6.685 keV, as shown in Fig. 2. Using the CSPAD (Cornell-SLAC hybrid pixel array detector) detector at LCLS with the minimum working distance of 5 mm, the 85 eV (1.3%) energy difference shifts the Bragg spots by approximately 20 pixels at the 2 Å resolution ring, which corresponds to the side edge of the detector. Since the relative displacement between the Bragg spots of the same Miller index increases with resolution, the Bragg reflections at low resolution can be separated by using a larger working distance or an additional back detector, illuminated by a central hole in the front detector. Over the past year, there have been dramatic advances in the ability to model partiality for SFX data from several groups, using iterative optimization algorithms and a suitable model for mosaicity.2,15,16,24 If we use these methods to model the partiality for each wavelength separately on the same detector readout, the resulting more realistic results will fall somewhere between the Monte-Carlo error curve and two-color error curve (Figure 1), since curve “2C” assumes no difference in partialities of the two wavelengths. Goniometer-based fixed-sample experiments provide accurate control of the crystal orientation which our SFX experiments are not capable of. In destructive mode, each X-ray shot drills a hole in the crystal, which must be translated to a fresh point for the next shot. Beam intensity is maximized to obtain the highest SNR and the error decreases as the inverse square root of the number of patterns, which is similar to the Monte-Carlo approach, except that the prefactor is much smaller due to the accurate control of crystal orientation. With the beam attenuated or 041714-14 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) FIG. 2. Simulated diffraction pattern ((100) plane) from I3C (“magic triangle”) crystals (orthorhombic. Pbca, a ¼ 9.214 Å, b ¼ 15.735 Å, and c ¼ 18.816 Å) using X-ray pulses at energies of 6.6 keV and 6.685 keV in two-color approach. Crystal size is 0.005 lm  1.3 lm  1.5 lm and identical intensity for all Bragg reflections is assumed just to show the Bragg spot positions. Red and blue colors indicate Bragg spots from 6.6 keV and 6.685 keV, respectively. Bragg spots of same index from two colors are clearly separated by detectable displacements. The displacement is approximately 20 pixels at 2 Å resolution ring on CSPAD detector at LCLS with the minimum working distance of 5 mm. defocussed to a level below the damage threshold, goniometer-based experiments allow us to probe the same region of a sample in different orientations from which local information on structures or dynamics can be extracted. The low dose limit gives poorer statistics, and the error in the measured structure factor is found to be independent of the number of sampling points (or patterns from the same region) and is only determined by the total dose deposited into the probed region of the crystal. ACKNOWLEDGMENTS The authors thank Dr. Nadia Zatsepin, Rick Kirian, and Yun Zhao for helpful discussions and assistance in diffraction pattern simulations. Supported by NSF STC award 1231306. APPENDIX A: STATISTICS OF TbCatB CRYSTAL SHAPE TRANSFORM CALCULATED BY MONTE-CARLO SIMULATION To characterize the source of errors in XFEL experiments, Monte-Carlo simulations were conducted to estimate the dominant contribution from the large intensity fluctuation across the shape transform based on its statistics. Shape transforms were modeled using Eq. (4) for TbCatB crystals13 of 0.9  0.9  11 lm average size with 10% Gaussian-distributed deviation. Statistics of intensity variation across the shape transform depends on the integration radius dt. Therefore, mean value, standard deviation, and their ratio (relative deviation) were calculated as functions of dt as a fraction of the scattering vector (Fig. 3). dt ranges from 0 to 0.1 with an increment of 0.01, and for each value of dt, 106 sampling points on the shape transform were randomly generated for a uniform distribution. At dt ¼ 0.01, which matches the average size of the crystal, the mean value and the relative deviation of the shape transform for TbcatB crystals were found to be 1.76  1012 and 5.7, respectively. 041714-15 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) FIG. 3. Statistics (mean, standard deviation, and relative deviation/ratio) of TbcatB crystal shape transform obtained from Monte-Carlo simulation. 10% standard deviation in crystal size was assumed based on experimental data.12 The abscissa dt is the integration radius around Bragg peaks; left vertical axis shows the mean and standard deviation values; right vertical axis shows the relative deviation/ratio. APPENDIX B: ERROR ANALYSIS FOR TWO-COLOR APPROACH According to error propagation theory,12 errors in different variables are related as follows: qffiffiffiffiffiffiffiffiffiffi 0ðiÞ ðiÞ rðRðhklÞ Þ ¼ rð AðhklÞ Þ; (B1) qffiffiffiffiffiffiffiffiffiffi   ðÞ ðiÞ r AðihklÞ 1 r AðhklÞ ; DqffiffiffiffiffiffiffiffiffiffiE ¼ ðÞ 2 hAðiÞ i AðihklÞ ðhklÞ (B2) ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi v0 u  12 0  1 2 u ðiÞ ðÞ  2 u r IðiÞð Þ r AðihklÞ rðk12 Þ Br I2;ðhklÞ C uB 1; hkl C @ A @ A þ þ ¼ ; t ðÞ ðÞ ðÞ k12 hAðihklÞ i I1;i ðhklÞ I2;i ðhklÞ     qffiffiffiffiffiffiffiffiffiffiffiffi ðÞ ðÞ r I1;i ðhklÞ ¼ I1;i ðhklÞ ; r  0ðiÞ RðhklÞ    qffiffiffiffiffiffiffiffiffiffiffiffi ðÞ ðÞ r I2;i ðhklÞ ¼ I2;i ðhklÞ ; a ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffi v AðhklÞ i u 1 u 1  t ðiÞ þ ðiÞ þ a2 : ¼ 2 I1;ðhklÞ I2;ðhklÞ h ðNÞ rðk12 Þ ; k12 (B3) (B4) (B5) In order to analyze the error in R0 ðhklÞ , we express it explicitly in terms of the experimentally measured parameters as below vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi uX u N ðiÞ ðÞ u I2;ðhklÞ  hk12i i u u ð Þ N R0 ðhklÞ ¼ u i¼1 N  1: (B6) u X ðiÞ t I 1;ðhklÞ i¼1 041714-16 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) ðNÞ To show that R0 ðhklÞ is indeed a valid estimate of RðhklÞ which is the true value of the relative change in structure factor magnitude jFðhklÞ j, we show that the average value (expectation) of ðNÞ R0 ðhklÞ approaches the true value RðhklÞ when the number of shots N is sufficiently large. The averðNÞ age value of R0 ðhklÞ is ðNÞ hR0 ðhklÞ ijintensity vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi uX u N ðiÞ ðÞ u hI2;ðhklÞ ijintensity  hk12i i u u ¼ u i¼1 N  1: u X ðiÞ intensity t hI ij (B7) 1;ðhklÞ i¼1 ðiÞ ðiÞ We define D(i) as the discrepancy between k12 and its expectation value hk12 i ðiÞ DðiÞ  k12  hk12 i: (B8) Then, Eq. (B7) can be rewritten as ðNÞ hR0 ðhklÞ ijintensity vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi uX N X u N ðiÞ ðÞ ðÞ u hI2;ðhklÞ ijintensity  k12i hI2;i ðhklÞ ijintensity  DðiÞ u u ¼ u i¼1 N  i¼1 N  1: u X X ð iÞ ðiÞ intensity intensity t hI1;ðhklÞ ij hI1;ðhklÞ ij i¼1 (B9) i¼1 If the number of shots N goes to infinity, or more practically, we have a sufficiently large number of shots from which diffraction patterns are collected; the second term under the square root sign approaches 0 0 1 N X ðÞ B hI2;i ðhklÞ ijintensity  DðiÞ C C B C B ðÞ (B10) lim B i¼1 N C ¼ hD i i ¼ 0: N!1B X ðiÞ C intensity @ A hI ij 1;ðhklÞ i¼1 ðNÞ Hence, the average of R0 ðhklÞ approaches the true value RðhklÞ ðNÞ lim R0 ðhklÞ ¼ RðhklÞ : (B11) N!1 ðNÞ Therefore, R0 ðhklÞ is a good estimate of the relative change in structure factor magnitude. We now omit the Bragg order index (hkl) from subscripts and (N) from superscripts. Additionally, we define some auxiliary variables for notational convenience as follows: pffiffiffi (B12) R0 ¼ A  1; N  P A ¼ i¼1 ðÞ I2i hk12 i N P i¼1 B¼ N X BðiÞ ;   ðÞ I1i ðiÞ BðiÞ ¼ ðI2 hk12 iÞ ; i¼1 B ; D D¼ (B13) N X ðiÞ I1 : (B14) i¼1 According to the theory of errors, the errors in the different variables are related as follows: pffiffiffi rðR0 Þ ¼ rð AÞ; (B15) 041714-17 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) r2 ð AÞ hAi2 ¼ r2 ð BÞ þ hBi2 r2 ð DÞ hDi2 ; (B16) ðiÞ BðiÞ ¼ ðI2  hk12 iÞ; r2 ðBðiÞ Þ hBðiÞ i2 ¼  1 ðÞ hI2i i þ r2 ð BÞ ¼ 2 rðk12 Þ hk12 i N X (B17) 1 ¼ ¼ 1 ðÞ hI2i i2 hk12 i2 ðÞ hI2i i i¼1 r2 ðDÞ ¼ N X (B18) !! (B19) r2 ðBðiÞ Þ; i¼1 N X þ a2 ; ðÞ hI2i i ðiÞ r2 ðI1 Þ ¼ þa 2 ; N X ðiÞ hI1 i: i¼1 (B20) i¼1 Combining Eqs. (B15)–(B20), we obtain the error in A, hence R0 N X r2 ð AÞ hAi2 ¼ ðÞ hI2i i2 hk12 i2 i¼1 * N X ðÞ I2i !! 1 þa ðÞ hI2i i +2 hk12 i N X ðÞ hI1i i 2 þ *i¼1 N X 2 i¼1 +2 ; ðÞ I1i i¼1 N X ðÞ hI2i i2 1 1 þ N þ a2  * i¼1 ¼ N +2 ; X ð iÞ X ð iÞ N X ð Þ i hI2 i hI1 i hI i i¼1 2 i¼1 i¼1 N  X  1 1 þ N þ a2  N X X ðÞ ðÞ I2i I1i i¼1 ðÞ I2i i¼1 N X ðÞ I2i i¼1 2 !2 ; i¼1  T1 þ T2 þ T3; rðR0 Þ ¼ r (B21) pffiffiffi 1 pffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi hAi  T1 þ T2 þ T3: A ¼ 2 (B22) In case of a large value of N, the sampling can cover the whole intensity distribution of Bragg reflections ergodically with much less fluctuation than for small values of N. Instead of using Eq. ðiÞ ðiÞ (B21), we estimate the error in R0 using the expectation value of the refection intensity I1 and I2 over the entire intensity distribution lim T1 ¼ N!1 lim T2 ¼ N!1 1 1 ; ð Þ i N hI ijintensity (B23) 1 1 ; N hIðiÞ ijintensity (B24) 2 1 shots shots 041714-18 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) N X   ðiÞ 2 intensity N hI i jshots 2 lim *i¼1 +2 ¼  2 ; N!1 X N ðiÞ intensity ðiÞ NhI2 ijshots I2 ðÞ hI2i i2 (B25) i¼1 N  ¼ 2 ðÞ ðÞ hI2i ijintensity þ r2 ðI2i Þ shots ;  2 ðÞ N 2 hI2i ijintensity shots 0   12 1 ðÞ r I2i 1B @ A C @ A; 1þ ¼ ðÞ N hI2i ijintensity shots 0 ¼ b  ðÞ r I2i ðÞ N!1  1 1 þ b2 ; N  hI2i ijintensity shots lim T3 ¼ (B26) ;  1 1 þ b2 a2 : N (B27) (B28) (B29) (B30) Therefore, the error in R0 can be estimated as below 1 pffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi hAi  T1 þ T2 þ T3; 2 sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi   1 pffiffiffiffiffiffiffi 1 1 1 hAi  N!1 þ ðiÞ intensity þ 1 þ b2 a2  pffiffiffiffi ; ð Þ intensity i 2 N hI2 ijshots hI1 ijshots rðR0 Þ ¼ ðiÞ (B31) (B32) ðiÞ and hI2 ijintensity are the expectation values over the rocking curve of the reflecwhere hI1 ijintensity shots shots tion intensities from the first and second pulses, respectively. b is the relative standard deviation ðiÞ in I2 over the rocking curve and a denotes the relative error in k12 . The ratio of the intensities of the two pulses k12 varies from shot to shot, and this amplitude of fluctuation is characterized by a and determined by the stability of the emittance spoiler in the delay line. 1 J. C. H. Spence, U. Weierstall, and H. N. Chapman, “X-ray lasers for structural and dynamic biology,” Rep. Prog. Phys. 75, 102601 (2012). J. Hattne, N. Echols, R. Tran, J. Kern, R. J. Gildea, A. S. Brewster, R. Alonso-Mori, C. Gl€ ockner, J. Hellmich, H. Laksmono et al., “Accurate macromolecular structures using minimal measurements from X-ray free-electron lasers,” Nat. Methods 11, 545–548 (2014). 3 A. Aquila, M. S. Hunter, R. B. Doak, R. A. Kirian, P. Fromme, T. A. White, J. Andreasson, D. Arnlund, S. Bajt, R. M. Thomas et al., “Time-resolved protein nanocrystallography using an X-ray free-electron laser,” Opt. Express 20, 2706–2716 (2012). 4 C. Kupitz, S. Basu, I. Grotjohann, R. Fromme, N. A. Zatsepin, K. N. Rendek, M. S. Hunter, R. L. Shoeman, T. A. White, D. Wang et al., “Serial time-resolved crystallography of photosystem II using a femtosecond X-ray laser,” Nature 513, 261 (2014). 5 R. A. Kirian, T. A. White, J. M. Holton, H. N. Chapman, P. Fromme, A. Barty, L. Lomb, A. Aquila, F. R. N. C. Maia, A. V. Martin, R. Fromme et al., “Structure-factor analysis of femtosecond microdiffraction patterns from protein nanocrystals,” Acta Crystallogr., Sect. A: Found. Crystallogr. 67, 131–140 (2011). 6 K. Ayyer, H. T. Philipp, M. W. Tate, V. Elser, S. M. Gruner et al., “Real-space x-ray tomographic reconstruction of randomly oriented objects with sparse data frames,” Opt. Express 22, 2403–2413 (2014). 7 M. Frank, D. B. Carlson, M. S. Hunter, G. J. Williams, M. Messerschmidt, N. A. Zatsepin, A. Barty, W. H. Benner, K. Chu, A. T. Graf et al., “Femtosecond X-ray diffraction from two-dimensional protein crystals,” IUCrJ 1, 95–100 (2014). 2 041714-19 8 Li, Schmidt, and Spence Struct. Dyn. 2, 041714 (2015) H. P. Stevenson, D. P. Deponte, A. M. Makhov, J. F. Conway, O. B. Zeldin, S. Boutet, G. Calero et al., “Transmission electron microscopy as a tool for nanocrystal characterization pre- and post-injector,” Philos. Trans. R. Soc., B 369, 20130322 (2014). 9 A. A. Lutman, R. Coffee, Y. Ding, Z. Huang, J. Krzywinski, T. Maxwell, M. Messerschmidt, H. D. Nuhn et al., “Experimental demonstration of femtosecond two-color X-ray free-electron lasers,” Phys. Rev. Lett. 110, 134801 (2013). 10 J. C. H. Spence, “Approaches to time-resolved diffraction using an XFEL,” Faraday Discuss. 171, 429 (2014). 11 R. A. Kirian, X. Wang, U. Weierstall, K. E. Schmidt, J. C. H. Spence, M. Hunter, P. Fromme, T. White, H. N. Chapman, J. Holton et al., “Femtosecond protein nanocrystallography - data analysis methods,” Opt. Express 18, 5713–5723 (2010). 12 P. R. Bevington and D. K. Robinson, Data Reduction and Error Analysis for the Physical Sciences, 2nd ed. (McGrawHill, Inc., 1992), pp. 50 and 71–72. 13 L. Redecke, K. Nass, D. P. DePonte, T. A. White, D. Rehders, A. Barty, F. Stellato, M. Liang, T. R. M. Barends, S. Boutet et al., “Natively inhibited Trypanosoma brucei cathepsin B structure determined by using an X-ray laser,” Science 339, 227–230 (2013). 14 W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes in C (Cambridge University Press, 1988), pp. 114–119. 15 T. White, “Post-refinement method for snapshot serial crystallography post-refinement method for snapshot serial crystallography,” Philos. Trans. R. Soc., B 369, 20130330 (2014). 16 W. Kabsch, “Processing of X-ray snapshots from crystals in random orientations,” Acta Crystallogr., Sect. D: Biol. Crystallogr. 70, 2204–2216 (2014). 17 R. Henderson and J. K. Moffat, “The difference Fourier technique in protein crystallography: Errors and their treatment,” Acta Crystallogr., Sect. B: Struct. Crystallogr. Cryst. Chem. 27, 1414–1420 (1971). 18 J. C. H. Spence, N. A. Zatsepin, and C. Li, “Coherent convergent-beam time-resolved X-ray diffraction coherent convergent-beam time-resolved X-ray diffraction,” Philos. Trans. R. Soc., B 369, 20130325 (2014). 19 W. H. Zachariasen, Theory of X-ray Diffraction in Crystals (Dover, 1967, Toronto). 20 J. Tenboer, S. Basu, N. Zatsepin, K. Pande, D. Milathianaki, M. Frank, M. Hunter, S. Boutet, G. J. Williams, J. E. Koglin et al., “Time-resolved serial crystallography captures high-resolution intermediates of photoactive yellow protein,” Science 346, 1242–1246 (2014). 21 K. Moffat, “Time-resolved biochemical crystallography: A mechanistic perspective,” Chem. Rev. 101, 1569–1582 (2001). 22 E. Snell, H. Bellamy, and G. Borgstahl, “Macromolecular crystal quality,” Methods Enzymol. 368, 268 (2003). 23 J. C. H. Spence, R. A. Kirian, X. Wang, U. Weierstall, K. E. Schmidt, T. White, A. Barty, H. N. Chapman, S. Marchesini, J. Holton et al., “Phasing of coherent femtosecond X-ray diffraction from size-varying nanocrystals,” Opt. Express 19, 2866–2873 (2011). 24 H. M. Ginn, M. Messerschmidt, X. Ji, H. Zhang, D. Axford, R. J. Gildea, G. Winter, A. S. Brewster, J. Hattne, A. Wagner et al., “Structure of CPV17 polyhedrin determined by the improved analysis of serial femtosecond crystallographic data,” Nat. Commun. 6, 6435 (2015). 25 A. Marinelli, D. Ratner, A. A. Lutman, J. Turner, J. Welch, F. Decker, H. Loos, C. Behrens, S. Gilevich, A. A. Miahnahri et al., “High-intensity double-pulse X-ray free-electron laser,” Nat. Commun. 6, 6369 (2015). 26 D. Zhu, M. Cammarata, J. M. Feldkamp, D. M. Fritz, J. B. Hastings, S. Lee, H. T. Lemke, A. Robert, J. L. Turner, Y. Feng et al., “A single-shot transmissive spectrometer for hard x-ray free electron lasers,” Appl. Phys. Lett. 101, 034103 (2012). 27 T. Beck and G. M. Sheldrick, “5-amino-2,4,6-triiodo-isophthalic acid monohydrate,” Acta Crystallogr., Sect. E: Struct. Rep. Online E64, 1286 (2008). 28 R. Henderson, “The potential and limitations of neutrons, electrons and X-rays for atomic resolution microscopy of unstained biological molecules,” Q. Rev. Biophys. 28, 171–193 (1995).