Quick Summary on Using X-GEN

X-GEN provides for data processing, beginning at raw images and ending at the creation and analysis of an ASCII file containing h,k,l,F,sigma, and some anomalous information. In between it enables

X-GEN can be run in at least four ways:

Another preconstructed script for the most straightforward case (processing a single data run's worth of images) is provided here.

How to use X-GEN

The steps you perform within the "xgen" executable are:

  1. setup, pdisplay, or xgnow. Any of these commands sets up environment variables and an index of the images you will be looking at. Setup is a Curses-based program for this purpose. pdisplay is a Motif-based GUI that is built around a two-dimensional display of detector images as well as a pushbutton GUI for navigating through the X-GEN commands. xgnow simply creates the environment variables and the index of images and writes those to the appropriate working directory, where they can be activated by source-ing the command file (*.com or *.cmd, depending on the user's shell). In the cases of setup and pdisplay, there are screens on which you fill in a few basic parameters, and you push some buttons to set up the run. In the current version of X-GEN it is generally unnecessary to explicitly map pixels to centimeters and back again using the calibrate application; see the special comments on calibrate, below.
  2. source *.com. This takes the environment variables you put into those command procedures and activates them. If you're running sh, ksh, or bash, you should source the file with the name that ends in .com; if you're running tcsh or csh, you should source the file with the name that ends in .cmd. Of course, if there is more than one file with the extension you've used, then you'll have to source the correct one.
  3. spots finds the reference reflections for indexing and refinement, creates starting 3-D profiles for profile fitting, and it determines the boundaries of the active area of the detector. Typing spots without arguments which will cause the application to search an appropriate number of images to find bright spots -- six images if you're using a CCD or image-plate detector, more if you're using a Xentronics or similar detector. If that doesn't give you enough spots (200-500 is typically plenty) then you can specify a larger number of images, e.g.
      spots 12 30

    will tell it to look for spots in images twelve through thirty.
  4. refall sets up indexing and refinement. This replaces the interactive programs "refine" and "refine1", which were almost the only detailedly interactive programs in the package. You can run "refall" without arguments in some instances; see the documentation for refall to see some exceptions. In particular, "refall -as" will go through the indexing step up through the determination of the Bravais lattice and then ask the user to confirm whether or not the program's idea of the most probable Bravais lattice is actually correct; once the user replies to the Bravais question, it then prompts for a spacegroup number from the International Tables. Then it goes on to complete the refinement.
  5. integrate -j6000. This integrates the data as described above; it also updates the cell and orientation as it goes along, and it updates the 3-D profiles. In this instance you're looking at the first few images in order to get better parameters and profiles. In this instance we have asked the system to integrate the entire set of images, but to perform a detector- coordinate remapping during the first on-the-fly refinement if the RMS (X,Y) residual is worse than 0.6 pixels (= 6000/10000).
  6. reduce. This does a preliminary filtering and reformatting of the integrated data.
  7. mrmerge run1.mrf run2.mrf. This merges multiple runs together. Obviously this is only necessary if there is more than one run to manage.
  8. scalem. This performs a multi-pass post-processing of the intensity data. It scales the data using a simple algorithm featuring one scaling parameter per omega range. It then flags outliers in the data, scales the data again using a spherical-harmonic model, redoes the outlier rejection step from scratch, and then performs an additional scaling using spherical harmonics and (if necessary) a nonlinear scaling model. If the redundancy in the data is high enough it will perform a final scaling step with the Friedel-mates separated. In any event it then calculates statistical indicia characterizing the quality of the data, rechecks the spacegroup, and outputs the merged, scaled data in six ascii formats.

Special comments on pixel-to-centimeter calibration:

With CCD or image-plate detectors the mapping from pixels to centimeters and back again has usually been done for you within the initial setup of your data run. Therefore you generally don't, as a user, need to run a calibration program. Instead, the system copies over a pixel-to-centimeter mapping from a common resources directory (typically /opt/xgen/res or /usr/share/xgen) into the current directory as the working .uca (USPLINE) file. When you first set up your run with setup, pdisplay, or xgnow, this copying operation should have been performed automatically for you. If the automated system guessed wrong about what kind of detector you have (e.g., it thought you have a Mar 165 when in fact you have a Mar 225), then you may have to do the copying manually, e.g.

  cp ~ahoward/xgensdk/res/q2104k.uca ./mycrystal.uca

That copying process will be done for you when you run pdisplay, setup, or xgnow, and you will only need to modify that if either the automatic process misunderstands what kind of detector you're using (so that you would need to copy the right file yourself), or if the beam-center values in the standard pixel-to-centimeter mapping file is incorrect. In this latter case you'll need to run the calibrate application itself.

Other commands:

To obtain specific details of the operation of the primary commands listed above, examine the documents describing the specific applications. For other X-GEN utilities, examine the documents describing them:

anal_u border cenmer cutoff editbord editcen
ewmask filterc filteru makemu mermask mukb
newr pickuref recycle refine refine1 reject
scalek spaceg stats strategy xdump


Each application within X-GEN reads and writes various files. Apart from dump and log files, which are appended to whenever accessed, most files are opened in a protected mode reminiscent of the way VAX/VMS opens files. Thus if a file $WORK/george.ams already exists, and an application needs to open a new copy of it, then the old version will be renamed to $WORK/george.ams,0 and a new $WORK/george.ams will be output. If $WORK/george.ams,0 already exists, then the previous $WORK/george.ams will be written to $WORK/george.ams,1 instead. This sequence continues up through $WORK/george.ams,99. If a user wishes to produce more than 100 versions of a file, he or she will have to save the oldest ones.

Documentation is available for some of the file formats used within or output from X-GEN. The formats include:

Environ. Typical Binary Role Description
Variable Name or Ascii   of format
AMASK $WORK/george.ams Binary Internal Detector Active Area
CENTROIDS $WORK/george.cen Binary Internal 3-D Reflection Centroids
MULIST $WORK/george.mu Ascii Output Ascii (h,k,l,F,σ,F+, . . .)
MULTIREF $WORK/george.mrf Binary Internal Integrated, merged refls
SCALEIN $WORK/george.si Ascii Internal Scaling functions
for batches of data
UDUMP $WORK/george.udu Ascii Appended Dump of X-GEN binaries
UPARAMS $WORK/george.upr Binary Internal Crystal & Detector Params
UREFLS $WORK/george.urf Binary Internal Integrated intensities
USPLINE $WORK/george.uca Binary Internal Pixel-to-cm conversions
WMASK $WORK/george.wms Binary Internal 3-D Reflection Profiles
XLOG $WORK/george.xls Ascii Appended Logfile
XASCA $WORK/george.asc Ascii Output SCALEPACK output;
Friedel-mates separated
XCNS $WORK/george.xcn Ascii Output CNS reflection format
XSHEL $WORK/george.hkl Ascii Output SHELX/XPREP format
XXPLOR $WORK/george.xxp Ascii Output X-PLOR V.3.1 format
XSCA $WORK/george.sca Ascii Output SCALEPACK output;
Friedel-mates merged

Report bugs to Andy Howard at howard@iit.edu or 312-567-5881.