(Demeter main page) (Cookbook main page)

Data Handling

This section covers the parts of Demeter that implement the functionality of Athena.

Contents

Data Handling

How do I import a data file and make a plot?

In this recipe, we import one of the iron foil examples that comes in the Demeter package. This file has already been converted from raw data into mu(E) data. Demeter will recognize that it is mu(E) data and process it accordingly.

   1 #!/usr/bin/perl   ### import and plot
   2 ## -----------------------------------------------------------
   3 ## boilerplate required in every script:
   4 use warnings;
   5 use strict;
   6 use Ifeffit::Demeter;
   7 my $plot = Ifeffit::Demeter::Plot -> new();
   8 ## -----------------------------------------------------------
   9 my $data = Ifeffit::Demeter::Data -> new({group => 'data0'});
  10 $data -> set({file        => "data/fe.060.xmu",
  11               label       => 'Fe 60K',
  12               bkg_rbkg    => 1.5,
  13               bkg_spl1    => 0,    bkg_spl2    => 18,
  14               bkg_nor1    => 100,  bkg_nor2    => 1800,
  15               fft_kmax    => 3,    fft_kmin    => 17,
  16              });
  17 $data -> plot('E');
  18 
  19 sleep 2;
  20 $data -> po -> start_plot;
  21 $data -> plot('k');
  22 
  23 sleep 2;
  24 $data -> po -> start_plot;
  25 $data -> plot('R');

After reading the data and setting several of the data processing parameters, the data are plotted in energy, then as chi(k), then as chi(R), with a two second pause in between plots. The calls to start_plot at lines 20 and 24 reinitialize the plotting system so that the subsequent plot is a new plot rather than an overplot.

Note that you do not need to explicitly do a background removal or a Fourier transform. Demeter keeps track of what needs to be done in order to make the requested plot.

How do I import raw data from the beamline?

To convert raw data into mu(E) data you need to know which columns contain which scalars from the measurement. In this example of transmission data, column 1 is the energy, column 2 is I0, and column 3 is transmission.

   1 #!/usr/bin/perl    ### import raw data
   2 ## -----------------------------------------------------------
   3 ## boilerplate required in every script:
   4 use warnings;
   5 use strict;
   6 use Ifeffit::Demeter;
   7 my $plot = Ifeffit::Demeter::Plot -> new();
   8 ## -----------------------------------------------------------
   9 my $data = Ifeffit::Demeter::Data -> new({group => 'scan1'});
  10 $data -> set({energy      => '$1', # column 1 is energy
  11               numerator   => '$2', # column 2 is I0
  12               denominator => '$3', # column 3 is It
  13               ln          => 1,    # these are transmission data
  14               bkg_pre1    => -30,  bkg_pre2    => -150,
  15               bkg_nor1    => 150,  bkg_nor2    => 1757.5,
  16               bkg_spl1    => 0.5,  bkg_spl2    => 22,
  17               fft_kmax    => 3,    fft_kmin    => 14,
  18              });
  19 $data -> set({file => "data/fe.060", label => 'Iron foil',});
  20 $data -> plot('E');

The energy, numerator, and denominator must be explicitly specified, as is a flag for whether the natural log should be taken. The "language" of the column selection was chosen to resemble gnuplot's syntax.

Fluorescence data would have the ln flag set to zero and the I0 column would be in the denominator. For multi-element data, you might have a sum of columns for the numerator -- something like '$5+$6+$7+$8'.

How do I plot multiple data sets?

Answered in the simplest possible terms, define more than one Data object and plot them all. Here is the simplest approach:

   1 #!/usr/bin/perl     ### plot more than one data set
   2 ## -----------------------------------------------------------
   3 ## boilerplate required in every script:
   4 use warnings;
   5 use strict;
   6 use Ifeffit::Demeter;
   7 my $plot = Ifeffit::Demeter::Plot -> new();
   8 ## -----------------------------------------------------------
   9 my $d1 = Ifeffit::Demeter::Data -> new({group => 'data0'});
  10 $d1 -> set({file        => "data/fe.060.xmu",
  11             label       => 'Fe 60K',
  12             bkg_rbkg    => 1.5,
  13             bkg_spl1    => 0,    bkg_spl2    => 18,
  14             bkg_nor1    => 100,  bkg_nor2    => 1800,
  15             fft_kmax    => 3,    fft_kmin    => 17,
  16            });
  17 $d1 -> plot('E');
  18 my $d2 = Ifeffit::Demeter::Data -> new({group => 'data0'});
  19 $d2 -> set({file        => "data/fe.300.xmu",
  20             label       => 'Fe 300K',
  21             bkg_rbkg    => 1.5,
  22             bkg_spl1    => 0,    bkg_spl2    => 18,
  23             bkg_nor1    => 100,  bkg_nor2    => 1800,
  24             fft_kmax    => 3,    fft_kmin    => 17,
  25              });
  26 $d2 -> plot('E');

This has a big problem -- it is very repetitious. Repetition is tedious and error-prone. There are two solutions to this problem:

Use hash references

The first is to recognize that the argument of the set method is an anonymous hash. That means we can use a hash reference to specify the parameters the two data sets have in common:

   1 #!/usr/bin/perl          ### defining attributes with an anonymous hash
   2 ## -----------------------------------------------------------
   3 ## boilerplate required in every script:
   4 use warnings;
   5 use strict;
   6 use Ifeffit::Demeter;
   7 my $plot = Ifeffit::Demeter::Plot -> new();
   8 ## -----------------------------------------------------------
   9 my %common_attributes = (bkg_rbkg    => 1.5,
  10                          bkg_spl1    => 0,    bkg_spl2    => 18,
  11                          bkg_nor1    => 100,  bkg_nor2    => 1800,
  12                          fft_kmax    => 3,    fft_kmin    => 17,
  13                         );
  14 
  15 my $d1 = Ifeffit::Demeter::Data -> new({group => 'data0'});
  16 $d1 -> set({file => "data/fe.060.xmu", label => 'Fe 60K'});
  17 $d1 -> set(\%common_attributes);
  18 
  19 my $d2 = Ifeffit::Demeter::Data -> new({group => 'data0'});
  20 $d2 -> set({file => "data/fe.300.xmu", label => 'Fe 300K'});
  21 $d2 -> set(\%common_attributes);
  22 
  23 foreach my $d ($d1, $d2) {
  24   $d -> plot('E');
  25 };

Clone objects

The other shortcut is to use the clone method:

   1 #!/usr/bin/perl     ### cloning data groups
   2 ## -----------------------------------------------------------
   3 ## boilerplate required in every script:
   4 use warnings;
   5 use strict;
   6 use Ifeffit::Demeter;
   7 my $plot = Ifeffit::Demeter::Plot -> new();
   8 ## -----------------------------------------------------------
   9 my $d1 = Ifeffit::Demeter::Data -> new({group => 'data0'});
  10 $d1 -> set({file        => "data/fe.060.xmu",
  11             label       => 'Fe 60K',
  12             bkg_rbkg    => 1.5,
  13             bkg_spl1    => 0,    bkg_spl2    => 18,
  14             bkg_nor1    => 100,  bkg_nor2    => 1800,
  15             fft_kmax    => 3,    fft_kmin    => 17,
  16            });
  17 my $d2 = $d1 -> clone({file => "data/fe.300.xmu", label => 'Fe 300K'});
  18 
  19 foreach my $d ($d1, $d2) {
  20   $d -> plot('E');
  21 };

The cloning at line 18 returns a replica of the original data, but with the specified attributes changed.

There is no advantage to one of these approaches over the other and they can be used interchangeably. Take your pick!

How do I align and merge data?

Once you have imported data and set the data processing parameters using one of the recipes above, you can align data sets. AS in Athena, merging data sets requires that you first align them.

Alignment

This script from the Demeter distribution demonstrates data alignment. The salient lines are
```
   1 $d0->align($d1);
   2 $d1->e0($d0);
```
These lines align data group $d1 to $d0 and sets the e0 value of $d1 to be the same as $d0.

Merging

This script from the Demeter distribution demonstrates data merging. The salient line is
```
   1 my $merge = $d0->merge("N", $d1, $d2);
```
which tells Demeter to merge Data objects $d1 and $d2 with $d0 after interpolating $d1 and $d2 onto the grid of $d0. The first argument to the merge method tells which form of the data to do the merge in. In this case "N" is for "normalized". The other options are "X" and "K" for merging raw mu(E) data or chi(k) data. Note that $merge is a Data object containing the merged data.

How do I <X> data? <X> = (calibrate|truncate|deglitch|smooth|add noise to|convolve)

Once you have imported data and set processing parameters using one of the recipes above, you can perform a number of data processing chores.

calibrate

Calibration is the process of setting a particular point in the data set to a particular value.
```
   1 $d0->calibrate($selected_energy, $edge_energy);
```
The two arguments are optional. If not provided, an energy very close to the first big peak in the first derivative of mu(E) will be used as the selected energy and the tabulated value will be used as the edge energy.

truncate

This simply removes data before or after a selected energy value:
```
   1 $d0 -> Truncate('before', 7100);
```
or
```
   1 $d0 -> Truncate('after', 7600);
```

deglitch

Individual points can be removed from a data set like so:
```
   1 $d0 -> deglitch(17385.686, 17655.5);
```
Demeter does not, at this time, provide a specific manner of obtaining the energy values of the glitches. The cursor method (see the documentation for Ifeffit::Demeter::Dispose) provides a way of plucking values interactively from the PGPLOT window. Note that Athena's algorithm for finding glitches en masse using margins around the normalization lines has not yet been implemented.

smooth

Ifeffit's three-point smoothing algorithm can be applied repeatedly:
```
   1 $d0 -> smooth($n);
```
Here $n is the number of repititions.

add noise

Noise can be added to mu(E) or chi(k) data. For mu(E) data, the noise parameter is a fraction of the edge step. For chi(k) data, the noise parameter is referenced to the size of the chi(k) data. Note that, unlike most other data processing methods, this method takes a hash reference as its argument.
```
   1 $d1 -> noise({noise=>0.02, which=>'xmu'});
```

convolve

Data can be convoluted with with either a Gaussian or a Lorentzian as mu(E) or chi(k). Note that, unlike most other data processing methods, this method takes a hash reference as its argument.
```
   1 $d1 -> convolve({width=>2, type=>'Gaussian', which=>'xmu'});
```
The width is measured in eV, the type can be either Gaussian or Lorentzian , which can be either xmu or chi.